LimeSDR fails to stream over USB 3.0 leading to a single-shot receive spectrum and USB Transfer Error state

Hello, I have a LimeSDR updated to the latest Firmware images (20.01) with gateway version 2 in revision 22. I use Debian Buster with the libusb-1.0-0 as driver. Opening and configuring the device with LimeSuiteGUI works fine over an USB 3.0 connection but I cannot obtain samples.

INFO: Connected Control port: LimeSDR-USB FW:4 HW:4 Protocol:1 GW:2.22 Ref Clk: 30.72 MHz

The problem I face is that the streaming of <= 20 MSPS is not working properly. I managed to get a single-shot spectrum in gqrx once but it is then fixed to one view. If I follow the basic LimeSDR USB Quick Test or this Getting Started guide, the FFT Viewer usually won’t display anything or if it does, then the displayed spectrum will not be updated and the RX rate gets zero. Further, an USB Transfer error occurs.

The LimeUtil (v20.01.0) and SoapySDRUtil (v0.8.0) are currently built from source but the issue was similar with older versions I got from the Debian repository.

$ SoapySDRUtil --info
######################################################
##     Soapy SDR -- the SDR abstraction library     ##
######################################################

Lib Version: v0.8.0-gf722f9ce
API Version: v0.8.0
ABI Version: v0.8
Install root: /usr/local
Search path:  /usr/local/lib/SoapySDR/modules0.8
Module found: /usr/local/lib/SoapySDR/modules0.8/libLMS7Support.so (20.01.0-c931854e)
Available factories... lime
Available converters...
 -  CF32 -> [CF32, CS16, CS8, CU16, CU8]
 -  CS16 -> [CF32, CS16, CS8, CU16, CU8]
 -  CS32 -> [CS32]
 -   CS8 -> [CF32, CS16, CS8, CU16, CU8]
 -  CU16 -> [CF32, CS16, CS8]
 -   CU8 -> [CF32, CS16, CS8]
 -   F32 -> [F32, S16, S8, U16, U8]
 -   S16 -> [F32, S16, S8, U16, U8]
 -   S32 -> [S32]
 -    S8 -> [F32, S16, S8, U16, U8]
 -   U16 -> [F32, S16, S8]
 -    U8 -> [F32, S16, S8]

When I perform the LimeQuickTest it commonly fails with the [RF Loopback Test] as

$ LimeQuickTest 

[ TESTING STARTED ]
->Start time: Sat Mar 28 19:35:10 2020

->Device: LimeSDR-USB, media=USB 3.0, module=FX3, addr=1d50:6108, serial=<SERIALNUMBER>
  Serial Number: <SERIALNUMBER>

[ Clock Network Test ]
->FX3 GPIF clock test
  Test results: 27918; 31674; 35430 - PASSED
->Si5351C test
  CLK0: 17554 / 17554 - PASSED
  CLK1: 17554 / 17554 - PASSED
  CLK2: 17554 / 17554 - PASSED
  CLK3: 17554 / 17554 - PASSED
  CLK4: 17554 / 17554 - PASSED
  CLK5: 17554 / 17554 - PASSED
  CLK6: 17554 / 17554 - PASSED
->ADF4002 Test
  Result: 10 - PASSED
->VCTCXO test
  Results : 5112948 (min); 5113087 (max) - PASSED
->Clock Network Test PASSED

[ FPGA EEPROM Test ]
->Read EEPROM
->Read data: 13 01 1C 13 01 1C 02
->FPGA EEPROM Test PASSED

[ LMS7002M Test ]
->Perform Registers Test
->External Reset line test
  Reg 0x20: Write value 0xFFFD, Read value 0xFFFD
  Reg 0x20: value after reset 0x0FFFF
->LMS7002M Test PASSED

[ RF Loopback Test ]
Note: The test should be run without anything connected to RF ports
->Configure LMS
SetPllFrequency: error configuring phase
SetPllFrequency: error configuring phase
SetPllFrequency: error configuring phase
SetPllFrequency: error configuring phase
SetPllFrequency: error configuring phase
SetPllFrequency: error configuring phase
SetPllFrequency: error configuring phase
SetPllFrequency: error configuring phase
SetPllFrequency: error configuring phase
SetPllFrequency: error configuring phase
LML TX phase search FAIL
Failed to set sample rate
->RF Loopback Test FAILED

=> Board tests FAILED <=

Elapsed time: 1.42 seconds

Sometimes, the [RF Loopback Test] will also fail with this output

[ RF Loopback Test ]
Note: The test should be run without anything connected to RF ports
->Configure LMS
->Run Tests (TX_2-> LNA_L):
  Failed to read samples: Timeout
  CH0 (SXR=800.0MHz, SXT=805.0MHz): Result:(-15.0 dBFS, 5.00 MHz) - FAILED
  Failed to read samples: Timeout
  CH1 (SXR=800.0MHz, SXT=805.0MHz): Result:(-15.0 dBFS, 5.00 MHz) - FAILED
->Run Tests (TX_1 -> LNA_W):
  Failed to read samples: Timeout
  CH0 (SXR=1800.0MHz, SXT=1805.0MHz): Result:(-15.0 dBFS, 5.00 MHz) - FAILED
  Failed to read samples: Timeout
  CH1 (SXR=1800.0MHz, SXT=1805.0MHz): Result:(-15.0 dBFS, 5.00 MHz) - FAILED
->Run Tests (TX_2-> LNA_H):
  Failed to read samples: Timeout
  CH0 (SXR=2500.0MHz, SXT=2505.0MHz): Result:(-15.0 dBFS, 5.00 MHz) - FAILED
  Failed to read samples: Timeout
  CH1 (SXR=2500.0MHz, SXT=2505.0MHz): Result:(-15.0 dBFS, 5.00 MHz) - FAILED
->RF Loopback Test FAILED

=> Board tests FAILED <=

Elapsed time: 7.45 seconds

When performing a test with the FFT viewer, the Rx rate is at first shortly different to 0 B/s but then returns to zero again. If I manually rescale the display, I can then see a frozen spectrum which is not updated. It is similar to the one I earlier saw when using gqrx.

Typically, the LEDs on the LimeSDR related to FX are both green and the outer FPGA one close to the edge is blinking green. But the spectrum and noise remains constant.

But mostly, I will run into an “USB Transfer error” as stated in the Lime Suite’s Log Window when trying to re-run the FFT Viewer. This happens for example when following the USB QuickTest Guide and active MIMO settings. Then, the outer LED starts alternating between green and red. This state is often also raised after device activation in e.g. gqrx and after an one-shot spectrum was computed. A similar behaviour occurs when I try to run a quick reception test with SoapySDRUtil which thereby produces the following output (see edit below for different syntax)

$ SoapySDRUtil --args=“driver=lime” --rate=1e6 --channels=“0” --direction=RX

######################################################
##     Soapy SDR -- the SDR abstraction library     ##
######################################################

[INFO] Make connection: 'LimeSDR-USB [USB 3.0] <SERIALNUMBER>'
[INFO] Reference clock 30.72 MHz
[INFO] Device name: LimeSDR-USB
[INFO] Reference: 30.72 MHz
[INFO] LMS7002M register cache: Disabled
Error in rate test: stoi

And again a green and red alternating outer FPGA LED indication happens similar to the case of “USB Transfer Error” in LimeSuiteGUI.

However, if I run SoapySDRUtil --probe="driver=lime", it will output the whole port configuration without any fault indication. The following udev rules are set and also activated.

$ cat /etc/udev/rules.d/64-limesuite.rules

SUBSYSTEM=="usb", ATTR{idVendor}=="04b4", ATTR{idProduct}=="8613", SYMLINK+="stream-%k", MODE="666"
SUBSYSTEM=="usb", ATTR{idVendor}=="04b4", ATTR{idProduct}=="00f1", SYMLINK+="stream-%k", MODE="666"
SUBSYSTEM=="usb", ATTR{idVendor}=="0403", ATTR{idProduct}=="601f", SYMLINK+="stream-%k", MODE="666"
SUBSYSTEM=="usb", ATTR{idVendor}=="1d50", ATTR{idProduct}=="6108", SYMLINK+="stream-%k", MODE="666"
SUBSYSTEM=="xillybus", MODE="666", OPTIONS="last_rule"

SUBSYSTEM=="tty", ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6001", MODE="0666", SYMLINK+="serial"

But it does not fix the problem of an interrupted stream resulting in the display of an one-shot frozen spectrum (if any). Board temperature is typically between 40 and 55 degrees.

  • What do I need to change to get a continuous stream?

  • How to properly react when the outer FPGA LED is alternating between green and red after an USB Transfer error mesage? Typically writing to the chip from LimeSuiteGUI resets this state.

If you require more information, then I can provide it.

EDITED TO ADD (1/2):

If I use SoapySDRUtil --args=“driver=lime” --rate=1e6 --channels=1 --direction=RX at the moment, I can now more or less reproduce the [ERROR] USB TRANSFER ERROR previously reported for gqrx and in the Lime Suite GUI’s log window.

SoapySDRUtil-read-1ch-rx-USB-Transfer-Error

In the case of --channels=0 the same will happen. But interestingly, the outer FPGA LED currently continues to blink green despite of the message. How can I get rid of this error?

EDITED TO ADD (2/2):

The issue seems to be related to USB 3.0 because over USB 2 the streaming is functional.

SoapySDRUtil-read-1ch-rx-USB2-works

But LimeQuickTest now always fails over USB 2 with SetPllFrequency: error configuring phase.

I hope you can support me in getting USB 3.0 working.

That’s odd.

@Zack, could you help diagnose, please.

Do you have any more news on this? I also tried an external power supply and the issue remains :-/ @andrewback @Zack

I’ve updated my Lime to Firmware 4, gateware 2.22 (part of Ubuntu 20.04) and now it seems to be happier with USB3.0.

Thank you for the information. I use this version already and it leads to the described interrupted transfer or respectively the USB TRANSFER ERROR on Linux over USB 3.0.

The USB 3.0 host controller is an Intel series 8 chipset. Therefore, the workaround for Linux USB3 AMD host controllers – 290 included in ConnectionFX3 class by @TehWan does not address my problem. But as described in 287 USB 2 is functional as well. Failure might be due to power constraints on these ports. I did not test latter in detail yet because I aim to establish an USB 3.0 connection.

More details

The first connection attempt via Soapy usually blocks at this point.

$ ./SoapySDRUtil --args=“driver=lime” --rate=1e6 --channels=1 --direction=RX
######################################################
##     Soapy SDR -- the SDR abstraction library     ##
######################################################

[INFO] Make connection: 'LimeSDR-USB [USB 3.0] <SERIALNUMBER>'
[INFO] Reference clock 30.72 MHz
[INFO] Device name: LimeSDR-USB
[INFO] Reference: 30.72 MHz
[INFO] LMS7002M register cache: Disabled
[INFO] RX LPF configured
Stream format: CS16
Num channels: 1
Element size: 4 bytes
Begin RX rate test at 1 Msps
Starting stream loop, press Ctrl+C to exit...
[INFO] Rx calibration finished

^C

Just after the second connection attempt, it directly continues with the USB Transfer Error Message which is ‘persistent’ until for example re-powering the LimeSDR.

^C

$ ./SoapySDRUtil --args=“driver=lime” --rate=1e6 --channels=1 --direction=RX
######################################################
##     Soapy SDR -- the SDR abstraction library     ##
######################################################

[INFO] Make connection: 'LimeSDR-USB [USB 3.0] <SERIALNUMBER>'
[INFO] Reference clock 30.72 MHz
[INFO] Device name: LimeSDR-USB
[INFO] Reference: 30.72 MHz
[INFO] LMS7002M register cache: Disabled
[INFO] RX LPF configured
Stream format: CS16
Num channels: 1
Element size: 4 bytes
Begin RX rate test at 1 Msps
Starting stream loop, press Ctrl+C to exit...
[INFO] Rx calibration finished
[ERROR] USB TRANSFER ERROR
[ERROR] USB TRANSFER ERROR
[ERROR] USB TRANSFER ERROR
[ERROR] USB TRANSFER ERROR

And the upper blinking LED did not turn red during testing this time.

I did a little bit of USB 3.0 debugging with modprobe usbmon and Wireshark on Debian Buster (see below). It seems that the ConnectionFX3 class waits in vain for an asynchronous callback from libusb.

Until CTRL+C interrupt was hit, no more data has been retrieved from the device.

The additional info and warning outputs were put to their respective cases in callback_libusbtransfer. This is the async callback function registered by libusb_fill_bulk_transfer() in ConnectionFX3::BeginDataReading/Sending.

This is one example in which at least some data got transferred in 4 callbacks calls from the beginning. Likely, this lead to the display of a constant single-shot receive spectrum described earlier. Despite the last UBX response lengths looks quite big to me (8192 bytes).

Since the device was freshly powered, it also did not crash into an USB TRANSFER ERROR but stalled.

  • Could this be because libusb is not triggered for event handling?

It seems that libusb_handle_events_timeout_completed() is correctly called repeatedly as required for asynchronous event handling in the ConnectionFX3Entry class.

  • What could help to continue from this stall and to fix the asynchronous USB 3.0 transfer?

Update (April 16, 2020)

I had the chance to connect my Lime device directly to an USB 3.0 port. This might be limited by your PC case and its placement of suitable ports. However, if it is connected with a direct connection, it will pass all LimeQuickTests from the start and LimeSuiteGUI’s FFTViewer displays a continuously updated spectrum not only via USB2. The only thing left is the cable as error source.

Suspect: Faulty claimed High Speed USB3.0 Cable

Its imprint reads High Speed USB3.0 cable and it is equipped with blue USB-SS connectors. This looks strange to me but it was the last likely source of error for the behaviour seen.

Connecting the device with this cable, trials were successful most of the time via USB 2 only. Probably, this hub had a stronger power supply than the one I used before. If the device is connect to any USB 3.0 port with this USB cable, the last RF Loopback test of LimeQuickTest always fails with either

1) Fails to read samples: No samples received / Timeout / cannot deliver frequency

or 2) SetPllFrequency: error configuring phase (partly also when connected by this cable to USB2)

Tl;dr Check your USB 3.0 SuperSpeed cable or try a direct connection (if possible) for these types of errors. In particular, if the device seems to work with USB 2 and bulk transfers fail with USB 3.0.

Likely solved! Thanks all for reading. Maybe it will save others with similar issues and faulty cables some time else spent for debugging. Now I am curious to move on to test all LimeSDR features. I will try other cables in the near future because I don’t intend solely direct connections.

The only issue remaining is wMaxPacketSize being just 1x 1024 Bytes in contrary to USB transfer sizes of 8192 bytes seen with libusb by usbmon – this however worked fine even for USB2 connection type.

1 Like

+1. Poor quality and faulty cables are a common cause of problems, particularly with the LimeSDR USB board, which has a larger FPGA etc. and draws more current. Power brownouts can result in all sorts of failure modes which send you off on a wild goose chase. And even a lot of pretty good looking cables can be of dubious quality. Always test with a direct connection wherever possible!

I am now able to confirm that the cable in question was the actual cause of above problems. A high quality USB 3.0 Super Speed cable works right out of the box.

1 Like