Discontinuities in Rx signal

[wowa]

(Continuing to receive rx stream frome LimeSDR with singleRX example under MS VS 2013…)

Now I use oversampling = 2, signal looks much better, but it still has some strange discontinuities. The plain AM signal is produced by an HF generator and supplied over the coax cable, Fc = 1 GHz, AM depth = 50%, Fmod = 100 kHz, output power is -20 dBm.

In a singleRX example I set up Fs = 10MHz, oversamp = 2, gain = 0 and receive several milliseconds of samples. It has some strange disruptions as you can see here:

On the next image, white trace is AM demodulated signal:

And here is magnified section where we see the moment of discontinuity. Interesting that I and Q break with some delay:

The same signal, received and demodulated using USRP X300:

If I increase oversampling, the sections between problem points tend to get shorter, but main picture remains the same.

When it starts streaming (I beleive on first LMS_RecvStream() call after LMS_StartStream()), it usually reports of some missed samples, but I understood it’s normal in the beginning of the process:

MCU algorithm time: 0 ms
Current MCU firmware: 3, DC/IQ calibration full
MCU Ref. clock: 30.72 MHz
MCU algorithm time: 239 ms
        Rx pktLoss: ts diff: 17680  pktLoss: 12

Except for this, it does not report about any lost packets.

What can be a reason of this?

Vladimir

[ricardas]

Is the board connected with USB 3.0?

[ricardas]

It would be best if you showed your code. How and at which point in code you are processing captured data.
Also compile your code in Release configuration, as Debug is not suitable for high sampling rate capturing

[wowa]

Ricardas,

Of course, it’s usb 3.0, it starts to show lost packets at significantly higher rates than 10 MSps that I use here. And yes, this time I did it in Release version, with the same result. Tried different sizes of buffer (40k samples/20k samples, I mean “const int bufersize” in singleRX).

I use slightly modified version of singleRX.cpp example. Unfortunately, I didn’t understand how to include source files here. Uploader takes only pictures, and when I paste it here, it autoformats the text in some weird manner which I don’t know how to cancel. Anyway, it doesn’t actually damage it, just formats. Please advise how to do it properly on this forum.

using namespace std;

//Device structure, should be initialize to NULL
lms_device_t* device = NULL;

const double Fc = 1e9; // Гц
//const double Fc = 800e6; // Гц
const double Fs = 10e6;
//const double Fs = 8e6;
const double FsOvrFact = 2;
//const double FsOvrFact = 2;
const double Flpw = 8e6;
const int Gain = 0; // дБ
const double TgtTime = 0.1; // секунд
const long TgtSampCnt = TgtTime*Fs;

int error()
{
//print last error message
cout << “ERROR:” << LMS_GetLastErrorMessage();
if (device != NULL)
LMS_Close(device);
exit(-1);
}

int main(int argc, char** argv)
{
//Find devices
//First we find number of devices, then allocate large enough list, and then populate the list
int n;
if ((n = LMS_GetDeviceList(NULL)) < 0)//Pass NULL to only obtain number of devices
error();
cout << "Devices found: " << n << endl;
if (n < 1)
return -1;

lms_info_str_t* list = new lms_info_str_t[n];   //allocate device list

if (LMS_GetDeviceList(list) < 0)                //Populate device list
    error();

for (int i = 0; i < n; i++)                     //print device list
    cout << i << ": " << list[i] << endl;
cout << endl;

//Open the first device
if (LMS_Open(&device, list[0], NULL))
    error();

delete [] list;                                 //free device list

//Initialize device with default configuration
//Do not use if you want to keep existing configuration
//Use LMS_LoadConfig(device, "/path/to/file.ini") to load config from INI
if (LMS_Init(device) != 0) {
  cout << "Init attempt failed (" << LMS_GetLastError() << "): " << LMS_GetLastErrorMessage() << endl;
  cout << "Trying to reset..." << endl;
  if (LMS_Reset(device) != 0)
    error();
  cout << "Init attempt 2:" << endl;
  if (LMS_Init(device) != 0)
    error();
} // if

//Enable RX channel
//Channels are numbered starting at 0
if (LMS_EnableChannel(device, LMS_CH_RX, 0, true) != 0)
    error();

//Set center frequency to 800 MHz
if (LMS_SetLOFrequency(device, LMS_CH_RX, 0, Fc) != 0)
    error();

//print currently set center frequency
float_type freq;
if (LMS_GetLOFrequency(device, LMS_CH_RX, 0, &freq) != 0)
    error();
cout << "\nCenter frequency: " << freq / 1e6 << " MHz\n";

//select antenna port
lms_name_t antenna_list[10];    //large enough list for antenna names.
                                //Alternatively, NULL can be passed to LMS_GetAntennaList() to obtain number of antennae
if ((n = LMS_GetAntennaList(device, LMS_CH_RX, 0, antenna_list)) < 0)
    error();

cout << "Available antennae:\n";            //print available antennae names
for (int i = 0; i < n; i++)
    cout << i << ": " << antenna_list[i] << endl;

if ((n = LMS_GetAntenna(device, LMS_CH_RX, 0)) < 0) //get currently selected antenna index
    error();
//print antenna index and name
cout << "Automatically selected antenna: " << n << ": " << antenna_list[n] << endl;

if (LMS_SetAntenna(device, LMS_CH_RX, 0, LMS_PATH_LNAW) != 0) // manually select antenna
    error();

if ((n = LMS_GetAntenna(device, LMS_CH_RX, 0)) < 0) //get currently selected antenna index
    error();

//print antenna index and name
cout << "Manually selected antenna: " << n << ": " << antenna_list[n] << endl;

//Set sample rate to 8 MHz, preferred oversampling in RF 8x
//This set sampling rate for all channels
if (LMS_SetSampleRate(device, Fs, FsOvrFact/*8e6, 8*/) != 0)
    error();
//print resulting sampling rates (interface to host , and ADC)
float_type rate, rf_rate;
if (LMS_GetSampleRate(device, LMS_CH_RX, 0, &rate, &rf_rate) != 0)  //NULL can be passed
    error();
cout << "\nHost interface sample rate: " << rate / 1e6 << " MHz\nRF ADC sample rate: " << rf_rate / 1e6 << "MHz\n\n";

//Example of getting allowed parameter value range
//There are also functions to get other parameter ranges (check LimeSuite.h)

//Get allowed LPF bandwidth range
lms_range_t range;

if (LMS_GetLOFrequencyRange(device, LMS_CH_RX, &range)!=0)
    error();
cout << "RX center freq range: " << range.min / 1e6 << " - " << range.max / 1e6 << " MHz\n";


if (LMS_GetSampleRateRange(device, LMS_CH_RX, &range)!=0)
    error();
cout << "RX sample rate range: " << range.min / 1e6 << " - " << range.max / 1e6 << " MHz\n";

if (LMS_GetLPFBWRange(device,LMS_CH_RX,&range)!=0)
    error();

cout << "RX LPF bandwitdh range: " << range.min / 1e6 << " - " << range.max / 1e6 << " MHz\n\n";

//Configure LPF, bandwidth 8 MHz
if (LMS_SetLPFBW(device, LMS_CH_RX, 0, Flpw) != 0)
    error();

//Set RX gain
if (LMS_SetGaindB(device, LMS_CH_RX, 0, Gain) != 0)
    error();

//Print RX gain
float_type gain; //normalized gain
if (LMS_GetNormalizedGain(device, LMS_CH_RX, 0, &gain) != 0)
    error();
cout << "Normalized RX Gain: " << gain << endl;

unsigned int gaindB; //gain in dB
if (LMS_GetGaindB(device, LMS_CH_RX, 0, &gaindB) != 0)
    error();
cout << "RX Gain: " << gaindB << " dB" << endl;

//Perform automatic calibration
if (LMS_Calibrate(device, LMS_CH_RX, 0, Flpw, 0) != 0)
    error();

//Enable test signal generation
//To receive data from RF, remove this line or change signal to LMS_TESTSIG_NONE
/*if (LMS_SetTestSignal(device, LMS_CH_RX, 0, LMS_TESTSIG_NONE, 0, 0) != 0)
    error();*/
/*if (LMS_SetTestSignal(device, LMS_CH_RX, 0, LMS_TESTSIG_NCODIV8, 0, 0) != 0)
    error();
if (LMS_SetTestSignal(device, LMS_CH_RX, 1, LMS_TESTSIG_NCODIV8, 0, 0) != 0)
    error();*/

if (!wrOpen("f:\\Sig\\lime_trace.dat")) {
  cout << "Can't open output file!" << endl;
  exit(-1);
} // if

if (!wrWriteSetCrTrHdrProps(pdu_DefLime, round(Fc), round(Fs), TgtSampCnt)) {
  cout << "Can't write output file header!" << endl;
  exit(-1);
} // if

//Streaming Setup

//Initialize stream
lms_stream_t streamId;
streamId.channel = 0; //channel number
streamId.fifoSize = 1024 * 1024; //fifo size in samples
streamId.throughputVsLatency = 0.5; //optimize for max throughput
streamId.isTx = false; //RX channel
streamId.dataFmt = lms_stream_t::LMS_FMT_I12; //32-bit floats
if (LMS_SetupStream(device, &streamId) != 0)
    error();

//Data buffers
const int bufersize = 40000; //complex samples per buffer
int16_t buffer[bufersize * 2]; //must hold I+Q values of each sample
//Start streaming
LMS_StartStream(&streamId);

auto t1 = chrono::high_resolution_clock::now();
auto t2 = t1;

long SampsWritten = 0;

//while (chrono::high_resolution_clock::now() - t1 < chrono::seconds(3)) //run for 10 seconds
while (SampsWritten < TgtSampCnt) {
    int samplesRead;
    //Receive samples
    samplesRead = LMS_RecvStream(&streamId, buffer, bufersize, NULL, 1000);
    //I and Q samples are interleaved in buffer: IQIQIQ...
    //samplesRead = LMS_RecvStream(&streamId[1], buffer[1], bufersize, NULL, 1000);

    if (SampsWritten == 0) {
      int16_t *p = buffer;
      printf("%4d:%-4d %4d:%-4d %4d:%-4d %4d:%-4d %4d:%-4d %4d:%-4d %4d:%-4d %4d:%-4d %4d:%-4d %4d:%-4d\n",
        p[0], p[1], p[2], p[3], p[4], p[5], p[6], p[7], p[8], p[9],
        p[10], p[11], p[12], p[13], p[14], p[15], p[16], p[17], p[18], p[19]);
      byte *b = (byte*)buffer;
      printf("%02X%02X:%02X%02X %02X%02X:%02X%02X %02X%02X:%02X%02X %02X%02X:%02X%02X %02X%02X:%02X%02X %02X%02X:%02X%02X %02X%02X:%02X%02X %02X%02X:%02X%02X %02X%02X:%02X%02X %02X%02X:%02X%02X\n\n",
        b[0], b[1], b[2], b[3], b[4], b[5], b[6], b[7], b[8], b[9],
        b[10], b[11], b[12], b[13], b[14], b[15], b[16], b[17], b[18], b[19],
        b[20], b[21], b[22], b[23], b[24], b[25], b[26], b[27], b[28], b[29],
        b[30], b[31], b[32], b[33], b[34], b[35], b[36], b[37], b[38], b[39]);
    } // if

    int Samps2Write = min(TgtSampCnt-SampsWritten, samplesRead);
    if (!wrWrite(buffer, 2 * Samps2Write * sizeof(__int16))) {
      cout << "Can't write samples to output file!" << endl;
      exit(-1);
    } // if

    SampsWritten += Samps2Write;

    //Print stats (once per second)
    if (chrono::high_resolution_clock::now() - t2 > chrono::milliseconds(500))
    {
        t2 = chrono::high_resolution_clock::now();
        lms_stream_status_t status;
        //Get stream status
        LMS_GetStreamStatus(&streamId, &status);
        cout << "RX_1 data rate: " << status.linkRate / 1e6 << " MB/s\n"; //link data rate
        cout << "RX_1 sample rate: " << status.sampleRate / 1e6 << " MSamples/s\n"; //link data rate
        cout << "RX_1 fifo: " << 100 * status.fifoFilledCount / status.fifoSize << "%" << endl; //percentage of FIFO filled
    }
} // while

wrClose();

//Stop streaming
LMS_StopStream(&streamId); //stream is stopped but can be started again with LMS_StartStream()
LMS_DestroyStream(device, &streamId); //stream is deallocated and can no longer be used

//Close device
LMS_Close(device);

return 0;

}

[wowa]

No, I see that it actually damages C source code - interpreting somehow “{ }” blocks. I try to apply “Preformatted text”, but with no effect… But anyway, main parts can be seen.

To save the samples to the file I use my own small helper that is in separate file, I can send it to you, but it essentially writes plain byte data of a given size and is reusable piece of code, I doubt that it’s introducing the problem. Plus, the fact that the effect does not depend of the portion size says against this suspision.

[wowa]

Here is the header Writer.h:

#pragma once
#include <type_traits>

#pragma pack(push, 1)
typedef struct CrTrHeader
{
  unsigned __int32 Signature;
  unsigned __int32 Version;       // +4
  unsigned __int16 CrType;	      // +8
  unsigned __int16 DataType;	    // +10
  double KScale;		              // +12
  unsigned __int64 CenterFreq;	  // +20
  unsigned __int32 SampleRate;	  // +28
  unsigned __int16 PrimDataType;	// +32
  unsigned __int16 PrimDataUnits;	// +34
  unsigned __int64 SampleCount;	  // +36
  unsigned __int64 CheckSum;      // +44
				                          // +52
  //float MaxAbsIQ;
} CRTR_HEADER_TYPE;
#define CRTR_HEADER_SIZE (sizeof(CRTR_HEADER_TYPE))
#pragma pack(pop)
static_assert(std::alignment_of<CRTR_HEADER_TYPE>::value <= 2, "Alignment of CRTR_HEADER_TYPE must be <= 2");

// DataType
const unsigned __int16 dt_IQ = 0;

// PrimDataType
const unsigned __int16 pdt_int16 = 0;

// PrimDataUnits
const unsigned __int16 pdu_Unknown = 0;
const unsigned __int16 pdu_dBuV    = 1;
const unsigned __int16 pdu_10dBuV  = 2;
const unsigned __int16 pdu_DefX300 = 3;
const unsigned __int16 pdu_DefLime = 4;

extern CRTR_HEADER_TYPE CrTrHdr;

bool wrOpen(char *FName);
bool wrWrite(void *buf, size_t ByteCnt);
bool wrClose();

void wrSetCrTrHdrProps(
  int PrimDataUnits,          // -1, do not change
  unsigned __int64 CenterFreq,
  unsigned __int32 SampleRate,
  unsigned __int64 SampleCount
);

bool wrWriteSetCrTrHdrProps(
  int PrimDataUnits,          // -1, do not change
  unsigned __int64 CenterFreq,
  unsigned __int32 SampleRate,
  unsigned __int64 SampleCount
);

[wowa]

And helper file Writer.cpp:

#include "stdafx.h"
#include "Writer.h"

FILE *wrf;

CRTR_HEADER_TYPE CrTrHdr = {
  0x52545243,           // Signature
  1,                    // Version
  0,                    // CrType: rwt_Any
  dt_IQ,                // DataType
  0.0,                  // KScale
  0,                    // CenterFreq
  0,                    // SampleRate
  pdt_int16,            // IQDataType
  pdu_10dBuV,           // IQUnits
  0,                    // SampleCount
  0,                    // CheckSum
}; // CrTrHdr

bool wrOpen(char *FName)
{
  wrf = fopen(FName, "w+b");
  return wrf != NULL;
} // wrOpen

bool wrWrite(void *buf, size_t ByteCnt)
{
  size_t wrcnt = fwrite(buf, ByteCnt, 1, wrf);
  return wrcnt == 1;
} // wrWrite

bool wrClose()
{
  fclose(wrf);
  return true;
} // wrClose

void wrSetCrTrHdrProps(
  int PrimDataUnits, // -1, если не надо менять
  unsigned __int64 CenterFreq,
  unsigned __int32 SampleRate,
  unsigned __int64 SampleCount)
{
  if (PrimDataUnits >= 0)
    CrTrHdr.PrimDataUnits = PrimDataUnits;
  CrTrHdr.CenterFreq = CenterFreq;
  CrTrHdr.SampleRate = SampleRate;
  CrTrHdr.SampleCount = SampleCount;
} // wrSetCrTrHdr

bool wrWriteSetCrTrHdrProps(
  int PrimDataUnits, // -1, если не надо менять
  unsigned __int64 CenterFreq,
  unsigned __int32 SampleRate,
  unsigned __int64 SampleCount)
{
  wrSetCrTrHdrProps(PrimDataUnits, CenterFreq, SampleRate, SampleCount);
  return wrWrite(&CrTrHdr, CRTR_HEADER_SIZE);
} // wrWriteSetCrTrHdrProps

[ricardas]

Not the best place to write data to disk inside the loop, the disk I/O can be quite slow. But this shoudn’t be the problem here.

It should be impossible to have misaligned I and Q signals, are you using latest software/firmware/gateware?

I see you have changed throughput and data formats, try the following settings, check if it makes any difference.
streamId.throughputVsLatency = 1.0;
streamId.dataFmt = lms_stream_t::LMS_FMT_I16;

[wowa]

Ricardas,

@ricardas

Not the best place to write data to disk inside the loop, the disk I/O can be quite slow. But this shoudn’t be the problem here.

Agree, but I beleive LimeSuite has stream integrity check based on timestamps, hasn’t it? Or it might happen that samples are being lost and there is no sign of it in the output log?

Besides of that, looking at the pictures, it does not resemble the sample loss, because it shifts not the way as it would if small amount of samples han been lost (taking into account that time is going from left to right). It looks more like some amplitude shift (of unclear nature for me though), may be like some rough version of AGC - impression is that it tries to lower the amplitude when it goes too far from zero… don’t know…

@ricardas

It should be impossible to have misaligned I and Q signals, are you using latest software/firmware/gateware?

As of LimeSuite/firmware - yes, I cloned/downloaded it from corresponding sites less than a week ago. About gateware I’m not sure what it is. Is it one of two images which are uploaded to the device? Then also yes, I downloaded/burned both images at once.

@ricardas

I see you have changed throughput and data formats, try the following settings, check if it makes any difference.
streamId.throughputVsLatency = 1.0;
streamId.dataFmt = lms_stream_t::LMS_FMT_I16;

OK, I’ll try this on monday (though I think for sure I tried at least that FMT_I16, with the same result; but will doublecheck it).

[ricardas]

Agree, but I beleive LimeSuite has stream integrity check based on timestamps, hasn’t it? Or it might happen that samples are being lost and there is no sign of it in the output log?

the Rx pktLoss: ts diff: 17680 pktLoss: 12 check you saw is done only in debug build, it just checks if data packets coming from the hardware are contiguous.
If it does not show any packet loss, that means data is received correctly and placed into a ring buffer.
When you use LMS_RecvStream(), it takes samples from that ring buffer. If it’s called too slow samples might be lost/overwritten by new incoming data. You can check if you are not losing samples by checking timstamps that are returned in metadata parameter from LMS_RecvStream().

Besides of that, looking at the pictures, it does not resemble the sample loss, because it shifts not the way as it would if small amount of samples han been lost (taking into account that time is going from left to right). It looks more like some amplitude shift (of unclear nature for me though), may be like some rough version of AGC - impression is that it tries to lower the amplitude when it goes too far from zero… don’t know…

Yes, the shift only in amplitude is strange, hard to tell the exact amounts from the plots, but it looks like it shifts by some power of 2. Might be some sort of integer bits shifting/overflow/endianess issue. As the data is displayed by other software I don’t know what data format it expects or how it parses it, so there is possibility for conversion errors. It would be best to get raw data for inspection (just upload the file somewhere dropbox/google drive)

Most puzzling part is the misalignment of the break in IQ channels.

[andrewback]

@wowa

Please advise how to do it properly on this forum.

Probably better if you post a link to a GitHub repo or Gist.

[mzs]

People who use IRC would typically use one of the following to share formatted text without it getting mangled:
http://sprunge.us/
http://termbin.com/
https://paste.ee/
https://pastebin.com/

[Zack]

Hi @wowa,

Could you write the data to the file in binary format for checking. It would be nice to see it in two columns, one for I another for Q, something like this:

I            Q
000000000000 000000000001
000000010000 001000000001

…

[wowa]

@andrewback, @mzs,

Thank you for links! I just thought I missed some way to do it here. OK, will try gist first. (After trying: not very convenient for large files though, and as I understood, works only for text files…)

@Zack,

I put the files here: https://gist.github.com/wowa-2017/1be7b8ac1efae8672fcbd51b96bc8fc1

Here, singleRX_05.cpp is the version of the example that produced data files. As you can see, I modified it so it grabs just 100000 samples by single recv call and outputs them in couple ways. Lime_I12.dat and Lime_16.dat - the outputs you requested with corresponding lms_stream_t::LMS_FMT_… settings.

Both were captured in Release mode. By ricardas’ suggestion I set streamId.throughputVsLatency = 1.0;, but it does not seem to change the way it runs.

[IgnasJ]

Hello,
I thought of one more thing that you can try. Insert
LMS_WriteParam(device,LMS7_DC_BYP_RXTSP,1);
right after LMS_Calibrate() and see if it helps with discontinuities.

[wowa]

Yes, it helps, they’re gone! Now what does this mean?

[IgnasJ]

It bypasses DC corrector in LMS7 chip. Your signal oscillates around 0 quite slowly and it seems that on-chip DC corrector window is too small for that, so DC corrector kicks in trying to compensate DC offset and bring signal closer to 0.

[wowa]

Ignas, thank you for solving this puzzle. Couple questions remain though:

@IgnasJ

Your signal oscillates around 0 quite slowly and it seems that on-chip DC corrector window is too small for that, so DC corrector kicks in trying to compensate DC offset and bring signal closer to 0.

Sounds a bit strange to me, as the modulation freq is 100 kHz, doesn’t seem to be so slow… Those big “waves” are not signal, it’s carrier frequencies’ difference between generator and Lime receiver. I think that here the corrector tries to struggle the carrier offset… Anyway, it’s good that this cleared out.

Can you advise what automatic features are turned on by default (or may be turned on) which may affect the signal? I noticed near this parameter two more alike - the gain corrector and phase corrector. Is AGC turned on by default? Is there anything else like that?

[IgnasJ]

@wowa

Sounds a bit strange to me, as the modulation freq is 100 kHz, doesn’t seem to be so slow… Those big “waves” are not signal, it’s carrier frequencies’ difference between generator and Lime receiver. I think that here the corrector tries to struggle the carrier offset… Anyway, it’s good that this cleared out.

Yea, I was talking about those ‘big waves’ that are present in your received signal.

@wowa

Can you advise what automatic features are turned on by default (or may be turned on) which may affect the signal? I noticed near this parameter two more alike - the gain corrector and phase corrector. Is AGC turned on by default? Is there anything else like that?

Gain and phase correctors are not automatic and are not doing any corrections at their default settings. They are configured when you run LMS_Calibrate().
AGC and digital filters (GFIR) are turned off by default

[wowa]

Ignas,

Thank you for your help, one channel is now working good.

But now I try to receive two streams in sync, and it seems I get the same thing but on one of the channels. I played with the sequence of operations (starting from trying to input two channels, but then reducing to the simplest scenario where it occurs) and now have the following way to reproduce the error. In my singleRX code which works well for one channel, I duplicate all ‘set’ actions for the second RX channel (I mean setting parameters, except for streaming itself).

I then start the streaming on channel 0 only. The quality of received signal depends on the sequence of LMS_Calibrate() calls. If I do Calibrate(ch 0); Calibrate(ch 1); WriteParam(LMS7_DC_BYP_RXTSP, 1); then I get distorted signal from ch 0, just like at the beginning of this topic.

If I swap calibrate calls and make it Calibrate(ch 1); Calibrate(ch 0); WriteParam(LMS7_DC_BYP_RXTSP, 1); - then the signal from channel 0 is undistorted.

Is it possible that some cross-interference happens in those calls? Looks like WriteParam sets this flag up for the last calibrated channel (of course it’s just a guess).

I put the source of this modified example here: https://gist.github.com/wowa-2017/49f15b2e47ad0fb6987a0c37a5d622aa#file-dualrx_01-cpp