My guess would be Nyquist limit, so maybe increase the sample rate ? It can go quite high. You will need to decimate it afterwards within the TSP to be less than the ~60MHz of bandwidth possible from the digital interface out from the TSP. Maybe even shift your centre frequency higher (or not) for a higher sample rate. The more you decimate the higher the dynamic range, but you will probably need to add an external lowpass/bandpass filter to limit unwanted signals (e.g. Look at the filters on the LimeRFE board).
I’ll point you to this thread where I learned about the NCO and decimation. Which is not that long ago actually.