Have you tried with all other USB devices unplugged (or as many as you can remove), like no USB hard drives, no USB flash drives, no USB video adapters, …
And have you tried using a different USB port (not all USB ports are equal, performance will depend on the chipset used, which can change from port to port even on the same computer, I’ve seen computers with 4 different USB controllers from different manufacturers).
You said Mac, but no info about model type/CPU/RAM.
The other thing is that pushing 40MSPS IQ samples about as 32-bit floats inside a computer will use a minimum of ~320MB/sec memory bandwidth, and then an additional 160MB/sec when being converted to 12-bit values stored as 16-bit integers and an 160MB/sec when being being bit packed to go across the USB bus. So even with the simple flow graph you used above you could be talking about an idealised minimum of 640MB/sec (in reality it will be more) which could be roughly 20% of the maximum for DDR2 (2003), or 10% of DDR3 memory (2007), or 5% of DDR4 memory (2014), … These numbers are more a guideline than fact, because there is a whole range of performance for each RAM type increasing with higher prices.
If there are other tasks running that are RAM intensive on your computer, and it has slightly lower performance RAM, you could easily be hitting a memory bandwidth bottleneck.
There is also the conversion from floats to integers and packing samples, that requires a good bit of processing power at 40 million samples per second. I would quit all other applications to maximise available processing power.
Then there is the relatively recent patches that help mitigate the Meltdown and Spectre speculative execution vulnerabilities. You are probably talking about a best case of a 10% performance loss and at worst case two to possibly even four fold loss in performance for some tasks depending explicitly on the system calls used. The LimeSDR boards were both created before these CPU flaws were publicly known about, so there is the possibly that the mitigation patches could have a negative influence as well.
Oh and anti-virus, but this is a given, as well power supply issues that the board is not getting enough power for the higher sample rates which will typically use more power than lower sample rates.
I would be inclined to workout what the upper limit is for your computer by incrementing the sample rate up in steps (1-10MSPS) from 1MSPS up to 40MSPS and narrowing down where the problem starts to happen. Knowing where might help in understanding the source of the problem. I’d also be inclined to see if the same thing happens at a centre frequency of say 350MHz (640/2+30). And I would see if the upper limit changes with different USB ports as well. That would be my approach anyhow.
