Do you have all other USB devices disconnected (keyboards, mice etc) and you are not using the 10/100 Mbps NIC which also shares the same one and only real USB 2.0 HighSpeed port that is common in all RPi products. (ref: Gqrx for limeSDR on RPI3 for a nice diagram).
The RPi has three main things going for it, cheap, large support base, reasonably number crunching ability, but pushing large amounts of data about fast is not one of its strengths.
I think of the USB configuration of a RPi using a LAN9514 chip (a 4-port USB hub with an integrated 10/100 USB Ethernet controller) like putting 5 funnels into one funnel. The flow rate of the bottom funnel is the limiting factor and using more than one funnel above will only reduce the flow rate available.