XTRX enumeration issue

Greetings!

I’m having an issue with a physical xtrx 1.2 device.

I’m wondering if this could be either the device doesn’t have any configuration on it, or perhaps EEPROM / Flash corruption?

  • The PCIe board and software work fine with a different xtrx 1.2 device.

  • My understanding is that this device has never been flashed or even powered on prior to this issue being visible.

  • I’m using commit 368efb0154a459e42317ae270bf9716e776b82ac of LimeSuiteNG to build the kernel module and the relevant lime* CLI tools.

  • The device does show up ( with the serial number 00-00-00-00-00-00-00-00 ) using lspci.

  • Running limeDevice results in LimePCIe: Failed to open (/dev/limepcie0/control0), errno(5) Input/output error.

  • Doing a modprobe limepcie, and subsequent rmmod limepcie results in the following dmesg:

[ 2686.631972] limepcie : module init
[ 2686.635501] limepcie 0000:01:00.0: [Probing device] vid:10EE pid:7023
[ 2686.643162] limepcie 0000:01:00.0: BAR0 address=0x000000002e27903a
[ 2686.669657] limepcie 0000:01:00.0: Identifier: \xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff
[ 2686.696487] limepcie 0000:01:00.0: [device info] LimeSDR-Unknown FW:255 HW:255 PROTOCOL:255 S/N:0xFFFFFFFFFFFFFFFF 
[ 2686.706980] limepcie 0000:01:00.0: 4.000 Gb/s available PCIe bandwidth, limited by 5.0 GT/s PCIe x1 link at 0000:00:00.0 (capable of 8.000 Gb/s with 5.0 GT/s PCIe x2 link)
[ 2686.722351] limepcie 0000:01:00.0: Required DMA mask ffffffffffffffff. Attempting 64bit mask.
[ 2686.730968] limepcie 0000:01:00.0: 1 MSI IRQs allocated.
[ 2686.736308] limepcie 0000:01:00.0: DMA buffer size not specified, defaulting to 8192
[ 2686.744080] limepcie 0000:01:00.0: Invalid DMA channel count(-1)
[ 2686.750108] limepcie 0000:01:00.0: Creating /dev/limepcie0/control0
[ 2725.930712] limepcie 0000:01:00.0: [Removing device] vid:10EE pid:7023
[ 2725.937279] limepcie 0000:01:00.0: FreeIRQs 1
[ 2725.942045] limepcie : module exit
  • The relevant section of lspci -vvvv is below:
0000:01:00.0 Multimedia audio controller: Xilinx Corporation Device 7023 (rev 01)                                                                                                  
        Subsystem: Xilinx Corporation Device 0007                                                                                                                                  
        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-                                                                      
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-                                                                       
        Region 0: Memory at 1b80000000 (32-bit, non-prefetchable) [virtual] [size=1M]                                                                                              
        Capabilities: [40] Power Management version 3                                                                                                                              
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-)                                                                                         
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-                                                                                                             
        Capabilities: [48] MSI: Enable- Count=1/1 Maskable- 64bit-                                                                                                                 
                Address: 00000000  Data: 0000                                                                                                                                      
        Capabilities: [60] Express (v2) Endpoint, MSI 00                                                                                                                           
                DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <64ns, L1 unlimited                                                                                         
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0W                                                                                          
                DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-                                                                                                                  
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+                                                                                                               
                        MaxPayload 128 bytes, MaxReadReq 512 bytes                                                                                                                 
                DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-                                                                                               
                LnkCap: Port #0, Speed 5GT/s, Width x2, ASPM L0s, Exit Latency L0s unlimited                                                                                       
                        ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-                                                                                                           
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-                                                                                                            
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-                                                                                                             
                LnkSta: Speed 5GT/s, Width x1 (downgraded)                                                                                                                         
                        TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-                                                                                                          
                DevCap2: Completion Timeout: Range B, TimeoutDis- NROPrPrP- LTR-                                                                                                   
                         10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-                                                                                       
                         EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-                                                                                       
                         FRS- TPHComp- ExtTPHComp-                                                                                                                                 
                         AtomicOpsCap: 32bit- 64bit- 128bitCAS-                                                                                                                    
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- 10BitTagReq- OBFF Disabled,                                                                            
                         AtomicOpsCtl: ReqEn-                                                                                                                                      
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-                                                                                                      
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-                                                                          
                         Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot                                                                                             
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-                                                                                
                         EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-                                                                                          
                         Retimer- 2Retimers- CrosslinkRes: unsupported                                                                                                             
        Capabilities: [100 v1] Device Serial Number 00-00-00-00-00-00-00-00                                                                                                        
        Kernel modules: limepcie

An earlier version of the LimeSuiteNG repository using litepcie kernel module seemed to have a similar non-functional response.

Is there something that anyone would recommend other than soldering to the JTAG port?

Thanks!

edit: forgot to mention, the kernel is forcing 32bit DMA ( comes out of this thread ), without it the device is not found at all.

Hi,

well, the PCIe communication is working and being enumerated, so the FPGA is configured. But the control and status registers aren’t functioning. I suspect that it has been flashed with incompatible gateware (For example flashing gateware intended for Fairwaves XTRX Rev5 into LimeSDR XTRX).
If that’s the case, then JTAG will be required to fix that.

Also I suggest to update to the latest ‘develop’ branch of LimeSuiteNG, it will not affect this issue, but there has been other fixes regarding DMA usage.