We are testing RecoNIC setup with two U55C's boards , with an HBM instead of a ddr4. The DMA tests work perfectly fine whereas rdma tests are malfunctioning, here's an output example of an rdma read from both client and server :
Output from Peer 1 (Client side) :
kamilgiorgio@kamilgiorgio-Standard-PC-Q35-ICH9-2009:~/Documents/RecoNIC/examples/rdma_test$ sudo env LD_LIBRARY_PATH=$LD_LIBRARY_PATH ./read -r 192.100.51.1 -i 192.100.52.1 -p /sys/bus/pci/devices/0000:07:00.0/resource2 -z 128 -l host_mem -d /dev/reconic-mm -c -u 22222 -t 11111 --dst_qp 2 -g 2>&1 | tee client_debug.log
src_ip_str = 192.100.51.1
dst_ip_str = 192.100.52.1
Info: mac_addr_t = 00:0a:35:ca:26:34
Info: PCIe resource file: /sys/bus/pci/devices/0000:07:00.0/resource2
Info: QP allocated at: host_mem
Info: Device - /dev/reconic-mm
Info: src_ip = 192.100.51.1
Info: Found network interface: enp7s0
Info: mac_addr_t = 00:0a:35:92:12:ab
Info: Creating rn_dev
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:321:create_rn_dev(): Info: scr(=4)) file open successfully
create_rn_dev - testing2
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:154:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4e000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:159:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:163:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4e000000
Info: pre-allocated hugepage buffer vir addr = 0x7f88d7c00000, physical addr = 0x4e000000
Info: Configuring 8 windows in QDMA AXI bridge BDF, each has 128GB mapping
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:206:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16420, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:207:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16424, bdf_addr_high=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16430, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:206:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16440, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:207:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16444, bdf_addr_high=0x20
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16450, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:206:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16460, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:207:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16464, bdf_addr_high=0x40
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16470, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:206:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16480, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:207:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16484, bdf_addr_high=0x60
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16490, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:206:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x164a0, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:207:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x164a4, bdf_addr_high=0x80
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x164b0, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:206:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x164c0, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:207:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x164c4, bdf_addr_high=0xa0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x164d0, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:206:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x164e0, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:207:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x164e4, bdf_addr_high=0xc0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x164f0, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:206:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16500, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:207:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16504, bdf_addr_high=0xe0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16510, bdf_win_config=0xc2000000
Info: CREATE RDMA DEVICE
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:154:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4e000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:159:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:163:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4e000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:240:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f88d7c00000, physical addr = 4e000000, rn_dev->buffer_offset = 0x200000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:241:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:154:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4e200
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:159:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:163:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4e200000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:240:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f88d7e00000, physical addr = 4e200000, rn_dev->buffer_offset = 0x1200000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:241:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:154:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4f200
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:159:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:163:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4f200000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:240:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f88d8e00000, physical addr = 4f200000, rn_dev->buffer_offset = 0x1202000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:241:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:154:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4f202
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:159:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:163:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4f202000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:240:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f88d8e02000, physical addr = 4f202000, rn_dev->buffer_offset = 0x1212000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:241:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:154:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4f212
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:159:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:163:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4f212000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:240:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f88d8e12000, physical addr = 4f212000, rn_dev->buffer_offset = 0x1222000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:241:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
Info: OPEN RDMA DEVICE
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:186:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_DATBUFBA=0x600a0, value=0x4e200000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:188:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_DATBUFBAMSB=0x600a4, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:190:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_DATBUFSZ=0x600a8, value=0x10001000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:193:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_IPKTERRQBA=0x60088, value=0x4f200000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:195:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_IPKTERRQBAMSB=0x6008c, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:197:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_ERRBUFSZ=0x60090, value=0x2000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:200:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_ERRBUFBA=0x60060, value=0x4f202000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:202:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_ERRBUFBAMSB=0x60064, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:204:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_ERRBUFSZ=0x60068, value=0x1000100
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:207:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_RESPERRPKTBA=0x600b0, value=0x4f212000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:209:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_RESPERRPKTBAMSB=0x600b4, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:211:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_RESPERRSZ=0x600b8, value=0x10000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:213:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_RESPERRSZMSB=0x600bc, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:217:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_INTEN=0x60180, value=0xff
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:221:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_MACXADDLSB=0x60010, value=0x359212ab
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:223:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_MACXADDMSB=0x60014, value=0xa
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:227:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_IPV4XADD=0x60070, value=0xc0643301
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:230:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_XRNICCONF=0x60000, value=0x56ce0821
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:233:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_XRNICADCONF=0x60004, value=0xa0004
Info: RDMA global control status registers are configured.
Info: rdma_dev opened
Info: ALLOCATE PD
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:254:allocate_rdma_pd(): [Register] RN_RDMA_PDT_PDPDNUM=0x40000, pd_num=0, value=0x0
Info: OPEN DEVICE FILE
Info: ALLOCATE RDMA QP
Allocating qp->sq
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:437:allocate_rdma_qp(): sq_size = 32768, cq_size = 2048, rq_size 262144, buf_location = host_mem
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:154:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4f222
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:159:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:163:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4f222000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:240:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f88d8e22000, physical addr = 4f222000, rn_dev->buffer_offset = 0x122a000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:241:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
Allocating qp->cq
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:154:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4f22a
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:159:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:163:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4f22a000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:240:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f88d8e2a000, physical addr = 4f22a000, rn_dev->buffer_offset = 0x122a800
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:241:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
Allocating qp->rq
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:154:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4f22b
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:159:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:163:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4f22b000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:240:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f88d8e2b000, physical addr = 4f22b000, rn_dev->buffer_offset = 0x126b000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:241:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
Info: queue pair setting is done! Configuring RDMA per-queu CSR registers
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:485:allocate_rdma_qp(): DEBUG: rdma_dev->rn_dev->axil_ctl = 0x7f88f7c15000, rdma_dev->axil_ctl = 0x7f88f7c15000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:499:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_IPDESADDR1i=0x60360, qpid=2, value=0xc0643401
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:506:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_MACDESADDLSBi=0x60350, qpid=2, value=0x35ca2634
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:513:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_MACDESADDMSBi=0x60354, qpid=2, value=0xa
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:519:allocate_rdma_qp(): DEBUG: win_size_high = 0xff, win_size_low = 0xffffffff
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:536:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_SQBAi=0x60310, qpid=2, value=0x4f222000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:543:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_SQBAMSBi=0x603c8, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:547:allocate_rdma_qp(): DEBUG: qp->sq->dma_addr = 0x4f222000, sq_addr_msb = 0x0, sq_addr_lsb = 0x4f222000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:565:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_CQBAi=0x60318, qpid=2, value=0x4f22a000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:572:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_CQBAMSBi=0x603d0, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:576:allocate_rdma_qp(): DEBUG: qp->cq->dma_addr = 0x4f22a000, cq_addr_msb = 0x0, cq_addr_lsb = 0x4f22a000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:594:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_RQBAi=0x60308, qpid=2, value=0x4f22b000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:601:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_RQBAMSBi=0x603c0, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:605:allocate_rdma_qp(): DEBUG: qp->rq->dma_addr = 0x4f22b000, rq_addr_msb = 0x0, rq_addr_lsb = 0x4f22b000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:614:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_CQDBADDi=0x60328, qpid=2, value=0x4e000000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:621:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_CQDBADDMSBi=0x6032c, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:625:allocate_rdma_qp(): DEBUG: cq_cidb_addr = 0x4e000000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:631:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_RQWPTRDBADDi=0x60320, qpid=2, value=0x4e000020
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:638:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_RQWPTRDBADDMSBi=0x60324, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:642:allocate_rdma_qp(): DEBUG: rq_cidb_addr = 0x4e000020
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:648:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_DESTQPCONFi=0x60348, qpid=2, value=0x2
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:657:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_QDEPTHi=0x6033c, qpid=2, value=0x400040
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:704:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_QPCONFi=0x60300, qpid=2, value=0x200043d
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:718:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_QPADVCONFi=0x60304, qpid=2, value=0x12344000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:727:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_PDi=0x603b0, qpid=2, value=0x0
Info: allocate_rdma_qp - Successfully allocated a rdma qp
Info: CONFIGURE PSN
[Register] RN_RDMA_QCSR_LSTRQREQi=0x60344, qpid=2, value=0xa000abc
[Register] RN_RDMA_QCSR_SQPSNi=0x60340, qpid=2, value=0xabd
payload_size = 128, payload_size>>2 = 32
Info: Client is connecting to a remote server
Info: Client is connected to a remote server
Info: client received remote offset of A = 0xa350000000000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:273:allocate_rdma_buffer(): Info: allocated device buffer physical addr = a350000000000000, rn_dev->dev_buffer_offset = 0x80
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:278:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma device buffer
Info: creating an RDMA read WQE for getting data
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:769:create_a_wqe(): Info: WQE mem_buffer = 0xa350000000000000, masked_mem_buffer = 0xa350000000000000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:796:create_a_wqe(): [WQE] wrid=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:797:create_a_wqe(): [WQE] laddr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:798:create_a_wqe(): [WQE] laddr_high=0xa3500000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:799:create_a_wqe(): [WQE] length=0x80
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:800:create_a_wqe(): [WQE] opcode=0x4
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:801:create_a_wqe(): [WQE] remote_offset_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:802:create_a_wqe(): [WQE] remote_offset_high=0xa3500000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:803:create_a_wqe(): [WQE] r_key=0x8
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:804:create_a_wqe(): [WQE] send_small_payload0=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:805:create_a_wqe(): [WQE] send_small_payload1=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:806:create_a_wqe(): [WQE] send_small_payload2=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:807:create_a_wqe(): [WQE] send_small_payload3=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:808:create_a_wqe(): [WQE] immdt_data=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:875:rdma_post_send(): DEBUG: Reading hardware SQPIi (0x60338) = 0x1
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:876:rdma_post_send(): DEBUG: original qp->sq_pidb = 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:882:rdma_post_send(): [Register] RN_RDMA_QCSR_SQPIi=0x60338, qpid=2, value=0x1
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:883:rdma_post_send(): DEBUG: Update hardware sq db idx from software = 1
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:884:rdma_post_send(): DEBUG: Reading hardware SQPIi (0x60338) = 0x1
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:844:poll_cq_cidb(): [Register] RN_RDMA_QCSR_CQHEADi=0x60330, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:846:poll_cq_cidb(): DEBUG: before polling: sq_cidb = 0; Polling CQ CIDB = 0
ERROR: poll_cq_cidb timeout! sq_cidb = 0; Polling CQ CIDB = 0
Info: Dump register values for debug purpose
Info: [RN_RDMA_GCSR_ERRBUFWPTR = 0x6006c] = 0x0
Info: [RN_RDMA_GCSR_IPKTERRQWPTR = 0x60094] = 0x0
Info: [RN_RDMA_GCSR_INSRRPKTCNT = 0x60100] = 0x0
Info: [RN_RDMA_GCSR_INAMPKTCNT = 0x60104] = 0x0
Info: [RN_RDMA_GCSR_OUTIOPKTCNT = 0x60108] = 0x10000
Info: [RN_RDMA_GCSR_OUTAMPKTCNT = 0x6010c] = 0x0
Info: [RN_RDMA_GCSR_LSTINPKT = 0x60110] = 0x2
Info: [RN_RDMA_GCSR_LSTOUTPKT = 0x60114] = 0x157a204
Info: [RN_RDMA_GCSR_ININVDUPCNT = 0x60118] = 0x0
Info: [RN_RDMA_GCSR_INNCKPKTSTS = 0x6011c] = 0x0
Info: [RN_RDMA_GCSR_OUTRNRPKTSTS = 0x60120] = 0x0
Info: [RN_RDMA_GCSR_WQEPROCSTS = 0x60124] = 0x1012040
Info: [RN_RDMA_GCSR_QPMSTS = 0x6012c] = 0x10002
Info: [RN_RDMA_GCSR_INALLDRPPKTCNT = 0x60130] = 0x0
Info: [RN_RDMA_GCSR_INNAKPKTCNT = 0x60134] = 0x0
Info: [RN_RDMA_GCSR_OUTNAKPKTCNT = 0x60138] = 0x0
Info: [RN_RDMA_GCSR_RESPHNDSTS = 0x6013c] = 0x30202
Info: [RN_RDMA_GCSR_RETRYCNTSTS = 0x60140] = 0x0
Info: [RN_RDMA_GCSR_INCNPPKTCNT = 0x60174] = 0x0
Info: [RN_RDMA_GCSR_OUTCNPPKTCNT = 0x60178] = 0x0
Info: [RN_RDMA_GCSR_OUTRDRSPPKTCNT = 0x6017c] = 0x0
Info: [RN_RDMA_GCSR_INTSTS = 0x60184] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS1 = 0x60190] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS2 = 0x60194] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS3 = 0x60198] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS4 = 0x6019c] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS5 = 0x601a0] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS6 = 0x601a4] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS7 = 0x601a8] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS8 = 0x601ac] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS1 = 0x601b0] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS2 = 0x601b4] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS3 = 0x601b8] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS4 = 0x601bc] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS5 = 0x601c0] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS6 = 0x601c4] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS7 = 0x601c8] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS8 = 0x601cc] = 0x0
Info: [RN_RDMA_QCSR_CQHEADi = 0x60330] = 0x0
Info: [RN_RDMA_QCSR_STATSSNi = 0x60380] = 0x1
Info: [RN_RDMA_QCSR_STATMSNi = 0x60384] = 0x0
Info: [RN_RDMA_QCSR_STATQPi = 0x60388] = 0x200
Info: [RN_RDMA_QCSR_STATCURSQPTRi = 0x6038c] = 0x1
Info: [RN_RDMA_QCSR_STATRESPSNi = 0x60390] = 0xabc
Info: [RN_RDMA_QCSR_STATRQBUFCAi = 0x60394] = 0x4f22b000
Info: [RN_RDMA_QCSR_STATWQEi = 0x60398] = 0x0
Info: [RN_RDMA_QCSR_STATRQPIDBi = 0x6039c] = 0x0
Info: [RN_RDMA_QCSR_STATRQBUFCAMSBi = 0x603d8] = 0x0
Info: [RN_RDMA_QCSR_SQPIi = 0x60338] = 0x1
Failed to send an RDMA read operation
Info: Dump register values for debug purpose
Info: [RN_RDMA_GCSR_ERRBUFWPTR = 0x6006c] = 0x0
Info: [RN_RDMA_GCSR_IPKTERRQWPTR = 0x60094] = 0x0
Info: [RN_RDMA_GCSR_INSRRPKTCNT = 0x60100] = 0x0
Info: [RN_RDMA_GCSR_INAMPKTCNT = 0x60104] = 0x0
Info: [RN_RDMA_GCSR_OUTIOPKTCNT = 0x60108] = 0x10000
Info: [RN_RDMA_GCSR_OUTAMPKTCNT = 0x6010c] = 0x0
Info: [RN_RDMA_GCSR_LSTINPKT = 0x60110] = 0x2
Info: [RN_RDMA_GCSR_LSTOUTPKT = 0x60114] = 0x157a204
Info: [RN_RDMA_GCSR_ININVDUPCNT = 0x60118] = 0x0
Info: [RN_RDMA_GCSR_INNCKPKTSTS = 0x6011c] = 0x0
Info: [RN_RDMA_GCSR_OUTRNRPKTSTS = 0x60120] = 0x0
Info: [RN_RDMA_GCSR_WQEPROCSTS = 0x60124] = 0x1012040
Info: [RN_RDMA_GCSR_QPMSTS = 0x6012c] = 0x10002
Info: [RN_RDMA_GCSR_INALLDRPPKTCNT = 0x60130] = 0x0
Info: [RN_RDMA_GCSR_INNAKPKTCNT = 0x60134] = 0x0
Info: [RN_RDMA_GCSR_OUTNAKPKTCNT = 0x60138] = 0x0
Info: [RN_RDMA_GCSR_RESPHNDSTS = 0x6013c] = 0x30202
Info: [RN_RDMA_GCSR_RETRYCNTSTS = 0x60140] = 0x0
Info: [RN_RDMA_GCSR_INCNPPKTCNT = 0x60174] = 0x0
Info: [RN_RDMA_GCSR_OUTCNPPKTCNT = 0x60178] = 0x0
Info: [RN_RDMA_GCSR_OUTRDRSPPKTCNT = 0x6017c] = 0x0
Info: [RN_RDMA_GCSR_INTSTS = 0x60184] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS1 = 0x60190] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS2 = 0x60194] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS3 = 0x60198] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS4 = 0x6019c] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS5 = 0x601a0] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS6 = 0x601a4] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS7 = 0x601a8] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS8 = 0x601ac] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS1 = 0x601b0] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS2 = 0x601b4] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS3 = 0x601b8] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS4 = 0x601bc] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS5 = 0x601c0] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS6 = 0x601c4] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS7 = 0x601c8] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS8 = 0x601cc] = 0x0
Info: [RN_RDMA_QCSR_CQHEADi = 0x60330] = 0x0
Info: [RN_RDMA_QCSR_STATSSNi = 0x60380] = 0x1
Info: [RN_RDMA_QCSR_STATMSNi = 0x60384] = 0x0
Info: [RN_RDMA_QCSR_STATQPi = 0x60388] = 0x200
Info: [RN_RDMA_QCSR_STATCURSQPTRi = 0x6038c] = 0x1
Info: [RN_RDMA_QCSR_STATRESPSNi = 0x60390] = 0xabc
Info: [RN_RDMA_QCSR_STATRQBUFCAi = 0x60394] = 0x4f22b000
Info: [RN_RDMA_QCSR_STATWQEi = 0x60398] = 0x0
Info: [RN_RDMA_QCSR_STATRQPIDBi = 0x6039c] = 0x0
Info: [RN_RDMA_QCSR_STATRQBUFCAMSBi = 0x603d8] = 0x0
Info: [RN_RDMA_QCSR_SQPIi = 0x60338] = 0x1
Info: All data has been received!
Info: buffer physical address is 0xa350000000000000
Info: Time spent 1174.769000 usec, size = 128 bytes, Bandwidth = 0.000872 gigabits/sec
Info: The value of rc is 128
Info: CHECK RECEIVED DATA
Error: received data mismatched: recv[1]=0, sw_golden[1]=1
Warning: QP in fatal status
***** QP2 FATAL RECOVERY *****
TIMEOUT: CQHEADi:0x0 and SQPIi:0x1 are different
Output from Peer 2 (Server side) :
kamilgiorgio@kamilgiorgio-Standard-PC-Q35-ICH9-2009:~/Documents/RecoNIC/examples/rdma_test$ sudo env LD_LIBRARY_PATH=$LD_LIBRARY_PATH ./read -r 192.100.52.1 -i 192.100.51.1 -p /sys/bus/pci/devices/0000:07:00.0/resource2 -z 128 -l host_mem -d /dev/reconic-mm -s -u 22222 -t 11111 --dst_qp 2 -g 2>&1 | tee server_debug.log
src_ip_str = 192.100.52.1
dst_ip_str = 192.100.51.1
Info: mac_addr_t = 00:0a:35:92:12:ab
Info: PCIe resource file: /sys/bus/pci/devices/0000:07:00.0/resource2
Info: QP allocated at: host_mem
Info: Device - /dev/reconic-mm
Info: src_ip = 192.100.52.1
Info: Found network interface: enp7s0
Info: mac_addr_t = 00:0a:35:ca:26:34
Info: Creating rn_dev
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:323:create_rn_dev(): Info: scr(=4)) file open successfully
create_rn_dev - testing2
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:156:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4e800
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:161:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:165:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4e800000
Info: pre-allocated hugepage buffer vir addr = 0x7f24afe00000, physical addr = 0x4e800000
Info: Configuring 8 windows in QDMA AXI bridge BDF, each has 128GB mapping
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16420, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:209:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16424, bdf_addr_high=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:210:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16430, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16440, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:209:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16444, bdf_addr_high=0x20
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:210:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16450, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16460, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:209:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16464, bdf_addr_high=0x40
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:210:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16470, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16480, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:209:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16484, bdf_addr_high=0x60
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:210:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16490, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x164a0, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:209:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x164a4, bdf_addr_high=0x80
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:210:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x164b0, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x164c0, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:209:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x164c4, bdf_addr_high=0xa0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:210:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x164d0, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x164e0, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:209:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x164e4, bdf_addr_high=0xc0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:210:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x164f0, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16500, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:209:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16504, bdf_addr_high=0xe0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:210:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16510, bdf_win_config=0xc2000000
Info: CREATE RDMA DEVICE
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:156:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4e800
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:161:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:165:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4e800000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:242:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f24afe00000, physical addr = 4e800000, rn_dev->buffer_offset = 0x200000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:243:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:156:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4ea00
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:161:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:165:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4ea00000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:242:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f24b0000000, physical addr = 4ea00000, rn_dev->buffer_offset = 0x1200000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:243:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:156:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4fa00
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:161:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:165:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4fa00000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:242:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f24b1000000, physical addr = 4fa00000, rn_dev->buffer_offset = 0x1202000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:243:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:156:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4fa02
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:161:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:165:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4fa02000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:242:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f24b1002000, physical addr = 4fa02000, rn_dev->buffer_offset = 0x1212000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:243:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:156:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4fa12
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:161:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:165:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4fa12000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:242:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f24b1012000, physical addr = 4fa12000, rn_dev->buffer_offset = 0x1222000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:243:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
Info: OPEN RDMA DEVICE
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:186:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_DATBUFBA=0x600a0, value=0x4ea00000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:188:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_DATBUFBAMSB=0x600a4, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:190:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_DATBUFSZ=0x600a8, value=0x10001000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:193:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_IPKTERRQBA=0x60088, value=0x4fa00000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:195:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_IPKTERRQBAMSB=0x6008c, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:197:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_ERRBUFSZ=0x60090, value=0x2000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:200:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_ERRBUFBA=0x60060, value=0x4fa02000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:202:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_ERRBUFBAMSB=0x60064, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:204:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_ERRBUFSZ=0x60068, value=0x1000100
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:207:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_RESPERRPKTBA=0x600b0, value=0x4fa12000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:209:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_RESPERRPKTBAMSB=0x600b4, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:211:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_RESPERRSZ=0x600b8, value=0x10000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:213:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_RESPERRSZMSB=0x600bc, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:217:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_INTEN=0x60180, value=0xff
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:221:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_MACXADDLSB=0x60010, value=0x35ca2634
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:223:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_MACXADDMSB=0x60014, value=0xa
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:227:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_IPV4XADD=0x60070, value=0xc0643401
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:230:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_XRNICCONF=0x60000, value=0x56ce0821
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:233:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_XRNICADCONF=0x60004, value=0xa0004
Info: RDMA global control status registers are configured.
Info: rdma_dev opened
Info: ALLOCATE PD
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:254:allocate_rdma_pd(): [Register] RN_RDMA_PDT_PDPDNUM=0x40000, pd_num=0, value=0x0
Info: OPEN DEVICE FILE
Info: ALLOCATE RDMA QP
Allocating qp->sq
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:437:allocate_rdma_qp(): sq_size = 32768, cq_size = 2048, rq_size 262144, buf_location = host_mem
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:156:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4fa22
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:161:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:165:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4fa22000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:242:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f24b1022000, physical addr = 4fa22000, rn_dev->buffer_offset = 0x122a000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:243:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
Allocating qp->cq
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:156:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4fa2a
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:161:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:165:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4fa2a000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:242:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f24b102a000, physical addr = 4fa2a000, rn_dev->buffer_offset = 0x122a800
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:243:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
Allocating qp->rq
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:156:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4fa2b
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:161:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:165:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4fa2b000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:242:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f24b102b000, physical addr = 4fa2b000, rn_dev->buffer_offset = 0x126b000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:243:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
Info: queue pair setting is done! Configuring RDMA per-queu CSR registers
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:485:allocate_rdma_qp(): DEBUG: rdma_dev->rn_dev->axil_ctl = 0x7f24cfeae000, rdma_dev->axil_ctl = 0x7f24cfeae000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:499:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_IPDESADDR1i=0x60360, qpid=2, value=0xc0643301
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:506:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_MACDESADDLSBi=0x60350, qpid=2, value=0x359212ab
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:513:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_MACDESADDMSBi=0x60354, qpid=2, value=0xa
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:519:allocate_rdma_qp(): DEBUG: win_size_high = 0xff, win_size_low = 0xffffffff
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:536:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_SQBAi=0x60310, qpid=2, value=0x4fa22000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:543:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_SQBAMSBi=0x603c8, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:547:allocate_rdma_qp(): DEBUG: qp->sq->dma_addr = 0x4fa22000, sq_addr_msb = 0x0, sq_addr_lsb = 0x4fa22000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:565:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_CQBAi=0x60318, qpid=2, value=0x4fa2a000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:572:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_CQBAMSBi=0x603d0, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:576:allocate_rdma_qp(): DEBUG: qp->cq->dma_addr = 0x4fa2a000, cq_addr_msb = 0x0, cq_addr_lsb = 0x4fa2a000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:594:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_RQBAi=0x60308, qpid=2, value=0x4fa2b000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:601:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_RQBAMSBi=0x603c0, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:605:allocate_rdma_qp(): DEBUG: qp->rq->dma_addr = 0x4fa2b000, rq_addr_msb = 0x0, rq_addr_lsb = 0x4fa2b000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:614:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_CQDBADDi=0x60328, qpid=2, value=0x4e800000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:621:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_CQDBADDMSBi=0x6032c, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:625:allocate_rdma_qp(): DEBUG: cq_cidb_addr = 0x4e800000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:631:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_RQWPTRDBADDi=0x60320, qpid=2, value=0x4e800020
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:638:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_RQWPTRDBADDMSBi=0x60324, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:642:allocate_rdma_qp(): DEBUG: rq_cidb_addr = 0x4e800020
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:648:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_DESTQPCONFi=0x60348, qpid=2, value=0x2
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:657:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_QDEPTHi=0x6033c, qpid=2, value=0x400040
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:704:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_QPCONFi=0x60300, qpid=2, value=0x200043d
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:718:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_QPADVCONFi=0x60304, qpid=2, value=0x12344000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:727:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_PDi=0x603b0, qpid=2, value=0x0
Info: allocate_rdma_qp - Successfully allocated a rdma qp
Info: CONFIGURE PSN
[Register] RN_RDMA_QCSR_LSTRQREQi=0x60344, qpid=2, value=0xa000abc
[Register] RN_RDMA_QCSR_SQPSNi=0x60340, qpid=2, value=0xabd
payload_size = 128, payload_size>>2 = 32
Info: Server is listening to a remote peer
Info: Server is connected to a remote peer
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:275:allocate_rdma_buffer(): Info: allocated device buffer physical addr = a350000000000000, rn_dev->dev_buffer_offset = 0x80
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:280:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma device buffer
Info: rdma_register_memory_region - registering memory region
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:316:rdma_register_memory_region(): [Register] RN_RDMA_PDT_VIRTADDRLSB=0x40004, pd_num=0, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:318:rdma_register_memory_region(): [Register] RN_RDMA_PDT_VIRTADDRMSB=0x40008, pd_num=0, value=0xa3500000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:320:rdma_register_memory_region(): [Register] RN_RDMA_PDT_BUFBASEADDRLSB=0x4000c, pd_num=0, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:322:rdma_register_memory_region(): [Register] RN_RDMA_PDT_BUFBASEADDRMSB=0x40010, pd_num=0, value=0xa3500000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:324:rdma_register_memory_region(): [Register] RN_RDMA_PDT_BUFRKEY=0x40014, pd_num=0, value=0x8
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:327:rdma_register_memory_region(): [Register] RN_RDMA_PDT_WRRDBUFLEN=0x40018, pd_num=0, value=0x80 B
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:330:rdma_register_memory_region(): [Register] RN_RDMA_PDT_ACCESSDESC=0x4001c, pd_num=0, value=0x2
Info: memory region for the 0-th PD is registered
Info: allocating buffer for payload data
Info: tmp_buffer->buffer = 0xa350000000000000, tmp_buffer->dma_addr = 0xa350000000000000
Info: copy payload data to the device memory
Info: copied payload data to the device memory succesfully rc = 128
Sending read_offset (a350000000000000) to the remote client
Does the client finish its RDMA read operation? If yes, please press any key
Info: Dump register values for debug purpose
Info: [RN_RDMA_GCSR_ERRBUFWPTR = 0x6006c] = 0x0
Info: [RN_RDMA_GCSR_IPKTERRQWPTR = 0x60094] = 0x0
Info: [RN_RDMA_GCSR_INSRRPKTCNT = 0x60100] = 0x0
Info: [RN_RDMA_GCSR_INAMPKTCNT = 0x60104] = 0x0
Info: [RN_RDMA_GCSR_OUTIOPKTCNT = 0x60108] = 0x0
Info: [RN_RDMA_GCSR_OUTAMPKTCNT = 0x6010c] = 0x0
Info: [RN_RDMA_GCSR_LSTINPKT = 0x60110] = 0xabd020c
Info: [RN_RDMA_GCSR_LSTOUTPKT = 0x60114] = 0x0
Info: [RN_RDMA_GCSR_ININVDUPCNT = 0x60118] = 0x0
Info: [RN_RDMA_GCSR_INNCKPKTSTS = 0x6011c] = 0x0
Info: [RN_RDMA_GCSR_OUTRNRPKTSTS = 0x60120] = 0x0
Info: [RN_RDMA_GCSR_WQEPROCSTS = 0x60124] = 0x2022040
Info: [RN_RDMA_GCSR_QPMSTS = 0x6012c] = 0x2
Info: [RN_RDMA_GCSR_INALLDRPPKTCNT = 0x60130] = 0x10000
Info: [RN_RDMA_GCSR_INNAKPKTCNT = 0x60134] = 0x0
Info: [RN_RDMA_GCSR_OUTNAKPKTCNT = 0x60138] = 0x0
Info: [RN_RDMA_GCSR_RESPHNDSTS = 0x6013c] = 0x10000
Info: [RN_RDMA_GCSR_RETRYCNTSTS = 0x60140] = 0x0
Info: [RN_RDMA_GCSR_INCNPPKTCNT = 0x60174] = 0x0
Info: [RN_RDMA_GCSR_OUTCNPPKTCNT = 0x60178] = 0x0
Info: [RN_RDMA_GCSR_OUTRDRSPPKTCNT = 0x6017c] = 0x1
Info: [RN_RDMA_GCSR_INTSTS = 0x60184] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS1 = 0x60190] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS2 = 0x60194] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS3 = 0x60198] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS4 = 0x6019c] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS5 = 0x601a0] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS6 = 0x601a4] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS7 = 0x601a8] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS8 = 0x601ac] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS1 = 0x601b0] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS2 = 0x601b4] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS3 = 0x601b8] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS4 = 0x601bc] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS5 = 0x601c0] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS6 = 0x601c4] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS7 = 0x601c8] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS8 = 0x601cc] = 0x0
Info: [RN_RDMA_QCSR_CQHEADi = 0x60330] = 0x0
Info: [RN_RDMA_QCSR_STATSSNi = 0x60380] = 0x0
Info: [RN_RDMA_QCSR_STATMSNi = 0x60384] = 0x0
Info: [RN_RDMA_QCSR_STATQPi = 0x60388] = 0x600
Info: [RN_RDMA_QCSR_STATCURSQPTRi = 0x6038c] = 0x0
Info: [RN_RDMA_QCSR_STATRESPSNi = 0x60390] = 0xabc
Info: [RN_RDMA_QCSR_STATRQBUFCAi = 0x60394] = 0x4fa2b000
Info: [RN_RDMA_QCSR_STATWQEi = 0x60398] = 0x0
Info: [RN_RDMA_QCSR_STATRQPIDBi = 0x6039c] = 0x0
Info: [RN_RDMA_QCSR_STATRQBUFCAMSBi = 0x603d8] = 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:1087:destroy_rdma_qp(): [DEBUG] Destroying dev: 0x7f24cfeae000, RN_RDMA_QCSR_CQHEADi=0x60330, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:1092:destroy_rdma_qp(): [DEBUG] Destroying dev: 0x7f24cfeae000, RN_RDMA_QCSR_CQHEADi=0x60330, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:1100:destroy_rdma_qp(): [DEBUG] Destroying dev: 0x7f24cfeae000, RN_RDMA_QCSR_CQHEADi=0x60330, qpid=2, value=0x0
kamilgiorgio@kamilgiorgio-Standard-PC-Q35-ICH9-2009:~/Documents/RecoNIC/examples/rdma_test$
We are testing RecoNIC setup with two U55C's boards , with an HBM instead of a ddr4. The DMA tests work perfectly fine whereas rdma tests are malfunctioning, here's an output example of an rdma read from both client and server :
Output from Peer 1 (Client side) :
kamilgiorgio@kamilgiorgio-Standard-PC-Q35-ICH9-2009:~/Documents/RecoNIC/examples/rdma_test$ sudo env LD_LIBRARY_PATH=$LD_LIBRARY_PATH ./read -r 192.100.51.1 -i 192.100.52.1 -p /sys/bus/pci/devices/0000:07:00.0/resource2 -z 128 -l host_mem -d /dev/reconic-mm -c -u 22222 -t 11111 --dst_qp 2 -g 2>&1 | tee client_debug.log
src_ip_str = 192.100.51.1
dst_ip_str = 192.100.52.1
Info: mac_addr_t = 00:0a:35:ca:26:34
Info: PCIe resource file: /sys/bus/pci/devices/0000:07:00.0/resource2
Info: QP allocated at: host_mem
Info: Device - /dev/reconic-mm
Info: src_ip = 192.100.51.1
Info: Found network interface: enp7s0
Info: mac_addr_t = 00:0a:35:92:12:ab
Info: Creating rn_dev
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:321:create_rn_dev(): Info: scr(=4)) file open successfully
create_rn_dev - testing2
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:154:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4e000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:159:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:163:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4e000000
Info: pre-allocated hugepage buffer vir addr = 0x7f88d7c00000, physical addr = 0x4e000000
Info: Configuring 8 windows in QDMA AXI bridge BDF, each has 128GB mapping
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:206:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16420, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:207:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16424, bdf_addr_high=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16430, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:206:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16440, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:207:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16444, bdf_addr_high=0x20
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16450, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:206:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16460, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:207:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16464, bdf_addr_high=0x40
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16470, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:206:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16480, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:207:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16484, bdf_addr_high=0x60
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16490, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:206:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x164a0, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:207:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x164a4, bdf_addr_high=0x80
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x164b0, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:206:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x164c0, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:207:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x164c4, bdf_addr_high=0xa0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x164d0, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:206:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x164e0, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:207:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x164e4, bdf_addr_high=0xc0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x164f0, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:206:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16500, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:207:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16504, bdf_addr_high=0xe0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16510, bdf_win_config=0xc2000000
Info: CREATE RDMA DEVICE
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:154:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4e000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:159:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:163:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4e000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:240:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f88d7c00000, physical addr = 4e000000, rn_dev->buffer_offset = 0x200000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:241:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:154:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4e200
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:159:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:163:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4e200000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:240:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f88d7e00000, physical addr = 4e200000, rn_dev->buffer_offset = 0x1200000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:241:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:154:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4f200
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:159:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:163:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4f200000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:240:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f88d8e00000, physical addr = 4f200000, rn_dev->buffer_offset = 0x1202000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:241:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:154:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4f202
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:159:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:163:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4f202000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:240:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f88d8e02000, physical addr = 4f202000, rn_dev->buffer_offset = 0x1212000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:241:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:154:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4f212
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:159:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:163:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4f212000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:240:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f88d8e12000, physical addr = 4f212000, rn_dev->buffer_offset = 0x1222000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:241:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
Info: OPEN RDMA DEVICE
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:186:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_DATBUFBA=0x600a0, value=0x4e200000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:188:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_DATBUFBAMSB=0x600a4, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:190:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_DATBUFSZ=0x600a8, value=0x10001000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:193:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_IPKTERRQBA=0x60088, value=0x4f200000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:195:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_IPKTERRQBAMSB=0x6008c, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:197:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_ERRBUFSZ=0x60090, value=0x2000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:200:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_ERRBUFBA=0x60060, value=0x4f202000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:202:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_ERRBUFBAMSB=0x60064, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:204:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_ERRBUFSZ=0x60068, value=0x1000100
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:207:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_RESPERRPKTBA=0x600b0, value=0x4f212000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:209:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_RESPERRPKTBAMSB=0x600b4, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:211:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_RESPERRSZ=0x600b8, value=0x10000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:213:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_RESPERRSZMSB=0x600bc, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:217:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_INTEN=0x60180, value=0xff
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:221:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_MACXADDLSB=0x60010, value=0x359212ab
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:223:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_MACXADDMSB=0x60014, value=0xa
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:227:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_IPV4XADD=0x60070, value=0xc0643301
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:230:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_XRNICCONF=0x60000, value=0x56ce0821
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:233:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_XRNICADCONF=0x60004, value=0xa0004
Info: RDMA global control status registers are configured.
Info: rdma_dev opened
Info: ALLOCATE PD
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:254:allocate_rdma_pd(): [Register] RN_RDMA_PDT_PDPDNUM=0x40000, pd_num=0, value=0x0
Info: OPEN DEVICE FILE
Info: ALLOCATE RDMA QP
Allocating qp->sq
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:437:allocate_rdma_qp(): sq_size = 32768, cq_size = 2048, rq_size 262144, buf_location = host_mem
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:154:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4f222
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:159:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:163:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4f222000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:240:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f88d8e22000, physical addr = 4f222000, rn_dev->buffer_offset = 0x122a000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:241:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
Allocating qp->cq
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:154:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4f22a
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:159:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:163:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4f22a000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:240:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f88d8e2a000, physical addr = 4f22a000, rn_dev->buffer_offset = 0x122a800
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:241:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
Allocating qp->rq
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:154:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4f22b
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:159:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:163:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4f22b000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:240:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f88d8e2b000, physical addr = 4f22b000, rn_dev->buffer_offset = 0x126b000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:241:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
Info: queue pair setting is done! Configuring RDMA per-queu CSR registers
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:485:allocate_rdma_qp(): DEBUG: rdma_dev->rn_dev->axil_ctl = 0x7f88f7c15000, rdma_dev->axil_ctl = 0x7f88f7c15000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:499:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_IPDESADDR1i=0x60360, qpid=2, value=0xc0643401
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:506:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_MACDESADDLSBi=0x60350, qpid=2, value=0x35ca2634
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:513:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_MACDESADDMSBi=0x60354, qpid=2, value=0xa
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:519:allocate_rdma_qp(): DEBUG: win_size_high = 0xff, win_size_low = 0xffffffff
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:536:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_SQBAi=0x60310, qpid=2, value=0x4f222000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:543:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_SQBAMSBi=0x603c8, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:547:allocate_rdma_qp(): DEBUG: qp->sq->dma_addr = 0x4f222000, sq_addr_msb = 0x0, sq_addr_lsb = 0x4f222000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:565:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_CQBAi=0x60318, qpid=2, value=0x4f22a000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:572:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_CQBAMSBi=0x603d0, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:576:allocate_rdma_qp(): DEBUG: qp->cq->dma_addr = 0x4f22a000, cq_addr_msb = 0x0, cq_addr_lsb = 0x4f22a000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:594:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_RQBAi=0x60308, qpid=2, value=0x4f22b000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:601:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_RQBAMSBi=0x603c0, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:605:allocate_rdma_qp(): DEBUG: qp->rq->dma_addr = 0x4f22b000, rq_addr_msb = 0x0, rq_addr_lsb = 0x4f22b000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:614:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_CQDBADDi=0x60328, qpid=2, value=0x4e000000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:621:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_CQDBADDMSBi=0x6032c, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:625:allocate_rdma_qp(): DEBUG: cq_cidb_addr = 0x4e000000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:631:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_RQWPTRDBADDi=0x60320, qpid=2, value=0x4e000020
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:638:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_RQWPTRDBADDMSBi=0x60324, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:642:allocate_rdma_qp(): DEBUG: rq_cidb_addr = 0x4e000020
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:648:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_DESTQPCONFi=0x60348, qpid=2, value=0x2
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:657:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_QDEPTHi=0x6033c, qpid=2, value=0x400040
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:704:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_QPCONFi=0x60300, qpid=2, value=0x200043d
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:718:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_QPADVCONFi=0x60304, qpid=2, value=0x12344000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:727:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_PDi=0x603b0, qpid=2, value=0x0
Info: allocate_rdma_qp - Successfully allocated a rdma qp
Info: CONFIGURE PSN
[Register] RN_RDMA_QCSR_LSTRQREQi=0x60344, qpid=2, value=0xa000abc
[Register] RN_RDMA_QCSR_SQPSNi=0x60340, qpid=2, value=0xabd
payload_size = 128, payload_size>>2 = 32
Info: Client is connecting to a remote server
Info: Client is connected to a remote server
Info: client received remote offset of A = 0xa350000000000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:273:allocate_rdma_buffer(): Info: allocated device buffer physical addr = a350000000000000, rn_dev->dev_buffer_offset = 0x80
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:278:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma device buffer
Info: creating an RDMA read WQE for getting data
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:769:create_a_wqe(): Info: WQE mem_buffer = 0xa350000000000000, masked_mem_buffer = 0xa350000000000000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:796:create_a_wqe(): [WQE] wrid=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:797:create_a_wqe(): [WQE] laddr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:798:create_a_wqe(): [WQE] laddr_high=0xa3500000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:799:create_a_wqe(): [WQE] length=0x80
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:800:create_a_wqe(): [WQE] opcode=0x4
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:801:create_a_wqe(): [WQE] remote_offset_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:802:create_a_wqe(): [WQE] remote_offset_high=0xa3500000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:803:create_a_wqe(): [WQE] r_key=0x8
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:804:create_a_wqe(): [WQE] send_small_payload0=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:805:create_a_wqe(): [WQE] send_small_payload1=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:806:create_a_wqe(): [WQE] send_small_payload2=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:807:create_a_wqe(): [WQE] send_small_payload3=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:808:create_a_wqe(): [WQE] immdt_data=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:875:rdma_post_send(): DEBUG: Reading hardware SQPIi (0x60338) = 0x1
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:876:rdma_post_send(): DEBUG: original qp->sq_pidb = 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:882:rdma_post_send(): [Register] RN_RDMA_QCSR_SQPIi=0x60338, qpid=2, value=0x1
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:883:rdma_post_send(): DEBUG: Update hardware sq db idx from software = 1
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:884:rdma_post_send(): DEBUG: Reading hardware SQPIi (0x60338) = 0x1
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:844:poll_cq_cidb(): [Register] RN_RDMA_QCSR_CQHEADi=0x60330, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:846:poll_cq_cidb(): DEBUG: before polling: sq_cidb = 0; Polling CQ CIDB = 0
ERROR: poll_cq_cidb timeout! sq_cidb = 0; Polling CQ CIDB = 0
Info: Dump register values for debug purpose
Info: [RN_RDMA_GCSR_ERRBUFWPTR = 0x6006c] = 0x0
Info: [RN_RDMA_GCSR_IPKTERRQWPTR = 0x60094] = 0x0
Info: [RN_RDMA_GCSR_INSRRPKTCNT = 0x60100] = 0x0
Info: [RN_RDMA_GCSR_INAMPKTCNT = 0x60104] = 0x0
Info: [RN_RDMA_GCSR_OUTIOPKTCNT = 0x60108] = 0x10000
Info: [RN_RDMA_GCSR_OUTAMPKTCNT = 0x6010c] = 0x0
Info: [RN_RDMA_GCSR_LSTINPKT = 0x60110] = 0x2
Info: [RN_RDMA_GCSR_LSTOUTPKT = 0x60114] = 0x157a204
Info: [RN_RDMA_GCSR_ININVDUPCNT = 0x60118] = 0x0
Info: [RN_RDMA_GCSR_INNCKPKTSTS = 0x6011c] = 0x0
Info: [RN_RDMA_GCSR_OUTRNRPKTSTS = 0x60120] = 0x0
Info: [RN_RDMA_GCSR_WQEPROCSTS = 0x60124] = 0x1012040
Info: [RN_RDMA_GCSR_QPMSTS = 0x6012c] = 0x10002
Info: [RN_RDMA_GCSR_INALLDRPPKTCNT = 0x60130] = 0x0
Info: [RN_RDMA_GCSR_INNAKPKTCNT = 0x60134] = 0x0
Info: [RN_RDMA_GCSR_OUTNAKPKTCNT = 0x60138] = 0x0
Info: [RN_RDMA_GCSR_RESPHNDSTS = 0x6013c] = 0x30202
Info: [RN_RDMA_GCSR_RETRYCNTSTS = 0x60140] = 0x0
Info: [RN_RDMA_GCSR_INCNPPKTCNT = 0x60174] = 0x0
Info: [RN_RDMA_GCSR_OUTCNPPKTCNT = 0x60178] = 0x0
Info: [RN_RDMA_GCSR_OUTRDRSPPKTCNT = 0x6017c] = 0x0
Info: [RN_RDMA_GCSR_INTSTS = 0x60184] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS1 = 0x60190] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS2 = 0x60194] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS3 = 0x60198] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS4 = 0x6019c] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS5 = 0x601a0] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS6 = 0x601a4] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS7 = 0x601a8] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS8 = 0x601ac] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS1 = 0x601b0] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS2 = 0x601b4] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS3 = 0x601b8] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS4 = 0x601bc] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS5 = 0x601c0] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS6 = 0x601c4] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS7 = 0x601c8] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS8 = 0x601cc] = 0x0
Info: [RN_RDMA_QCSR_CQHEADi = 0x60330] = 0x0
Info: [RN_RDMA_QCSR_STATSSNi = 0x60380] = 0x1
Info: [RN_RDMA_QCSR_STATMSNi = 0x60384] = 0x0
Info: [RN_RDMA_QCSR_STATQPi = 0x60388] = 0x200
Info: [RN_RDMA_QCSR_STATCURSQPTRi = 0x6038c] = 0x1
Info: [RN_RDMA_QCSR_STATRESPSNi = 0x60390] = 0xabc
Info: [RN_RDMA_QCSR_STATRQBUFCAi = 0x60394] = 0x4f22b000
Info: [RN_RDMA_QCSR_STATWQEi = 0x60398] = 0x0
Info: [RN_RDMA_QCSR_STATRQPIDBi = 0x6039c] = 0x0
Info: [RN_RDMA_QCSR_STATRQBUFCAMSBi = 0x603d8] = 0x0
Info: [RN_RDMA_QCSR_SQPIi = 0x60338] = 0x1
Failed to send an RDMA read operation
Info: Dump register values for debug purpose
Info: [RN_RDMA_GCSR_ERRBUFWPTR = 0x6006c] = 0x0
Info: [RN_RDMA_GCSR_IPKTERRQWPTR = 0x60094] = 0x0
Info: [RN_RDMA_GCSR_INSRRPKTCNT = 0x60100] = 0x0
Info: [RN_RDMA_GCSR_INAMPKTCNT = 0x60104] = 0x0
Info: [RN_RDMA_GCSR_OUTIOPKTCNT = 0x60108] = 0x10000
Info: [RN_RDMA_GCSR_OUTAMPKTCNT = 0x6010c] = 0x0
Info: [RN_RDMA_GCSR_LSTINPKT = 0x60110] = 0x2
Info: [RN_RDMA_GCSR_LSTOUTPKT = 0x60114] = 0x157a204
Info: [RN_RDMA_GCSR_ININVDUPCNT = 0x60118] = 0x0
Info: [RN_RDMA_GCSR_INNCKPKTSTS = 0x6011c] = 0x0
Info: [RN_RDMA_GCSR_OUTRNRPKTSTS = 0x60120] = 0x0
Info: [RN_RDMA_GCSR_WQEPROCSTS = 0x60124] = 0x1012040
Info: [RN_RDMA_GCSR_QPMSTS = 0x6012c] = 0x10002
Info: [RN_RDMA_GCSR_INALLDRPPKTCNT = 0x60130] = 0x0
Info: [RN_RDMA_GCSR_INNAKPKTCNT = 0x60134] = 0x0
Info: [RN_RDMA_GCSR_OUTNAKPKTCNT = 0x60138] = 0x0
Info: [RN_RDMA_GCSR_RESPHNDSTS = 0x6013c] = 0x30202
Info: [RN_RDMA_GCSR_RETRYCNTSTS = 0x60140] = 0x0
Info: [RN_RDMA_GCSR_INCNPPKTCNT = 0x60174] = 0x0
Info: [RN_RDMA_GCSR_OUTCNPPKTCNT = 0x60178] = 0x0
Info: [RN_RDMA_GCSR_OUTRDRSPPKTCNT = 0x6017c] = 0x0
Info: [RN_RDMA_GCSR_INTSTS = 0x60184] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS1 = 0x60190] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS2 = 0x60194] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS3 = 0x60198] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS4 = 0x6019c] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS5 = 0x601a0] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS6 = 0x601a4] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS7 = 0x601a8] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS8 = 0x601ac] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS1 = 0x601b0] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS2 = 0x601b4] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS3 = 0x601b8] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS4 = 0x601bc] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS5 = 0x601c0] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS6 = 0x601c4] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS7 = 0x601c8] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS8 = 0x601cc] = 0x0
Info: [RN_RDMA_QCSR_CQHEADi = 0x60330] = 0x0
Info: [RN_RDMA_QCSR_STATSSNi = 0x60380] = 0x1
Info: [RN_RDMA_QCSR_STATMSNi = 0x60384] = 0x0
Info: [RN_RDMA_QCSR_STATQPi = 0x60388] = 0x200
Info: [RN_RDMA_QCSR_STATCURSQPTRi = 0x6038c] = 0x1
Info: [RN_RDMA_QCSR_STATRESPSNi = 0x60390] = 0xabc
Info: [RN_RDMA_QCSR_STATRQBUFCAi = 0x60394] = 0x4f22b000
Info: [RN_RDMA_QCSR_STATWQEi = 0x60398] = 0x0
Info: [RN_RDMA_QCSR_STATRQPIDBi = 0x6039c] = 0x0
Info: [RN_RDMA_QCSR_STATRQBUFCAMSBi = 0x603d8] = 0x0
Info: [RN_RDMA_QCSR_SQPIi = 0x60338] = 0x1
Info: All data has been received!
Info: buffer physical address is 0xa350000000000000
Info: Time spent 1174.769000 usec, size = 128 bytes, Bandwidth = 0.000872 gigabits/sec
Info: The value of rc is 128
Info: CHECK RECEIVED DATA
Error: received data mismatched: recv[1]=0, sw_golden[1]=1
Warning: QP in fatal status
***** QP2 FATAL RECOVERY *****
TIMEOUT: CQHEADi:0x0 and SQPIi:0x1 are different
Output from Peer 2 (Server side) :
kamilgiorgio@kamilgiorgio-Standard-PC-Q35-ICH9-2009:~/Documents/RecoNIC/examples/rdma_test$ sudo env LD_LIBRARY_PATH=$LD_LIBRARY_PATH ./read -r 192.100.52.1 -i 192.100.51.1 -p /sys/bus/pci/devices/0000:07:00.0/resource2 -z 128 -l host_mem -d /dev/reconic-mm -s -u 22222 -t 11111 --dst_qp 2 -g 2>&1 | tee server_debug.log
src_ip_str = 192.100.52.1
dst_ip_str = 192.100.51.1
Info: mac_addr_t = 00:0a:35:92:12:ab
Info: PCIe resource file: /sys/bus/pci/devices/0000:07:00.0/resource2
Info: QP allocated at: host_mem
Info: Device - /dev/reconic-mm
Info: src_ip = 192.100.52.1
Info: Found network interface: enp7s0
Info: mac_addr_t = 00:0a:35:ca:26:34
Info: Creating rn_dev
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:323:create_rn_dev(): Info: scr(=4)) file open successfully
create_rn_dev - testing2
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:156:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4e800
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:161:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:165:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4e800000
Info: pre-allocated hugepage buffer vir addr = 0x7f24afe00000, physical addr = 0x4e800000
Info: Configuring 8 windows in QDMA AXI bridge BDF, each has 128GB mapping
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16420, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:209:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16424, bdf_addr_high=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:210:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16430, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16440, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:209:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16444, bdf_addr_high=0x20
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:210:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16450, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16460, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:209:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16464, bdf_addr_high=0x40
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:210:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16470, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16480, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:209:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16484, bdf_addr_high=0x60
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:210:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16490, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x164a0, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:209:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x164a4, bdf_addr_high=0x80
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:210:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x164b0, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x164c0, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:209:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x164c4, bdf_addr_high=0xa0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:210:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x164d0, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x164e0, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:209:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x164e4, bdf_addr_high=0xc0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:210:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x164f0, bdf_win_config=0xc2000000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:208:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_LSB=0x16500, bdf_addr_low=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:209:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_ADDR_TRANSLATE_ADDR_MSB=0x16504, bdf_addr_high=0xe0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:210:config_rn_dev_axib_bdf(): [BDF] AXIB_BDF_MAP_CONTROL_ADDR=0x16510, bdf_win_config=0xc2000000
Info: CREATE RDMA DEVICE
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:156:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4e800
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:161:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:165:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4e800000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:242:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f24afe00000, physical addr = 4e800000, rn_dev->buffer_offset = 0x200000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:243:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:156:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4ea00
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:161:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:165:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4ea00000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:242:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f24b0000000, physical addr = 4ea00000, rn_dev->buffer_offset = 0x1200000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:243:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:156:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4fa00
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:161:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:165:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4fa00000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:242:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f24b1000000, physical addr = 4fa00000, rn_dev->buffer_offset = 0x1202000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:243:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:156:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4fa02
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:161:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:165:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4fa02000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:242:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f24b1002000, physical addr = 4fa02000, rn_dev->buffer_offset = 0x1212000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:243:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:156:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4fa12
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:161:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:165:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4fa12000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:242:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f24b1012000, physical addr = 4fa12000, rn_dev->buffer_offset = 0x1222000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:243:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
Info: OPEN RDMA DEVICE
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:186:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_DATBUFBA=0x600a0, value=0x4ea00000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:188:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_DATBUFBAMSB=0x600a4, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:190:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_DATBUFSZ=0x600a8, value=0x10001000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:193:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_IPKTERRQBA=0x60088, value=0x4fa00000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:195:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_IPKTERRQBAMSB=0x6008c, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:197:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_ERRBUFSZ=0x60090, value=0x2000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:200:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_ERRBUFBA=0x60060, value=0x4fa02000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:202:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_ERRBUFBAMSB=0x60064, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:204:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_ERRBUFSZ=0x60068, value=0x1000100
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:207:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_RESPERRPKTBA=0x600b0, value=0x4fa12000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:209:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_RESPERRPKTBAMSB=0x600b4, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:211:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_RESPERRSZ=0x600b8, value=0x10000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:213:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_RESPERRSZMSB=0x600bc, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:217:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_INTEN=0x60180, value=0xff
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:221:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_MACXADDLSB=0x60010, value=0x35ca2634
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:223:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_MACXADDMSB=0x60014, value=0xa
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:227:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_IPV4XADD=0x60070, value=0xc0643401
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:230:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_XRNICCONF=0x60000, value=0x56ce0821
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:233:config_rdma_global_csr(): [Register] RN_RDMA_GCSR_XRNICADCONF=0x60004, value=0xa0004
Info: RDMA global control status registers are configured.
Info: rdma_dev opened
Info: ALLOCATE PD
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:254:allocate_rdma_pd(): [Register] RN_RDMA_PDT_PDPDNUM=0x40000, pd_num=0, value=0x0
Info: OPEN DEVICE FILE
Info: ALLOCATE RDMA QP
Allocating qp->sq
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:437:allocate_rdma_qp(): sq_size = 32768, cq_size = 2048, rq_size 262144, buf_location = host_mem
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:156:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4fa22
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:161:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:165:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4fa22000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:242:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f24b1022000, physical addr = 4fa22000, rn_dev->buffer_offset = 0x122a000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:243:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
Allocating qp->cq
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:156:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4fa2a
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:161:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:165:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4fa2a000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:242:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f24b102a000, physical addr = 4fa2a000, rn_dev->buffer_offset = 0x122a800
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:243:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
Allocating qp->rq
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:156:get_buffer_paddr(): Info: get_buffer_paddr - Page frame: 0x4fa2b
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:161:get_buffer_paddr(): Info: get_buffer_paddr - distance from page boundary: 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:165:get_buffer_paddr(): Info: get_buffer_paddr - Physical address of buffer: 0x4fa2b000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:242:allocate_rdma_buffer(): Info: allocated host buffer vir addr = 0x7f24b102b000, physical addr = 4fa2b000, rn_dev->buffer_offset = 0x126b000
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:243:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma host buffer
Info: queue pair setting is done! Configuring RDMA per-queu CSR registers
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:485:allocate_rdma_qp(): DEBUG: rdma_dev->rn_dev->axil_ctl = 0x7f24cfeae000, rdma_dev->axil_ctl = 0x7f24cfeae000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:499:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_IPDESADDR1i=0x60360, qpid=2, value=0xc0643301
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:506:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_MACDESADDLSBi=0x60350, qpid=2, value=0x359212ab
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:513:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_MACDESADDMSBi=0x60354, qpid=2, value=0xa
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:519:allocate_rdma_qp(): DEBUG: win_size_high = 0xff, win_size_low = 0xffffffff
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:536:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_SQBAi=0x60310, qpid=2, value=0x4fa22000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:543:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_SQBAMSBi=0x603c8, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:547:allocate_rdma_qp(): DEBUG: qp->sq->dma_addr = 0x4fa22000, sq_addr_msb = 0x0, sq_addr_lsb = 0x4fa22000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:565:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_CQBAi=0x60318, qpid=2, value=0x4fa2a000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:572:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_CQBAMSBi=0x603d0, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:576:allocate_rdma_qp(): DEBUG: qp->cq->dma_addr = 0x4fa2a000, cq_addr_msb = 0x0, cq_addr_lsb = 0x4fa2a000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:594:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_RQBAi=0x60308, qpid=2, value=0x4fa2b000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:601:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_RQBAMSBi=0x603c0, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:605:allocate_rdma_qp(): DEBUG: qp->rq->dma_addr = 0x4fa2b000, rq_addr_msb = 0x0, rq_addr_lsb = 0x4fa2b000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:614:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_CQDBADDi=0x60328, qpid=2, value=0x4e800000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:621:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_CQDBADDMSBi=0x6032c, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:625:allocate_rdma_qp(): DEBUG: cq_cidb_addr = 0x4e800000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:631:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_RQWPTRDBADDi=0x60320, qpid=2, value=0x4e800020
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:638:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_RQWPTRDBADDMSBi=0x60324, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:642:allocate_rdma_qp(): DEBUG: rq_cidb_addr = 0x4e800020
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:648:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_DESTQPCONFi=0x60348, qpid=2, value=0x2
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:657:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_QDEPTHi=0x6033c, qpid=2, value=0x400040
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:704:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_QPCONFi=0x60300, qpid=2, value=0x200043d
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:718:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_QPADVCONFi=0x60304, qpid=2, value=0x12344000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:727:allocate_rdma_qp(): [Register] RN_RDMA_QCSR_PDi=0x603b0, qpid=2, value=0x0
Info: allocate_rdma_qp - Successfully allocated a rdma qp
Info: CONFIGURE PSN
[Register] RN_RDMA_QCSR_LSTRQREQi=0x60344, qpid=2, value=0xa000abc
[Register] RN_RDMA_QCSR_SQPSNi=0x60340, qpid=2, value=0xabd
payload_size = 128, payload_size>>2 = 32
Info: Server is listening to a remote peer
Info: Server is connected to a remote peer
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:275:allocate_rdma_buffer(): Info: allocated device buffer physical addr = a350000000000000, rn_dev->dev_buffer_offset = 0x80
/home/kamilgiorgio/Documents/RecoNIC/lib/reconic.c:280:allocate_rdma_buffer(): Info: allocate_rdma_buffer - successfully allocated rdma device buffer
Info: rdma_register_memory_region - registering memory region
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:316:rdma_register_memory_region(): [Register] RN_RDMA_PDT_VIRTADDRLSB=0x40004, pd_num=0, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:318:rdma_register_memory_region(): [Register] RN_RDMA_PDT_VIRTADDRMSB=0x40008, pd_num=0, value=0xa3500000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:320:rdma_register_memory_region(): [Register] RN_RDMA_PDT_BUFBASEADDRLSB=0x4000c, pd_num=0, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:322:rdma_register_memory_region(): [Register] RN_RDMA_PDT_BUFBASEADDRMSB=0x40010, pd_num=0, value=0xa3500000
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:324:rdma_register_memory_region(): [Register] RN_RDMA_PDT_BUFRKEY=0x40014, pd_num=0, value=0x8
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:327:rdma_register_memory_region(): [Register] RN_RDMA_PDT_WRRDBUFLEN=0x40018, pd_num=0, value=0x80 B
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:330:rdma_register_memory_region(): [Register] RN_RDMA_PDT_ACCESSDESC=0x4001c, pd_num=0, value=0x2
Info: memory region for the 0-th PD is registered
Info: allocating buffer for payload data
Info: tmp_buffer->buffer = 0xa350000000000000, tmp_buffer->dma_addr = 0xa350000000000000
Info: copy payload data to the device memory
Info: copied payload data to the device memory succesfully rc = 128
Sending read_offset (a350000000000000) to the remote client
Does the client finish its RDMA read operation? If yes, please press any key
Info: Dump register values for debug purpose
Info: [RN_RDMA_GCSR_ERRBUFWPTR = 0x6006c] = 0x0
Info: [RN_RDMA_GCSR_IPKTERRQWPTR = 0x60094] = 0x0
Info: [RN_RDMA_GCSR_INSRRPKTCNT = 0x60100] = 0x0
Info: [RN_RDMA_GCSR_INAMPKTCNT = 0x60104] = 0x0
Info: [RN_RDMA_GCSR_OUTIOPKTCNT = 0x60108] = 0x0
Info: [RN_RDMA_GCSR_OUTAMPKTCNT = 0x6010c] = 0x0
Info: [RN_RDMA_GCSR_LSTINPKT = 0x60110] = 0xabd020c
Info: [RN_RDMA_GCSR_LSTOUTPKT = 0x60114] = 0x0
Info: [RN_RDMA_GCSR_ININVDUPCNT = 0x60118] = 0x0
Info: [RN_RDMA_GCSR_INNCKPKTSTS = 0x6011c] = 0x0
Info: [RN_RDMA_GCSR_OUTRNRPKTSTS = 0x60120] = 0x0
Info: [RN_RDMA_GCSR_WQEPROCSTS = 0x60124] = 0x2022040
Info: [RN_RDMA_GCSR_QPMSTS = 0x6012c] = 0x2
Info: [RN_RDMA_GCSR_INALLDRPPKTCNT = 0x60130] = 0x10000
Info: [RN_RDMA_GCSR_INNAKPKTCNT = 0x60134] = 0x0
Info: [RN_RDMA_GCSR_OUTNAKPKTCNT = 0x60138] = 0x0
Info: [RN_RDMA_GCSR_RESPHNDSTS = 0x6013c] = 0x10000
Info: [RN_RDMA_GCSR_RETRYCNTSTS = 0x60140] = 0x0
Info: [RN_RDMA_GCSR_INCNPPKTCNT = 0x60174] = 0x0
Info: [RN_RDMA_GCSR_OUTCNPPKTCNT = 0x60178] = 0x0
Info: [RN_RDMA_GCSR_OUTRDRSPPKTCNT = 0x6017c] = 0x1
Info: [RN_RDMA_GCSR_INTSTS = 0x60184] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS1 = 0x60190] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS2 = 0x60194] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS3 = 0x60198] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS4 = 0x6019c] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS5 = 0x601a0] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS6 = 0x601a4] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS7 = 0x601a8] = 0x0
Info: [RN_RDMA_GCSR_RQINTSTS8 = 0x601ac] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS1 = 0x601b0] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS2 = 0x601b4] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS3 = 0x601b8] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS4 = 0x601bc] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS5 = 0x601c0] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS6 = 0x601c4] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS7 = 0x601c8] = 0x0
Info: [RN_RDMA_GCSR_CQINTSTS8 = 0x601cc] = 0x0
Info: [RN_RDMA_QCSR_CQHEADi = 0x60330] = 0x0
Info: [RN_RDMA_QCSR_STATSSNi = 0x60380] = 0x0
Info: [RN_RDMA_QCSR_STATMSNi = 0x60384] = 0x0
Info: [RN_RDMA_QCSR_STATQPi = 0x60388] = 0x600
Info: [RN_RDMA_QCSR_STATCURSQPTRi = 0x6038c] = 0x0
Info: [RN_RDMA_QCSR_STATRESPSNi = 0x60390] = 0xabc
Info: [RN_RDMA_QCSR_STATRQBUFCAi = 0x60394] = 0x4fa2b000
Info: [RN_RDMA_QCSR_STATWQEi = 0x60398] = 0x0
Info: [RN_RDMA_QCSR_STATRQPIDBi = 0x6039c] = 0x0
Info: [RN_RDMA_QCSR_STATRQBUFCAMSBi = 0x603d8] = 0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:1087:destroy_rdma_qp(): [DEBUG] Destroying dev: 0x7f24cfeae000, RN_RDMA_QCSR_CQHEADi=0x60330, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:1092:destroy_rdma_qp(): [DEBUG] Destroying dev: 0x7f24cfeae000, RN_RDMA_QCSR_CQHEADi=0x60330, qpid=2, value=0x0
/home/kamilgiorgio/Documents/RecoNIC/lib/rdma_api.c:1100:destroy_rdma_qp(): [DEBUG] Destroying dev: 0x7f24cfeae000, RN_RDMA_QCSR_CQHEADi=0x60330, qpid=2, value=0x0
kamilgiorgio@kamilgiorgio-Standard-PC-Q35-ICH9-2009:~/Documents/RecoNIC/examples/rdma_test$