Hi Rama,
could you please tell me the device type (ConnectX3 or ConnectX4) and the Link type( Ethernet or Infiniband )
I guess that you use Ethernet and you want to capture RoCE packets.
you can see HowTo Enable, Verify and Troubleshoot RDMA
To be able to use ibdump, you need to enable flow steering.
a. To enable flow-steering:
- add/create /etc/modprobe.d/mlx4.conf file and add this line:
options mlx4_core log_num_mgm_entry_size=-1
- restart the driver
#/etc/init.d/openibd restart
(Make sure the you still have IP configured on the interface)
b. Run some RDMA traffic (e.g. ib_send_bw or similar above)
c. run ibdump to create *.pcap file.
# ibdump
Initiating resources ...
searching for IB devices in host
Port active_mtu=1024
MR was registered with addr=0x61b8f0, lkey=0x10010d00, rkey=0x10010d00, flags=0x1
------------------------------------------------
Device : "mlx4_0"
Physical port : 1
Link layer : Ethernet
Dump file : sniffer.pcap
Sniffer WQEs (max burst size) : 4096
------------------------------------------------
Ready to capture (Press ^c to stop):
Captured: 82133 packets, 88626986 bytes
Interrupted (signal 2) - exiting ...
Captured: 82133 packets, 88626986 bytes
# ls
sniffer.pcap
#
d. Open the pcap file using wireshark (or similar program)
In this case RoCE V1 was used (ethertype 0x8915)
Thanks,
Talat