Hello,
I'm seeing very poor IPoIB performance with QDR switch, RH 6.5, when measured with iPerf.
[ 3] 0.0-10.0 sec 3.31 GBytes 2.84 Gbits/sec
I implemented the changes recommended in performance tuning guide, still no improvement.
Some of the suggestions, I read online was to change the MTU size to highest 65520, and run iperf with multiple threads.
1. How do I change the IPoIB MTU size to highest? Is that the setting in rdma.conf file to include line IPOIB_MTU=65520.
2. Running ibdiagnet -r, I see line rate at 10Gbps instead of 40. Is there a setting to change line rate.
0xff12601bffff0000:0x0000000000000001 | 0xc003 | 0xffff | 0x00000b1b | =2048 | =10Gbps | 45
0xff12601bffff0000:0x00000001ff4bebeb | 0xc002 | 0xffff | 0x00000b1b | =2048 | =10Gbps | 1
-I---------------------------------------------------
-I- IPoIB Subnets Check
-I---------------------------------------------------
-I- Subnet: IPv4 PKey:0x7fff QKey:0x00000b1b MTU:2048Byte rate:10Gbps SL:0x00
-W- Suboptimal rate for group. Lowest member rate:40Gbps > group-rate:10Gbps
Thanks