Well I got the 9024D today and connected it up. At first my storage server and one VM server could not connect via IPoIB. I had to downgrade the firmware on the MHGH28-XTC IB cards to 2.7.0 from firmware 2.9.1 in order for IPoIB traffic to work with this switch.
Now I'm trying to figure out how to get them running at 4X DDR speeds. Currently they are still running at 4X SDR speeds. Per the specs on the MHGH28-XTC cards, they should be capable of running at 4X DDR speeds.
Switch 24 "S-0008f10400411aba" # "ISR9024D Voltaire" base port 0 lid 1 lmc 0
[13] "H-0002c9030000fdb0"[2](2c9030000fdb2) # "opensm mlx4_0" lid 9 4xSDR
[14] "H-0002c9030002861c"[2](2c9030002861e) # "SAN" lid 5 4xSDR
[15] "H-0002c9030002861c"[1](2c9030002861d) # "SAN" lid 6 4xSDR
[16] "H-0002c903000284b0"[2](2c903000284b2) # "VMHOST2" lid 7 4xSDR
[18] "H-0002c90300028a10"[1](2c90300028a11) # "VMHOST" lid 3 4xSDR
[19] "H-0002c90300028a10"[2](2c90300028a12) # "VMHOST" lid 4 4xSDR
[20] "H-0002c903000284b0"[1](2c903000284b1) # "VMHOST2" lid 8 4xSDR
Any advice?