Good question. For example MHJH29-XTC Dual 4X 40Gb/s InfiniBand
has PCIe x8 2.0 5.0GT/s (4 GB/s) so how will one be able to push 2*40Gb/s thru it?
A PCIe 3.0 x8 or 2.0 x16 might be able to handle such transfer.
Going back to my original question I just saw Ethernet over InfiniBand (EoIB) - YouTube
and also think about creating 2 ethernet interfaces using IBoE driver and then connect the 2 interfaces with linux bonding driver (like I would do with usuall eth0 and eth1) to get 80Gb/s link. Anyone tried bonding IBoE?