Someone pointed me the chelsio website which has white papers for 40GbE over SMB between Windows 2012 servers setup just like mine getting a whopping 36Gbps... Another poster confirmed that he's getting 31Gbps over SMB using Mellanox ConnectX-2 QDR cards with IPoIB.
I've ordered myself a Mellanox 40GbE passive copper cable to see if my Fiber Optic FDR cable is what's causing the problem.