Hello Amirul,
In order to maximize performance I would suggest to do the following:
1. Make sure nodes and switches are running with the latest and greatest FW/driver versions.
2. Tune the system to get a better performance. Please refer to our performance tuning guide:
In case you still observe a poor BW and you are a contracted customer, you may open a case with Mellanox support and we will be glad to assist.
Kind Regards,
Einav Ezra