Hello,
I'm working on SR-IOV using Mellanox ConnectX-3 card (switch: Voltaire 4036) on CentOS 7.
Mellanox OFED Driver Installation and Configuration for SR-IOV
Mellanox-Neutron-Icehouse-Redhat-Ethernet - OpenStack
Nova-neutron-sriov - OpenStack
Most description/packages are written/made based on CentOS 6.*/python 2.6.
I'm working on CentOS 7 so I've installed needed packages from git sources and tar balls.
I could verify that SR-IOV is installed using lspci command.
# lspci -nn | grep Mell
21:00.0 Network controller [0280]: Mellanox Technologies MT27500 Family [ConnectX-3] [15b3:1003]
21:00.1 Network controller [0280]: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [15b3:1004]
21:00.2 Network controller [0280]: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [15b3:1004]
21:00.3 Network controller [0280]: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [15b3:1004]
21:00.4 Network controller [0280]: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [15b3:1004]
21:00.5 Network controller [0280]: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [15b3:1004]
21:00.6 Network controller [0280]: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [15b3:1004]
21:00.7 Network controller [0280]: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [15b3:1004]
21:01.0 Network controller [0280]: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [15b3:1004]
21:01.1 Network controller [0280]: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [15b3:1004]
21:01.2 Network controller [0280]: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [15b3:1004]
21:01.3 Network controller [0280]: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [15b3:1004]
21:01.4 Network controller [0280]: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [15b3:1004]
21:01.5 Network controller [0280]: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [15b3:1004]
21:01.6 Network controller [0280]: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [15b3:1004]
21:01.7 Network controller [0280]: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [15b3:1004]
21:02.0 Network controller [0280]: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [15b3:1004]
I could verify that this VF can be attached to VM (SR-IOV, hostdev) using virsh.
But, "VLAN operation failed" while launching OpenStack VM, according to the following mlnx-agent and eswitchd logs.
2015-02-24 16:51:31,346 DEBUG eswitchd [-] Handling message - {u'action': u'set_vlan', u'vlan': 1000, u'fabric': u'physnet1', u'port_mac': u'fa:16:3e:cc:76:bd'}
2015-02-24 16:51:31,346 DEBUG eswitchd [-] Running command: sudo eswitch-rootwrap /etc/eswitchd/rootwrap.conf ip link set ens4 vf 9 vlan 1000 qos 0
2015-02-24 16:51:31,441 DEBUG eswitchd [-]
Command: ['sudo', 'eswitch-rootwrap', '/etc/eswitchd/rootwrap.conf', 'ip', 'link', 'set', 'ens4', 'vf', '9', 'vlan', '1000', 'qos', '0']
Exit code: 2
Stdout: ''
Stderr: 'RTNETLINK answers: Operation not supported\n'
2015-02-24 16:51:31,442 ERROR eswitchd [-] Set VLAN operation failed
Also, I tried manually, but it's same as follows. Assigning MAC is fine but VLAN setup is failed. The iproute2-3.19.0 is installed.
# ip link set ens4 vf 9 mac fa:16:3e:cc:76:bd
# ip link show ens4
11: ens4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master ovs-system state DOWN mode DEFAULT qlen 1000
link/ether 00:02:c9:fb:a4:50 brd ff:ff:ff:ff:ff:ff
vf 0 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
vf 1 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
vf 2 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
vf 3 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
vf 4 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
vf 5 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
vf 6 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
vf 7 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
vf 8 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
vf 9 MAC fa:16:3e:cc:76:bd, vlan 4095, spoof checking off, link-state auto
vf 10 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
vf 11 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
vf 12 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
vf 13 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
vf 14 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
vf 15 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
# ip link set ens4 vf 9 vlan 1000
RTNETLINK answers: Operation not supported
# cat /boot/config-3.10.0-123.20.1.el7.x86_64 | grep NETFILTER_NETLINK
CONFIG_NETFILTER_NETLINK=m
CONFIG_NETFILTER_NETLINK_ACCT=m
CONFIG_NETFILTER_NETLINK_QUEUE=m
CONFIG_NETFILTER_NETLINK_LOG=m
CONFIG_NETFILTER_NETLINK_QUEUE_CT=y
Any suggestions are welcome.