Configure OVS and DPDK on the Host

Log in to the KVM host CLI.

If you are replacing or reconfiguring an existing OVS-DPDK setup, execute the following commands to reset any previous configuration. Repeat the command for each interface.

rm /usr/local/var/run/openvswitch/
<interface-name>

Configure initial huge pages for OVS.

echo 16384 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages

Mount huge pages for QEMU:

mkdir /dev/hugepages 
mkdir /dev/hugepages/libvirt 
mkdir /dev/hugepages/libvirt/qemu 
mount -t hugetlbfs hugetlbfs /dev/hugepages/libvirt/qemu 

Use the following command to kill any currently existing OVS daemon.

killall ovsdb-server ovs-vswitchd

Create directories for the OVS daemon.

mkdir -p /usr/local/etc/openvswitch 
mkdir -p /usr/local/var/run/openvswitch 

Clear old directories.

rm -f /var/run/openvswitch/vhost-user* 
rm -f /usr/local/etc/openvswitch/conf.db 

Initialize the configuration database.

ovsdb-tool create /usr/local/etc/openvswitch/conf.db\ 
/usr/local/share/openvswitch/vswitch.ovsschema 

Create an OVS DB server.

ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \ 
--remote=db:Open_vSwitch,Open_vSwitch,manager_options \ 
--private-key=db:Open_vSwitch,SSL,private_key \ 
--certificate=db:Open_vSwitch,SSL,certificate \ 
--bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert \ 
--pidfile --detach 

Initialize OVS.

ovs-vsctl --no-wait init

Start the database server.

export DB_SOCK=/usr/local/var/run/openvswitch/db.sock

Install the igb_uio module (network device driver) for DPDK.

cd ~/dpdk-2.2.0/x86_64-native-linuxapp-gcc/kmod 
modprobe uio 
insmod igb_uio.ko 
cd ~/dpdk-2.2.0/tools/ 

Enable DPDK on interfaces using PCI-ID or interface name.

./dpdk_nic_bind.py --bind=igb_uio <your first data interface> 
./dpdk_nic_bind.py --bind=igb_uio <your second data interface> 

Start the OVS daemon in DPDK mode. You can change the number of cores for ovs-vswitchd. By changing -c 0x1 to -c 0x3, you can have two core run this daemon.

ovs-vswitchd --dpdk -c 0x3 -n 4 -- unix:$DB_SOCK --pidfile --detach 
echo 50000 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages 

Create the OVS bridge and attach ports to the OVS bridge.

ovs-vsctl add-br ovs-br0 -- set bridge ovs-br0 datapath_type=netdev 
ovs-vsctl add-port ovs-br0 dpdk0 -- set Interface dpdk0 type=dpdk 
ovs-vsctl add-br ovs-br1 -- set bridge ovs-br1 datapath_type=netdev 
ovs-vsctl add-port ovs-br1 dpdk1 -- set Interface dpdk1 type=dpdk  

Create DPDK vhost user ports for OVS.

ovs-vsctl add-port ovs-br0 vhost-user1 -- set Interface vhost-user1 type=dpdkvhostuser 
ovs-vsctl add-port ovs-br1 vhost-user2 -- set Interface vhost-user2 type=dpdkvhostuser 

Set the number of hardware queues of the NIC used by the host.

ovs-vsctl set Open_vSwitch . other_config:n-dpdk-rxqs=8 
ovs-vsctl set Open_vSwitch . other_config:n-dpdk-txqs=8 

Set the CPU mask used for OVS.

ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0xffff 

Set the necessary permissions for DPDK vhost user ports. In the example below, 777 is used to give read, write, and executable permissions.

chmod 777 /usr/local/var/run/openvswitch/vhost-user1 
chmod 777 /usr/local/var/run/openvswitch/vhost-user2 
chmod 777 /dev/hugepages/libvirt/qemu