Skip to content

Home

Jump to bottom
Hrishi Hiraskar edited this page Sep 15, 2020 · 15 revisions

Welcome to the ns-3-dpdk wiki!

Installation

Install DPDK

Is my NIC supported by DPDK?

Check Supported Devices.

Not supported? Use Virtual Machine instead

Install Oracle VM VirtualBox. Create a new VM and install Ubuntu on it. Open settings, create a network adapter with following configuration:

  • Attached to: Bridged Adapter
  • Name: The host network device you want to use
  • In Advanced
    • Adapter Type: Intel PRO/1000 MT Server (82545EM)
    • Promiscuous Mode: Allow All
    • Select Cable Connected

Then rest of the steps are same as follows.

Download the source

Head over to DPDK Downloads page to get the latest stable source. (We have used version dpdk-stable-18.02.2)

Install the source

Refer Installation for detailed instructions.

For a 64 bit linux machine with gcc, run:

make install T=x86_64-native-linuxapp-gcc DESTDIR=install

Build DPDK as a shared library

Goto the build directory. Build directory is the target(T) you used in previous make command.

In build directory, edit .config file to change following line to compile DPDK as a shared library:

# Compile to share library
CONFIG_RTE_BUILD_SHARED_LIB=y

Then inside target directory, compile dpdk again:

make

Export DPDK Environment variables

Export following environment variables:

  • RTE_SDK as the your DPDK source folder.
  • RTE_TARGET as the build target directory.

For example:

export RTE_SDK=/home/hrishi/dpdk/dpdk-stable-18.02.2
export RTE_TARGET=x86_64-native-linuxapp-gcc

It is advisable that you export these variables in .bashrc or similar for reusability.

Load DPDK Drivers to kernel

Execute following:

sudo modprobe uio_pci_generic
sudo modprobe uio
sudo insmod $RTE_SDK/$RTE_TARGET/kmod/igb_uio.ko
sudo modprobe vfio-pci

These should be done every time you reboot your system.

Configure hugepages

Refer System Requirements for detailed instructions.

To allocate hugepages at runtime, edit /etc/default/grub, and add the following to value of variable GRUB_CMDLINE_LINUX_DEFAULT:

hugepages=256

We suggest minimum of number of 256 to run our applications. Then update the grub configurations using:

sudo update-grub

OR

sudo update-grub2

You will need to reboot your system in order to see these changes.

To check allocation of hugepages, run:

cat /proc/meminfo | grep HugePages

You will see the number of hugepages allocated, they should be equal to the number you used above.

Once the hugepage memory is reserved, to make the memory available for DPDK use, perform the following steps:

mkdir /mnt/huge
mount -t hugetlbfs nodev /mnt/huge

The mount point can be made permanent across reboots, by adding the following line to the /etc/fstab file:

nodev /mnt/huge hugetlbfs defaults 0 0

Install ns-3 with DPDK applications

Clone/Download the project

Clone by:

git clone https://github.com/ns-3-dpdk-integration/ns-3-dpdk

Or download the zip and extract.

Resolve dependencies and Configure ns-3

In folder ns-allinone-3.28.1, run the command:

./build.py

This will resolve all the dependencies for compiling and building ns-3.

In folder ns-allinone-3.28.1/ns-3.28.1, run:

./waf configure --build-profile=optimized --enable-examples --enable-tests --enable-sudo

--enable-sudo is required for ns-3 network emulation

Build ns-3

In folder ns-allinone-3.28.1/ns-3.28.1, execute:

./waf

Run ping application using DPDK

Edit src/fd-net-device/examples/fd-dpdk-emu-ping.cc, and change following variables:

  • deviceName : The device address of your ethernet. Use lspci to find this. For example: 0000:00:11.0
  • macClient : (Optional) The mac address of your ethernet device.
  • remote : The IP address of the remote you want to ping.
  • localIp : The local IP address of ethernet device.
  • localMask : The mask that your subnet uses. Use netstat -rn to find this.
  • gateway : The gateway of your subnet. Use netstat -rn to find this.

Since DPDK requires root access, switch to root user using following:

sudo -E su

-E flag to use RTE_SDK and RTE_TARGET environment variables of user

Run the DPDK ping application:

./waf --run src/fd-net-device/examples/fd-dpdk-emu-ping

Run the OnOff Application

The following arguments are required for OnOff Application:

  • deviceName : In case of DPDK mode, the device address of your ethernet (Use lspci to find this), for example: 0000:00:11.0. Otherwise, the interface id which linux uses, for example: eno1.
  • client : The IP address of client.
  • server : The IP address of server.
  • mac-server : (Optional) The mac address of server.
  • data-rate : The application rate of application.
  • transportPort : The transport protocol to use.
  • dpdkMode : true to use DpdkNetDevice.

You can run the OnOff Application using following command as a root user:

./waf --run 'src/fd-net-device/examples/fd-dpdk-emu-onoff \
    --deviceName=0000:00:11.0 \
    --client=192.168.43.44 \
    --server=192.168.43.142 \
    --mac-server=20:39:56:6c:47:1b \
    --data-rate=10Mb/s \
    --transportPort=Udp \
    --dpdkMode=true'

On the remote device, iperf server can be started using following command:

For TCP:

iperf -s -p 8000

For UDP:

iperf -s -p 8000 -u

Statistics

The OnOff Application creates following files which can be used for calculating various statistics:

  • fd-client-0-0.pcap : Throughput and RTT using TCP analysis.
  • inflight.plotme : TCP bytes in flight values at regular interval (1ms).
  • ping.plotme : Ping RTT at regular interval (1ms).
  • cwnd.plotme : TCP CWND at values at regular interval (1ms).
  • drops.plotme : Packet drops at Physical Layer.

Additionally, iperf can be used with -i flag to generate throughput statistics.

To generate plots, refer dpdk-plot-scripts.

VTune Hotspot profiling

For installation check Installation Guide.

When OnOff Application is executed, it prints it's pid and waits for user input. This pid can be used to profile the program in VTune. To do so:

  1. Run VTune Amplifier.
  2. Select Configure Analysis.
  3. Select Hotspots in analysis type.
  4. Insert pid provided by OnOff Application.
  5. Start analysis.
  6. Resume the OnOff Application.

VTune provides various profiling statistics. The important ones are:

  • CPU Utilization (the measure of parallel efficiency of the program).
  • Simultaneously utilized CPU cores.
  • Hotspot analysis (total time spent in each function of the program).

netmap in ns-3

Similar to this project, netmap support in ns-3 was added by Pasquale Imputato and Stefano Avallone. Link https://github.com/pasquimp/ns-3-dev-git/tree/socis-next. The installation and usage can be found in their wiki.

OnOff Application (fd-emu-onoff.cc) in netmap can give above statistics. To configure the example to give these statistics, refer fd-dpdk-emu-onoff.cc for adding trace sources.

Clone this wiki locally