Data Plane Development Kit
  • Getting Started Guide for Linux
    • 1. Introduction
    • 2. System Requirements
    • 3. Compiling the DPDK Target from Source
    • 4. Linux Drivers
    • 5. Compiling and Running Sample Applications
    • 6. Enabling Additional Functionality
    • 7. Quick Start Setup Script
    • 8. How to get best performance with NICs on Intel platforms
  • Getting Started Guide for FreeBSD
    • 1. Introduction
    • 2. Installing DPDK from the Ports Collection
    • 3. Compiling the DPDK Target from Source
    • 4. Compiling and Running Sample Applications
  • Sample Applications User Guides
    • 1. Introduction to the DPDK Sample Applications
    • 2. Compiling the Sample Applications
    • 3. Command Line Sample Application
    • 4. Ethtool Sample Application
    • 5. Exception Path Sample Application
    • 6. Hello World Sample Application
    • 7. Basic Forwarding Sample Application
    • 8. RX/TX Callbacks Sample Application
    • 9. Flow Classify Sample Application
    • 10. Basic RTE Flow Filtering Sample Application
    • 11. IP Fragmentation Sample Application
    • 12. IPv4 Multicast Sample Application
    • 13. IP Reassembly Sample Application
    • 14. Kernel NIC Interface Sample Application
    • 15. Keep Alive Sample Application
    • 16. L2 Forwarding with Crypto Sample Application
    • 17. L2 Forwarding Sample Application (in Real and Virtualized Environments) with core load statistics.
    • 18. L2 Forwarding Sample Application (in Real and Virtualized Environments)
    • 19. L2 Forwarding Sample Application with Cache Allocation Technology (CAT)
    • 20. L3 Forwarding Sample Application
    • 21. L3 Forwarding with Power Management Sample Application
    • 22. L3 Forwarding with Access Control Sample Application
    • 23. L3 Forwarding in a Virtualization Environment Sample Application
    • 24. Link Status Interrupt Sample Application
    • 25. Load Balancer Sample Application
    • 26. Server-Node EFD Sample Application
    • 27. Service Cores Sample Application
    • 28. Multi-process Sample Application
    • 29. QoS Metering Sample Application
    • 30. QoS Scheduler Sample Application
    • 31. Quota and Watermark Sample Application
    • 32. Timer Sample Application
    • 33. Packet Ordering Application
    • 34. VMDQ and DCB Forwarding Sample Application
    • 35. Vhost Sample Application
    • 36. Vhost_scsi Sample Application
    • 37. Netmap Compatibility Sample Application
    • 38. Internet Protocol (IP) Pipeline Application
    • 39. Test Pipeline Application
    • 40. Eventdev Pipeline SW PMD Sample Application
    • 41. Distributor Sample Application
    • 42. VM Power Management Application
    • 43. TEP termination Sample Application
    • 44. PTP Client Sample Application
    • 45. Performance Thread Sample Application
    • 46. IPsec Security Gateway Sample Application
  • Programmer’s Guide
    • 1. Introduction
    • 2. Overview
    • 3. Environment Abstraction Layer
    • 4. Service Cores
    • 5. Ring Library
    • 6. Mempool Library
    • 7. Mbuf Library
    • 8. Poll Mode Driver
    • 9. Generic flow API (rte_flow)
    • 10. Traffic Metering and Policing API
    • 11. Traffic Management API
    • 12. Cryptography Device Library
    • 13. Security Library
    • 14. Link Bonding Poll Mode Driver Library
    • 15. Timer Library
    • 16. Hash Library
    • 17. Elastic Flow Distributor Library
    • 18. Membership Library
    • 19. LPM Library
    • 20. LPM6 Library
    • 21. Flow Classification Library
    • 22. Packet Distributor Library
    • 23. Reorder Library
    • 24. IP Fragmentation and Reassembly Library
    • 25. Generic Receive Offload Library
    • 26. Generic Segmentation Offload Library
    • 27. The librte_pdump Library
    • 28. Multi-process Support
    • 29. Kernel NIC Interface
    • 30. Thread Safety of DPDK Functions
    • 31. Event Device Library
    • 32. Event Ethernet Rx Adapter Library
    • 33. Quality of Service (QoS) Framework
    • 34. Power Management
    • 35. Packet Classification and Access Control
    • 36. Packet Framework
    • 37. Vhost Library
    • 38. Metrics Library
    • 39. Port Hotplug Framework
    • 40. Source Organization
    • 41. Development Kit Build System
    • 42. Development Kit Root Makefile Help
    • 43. Extending the DPDK
    • 44. Building Your Own Application
    • 45. External Application/Library Makefile help
    • 46. Performance Optimization Guidelines
    • 47. Writing Efficient Code
    • 48. Profile Your Application
    • 49. Glossary
  • HowTo Guides
    • 1. Live Migration of VM with SR-IOV VF
    • 2. Live Migration of VM with Virtio on host running vhost_user
    • 3. Flow Bifurcation How-to Guide
    • 4. Generic flow API - examples
    • 5. PVP reference benchmark setup using testpmd
    • 6. VF daemon (VFd)
    • 7. Virtio_user for Container Networking
    • 8. Virtio_user as Exceptional Path
    • 9. DPDK pdump Library and pdump Tool
  • DPDK Tools User Guides
    • 1. dpdk-procinfo Application
    • 2. dpdk-pdump Application
    • 3. dpdk-pmdinfo Application
    • 4. dpdk-devbind Application
    • 5. dpdk-test-crypto-perf Application
    • 6. dpdk-test-eventdev Application
  • Testpmd Application User Guide
    • 1. Introduction
    • 2. Compiling the Application
    • 3. Running the Application
    • 4. Testpmd Runtime Functions
  • Network Interface Controller Drivers
    • 1. Overview of Networking Drivers
    • 2. Features Overview
    • 3. Compiling and testing a PMD for a NIC
    • 4. ARK Poll Mode Driver
    • 5. AVP Poll Mode Driver
    • 6. BNX2X Poll Mode Driver
    • 7. BNXT Poll Mode Driver
    • 8. CXGBE Poll Mode Driver
    • 9. DPAA Poll Mode Driver
    • 10. DPAA2 Poll Mode Driver
    • 11. Driver for VM Emulated Devices
    • 12. ENA Poll Mode Driver
    • 13. ENIC Poll Mode Driver
    • 14. FM10K Poll Mode Driver
    • 15. I40E Poll Mode Driver
    • 16. IXGBE Driver
    • 17. I40E/IXGBE/IGB Virtual Function Driver
    • 18. KNI Poll Mode Driver
    • 19. LiquidIO VF Poll Mode Driver
    • 20. MLX4 poll mode driver library
    • 21. MLX5 poll mode driver
    • 22. MRVL Poll Mode Driver
    • 23. NFP poll mode driver library
    • 24. OCTEONTX Poll Mode driver
    • 25. QEDE Poll Mode Driver
    • 26. Solarflare libefx-based Poll Mode Driver
    • 27. SZEDATA2 poll mode driver library
    • 28. Tun/Tap Poll Mode Driver
    • 29. ThunderX NICVF Poll Mode Driver
    • 30. Poll Mode Driver for Emulated Virtio NIC
    • 31. Poll Mode Driver that wraps vhost library
    • 32. Poll Mode Driver for Paravirtual VMXNET3 NIC
    • 33. Libpcap and Ring Based Poll Mode Drivers
    • 34. Fail-safe poll mode driver library
  • Crypto Device Drivers
    • 1. Crypto Device Supported Functionality Matrices
    • 2. AESN-NI Multi Buffer Crypto Poll Mode Driver
    • 3. AES-NI GCM Crypto Poll Mode Driver
    • 4. ARMv8 Crypto Poll Mode Driver
    • 5. NXP DPAA2 CAAM (DPAA2_SEC)
    • 6. NXP DPAA CAAM (DPAA_SEC)
    • 7. KASUMI Crypto Poll Mode Driver
    • 8. OpenSSL Crypto Poll Mode Driver
    • 9. MRVL Crypto Poll Mode Driver
    • 10. Null Crypto Poll Mode Driver
    • 11. Cryptodev Scheduler Poll Mode Driver Library
    • 12. SNOW 3G Crypto Poll Mode Driver
    • 13. Intel(R) QuickAssist (QAT) Crypto Poll Mode Driver
    • 14. ZUC Crypto Poll Mode Driver
  • Event Device Drivers
    • 1. NXP DPAA2 Eventdev Driver
    • 2. Software Eventdev Poll Mode Driver
    • 3. OCTEONTX SSOVF Eventdev Driver
  • Mempool Device Driver
    • 1. OCTEONTX FPAVF Mempool Driver
  • Platform Specific Guides
    • 1. OCTEONTX Board Support Package
  • Contributor’s Guidelines
    • 1. DPDK Coding Style
    • 2. Design
    • 3. Managing ABI updates
    • 4. DPDK Documentation Guidelines
    • 5. Contributing Code to DPDK
    • 6. DPDK Stable Releases and Long Term Support
    • 7. Patch Cheatsheet
  • Release Notes
    • 1. Description of Release
    • 2. DPDK Release 17.11
    • 3. DPDK Release 17.08
    • 4. DPDK Release 17.05
    • 5. DPDK Release 17.02
    • 6. DPDK Release 16.11
    • 7. DPDK Release 16.07
    • 8. DPDK Release 16.04
    • 9. DPDK Release 2.2
    • 10. DPDK Release 2.1
    • 11. DPDK Release 2.0
    • 12. DPDK Release 1.8
    • 13. Known Issues and Limitations in Legacy Releases
    • 14. ABI and API Deprecation
  • FAQ
    • 1. What does “EAL: map_all_hugepages(): open failed: Permission denied Cannot init memory” mean?
    • 2. If I want to change the number of hugepages allocated, how do I remove the original pages allocated?
    • 3. If I execute “l2fwd -l 0-3 -m 64 -n 3 – -p 3”, I get the following output, indicating that there are no socket 0 hugepages to allocate the mbuf and ring structures to?
    • 4. I am running a 32-bit DPDK application on a NUMA system, and sometimes the application initializes fine but cannot allocate memory. Why is that happening?
    • 5. On application startup, there is a lot of EAL information printed. Is there any way to reduce this?
    • 6. How can I tune my network application to achieve lower latency?
    • 7. Without NUMA enabled, my network throughput is low, why?
    • 8. I am getting errors about not being able to open files. Why?
    • 9. VF driver for IXGBE devices cannot be initialized
    • 10. Is it safe to add an entry to the hash table while running?
    • 11. What is the purpose of setting iommu=pt?
    • 12. When trying to send packets from an application to itself, meaning smac==dmac, using Intel(R) 82599 VF packets are lost.
    • 13. Can I split packet RX to use DPDK and have an application’s higher order functions continue using Linux pthread?
    • 14. Is it possible to exchange data between DPDK processes and regular userspace processes via some shared memory or IPC mechanism?
    • 15. Can the multiple queues in Intel(R) I350 be used with DPDK?
    • 16. How can hugepage-backed memory be shared among multiple processes?
    • 17. Why can’t my application receive packets on my system with UEFI Secure Boot enabled?
 
Data Plane Development Kit
  • Docs »
  • Sample Applications User Guides »
  • 24. Link Status Interrupt Sample Application
  • View page source

24. Link Status Interrupt Sample Application

The Link Status Interrupt sample application is a simple example of packet processing using the Data Plane Development Kit (DPDK) that demonstrates how network link status changes for a network port can be captured and used by a DPDK application.

24.1. Overview

The Link Status Interrupt sample application registers a user space callback for the link status interrupt of each port and performs L2 forwarding for each packet that is received on an RX_PORT. The following operations are performed:

  • RX_PORT and TX_PORT are paired with available ports one-by-one according to the core mask
  • The source MAC address is replaced by the TX_PORT MAC address
  • The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID

This application can be used to demonstrate the usage of link status interrupt and its user space callbacks and the behavior of L2 forwarding each time the link status changes.

24.2. Compiling the Application

To compile the sample application see Compiling the Sample Applications.

The application is located in the link_status_interrupt sub-directory.

24.3. Running the Application

The application requires a number of command line options:

./build/link_status_interrupt [EAL options] -- -p PORTMASK [-q NQ][-T PERIOD]

where,

  • -p PORTMASK: A hexadecimal bitmask of the ports to configure
  • -q NQ: A number of queues (=ports) per lcore (default is 1)
  • -T PERIOD: statistics will be refreshed each PERIOD seconds (0 to disable, 10 default)

To run the application in a linuxapp environment with 4 lcores, 4 memory channels, 16 ports and 8 RX queues per lcore, issue the command:

$ ./build/link_status_interrupt -l 0-3 -n 4-- -q 8 -p ffff

Refer to the DPDK Getting Started Guide for general information on running applications and the Environment Abstraction Layer (EAL) options.

24.4. Explanation

The following sections provide some explanation of the code.

24.4.1. Command Line Arguments

The Link Status Interrupt sample application takes specific parameters, in addition to Environment Abstraction Layer (EAL) arguments (see Section Running the Application).

Command line parsing is done in the same way as it is done in the L2 Forwarding Sample Application. See Command Line Arguments for more information.

24.4.2. Mbuf Pool Initialization

Mbuf pool initialization is done in the same way as it is done in the L2 Forwarding Sample Application. See Mbuf Pool Initialization for more information.

24.4.3. Driver Initialization

The main part of the code in the main() function relates to the initialization of the driver. To fully understand this code, it is recommended to study the chapters that related to the Poll Mode Driver in the DPDK Programmer’s Guide and the DPDK API Reference.

if (rte_pci_probe() < 0)
    rte_exit(EXIT_FAILURE, "Cannot probe PCI\n");

nb_ports = rte_eth_dev_count();
if (nb_ports == 0)
    rte_exit(EXIT_FAILURE, "No Ethernet ports - bye\n");

/*
 * Each logical core is assigned a dedicated TX queue on each port.
 */

for (portid = 0; portid < nb_ports; portid++) {
    /* skip ports that are not enabled */

    if ((lsi_enabled_port_mask & (1 << portid)) == 0)
        continue;

    /* save the destination port id */

    if (nb_ports_in_mask % 2) {
        lsi_dst_ports[portid] = portid_last;
        lsi_dst_ports[portid_last] = portid;
    }
    else
        portid_last = portid;

    nb_ports_in_mask++;

    rte_eth_dev_info_get((uint8_t) portid, &dev_info);
}

Observe that:

  • rte_pci_probe() parses the devices on the PCI bus and initializes recognized devices.

The next step is to configure the RX and TX queues. For each port, there is only one RX queue (only one lcore is able to poll a given port). The number of TX queues depends on the number of available lcores. The rte_eth_dev_configure() function is used to configure the number of queues for a port:

ret = rte_eth_dev_configure((uint8_t) portid, 1, 1, &port_conf);
if (ret < 0)
    rte_exit(EXIT_FAILURE, "Cannot configure device: err=%d, port=%u\n", ret, portid);

The global configuration is stored in a static structure:

static const struct rte_eth_conf port_conf = {
    .rxmode = {
        .split_hdr_size = 0,
        .header_split = 0,   /**< Header Split disabled */
        .hw_ip_checksum = 0, /**< IP checksum offload disabled */
        .hw_vlan_filter = 0, /**< VLAN filtering disabled */
        .hw_strip_crc= 0,    /**< CRC stripped by hardware */
    },
    .txmode = {},
    .intr_conf = {
        .lsc = 1, /**< link status interrupt feature enabled */
    },
};

Configuring lsc to 0 (the default) disables the generation of any link status change interrupts in kernel space and no user space interrupt event is received. The public interface rte_eth_link_get() accesses the NIC registers directly to update the link status. Configuring lsc to non-zero enables the generation of link status change interrupts in kernel space when a link status change is present and calls the user space callbacks registered by the application. The public interface rte_eth_link_get() just reads the link status in a global structure that would be updated in the interrupt host thread only.

24.4.4. Interrupt Callback Registration

The application can register one or more callbacks to a specific port and interrupt event. An example callback function that has been written as indicated below.

static void
lsi_event_callback(uint16_t port_id, enum rte_eth_event_type type, void *param)
{
    struct rte_eth_link link;

    RTE_SET_USED(param);

    printf("\n\nIn registered callback...\n");

    printf("Event type: %s\n", type == RTE_ETH_EVENT_INTR_LSC ? "LSC interrupt" : "unknown event");

    rte_eth_link_get_nowait(port_id, &link);

    if (link.link_status) {
        printf("Port %d Link Up - speed %u Mbps - %s\n\n", port_id, (unsigned)link.link_speed,
              (link.link_duplex == ETH_LINK_FULL_DUPLEX) ? ("full-duplex") : ("half-duplex"));
    } else
        printf("Port %d Link Down\n\n", port_id);
}

This function is called when a link status interrupt is present for the right port. The port_id indicates which port the interrupt applies to. The type parameter identifies the interrupt event type, which currently can be RTE_ETH_EVENT_INTR_LSC only, but other types can be added in the future. The param parameter is the address of the parameter for the callback. This function should be implemented with care since it will be called in the interrupt host thread, which is different from the main thread of its caller.

The application registers the lsi_event_callback and a NULL parameter to the link status interrupt event on each port:

rte_eth_dev_callback_register((uint8_t)portid, RTE_ETH_EVENT_INTR_LSC, lsi_event_callback, NULL);

This registration can be done only after calling the rte_eth_dev_configure() function and before calling any other function. If lsc is initialized with 0, the callback is never called since no interrupt event would ever be present.

24.4.5. RX Queue Initialization

The application uses one lcore to poll one or several ports, depending on the -q option, which specifies the number of queues per lcore.

For example, if the user specifies -q 4, the application is able to poll four ports with one lcore. If there are 16 ports on the target (and if the portmask argument is -p ffff), the application will need four lcores to poll all the ports.

ret = rte_eth_rx_queue_setup((uint8_t) portid, 0, nb_rxd, SOCKET0, &rx_conf, lsi_pktmbuf_pool);
if (ret < 0)
    rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup: err=%d, port=%u\n", ret, portid);

The list of queues that must be polled for a given lcore is stored in a private structure called struct lcore_queue_conf.

struct lcore_queue_conf {
    unsigned n_rx_port;
    unsigned rx_port_list[MAX_RX_QUEUE_PER_LCORE]; unsigned tx_queue_id;
    struct mbuf_table tx_mbufs[LSI_MAX_PORTS];
} rte_cache_aligned;

struct lcore_queue_conf lcore_queue_conf[RTE_MAX_LCORE];

The n_rx_port and rx_port_list[] fields are used in the main packet processing loop (see Receive, Process and Transmit Packets).

The global configuration for the RX queues is stored in a static structure:

static const struct rte_eth_rxconf rx_conf = {
    .rx_thresh = {
        .pthresh = RX_PTHRESH,
        .hthresh = RX_HTHRESH,
        .wthresh = RX_WTHRESH,
    },
};

24.4.6. TX Queue Initialization

Each lcore should be able to transmit on any port. For every port, a single TX queue is initialized.

/* init one TX queue logical core on each port */

fflush(stdout);

ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, rte_eth_dev_socket_id(portid), &tx_conf);
if (ret < 0)
    rte_exit(EXIT_FAILURE, "rte_eth_tx_queue_setup: err=%d,port=%u\n", ret, (unsigned) portid);

The global configuration for TX queues is stored in a static structure:

static const struct rte_eth_txconf tx_conf = {
    .tx_thresh = {
        .pthresh = TX_PTHRESH,
        .hthresh = TX_HTHRESH,
        .wthresh = TX_WTHRESH,
    },
    .tx_free_thresh = RTE_TEST_TX_DESC_DEFAULT + 1, /* disable feature */
};

24.4.7. Receive, Process and Transmit Packets

In the lsi_main_loop() function, the main task is to read ingress packets from the RX queues. This is done using the following code:

/*
 *   Read packet from RX queues
 */

for (i = 0; i < qconf->n_rx_port; i++) {
    portid = qconf->rx_port_list[i];
    nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, pkts_burst, MAX_PKT_BURST);
    port_statistics[portid].rx += nb_rx;

    for (j = 0; j < nb_rx; j++) {
        m = pkts_burst[j];
        rte_prefetch0(rte_pktmbuf_mtod(m, void *));
        lsi_simple_forward(m, portid);
    }
}

Packets are read in a burst of size MAX_PKT_BURST. The rte_eth_rx_burst() function writes the mbuf pointers in a local table and returns the number of available mbufs in the table.

Then, each mbuf in the table is processed by the lsi_simple_forward() function. The processing is very simple: processes the TX port from the RX port and then replaces the source and destination MAC addresses.

Note

In the following code, the two lines for calculating the output port require some explanation. If portId is even, the first line does nothing (as portid & 1 will be 0), and the second line adds 1. If portId is odd, the first line subtracts one and the second line does nothing. Therefore, 0 goes to 1, and 1 to 0, 2 goes to 3 and 3 to 2, and so on.

static void
lsi_simple_forward(struct rte_mbuf *m, unsigned portid)
{
    struct ether_hdr *eth;
    void *tmp;
    unsigned dst_port = lsi_dst_ports[portid];

    eth = rte_pktmbuf_mtod(m, struct ether_hdr *);

    /* 02:00:00:00:00:xx */

    tmp = &eth->d_addr.addr_bytes[0];

    *((uint64_t *)tmp) = 0x000000000002 + (dst_port << 40);

    /* src addr */
    ether_addr_copy(&lsi_ports_eth_addr[dst_port], &eth->s_addr);

    lsi_send_packet(m, dst_port);
}

Then, the packet is sent using the lsi_send_packet(m, dst_port) function. For this test application, the processing is exactly the same for all packets arriving on the same RX port. Therefore, it would have been possible to call the lsi_send_burst() function directly from the main loop to send all the received packets on the same TX port using the burst-oriented send function, which is more efficient.

However, in real-life applications (such as, L3 routing), packet N is not necessarily forwarded on the same port as packet N-1. The application is implemented to illustrate that so the same approach can be reused in a more complex application.

The lsi_send_packet() function stores the packet in a per-lcore and per-txport table. If the table is full, the whole packets table is transmitted using the lsi_send_burst() function:

/* Send the packet on an output interface */

static int
lsi_send_packet(struct rte_mbuf *m, uint16_t port)
{
    unsigned lcore_id, len;
    struct lcore_queue_conf *qconf;

    lcore_id = rte_lcore_id();
    qconf = &lcore_queue_conf[lcore_id];
    len = qconf->tx_mbufs[port].len;
    qconf->tx_mbufs[port].m_table[len] = m;
    len++;

    /* enough pkts to be sent */

    if (unlikely(len == MAX_PKT_BURST)) {
        lsi_send_burst(qconf, MAX_PKT_BURST, port);
        len = 0;
    }
    qconf->tx_mbufs[port].len = len;

    return 0;
}

To ensure that no packets remain in the tables, each lcore does a draining of the TX queue in its main loop. This technique introduces some latency when there are not many packets to send. However, it improves performance:

 cur_tsc = rte_rdtsc();

 /*
  *    TX burst queue drain
  */

 diff_tsc = cur_tsc - prev_tsc;

 if (unlikely(diff_tsc > drain_tsc)) {
     /* this could be optimized (use queueid instead of * portid), but it is not called so often */

     for (portid = 0; portid < RTE_MAX_ETHPORTS; portid++) {
         if (qconf->tx_mbufs[portid].len == 0)
             continue;

         lsi_send_burst(&lcore_queue_conf[lcore_id],
         qconf->tx_mbufs[portid].len, (uint8_t) portid);
         qconf->tx_mbufs[portid].len = 0;
     }

     /* if timer is enabled */

     if (timer_period > 0) {
         /* advance the timer */

         timer_tsc += diff_tsc;

         /* if timer has reached its timeout */

         if (unlikely(timer_tsc >= (uint64_t) timer_period)) {
             /* do this only on master core */

             if (lcore_id == rte_get_master_lcore()) {
                 print_stats();

                 /* reset the timer */
                 timer_tsc = 0;
             }
         }
     }
     prev_tsc = cur_tsc;
}
Next Previous

Sphinx theme provided by Read the Docs