Lines Matching +full:tie +full:- +full:off
1 .. SPDX-License-Identifier: GPL-2.0
20 XDP programs to redirect frames to a memory buffer in a user-space
64 single-consumer / single-producer (for performance reasons), the new
72 user-space application can place an XSK at an arbitrary place in this
99 http://vger.kernel.org/lpc_net2018_talks/lpc18_paper_af_xdp_perf-v2.pdf. Do
106 ----
109 equal-sized frames. An UMEM is associated to a netdev and a specific
121 The UMEM has two single-producer/single-consumer rings that are used
123 user-space application.
126 -----
129 TX. All rings are single-producer/single-consumer, so the user-space
145 calls and mmapped to user-space using the appropriate offset to mmap()
155 user-space to kernel-space. The UMEM addrs are passed in the ring. As
164 the addr will be masked off, meaning that 2048, 2050 and 3000 refers
173 kernel-space to user-space. Just like the FILL ring, UMEM indices are
176 Frames passed from the kernel to user-space are frames that has been
177 sent (TX ring) and can be used by user-space again.
247 ------------------------------------
249 When you bind to a socket, the kernel will first try to use zero-copy
250 copy. If zero-copy is not supported, it will fall back on using copy
256 socket into zero-copy mode or fail.
259 -------------------------
283 round-robin example of distributing packets is shown below:
285 .. code-block:: c
303 rr = (rr + 1) & (MAX_SOCKS - 1);
351 -----------------------------
364 the NIC. In this case, interrupts are turned off as the NIC cannot
379 .. code-block:: c
393 ------------------------------------------------------
405 be used. Note, that the rings are single-producer single-consumer, so
409 In libbpf, you can create Rx-only and Tx-only sockets by supplying
413 If you create a Tx-only socket, we recommend that you do not put any
419 -----------------------
437 --------------------------
439 This is a generic SOL_SOCKET option that can be used to tie AF_XDP
441 is created by a privileged process and passed to a non-privileged one.
446 -------------------------
451 .. code-block:: c
460 ----------------------
463 XDP_OPTIONS_ZEROCOPY which tells you if zero-copy is on or not.
465 Multi-Buffer Support
468 With multi-buffer support, programs using AF_XDP sockets can receive
470 zero-copy mode. For example, a packet can consist of two
483 To enable multi-buffer support for an AF_XDP socket, use the new bind
484 flag XDP_USE_SG. If this is not provided, all multi-buffer packets
486 needs to be in multi-buffer mode. This can be accomplished by using
493 of the packet. Why the reverse logic of end-of-packet (eop) flag found
494 in many NICs? Just to preserve compatibility with non-multi-buffer
518 * For zero-copy mode, the limit is up to what the NIC HW
521 CONFIG_MAX_SKB_FRAGS + 1) for zero-copy mode, as it would have
523 NIC supports. Kind of defeats the purpose of zero-copy mode. How to
524 probe for this limit is explained in the "probe for multi-buffer
527 On the Rx path in copy-mode, the xsk core copies the XDP data into
529 detailed before. Zero-copy mode works the same, though the data is not
546 An example program each for Rx and Tx multi-buffer support can be found
550 -----
553 user-space application and the XDP program. For a complete setup and
554 usage example, please refer to the sample application. The user-space
559 .. code-block:: c
563 int index = ctx->rx_queue_index;
576 .. code-block:: c
598 __u32 entries = *ring->producer - *ring->consumer;
601 return -1;
603 // read-barrier!
605 *item = ring->desc[*ring->consumer & (RING_SIZE - 1)];
606 (*ring->consumer)++;
612 u32 free_entries = RING_SIZE - (*ring->producer - *ring->consumer);
615 return -1;
617 ring->desc[*ring->producer & (RING_SIZE - 1)] = *item;
619 // write-barrier!
621 (*ring->producer)++;
628 Usage Multi-Buffer Rx
629 ---------------------
631 Here is a simple Rx path pseudo-code example (using libxdp interfaces
634 .. code-block:: c
642 int rcvd = xsk_ring_cons__peek(&xsk->rx, opt_batch_size, &idx_rx);
644 xsk_ring_prod__reserve(&xsk->umem->fq, rcvd, &idx_fq);
647 struct xdp_desc *desc = xsk_ring_cons__rx_desc(&xsk->rx, idx_rx++);
648 char *frag = xsk_umem__get_data(xsk->umem->buffer, desc->addr);
649 bool eop = !(desc->options & XDP_PKT_CONTD);
661 *xsk_ring_prod__fill_addr(&xsk->umem->fq, idx_fq++) = desc->addr;
664 xsk_ring_prod__submit(&xsk->umem->fq, rcvd);
665 xsk_ring_cons__release(&xsk->rx, rcvd);
668 Usage Multi-Buffer Tx
669 ---------------------
671 Here is an example Tx path pseudo-code (using libxdp interfaces for
676 .. code-block:: c
683 xsk_ring_prod__reserve(&xsk->tx, batch_size, &idx);
692 tx_desc = xsk_ring_prod__tx_desc(&xsk->tx, idx + i++);
693 tx_desc->addr = addr;
696 tx_desc->len = xsk_frame_size;
697 tx_desc->options = XDP_PKT_CONTD;
699 tx_desc->len = len;
700 tx_desc->options = 0;
703 len -= tx_desc->len;
715 xsk_ring_prod__submit(&xsk->tx, i);
718 Probing for Multi-Buffer Support
719 --------------------------------
721 To discover if a driver supports multi-buffer AF_XDP in SKB or DRV
724 querying for XDP multi-buffer support. If XDP supports multi-buffer in
727 To discover if a driver supports multi-buffer AF_XDP in zero-copy
729 flag. If it is set, it means that at least zero-copy is supported and
733 supported by this device in zero-copy mode. These are the possible
736 1: Multi-buffer for zero-copy is not supported by this device, as max
737 one fragment supported means that multi-buffer is not possible.
739 >=2: Multi-buffer is supported in zero-copy mode for this device. The
745 Multi-Buffer Support for Zero-Copy Drivers
746 ------------------------------------------
748 Zero-copy drivers usually use the batched APIs for Rx and Tx
751 to facilitate extending a zero-copy driver with multi-buffer support.
761 ethtool -N p3p2 rx-flow-hash udp4 fn
762 ethtool -N p3p2 flow-type udp4 src-port 4242 dst-port 4242 \
768 samples/bpf/xdpsock -i p3p2 -q 16 -r -N
770 For XDP_SKB mode, use the switch "-S" instead of "-N" and all options
771 can be displayed with "-h", as usual.
799 sudo ethtool -L <interface> combined 1
806 sudo ethtool -N <interface> rx-flow-hash udp4 fn
807 sudo ethtool -N <interface> flow-type udp4 src-port 4242 dst-port \
821 to the same queue id Y. In zero-copy mode, you should use the
839 - Björn Töpel (AF_XDP core)
840 - Magnus Karlsson (AF_XDP core)
841 - Alexander Duyck
842 - Alexei Starovoitov
843 - Daniel Borkmann
844 - Jesper Dangaard Brouer
845 - John Fastabend
846 - Jonathan Corbet (LWN coverage)
847 - Michael S. Tsirkin
848 - Qi Z Zhang
849 - Willem de Bruijn