Lines Matching +full:rx +full:- +full:only
1 .. SPDX-License-Identifier: GPL-2.0
20 XDP programs to redirect frames to a memory buffer in a user-space
24 syscall. Associated with each XSK are two rings: the RX ring and the
25 TX ring. A socket can receive packets on the RX ring and it can send
28 to have at least one of these rings for each socket. An RX or TX
30 UMEM. RX and TX can share the same UMEM so that a packet does not have
31 to be copied between RX and TX. Moreover, if a packet needs to be kept
44 to fill in with RX packet data. References to these frames will then
45 appear in the RX ring once each packet has been received. The
48 space, for either TX or RX. Thus, the frame addrs appearing in the
50 TX ring. In summary, the RX and FILL rings are used for the RX path
62 then receive frame addr references in its own RX ring that point to
64 single-consumer / single-producer (for performance reasons), the new
65 process has to create its own socket with associated RX and TX rings,
67 reason that there is only one set of FILL and COMPLETION rings per
72 user-space application can place an XSK at an arbitrary place in this
99 http://vger.kernel.org/lpc_net2018_talks/lpc18_paper_af_xdp_perf-v2.pdf. Do
106 ----
109 equal-sized frames. An UMEM is associated to a netdev and a specific
121 The UMEM has two single-producer/single-consumer rings that are used
123 user-space application.
126 -----
128 There are a four different kind of rings: FILL, COMPLETION, RX and
129 TX. All rings are single-producer/single-consumer, so the user-space
134 with the UMEM must have an RX queue, TX queue or both. Say, that there
135 is a setup with four sockets (all doing TX and RX). Then there will be
136 one FILL ring, one COMPLETION ring, four TX rings and four RX rings.
145 calls and mmapped to user-space using the appropriate offset to mmap()
155 user-space to kernel-space. The UMEM addrs are passed in the ring. As
159 Frames passed to the kernel are used for the ingress path (RX rings).
173 kernel-space to user-space. Just like the FILL ring, UMEM indices are
176 Frames passed from the kernel to user-space are frames that has been
177 sent (TX ring) and can be used by user-space again.
182 RX Ring
185 The RX ring is the receiving side of a socket. Each entry in the ring
190 descriptors will (or can) appear on the RX ring.
236 queue 17. Only the XDP program executing for eth0 and queue 17 will
247 ------------------------------------
249 When you bind to a socket, the kernel will first try to use zero-copy
250 copy. If zero-copy is not supported, it will fall back on using copy
256 socket into zero-copy mode or fail.
259 -------------------------
263 netdevs/devices. In this mode, each socket has their own RX and TX
270 fist socket created) will only have a single FILL ring and a single
271 COMPLETION ring as there is only on unique netdev,queue_id tuple that
273 it in the normal way. Create a second socket and create an RX and a TX
283 round-robin example of distributing packets is shown below:
285 .. code-block:: c
303 rr = (rr + 1) & (MAX_SOCKS - 1);
308 Note, that since there is only a single set of FILL and COMPLETION
325 and bind it in the normal way. Create a second socket and create an RX
335 the packets to the right queue. In the previous example, there is only
337 can only steer between queues.
351 -----------------------------
361 poll() to be able to continue to receive packets on the RX ring. This
363 more buffers on the FILL ring and no buffers left on the RX HW ring of
373 RX path, or by calling sendto().
379 .. code-block:: c
384 I.e., only use the syscall if the flag is set.
392 XDP_{RX|TX|UMEM_FILL|UMEM_COMPLETION}_RING setsockopts
393 ------------------------------------------------------
395 These setsockopts sets the number of descriptors that the RX, TX,
397 to set the size of at least one of the RX and TX rings. If you set
399 application, but if you only want to do one of them, you can save
400 resources by only setting up one of them. Both the FILL ring and the
405 be used. Note, that the rings are single-producer single-consumer, so
409 In libbpf, you can create Rx-only and Tx-only sockets by supplying
410 NULL to the rx and tx arguments, respectively, to the
413 If you create a Tx-only socket, we recommend that you do not put any
419 -----------------------
425 divided into. It can only be 2K or 4K at the moment. If you have an
437 --------------------------
441 is created by a privileged process and passed to a non-privileged one.
446 -------------------------
451 .. code-block:: c
460 ----------------------
462 Gets options from an XDP socket. The only one supported so far is
463 XDP_OPTIONS_ZEROCOPY which tells you if zero-copy is on or not.
465 Multi-Buffer Support
468 With multi-buffer support, programs using AF_XDP sockets can receive
470 zero-copy mode. For example, a packet can consist of two
483 To enable multi-buffer support for an AF_XDP socket, use the new bind
484 flag XDP_USE_SG. If this is not provided, all multi-buffer packets
486 needs to be in multi-buffer mode. This can be accomplished by using
490 XDP_PKT_CONTD is introduced in the options field of the Rx and Tx
493 of the packet. Why the reverse logic of end-of-packet (eop) flag found
494 in many NICs? Just to preserve compatibility with non-multi-buffer
495 applications that have this bit set to false for all packets on Rx,
518 * For zero-copy mode, the limit is up to what the NIC HW
521 CONFIG_MAX_SKB_FRAGS + 1) for zero-copy mode, as it would have
523 NIC supports. Kind of defeats the purpose of zero-copy mode. How to
524 probe for this limit is explained in the "probe for multi-buffer
527 On the Rx path in copy-mode, the xsk core copies the XDP data into
529 detailed before. Zero-copy mode works the same, though the data is not
535 that only a complete packet (all frames in the packet) is sent to the
536 application. If there is not enough space in the AF_XDP Rx ring, all
546 An example program each for Rx and Tx multi-buffer support can be found
550 -----
553 user-space application and the XDP program. For a complete setup and
554 usage example, please refer to the sample application. The user-space
559 .. code-block:: c
563 int index = ctx->rx_queue_index;
576 .. code-block:: c
598 __u32 entries = *ring->producer - *ring->consumer;
601 return -1;
603 // read-barrier!
605 *item = ring->desc[*ring->consumer & (RING_SIZE - 1)];
606 (*ring->consumer)++;
612 u32 free_entries = RING_SIZE - (*ring->producer - *ring->consumer);
615 return -1;
617 ring->desc[*ring->producer & (RING_SIZE - 1)] = *item;
619 // write-barrier!
621 (*ring->producer)++;
628 Usage Multi-Buffer Rx
629 ---------------------
631 Here is a simple Rx path pseudo-code example (using libxdp interfaces
634 .. code-block:: c
642 int rcvd = xsk_ring_cons__peek(&xsk->rx, opt_batch_size, &idx_rx);
644 xsk_ring_prod__reserve(&xsk->umem->fq, rcvd, &idx_fq);
647 struct xdp_desc *desc = xsk_ring_cons__rx_desc(&xsk->rx, idx_rx++);
648 char *frag = xsk_umem__get_data(xsk->umem->buffer, desc->addr);
649 bool eop = !(desc->options & XDP_PKT_CONTD);
661 *xsk_ring_prod__fill_addr(&xsk->umem->fq, idx_fq++) = desc->addr;
664 xsk_ring_prod__submit(&xsk->umem->fq, rcvd);
665 xsk_ring_cons__release(&xsk->rx, rcvd);
668 Usage Multi-Buffer Tx
669 ---------------------
671 Here is an example Tx path pseudo-code (using libxdp interfaces for
676 .. code-block:: c
683 xsk_ring_prod__reserve(&xsk->tx, batch_size, &idx);
692 tx_desc = xsk_ring_prod__tx_desc(&xsk->tx, idx + i++);
693 tx_desc->addr = addr;
696 tx_desc->len = xsk_frame_size;
697 tx_desc->options = XDP_PKT_CONTD;
699 tx_desc->len = len;
700 tx_desc->options = 0;
703 len -= tx_desc->len;
715 xsk_ring_prod__submit(&xsk->tx, i);
718 Probing for Multi-Buffer Support
719 --------------------------------
721 To discover if a driver supports multi-buffer AF_XDP in SKB or DRV
724 querying for XDP multi-buffer support. If XDP supports multi-buffer in
727 To discover if a driver supports multi-buffer AF_XDP in zero-copy
729 flag. If it is set, it means that at least zero-copy is supported and
733 supported by this device in zero-copy mode. These are the possible
736 1: Multi-buffer for zero-copy is not supported by this device, as max
737 one fragment supported means that multi-buffer is not possible.
739 >=2: Multi-buffer is supported in zero-copy mode for this device. The
745 Multi-Buffer Support for Zero-Copy Drivers
746 ------------------------------------------
748 Zero-copy drivers usually use the batched APIs for Rx and Tx
751 to facilitate extending a zero-copy driver with multi-buffer support.
761 ethtool -N p3p2 rx-flow-hash udp4 fn
762 ethtool -N p3p2 flow-type udp4 src-port 4242 dst-port 4242 \
768 samples/bpf/xdpsock -i p3p2 -q 16 -r -N
770 For XDP_SKB mode, use the switch "-S" instead of "-N" and all options
771 can be displayed with "-h", as usual.
784 allocates one RX and TX queue pair per core. So on a 8 core system,
787 specify a specific queue id to bind to and it is only the traffic
796 all the traffic, you can force the netdev to only have 1 queue, queue
799 sudo ethtool -L <interface> combined 1
801 If you want to only see part of the traffic, you can program the
806 sudo ethtool -N <interface> rx-flow-hash udp4 fn
807 sudo ethtool -N <interface> flow-type udp4 src-port 4242 dst-port \
817 XSKMAP can only be used to switch traffic coming in on queue id X
819 sockets bound to different queue ids, for example X and Y, but only
821 to the same queue id Y. In zero-copy mode, you should use the
839 - Björn Töpel (AF_XDP core)
840 - Magnus Karlsson (AF_XDP core)
841 - Alexander Duyck
842 - Alexei Starovoitov
843 - Daniel Borkmann
844 - Jesper Dangaard Brouer
845 - John Fastabend
846 - Jonathan Corbet (LWN coverage)
847 - Michael S. Tsirkin
848 - Qi Z Zhang
849 - Willem de Bruijn