media/v4l/dev-encoder.rst

1 .. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
6 Memory-to-Memory Stateful Video Encoder Interface
12 further post-processing by the client.
34 5. Single-planar API (see :ref:`planar-apis`) and applicable structures may be
35    used interchangeably with multi-planar API, unless specified otherwise,
36    depending on encoder capabilities and following the general V4L2 guidelines.
41 7. Given an ``OUTPUT`` buffer A, then A' represents a buffer on the ``CAPTURE``
47 Refer to :ref:`decoder-glossary`.
52 .. kernel-render:: DOT
62        node [shape = circle, label="Reset"] Reset;
65        qi -> Initialization [ label = "open()" ];
67        Initialization -> Encoding [ label = "Both queues streaming" ];
69        Encoding -> Drain [ label = "V4L2_ENC_CMD_STOP" ];
70        Encoding -> Reset [ label = "VIDIOC_STREAMOFF(CAPTURE)" ];
71        Encoding -> Stopped [ label = "VIDIOC_STREAMOFF(OUTPUT)" ];
72        Encoding -> Encoding;
74        Drain -> Stopped [ label = "All CAPTURE\nbuffers dequeued\nor\nVIDIOC_STREAMOFF(OUTPUT)" ];
75        Drain -> Reset [ label = "VIDIOC_STREAMOFF(CAPTURE)" ];
77        Reset -> Encoding [ label = "VIDIOC_STREAMON(CAPTURE)" ];
78        Reset -> Initialization [ label = "VIDIOC_REQBUFS(OUTPUT, 0)" ];
80        Stopped -> Encoding [ label = "V4L2_ENC_CMD_START\nor\nVIDIOC_STREAMON(OUTPUT)" ];
81        Stopped -> Reset [ label = "VIDIOC_STREAMOFF(CAPTURE)" ];
88    client may call :c:func:`VIDIOC_ENUM_FMT` on ``CAPTURE``.
91      format set on ``OUTPUT``.
94    :c:func:`VIDIOC_ENUM_FMT` on ``OUTPUT``.
96    * Only the formats supported for the format currently active on ``CAPTURE``
100      the client must first set that coded format on ``CAPTURE`` and then
101      enumerate the formats on ``OUTPUT``.
113      encoder for the given raw pixel format and coded format currently set on
129      coded format, coded resolution and coded frame interval currently set on
136 5. Supported profiles and levels for the coded format currently set on
146 1. Set the coded format on the ``CAPTURE`` queue via :c:func:`VIDIOC_S_FMT`.
161          ignored (read-only).
172          the coded size selected by the encoder based on current state, e.g.
173          ``OUTPUT`` format, selection rectangles, etc. (read-only).
179       and the client must ensure it matches its needs afterwards.
195          raw format supported for the coded format currently selected on
201 3. Set the raw source format on the ``OUTPUT`` queue via
228    * Setting the ``OUTPUT`` format will reset the selection rectangles to their
229      default values, based on the new resolution, as described in the next
232 4. Set the raw frame interval on the ``OUTPUT`` queue via
233    :c:func:`VIDIOC_S_PARM`. This also sets the coded frame interval on the
264       changed to a different value. This is useful for off-line encoding
277       interval also changes the coded frame interval on the ``CAPTURE``
281 5. **Optional** Set the coded frame interval on the ``CAPTURE`` queue via
284    the case for off-line encoding. Support for this feature is signalled
285    by the :ref:`V4L2_FMT_FLAG_ENC_CAP_FRAME_INTERVAL <fmtdesc-flags>` format flag.
307       coded video. It does *not* set the rate at which buffers arrive on the
308       ``CAPTURE`` queue, that depends on how fast the encoder is and how
309       fast raw frames are queued on the ``OUTPUT`` queue.
322       However, drivers that can schedule multiple encoders based on the
326    :c:func:`VIDIOC_S_SELECTION` on the ``OUTPUT`` queue if it is desired
347    * The following selection targets are supported on ``OUTPUT``:
366             size. To avoid encoding the padding, the client needs to explicitly
372       supported ones to meet codec and hardware requirements. The client needs
428 8. Begin streaming on both ``OUTPUT`` and ``CAPTURE`` queues via
437    on the coded format, but may include the following implications:
454 The content of encoded ``CAPTURE`` buffers depends on the active coded pixel
455 format and may be affected by codec-specific extended controls, as stated
459 queues and memory-to-memory devices. In addition, the order of encoded frames
468 * a buffer queued to ``OUTPUT`` may result in more than one buffer produced on
473 * a buffer queued to ``OUTPUT`` may result in a buffer being produced on
478 * buffers may become available on the ``CAPTURE`` queue without additional
485   encoder needs to use the frame as a reference for encoding further frames.
502      ``CAPTURE`` buffers are out-of-order compared to the ``OUTPUT`` buffers):
508    frames; the exact list of types depends on the coded format), the
515 of details depending on the encoder capabilities. Specifically:
532 further operations on corresponding encoder file handle will return the -EIO
534 alternatively reinitialize the instance by stopping streaming on both queues,
541 parameters at any time. The availability of parameters is encoder-specific
544 The ability to change each parameter during encoding is encoder-specific, as
547 -EBUSY error code, the ``CAPTURE`` queue needs to be stopped for the
551 The timing of parameter updates is encoder-specific, as per the standard
552 semantics of the V4L2 control interface. If the client needs to apply the
554 (:ref:`media-request-api`) should be considered, if supported by the encoder.
603         marked with ``V4L2_BUF_FLAG_LAST`` will result in a -EPIPE error from
615       dequeued. It is deprecated behavior and the client must not rely on it.
623    * ``V4L2_ENC_CMD_START`` - the encoder will not be reset and will resume
626    * a pair of :c:func:`VIDIOC_STREAMOFF` and :c:func:`VIDIOC_STREAMON` on the
627      ``CAPTURE`` queue - the encoder will be reset (see the `Reset` sequence)
628      and then resume encoding,
630    * a pair of :c:func:`VIDIOC_STREAMOFF` and :c:func:`VIDIOC_STREAMON` on the
631      ``OUTPUT`` queue - the encoder will resume operation normally, however any
637    Once the drain sequence is initiated, the client needs to drive it to
639    issuing :c:func:`VIDIOC_STREAMOFF` on any of the ``OUTPUT`` or ``CAPTURE``
642    will fail with -EBUSY error code if attempted.
656    * If :c:func:`VIDIOC_STREAMOFF` is called on the ``CAPTURE`` queue in the
660    * If :c:func:`VIDIOC_STREAMOFF` is called on the ``OUTPUT`` queue in the
668 Reset  chapter
673 generated before. Depending on the coded format, that may imply that:
681 This can be achieved by performing the reset sequence.
683 1. Perform the `Drain` sequence to ensure all the in-flight encoding finishes
686 2. Stop streaming on the ``CAPTURE`` queue via :c:func:`VIDIOC_STREAMOFF`. This
690 3. Start streaming on the ``CAPTURE`` queue via :c:func:`VIDIOC_STREAMON` and
692    ``CAPTURE`` buffers from now on will contain a standalone stream that can be
693    decoded without the need for frames encoded before the reset sequence,
698 without the ability to change the parameters on the fly.
706 1. Setting the format on the ``CAPTURE`` queue may change the set of formats
707    supported/advertised on the ``OUTPUT`` queue. In particular, it also means
708    that the ``OUTPUT`` format may be reset and the client must not rely on the
711 2. Enumerating formats on the ``OUTPUT`` queue always returns only formats
714 3. Setting the format on the ``OUTPUT`` queue does not change the list of
715    formats available on the ``CAPTURE`` queue. An attempt to set the ``OUTPUT``
720 4. Enumerating formats on the ``CAPTURE`` queue always returns the full set of
723 5. While buffers are allocated on any of the ``OUTPUT`` or ``CAPTURE`` queues,
724    the client must not change the format on the ``CAPTURE`` queue. Drivers will
725    return the -EBUSY error code for any such format change attempt.