media/v4l/dev-encoder.rst

1 .. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
6 Memory-to-Memory Stateful Video Encoder Interface
9 A stateful video encoder takes raw video frames in display order and encodes
12 further post-processing by the client.
16 operations are needed, use of the Stateless Video Encoder Interface (in
34 5. Single-planar API (see :ref:`planar-apis`) and applicable structures may be
35    used interchangeably with multi-planar API, unless specified otherwise,
36    depending on encoder capabilities and following the general V4L2 guidelines.
47 Refer to :ref:`decoder-glossary`.
52 .. kernel-render:: DOT
53    :alt: DOT digraph of encoder state machine
54    :caption: Encoder State Machine
65        qi -> Initialization [ label = "open()" ];
67        Initialization -> Encoding [ label = "Both queues streaming" ];
69        Encoding -> Drain [ label = "V4L2_ENC_CMD_STOP" ];
70        Encoding -> Reset [ label = "VIDIOC_STREAMOFF(CAPTURE)" ];
71        Encoding -> Stopped [ label = "VIDIOC_STREAMOFF(OUTPUT)" ];
72        Encoding -> Encoding;
74        Drain -> Stopped [ label = "All CAPTURE\nbuffers dequeued\nor\nVIDIOC_STREAMOFF(OUTPUT)" ];
75        Drain -> Reset [ label = "VIDIOC_STREAMOFF(CAPTURE)" ];
77        Reset -> Encoding [ label = "VIDIOC_STREAMON(CAPTURE)" ];
78        Reset -> Initialization [ label = "VIDIOC_REQBUFS(OUTPUT, 0)" ];
80        Stopped -> Encoding [ label = "V4L2_ENC_CMD_START\nor\nVIDIOC_STREAMON(OUTPUT)" ];
81        Stopped -> Reset [ label = "VIDIOC_STREAMOFF(CAPTURE)" ];
87 1. To enumerate the set of coded formats supported by the encoder, the
109      encoder for the given coded pixel format.
113      encoder for the given raw pixel format and coded format currently set on
124      supported by the encoder for the given coded pixel format and resolution.
128      by the encoder for the given raw pixel format and resolution and for the
140 6. Any additional encoder capabilities may be discovered by querying
157          desired size of ``CAPTURE`` buffers; the encoder may adjust it to
161          ignored (read-only).
172          the coded size selected by the encoder based on current state, e.g.
173          ``OUTPUT`` format, selection rectangles, etc. (read-only).
178       format. How the new ``OUTPUT`` format is determined is up to the encoder
221          may be adjusted to match encoder minimums, maximums and alignment
245 	 the desired frame interval; the encoder may adjust it to
256       the encoder uses to encode the video. So setting the frame interval
257       to 1/24 (or 24 frames per second) will produce a coded video stream
264       changed to a different value. This is useful for off-line encoding
284    the case for off-line encoding. Support for this feature is signalled
285    by the :ref:`V4L2_FMT_FLAG_ENC_CAP_FRAME_INTERVAL <fmtdesc-flags>` format flag.
296 	 the desired coded frame interval; the encoder may adjust it to
307       coded video. It does *not* set the rate at which buffers arrive on the
308       ``CAPTURE`` queue, that depends on how fast the encoder is and how
345          visible rectangle adjusted by the encoder.
363             video with a resolution that is not a multiple of a macroblock,
371       The encoder may adjust the crop/compose rectangles to the nearest
435    restarts it again, the encoder will begin generating a stream independent
455 format and may be affected by codec-specific extended controls, as stated
459 queues and memory-to-memory devices. In addition, the order of encoded frames
469   ``CAPTURE`` (for example, if returning an encoded frame allowed the encoder
485   encoder needs to use the frame as a reference for encoding further frames.
502      ``CAPTURE`` buffers are out-of-order compared to the ``OUTPUT`` buffers):
515 of details depending on the encoder capabilities. Specifically:
520 * if the encoder is able to precisely report the ``OUTPUT`` buffer(s) that triggered
532 further operations on corresponding encoder file handle will return the -EIO
540 The client is allowed to use :c:func:`VIDIOC_S_CTRL` to change encoder
541 parameters at any time. The availability of parameters is encoder-specific
542 and the client must query the encoder to find the set of available controls.
544 The ability to change each parameter during encoding is encoder-specific, as
547 -EBUSY error code, the ``CAPTURE`` queue needs to be stopped for the
551 The timing of parameter updates is encoder-specific, as per the standard
554 (:ref:`media-request-api`) should be considered, if supported by the encoder.
603         marked with ``V4L2_BUF_FLAG_LAST`` will result in a -EPIPE error from
613       For backwards compatibility, the encoder will signal a ``V4L2_EVENT_EOS``
619    dequeued and the last ``CAPTURE`` buffer is dequeued, the encoder is stopped
623    * ``V4L2_ENC_CMD_START`` - the encoder will not be reset and will resume
627      ``CAPTURE`` queue - the encoder will be reset (see the `Reset` sequence)
631      ``OUTPUT`` queue - the encoder will resume operation normally, however any
642    will fail with -EBUSY error code if attempted.
648      immediately and the encoder returns an empty ``CAPTURE`` buffer with the
665    Although not mandatory, the availability of encoder commands may be queried
671 The client may want to request the encoder to reinitialize the encoding, so
683 1. Perform the `Drain` sequence to ensure all the in-flight encoding finishes
704 encoder.
717    will result in the encoder adjusting the requested ``OUTPUT`` format to a
725    return the -EBUSY error code for any such format change attempt.