sound/designs/compress-offload.rst

2 ALSA Compress-Offload API
5 Pierre-Louis.Bossart <pierre-louis.bossart@linux.intel.com>
17 In recent years, audio digital signal processors (DSP) were integrated
18 in system-on-chip designs, and DSPs are also integrated in audio
20 reduction of power consumption compared to host-based
27 provide a control and data-streaming interface for audio DSPs.
29 The design of this API was inspired by the 2-year experience with the
39 - separation between byte counts and time. Compressed formats may have
41   may vary from frame-to-frame. As a result, it is not possible to
44   reliable audio-video synchronization, which requires precise
47 - Handling of multiple formats. PCM data only requires a specification
54 - Focus on main formats. This API provides support for the most
59 - Handling of multiple configurations. Even for a given format like
60   AAC, some implementations may support AAC multichannel but HE-AAC
65 - Rendering/Grabbing only. This API does not provide any means of
67   user-space for additional processing. This API focuses instead on
68   streaming compressed data to a DSP, with the assumption that the
69   decoded samples are routed to a physical output or logical back-end.
71 - Complexity hiding. Existing user-space multimedia frameworks all
73   API assumes the existence of a platform-specific compatibility layer
75   DSP, eg. Android HAL or PulseAudio sinks. By construction, regular
95 is transmitted to the audio DSP. DMA transfers from main memory to an
132   This routines returns the actual settings used by the DSP. Changes to
140   refilled or the delay due to decoding/encoding/io on the DSP.
145 - Addition of FLAC and IEC formats
146 - Merge of encoder/decoder capabilities
147 - Profiles/modes listed as bitmasks to make descriptors more compact
148 - Addition of set_params for decoders (missing in OpenMAX AL)
149 - Addition of AMR/AMR-WB encoding modes (missing in OpenMAX AL)
150 - Addition of format information for WMA
151 - Addition of encoding options when required (derived from OpenMAX IL)
152 - Addition of rateControlSupported (missing in OpenMAX AL)
159                                         +----------+
163                                         +----------+
169          compr_free()                  +----------+
170   +------------------------------------|          |
172   |          +-------------------------|          |<-------------------------+
173   |          |       compr_write()     +----------+                          |
179   |          |                         +----------+                          |
183   |          |                         +----------+                          |
189   |    +----------+                    +----------+                          |
191   |    | PREPARE  |------------------->|  RUNNING |--------------------------+
193   |    +----------+                    +----------+                          |
199   |    +----------+                   +----------+                           |
201   +--->|   FREE   |                   |  PAUSE   |---------------------------+
203        +----------+                   +----------+
213 Also, there might be low-intensity noises due to encoding. Perfect gapless is
216 So we need to pass this to DSP. This metadata is extracted from ID3/MP4 headers
218 interface to pass this information to the DSP. Also DSP and userspace needs to
229   This routine tells DSP that metadata and write operation sent after this would
233   This is called when end of file is reached. The userspace can inform DSP that
234   EOF is reached and now DSP can start skipping padding delay. Also next write
238 - Open
239 - Get caps / codec caps
240 - Set params
241 - Set metadata of the first track
242 - Fill data of the first track
243 - Trigger start
244 - User-space finished sending all,
245 - Indicate next track data by sending set_next_track
246 - Set metadata of the next track
247 - then call partial_drain to flush most of buffer in DSP
248 - Fill data of the next track
249 - DSP switches to second track
260                                         +----------+
262               +------------------------>|  RUNNING |
264               |                         +----------+
270               |                         +----------+
272               |             +-----------|NEXT_TRACK|
274               |             |           +--+-------+
276               |             +--------------+ |
281               |                         +----------+
283               +------------------------ | PARTIAL_ |
285                                         +----------+
289 - Support for VoIP/circuit-switched calls is not the target of this
290   API. Support for dynamic bit-rate changes would require a tight
291   coupling between the DSP and the host stack, limiting power savings.
293 - Packet-loss concealment is not supported. This would require an
297 - Volume control/routing is not handled by this API. Devices exposing a
302 - Embedded audio effects. Such effects should be enabled in the same
305 - multichannel IEC encoding. Unclear if this is required.
307 - Encoding/decoding acceleration is not supported as mentioned
312 - Audio policy/resource management. This API does not provide any
313   hooks to query the utilization of the audio DSP, nor any preemption
316 - No notion of underrun/overrun. Since the bytes written are compressed
319   maybe dealt in user-library
324 - Mark Brown and Liam Girdwood for discussions on the need for this API
325 - Harsha Priya for her work on intel_sst compressed API
326 - Rakesh Ughreja for valuable feedback
327 - Sing Nallasellan, Sikkandar Madar and Prasanna Samaga for