Lines Matching full:compression
35 zstd, short for Zstandard, is a fast lossless compression algorithm, targeting
36 real-time compression scenarios at zlib-level and better compression ratios.
37 The zstd compression library provides in-memory compression and decompression
40 The library supports regular compression levels from 1 up to ZSTD_maxCLevel(),
43 compression levels, which extend the range of speed vs. ratio preferences.
44 The lower the level, the faster the speed (at the cost of compression).
46 Compression can be done in:
49 - unbounded multiple steps (described as Streaming compression)
51 The compression ratio achievable on small data can be highly improved using
52 a dictionary. Dictionary compression can be performed in:
110 * Hint : compression runs faster if `dstCapacity` >= `ZSTD_compressBound(srcSize)`.
140 …* note 3 : decompressed size is always present when compression is completed using single-pass f…
175 ZSTDLIB_API int ZSTD_minCLevel(void); /*!< minimum negative compression level…
176 ZSTDLIB_API int ZSTD_maxCLevel(void); /*!< maximum compression level availabl…
177 ZSTDLIB_API int ZSTD_defaultCLevel(void); /*!< default compression level, specifi…
183 /*= Compression context
186 * and re-use it for each successive compression operation.
189 * It doesn't change the compression ratio, which remains identical.
200 * this function compresses at requested compression level,
213 * and re-use it for each successive compression operation.
231 * Advanced compression API (Requires v1.4.0+)
248 /* Compression strategies, listed from fastest to strongest */
264 /* compression parameters
268 … ZSTD_c_compressionLevel=100, /* Set compression parameters according to pre-defined cLevel table.
269 * Note that exact compression parameters are dynamically determined,
270 * depending on both compression level and srcSize (when known).
273 * Note 1 : it's possible to pass a negative compression level.
274 … * Note 2 : setting a level does not automatically set all other compression parameters
275 … * to default. Setting this will however eventually dynamically impact the compression
278 /* Advanced compression parameters :
279 * It's possible to pin down compression parameters to some specific values.
292 * Larger tables improve compression ratio of strategies <= dFast,
298 * Larger tables result in better and slower compression.
304 * More attempts result in better and slower compression.
310 … * Larger values increase compression and decompression speed, but decrease ratio.
318 * Larger values make compression stronger, and slower.
321 * Larger values make compression faster, and weaker.
325 * resulting in stronger and slower compression.
329 * This parameter is designed to improve compression ratio
335 … * compression strategy >= ZSTD_btopt (== compression level 16+) */
337 * Larger values increase memory usage and compression ratio,
338 * but decrease compression speed.
343 * Larger/too small values usually decrease compression ratio.
347 … * Larger values improve collision resolution but decrease compression speed.
353 * Larger values improve compression speed.
354 … * Deviating far from default value will likely result in a compression ratio decrease.
359 * Content size must be known at the beginning of compression.
374 * while compression is performed in parallel, within worker thread(s).
379 … * compression is performed inside Caller's thread, and all invocations are blocking */
380 …ZSTD_c_jobSize=401, /* Size of a compression job. This value is enforced only when nbWorkers …
381 …* Each compression job is completed in parallel, so this value can indirectly impact the nb of act…
382 … * 0 means default, which is dynamically determined based on compression parameters.
387 … * It helps preserve compression ratio, while each job is compressed in parallel.
389 * Larger values increase compression ratio, but decrease speed.
452 * Set one compression parameter, selected by enum ZSTD_cParameter.
455 …g a parameter is generally only possible during frame initialization (before starting compression).
457 * the following parameters can be updated _during_ compression (within same frame):
490 * Useful after an error, or to interrupt any ongoing compression.
492 * Compression parameters and dictionary remain unchanged.
497 …* Parameters can only be changed between 2 sessions (i.e. no compression is curre…
504 …* Behave the same as ZSTD_compressCCtx(), but compression parameters are set using the advanced A…
507 …* - Compression parameters are pushed into CCtx before starting compression, using ZSTD_CCtx_set*…
508 * - The function is always blocking, returns when compression is completed.
509 * Hint : compression runs faster if `dstCapacity` >= `ZSTD_compressBound(srcSize)`.
566 * Set one compression parameter, selected by enum ZSTD_dParameter.
602 * Streaming compression - HowTo
606 * ZSTD_CStream objects can be reused multiple times on consecutive compression operations.
613 * Parameters are sticky : when starting a new compression on the same context,
614 * it will re-use the same sticky parameters as previous compression session.
668 /*===== Streaming compression functions =====*/
670 …llect more data, encoder decides when to output compressed result, for optimal compression ratio */
673 …ll continue: any future data can still reference previously compressed data, improving compression.
674 … * note : multithreaded compression will block to flush as much output as possible. */
679 … : note : multithreaded compression will block to flush as much output as possible. */
684 …* - Compression parameters are pushed into CCtx before starting compression, using ZSTD_CCtx_set*…
685 …* - Compression parameters cannot be changed once compression is started (save a list of exceptio…
701 * Before starting a new compression job, or changing compression parameters,
719 …* It's not rare that performance ends being spent more into the interface, rather than compression…
731 * Streaming in combination with advanced parameters and dictionary compression
808 * Compression at an explicit compression level using a Dictionary.
822 * Dictionary must be identical to the one used during compression.
841 …* The resulting state can be used for future compression operations with very limited startup cos…
858 * Compression using a digested Dictionary.
860 * Note : compression level is _decided at dictionary creation time_,
944 * Tables are dependent on compression parameters, and for this reason,
945 * compression parameters can no longer be changed after loading a dictionary.
955 * Note that compression parameters are enforced from within CDict,
956 * and supersede any compression parameter previously set within CCtx.
972 …erforms much faster, especially during decompression (compression speed is tunable with compressio…
975 * Note 1 : Prefix buffer is referenced. It **must** outlive compression.
976 * Its content must remain unmodified during compression.
980 …* Note 3 : Referencing a prefix involves building tables, which are dependent on compression para…
1026 * and must use the same prefix as the one used during compression.
1107 /* compression parameter bounds */
1197 …unsigned windowLog; /*< largest match distance : larger == more compression, more memory nee…
1198 …unsigned chainLog; /*< fully searched segment : larger == more compression, slower, more me…
1200 unsigned searchLog; /*< nb of searches : larger == more compression, slower */
1201 …h; /*< match length searched : larger == faster decompression, sometimes less compression */
1202 …gth; /*< acceptable match size for optimal parser (only) : larger == more compression, slower */
1209 … 1: no dictID will be saved into frame header (dictID is only useful for dictionary compression) */
1256 * means that the compression can search both the dictionary and input
1258 * the compression faster per-byte of input. However, the initial copy of
1260 * compression. For small compressions (< 8 KB), that copy can dominate
1261 * the cost of the compression.
1263 * - The CDict's tables can be used in-place. In this model, compression is
1276 * at the beginning of a compression. However, if experimentation shows that
1287 …_lcm_auto = 0, /*< Automatically determine the compression mode based on the compression …
1288 … * Negative compression levels will be uncompressed, and positive compression
1290 …ZSTD_lcm_huffman = 1, /*< Always attempt Huffman compression. Uncompressed literals will sti…
1291 * emitted if Huffman compression is not profitable. */
1320 * note 2 : decompressed size is always present when compression is done with ZSTD_compress()
1366 * zc can be used to insert custom compression params.
1395 * The compression behavior changes based on cctx params. In particular:
1409 …ionLevel accordingly adjusts the strength of the entropy coder, as it would in typical compression.
1469 * for any compression level up to selected one.
1478 * because the CCtx compression context will need less memory.
1482 …* Both can be used to estimate memory using custom compression parameters and arbitrary srcSize l…
1484 * Note 2 : only single-threaded compression is supported.
1493 …* ZSTD_estimateCStreamSize() will provide a budget large enough for any compression level up to s…
1497 …ed in tandem with ZSTD_CCtxParams_setParameter(). Only single-threaded compression is supported. T…
1498 * Note : CStream size estimation is only correct for single-threaded compression.
1513 …* ZSTD_estimateCDictSize_advanced() makes it possible to control compression parameters precisely…
1534 * Note 3 : cParams : use ZSTD_getCParams() to convert a compression level
1585 * These prototypes make it possible to share a thread pool among multiple compression contexts.
1587 * a threaded compression mode (via ZSTD_c_nbWorkers parameter).
1618 * Advanced compression functions
1622 * Create a digested dictionary for compression
1630 …* @return ZSTD_compressionParameters structure for a selected compression level and estimated srcS…
1702 * it will result in compression ratio degradation.
1707 * NOTE 3: Rsyncable mode limits maximum compression speed to ~400 MB/s.
1708 * If the selected compression level is already running significantly slower,
1713 /* Select a compression format.
1735 * literals compression based on the compression parameters - specifically,
1736 * negative compression levels do not use literal compression.
1748 * but compression ratio may regress significantly if guess considerably underestimates */
1759 * construction (via ZSTD_createCDict_advanced2()). A compression will then
1771 * To an approximation, a compression using a dictionary can then use those
1773 * compression where the simulated compression of the dictionary left off.
1780 * written as the compression goes along. This means we can choose a search
1787 * highly recommended that you simply set a compression level in the CCtxParams
1801 * In general, you should expect compression to be faster--sometimes very much
1814 * compression will fail if it ever changes. This means the only flush
1817 * MUST not be modified during compression or you will get data corruption.
1830 * memory. However, compression WILL fail if you violate the preconditions.
1833 * not be modified during compression or you will get data corruption. This
1863 * For use with sequence compression API: ZSTD_compressSequences().
1874 * For use with sequence compression API: ZSTD_compressSequences().
1895 * block splitting based on the compression parameters.
1907 * Note that this only pertains to compression strategies: greedy, lazy, and lazy2
1914 * Zstd produces different results for prefix compression when the prefix is
1923 * like when doing delta compression, you should select this option. It comes
1932 * Get the requested compression parameter value, selected by enum ZSTD_cParameter,
1950 * - ZSTD_compressStream2() : Do compression using the CCtx.
1954 * for static allocation of CCtx for single-threaded compression.
1965 * Initializes the compression parameters of cctxParams according to
1966 * compression level. All other parameters are reset to their default values.
1971 * Initializes the compression and frame parameters of cctxParams according to
1978 * Set one compression parameter, selected by enum ZSTD_cParameter.
1988 * Get the requested value of one compression parameter, selected by enum ZSTD_cParameter.
1994 * Apply a set of ZSTD_CCtx_params to the compression context.
1995 * This can be done even after compression is started,
1996 * if nbWorkers==0, this will have no impact until a new compression is started.
2071 * allowing selection between ZSTD_format_e input compression formats
2111 * of whether checksumming was specified during compression. This offers some
2169 /*===== Advanced Streaming compression functions =====*/
2231 * note : cdict will just be referenced, and must outlive compression session
2357 Buffer-less streaming compression (synchronous mode)
2361 ZSTD_CCtx object can be re-used multiple times within successive compression operations.
2364 Use ZSTD_compressBegin(), or ZSTD_compressBegin_usingDict() for dictionary compression.
2373 ZSTD_compressContinue() doesn't guarantee recover after a failed compression.
2386 /*===== Buffer-less streaming compression functions =====*/
2399 …rameters const fParams, unsigned long long const pledgedSrcSize); /* compression parameters are …
2440 Finally, if you control the compression process, you can also ignore all buffer size rules,
2525 + compression : any ZSTD_compressBegin*() variant, including with dictionary