admin-guide/device-mapper/vdo-design.rst

4 Design of dm-vdo
7 The dm-vdo (virtual data optimizer) target provides inline deduplication,
8 compression, zero-block elimination, and thin provisioning. A dm-vdo target
14 dm-vdo. For usage, see vdo.rst in the same directory as this file.
17 vdo target has a maximum block size of 4K. However, it can achieve
25 The design of dm-vdo is based on the idea that deduplication is a two-part
27 storing multiple copies of those duplicates. Therefore, dm-vdo has two main
37 structures involved in a single write operation to a vdo target is larger
38 than most other targets. Furthermore, because vdo must operate on small
40 performance can only be achieved through parallelism. Therefore, vdo's
43 Most of a vdo's main data structures are designed to be easily divided into
55 operations, because vdo guarantees that no other thread will alter those
61 can be reconfigured each time a vdo target is started.
66 In order to identify duplicate data efficiently, vdo was designed to
80 those savings, vdo does not attempt to find every last duplicate block. It
93 vdo does not use a cryptographic hash, a malicious workload could be
94 constructed. Because of these inaccuracies, vdo treats the locations in the
135 in memory and is saved to storage only when the vdo target is shut down.
169 200 GB file to a vdo target and then immediately writes it again, the two
176 deduplication beyond the 256GB threshold, vdo can be configured to use a
179 later. It is important to consider the expected workload for a vdo target
199 A vio (short for Vdo I/O) is conceptually similar to a bio, with additional
200 fields and data to track vdo-specific information. A struct vio maintains a
202 vdo. The vio is kept separate from its related bio because there are many
203 circumstances where vdo completes the bio but must continue to do work
206 Metadata reads and writes, and other writes that originate within vdo, use
210 related to deduplication and other vdo features. The data_vio is the
211 primary unit of application work in vdo. Each data_vio proceeds through a
229 Most of the vdo volume belongs to the slab depot. The depot contains a
290 makes growing the logical size of a vdo relatively easy.
316 be updated to reflect the operation. These entries allow the vdo device to
322 All write I/O to vdo is asynchronous. Each bio will be acknowledged as soon
323 as vdo has done enough work to guarantee that it can complete the write
328 the vdo target will also flush any remaining I/O.
427     acknowledged because the vdo device may be out of space.
429 6.  At this point vdo must determine where to store the application data.
497        data block. Compression is only helpful if vdo can pack at least 2
500        to fill out the compressed block. There is a mechanism for vdo to
600 All storage within vdo is managed as 4KB blocks, but it can accept writes
610 When a vdo is restarted after a crash, it will attempt to recover from the
617 the vdo comes back online, while the remainder of the slab journals are
622 If a vdo encounters an unrecoverable error, it will enter read-only mode.
624 lost. The vdo may be instructed to rebuild as best it can in order to
632 consistent with each other. This allows vdo to resume normal operation and