Documentation/block/bfq-iosched.rst

5 BFQ is a proportional-share I/O scheduler, with some extra
6 low-latency capabilities. In addition to cgroups support (blkio or io
9 - BFQ guarantees a high system and application responsiveness, and a
10   low latency for time-sensitive applications, such as audio or video
12 - BFQ distributes bandwidth, not just time, among processes or
19 goal, for a given device, is to achieve the maximum-possible
20 throughput at all times, then do switch off all low-latency heuristics
25 As every I/O scheduler, BFQ adds some overhead to per-I/O-request
27 single-lock-protected, per-request processing time of BFQ---i.e., the
29 completion hooks---is, e.g., 1.9 us on an Intel Core i7-2760QM@2.40GHz
31 instrumentation, and using the throughput-sync.sh script of the S
32 suite [1], in performance-profiling mode). To put this result into
33 context, the total, single-lock-protected, per-request execution time
34 of the lightest I/O scheduler available in blk-mq, mq-deadline, is 0.7
35 us (mq-deadline is ~800 LOC, against ~10500 LOC for BFQ).
44 set (Section 4-2):
45 - Intel i7-4850HQ: 400 KIOPS
46 - AMD A8-3850: 250 KIOPS
47 - ARM CortexTM-A53 Octa-core: 80 KIOPS
52 (Section 4-2). For BFQ, this leads to the following maximum
54 - Intel i7-4850HQ: 310 KIOPS
55 - AMD A8-3850: 200 KIOPS
56 - ARM CortexTM-A53 Octa-core: 56 KIOPS
58 BFQ works for multi-queue devices too.
65     1-1 Personal systems
66     1-2 Server systems
70     4-1 Service guarantees provided
71     4-2 Interface
78 1-1 Personal systems
79 --------------------
81 Low latency for interactive applications
89 - one or more large files are being read, written or copied,
90 - a tree of source files is being compiled,
91 - one or more virtual machines are performing I/O,
92 - a software update is in progress,
93 - indexing daemons are scanning filesystems and updating their
102 Low latency for soft real-time applications
104 Also soft real-time applications, such as audio and video
105 players/streamers, enjoy a low latency and a low drop rate, regardless
109 Higher speed for code-development tasks
113 BFQ executes the I/O-related components of typical code-development
123 and with all the workloads on flash-based devices, BFQ achieves,
126 Strong fairness, bandwidth and delay guarantees
130 among I/O-bound applications in proportion to their weights, with any
131 workload and regardless of the device parameters. From these bandwidth
132 guarantees, it is possible to compute a tight per-I/O-request delay
134 guarantees, BFQ switches to time-based resource sharing (only) for
137 1-2 Server systems
138 ------------------
144 * audio and video-streaming with zero or very low jitter and drop
149 * real-time recording of data in live-dumping applications (e.g.,
158 BFQ is a proportional-share I/O scheduler, whose general structure,
161 - Each process doing I/O on a device is associated with a weight and a
164 - BFQ grants exclusive access to the device, for a while, to one queue
169   - After a queue is granted access to the device, the budget of the
173   - The in-service queue is expired, i.e., its service is suspended,
177     - The budget timeout prevents processes doing random I/O from
181     - Actually, as in CFQ, a queue associated with a process issuing
186       throughput on rotational devices and on non-queueing flash-based
193       - With respect to idling for service guarantees, if several
200      - On flash-based storage with internal queueing of commands
204        guaranteeing low latency or fairness. In these cases, overall
205        throughput may be sub-optimal. No solution currently exists to
209   - If low-latency mode is enabled (default configuration), BFQ
211     real-time applications (e.g., video or audio players/streamers),
215     throughput. For brevity, we call it just "weight-raising" the whole
217     particular, BFQ provides a milder form of weight-raising for
218     interactive applications, and a stronger form for soft real-time
221   - BFQ automatically deactivates idling for queues born in a burst of
227   - As CFQ, BFQ merges queues performing interleaved I/O, i.e.,
238   - Queues are scheduled according to a variant of WF2Q+, named
239     B-WF2Q+, and implemented using an augmented rb-tree to preserve an
240     O(log N) overall complexity.  See [2] for more details. B-WF2Q+ is
243   - B-WF2Q+ guarantees a tight deviation with respect to an ideal,
244     perfectly fair, and smooth service. In particular, B-WF2Q+
250   - The last, budget-independence, property (although probably
254     - First, with any proportional-share scheduler, the maximum
258       accurate service of B-WF2Q+, but also because BFQ *does not*
262     - Second, BFQ is free to choose, for every process (queue), the
265       updates queue budgets with a simple feedback-loop algorithm that
267       tight latency guarantees to time-sensitive applications. When
268       the in-service queue expires, this algorithm computes the next
271       - Let large budgets be eventually assigned to the queues
272 	associated with I/O-bound applications performing sequential
276       - Let small budgets be eventually assigned to the queues
277 	associated with time-sensitive applications (which typically
280 	B-WF2Q+ will serve that queue (Subsec 3.3 in [2]).
282 - If several processes are competing for the device at the same time,
288 - ioprio classes are served in strict priority order, i.e.,
289   lower-priority queues are not served as long as there are
290   higher-priority queues.  Among queues in the same class, the
291   bandwidth is distributed in proportion to the weight of each
292   queue. A very thin extra bandwidth is however guaranteed to
304 max_budget. The other performance-related parameters have been
309 In particular, the tunables back_seek-max, back_seek_penalty,
315 per-process ioprio and weight
316 -----------------------------
321 weight = (IOPRIO_BE_NR - ioprio) * 10.
323 Beware that, if low-latency is set, then BFQ automatically raises the
324 weight of the queues associated with interactive and soft real-time
328 ----------
332 slice_idle is a non-zero value. Idling has a double purpose: boosting
344 as flash-based storage with internal command queueing (and
352 flash-based fast storage, setting slice_idle=0 might end up in better
356 case of differentiated weights or differentiated I/O-request lengths.
364 terms of I/O-request dispatches. To guarantee that the actual service
374 strict_guarantees tunable, short-term service guarantees may be
377 video playing/streaming, a very low drop rate may be more important
382 -------------
389 -----------------
393 - always performs idling when the in-service queue becomes empty;
395 - forces the device to serve one I/O request at a time, by dispatching a
398 In the presence of differentiated weights or I/O-request sizes, both
400 receives its allotted share of the bandwidth. The first condition is
403 devices reorder internally-queued requests, which may trivially break
409 -------------
420 -----------------
430 -----------------
436 ----------------
443 -----------
445 This parameter is used to enable/disable BFQ's low latency mode. By
446 default, low latency mode is enabled. If enabled, interactive and soft
447 real-time applications are privileged and experience a lower latency,
450 DISABLE this mode if you need full control on bandwidth
452 increases the bandwidth share of privileged applications, as the main
458 entail a lower throughput. To achieve the highest-possible throughput
459 on a non-rotational device, setting slice_idle to 0 may be needed too
460 (at the cost of giving up any strong guarantee on fairness and low
464 ------------
470 short-term bandwidth and latency guarantees, especially if the
474 ----------
481 values is that they coarsen the granularity of short-term bandwidth
484 The default value is 0, which enables auto-tuning: BFQ sets max_budget
492 it with auto-tuning. An alternative way to achieve this goal is to
498 BFQ supports both cgroups-v1 and cgroups-v2 io controllers, namely
499 blkio and io. In particular, BFQ supports weight-based proportional
502 4-1 Service guarantees provided
503 -------------------------------
506 device bandwidth, according to group weights. For example, a group
507 with weight 200 gets twice the bandwidth, and not just twice the time,
510 BFQ supports hierarchies (group trees) of any depth. Bandwidth is
512 group, the children of the group share the whole bandwidth of the
515 same share of the whole group bandwidth, unless the ioprio of the
518 The resource-sharing guarantee for a group may partially or totally
519 switch from bandwidth to time, if providing bandwidth guarantees to
521 per-process basis: if a process of a leaf group causes throughput loss
522 if served in such a way to receive its share of the bandwidth, then
523 BFQ switches back to just time-based proportional share for that
526 4-2 Interface
527 -------------
529 To get proportional sharing of bandwidth with BFQ for a given device,
533 BFQ-specific cgroup parameters and stats begin with the "bfq."
534 prefix. So, with cgroups-v1 or cgroups-v2, the full prefix for
535 BFQ-specific files is "blkio.bfq." or "io.bfq." For example, the group
539 As for cgroups-v1 (blkio controller), the exact set of stat files
540 created, and kept up-to-date by bfq, depends on whether
543 Documentation/admin-guide/cgroup-v1/blkio-controller.rst. If, instead,
557 ----------
574         Recall that, if low-latency is set, then BFQ automatically raises the
575         weight of the queues associated with interactive and soft real-time
579         This specifies a per-device weight for the cgroup. The syntax is
591     Technologies (MST-2015), May 2015.
593     http://algogroup.unimore.it/people/paolo/disk_sched/mst-2015.pdf
603     http://algogroup.unimore.it/people/paolo/disk_sched/bfq-v1-suite-results.pdf
606    https://github.com/Algodev-github/S