linux-6.12.1/Documentation/memory-barriers.txt

16 meant as a guide to using the various memory barriers provided by Linux, but
31  (2) to provide a guide as to how to use the barriers that are available.
51  (*) What are memory barriers?
54      - What may not be assumed about memory barriers?
55      - Address-dependency barriers (historical).
59      - Read memory barriers vs load speculation.
62  (*) Explicit kernel barriers.
65      - CPU memory barriers.
67  (*) Implicit kernel memory barriers.
78  (*) Where are memory barriers needed?
363 Memory barriers are such interventions.  They impose a perceived partial
369 branch prediction and various types of caching.  Memory barriers are used to
377 Memory barriers come in four basic varieties:
379  (1) Write (or store) memory barriers.
393      [!] Note that write barriers should normally be paired with read or
394      address-dependency barriers; see the "SMP barrier pairing" subsection.
397  (2) Address-dependency barriers (historical).
434      [!] Note that address-dependency barriers should normally be paired with
435      write barriers; see the "SMP barrier pairing" subsection.
438      dependency barriers.  Nowadays, APIs for marking loads from shared
440      address-dependency barriers.
442  (3) Read (or load) memory barriers.
452      Read memory barriers imply address-dependency barriers, and so can
455      [!] Note that read barriers should normally be paired with write barriers;
459  (4) General memory barriers.
468      General memory barriers imply both read and write memory barriers, and so
518 Memory barriers are only required where there's a possibility of interaction
521 memory barriers are unnecessary in that piece of code.
532 There are certain things that the Linux kernel memory barriers do not guarantee:
574 address-dependency barriers.
577 load-to-store relations, address-dependency barriers are not necessary
580 The requirement of address-dependency barriers is a little subtle, and
718 Control dependencies pair normally with other types of barriers.
768 memory barriers, for example, smp_store_release():
779 In contrast, without explicit memory barriers, two-legged-if control
930   (*) Control dependencies pair normally with other types of barriers.
945 General barriers pair with each other, though they also pair with most
946 other types of barriers, albeit without multicopy atomicity.  An acquire
948 barriers, including of course general barriers.  A write barrier pairs
1005 Firstly, write barriers act as partial orderings on store operations.
1043 Secondly, address-dependency barriers act as partial orderings on address-
1395 CPU 3's load from Y.  In addition, the memory barriers guarantee that
1413 However, dependencies, read barriers, and write barriers are not always
1433 writes.  General barriers are therefore required to ensure that all CPUs
1436 General barriers can compensate not only for non-multicopy atomicity,
1517 use general barriers throughout.
1524 The Linux kernel has a variety of different barriers that act at different
1529   (*) CPU memory barriers.
1831 Please note that these compiler barriers have no direct effect on the CPU,
1838 The Linux kernel has seven basic CPU memory barriers:
1848 All memory barriers except the address-dependency barriers imply a compiler
1861 SMP memory barriers are reduced to compiler barriers on uniprocessor compiled
1866 [!] Note that SMP memory barriers _must_ be used to control the ordering of
1870 Mandatory barriers should not be used to control SMP effects, since mandatory
1871 barriers impose unnecessary overhead on both SMP and UP systems. They may,
1873 windows.  These barriers are required even on non-SMP systems as they affect
1891      barriers, but where the code needs a memory barrier. Examples for atomic
1954      Note that the dma_*() barriers do not provide any ordering guarantees for
1970      For load from persistent memory, existing read memory barriers are sufficient
1985 Some of the other functions in the linux kernel imply memory barriers, amongst
2005 for each construct.  These operations all imply certain barriers:
2041 one-way barriers is that the effects of instructions outside of a critical
2143 (RELEASE equivalent) will act as compiler barriers only.  So if memory or I/O
2144 barriers are required in such a situation, they must be provided from some
2156 barriers.
2264 [!] Note that the memory barriers implied by the sleeper and the waker do _not_
2283 code on both sides must interpolate its own memory barriers between the
2303 Other functions that imply barriers:
2305  (*) schedule() and similar imply full memory barriers.
2469 operations are noted specially as some of them imply full memory barriers and
2492 use of memory barriers unnecessary, if the accessor functions are used to refer
2494 memory barriers are required to enforce ordering.
2535 explicit barriers are used.
2539 registers that form implicit I/O barriers.
2702 barriers for the most part act at the interface between the CPU and its cache
2703 (memory barriers logically act on the dotted line in the following diagram):
2740 What memory barriers are concerned with is controlling the order in which
2745 [!] Memory barriers are _not_ needed within a given CPU, as CPUs always see
2927 barriers for this use-case would be possible but is often suboptimal.
2937 MMIO effects, use mandatory barriers.
2947 Memory barriers can be used to implement circular buffering without the need