admin-guide/mm/numa_memory_policy.rst

2 NUMA Memory Policy
5 What is NUMA Memory Policy?
8 In the Linux kernel, "memory policy" determines from which node the kernel will
10 supported platforms with Non-Uniform Memory Access architectures since 2.4.?.
11 The current memory policy support was added to Linux 2.6 around May 2004.  This
12 document attempts to describe the concepts and APIs of the 2.6 memory policy
16 (``Documentation/admin-guide/cgroup-v1/cpusets.rst``)
19 programming interface that a NUMA-aware application can take advantage of.  When
24 Memory Policy Concepts
28 ------------------------
30 The Linux kernel supports _scopes_ of memory policy, described here from
33 System Default Policy
34 	this policy is "hard coded" into the kernel.  It is the policy
36 	one of the more specific policy scopes discussed below.  When
37 	the system is "up and running", the system default policy will
39 	up, the system default policy will be set to interleave
41 	not to overload the initial boot node with boot-time
44 Task/Process Policy
45 	this is an optional, per-task policy.  When defined for a
46 	specific task, this policy controls all page allocations made
48 	specific scope. If a task does not define a task policy, then
50 	task policy "fall back" to the System Default Policy.
52 	The task policy applies to the entire address space of a task. Thus,
55 	to establish the task policy for a child task exec()'d from an
56 	executable image that has no awareness of memory policy.  See the
57 	:ref:`Memory Policy APIs <memory_policy_apis>` section,
59 	that a task may use to set/change its task/process policy.
61 	In a multi-threaded task, task policies apply only to the thread
62 	[Linux kernel task] that installs the policy and any threads
64 	at the time a new task policy is installed retain their current
65 	policy.
67 	A task policy applies only to pages allocated after the policy is
69 	changes its task policy remain where they were allocated based on
70 	the policy at the time they were allocated.
74 VMA Policy
76 	virtual address space.  A task may define a specific policy for a range
78 	:ref:`Memory Policy APIs <memory_policy_apis>` section,
80 	policy.
82 	A VMA policy will govern the allocation of pages that back
84 	address space that don't have an explicit VMA policy will fall
85 	back to the task policy, which may itself fall back to the
86 	System Default Policy.
90 	* VMA policy applies ONLY to anonymous pages.  These include
93 	  mmap()ed with the MAP_ANONYMOUS flag.  If a VMA policy is
96 	  MAP_PRIVATE flag, the VMA policy will only be applied when
98 	  mapping-- i.e., at Copy-On-Write.
101 	  virtual address space--a.k.a. threads--independent of when
102 	  the policy is installed; and they are inherited across
106 	  are NOT inheritable across exec().  Thus, only NUMA-aware
109 	* A task may install a new VMA policy on a sub-range of a
112 	  its own policy.
114 	* By default, VMA policy applies only to pages allocated after
115 	  the policy is installed.  Any pages already faulted into the
117 	  policy at the time they were allocated.  However, since
120 	  installed policy.
122 Shared Policy
126 	policies--using the mbind() system call specifying a range of
131 	the object share the policy, and all pages allocated for the
132 	shared object, by any task, will obey the shared policy.
135 	mmap(MAP_ANONYMOUS|MAP_SHARED), support shared policy.  When shared
136 	policy support was added to Linux, the associated data structures were
138 	support allocation at fault time--a.k.a lazy allocation--so hugetlbfs
139 	shmem segments were never "hooked up" to the shared policy support.
141 	for shared policy has not been completed.
145 	with MAP_SHARED ignore any VMA policy installed on the virtual
149 	task policy, if any, else System Default Policy.
151 	The shared policy infrastructure supports different policies on subset
153 	the task that installs the policy for each range of distinct policy.
157 	a shared memory region, when one task has installed shared policy on
161 -----------------------------
163 A NUMA memory policy consists of a "mode", optional mode flags, and
165 policy, the optional mode flags determine the behavior of the mode,
167 policy behavior.
173 NUMA memory policy supports the following 4 behavioral modes:
175 Default Mode--MPOL_DEFAULT
176 	This mode is only used in the memory policy APIs.  Internally,
177 	MPOL_DEFAULT is converted to the NULL memory policy in all
178 	policy scopes.  Any existing non-default policy will simply be
180 	MPOL_DEFAULT means "fall back to the next most specific policy
183 	For example, a NULL or default task policy will fall back to the
184 	system default policy.  A NULL or default vma policy will fall
185 	back to the task policy.
187 	When specified in one of the memory policy APIs, the Default mode
190 	It is an error for the set of nodes specified for this policy to
191 	be non-empty.
195 	nodes specified by the policy.  Memory will be allocated from
201 	from the single node specified in the policy.  If that
202 	allocation fails, the kernel will search other nodes, in order
206 	Internally, the Preferred policy uses a single node--the
209 	and the policy is interpreted as local allocation.  "Local"
210 	allocation policy can be viewed as a Preferred policy that
216 	mode.  If an empty nodemask is passed, the policy cannot use
222 	page granularity, across the nodes specified in the policy.
228 	policy using the page offset of the faulting address into the
230 	nodes specified by the policy.  It then attempts to allocate a
232 	specified by a Preferred policy or had been selected by a
237 	the set of nodes specified by the policy using a node counter
241 	specified by the policy based on the order in which they are
244 	interleaved system default policy works in this mode.
248 	satisfied from the nodemask specified in the policy. If there is
256 	/sys/kernel/mm/mempolicy/weighted_interleave/
262 NUMA memory policy supports the following optional mode flags:
267 	nodes changes after the memory policy has been defined.
275 	With this flag, if the user-specified nodes overlap with the
276 	nodes allowed by the task's cpuset, then the memory policy is
278 	overlap, the Default policy is used.
281 	mems 1-3 that sets an Interleave policy over the same set.  If
282 	the cpuset's mems change to 3-5, the Interleave will now occur
296 	set of allowed nodes.  The kernel stores the user-passed nodemask,
306 	1,3,5 may be remapped to 7-9 and then to 1-3 if the set of
312 	nodemask, the policy will be effected over the first (and in the
319 	the user's nodemask when the set of allowed nodes is only 0-3),
324 	mems 2-5 that sets an Interleave policy over the same set with
325 	MPOL_F_RELATIVE_NODES.  If the cpuset's mems change to 3-7, the
326 	interleave now occurs over nodes 3,5-7.  If the cpuset's mems
327 	then change to 0,2-3,5, then the interleave occurs over nodes
328 	0,2-3,5.
334 	memory nodes 0 to N-1, where N is the number of memory nodes the
335 	policy is intended to manage.  Let the kernel then remap to the
344 Memory Policy Reference Counting
353 When a new memory policy is allocated, its reference count is initialized
355 new policy.  When a pointer to a memory policy structure is stored in another
357 on completion of the policy installation.
359 During run-time "usage" of the policy, we attempt to minimize atomic operations
363 1) querying of the policy, either by the task itself [using the get_mempolicy()
367 2) examination of the policy to determine the policy mode and associated node
371    BIND policy nodemask is used, by reference, to filter ineligible nodes.
376 1) we never need to get/free the system default policy as this is never
379 2) for querying the policy, we do not need to take an extra reference on the
380    target task's task policy nor vma policies because we always acquire the
384    of a task or thread freeing a policy while another task or thread is
387 3) Page allocation usage of task or vma policy occurs in the fault path where
389    policy requires that the mmap_lock be held for write, the policy can't be
393    shared memory policy while another task, with a distinct mmap_lock, is
394    querying or allocating a page based on the policy.  To resolve this
395    potential race, the shared policy infrastructure adds an extra reference
396    to the shared policy during lookup while holding a spin lock on the shared
397    policy management structure.  This requires that we drop this extra
398    reference when we're finished "using" the policy.  We must drop the
400    used for non-shared policies.  For this reason, shared policies are marked
401    as such, and the extra reference is dropped "conditionally"--i.e., only
409    falling back to task or system default policy for shared memory regions,
415 Memory Policy APIs
418 Linux supports 4 system calls for controlling memory policy.  These APIS
425    Linux kernel.  The kernel system call interfaces, with the 'sys\_'
429 Set [Task] Memory Policy::
434 Set's the calling task's "task/process memory policy" to mode
444 Get [Task] Memory Policy or Related Information::
450 Queries the "task/process memory policy" of the calling task, or the
451 policy or location of a specified virtual address, depending on the
457 Install VMA/Shared Policy for a Range of Task's Address Space::
463 mbind() installs the policy specified by (mode, nmask, maxnodes) as a
464 VMA policy for the range of the calling task's address space specified
476 sys_set_mempolicy_home_node set the home node for a VMA policy present in the
480 the default allocation policy to allocate memory close to the local node for an
484 Memory Policy Command Line Interface
487 Although not strictly part of the Linux implementation of memory policy,
490 + set the task policy for a specified program via set_mempolicy(2), fork(2) and
493 + set the shared policy for a shared memory segment via mbind(2)
495 The numactl(8) tool is packaged with the run-time version of the library
496 containing the memory policy system call wrappers.  Some distributions
497 package the headers and compile-time libraries in a separate development
508 specified for the policy contains nodes that are not allowed by the cpuset and
510 specified for the policy and the set of nodes with memory is used.  If the
511 result is the empty set, the policy is considered invalid and cannot be
512 installed.  If MPOL_F_RELATIVE_NODES is used, the policy's nodes are mapped
518 any of the tasks install shared policy on the region, only nodes whose
520 this information requires "stepping outside" the memory policy APIs to use the
523 memory sets are disjoint, "local" allocation is the only valid policy.