admin-guide/cgroup-v1/cgroups.rst

6 Documentation/admin-guide/cgroup-v1/cpusets.rst
12 Portions Copyright (c) 2004-2006 Silicon Graphics, Inc.
42 ----------------------
44 Control Groups provide a mechanism for aggregating/partitioning sets of
50 A *cgroup* associates a set of tasks with a set of parameters for one
53 A *subsystem* is a module that makes use of the task grouping
55 particular ways. A subsystem is typically a "resource controller" that
56 schedules a resource or applies per-cgroup limits, but it may be
57 anything that wants to act on a group of processes, e.g. a
60 A *hierarchy* is a set of cgroups arranged in a tree, such that
62 hierarchy, and a set of subsystems; each subsystem has system-specific
67 cgroups. Each hierarchy is a partition of all tasks in the system.
69 User-level code may create and destroy cgroups by name in an
71 which cgroup a task is assigned, and list the task PIDs assigned to
72 a cgroup. Those creations and assignments only affect the hierarchy
78 accounting/limiting the resources which processes in a cgroup can
79 access. For example, cpusets (see Documentation/admin-guide/cgroup-v1/cpusets.rst) allow
80 you to associate a set of CPUs and a set of memory nodes with the
83 .. _cgroups-why-needed:
86 ----------------------------
89 Linux kernel, mainly for resource-tracking purposes. Such efforts
91 namespaces. These all require the basic notion of a
103 different subsystems - having parallel hierarchies allows each
104 hierarchy to be a natural division of tasks, without having to handle
109 At one extreme, each resource controller or subsystem could be in a
113 As an example of a scenario (originally proposed by vatsa@in.ibm.com)
114 that can benefit from multiple hierarchies, consider a large
115 university server with various users - students, professors, system
126                that they can run anywhere) with a limit of 20%
144 the admin can easily set up a script which receives exec notifications
149 With only a single hierarchy, he now would potentially have to create
150 a separate cgroup for every browser launched and associate it with
155 access temporarily to a student's browser (since it is night and the user
159 With ability to write PIDs directly to resource classes, it's just a
173 ---------------------------------
177  - Each task in the system has a reference-counted pointer to a
180  - A css_set contains a set of reference-counted pointers to
182    registered in the system. There is no direct link from a task to
183    the cgroup of which it's a member in each hierarchy, but this
187    and in performance-critical code, whereas operations that require a
189    cgroups) are less common. A linked list runs through the cg_list
191    css_set->tasks.
193  - A cgroup hierarchy filesystem can be mounted for browsing and
196  - You can list all the tasks (by PID) attached to any cgroup.
198 The implementation of cgroups requires a few, simple hooks
199 into the rest of the kernel, none in performance-critical paths:
201  - in init/main.c, to initialize the root cgroups and initial
204  - in fork and exit, to attach and detach a task from its css_set.
206 In addition, a new file system of type "cgroup" may be mounted, to
208 kernel.  When mounting a cgroup hierarchy, you may specify a
209 comma-separated list of subsystems to mount as the filesystem mount
211 mount a hierarchy containing all registered subsystems.
216 hierarchy, the mount will fail with -EBUSY. Otherwise, a new hierarchy
219 It's not currently possible to bind a new subsystem to an active
220 cgroup hierarchy, or to unbind a subsystem from an active cgroup
222 error-recovery issues.
224 When a cgroup filesystem is unmounted, if there are any
225 child cgroups created below the top-level cgroup, that hierarchy
229 No new system calls are added for cgroups - all support for
236 Each cgroup is represented by a directory in the cgroup file system
239  - tasks: list of tasks (by PID) attached to that cgroup.  This list
240    is not guaranteed to be sorted.  Writing a thread ID into this file
242  - cgroup.procs: list of thread group IDs in the cgroup.  This list is
245    Writing a thread group ID into this file moves all threads in that
247  - notify_on_release flag: run the release agent on exit?
248  - release_agent: the path to use for release notifications (this file
255 command.  The properties of a cgroup, such as its flags, are
260 a large system into nested, dynamically changeable, "soft-partitions".
263 children of that task, to a cgroup allows organizing the work load
264 on a system into related sets of tasks.  A task may be re-attached to
268 When a task is moved from one cgroup to another, it gets a new
269 css_set pointer - if there's an already existing css_set with the
270 desired collection of cgroups then that group is reused, otherwise a new
272 looking into a hash table.
274 To allow access from a cgroup to the css_sets (and hence tasks)
275 that comprise it, a set of cg_cgroup_link objects form a lattice;
276 each cg_cgroup_link is linked into a list of cg_cgroup_links for
277 a single cgroup on its cgrp_link_list field, and a list of
278 cg_cgroup_links for a single css_set on its cg_link_list.
280 Thus the set of tasks in a cgroup can be listed by iterating over
281 each css_set that references the cgroup, and sub-iterating over
284 The use of a Linux virtual file system (vfs) to represent the
285 cgroup hierarchy provides for a familiar permission and name space
286 for cgroups, with a minimum of additional kernel code.
289 ------------------------------------
291 If the notify_on_release flag is enabled (1) in a cgroup, then
302 a cgroup hierarchy's release_agent path is empty.
305 ---------------------------------
308 flag is enabled (1) in a cgroup, a new cpuset cgroup will copy its
312 --------------------------
314 To start a new job that is to be contained within a cgroup, using
317  1) mount -t tmpfs cgroup_root /sys/fs/cgroup
319  3) mount -t cgroup -ocpuset cpuset /sys/fs/cgroup/cpuset
322  5) Start a task that will be the "founding father" of the new job.
327 For example, the following sequence of commands will setup a cgroup
329 and then start a subshell 'sh' in that cgroup::
331   mount -t tmpfs cgroup_root /sys/fs/cgroup
333   mount -t cgroup cpuset -ocpuset /sys/fs/cgroup/cpuset
337   /bin/echo 2-3 > cpuset.cpus
349 ---------------
354 To mount a cgroup hierarchy with all available subsystems, type::
356   # mount -t cgroup xxx /sys/fs/cgroup
367 resources you want to control. Therefore, you should mount a tmpfs on
371   # mount -t tmpfs cgroup_root /sys/fs/cgroup
374 To mount a cgroup hierarchy with just the cpuset and memory
377   # mount -t cgroup -o cpuset,memory hier1 /sys/fs/cgroup/rg1
386 To Specify a hierarchy's release_agent::
388   # mount -t cgroup -o cpuset,release_agent="/sbin/cpuset_release_agent" \
394 when the hierarchy consists of a single (root) cgroup. Supporting
398 Then under /sys/fs/cgroup/rg1 you can find a tree that corresponds to the
408 If you want to create a new cgroup under /sys/fs/cgroup/rg1::
432 To remove a cgroup, just use rmdir::
437 has processes attached, or is held alive by other subsystem-specific
441 -----------------------
447 Note that it is PID, not PIDs. You can only attach ONE task at a time.
460 threads in a threadgroup at once. Echoing the PID of any task in a
465 Note: Since every task is always a member of exactly one cgroup in each
466 mounted hierarchy, to remove a task from its current cgroup you must
467 move it into a new cgroup (possibly the root cgroup) by writing to the
471 a process to another cgroup can fail.
474 --------------------------------
476 Passing the name=<x> option when mounting a cgroups hierarchy
478 mounting a pre-existing hierarchy, in order to refer to it by name
480 nameless, or has a unique name.
482 The name should match [\w.-]+
484 When passing a name=<x> option for a new hierarchy, you need to
487 you give a subsystem a name.
497 ------------
500 system needs to create a cgroup_subsys object. This contains
502 with a subsystem ID which will be assigned by the cgroup system.
506 - subsys_id: a unique array index for the subsystem, indicating which
507   entry in cgroup->subsys[] this subsystem should be managing.
509 - name: should be initialized to a unique subsystem name. Should be
512 - early_init: indicate if the subsystem needs early initialization
520 -------------------
522 There is a global mutex, cgroup_mutex, used by the cgroup
523 system. This should be taken by anything that wants to modify a
533 Accessing a task's cgroup pointer may be done in the following ways:
534 - while holding cgroup_mutex
535 - while holding the task's alloc_lock (via task_lock())
536 - inside an rcu_read_lock() section via rcu_dereference()
539 -----------------
543 - add an entry in linux/cgroup_subsys.h
544 - define a cgroup_subsys object called <name>_cgrp_subsys
548 be successful no-ops.
553 Called to allocate a subsystem state object for a cgroup. The
555 cgroup, returning a pointer to the new object on success or a
557 a structure of type cgroup_subsys_state (typically embedded in a
558 larger subsystem-specific object), which will be initialized by the
561 identified by the passed cgroup object having a NULL parent (since
570 subsystem may choose to fail creation by returning -errno. This
582 cgroup removal will proceed to the next step - css_free(). After this
590 is completely unused; @cgrp->parent is still valid. (Note - can also
591 be called for a newly-created cgroup if an error occurs after this
597 Called prior to moving one or more tasks into a cgroup; if the
603   - it's guaranteed that all are from the same thread group
604   - @tset contains all tasks from the thread group whether or not
606   - the first task is the leader
610 cgroup_taskset_for_each() iterator. Note that this isn't called on a
620 when a subsystem is disabled on a cgroup through
622 subsystems depend on it.  cgroup core makes such a css invisible by
625 This prevents unexpected resource control from a hidden css and
632 Called when a task attach operation has failed after can_attach() has succeeded.
633 A subsystem whose can_attach() has some side-effects should provide this
634 function, so that the subsystem can implement a rollback. If not, not necessary.
642 post-attachment activity that requires memory allocations or blocking.
647 Called when a task is forked into a cgroup.
660 Called when a cgroup subsystem is rebound to a different hierarchy
662 the default hierarchy (which never has sub-cgroups) and a hierarchy
663 that is being created/destroyed (and hence has no sub-cgroups).
671 	- Trusted (XATTR_TRUSTED)
672 	- Security (XATTR_SECURITY)
682 in containers and systemd for assorted meta data like main PID in a cgroup
683 (systemd creates a cgroup per service).
691   A: bash's builtin 'echo' command does not check calls to write() against
693      able to tell whether a command succeeded or failed.
696   A: We can only return one error code per call to write(). So you should also