Documentation/filesystems/relay.rst

1 .. SPDX-License-Identifier: GPL-2.0
9 to userspace via user-defined 'relay channels'.
11 A 'relay channel' is a kernel->user data relay mechanism implemented
12 as a set of per-cpu kernel buffers ('channel buffers'), each
14 clients write into the channel buffers using efficient write
15 functions; these automatically log into the current cpu's channel
19 are associated with the channel buffers using the API described below.
21 The format of the data logged into the channel buffers is completely
25 filtering - this also is left to the kernel client.  The purpose is to
30 functions in the relay interface code - please see that for details.
35 Each relay channel has one buffer per CPU, each buffer has one or more
36 sub-buffers.  Messages are written to the first sub-buffer until it is
38 the next (if available).  Messages are never split across sub-buffers.
40 sub-buffer, while the kernel continues writing to the next.
42 When notified that a sub-buffer is full, the kernel knows how many
44 message couldn't fit into a sub-buffer.  Userspace can use this
47 After copying it, userspace can notify the kernel that a sub-buffer
50 A relay channel can operate in a mode where it will overwrite data not
53 The relay channel itself does not provide for communication of such
60 read sub-buffers; thus in cases where read(2) is being used to drain
61 the channel buffers, special-purpose communication between kernel and
66 read() interface is easy to use, it's not as efficient as the mmap()
70 klog and relay-apps example code
73 The relay interface itself is ready to use, but to make things easier,
76 The relay-apps example tarball, available on the relay sourceforge
77 site, contains a set of self-contained examples, each consisting of a
83 The 'klog debugging functions' patch (klog.patch in the relay-apps
84 tarball) provides a couple of high-level logging functions to the
85 kernel which allow writing formatted text or raw data to a channel,
86 regardless of whether a channel to write into exists or not, or even
93 It is of course possible to use the relay interface from scratch,
94 i.e. without using any of the relay-apps example code or klog, but
98 consumes the read sub-buffers; thus in cases where read(2) is being
99 used to drain the channel buffers, special-purpose communication
101 such as buffer-full conditions would still need to be communicated via
102 some channel though.
104 klog and the relay-apps examples can be found in the relay-apps
111 access to relay channel buffer data.  Here are the file operations
115 open()	    enables user to open an _existing_ channel buffer.
117 mmap()      results in channel buffer being mapped into the caller's
118 	    memory space. Note that you can't do a partial mmap - you
121 read()      read the contents of a channel buffer.  The bytes read are
123 	    again to subsequent reads.  If the channel is being used
124 	    in no-overwrite mode (the default), it can be read at any
126 	    channel is being used in overwrite mode and there are
127 	    active channel writers, results may be unpredictable -
128 	    users should make sure that all logging to the channel has
129 	    ended before using read() with overwrite mode.  Sub-buffer
133 sendfile()  transfer data from a channel buffer to an output file
134 	    descriptor. Sub-buffer padding is automatically removed
138 	    notified when sub-buffer boundaries are crossed.
140 close()     decrements the channel buffer's refcount.  When the refcount
142 	    buffer open, the channel buffer is freed.
145 In order for a user application to make use of relay files, the
148 	mount -t debugfs debugfs /sys/kernel/debug
153 	clients to create or use channels - it only needs to be
161 Here's a summary of the API the relay interface provides to in-kernel clients:
164   channel management functions::
172   channel management typically called on instigation of userspace::
196 Creating a channel
197 ------------------
199 relay_open() is used to create a channel, along with its per-cpu
200 channel buffers.  Each channel buffer will have an associated file
202 read from in user space.  The files are named basename0...basenameN-1
209 structure they create, when the channel is closed - again the host
213 In order for a channel to be created and the host filesystem's files
214 associated with its channel buffers, the user must provide definitions
216 create_buf_file() is called once for each per-cpu buffer from
218 to represent the corresponding channel buffer.  The callback should
219 return the dentry of the file created to represent the channel buffer.
263 If the create_buf_file() callback fails, or isn't defined, channel
266 The total size of each per-cpu buffer is calculated by multiplying the
267 number of sub-buffers by the sub-buffer size passed into relay_open().
268 The idea behind sub-buffers is that they're basically an extension of
269 double-buffering to N buffers, and they also allow applications to
270 easily implement random-access-on-buffer-boundary schemes, which can
271 be important for some high-volume applications.  The number and size
272 of sub-buffers is completely dependent on the application and even for
275 values to use are best decided after some experimentation; in general,
276 though, it's safe to assume that having only 1 sub-buffer is a bad
277 idea - you're guaranteed to either overwrite data or lose events
278 depending on the channel mode being used.
282 default per-cpu set.  This can be useful for applications interested
283 mainly in seeing the relative ordering of system-wide events without
285 merging/sorting per-cpu files in a postprocessing step.
289 non-zero value in addition to creating the file that will be used to
292 normal channel-writing functions, e.g. relay_write(), can still be
293 used - writes from any cpu will transparently end up in the global
294 buffer - but since it is a global buffer, callers should make sure
295 they use the proper locking for such a buffer, either by wrapping
300 user-defined data with a channel, and is immediately available
301 (including in create_buf_file()) via chan->private_data or
302 buf->chan->private_data.
304 Buffer-only channels
305 --------------------
310 cases, one may open a buffer-only channel and then call
314 Channel 'modes'
315 ---------------
317 relay channels can be used in either of two modes - 'overwrite' or
318 'no-overwrite'.  The mode is entirely determined by the implementation
320 subbuf_start() callback is defined is 'no-overwrite' mode.  If the
321 default mode suits your needs, and you plan to use the read()
322 interface to retrieve channel data, you can ignore the details of this
328 been consumed.  In no-overwrite mode, writes will fail, i.e. data will
329 be lost, if the number of unconsumed sub-buffers equals the total
330 number of sub-buffers in the channel.  It should be clear that if
331 there is no consumer or if the consumer can't consume sub-buffers fast
335 As explained above, a relay channel is made of up one or more
336 per-cpu channel buffers, each implemented as a circular buffer
337 subdivided into one or more sub-buffers.  Messages are written into
338 the current sub-buffer of the channel's current per-cpu buffer via the
340 the current sub-buffer, because there's no room left for it, the
342 new sub-buffer is about to occur.  The client uses this callback to 1)
343 initialize the next sub-buffer if appropriate 2) finalize the previous
344 sub-buffer if appropriate and 3) return a boolean value indicating
345 whether or not to actually move on to the next sub-buffer.
347 To implement 'no-overwrite' mode, the userspace client would provide
367 If the current buffer is full, i.e. all sub-buffers remain unconsumed,
370 current set of ready sub-buffers.  For the relay_buf_full() function
372 interface when sub-buffers have been consumed via
376 ready sub-buffers will relay_buf_full() return 0, in which case the
397 unconditionally.  It's also meaningless for the client to use the
403 implements the simplest possible 'no-overwrite' mode, i.e. it does
406 Header information can be reserved at the beginning of each sub-buffer
410 reserved in each sub-buffer to store the padding count for that
411 sub-buffer.  This is filled in for the previous sub-buffer in the
413 sub-buffer is passed into the subbuf_start() callback along with a
414 pointer to the previous sub-buffer, since the padding value isn't
415 known until a sub-buffer is filled.  The subbuf_start() callback is
416 also called for the first sub-buffer when the channel is opened, to
418 previous sub-buffer pointer passed into the callback will be NULL, so
420 writing into the previous sub-buffer.
422 Writing to a channel
423 --------------------
425 Kernel clients write data into the current cpu's channel buffer using
427 function - it uses local_irqsave() to protect the buffer and should be
429 you'll never be logging from interrupt context, you can use
432 failed - the assumption is that you wouldn't want to check a return
434 unless the buffer is full and no-overwrite mode is being used, in
438 relay_reserve() is used to reserve a slot in a channel buffer which
440 that need to write directly into a channel buffer without having to
444 written, either in space reserved in the sub-buffers themselves or as
445 a separate array.  See the 'reserve' example in the relay-apps tarball
449 at all - it's up to the client to provide the appropriate
452 Closing a channel
453 -----------------
455 The client calls relay_close() when it's finished using the channel.
456 The channel and its associated buffers are destroyed when there are no
457 longer any references to any of the channel buffers.  relay_flush()
458 forces a sub-buffer switch on all the channel buffers, and can be used
459 to finalize and process the last sub-buffers before the channel is
463 ----
465 Some applications may want to keep a channel around and re-use it
466 rather than open and close a new channel for each use.  relay_reset()
467 can be used for this purpose - it resets a channel to its initial
468 state without reallocating channel buffer memory or destroying
470 do so, i.e. when the channel isn't currently being written to.
473 different purposes.  buf_mapped() is called whenever a channel buffer
475 unmapped.  The client can use this notification to trigger actions
477 the channel.