Lines Matching +full:read +full:- +full:to +full:- +full:read
1 .. SPDX-License-Identifier: GPL-2.0
9 - Overview.
10 - Per-inode context.
11 - Inode context helper functions.
12 - Buffered read helpers.
13 - Read helper functions.
14 - Read helper structures.
15 - Read helper operations.
16 - Read helper procedure.
17 - Read helper cache API.
23 The network filesystem helper library is a set of functions designed to aid a
25 just includes turning various VM buffered read operations into requests to read
33 Per-Inode Context
36 The network filesystem helper library needs a place to store a bit of state for
37 its use on each netfs inode it is helping to manage. To this end, a context
46 A network filesystem that wants to use netfs lib must place one of these in its
48 a way similar to the following::
55 This allows netfslib to find its state by using ``container_of()`` from the
56 inode pointer, thereby allowing the netfslib helper functions to be pointed to
67 The set of operations provided by the network filesystem to netfslib.
76 ------------------------------
78 To help deal with the per-inode context, a number helper functions are
79 provided. Firstly, a function to perform basic initialisation on a context and
85 then a function to cast from the VFS inode structure to the netfs context::
89 and finally, a function to get the cache cookie pointer from the context
90 attached to an inode (or NULL if fscache is disabled)::
95 Buffered Read Helpers
98 The library provides a set of read helpers that handle the ->read_folio(),
99 ->readahead() and much of the ->write_begin() VM operations and translate them
108 * Allow the netfs to arbitrarily split reads up into pieces, even ones that
111 * Allow the netfs to expand a readahead request in both directions to meet its
114 * Allow the netfs to partially fulfil a read, which will then be resubmitted.
116 * Handle local caching, allowing cached data and server-read data to be
121 * Handle retrying of reads that failed, switching reads from the cache to the
125 local encryption of data to be stored remotely or in the cache.
128 includes a mandatory method to issue a read operation along with a number of
132 Read Helper Functions
133 ---------------------
135 Three read helpers are provided::
148 Each corresponds to a VM address space operation. These operations use the
149 state in the per-inode context.
151 For ->readahead() and ->read_folio(), the network filesystem just point directly
152 at the corresponding read helper; whereas for ->write_begin(), it may be a
153 little more complicated as the network filesystem might want to flush
154 conflicting writes or track dirty data and needs to put the acquired folio if
157 The helpers manage the read request, calling back into the network filesystem
159 necessary before returning for helpers that are meant to be synchronous.
161 If an error occurs, the ->free_request() will be called to clean up the
164 sufficient data is read.
172 which should be called to complete a read subrequest. This is given the number
174 the operation was asynchronous (ie. whether the follow-on processing can be
178 Read Helper Structures
179 ----------------------
181 The read helpers make use of a couple of structures to maintain the state of
182 the read. The first is a structure that manages a read request as a whole::
202 The inode and the address space of the file being read from. The mapping
203 may or may not point to inode->i_data.
207 Resources for the local cache to use, if present.
212 to the helper functions or set during the request.
217 The file position of the start of the read request and the length. These
218 may be altered by the ->expand_readahead() op.
226 A pointer to the operation table. The value for this is passed into the
231 A number allocated to this operation that can be displayed in trace lines
235 The second structure is used to manage individual slices of the overall read
248 Each subrequest is expected to access a single source, though the helpers will
249 handle falling back from one source type to another. The members are:
253 A pointer to the read request.
258 The file position of the start of this slice of the read request and the
265 slice. If a short read occurs, the helpers will call again, having updated
266 this to reflect the amount read so far.
270 Flags pertaining to the read. There are two of interest to the filesystem
275 This can be set to indicate that the remainder of the slice, from
276 transferred to len, should be cleared.
280 This is a hint to the cache that it might want to try skipping ahead to
285 A number allocated to this slice that can be displayed in trace lines for
289 Read Helper Operations
290 ----------------------
292 The network filesystem must provide the read helpers with a table of operations
311 [Optional] This is called to initialise the request structure. It is given
321 [Optional] This is called to allow the filesystem to expand the size of a
322 readahead read request. The filesystem gets to expand the request in both
323 directions, though it's not permitted to reduce it as the numbers may
325 to expand the request first.
327 Expansion is communicated by changing ->start and ->len in the request
328 structure. Note that if any change is made, ->len must be increased by at
329 least as much as ->start is reduced.
333 [Optional] This is called to allow the filesystem to reduce the size of a
334 subrequest. The filesystem can use this, for example, to chop up a request
335 that has to be split across multiple servers or to put multiple reads in
342 [Required] The helpers use this to dispatch a subrequest to the server for
343 reading. In the subrequest, ->start, ->len and ->transferred indicate what
344 data should be read from the server.
347 called to indicate whether or not the operation succeeded and how much data
349 uptodate, unlocking them or dropping their refs - the helpers need to deal
350 with this as they have to coordinate with copying to the local cache.
353 possible to use the ITER_XARRAY iov iterator to refer to the range of the
354 inode that is being operated upon without the need to allocate large bvec
359 [Optional] This is called to find out if the data just read from the local
366 allocated/grabbed the folio to be modified to allow the filesystem to flush
367 conflicting state before allowing it to be modified.
370 pointer to NULL. It should return 0 if everything is now fine (``*foliop``
372 error code to abort the operation.
381 Read Helper Procedure
382 ---------------------
384 The read helpers work by the following general procedure:
388 * For readahead, allow the local cache and then the network filesystem to
389 propose expansions to the read request. This is then proposed to the VM.
390 If the VM cannot fully perform the expansion, a partially expanded read will
391 be performed, though this may not get written to the cache in its entirety.
393 * Loop around slicing chunks off of the request to form subrequests:
395 * If a local cache is present, it gets to do the slicing, otherwise the
396 helpers just try to generate maximal slices.
398 * The network filesystem gets to clamp the size of each slice if it is to be
399 the source. This allows rsize and chunking to be implemented.
401 * The helpers issue a read from the cache or a read from the server or just
406 * As slices finish being read, they terminate.
418 * The cache may need to skip holes that it can't do DIO from.
420 * If NETFS_SREQ_CLEAR_TAIL was set, a short read will be cleared to the
423 * Once the data is read, the folios that have been fully read/cleared:
431 * Any folios that need writing to the cache will then have DIO writes issued.
433 * Synchronous operations will wait for reading to be complete.
435 * Writes to the cache will proceed asynchronously and the folios will have the
441 Read Helper Cache API
442 ---------------------
444 When implementing a local cache to be used by the read helpers, two things are
445 required: some way for the network filesystem to initialise the caching for a
446 read request and a table of operations for the helpers to call.
448 To begin a cache operation on an fscache object, the following function is
454 passing in the request pointer and the cookie corresponding to the file. This
457 The netfs_io_request object contains a place for the cache to hang its
478 int (*read)(struct netfs_cache_resources *cres,
510 [Required] Called to clean up the resources at the end of the read request.
514 [Optional] Called at the beginning of a netfs_readahead() operation to allow
515 the cache to expand a request in either direction. This allows the cache to
518 The function is passed poiners to the start and length in its parameters,
527 to indicate whether the slice should just be cleared or whether it should be
528 downloaded from the server or read from the cache - or whether slicing
533 [Required] Called to configure the next slice of a request. ->start and
534 ->len in the subrequest indicate where and how big the next slice can be;
535 the cache gets to reduce the length to match its granularity requirements.
537 * ``read()``
539 [Required] Called to read from the cache. The start file offset is given
540 along with an iterator to read to, which gives the length also. It can be
544 Also provided is a pointer to a termination handler function and private
545 data to pass to that function. The termination function should be called
552 [Required] Called to prepare a write to the cache to take place. This
553 involves checking to see whether the cache has sufficient space to honour
554 the write. ``*_start`` and ``*_len`` indicate the region to be written; the
555 region can be shrunk or it can be expanded to a page boundary either way as
556 necessary to align for direct I/O. i_size holds the size of the object and
557 is provided for reference. no_space_allocated_yet is set to true if the
558 caller is certain that no data has been written to that region - for example
559 if it tried to do a read from there already.
563 [Required] Called to write to the cache. The start file offset is given
564 along with an iterator to write from, which gives the length also.
566 Also provided is a pointer to a termination handler function and private
567 data to pass to that function. The termination function should be called
574 [Required] Called to find out where the next piece of data is within a
575 particular region of the cache. The start and length of the region to be
576 queried are passed in, along with the granularity to which the answer needs
577 to be aligned. The function passes back the start and length of the data,
581 It returns 0 if some data was found, -ENODATA if there was no usable data
582 within the region or -ENOBUFS if there is no caching on this file.
584 Note that these methods are passed a pointer to the cache resource structure,
585 not the read request structure as they could be used in other situations where
586 there isn't a read request structure as well, such as writing dirty data to the
593 .. kernel-doc:: include/linux/netfs.h
594 .. kernel-doc:: fs/netfs/buffered_read.c