filesystems/xfs/xfs-delayed-logging-design.rst

15 We begin with an overview of transactions in XFS, followed by describing how
17 guarantee forwards progress for long running transactions with finite initial
54 Transactions in XFS
57 XFS has two types of high level transactions, defined by the type of log space
59 transactions. Permanent transaction reservations can take reservations that span
60 commit boundaries, whilst "one shot" transactions are for a single atomic
64 place.  This means that permanent transactions can be used for one-shot
66 transactions.
82 transactions, and the pattern looks like this::
99 transactions together::
105 This results in a series of "rolling transactions" where the inode is locked
106 across the entire chain of transactions.  Hence while this series of rolling
107 transactions is running, nothing else can read from or write to the inode and
111 It is important to note that a series of rolling transactions in a permanent
117 This affects long running permanent transactions in that it is not possible to
120 if a long running operation requires multiple transactions to fully complete,
122 recovery can complete the operation once the first transactions is persisted in
126 Transactions are Asynchronous
129 In XFS, all high level transactions are asynchronous by default. This means that
132 completed transactions will be replayed during recovery.
141 a "log force" to flush the outstanding committed transactions to stable storage
144 Synchronous transactions are rarely used, however, because they limit logging
157 a transaction is first allocated. For permanent transactions, these reservations
183 For one-shot transactions, a single unit space reservation is all that is
184 required for the transaction to proceed. For permanent transactions, however, we
191 know from the implementation of the permanent transactions how many transaction
194 For example, an inode allocation is typically two transactions - one to
198 that the common/fast path transaction will commit two linked transactions in a
233 reservations currently held by active transactions. It is a purely in-memory
242 transactions from taking new reservations when the head reaches the current
253 forwards progress guarantees that rolling permanent transactions require.
277 As a result, permanent transactions only "regrant" reservation space during
296 If all rolling transactions obey this rule, then they can all make forwards
299 (eventually) made available to permanent transactions no matter how many times
315 of transactions, their contents and the log sequence number (LSN) of the
334 implement long-running, multiple-commit permanent transactions.
352 It should now also be obvious how relogging and asynchronous transactions go
353 hand in hand. That is, transactions don't get written to the physical journal
355 transactions) or a synchronous operation forces the log buffers holding the
356 transactions to disk. This means that XFS is doing aggregation of transactions
367 buffers are full and under IO, then no more transactions can be committed until
369 be to able to issue enough transactions to keep the log buffers full and under
380 transactions A through D are committed to disk in the same log buffer.
418 recovered filesystem is concerned, there may be many thousands of transactions
455 This introduces lots of scope for deadlocks with transactions that are already
480 asynchronous transactions to the log. The differences between the existing
579 no later transactions should be replayed, either.
615 context and attach that to the CIL for aggregation of new transactions.
618 committed items and effectively allows new transactions to be issued while we
678 Once this transfer is done, the CIL can be unlocked and new transactions can
706 committed transactions with the log sequence number of the transaction commit.
707 This allows transactions to be issued asynchronously even though there may be
713 To do this, transactions need to record the LSN of the commit record of the
716 mechanism, it does not work for delayed logging because transactions are not
718 transactions is required.
730 operations that track transactions that have not yet completed know what
745 aggregation of multiple synchronous transactions if there are already
746 synchronous transactions being flushed. Investigation of the performance of the
762 transactions to remain untouched (i.e. commit an asynchronous transaction, then
786 there are lots of transactions that only contain an inode core and an inode log
807 large enough to handle arbitrary sized checkpoint transactions. This
820 rolling transactions for an example of this).  Hence we *must* always have
830 the change. Transactions at this level will *always* have enough space
840 As mentioned early, transactions can't grow to more than half the size of the
867 transactions is completed, they will unpin the item once. As a result, the item
868 only becomes unpinned when all the transactions complete and there are no
869 pending transactions. Thus the pinning and unpinning of a log item is symmetric
904 code does not break down even when there are transactions coming from 2048
918 the number of concurrent transactions that can be trying to commit at once is
920 limit here is in the order of several hundred concurrent transactions for a
947 are inserted into the CIL. Because transactions can enter this code