fs/unicode/utf8-norm.c

24  * The UTF-8 encoding spreads the bits of a 32bit word over several
29  * 0x00000000 0x000007FF: 110xxxxx 10xxxxxx
30  * 0x00000000 0x0000FFFF: 1110xxxx 10xxxxxx 10xxxxxx
31  * 0x00000000 0x001FFFFF: 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx
32  * 0x00000000 0x03FFFFFF: 111110xx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx
33  * 0x00000000 0x7FFFFFFF: 1111110x 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx
36  * shortest representation of a 32bit value is to be used.  A decoder
38  * Thus the allowed ranges have a lower bound.
41  * 0x00000080 0x000007FF: 110xxxxx 10xxxxxx
42  * 0x00000800 0x0000FFFF: 1110xxxx 10xxxxxx 10xxxxxx
43  * 0x00010000 0x001FFFFF: 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx
44  * 0x00200000 0x03FFFFFF: 111110xx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx
45  * 0x04000000 0x7FFFFFFF: 1111110x 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx
59  * the same a single UTF-32 character.  This makes the UTF-8
71  * Assumes the input points to the first byte of a valid UTF-8
82  * Decode a 3-byte UTF-8 sequence.
99  * Encode a 3-byte UTF-8 sequence.
116  * A compact binary tree, used to decode UTF-8 characters.
128  *                            node, otherwise it is a leaf node
134  * NEXTBYTE set, and moreover those nodes always have a righthand
153  * leaf[0]: The unicode version, stored as a generation number that is
156  *          defined.  The CCC of a non-defined code point is 0.
158  *          to do a stable sort into ascending order of all characters
159  *          with a non-zero CCC that occur between two characters with
160  *          a CCC of 0, or at the begin or end of a string.
163  *          a special value.
166  *          start of a NUL-terminated string that is the decomposition
168  *          The CCC of a decomposable character is the same as the CCC
179  * The trie is constructed in such a way that leaves exist for all
181  * ranges" comment above, and only for those sequences.  Therefore a
278 	/* Add LPart, a 3-byte UTF-8 sequence. */  in utf8hangul()
281 	/* Add VPart, a 3-byte UTF-8 sequence. */  in utf8hangul()
284 	/* Add TPart if required, also a 3-byte UTF-8 sequence. */  in utf8hangul()
298  * A non-NULL return guarantees that the UTF-8 sequence starting at s
299  * is well-formed and corresponds to a known unicode code point.  The
450  * When a character is decomposed, the current location is stored in
452  * that bytes from a decomposition do not count against u8c->len.
467  *  u8c->p  != NULL -> a decomposition is being scanned.
468  *  u8c->ss != NULL -> this is a repeating scan.
469  *  u8c->ccc == -1   -> this is the first scan of a repeating scan.
477 		/* Check for the end of a decomposed character. */  in utf8byte()
488 			/* End-of-string during a scan counts as a stopper. */  in utf8byte()
492 			/* This is a continuation of the current character. */  in utf8byte()
506 		/* No leaf found implies that the input is a binary blob. */  in utf8byte()
534 		 * If this is not a stopper, then see if it updates  in utf8byte()
567 			/* Not a stopper, and not the ccc we're emitting. */  in utf8byte()
572 			/* At a stopper, restart for next ccc. */  in utf8byte()