Enumeration

CFStringTokenizerTokenType

Declaration

typedef enum CFStringTokenizerTokenType : CFOptionFlags {
    ...
} CFStringTokenizerTokenType;

Overview

See http://www.unicode.org/reports/tr29/#Word_Boundaries for a detailed description of word boundaries.

Topics

Constants

kCFStringTokenizerTokenHasSubTokensMask

Compound token which may contain subtokens but with no derived subtokens.

kCFStringTokenizerTokenHasDerivedSubTokensMask

Compound token which may contain derived subtokens.

kCFStringTokenizerTokenHasHasNumbersMask

Appears to contain a number.

kCFStringTokenizerTokenHasNonLettersMask

Contains punctuation, symbols, and so on.

kCFStringTokenizerTokenIsCJWordMask

Contains kana and/or ideographs.

Beta Software

This documentation contains preliminary information about an API or technology in development. This information is subject to change, and software implemented according to this documentation should be tested with final operating system software.

Learn more about using Apple's beta software