Returns a tokenizer for a given string.


CFStringTokenizerRef CFStringTokenizerCreate(CFAllocatorRef alloc, CFStringRef string, CFRange range, CFOptionFlags options, CFLocaleRef locale);



The allocator to use to allocate memory for the new object. Pass NULL or kCFAllocatorDefault to use the current default allocator.


The string to tokenize.


The range of the characters in string to tokenize.


A tokenization unit option that specifies how string should be tokenized. The options can be modified by adding unit modifier options to tell the tokenizer to prepare specified attributes when it tokenizes string.

For possible values, see Tokenization Modifiers.


A locale that specifies language- or region-specific behavior for the tokenization. You can pass NULL to use the default system locale, although this is typically not recommended—instead use CFLocaleCopyCurrent to specify the locale of the current user.

For more information, see Tokenization Modifiers.

Return Value

A tokenizer to analyze the range range of string for the given locale and options. Ownership follows the The Create Rule.

Beta Software

This documentation contains preliminary information about an API or technology in development. This information is subject to change, and software implemented according to this documentation should be tested with final operating system software.

Learn more about using Apple's beta software