Advances the tokenizer to the next token and sets that as the current token.


CFStringTokenizerTokenType CFStringTokenizerAdvanceToNextToken(CFStringTokenizerRef tokenizer);



A CFStringTokenizer object.

Return Value

The type of the token if the tokenizer succeeded in finding a token and setting it as current token. Returns kCFStringTokenizerTokenNone if the tokenizer failed to find a token. For possible values, see CFStringTokenizerTokenType.


If there is no preceding call to CFStringTokenizerGoToTokenAtIndex or CFStringTokenizerAdvanceToNextToken, the function finds the first token in the range specified by the CFStringTokenizerCreate. If there is a preceding, successful, call to CFStringTokenizerGoToTokenAtIndex or CFStringTokenizerAdvanceToNextToken and there is a current token, proceeds to the next token. If a token is found, it is set as the current token and the function returns true; otherwise the current token is invalidated and the function returns false.

You can obtain the range and attribute of the token calling CFStringTokenizerGetCurrentTokenRange and CFStringTokenizerCopyCurrentTokenAttribute. If the token is a compound (with type kCFStringTokenizerTokenHasSubTokensMask or kCFStringTokenizerTokenHasDerivedSubTokensMask), you can obtain its subtokens and (or) derived subtokens by calling CFStringTokenizerGetCurrentSubTokens.

See Also

Changing the Location


Finds a token that includes the character at a given index, and set it as the current token.