Structure

CFStringTokenizerTokenType

Declaration

struct CFStringTokenizerTokenType

Overview

See http://www.unicode.org/reports/tr29/#Word_Boundaries for a detailed description of word boundaries.

Topics

Constants

static var hasSubTokensMask: CFStringTokenizerTokenType

Compound token which may contain subtokens but with no derived subtokens.

static var hasDerivedSubTokensMask: CFStringTokenizerTokenType

Compound token which may contain derived subtokens.

static var hasNonLettersMask: CFStringTokenizerTokenType

Contains punctuation, symbols, and so on.

Relationships

Conforms To