Instance Property

unit

The linguistic unit that this tokenizer uses.

Declaration

var unit: NLTokenUnit { get }

See Also

Configuring a Tokenizer

var string: String?

The text to be tokenized.

func setLanguage(NLLanguage)

Sets the language of the text to be tokenized.

struct NLTokenizer.Attributes

Hints about the contents of the string for the tokenizer.