<!--
{
  "availability" : [
    "iOS: 12.0.0 -",
    "iPadOS: 12.0.0 -",
    "macCatalyst: 12.0.0 -",
    "macOS: 10.14.0 -",
    "tvOS: 12.0.0 -",
    "visionOS: 1.0.0 -",
    "watchOS: 5.0.0 -"
  ],
  "documentType" : "symbol",
  "framework" : "NaturalLanguage",
  "identifier" : "/documentation/NaturalLanguage/NLTokenizer/tokens(for:)",
  "metadataVersion" : "0.1.0",
  "role" : "Instance Method",
  "symbol" : {
    "kind" : "Instance Method",
    "modules" : [
      "Natural Language",
      "NaturalLanguage"
    ],
    "preciseIdentifier" : "s:So11NLTokenizerC15NaturalLanguageE6tokens3forSaySnySS5IndexVGGAH_tF"
  },
  "title" : "tokens(for:)"
}
-->

# tokens(for:)

Tokenizes the string within the provided range.

```
@nonobjc func tokens(for range: Range<String.Index>) -> [Range<String.Index>]
```

## Parameters

`range`

The range within the string that should be tokenzied.

## Return Value

Returns the ranges corresponding to the tokens for the tokenizer’s unit that intersect the given range.

---

Copyright &copy; 2026 Apple Inc. All rights reserved. | [Terms of Use](https://www.apple.com/legal/internet-services/terms/site.html) | [Privacy Policy](https://www.apple.com/privacy/privacy-policy)