{"abstract":[{"text":"Optimize your app’s token usage when prompting a model with the Foundation Models","type":"text"},{"text":" ","type":"text"},{"text":"framework.","type":"text"}],"hierarchy":{"paths":[["doc:\/\/com.apple.documentation\/documentation\/technologies","doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels"]]},"identifier":{"interfaceLanguage":"swift","url":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/managing-the-context-window"},"kind":"article","legalNotices":{"copyright":"Copyright &copy; 2026 Apple Inc. All rights reserved.","privacyPolicy":"https:\/\/www.apple.com\/privacy\/privacy-policy","termsOfUse":"https:\/\/www.apple.com\/legal\/internet-services\/terms\/site.html"},"metadata":{"availableLanguages":["de-DE","en-US","es-419","fr-FR","it-IT","ja-JP","ko-KR","pt-BR","zh-CN"],"modules":[{"name":"Foundation Models"}],"role":"article","roleHeading":"Article","title":"Managing the context window"},"primaryContentSections":[{"content":[{"anchor":"Overview","level":2,"text":"Overview","type":"heading"},{"inlineContent":[{"text":"When working with a model — like the default on-device foundation model —","type":"text"},{"text":" ","type":"text"},{"text":"you have a fixed ","type":"text"},{"inlineContent":[{"text":"context window","type":"text"}],"type":"emphasis"},{"text":" that limits the number of tokens the model can","type":"text"},{"text":" ","type":"text"},{"text":"process in a single ","type":"text"},{"inlineContent":[{"text":"session","type":"text"}],"type":"emphasis"},{"text":". Apple’s on-device foundation model has a context","type":"text"},{"text":" ","type":"text"},{"text":"window of 4096 tokens per session, with a token representing each word, or","type":"text"},{"text":" ","type":"text"},{"text":"partial word.","type":"text"}],"type":"paragraph"},{"inlineContent":[{"text":"The size of the context window matters, because every session interaction with","type":"text"},{"text":" ","type":"text"},{"text":"the model consumes input and output tokens from this window. This includes all","type":"text"},{"text":" ","type":"text"},{"text":"prompts, instructions, tool definitions and their input and output, generable","type":"text"},{"text":" ","type":"text"},{"text":"type schemas, and all of the model’s responses.","type":"text"}],"type":"paragraph"},{"inlineContent":[{"text":"Understanding and managing the context window is essential for building reliable","type":"text"},{"text":" ","type":"text"},{"text":"AI features in your app. This article covers several strategies for staying","type":"text"},{"text":" ","type":"text"},{"text":"within a token budget, along with how to recover from exceeding the budget.","type":"text"}],"type":"paragraph"},{"anchor":"Understand-tokens-and-the-context-window","level":2,"text":"Understand tokens and the context window","type":"heading"},{"inlineContent":[{"text":"In Latin alphabet languages such as English, a token typically represents three to","type":"text"},{"text":" ","type":"text"},{"text":"four characters. For multibyte languages such as Chinese, Japanese, Korean, and","type":"text"},{"text":" ","type":"text"},{"text":"Vietnamese a token typically represents one character.","type":"text"}],"type":"paragraph"},{"inlineContent":[{"text":"The context window defines the maximum number of tokens the model can process in","type":"text"},{"text":" ","type":"text"},{"text":"a ","type":"text"},{"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/LanguageModelSession","isActive":true,"type":"reference"},{"text":" instance. When you prompt the model using the same","type":"text"},{"text":" ","type":"text"},{"text":"session, you consume more of the context window. Use","type":"text"},{"text":" ","type":"text"},{"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/SystemLanguageModel\/tokenCount(for:)","isActive":true,"type":"reference"},{"text":" to understand how many tokens your","type":"text"},{"text":" ","type":"text"},{"text":"prompt, instruction, or tool uses. You can also call ","type":"text"},{"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/SystemLanguageModel\/contextSize","isActive":true,"type":"reference"},{"text":" ","type":"text"},{"text":"to retrieve the maximum size (in tokens) that the model supports.","type":"text"}],"type":"paragraph"},{"inlineContent":[{"text":"The following diagram shows the context window filling over a number of turns of","type":"text"},{"text":" ","type":"text"},{"text":"a conversation. In a later turn, the context window is nearly full with the","type":"text"},{"text":" ","type":"text"},{"text":"accumulated content from the previous turns, and eventually appears with","type":"text"},{"text":" ","type":"text"},{"text":"a dashed red border to indicate that the session exceeded the content size:","type":"text"}],"type":"paragraph"},{"inlineContent":[{"identifier":"managing-the-context-window-context-window","type":"image"}],"type":"paragraph"},{"inlineContent":[{"text":"After reaching the context size, the session can no longer process additional","type":"text"},{"text":" ","type":"text"},{"text":"requests and throws an ","type":"text"},{"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/LanguageModelError\/contextSizeExceeded(_:)","isActive":true,"type":"reference"},{"text":" ","type":"text"},{"text":"error. When this happens, trim the session history or create a new session to","type":"text"},{"text":" ","type":"text"},{"text":"get a fresh context window. Use ","type":"text"},{"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/LanguageModelSession\/init(model:tools:transcript:)","isActive":true,"type":"reference"},{"text":"","type":"text"},{"text":" ","type":"text"},{"text":"and include any state from the original session that’s necessary for the request,","type":"text"},{"text":" ","type":"text"},{"text":"then retry:","type":"text"}],"type":"paragraph"},{"inlineContent":[{"identifier":"managing-the-context-window-new-context-window","type":"image"}],"type":"paragraph"},{"anchor":"Profile-token-usage-in-Xcode","level":2,"text":"Profile token usage in Xcode","type":"heading"},{"inlineContent":[{"text":"Before optimizing your token usage, it’s important to understand how your interaction","type":"text"},{"text":" ","type":"text"},{"text":"with the model impacts token growth. The Foundation Models instrument in Xcode","type":"text"},{"text":" ","type":"text"},{"text":"provides real-time visibility into token usage. To profile your app in Xcode:","type":"text"}],"type":"paragraph"},{"items":[{"content":[{"inlineContent":[{"text":"Choose Product > Profile to launch Instruments.","type":"text"}],"type":"paragraph"}]},{"content":[{"inlineContent":[{"text":"Select the Foundation Models template, then click Choose.","type":"text"}],"type":"paragraph"}]},{"content":[{"inlineContent":[{"text":"Click the Record button and interact with your app’s AI features.","type":"text"}],"type":"paragraph"}]},{"content":[{"inlineContent":[{"text":"Observe the token count as your app interacts with the model.","type":"text"}],"type":"paragraph"}]}],"type":"orderedList"},{"inlineContent":[{"identifier":"analyzing-the-runtime-performance-of-your-foundation-models-app-record-analysis","type":"image"}],"type":"paragraph"},{"inlineContent":[{"text":"The instrument shows you exactly how many tokens each interaction uses for your","type":"text"},{"text":" ","type":"text"},{"text":"app’s input and output tokens the model generates. It provides a detailed","type":"text"},{"text":" ","type":"text"},{"text":"breakdown of your session requests, prompts, responses, and tool calls to help","type":"text"},{"text":" ","type":"text"},{"text":"you identify opportunities for optimization, like:","type":"text"}],"type":"paragraph"},{"items":[{"content":[{"inlineContent":[{"text":"Reducing the amount of tokens necessary for the prompts and instructions.","type":"text"}],"type":"paragraph"}]},{"content":[{"inlineContent":[{"text":"Limiting model responses to what your app needs.","type":"text"}],"type":"paragraph"}]},{"content":[{"inlineContent":[{"text":"Managing the amount of context your generable types and tools use.","type":"text"}],"type":"paragraph"}]},{"content":[{"inlineContent":[{"text":"Identifying areas of opportunity to break a task into separate sessions with","type":"text"},{"text":" ","type":"text"},{"text":"a fresh context window.","type":"text"}],"type":"paragraph"}]}],"type":"unorderedList"},{"inlineContent":[{"text":"For more information about profiling, see ","type":"text"},{"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/analyzing-the-runtime-performance-of-your-foundation-models-app","isActive":true,"type":"reference"},{"text":".","type":"text"}],"type":"paragraph"},{"anchor":"Write-concise-prompts-and-instructions","level":2,"text":"Write concise prompts and instructions","type":"heading"},{"inlineContent":[{"text":"Prompts and instructions often consume the most tokens, especially in multiturn","type":"text"},{"text":" ","type":"text"},{"text":"conversations. Shorter, clearer prompts not only save tokens but they often","type":"text"},{"text":" ","type":"text"},{"text":"produce better results. Focus on clarity and brevity, along with:","type":"text"}],"type":"paragraph"},{"items":[{"content":[{"inlineContent":[{"text":"Use imperative verbs that clearly state what you want the model to do:","type":"text"},{"text":" ","type":"text"},{"text":"“Generate a story about…,” or “List five reasons why…”.","type":"text"}],"type":"paragraph"}]},{"content":[{"inlineContent":[{"text":"Provide only the information the model needs for the specific task.","type":"text"}],"type":"paragraph"}]},{"content":[{"inlineContent":[{"text":"Avoid lengthy background information, policies, or unnecessary context.","type":"text"}],"type":"paragraph"}]},{"content":[{"inlineContent":[{"text":"Reduce prompts to no more than three paragraphs in length.","type":"text"}],"type":"paragraph"}]},{"content":[{"inlineContent":[{"text":"Eliminate indirect language, excessive formality, and ambiguous jargon.","type":"text"}],"type":"paragraph"}]}],"type":"unorderedList"},{"inlineContent":[{"text":"For more information on prompting a model, see ","type":"text"},{"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/prompting-an-on-device-foundation-model","isActive":true,"type":"reference"},{"text":".","type":"text"}],"type":"paragraph"},{"anchor":"Ask-the-model-for-less-content","level":2,"text":"Ask the model for less content","type":"heading"},{"inlineContent":[{"text":"If you notice the model produces long, detailed responses, try limiting how many","type":"text"},{"text":" ","type":"text"},{"text":"tokens the model uses in its responses:","type":"text"}],"type":"paragraph"},{"items":[{"content":[{"inlineContent":[{"text":"Specify the desired response length in your prompt, by starting the prompt with","type":"text"},{"text":" ","type":"text"},{"text":"text like “In three sentences…,” or “List three reasons…”.","type":"text"}],"type":"paragraph"}]},{"content":[{"inlineContent":[{"text":"For ","type":"text"},{"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/Generable","isActive":true,"type":"reference"},{"text":" arrays, add an ","type":"text"},{"code":"@Guide","type":"codeVoice"},{"text":" macro that specifies a ","type":"text"},{"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/GenerationGuide\/maximumCount(_:)","isActive":true,"type":"reference"},{"text":"","type":"text"},{"text":" ","type":"text"},{"text":"to limit the number of items the model produces.","type":"text"}],"type":"paragraph"}]}],"type":"unorderedList"},{"code":["@Generable ","struct GameSettings {","    @Guide(.minimumCount(1), .maximumCount(20))","    @Guide(description: \"Keyboard shortcuts for desktop\")","    var keyboardShortcuts: [String]","}"],"copyToClipboard":true,"lineAnnotations":[],"showLineNumbers":false,"syntax":"swift","type":"codeListing","wrap":0},{"content":[{"inlineContent":[{"text":"Only use ","type":"text"},{"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/GenerationOptions\/maximumResponseTokens","isActive":true,"type":"reference"},{"text":" to","type":"text"},{"text":" ","type":"text"},{"text":"prevent verbose responses. Limiting tokens can cause the model to generate","type":"text"},{"text":" ","type":"text"},{"text":"incomplete or grammatically incorrect responses, like “A cat is a small.”","type":"text"}],"type":"paragraph"}],"name":"Important","style":"important","type":"aside"},{"anchor":"Simplify-generable-types","level":2,"text":"Simplify generable types","type":"heading"},{"inlineContent":[{"text":"Generable types consume tokens in multiple ways. The framework converts each","type":"text"},{"text":" ","type":"text"},{"text":"type’s structure into a JSON schema and sends that to the model.","type":"text"}],"type":"paragraph"},{"inlineContent":[{"text":"Any ","type":"text"},{"code":"@Guide","type":"codeVoice"},{"text":" descriptions you add also consume tokens because they become part","type":"text"},{"text":" ","type":"text"},{"text":"of the schema that’s sent to the model. Similar to the instructions you provide","type":"text"},{"text":" ","type":"text"},{"text":"the model, these descriptions help the model understand more about your custom","type":"text"},{"text":" ","type":"text"},{"text":"type but uses context to do so. To minimize token usage:","type":"text"}],"type":"paragraph"},{"items":[{"content":[{"inlineContent":[{"text":"Keep your types simple and focused — complex type definitions consume more","type":"text"},{"text":" ","type":"text"},{"text":"tokens.","type":"text"}],"type":"paragraph"}]},{"content":[{"inlineContent":[{"text":"Use short, descriptive property names.","type":"text"}],"type":"paragraph"}]},{"content":[{"inlineContent":[{"text":"Use ","type":"text"},{"code":"@Guide","type":"codeVoice"},{"text":" only where necessary. If your property names are clear, the model","type":"text"},{"text":" ","type":"text"},{"text":"may not need additional guidance.","type":"text"}],"type":"paragraph"}]}],"type":"unorderedList"},{"inlineContent":[{"text":"The following example shows a custom type with only the properties necessary for","type":"text"},{"text":" ","type":"text"},{"text":"the task and clear names that the model can infer:","type":"text"}],"type":"paragraph"},{"code":["@Generable","struct ContentAnalysis {","    \/\/ Clear property names that the model can infer won't need an `@Guide`.","    var title: String","    var summary: String","            ","    \/\/ A minimum and maximum to a rating provides details to the model that it","    \/\/ otherwise wouldn't know. There is no need for a description because the ","    \/\/ property name is clear.","    @Guide(.minimum(1), .maximum(10))","    var rating: Int","}"],"copyToClipboard":true,"lineAnnotations":[],"showLineNumbers":false,"syntax":"swift","type":"codeListing","wrap":0},{"inlineContent":[{"text":"Test Generable types without an ","type":"text"},{"code":"@Guide","type":"codeVoice"},{"text":" macro first, then add guide annotations","type":"text"},{"text":" ","type":"text"},{"text":"to the properties that need more clarity to help the model improve the response","type":"text"},{"text":" ","type":"text"},{"text":"quality. If you have an unclear property name, it won’t convey the right intent","type":"text"},{"text":" ","type":"text"},{"text":"to the model. Instead, consider renaming the property so it’s easier to interpret,","type":"text"},{"text":" ","type":"text"},{"text":"or apply an ","type":"text"},{"code":"@Guide","type":"codeVoice"},{"text":" macro to provide a description that the model uses when","type":"text"},{"text":" ","type":"text"},{"text":"generating a response.","type":"text"}],"type":"paragraph"},{"anchor":"Use-tool-calling-efficiently","level":2,"text":"Use tool calling efficiently","type":"heading"},{"inlineContent":[{"text":"Like ","type":"text"},{"code":"@Generable","type":"codeVoice"},{"text":" types, each tool call and its output consume tokens. The","type":"text"},{"text":" ","type":"text"},{"text":"framework sends tool definitions — including names, descriptions, and","type":"text"},{"text":" ","type":"text"},{"text":"parameters — to the model so it can decide when to call them. To use tools","type":"text"},{"text":" ","type":"text"},{"text":"efficiently:","type":"text"}],"type":"paragraph"},{"items":[{"content":[{"inlineContent":[{"text":"Limit tool descriptions and ","type":"text"},{"code":"@Guide","type":"codeVoice"},{"text":" annotations to short phrases.","type":"text"}],"type":"paragraph"}]},{"content":[{"inlineContent":[{"text":"Provide no more than three to five tools per request.","type":"text"}],"type":"paragraph"}]},{"content":[{"inlineContent":[{"text":"Skip tool calling when you don’t need the model to make decisions. If the","type":"text"},{"text":" ","type":"text"},{"text":"model always needs specific information, retrieve it directly and include it","type":"text"},{"text":" ","type":"text"},{"text":"in your prompt rather than relying on tool calling.","type":"text"}],"type":"paragraph"}]}],"type":"unorderedList"},{"inlineContent":[{"text":"The following shows a tool with a clear description and provides the model with","type":"text"},{"text":" ","type":"text"},{"text":"additional context about the parameters it fills in when performing a tool call:","type":"text"}],"type":"paragraph"},{"code":["@Observable","final class FindPointsOfInterestTool: Tool {","    let name = \"findPointsOfInterest\"","    let description = \"Finds points of interest for a landmark.\"","    ","    @Generable","    enum Category: String, CaseIterable {","        case campground","        case hotel","        case cafe","        case museum","        case marina","        case restaurant","        case nationalMonument","    }","","    @Generable","    struct Arguments {","        @Guide(description: \"The type of destination to look up.\")","        let pointOfInterest: Category","","        @Guide(description: \"The natural language query of what to search for.\")","        let naturalLanguageQuery: String","    }","    ","    func call(arguments: Arguments) async throws -> String {","        \/\/ Implement the logic your app needs when the model calls this tool.","    }","}"],"copyToClipboard":true,"lineAnnotations":[],"showLineNumbers":false,"syntax":"swift","type":"codeListing","wrap":0},{"inlineContent":[{"text":"For complex workflows that approach the context window limit, split tool usage","type":"text"},{"text":" ","type":"text"},{"text":"across multiple sessions. When you need the model to generate tool arguments,","type":"text"},{"text":" ","type":"text"},{"text":"consider asking the model to generate those in one session, execute the tool in","type":"text"},{"text":" ","type":"text"},{"text":"your code, then have the model process the tool’s output in a new session.","type":"text"}],"type":"paragraph"},{"anchor":"Split-large-tasks-across-multiple-sessions","level":2,"text":"Split large tasks across multiple sessions","type":"heading"},{"inlineContent":[{"text":"When a task requires more than the size of the context window, divide it into","type":"text"},{"text":" ","type":"text"},{"text":"smaller steps that each run in a new language model session. For example, to","type":"text"},{"text":" ","type":"text"},{"text":"summarize a long article:","type":"text"}],"type":"paragraph"},{"items":[{"content":[{"inlineContent":[{"text":"Split the article into chunks that fit within the context window.","type":"text"}],"type":"paragraph"}]},{"content":[{"inlineContent":[{"text":"Summarize each chunk in a separate session.","type":"text"}],"type":"paragraph"}]},{"content":[{"inlineContent":[{"text":"Combine the chunk summaries.","type":"text"}],"type":"paragraph"}]},{"content":[{"inlineContent":[{"text":"If needed, summarize the combined summaries in another session.","type":"text"}],"type":"paragraph"}]},{"content":[{"inlineContent":[{"text":"Repeat until you reach your target length.","type":"text"}],"type":"paragraph"}]}],"type":"orderedList"},{"inlineContent":[{"text":"To maintain context between chunks, include the previous chunk’s summary in the","type":"text"},{"text":" ","type":"text"},{"text":"next chunk’s prompt:","type":"text"}],"type":"paragraph"},{"code":["let chunks: [String] = \/\/ Split a long article into separate chunks.","var chunkSummaries: [String] = []","","\/\/ Use a separate session to summarize each chunk.","for (index, chunk) in chunks.enumerated() {","    ","    let session = LanguageModelSession()","    var prompt = \"\"\"","        Summarize this section of an article:","        ","        \\(chunk)","        \"\"\"","","    \/\/ Include the previous summary to maintain continuity.","    if index > 0 {","        prompt = \"\"\"","        Previous section summary: \\(chunkSummaries[index - 1])","","        \\(prompt)","        \"\"\"","    }","","    \/\/ Perform the request and track the response.","    let response = try await session.respond(to: prompt)","    chunkSummaries.append(response.content)","}","","\/\/ Combine all chunk summaries in a final session.","let finalSession = LanguageModelSession()","let combined = chunkSummaries.joined(separator: \"\\n\")","","let prompt = \"\"\"","    Combine these section summaries into one cohesive summary:","","    \\(combined)","    \"\"\"","","\/\/ Perform the request to get the final summary.","let finalSummary = try await finalSession.respond(to: prompt).content"],"copyToClipboard":true,"lineAnnotations":[],"showLineNumbers":false,"syntax":"swift","type":"codeListing","wrap":0},{"anchor":"Handle-context-window-errors","level":2,"text":"Handle context window errors","type":"heading"},{"inlineContent":[{"text":"Even with careful optimization, you may exceed the context window limit. This","type":"text"},{"text":" ","type":"text"},{"text":"commonly occurs in open-ended conversations where people continue chatting until","type":"text"},{"text":" ","type":"text"},{"text":"they reach the token limit. When your app exceeds the limit, the framework throws","type":"text"},{"text":" ","type":"text"},{"text":"an ","type":"text"},{"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/LanguageModelError\/contextSizeExceeded(_:)","isActive":true,"type":"reference"},{"text":" error","type":"text"},{"text":" ","type":"text"},{"text":"and the session stops responding. When this occurs, create a new session and","type":"text"},{"text":" ","type":"text"},{"text":"evaluate your user experience so people understand what happened. You can catch","type":"text"},{"text":" ","type":"text"},{"text":"the error in your code, like the following example shows:","type":"text"}],"type":"paragraph"},{"code":["do {","    \/\/ Perform a request that exceeds the context window.","    let response = try await session.respond(to: prompt)","} catch LanguageModelError.contextSizeExceeded(let context) {","    \/\/ Handle exceeding the context window size by creating a new session.","} catch {","    \/\/ Handle other errors that are thrown.","}"],"copyToClipboard":true,"lineAnnotations":[],"showLineNumbers":false,"syntax":"swift","type":"codeListing","wrap":0},{"inlineContent":[{"text":"When you create a new session, you get a fresh context window that doesn’t retain","type":"text"},{"text":" ","type":"text"},{"text":"the state from the original session. To preserve context, either summarize the","type":"text"},{"text":" ","type":"text"},{"text":"original session’s transcript and use that summary to initialize the new session,","type":"text"},{"text":" ","type":"text"},{"text":"or extract key entries from the original transcript and use them to seed the new","type":"text"},{"text":" ","type":"text"},{"text":"session. The following example creates a new session using the first and last","type":"text"},{"text":" ","type":"text"},{"text":"entries from the original session:","type":"text"}],"type":"paragraph"},{"code":["func newContextualSession(with originalSession: LanguageModelSession) -> LanguageModelSession {","    let allEntries = originalSession.transcript","    let condensedEntries = [allEntries.first, allEntries.last].compactMap { $0 }","    let condensedTranscript = Transcript(entries: condensedEntries)","    let newSession = LanguageModelSession(transcript: condensedTranscript)","    newSession.prewarm()","    return newSession","}"],"copyToClipboard":true,"lineAnnotations":[],"showLineNumbers":false,"syntax":"swift","type":"codeListing","wrap":0},{"inlineContent":[{"text":"The first transcript entry often contains important instructions and the last","type":"text"},{"text":" ","type":"text"},{"text":"entry contains the most recent context. By preserving the first and last entry, you","type":"text"},{"text":" ","type":"text"},{"text":"maintain continuity while dramatically reducing token usage.","type":"text"}],"type":"paragraph"}],"kind":"content"}],"schemaVersion":{"major":0,"minor":3,"patch":0},"sections":[],"seeAlsoSections":[{"anchor":"Sessions-and-prompts","generated":true,"identifiers":["doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/prompting-an-on-device-foundation-model","doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/updating-prompts-for-new-model-versions","doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/LanguageModelSession","doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/Instructions","doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/Prompt","doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/GenerationOptions","doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/ContextOptions"],"title":"Sessions and prompts"}],"variants":[{"paths":["\/documentation\/foundationmodels\/managing-the-context-window"],"traits":[{"interfaceLanguage":"swift"}]}],"references":{"analyzing-the-runtime-performance-of-your-foundation-models-app-record-analysis":{"alt":"A screenshot of Instruments displaying the Foundation Models instrument track","identifier":"analyzing-the-runtime-performance-of-your-foundation-models-app-record-analysis","type":"image","variants":[{"traits":["2x","light"],"url":"\/images\/com.apple.foundationmodels\/analyzing-the-runtime-performance-of-your-foundation-models-app-record-analysis@2x.png"},{"traits":["2x","dark"],"url":"\/images\/com.apple.foundationmodels\/analyzing-the-runtime-performance-of-your-foundation-models-app-record-analysis~dark@2x.png"}]},"doc://com.apple.documentation/documentation/technologies":{"abstract":[],"identifier":"doc:\/\/com.apple.documentation\/documentation\/technologies","kind":"technologies","role":"overview","title":"Technologies","type":"topic","url":"\/documentation\/technologies"},"doc://com.apple.foundationmodels/documentation/FoundationModels":{"abstract":[{"text":"Perform tasks with models that specialize in language understanding, structured output, and tool calling.","type":"text"}],"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels","images":[{"identifier":"foundation-models-framework-landing-PageImage-card.png","type":"card"}],"kind":"symbol","navigatorTitle":[{"kind":"text","text":"Foundation Models"}],"role":"collection","title":"Foundation Models","type":"topic","url":"\/documentation\/foundationmodels"},"doc://com.apple.foundationmodels/documentation/FoundationModels/ContextOptions":{"abstract":[{"text":"Options that configure details that should appear in the prompt.","type":"text"}],"fragments":[{"kind":"keyword","text":"struct"},{"kind":"text","text":" "},{"kind":"identifier","text":"ContextOptions"}],"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/ContextOptions","kind":"symbol","navigatorTitle":[{"kind":"identifier","text":"ContextOptions"}],"role":"symbol","title":"ContextOptions","type":"topic","url":"\/documentation\/foundationmodels\/contextoptions"},"doc://com.apple.foundationmodels/documentation/FoundationModels/Generable":{"abstract":[{"text":"A type that the model uses when responding to prompts.","type":"text"}],"fragments":[{"kind":"keyword","text":"protocol"},{"kind":"text","text":" "},{"kind":"identifier","text":"Generable"}],"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/Generable","kind":"symbol","navigatorTitle":[{"kind":"identifier","text":"Generable"}],"role":"symbol","title":"Generable","type":"topic","url":"\/documentation\/foundationmodels\/generable"},"doc://com.apple.foundationmodels/documentation/FoundationModels/GenerationGuide/maximumCount(_:)":{"abstract":[{"text":"Enforces a maximum number of elements in the array.","type":"text"}],"fragments":[{"kind":"keyword","text":"static"},{"kind":"text","text":" "},{"kind":"keyword","text":"func"},{"kind":"text","text":" "},{"kind":"identifier","text":"maximumCount"},{"kind":"text","text":"<"},{"kind":"genericParameter","text":"Element"},{"kind":"text","text":">("},{"kind":"typeIdentifier","preciseIdentifier":"s:Si","text":"Int"},{"kind":"text","text":") -> "},{"kind":"typeIdentifier","preciseIdentifier":"s:16FoundationModels15GenerationGuideV","text":"GenerationGuide"},{"kind":"text","text":"<["},{"kind":"typeIdentifier","text":"Element"},{"kind":"text","text":"]>"}],"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/GenerationGuide\/maximumCount(_:)","kind":"symbol","role":"symbol","title":"maximumCount(_:)","type":"topic","url":"\/documentation\/foundationmodels\/generationguide\/maximumcount(_:)"},"doc://com.apple.foundationmodels/documentation/FoundationModels/GenerationOptions":{"abstract":[{"text":"Options that control how the model generates its response to a prompt.","type":"text"}],"fragments":[{"kind":"keyword","text":"struct"},{"kind":"text","text":" "},{"kind":"identifier","text":"GenerationOptions"}],"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/GenerationOptions","kind":"symbol","navigatorTitle":[{"kind":"identifier","text":"GenerationOptions"}],"role":"symbol","title":"GenerationOptions","type":"topic","url":"\/documentation\/foundationmodels\/generationoptions"},"doc://com.apple.foundationmodels/documentation/FoundationModels/GenerationOptions/maximumResponseTokens":{"abstract":[{"text":"The maximum number of tokens the model is allowed to produce in its response.","type":"text"}],"fragments":[{"kind":"keyword","text":"var"},{"kind":"text","text":" "},{"kind":"identifier","text":"maximumResponseTokens"},{"kind":"text","text":": "},{"kind":"typeIdentifier","preciseIdentifier":"s:Si","text":"Int"},{"kind":"text","text":"?"}],"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/GenerationOptions\/maximumResponseTokens","kind":"symbol","role":"symbol","title":"maximumResponseTokens","type":"topic","url":"\/documentation\/foundationmodels\/generationoptions\/maximumresponsetokens"},"doc://com.apple.foundationmodels/documentation/FoundationModels/Instructions":{"abstract":[{"text":"Details you provide that define the model’s intended behavior on prompts.","type":"text"}],"fragments":[{"kind":"keyword","text":"struct"},{"kind":"text","text":" "},{"kind":"identifier","text":"Instructions"}],"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/Instructions","kind":"symbol","navigatorTitle":[{"kind":"identifier","text":"Instructions"}],"role":"symbol","title":"Instructions","type":"topic","url":"\/documentation\/foundationmodels\/instructions"},"doc://com.apple.foundationmodels/documentation/FoundationModels/LanguageModelError/contextSizeExceeded(_:)":{"abstract":[{"text":"The session’s transcript exceeded the model’s context size.","type":"text"}],"fragments":[{"kind":"keyword","text":"case"},{"kind":"text","text":" "},{"kind":"identifier","text":"contextSizeExceeded"},{"kind":"text","text":"("},{"kind":"typeIdentifier","preciseIdentifier":"s:16FoundationModels18LanguageModelErrorO","text":"LanguageModelError"},{"kind":"text","text":"."},{"kind":"typeIdentifier","preciseIdentifier":"s:16FoundationModels18LanguageModelErrorO19ContextSizeExceededV","text":"ContextSizeExceeded"},{"kind":"text","text":")"}],"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/LanguageModelError\/contextSizeExceeded(_:)","kind":"symbol","role":"symbol","title":"LanguageModelError.contextSizeExceeded(_:)","type":"topic","url":"\/documentation\/foundationmodels\/languagemodelerror\/contextsizeexceeded(_:)"},"doc://com.apple.foundationmodels/documentation/FoundationModels/LanguageModelSession":{"abstract":[{"text":"An object that represents a session that interacts with a language model.","type":"text"}],"fragments":[{"kind":"keyword","text":"class"},{"kind":"text","text":" "},{"kind":"identifier","text":"LanguageModelSession"}],"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/LanguageModelSession","kind":"symbol","navigatorTitle":[{"kind":"identifier","text":"LanguageModelSession"}],"role":"symbol","title":"LanguageModelSession","type":"topic","url":"\/documentation\/foundationmodels\/languagemodelsession"},"doc://com.apple.foundationmodels/documentation/FoundationModels/LanguageModelSession/init(model:tools:transcript:)":{"abstract":[{"text":"Creates a session by rehydrating from a transcript.","type":"text"}],"fragments":[{"kind":"keyword","text":"convenience"},{"kind":"text","text":"("},{"kind":"externalParam","text":"model"},{"kind":"text","text":":"},{"kind":"externalParam","text":"tools"},{"kind":"text","text":":"},{"kind":"externalParam","text":"transcript"},{"kind":"text","text":":)"}],"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/LanguageModelSession\/init(model:tools:transcript:)","kind":"symbol","role":"symbol","title":"init(model:tools:transcript:)","type":"topic","url":"\/documentation\/foundationmodels\/languagemodelsession\/init(model:tools:transcript:)"},"doc://com.apple.foundationmodels/documentation/FoundationModels/Prompt":{"abstract":[{"text":"A prompt from a person to the model.","type":"text"}],"fragments":[{"kind":"keyword","text":"struct"},{"kind":"text","text":" "},{"kind":"identifier","text":"Prompt"}],"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/Prompt","kind":"symbol","navigatorTitle":[{"kind":"identifier","text":"Prompt"}],"role":"symbol","title":"Prompt","type":"topic","url":"\/documentation\/foundationmodels\/prompt"},"doc://com.apple.foundationmodels/documentation/FoundationModels/SystemLanguageModel/contextSize":{"abstract":[{"text":"The maximum context size in tokens that the model supports.","type":"text"}],"fragments":[{"kind":"keyword","text":"var"},{"kind":"text","text":" "},{"kind":"identifier","text":"contextSize"},{"kind":"text","text":": "},{"kind":"typeIdentifier","preciseIdentifier":"s:Si","text":"Int"}],"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/SystemLanguageModel\/contextSize","kind":"symbol","role":"symbol","title":"contextSize","type":"topic","url":"\/documentation\/foundationmodels\/systemlanguagemodel\/contextsize"},"doc://com.apple.foundationmodels/documentation/FoundationModels/SystemLanguageModel/tokenCount(for:)":{"abstract":[{"text":"Returns the token count for the specified instructions.","type":"text"}],"fragments":[{"kind":"keyword","text":"func"},{"kind":"text","text":" "},{"kind":"identifier","text":"tokenCount"},{"kind":"text","text":"("},{"kind":"externalParam","text":"for"},{"kind":"text","text":":)"}],"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/SystemLanguageModel\/tokenCount(for:)","kind":"symbol","role":"symbol","title":"tokenCount(for:)","type":"topic","url":"\/documentation\/foundationmodels\/systemlanguagemodel\/tokencount(for:)"},"doc://com.apple.foundationmodels/documentation/FoundationModels/analyzing-the-runtime-performance-of-your-foundation-models-app":{"abstract":[{"text":"Measure how prompts, responses, and tool calls affect token consumption and","type":"text"},{"text":" ","type":"text"},{"text":"response times in Instruments.","type":"text"}],"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/analyzing-the-runtime-performance-of-your-foundation-models-app","kind":"article","role":"article","title":"Analyzing the runtime performance of your Foundation Models app","type":"topic","url":"\/documentation\/foundationmodels\/analyzing-the-runtime-performance-of-your-foundation-models-app"},"doc://com.apple.foundationmodels/documentation/FoundationModels/prompting-an-on-device-foundation-model":{"abstract":[{"text":"Tailor your prompts to get effective results from an on-device model.","type":"text"}],"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/prompting-an-on-device-foundation-model","kind":"article","role":"article","title":"Prompting an on-device foundation model","type":"topic","url":"\/documentation\/foundationmodels\/prompting-an-on-device-foundation-model"},"doc://com.apple.foundationmodels/documentation/FoundationModels/updating-prompts-for-new-model-versions":{"abstract":[{"text":"Manage the prompts your app uses by versioning them to make the most out of","type":"text"},{"text":" ","type":"text"},{"text":"model improvements.","type":"text"}],"identifier":"doc:\/\/com.apple.foundationmodels\/documentation\/FoundationModels\/updating-prompts-for-new-model-versions","images":[{"identifier":"updating-prompts-for-new-model-versions-PageImage-card.png","type":"card"}],"kind":"article","role":"article","title":"Updating prompts for new model versions","type":"topic","url":"\/documentation\/foundationmodels\/updating-prompts-for-new-model-versions"},"foundation-models-framework-landing-PageImage-card.png":{"alt":"An illustration that represents a foundation model.","identifier":"foundation-models-framework-landing-PageImage-card.png","type":"image","variants":[{"traits":["2x","light"],"url":"\/images\/com.apple.foundationmodels\/foundation-models-framework-landing-PageImage-card@2x.png"},{"traits":["2x","dark"],"url":"\/images\/com.apple.foundationmodels\/foundation-models-framework-landing-PageImage-card~dark@2x.png"}]},"managing-the-context-window-context-window":{"alt":"A diagram showing how the context window fills over a number of turns of a conversation.","identifier":"managing-the-context-window-context-window","type":"image","variants":[{"traits":["2x","light"],"url":"\/images\/com.apple.foundationmodels\/managing-the-context-window-context-window@2x.png"},{"traits":["2x","dark"],"url":"\/images\/com.apple.foundationmodels\/managing-the-context-window-context-window~dark@2x.png"}]},"managing-the-context-window-new-context-window":{"alt":"A diagram showing a new session with a fresh context window. The session","identifier":"managing-the-context-window-new-context-window","type":"image","variants":[{"traits":["2x","light"],"url":"\/images\/com.apple.foundationmodels\/managing-the-context-window-new-context-window@2x.png"},{"traits":["2x","dark"],"url":"\/images\/com.apple.foundationmodels\/managing-the-context-window-new-context-window~dark@2x.png"}]},"updating-prompts-for-new-model-versions-PageImage-card.png":{"alt":"An abstract illustration that contains a circle, representing the model, emitting model outputs as lines.","identifier":"updating-prompts-for-new-model-versions-PageImage-card.png","type":"image","variants":[{"traits":["2x","light"],"url":"\/images\/com.apple.foundationmodels\/updating-prompts-for-new-model-versions-PageImage-card@2x.png"},{"traits":["2x","dark"],"url":"\/images\/com.apple.foundationmodels\/updating-prompts-for-new-model-versions-PageImage-card~dark@2x.png"}]}}}