Foundation Models

RSS for tag

Discuss the Foundation Models framework which provides access to Apple’s on-device large language model that powers Apple Intelligence to help you perform intelligent tasks specific to your app.

Foundation Models Documentation

Posts under Foundation Models subtopic

Post

Replies

Boosts

Views

Activity

Provide actionable feedback for the Foundation Models framework and the on-device LLM
We are really excited to have introduced the Foundation Models framework in WWDC25. When using the framework, you might have feedback about how it can better fit your use cases. Starting in macOS/iOS 26 Beta 4, the best way to provide feedback is to use #Playground in Xcode. To do so: In Xcode, create a playground using #Playground. Fore more information, see Running code snippets using the playground macro. Reproduce the issue by setting up a session and generating a response with your prompt. In the canvas on the right, click the thumbs-up icon to the right of the response. Follow the instructions on the pop-up window and submit your feedback by clicking Share with Apple. Another way to provide your feedback is to file a feedback report with relevant details. Specific to the Foundation Models framework, it’s super important to add the following information in your report: Language model feedback This feedback contains the session transcript, including the instructions, the prompts, the responses, etc. Without that, we can’t reason the model’s behavior, and hence can hardly take any action. Use logFeedbackAttachment(sentiment:issues:desiredOutput: ) to retrieve the feedback data of your current model session, as shown in the usage example, write the data into a file, and then attach the file to your feedback report. If you believe what you’d report is related to the system configuration, please capture a sysdiagnose and attach it to your feedback report as well. The framework is still new. Your actionable feedback helps us evolve the framework quickly, and we appreciate that. Thanks, The Foundation Models framework team
0
0
1.3k
Aug ’25
Why is SystemLanguageModel.default.availability tied to user enabling talk / press side button for Siri?
On iOS 27 Beta 1, it looks like the user must enable either "Siri"/"Hey Siri" or "Press Side Button for Siri" in iOS settings for SystemLanguageModel.default.availability to report true. Otherwise, it returns .appleIntelligenceNotEnabled. Is this expected behavior? This doesn't seem very intuitive. The user might very well want to use in-app AI functionalities without wanting to talk / press side button for Siri. Also, with the new "pull down for Siri" UX these are not the only way to interact with Siri anyway.
0
0
23
2d
SkillActivation Framework Fails to Build in Xcode 26 When Using foundation-models-utilities
Hi Apple Team, I'm trying to use the SkillActivation framework from the Foundation Models Utilities repository: https://github.com/apple/foundation-models-utilities Environment: Xcode 26 Beta iPadOS/macOS 26 Beta Apple Intelligence enabled Foundation Models Utilities: latest version from GitHub Issue: As soon as I import or use SkillActivation-related APIs, Xcode reports build errors and the project fails to compile. The rest of the Foundation Models framework works correctly, but the problem appears specifically when SkillActivation is added. Steps to Reproduce: Create a new project. Add foundation-models-utilities via Swift Package Manager. Import SkillActivation / follow the sample implementation. Build the project. Expected Result: The project should compile successfully and SkillActivation should be available. Actual Result: Xcode reports compilation errors and the build fails. Questions: Is there any additional entitlement, capability, or configuration required for SkillActivation? Is SkillActivation currently supported in Xcode 26 Beta? Are there any known issues with the current version of foundation-models-utilities? Thank you.
2
0
42
2d
Accessing Private Cloud Compute
Hello, I recently learned about Private Cloud Compute (PCC): https://developer.apple.com/private-cloud-compute/ I am currently using a standard Developer Program account, and it seems that I cannot apply for the program directly. Is there an alternative? Also, is there any additional fee for using this service? If I want to call PCC in the app, for example, using the following code: let session = LanguageModelSession( model: PrivateCloudComputeLanguageModel() ) Do I need to apply for a specific plan to ensure that my App is successfully published on the App Store and available to users? Thank you!
0
0
45
5d
Can any Apple Watch running WatchOS 27 access PCC via Foundation Models?
Apologies, if I've missed the answer already here, I've searched around but can't find it. Foundation models and Private Cloud Compute require Apple Intelligence to be enabled in Settings as mentioned here. At the same time it says that Foundation Models PCC calls are supported on all Apple Watch models that run WatchOS 27. So, will there be a seperate Apple Intelligence setting in WatchOS 27 for those devices? Otherwise if a user has an Apple Watch Series 11 (which does support Apple Intelligence) paired with an iPhone 15 (which doesn't support Apple Intelligence), will they be unable to use the Foundation Models PCC calls from WatchOS in my app? Despite the fact the iPhone isn't involved in these queries anyway?
0
1
56
6d
Siri As Coding Agent
In the new Xcode we saw examples of Claude, OAI & Google coding agents that you can start conversations with inside your project, giving it access to your project files context. As far as I understand, this requires an API key for those models & the processing is run on Anthropic / Google servers, not locally nor on Private Cloud Compute. Is it possible to instead, use the LLM powering Foundation Models, for a “Siri Code Agent” which operates in the place of those models, but runs on device or in Private Cloud Compute? I like how this works for Siri AI requests, and would love to have a coding assistant agent that can operate in the same privacy preserving way! Is this possible with any of the open source frameworks or the command line tools? If not, what is the best way to request this feature?
2
1
148
6d
Questions About Apple Foundation Models, Context Window Limits, and the New Core AI Framework
After reviewing the WWDC sessions on Foundation Models and Core AI, I had a few questions around the practical limits and architectural direction of the platform. From my understanding, on-device Foundation Models remain optimized for privacy, latency, and efficiency, which naturally introduces constraints around context length and agent complexity. Has anything changed regarding the effective context window available to developers, or should we still design around similar context-management constraints as before? Core AI appears to introduce a more structured approach to building AI-powered applications. For developers building sophisticated assistants, how should we think about the boundary between application-level orchestration and framework-level orchestration? For example, are advanced patterns such as sub-agents, hierarchical planning, dynamic tool availability, and workflow decomposition expected to remain developer-managed, or are these areas Core AI aims to support more directly over time? I am also curious about Apple's vision for model interoperability. While Foundation Models provide an excellent on-device experience, many production-grade agent systems combine multiple specialized models for planning, reasoning, retrieval, and execution. Does Apple envision future pathways for integrating external models into Core AI driven workflows while maintaining the privacy and performance principles of the platform? Finally, for teams pushing the limits of on-device AI assistants, what architectural patterns do you recommend for handling long-horizon tasks, large context requirements, evolving toolsets, and multi-step reasoning within the current Foundation Models ecosystem?
0
0
71
1w
Questions About Apple Foundation Models, Context Window Limits and the New Core AI Framework
After reviewing the WWDC sessions on Foundation Models and Core AI, I had a few questions around the practical limits and architectural direction of the platform. From my understanding, on-device Foundation Models remain optimized for privacy, latency, and efficiency, which naturally introduces constraints around context length and agent complexity. Has anything changed regarding the effective context window available to developers, or should we still design around similar context-management constraints as before? Core AI appears to introduce a more structured approach to building AI-powered applications. For developers building sophisticated assistants, how should we think about the boundary between application-level orchestration and framework-level orchestration? For example, are advanced patterns such as sub-agents, hierarchical planning, dynamic tool availability, and workflow decomposition expected to remain developer-managed, or are these areas Core AI aims to support more directly over time? I am also curious about Apple's vision for model interoperability. While Foundation Models provide an excellent on-device experience, many production-grade agent systems combine multiple specialized models for planning, reasoning, retrieval, and execution. Does Apple envision future pathways for integrating external models into Core AI driven workflows while maintaining the privacy and performance principles of the platform? Finally, for teams pushing the limits of on-device AI assistants, what architectural patterns do you recommend for handling long-horizon tasks, large context requirements, evolving toolsets, and multi-step reasoning within the current Foundation Models ecosystem?
0
0
47
1w
On Agentic Testing & Accessibility
Since agents in Xcode 27 can now interact with the accessibility tree and screenshots, can we provide 'developer hints' in our code to help the agent distinguish between decorative UI and critical interactive elements during automated testing? Can the Evaluations framework be used to 'score' the efficiency of an agent’s navigation path through the app, helping us identify where our App Intents might be creating confusing or redundant loops for Apple Intelligence?
0
0
42
1w
In-app text input vs system speech paths
If users dictate into a standard TextField via the keyboard mic instead of a dedicated in-app record button, does that text still benefit from App Intents entity resolution and indexed entities — or is keyboard dictation a separate pipeline where we lose domain vocabulary unless the user invokes Siri directly?
0
1
16
1w
On Protocol Extensibility & Multi-Modal Data
The Foundation Models framework is adding built-in OCR and barcode reader tools this year . If we implement a custom backend using the Language Model Protocol, can we return complex multi-modal objects (like bounding boxes or segmentation masks) back to the agentic flow, or is the protocol currently limited to text-based responses? For the 'Phone a Friend' pattern, is there a standard way to pass 'privacy-preserving embeddings' instead of raw text when calling a third-party model to maintain a higher level of user data protection?
1
0
24
1w
Disambiguation when multiple entities match
When a spoken phrase could match several entities in our catalog — same region, similar names, or partial matches — who is responsible for disambiguation: Siri via App Schemas and entity resolution, or the app via EntityStringQuery returning multiple candidates? What’s the recommended UX pattern for ‘Did you mean A or B?’
5
0
64
1w
Provide actionable feedback for the Foundation Models framework and the on-device LLM
We are really excited to have introduced the Foundation Models framework in WWDC25. When using the framework, you might have feedback about how it can better fit your use cases. Starting in macOS/iOS 26 Beta 4, the best way to provide feedback is to use #Playground in Xcode. To do so: In Xcode, create a playground using #Playground. Fore more information, see Running code snippets using the playground macro. Reproduce the issue by setting up a session and generating a response with your prompt. In the canvas on the right, click the thumbs-up icon to the right of the response. Follow the instructions on the pop-up window and submit your feedback by clicking Share with Apple. Another way to provide your feedback is to file a feedback report with relevant details. Specific to the Foundation Models framework, it’s super important to add the following information in your report: Language model feedback This feedback contains the session transcript, including the instructions, the prompts, the responses, etc. Without that, we can’t reason the model’s behavior, and hence can hardly take any action. Use logFeedbackAttachment(sentiment:issues:desiredOutput: ) to retrieve the feedback data of your current model session, as shown in the usage example, write the data into a file, and then attach the file to your feedback report. If you believe what you’d report is related to the system configuration, please capture a sysdiagnose and attach it to your feedback report as well. The framework is still new. Your actionable feedback helps us evolve the framework quickly, and we appreciate that. Thanks, The Foundation Models framework team
Replies
0
Boosts
0
Views
1.3k
Activity
Aug ’25
Why is SystemLanguageModel.default.availability tied to user enabling talk / press side button for Siri?
On iOS 27 Beta 1, it looks like the user must enable either "Siri"/"Hey Siri" or "Press Side Button for Siri" in iOS settings for SystemLanguageModel.default.availability to report true. Otherwise, it returns .appleIntelligenceNotEnabled. Is this expected behavior? This doesn't seem very intuitive. The user might very well want to use in-app AI functionalities without wanting to talk / press side button for Siri. Also, with the new "pull down for Siri" UX these are not the only way to interact with Siri anyway.
Replies
0
Boosts
0
Views
23
Activity
2d
SkillActivation Framework Fails to Build in Xcode 26 When Using foundation-models-utilities
Hi Apple Team, I'm trying to use the SkillActivation framework from the Foundation Models Utilities repository: https://github.com/apple/foundation-models-utilities Environment: Xcode 26 Beta iPadOS/macOS 26 Beta Apple Intelligence enabled Foundation Models Utilities: latest version from GitHub Issue: As soon as I import or use SkillActivation-related APIs, Xcode reports build errors and the project fails to compile. The rest of the Foundation Models framework works correctly, but the problem appears specifically when SkillActivation is added. Steps to Reproduce: Create a new project. Add foundation-models-utilities via Swift Package Manager. Import SkillActivation / follow the sample implementation. Build the project. Expected Result: The project should compile successfully and SkillActivation should be available. Actual Result: Xcode reports compilation errors and the build fails. Questions: Is there any additional entitlement, capability, or configuration required for SkillActivation? Is SkillActivation currently supported in Xcode 26 Beta? Are there any known issues with the current version of foundation-models-utilities? Thank you.
Replies
2
Boosts
0
Views
42
Activity
2d
Accessing Private Cloud Compute
Hello, I recently learned about Private Cloud Compute (PCC): https://developer.apple.com/private-cloud-compute/ I am currently using a standard Developer Program account, and it seems that I cannot apply for the program directly. Is there an alternative? Also, is there any additional fee for using this service? If I want to call PCC in the app, for example, using the following code: let session = LanguageModelSession( model: PrivateCloudComputeLanguageModel() ) Do I need to apply for a specific plan to ensure that my App is successfully published on the App Store and available to users? Thank you!
Replies
0
Boosts
0
Views
45
Activity
5d
Can any Apple Watch running WatchOS 27 access PCC via Foundation Models?
Apologies, if I've missed the answer already here, I've searched around but can't find it. Foundation models and Private Cloud Compute require Apple Intelligence to be enabled in Settings as mentioned here. At the same time it says that Foundation Models PCC calls are supported on all Apple Watch models that run WatchOS 27. So, will there be a seperate Apple Intelligence setting in WatchOS 27 for those devices? Otherwise if a user has an Apple Watch Series 11 (which does support Apple Intelligence) paired with an iPhone 15 (which doesn't support Apple Intelligence), will they be unable to use the Foundation Models PCC calls from WatchOS in my app? Despite the fact the iPhone isn't involved in these queries anyway?
Replies
0
Boosts
1
Views
56
Activity
6d
TTS Advanced Speech Generation: Expressive voices
During WWDC26 Keynote a second generation on-device model was announced with better speech generation capabilities. Is there a new API available for developers to generate speech?
Replies
0
Boosts
1
Views
62
Activity
1w
Image size, format, and background vs other VLMs
With different VLMs supporting different size and background color if padding is needed… and iOS 27 AFM being the most flexible… the previous talk mentioned that the context size suffers for this flexibility… so what’s the best format/size/background for the app to pre-process to minimize token use… much thanks
Replies
0
Boosts
0
Views
39
Activity
1w
Siri As Coding Agent
In the new Xcode we saw examples of Claude, OAI & Google coding agents that you can start conversations with inside your project, giving it access to your project files context. As far as I understand, this requires an API key for those models & the processing is run on Anthropic / Google servers, not locally nor on Private Cloud Compute. Is it possible to instead, use the LLM powering Foundation Models, for a “Siri Code Agent” which operates in the place of those models, but runs on device or in Private Cloud Compute? I like how this works for Siri AI requests, and would love to have a coding assistant agent that can operate in the same privacy preserving way! Is this possible with any of the open source frameworks or the command line tools? If not, what is the best way to request this feature?
Replies
2
Boosts
1
Views
148
Activity
6d
Questions About Apple Foundation Models, Context Window Limits, and the New Core AI Framework
After reviewing the WWDC sessions on Foundation Models and Core AI, I had a few questions around the practical limits and architectural direction of the platform. From my understanding, on-device Foundation Models remain optimized for privacy, latency, and efficiency, which naturally introduces constraints around context length and agent complexity. Has anything changed regarding the effective context window available to developers, or should we still design around similar context-management constraints as before? Core AI appears to introduce a more structured approach to building AI-powered applications. For developers building sophisticated assistants, how should we think about the boundary between application-level orchestration and framework-level orchestration? For example, are advanced patterns such as sub-agents, hierarchical planning, dynamic tool availability, and workflow decomposition expected to remain developer-managed, or are these areas Core AI aims to support more directly over time? I am also curious about Apple's vision for model interoperability. While Foundation Models provide an excellent on-device experience, many production-grade agent systems combine multiple specialized models for planning, reasoning, retrieval, and execution. Does Apple envision future pathways for integrating external models into Core AI driven workflows while maintaining the privacy and performance principles of the platform? Finally, for teams pushing the limits of on-device AI assistants, what architectural patterns do you recommend for handling long-horizon tasks, large context requirements, evolving toolsets, and multi-step reasoning within the current Foundation Models ecosystem?
Replies
0
Boosts
0
Views
71
Activity
1w
Questions About Apple Foundation Models, Context Window Limits and the New Core AI Framework
After reviewing the WWDC sessions on Foundation Models and Core AI, I had a few questions around the practical limits and architectural direction of the platform. From my understanding, on-device Foundation Models remain optimized for privacy, latency, and efficiency, which naturally introduces constraints around context length and agent complexity. Has anything changed regarding the effective context window available to developers, or should we still design around similar context-management constraints as before? Core AI appears to introduce a more structured approach to building AI-powered applications. For developers building sophisticated assistants, how should we think about the boundary between application-level orchestration and framework-level orchestration? For example, are advanced patterns such as sub-agents, hierarchical planning, dynamic tool availability, and workflow decomposition expected to remain developer-managed, or are these areas Core AI aims to support more directly over time? I am also curious about Apple's vision for model interoperability. While Foundation Models provide an excellent on-device experience, many production-grade agent systems combine multiple specialized models for planning, reasoning, retrieval, and execution. Does Apple envision future pathways for integrating external models into Core AI driven workflows while maintaining the privacy and performance principles of the platform? Finally, for teams pushing the limits of on-device AI assistants, what architectural patterns do you recommend for handling long-horizon tasks, large context requirements, evolving toolsets, and multi-step reasoning within the current Foundation Models ecosystem?
Replies
0
Boosts
0
Views
47
Activity
1w
Framework Boundaries
Given that Foundation Models focus on native Swift APIs, is there any supported bridge for a WebKit-based app to access the Language Model Protocol?
Replies
1
Boosts
0
Views
81
Activity
1w
On Agentic Testing & Accessibility
Since agents in Xcode 27 can now interact with the accessibility tree and screenshots, can we provide 'developer hints' in our code to help the agent distinguish between decorative UI and critical interactive elements during automated testing? Can the Evaluations framework be used to 'score' the efficiency of an agent’s navigation path through the app, helping us identify where our App Intents might be creating confusing or redundant loops for Apple Intelligence?
Replies
0
Boosts
0
Views
42
Activity
1w
On Advanced Context Management
When using the 'Summarize History' modifier, can we configure the summarization prompt to specifically preserve certain metadata like tool call IDs so that a resumed conversation can still reference previously executed app actions?
Replies
2
Boosts
0
Views
83
Activity
1w
In-app text input vs system speech paths
If users dictate into a standard TextField via the keyboard mic instead of a dedicated in-app record button, does that text still benefit from App Intents entity resolution and indexed entities — or is keyboard dictation a separate pipeline where we lose domain vocabulary unless the user invokes Siri directly?
Replies
0
Boosts
1
Views
16
Activity
1w
Strict RAG implementation via .required tool calling and temp=0
Any guidance if we want the iOS 27 SystemLanguageModel to always defer to our app for all answers and not its built-in training for responses
Replies
1
Boosts
0
Views
49
Activity
1w
On Protocol Extensibility & Multi-Modal Data
The Foundation Models framework is adding built-in OCR and barcode reader tools this year . If we implement a custom backend using the Language Model Protocol, can we return complex multi-modal objects (like bounding boxes or segmentation masks) back to the agentic flow, or is the protocol currently limited to text-based responses? For the 'Phone a Friend' pattern, is there a standard way to pass 'privacy-preserving embeddings' instead of raw text when calling a third-party model to maintain a higher level of user data protection?
Replies
1
Boosts
0
Views
24
Activity
1w
Mixed languages and foreign proper nouns
If the user’s device language is French but they speak English, or they use one language for the sentence and another for proper nouns, how does Siri handle transcription and entity resolution? Do we need per-locale entity indexing, aliases, or can semantic indexing work across languages?
Replies
0
Boosts
0
Views
23
Activity
1w
Disambiguation when multiple entities match
When a spoken phrase could match several entities in our catalog — same region, similar names, or partial matches — who is responsible for disambiguation: Siri via App Schemas and entity resolution, or the app via EntityStringQuery returning multiple candidates? What’s the recommended UX pattern for ‘Did you mean A or B?’
Replies
5
Boosts
0
Views
64
Activity
1w
Custom vocabulary for speech and entity resolution
Whisper and other STT APIs let you pass a custom vocabulary or initial_prompt to bias recognition toward domain-specific proper nouns. In the App Intents / Siri stack, is there an equivalent way to supply dynamic, per-user term lists — for example favorites or recently used items — to improve how spoken names are transcribed or resolved?
Replies
1
Boosts
0
Views
83
Activity
1w
RAG boundary: static knowledge vs live data
Should static domain documentation live in on-device RAG (local embeddings + FM), while time-sensitive data always comes from network tools — and are there practical size/latency budgets for on-device embedding indexes?
Replies
1
Boosts
0
Views
70
Activity
1w
Summarization that must not hallucinate numbers
What’s Apple’s guidance for using on-device models to turn structured JSON (time series, metrics, units) into a one-line natural-language summary without inventing values?
Replies
1
Boosts
0
Views
29
Activity
1w