What’s new in Apple Intelligence

The Foundation Models Framework is a native Swift API that gives you direct access to the same on-device model that powers Apple Intelligence. You can now work with any language model, including Apple Foundation Models, cloud models like Claude and Gemini, or any other provider that conforms to the Language Model protocol.

Multimodal prompts let you pass images alongside text so your app can reason about visual content, and Vision framework tools like OCR and barcode readers are available for your model to call directly, all on-device. Dynamic Profiles let you swap models, tools, and instructions on the fly, so your app’s behavior can adapt within a continuous session.

If you’re enrolled in the App Store Small Business Program and your app has fewer than 2 million total first-time App Store downloads, you can access the next generation of Apple Foundation Models running on Private Cloud Compute at no cloud API cost. And with the Evaluations framework, you can verify that your AI features behave correctly across dynamic conditions, going beyond what unit tests alone can catch.

The App Intents framework is how you can connect your app to Apple Intelligence and features like Siri AI. By adopting App Intents schemas — the recognizable structures Siri understands deeply, built on years of language model training — you can make your app's content discoverable and its capabilities available through natural language with less code.

Entity schemas contribute your app's content to the Spotlight semantic index, enabling personal context understanding with attribution back to your app, and intent schemas let people take action on that content naturally — no specific phrases to define, and no code changes needed as Siri's language understanding evolves or expands to new languages and regional dialects. Schemas cover common app categories like task management, photo editing, and communication, and because they're system-defined, your intents benefit automatically from future improvements.

The new View Annotations API adds on-screen awareness on top of all of this, letting you map your views to entities so people can reference and act on what's right in front of them conversationally. And the new App Intents Testing framework lets you validate your entire integration — Siri, Shortcuts, and Spotlight — through real system pathways, without UI automation, so you can catch issues early and ship with confidence.

Image Playground has been reimagined with a powerful new generative model running on Private Cloud Compute, letting people create high-quality images in virtually any style — including photorealistic — right in your app. People can start from a photo or a description, then refine their images using natural language or touch, and choose from landscape or portrait dimensions to use their creations in even more places. All of these new capabilities are available to your app through the Image Playground API.

Gain insight on how visual intelligence can transform content discovery in your app. Explore how to define entities, process images, and handle multiple result types effectively. Learn best practices for optimizing speed and relevance, and discover how intents enable direct actions like opening or playing content with a single tap.