-
Intégrez des modèles d’IA embarqués à votre app à l’aide de Core AI
Découvrez une sélection de modèles open source populaires, notamment Qwen, Mistral, SAM3 et bien d'autres, optimisés pour les puces Apple grâce au nouveau framework Core AI. Découvrez comment télécharger, exécuter et comparer des modèles sur votre Mac, et les intégrer à votre app en quelques lignes de code. Explorez un nouveau flux de travail pour la compilation de modèles et la spécialisation sur l'appareil afin d'accélérer le chargement initial du modèle. Découvrez comment profiler et optimiser les performances d'exécution avec les outils Core AI dans Xcode.
Chapitres
- 0:00 - Introduction
- 1:16 - App concept: camera-based vocab learning
- 2:52 - Model discovery
- 7:40 - Getting models with the Core AI models repository
- 8:37 - Integration
- 10:55 - Writing the Swift integration code
- 13:05 - Diagnosing model specialization latency
- 14:40 - Deployment
- 17:00 - Ahead-of-time (AOT) compilation
- 18:03 - iOS demo
- 19:57 - Multiplatform
- 23:06 - Next steps
Ressources
- Core AI PyTorch Extensions
- Core AI Python
- Core AI Optimization
- Core AI
- Compiling Core AI models ahead of time
Vidéos connexes
WWDC26
-
Rechercher dans cette vidéo…
-
-
11:01 - Load and run SAM3 image segmentation
import CoreAIImageSegmenter // Load let segmenter = try await ImageSegmenter(resourcesAt: sam3ModelURL) // Use let response = try await segmenter.segment(image: inputImage, prompt: "flower") let mask = response.segments.first?.mask -
11:28 - Load a language model and create a session
import FoundationModels import CoreAILanguageModels // Create model instance let model = try await CoreAILanguageModel(resourcesAt: qwen3ModelURL) // Create session using the model let session = LanguageModelSession(model: model) // Generate response let response = try await session.respond(to: "...") -
12:29 - Generate structured output with @Generable
import FoundationModels import CoreAILanguageModels @Generable struct VocabCard { let chineseWord: String let englishMeaning: String let exampleSentence: String } let model = try await CoreAILanguageModel(resourcesAt: modelURL) let session = LanguageModelSession(model: model) let response = try await session.respond( to: "Create a vocab card for flower", generating: VocabCard.self ) let card: VocabCard = response.content -
17:22 - Compile a Core AI model ahead of time
$ xcrun coreai-build compile MyModel.aimodel --platform iOS
-
-
- 0:00 - Introduction
Overview of Core AI — a new set of technologies that lets you bring advanced on-device AI capabilities to your apps with no server, no cost per token, and no cloud latency.
- 1:16 - App concept: camera-based vocab learning
Introduction to the demo app — an iOS language-learning app where students point their camera at real-world objects to generate vocab cards with translations, example sentences, and segmented images, all running on-device.
- 2:52 - Model discovery
How to define your app's AI requirements — content, language, and device constraints — and select the right models: SAM3 for text-prompted image segmentation and Qwen 0.6B (a 119-language reasoning model) for vocab card generation.
- 7:40 - Getting models with the Core AI models repository
How to use the coreai-models GitHub repository to find popular models with ready-made export recipes — browsing the catalog, running the export script for SAM3 and Qwen, and getting optimized .aimodel files.
- 8:37 - Integration
How to inspect .aimodel files in Xcode (size, platform targets, function signatures, tensor shapes), add the coreai-models Swift package, and select the CoreAILM and CoreAISegmentation libraries as app dependencies.
- 10:55 - Writing the Swift integration code
How to write the Swift code to use both models — loading SAM3 and running text-prompted segmentation, loading Qwen with a single CoreAILanguageModel line, and using the familiar LanguageModelSession API from Foundation Models with structured @Generable output for typed vocab card fields.
- 13:05 - Diagnosing model specialization latency
Using the new Core AI Instruments template to identify that first-run latency is caused by model specialization — the process that compiles a Core AI model for the specific device — and understanding when and how to handle it gracefully.
- 14:40 - Deployment
How to design a deliberate deployment strategy: using a first-run experience to introduce the feature, keeping models out of the app bundle to avoid bloating update size for all users, and triggering on-demand model download via Background Assets only when the user opts in.
- 17:00 - Ahead-of-time (AOT) compilation
How to use the coreai-build command to perform compilation ahead-of-time on your development machine — generating device-architecture-specific compiled model assets that dramatically reduce on-device specialization time during the first-run experience.
- 18:03 - iOS demo
Live demo of the complete iOS experience: fast model preparation with AOT compilation, SAM3 segmenting real objects (rocks, wood, sunflower), and Qwen generating Mandarin vocab cards — with seamless subsequent inferences from the cached model.
- 19:57 - Multiplatform
How the same Swift code runs on macOS with no changes — adding batch processing for folders of photos, stepping up to Qwen3 8B for higher-quality reasoning and pinyin generation, using longer context for curriculum generation, and a live macOS demo processing road trip photos into a full lesson plan.
- 23:06 - Next steps
Summary: Core AI gives you everything you need to build private, multi-platform on-device AI experiences — no server, no cost per token, no cloud latency.