-
利用 Core AI 将设备端 AI 模型整合到你的 App 中
探索一系列精选的热门开源模型 (包括 Qwen、Mistral、SAM3 等)，这些模型已针对 Apple 芯片使用全新 Core AI 框架进行了优化。了解如何将模型下载到 Mac 上运行并进行基准测试，然后只用几行代码将模型整合到你的 App 中。探索模型编译和设备端定制的全新工作流程，从而加快首次模型加载速度。了解如何使用 Xcode 中的 Core AI 工具来分析和优化运行时性能。
章节
- 0:00 - Introduction
- 1:16 - App concept: camera-based vocab learning
- 2:52 - Model discovery
- 7:40 - Getting models with the Core AI models repository
- 8:37 - Integration
- 10:55 - Writing the Swift integration code
- 13:05 - Diagnosing model specialization latency
- 14:40 - Deployment
- 17:00 - Ahead-of-time (AOT) compilation
- 18:03 - iOS demo
- 19:57 - Multiplatform
- 23:06 - Next steps
资源
- Core AI PyTorch Extensions
- Core AI Python
- Core AI Optimization
- Core AI
- Compiling Core AI models ahead of time
相关视频
WWDC26
-
搜索此视频…
-
-
11:01 - Load and run SAM3 image segmentation
import CoreAIImageSegmenter // Load let segmenter = try await ImageSegmenter(resourcesAt: sam3ModelURL) // Use let response = try await segmenter.segment(image: inputImage, prompt: "flower") let mask = response.segments.first?.mask -
11:28 - Load a language model and create a session
import FoundationModels import CoreAILanguageModels // Create model instance let model = try await CoreAILanguageModel(resourcesAt: qwen3ModelURL) // Create session using the model let session = LanguageModelSession(model: model) // Generate response let response = try await session.respond(to: "...") -
12:29 - Generate structured output with @Generable
import FoundationModels import CoreAILanguageModels @Generable struct VocabCard { let chineseWord: String let englishMeaning: String let exampleSentence: String } let model = try await CoreAILanguageModel(resourcesAt: modelURL) let session = LanguageModelSession(model: model) let response = try await session.respond( to: "Create a vocab card for flower", generating: VocabCard.self ) let card: VocabCard = response.content -
17:22 - Compile a Core AI model ahead of time
$ xcrun coreai-build compile MyModel.aimodel --platform iOS
-
-
- 0:00 - Introduction
Overview of Core AI — a new set of technologies that lets you bring advanced on-device AI capabilities to your apps with no server, no cost per token, and no cloud latency.
- 1:16 - App concept: camera-based vocab learning
Introduction to the demo app — an iOS language-learning app where students point their camera at real-world objects to generate vocab cards with translations, example sentences, and segmented images, all running on-device.
- 2:52 - Model discovery
How to define your app's AI requirements — content, language, and device constraints — and select the right models: SAM3 for text-prompted image segmentation and Qwen 0.6B (a 119-language reasoning model) for vocab card generation.
- 7:40 - Getting models with the Core AI models repository
How to use the coreai-models GitHub repository to find popular models with ready-made export recipes — browsing the catalog, running the export script for SAM3 and Qwen, and getting optimized .aimodel files.
- 8:37 - Integration
How to inspect .aimodel files in Xcode (size, platform targets, function signatures, tensor shapes), add the coreai-models Swift package, and select the CoreAILM and CoreAISegmentation libraries as app dependencies.
- 10:55 - Writing the Swift integration code
How to write the Swift code to use both models — loading SAM3 and running text-prompted segmentation, loading Qwen with a single CoreAILanguageModel line, and using the familiar LanguageModelSession API from Foundation Models with structured @Generable output for typed vocab card fields.
- 13:05 - Diagnosing model specialization latency
Using the new Core AI Instruments template to identify that first-run latency is caused by model specialization — the process that compiles a Core AI model for the specific device — and understanding when and how to handle it gracefully.
- 14:40 - Deployment
How to design a deliberate deployment strategy: using a first-run experience to introduce the feature, keeping models out of the app bundle to avoid bloating update size for all users, and triggering on-demand model download via Background Assets only when the user opts in.
- 17:00 - Ahead-of-time (AOT) compilation
How to use the coreai-build command to perform compilation ahead-of-time on your development machine — generating device-architecture-specific compiled model assets that dramatically reduce on-device specialization time during the first-run experience.
- 18:03 - iOS demo
Live demo of the complete iOS experience: fast model preparation with AOT compilation, SAM3 segmenting real objects (rocks, wood, sunflower), and Qwen generating Mandarin vocab cards — with seamless subsequent inferences from the cached model.
- 19:57 - Multiplatform
How the same Swift code runs on macOS with no changes — adding batch processing for folders of photos, stepping up to Qwen3 8B for higher-quality reasoning and pinyin generation, using longer context for curriculum generation, and a live macOS demo processing road trip photos into a full lesson plan.
- 23:06 - Next steps
Summary: Core AI gives you everything you need to build private, multi-platform on-device AI experiences — no server, no cost per token, no cloud latency.