At WWDC25 we launched a new type of Lab event for the developer community - Group Labs. A Group Lab is a panel Q&A designed for a large audience of developers. Group Labs are a unique opportunity for the community to submit questions directly to a panel of Apple engineers and designers. Here are the highlights from the WWDC25 Group Lab for Machine Learning and AI Frameworks. What are you most excited about in the Foundation Models framework? The Foundation Models framework provides access to an on-device Large Language Model (LLM), enabling entirely on-device processing for intelligent features. This allows you to build features such as personalized search suggestions and dynamic NPC generation in games. The combination of guided generation and streaming capabilities is particularly exciting for creating delightful animations and features with reliable output. The seamless integration with SwiftUI and the new design material Liquid Glass is also a major advantage. When should I still bring my own LLM via CoreML? It's generally recommended to first explore Apple's built-in system models and APIs, including the Foundation Models framework, as they are highly optimized for Apple devices and cover a wide range of use cases. However, Core ML is still valuable if you need more control or choice over the specific model being deployed, such as customizing existing system models or augmenting prompts. Core ML provides the tools to get these models on-device, but you are responsible for model distribution and updates. Should I migrate PyTorch code to MLX? MLX is an open-source, general-purpose machine learning framework designed for Apple Silicon from the ground up. It offers a familiar API, similar to PyTorch, and supports C, C++, Python, and Swift. MLX emphasizes unified memory, a key feature of Apple Silicon hardware, which can improve performance. It's recommended to try MLX and see if its programming model and features better suit your application's needs. MLX shines when working with state-of-the-art, larger models. Can I test Foundation Models in Xcode simulator or device? Yes, you can use the Xcode simulator to test Foundation Models use cases. However, your Mac must be running macOS Tahoe. You can test on a physical iPhone running iOS 18 by connecting it to your Mac and running Playgrounds or live previews directly on the device. Which on-device models will be supported? any open source models? The Foundation Models framework currently supports Apple's first-party models only. This allows for platform-wide optimizations, improving battery life and reducing latency. While Core ML can be used to integrate open-source models, it's generally recommended to first explore the built-in system models and APIs provided by Apple, including those in the Vision, Natural Language, and Speech frameworks, as they are highly optimized for Apple devices. For frontier models, MLX can run very large models. How often will the Foundational Model be updated? How do we test for stability when the model is updated? The Foundation Model will be updated in sync with operating system updates. You can test your app against new model versions during the beta period by downloading the beta OS and running your app. It is highly recommended to create an "eval set" of golden prompts and responses to evaluate the performance of your features as the model changes or as you tweak your prompts. Report any unsatisfactory or satisfactory cases using Feedback Assistant. Which on-device model/API can I use to extract text data from images such as: nutrition labels, ingredient lists, cashier receipts, etc? Thank you. The Vision framework offers the RecognizeDocumentRequest which is specifically designed for these use cases. It not only recognizes text in images but also provides the structure of the document, such as rows in a receipt or the layout of a nutrition label. It can also identify data like phone numbers, addresses, and prices. What is the context window for the model? What are max tokens in and max tokens out? The context window for the Foundation Model is 4,096 tokens. The split between input and output tokens is flexible. For example, if you input 4,000 tokens, you'll have 96 tokens remaining for the output. The API takes in text, converting it to tokens under the hood. When estimating token count, a good rule of thumb is 3-4 characters per token for languages like English, and 1 character per token for languages like Japanese or Chinese. Handle potential errors gracefully by asking for shorter prompts or starting a new session if the token limit is exceeded. Is there a rate limit for Foundation Models API that is limited by power or temperature condition on the iPhone? Yes, there are rate limits, particularly when your app is in the background. A budget is allocated for background app usage, but exceeding it will result in rate-limiting errors. In the foreground, there is no rate limit unless the device is under heavy load (e.g., camera open, game mode). The system dynamically balances performance, battery life, and thermal conditions, which can affect the token throughput. Use appropriate quality of service settings for your tasks (e.g., background priority for background work) to help the system manage resources effectively. Do the foundation models support languages other than English? Yes, the on-device Foundation Model is multilingual and supports all languages supported by Apple Intelligence. To get the model to output in a specific language, prompt it with instructions indicating the user's preferred language using the locale API (e.g., "The user's preferred language is en-US"). Putting the instructions in English, but then putting the user prompt in the desired output language is a recommended practice. Are larger server-based models available through Foundation Models? No, the Foundation Models API currently only provides access to the on-device Large Language Model at the core of Apple Intelligence. It does not support server-side models. On-device models are preferred for privacy and for performance reasons. Is it possible to run Retrieval-Augmented Generation (RAG) using the Foundation Models framework? Yes, it is possible to run RAG on-device, but the Foundation Models framework does not include a built-in embedding model. You'll need to use a separate database to store vectors and implement nearest neighbor or cosine distance searches. The Natural Language framework offers simple word and sentence embeddings that can be used. Consider using a combination of Foundation Models and Core ML, using Core ML for your embedding model.

We built an open-source macOS menu bar app that turns speech into text and pastes it into the active app — using SpeechAnalyzer for on-device transcription, ScreenCaptureKit + Vision for screen-aware context, and FluidAudio for speaker diarization in meeting mode. Here's what we learned shipping it on macOS 26. GitHub: github.com/Marvinngg/ambient-voice Architecture The app has two modes: hotkey dictation (press to talk, release to inject) and meeting recording (continuous transcription with a floating panel). Dictation Mode Audio capture uses AVCaptureSession (more on why below). The captured audio feeds into SpeechAnalyzer via an AsyncStream: let transcriber = SpeechTranscriber( locale: locale, transcriptionOptions: [], reportingOptions: [.volatileResults, .alternativeTranscriptions], attributeOptions: [.audioTimeRange, .transcriptionConfidence] ) let analyzer = SpeechAnalyzer(modules: [transcriber]) let (inputSequence, inputBuilder) = AsyncStream.makeStream() try await analyzer.start(inputSequence: inputSequence) While recording, we capture a screenshot of the focused window using ScreenCaptureKit, run Vision OCR (VNRecognizeTextRequest), extract keywords, and inject them into SpeechAnalyzer as contextual bias: let context = AnalysisContext() context.contextualStrings[.general] = ocrKeywords try await analyzer.setContext(context) This improves accuracy for technical terms and proper nouns visible on screen. If your screen shows "SpeechAnalyzer", saying it out loud is more likely to be transcribed correctly. After transcription, an optional L2 step sends the text through a local LLM (ollama) for spoken-to-written cleanup, then CGEvent simulates Cmd+V to paste into the active app. Meeting Mode Meeting mode forks the same audio stream to two consumers: SpeechAnalyzer — real-time streaming transcription, displayed in a floating NSPanel FluidAudio buffer — accumulates 16kHz Float32 mono samples for batch speaker diarization after recording stops When the user ends the meeting, FluidAudio's performCompleteDiarization() runs on the accumulated audio. We align transcription segments with speaker segments using audioTimeRange overlap matching — each transcription segment gets assigned the speaker ID with the most time overlap. Results export to Markdown. Pitfalls We Hit on macOS 26 1. AVAudioEngine installTap doesn't fire with Bluetooth devices We started with AVAudioEngine.inputNode.installTap() for audio capture. It worked fine with built-in mics but the tap callback never fired with Bluetooth devices (tested with vivo TWS 4 Hi-Fi). Fix: switched to AVCaptureSession. The delegate callback captureOutput(_:didOutput:from:) fires reliably regardless of audio device. The tradeoff is you get CMSampleBuffer instead of AVAudioPCMBuffer, so you need a conversion step. 2. NSEvent addGlobalMonitorForEvents crashes Our global hotkey listener used NSEvent.addGlobalMonitorForEvents. On macOS 26, this crashes with a Bus error inside GlobalObserverHandler — appears to be a Swift actor runtime issue. Fix: switched to CGEventTap. Works reliably, but the callback runs on a CFRunLoop context, which Swift doesn't recognize as MainActor. 3. CGEventTap callbacks aren't on MainActor If your CGEventTap callback touches any @MainActor state, you'll get concurrency violations. The callback runs on whatever thread owns the CFRunLoop. Fix: bridge with DispatchQueue.main.async {} inside the tap callback before touching any MainActor state. 4. CGPreflightScreenCaptureAccess doesn't request permission We used CGPreflightScreenCaptureAccess() as a guard before calling ScreenCaptureKit. If it returned false, we'd bail out. The problem: this function only checks — it never triggers macOS to add your app to the Screen Recording permission list. Chicken-and-egg: you can't get permission because you never ask for it. Fix: call CGRequestScreenCaptureAccess() at app startup. This adds your app to System Settings → Screen Recording. Then let ScreenCaptureKit calls proceed without the preflight guard — SCShareableContent will also trigger the permission prompt on first use. 5. Ad-hoc signing breaks TCC permissions on every rebuild During development, codesign --sign - (ad-hoc) generates a different code directory hash on every build. macOS TCC tracks permissions by this hash, so every rebuild = new app identity = all permissions reset. Fix: sign with a stable certificate. If you have an Apple Development certificate, use that. The TeamIdentifier stays constant across rebuilds, so TCC permissions persist. We also discovered that launching via open WE.app (LaunchServices) instead of directly executing the binary is required — otherwise macOS attributes TCC permissions to Terminal, not your app. Benchmarks We ran end-to-end benchmarks on public datasets (Mac Mini M4 16GB, macOS 26): Transcription (SpeechAnalyzer, AliMeeting Chinese): • Near-field CER 34% (excluding outliers ~25%) • Far-field CER 40% (single channel, no beamforming, >30% overlap) • Processing speed 74-89x real-time Speaker diarization (FluidAudio offline): • AMI English 16 meetings: avg DER 23.2% (collar=0.25s, ignoreOverlap=True) • AliMeeting Chinese 8 meetings: DER 48.5% (including overlap regions) • Memory: RSS ~500MB, peak 730-930MB Full evaluation methodology, scripts, and raw results are in the repo. Open Source The project is MIT licensed: github.com/Marvinngg/ambient-voice It includes the macOS client (Swift 6.2, SPM), server-side distillation/training scripts (Python), and a complete evaluation framework with reproducible benchmarks. Feedback and contributions welcome.

We built an open-source macOS menu bar app that turns speech into text and pastes it into the active app — using SpeechAnalyzer for on-device transcription, ScreenCaptureKit + Vision for screen-aware context, and FluidAudio for speaker diarization in meeting mode. Here's what we learned shipping it on macOS 26. GitHub: github.com/Marvinngg/ambient-voice Architecture The app has two modes: hotkey dictation (press to talk, release to inject) and meeting recording (continuous transcription with a floating panel). Dictation Mode Audio capture uses AVCaptureSession (more on why below). The captured audio feeds into SpeechAnalyzer via an AsyncStream: let transcriber = SpeechTranscriber( locale: locale, transcriptionOptions: [], reportingOptions: [.volatileResults, .alternativeTranscriptions], attributeOptions: [.audioTimeRange, .transcriptionConfidence] ) let analyzer = SpeechAnalyzer(modules: [transcriber]) let (inputSequence, inputBuilder) = AsyncStream.makeStream() try await analyzer.start(inputSequence: inputSequence) While recording, we capture a screenshot of the focused window using ScreenCaptureKit, run Vision OCR (VNRecognizeTextRequest), extract keywords, and inject them into SpeechAnalyzer as contextual bias: let context = AnalysisContext() context.contextualStrings[.general] = ocrKeywords try await analyzer.setContext(context) This improves accuracy for technical terms and proper nouns visible on screen. If your screen shows "SpeechAnalyzer", saying it out loud is more likely to be transcribed correctly. After transcription, an optional L2 step sends the text through a local LLM (ollama) for spoken-to-written cleanup, then CGEvent simulates Cmd+V to paste into the active app. Meeting Mode Meeting mode forks the same audio stream to two consumers: SpeechAnalyzer — real-time streaming transcription, displayed in a floating NSPanel FluidAudio buffer — accumulates 16kHz Float32 mono samples for batch speaker diarization after recording stops When the user ends the meeting, FluidAudio's performCompleteDiarization() runs on the accumulated audio. We align transcription segments with speaker segments using audioTimeRange overlap matching — each transcription segment gets assigned the speaker ID with the most time overlap. Results export to Markdown. Pitfalls We Hit on macOS 26 1. AVAudioEngine installTap doesn't fire with Bluetooth devices We started with AVAudioEngine.inputNode.installTap() for audio capture. It worked fine with built-in mics but the tap callback never fired with Bluetooth devices (tested with vivo TWS 4 Hi-Fi). Fix: switched to AVCaptureSession. The delegate callback captureOutput(_:didOutput:from:) fires reliably regardless of audio device. The tradeoff is you get CMSampleBuffer instead of AVAudioPCMBuffer, so you need a conversion step. 2. NSEvent addGlobalMonitorForEvents crashes Our global hotkey listener used NSEvent.addGlobalMonitorForEvents. On macOS 26, this crashes with a Bus error inside GlobalObserverHandler — appears to be a Swift actor runtime issue. Fix: switched to CGEventTap. Works reliably, but the callback runs on a CFRunLoop context, which Swift doesn't recognize as MainActor. 3. CGEventTap callbacks aren't on MainActor If your CGEventTap callback touches any @MainActor state, you'll get concurrency violations. The callback runs on whatever thread owns the CFRunLoop. Fix: bridge with DispatchQueue.main.async {} inside the tap callback before touching any MainActor state. 4. CGPreflightScreenCaptureAccess doesn't request permission We used CGPreflightScreenCaptureAccess() as a guard before calling ScreenCaptureKit. If it returned false, we'd bail out. The problem: this function only checks — it never triggers macOS to add your app to the Screen Recording permission list. Chicken-and-egg: you can't get permission because you never ask for it. Fix: call CGRequestScreenCaptureAccess() at app startup. This adds your app to System Settings → Screen Recording. Then let ScreenCaptureKit calls proceed without the preflight guard — SCShareableContent will also trigger the permission prompt on first use. 5. Ad-hoc signing breaks TCC permissions on every rebuild During development, codesign --sign - (ad-hoc) generates a different code directory hash on every build. macOS TCC tracks permissions by this hash, so every rebuild = new app identity = all permissions reset. Fix: sign with a stable certificate. If you have an Apple Development certificate, use that. The TeamIdentifier stays constant across rebuilds, so TCC permissions persist. We also discovered that launching via open WE.app (LaunchServices) instead of directly executing the binary is required — otherwise macOS attributes TCC permissions to Terminal, not your app. Benchmarks We ran end-to-end benchmarks on public datasets (Mac Mini M4 16GB, macOS 26): Transcription (SpeechAnalyzer, AliMeeting Chinese): • Near-field CER 34% (excluding outliers ~25%) • Far-field CER 40% (single channel, no beamforming, >30% overlap) • Processing speed 74-89x real-time Speaker diarization (FluidAudio offline): • AMI English 16 meetings: avg DER 23.2% (collar=0.25s, ignoreOverlap=True) • AliMeeting Chinese 8 meetings: DER 48.5% (including overlap regions) • Memory: RSS ~500MB, peak 730-930MB Full evaluation methodology, scripts, and raw results are in the repo. Open Source The project is MIT licensed: github.com/Marvinngg/ambient-voice It includes the macOS client (Swift 6.2, SPM), server-side distillation/training scripts (Python), and a complete evaluation framework with reproducible benchmarks. Feedback and contributions welcome.

Hello, Could you please provide details for maximum string length of the prompt and the title when using ImageCreator and the method extracted(from:title:)? static func extracted( from text: String, title: String? = nil ) -> ImagePlaygroundConcept Any additional details or example of prompt and title would help. Additionally, are ImagePlaygroundStyle.animation, ImagePlaygroundStyle.illustration and ImagePlaygroundStyle.sketch all available when using extracted(from:title:)? I am trying to generate images programmatically and would appreciate your guidance. Thank you.

In macOS Tahoe 26.2 an RDMA capability was added for Thunderbolt-5 interfaces. This has been demonstrated to significantly decrease the latency and maintain bandwidth for "clustered" Apple Silicon devices with TB5. What is the ideal and the maximum RDMA burst width for transfers over RDMA-enabled Thunderbolt-5 interfaces?

Application is getting Crashed: AXSpeech EXC_BAD_ACCESS KERN_INVALID_ADDRESS 0x000056f023efbeb0 Crashed: AXSpeech 0 libobjc.A.dylib 0x4820 objc_msgSend + 32 1 libsystem_trace.dylib 0x6c34 _os_log_fmt_flatten_object + 116 2 libsystem_trace.dylib 0x5344 _os_log_impl_flatten_and_send + 1884 3 libsystem_trace.dylib 0x4bd0 _os_log + 152 4 libsystem_trace.dylib 0x9c48 _os_log_error_impl + 24 5 TextToSpeech 0xd0a8c _pcre2_xclass_8 6 TextToSpeech 0x3bc04 TTSSpeechUnitTestingMode 7 TextToSpeech 0x3f128 TTSSpeechUnitTestingMode 8 AXCoreUtilities 0xad38 -[NSArray(AXExtras) ax_flatMappedArrayUsingBlock:] + 204 9 TextToSpeech 0x3eb18 TTSSpeechUnitTestingMode 10 TextToSpeech 0x3c948 TTSSpeechUnitTestingMode 11 TextToSpeech 0x48824 AXAVSpeechSynthesisVoiceFromTTSSpeechVoice 12 TextToSpeech 0x49804 AXAVSpeechSynthesisVoiceFromTTSSpeechVoice 13 Foundation 0xf6064 __NSThreadPerformPerform + 264 14 CoreFoundation 0x37acc CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION + 28 15 CoreFoundation 0x36d48 __CFRunLoopDoSource0 + 176 16 CoreFoundation 0x354fc __CFRunLoopDoSources0 + 244 17 CoreFoundation 0x34238 __CFRunLoopRun + 828 18 CoreFoundation 0x33e18 CFRunLoopRunSpecific + 608 19 Foundation 0x2d4cc -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 212 20 TextToSpeech 0x24b88 TTSCFAttributedStringCreateStringByBracketingAttributeWithString 21 Foundation 0xb3154 NSThread__start + 732 com.livingMedia.AajTakiPhone_issue_3ceba855a8ad2d1af83655803dc13f70_crash_session_9081fa41ced440ae9a57c22cb432f312_DNE_0_v2_stacktrace.txt 22 libsystem_pthread.dylib 0x24d4 _pthread_start + 136 23 libsystem_pthread.dylib 0x1a10 thread_start + 8

I have a very terrible crash problem in my App when I use AVSpeechSynthesizer and I can't repetition it.Here is my code, It's a singleton- (void)stopSpeech { if ([self.synthesizer isPaused]) { return; } if ([self.synthesizer isSpeaking]) { BOOL isSpeech = [self.synthesizer stopSpeakingAtBoundary:AVSpeechBoundaryImmediate]; if (!isSpeech) { [self.synthesizer stopSpeakingAtBoundary:AVSpeechBoundaryWord]; } } self.stopBlock ? self.stopBlock() : nil; } -(AVSpeechSynthesizer *)synthesizer { if (!_synthesizer) { _synthesizer = [[AVSpeechSynthesizer alloc] init]; _synthesizer.delegate = self; } return _synthesizer; }When the user leaves the page, I call the stopSpeech method。Then I got a lot of crash messagesHere is a crash log:# Crashlytics - plaintext stacktrace downloaded by liweican at Mon, 13 May 2019 03:03:24 GMT # URL: https://fabric.io/youdao-dict/ios/apps/com.youdao.udictionary/issues/5a904ed88cb3c2fa63ad7ed3?time=last-thirty-days/sessions/b1747d91bafc4680ab0ca8e3a702c52c_DNE_0_v2 # Organization: zzz # Platform: ios # Application: U-Dictionary # Version: 3.0.5.4 # Bundle Identifier: com.youdao.UDictionary # Issue ID: 5a904ed88cb3c2fa63ad7ed3 # Session ID: b1747d91bafc4680ab0ca8e3a702c52c_DNE_0_v2 # Date: 2019-05-13T02:27:00Z # OS Version: 12.2.0 (16E227) # Device: iPhone 8 Plus # RAM Free: 17% # Disk Free: 64.6% #19. Crashed: AXSpeech 0 libsystem_pthread.dylib 0x19c15e5b8 pthread_mutex_lock$VARIANT$armv81 + 102 1 CoreFoundation 0x19c4cf84c CFRunLoopSourceSignal + 68 2 Foundation 0x19cfc7280 performQueueDequeue + 464 3 Foundation 0x19cfc680c __NSThreadPerformPerform + 136 4 CoreFoundation 0x19c4d22bc __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 24 5 CoreFoundation 0x19c4d223c __CFRunLoopDoSource0 + 88 6 CoreFoundation 0x19c4d1b74 __CFRunLoopDoSources0 + 256 7 CoreFoundation 0x19c4cca60 __CFRunLoopRun + 1004 8 CoreFoundation 0x19c4cc354 CFRunLoopRunSpecific + 436 9 Foundation 0x19ce99fcc -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 300 10 libAXSpeechManager.dylib 0x1ac16c94c -[AXSpeechThread main] + 264 11 Foundation 0x19cfc66e4 __NSThread__start__ + 984 12 libsystem_pthread.dylib 0x19c1602c0 _pthread_body + 128 13 libsystem_pthread.dylib 0x19c160220 _pthread_start + 44 14 libsystem_pthread.dylib 0x19c163cdc thread_start + 4 -- #0. com.apple.main-thread 0 libsystem_malloc.dylib 0x19c11ce24 small_free_list_remove_ptr_no_clear + 768 1 libsystem_malloc.dylib 0x19c11f094 small_malloc_from_free_list + 296 2 libsystem_malloc.dylib 0x19c11f094 small_malloc_from_free_list + 296 3 libsystem_malloc.dylib 0x19c11d63c small_malloc_should_clear + 224 4 libsystem_malloc.dylib 0x19c11adcc szone_malloc_should_clear + 132 5 libsystem_malloc.dylib 0x19c123c18 malloc_zone_malloc + 156 6 CoreFoundation 0x19c569ab4 __CFBasicHashRehash + 300 7 CoreFoundation 0x19c56b430 __CFBasicHashAddValue + 96 8 CoreFoundation 0x19c56ab9c CFBasicHashAddValue + 2160 9 CoreFoundation 0x19c49f3bc CFDictionaryAddValue + 260 10 CoreFoundation 0x19c572ee8 __54-[CFPrefsSource mergeIntoDictionary:sourceDictionary:]_block_invoke + 28 11 CoreFoundation 0x19c49f0b4 __CFDictionaryApplyFunction_block_invoke + 24 12 CoreFoundation 0x19c568b7c CFBasicHashApply + 116 13 CoreFoundation 0x19c49f090 CFDictionaryApplyFunction + 168 14 CoreFoundation 0x19c42f504 -[CFPrefsSource mergeIntoDictionary:sourceDictionary:] + 136 15 CoreFoundation 0x19c4bcd38 -[CFPrefsSearchListSource alreadylocked_getDictionary:] + 644 16 CoreFoundation 0x19c42e71c -[CFPrefsSearchListSource alreadylocked_copyValueForKey:] + 152 17 CoreFoundation 0x19c42e660 -[CFPrefsSource copyValueForKey:] + 60 18 CoreFoundation 0x19c579e88 __76-[_CFXPreferences copyAppValueForKey:identifier:container:configurationURL:]_block_invoke + 40 19 CoreFoundation 0x19c4bdff4 __108-[_CFXPreferences(SearchListAdditions) withSearchListForIdentifier:container:cloudConfigurationURL:perform:]_block_invoke + 272 20 CoreFoundation 0x19c4bda38 normalizeQuintuplet + 340 21 CoreFoundation 0x19c42c634 -[_CFXPreferences(SearchListAdditions) withSearchListForIdentifier:container:cloudConfigurationURL:perform:] + 108 22 CoreFoundation 0x19c42cec0 -[_CFXPreferences copyAppValueForKey:identifier:container:configurationURL:] + 148 23 CoreFoundation 0x19c57c2d0 _CFPreferencesCopyAppValueWithContainerAndConfiguration + 124 24 TextInput 0x1a450e550 -[TIPreferencesController valueForPreferenceKey:] + 460 25 UIKitCore 0x1c87c71f8 -[UIKeyboardPreferencesController handBias] + 36 26 UIKitCore 0x1c887275c -[UIKeyboardLayoutStar showKeyboardWithInputTraits:screenTraits:splitTraits:] + 320 27 UIKitCore 0x1c88f4240 -[UIKeyboardImpl finishLayoutChangeWithArguments:] + 492 28 UIKitCore 0x1c88f47c8 -[UIKeyboardImpl updateLayout] + 1208 29 UIKitCore 0x1c88eaad0 -[UIKeyboardImpl updateLayoutIfNecessary] + 448 30 UIKitCore 0x1c88eab9c -[UIKeyboardImpl setFrame:] + 140 31 UIKitCore 0x1c88d5d60 -[UIKeyboard activate] + 652 32 UIKitCore 0x1c894c90c -[UIKeyboardAutomatic activate] + 128 33 UIKitCore 0x1c88d5158 -[UIKeyboard setFrame:] + 296 34 UIKitCore 0x1c88d81b0 -[UIKeyboard _didChangeKeyplaneWithContext:] + 228 35 UIKitCore 0x1c88f4aa0 -[UIKeyboardImpl didMoveToSuperview] + 136 36 UIKitCore 0x1c8f2ad84 __45-[UIView(Hierarchy) _postMovedFromSuperview:]_block_invoke + 888 37 UIKitCore 0x1c8f2a970 -[UIView(Hierarchy) _postMovedFromSuperview:] + 760 38 UIKitCore 0x1c8f39ddc -[UIView(Internal) _addSubview:positioned:relativeTo:] + 1740 39 UIKitCore 0x1c88d5d84 -[UIKeyboard activate] + 688 40 UIKitCore 0x1c894c90c -[UIKeyboardAutomatic activate] + 128 41 UIKitCore 0x1c893b3a4 -[UIPeripheralHost(UIKitInternal) _reloadInputViewsForResponder:] + 1332 42 UIKitCore 0x1c8ae66d8 -[UIResponder(UIResponderInputViewAdditions) reloadInputViews] + 80 43 UIKitCore 0x1c8ae23bc -[UIResponder becomeFirstResponder] + 804 44 UIKitCore 0x1c8f2a560 -[UIView(Hierarchy) becomeFirstResponder] + 156 45 UIKitCore 0x1c8d93e84 -[UITextField becomeFirstResponder] + 244 46 UIKitCore 0x1c8d578dc -[UITextInteractionAssistant(UITextInteractionAssistant_Internal) setFirstResponderIfNecessary] + 192 47 UIKitCore 0x1c8d45d8c -[UITextSelectionInteraction oneFingerTap:] + 3136 48 UIKitCore 0x1c86e0bcc -[UIGestureRecognizerTarget _sendActionWithGestureRecognizer:] + 64 49 UIKitCore 0x1c86e8dd4 _UIGestureRecognizerSendTargetActions + 124 50 UIKitCore 0x1c86e6778 _UIGestureRecognizerSendActions + 316 51 UIKitCore 0x1c86e5ca4 -[UIGestureRecognizer _updateGestureWithEvent:buttonEvent:] + 760 52 UIKitCore 0x1c86d9d80 _UIGestureEnvironmentUpdate + 2180 53 UIKitCore 0x1c86d94b0 -[UIGestureEnvironment _deliverEvent:toGestureRecognizers:usingBlock:] + 384 54 UIKitCore 0x1c86d9290 -[UIGestureEnvironment _updateForEvent:window:] + 204 55 UIKitCore 0x1c8af14a8 -[UIWindow sendEvent:] + 3112 56 UIKitCore 0x1c8ad1534 -[UIApplication sendEvent:] + 340 57 UIKitCore 0x1c8b977c0 __dispatchPreprocessedEventFromEventQueue + 1768 58 UIKitCore 0x1c8b99eec __handleEventQueueInternal + 4828 59 UIKitCore 0x1c8b9311c __handleHIDEventFetcherDrain + 152 60 CoreFoundation 0x19c4d22bc __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 24 61 CoreFoundation 0x19c4d223c __CFRunLoopDoSource0 + 88 62 CoreFoundation 0x19c4d1b24 __CFRunLoopDoSources0 + 176 63 CoreFoundation 0x19c4cca60 __CFRunLoopRun + 1004 64 CoreFoundation 0x19c4cc354 CFRunLoopRunSpecific + 436 65 GraphicsServices 0x19e6cc79c GSEventRunModal + 104 66 UIKitCore 0x1c8ab7b68 UIApplicationMain + 212 67 UDictionary 0x10517e138 main (main.m:17) 68 libdyld.dylib 0x19bf928e0 start + 4 #1. Thread 0 libsystem_kernel.dylib 0x19c0deb74 __workq_kernreturn + 8 1 libsystem_pthread.dylib 0x19c161138 _pthread_wqthread + 340 2 libsystem_pthread.dylib 0x19c163cd4 start_wqthread + 4 #2. com.apple.uikit.eventfetch-thread 0 libsystem_kernel.dylib 0x19c0d30f4 mach_msg_trap + 8 1 libsystem_kernel.dylib 0x19c0d25a0 mach_msg + 72 2 CoreFoundation 0x19c4d1cb4 __CFRunLoopServiceMachPort + 236 3 CoreFoundation 0x19c4ccbc4 __CFRunLoopRun + 1360 4 CoreFoundation 0x19c4cc354 CFRunLoopRunSpecific + 436 5 Foundation 0x19ce99fcc -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 300 6 Foundation 0x19ce99e5c -[NSRunLoop(NSRunLoop) runUntilDate:] + 96 7 UIKitCore 0x1c8b9d540 -[UIEventFetcher threadMain] + 136 8 Foundation 0x19cfc66e4 __NSThread__start__ + 984 9 libsystem_pthread.dylib 0x19c1602c0 _pthread_body + 128 10 libsystem_pthread.dylib 0x19c160220 _pthread_start + 44 11 libsystem_pthread.dylib 0x19c163cdc thread_start + 4 #3. JavaScriptCore bmalloc scavenger 0 libsystem_kernel.dylib 0x19c0ddee4 __psynch_cvwait + 8 1 libsystem_pthread.dylib 0x19c15d4a4 _pthread_cond_wait$VARIANT$armv81 + 628 2 libc++.1.dylib 0x19b6b5090 std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 24 3 JavaScriptCore 0x1a36a2238 void std::__1::condition_variable_any::wait<std::__1::unique_lock<bmalloc::Mutex> >(std::__1::unique_lock<bmalloc::Mutex>&) + 108 4 JavaScriptCore 0x1a36a622c bmalloc::Scavenger::threadRunLoop() + 176 5 JavaScriptCore 0x1a36a59a4 bmalloc::Scavenger::Scavenger(std::__1::lock_guard<bmalloc::Mutex>&) + 10 6 JavaScriptCore 0x1a36a73e4 std::__1::__thread_specific_ptr<std::__1::__thread_struct>::set_pointer(std::__1::__thread_struct*) + 38 7 libsystem_pthread.dylib 0x19c1602c0 _pthread_body + 128 8 libsystem_pthread.dylib 0x19c160220 _pthread_start + 44 9 libsystem_pthread.dylib 0x19c163cdc thread_start + 4 #4. WebThread 0 libsystem_kernel.dylib 0x19c0d30f4 mach_msg_trap + 8 1 libsystem_kernel.dylib 0x19c0d25a0 mach_msg + 72 2 CoreFoundation 0x19c4d1cb4 __CFRunLoopServiceMachPort + 236 3 CoreFoundation 0x19c4ccbc4 __CFRunLoopRun + 1360 4 CoreFoundation 0x19c4cc354 CFRunLoopRunSpecific + 436 5 WebCore 0x1a5126480 RunWebThread(void*) + 600 6 libsystem_pthread.dylib 0x19c1602c0 _pthread_body + 128 7 libsystem_pthread.dylib 0x19c160220 _pthread_start + 44 8 libsystem_pthread.dylib 0x19c163cdc thread_start + 4 #5. com.twitter.crashlytics.ios.MachExceptionServer 0 UDictionary 0x1058a5564 CLSProcessRecordAllThreads (CLSProcess.c:376) 1 UDictionary 0x1058a594c CLSProcessRecordAllThreads (CLSProcess.c:407) 2 UDictionary 0x1058952dc CLSHandler (CLSHandler.m:26) 3 UDictionary 0x1058906cc CLSMachExceptionServer (CLSMachException.c:446) 4 libsystem_pthread.dylib 0x19c1602c0 _pthread_body + 128 5 libsystem_pthread.dylib 0x19c160220 _pthread_start + 44 6 libsystem_pthread.dylib 0x19c163cdc thread_start + 4 #6. com.apple.NSURLConnectionLoader 0 libsystem_kernel.dylib 0x19c0d30f4 mach_msg_trap + 8 1 libsystem_kernel.dylib 0x19c0d25a0 mach_msg + 72 2 CoreFoundation 0x19c4d1cb4 __CFRunLoopServiceMachPort + 236 3 CoreFoundation 0x19c4ccbc4 __CFRunLoopRun + 1360 4 CoreFoundation 0x19c4cc354 CFRunLoopRunSpecific + 436 5 CFNetwork 0x19cae574c -[__CoreSchedulingSetRunnable runForever] + 216 6 Foundation 0x19cfc66e4 __NSThread__start__ + 984 7 libsystem_pthread.dylib 0x19c1602c0 _pthread_body + 128 8 libsystem_pthread.dylib 0x19c160220 _pthread_start + 44 9 libsystem_pthread.dylib 0x19c163cdc thread_start + 4 #7. AVAudioSession Notify Thread 0 libsystem_kernel.dylib 0x19c0d30f4 mach_msg_trap + 8 1 libsystem_kernel.dylib 0x19c0d25a0 mach_msg + 72 2 CoreFoundation 0x19c4d1cb4 __CFRunLoopServiceMachPort + 236 3 CoreFoundation 0x19c4ccbc4 __CFRunLoopRun + 1360 4 CoreFoundation 0x19c4cc354 CFRunLoopRunSpecific + 436 5 AVFAudio 0x1a238a378 GenericRunLoopThread::Entry(void*) + 156 6 AVFAudio 0x1a23b4c60 CAPThread::Entry(CAPThread*) + 88 7 libsystem_pthread.dylib 0x19c1602c0 _pthread_body + 128 8 libsystem_pthread.dylib 0x19c160220 _pthread_start + 44 9 libsystem_pthread.dylib 0x19c163cdc thread_start + 4 #8. WebCore: LocalStorage 0 libsystem_kernel.dylib 0x19c0ddee4 __psynch_cvwait + 8 1 libsystem_pthread.dylib 0x19c15d4a4 _pthread_cond_wait$VARIANT$armv81 + 628 2 JavaScriptCore 0x1a3668ce4 ***::ThreadCondition::timedWait(***::Mutex&, ***::WallTime) + 80 3 JavaScriptCore 0x1a364f96c ***::ParkingLot::parkConditionallyImpl(void const*, ***::ScopedLambda<bool ()> const&, ***::ScopedLambda<void ()> const&, ***::TimeWithDynamicClockType const&) + 2004 4 WebKitLegacy 0x1a67b6ea8 bool ***::Condition::waitUntil<***::Lock>(***::Lock&, ***::TimeWithDynamicClockType const&) + 184 5 WebKitLegacy 0x1a67b9ba4 std::__1::unique_ptr<***::Function<void ()>, std::__1::default_delete<***::Function<void ()> > > ***::MessageQueue<***::Function<void ()> >::waitForMessageFilteredWithTimeout<***::MessageQueue<***::Function<void ()> >::waitForMessage()::'lambda'(***::Function<void ()> const&)>(***::MessageQueueWaitResult&, ***::MessageQueue<***::Function<void ()> >::waitForMessage()::'lambda'(***::Function<void ()> const&)&&, ***::WallTime) + 156 6 WebKitLegacy 0x1a67b91c0 WebCore::StorageThread::threadEntryPoint() + 68 7 JavaScriptCore 0x1a3666f88 ***::Thread::entryPoint(***::Thread::NewThreadContext*) + 260 8 JavaScriptCore 0x1a3668494 ***::wtfThreadEntryPoint(void*) + 12 9 libsystem_pthread.dylib 0x19c1602c0 _pthread_body + 128 10 libsystem_pthread.dylib 0x19c160220 _pthread_start + 44 11 libsystem_pthread.dylib 0x19c163cdc thread_start + 4 #9. com.apple.CoreMotion.MotionThread 0 libsystem_kernel.dylib 0x19c0d30f4 mach_msg_trap + 8 1 libsystem_kernel.dylib 0x19c0d25a0 mach_msg + 72 2 CoreFoundation 0x19c4d1cb4 __CFRunLoopServiceMachPort + 236 3 CoreFoundation 0x19c4ccbc4 __CFRunLoopRun + 1360 4 CoreFoundation 0x19c4cc354 CFRunLoopRunSpecific + 436 5 CoreFoundation 0x19c4cd0b0 CFRunLoopRun + 80 6 CoreMotion 0x1a1df0240 (Missing) 7 libsystem_pthread.dylib 0x19c1602c0 _pthread_body + 128 8 libsystem_pthread.dylib 0x19c160220 _pthread_start + 44 9 libsystem_pthread.dylib 0x19c163cdc thread_start + 4 #10. Thread 0 libsystem_kernel.dylib 0x19c0deb74 __workq_kernreturn + 8 1 libsystem_pthread.dylib 0x19c161138 _pthread_wqthread + 340 2 libsystem_pthread.dylib 0x19c163cd4 start_wqthread + 4 #11. Thread 0 libsystem_kernel.dylib 0x19c0deb74 __workq_kernreturn + 8 1 libsystem_pthread.dylib 0x19c1611f8 _pthread_wqthread + 532 2 libsystem_pthread.dylib 0x19c163cd4 start_wqthread + 4 #12. com.apple.CFStream.LegacyThread 0 libsystem_kernel.dylib 0x19c0d30f4 mach_msg_trap + 8 1 libsystem_kernel.dylib 0x19c0d25a0 mach_msg + 72 2 CoreFoundation 0x19c4d1cb4 __CFRunLoopServiceMachPort + 236 3 CoreFoundation 0x19c4ccbc4 __CFRunLoopRun + 1360 4 CoreFoundation 0x19c4cc354 CFRunLoopRunSpecific + 436 5 CoreFoundation 0x19c4e5094 _legacyStreamRunLoop_workThread + 260 6 libsystem_pthread.dylib 0x19c1602c0 _pthread_body + 128 7 libsystem_pthread.dylib 0x19c160220 _pthread_start + 44 8 libsystem_pthread.dylib 0x19c163cdc thread_start + 4 #13. Thread 0 libsystem_pthread.dylib 0x19c163cd0 start_wqthread + 190 #14. Thread 0 libsystem_kernel.dylib 0x19c0deb74 __workq_kernreturn + 8 1 libsystem_pthread.dylib 0x19c161138 _pthread_wqthread + 340 2 libsystem_pthread.dylib 0x19c163cd4 start_wqthread + 4 #15. Thread 0 libsystem_kernel.dylib 0x19c0deb74 __workq_kernreturn + 8 1 libsystem_pthread.dylib 0x19c161138 _pthread_wqthread + 340 2 libsystem_pthread.dylib 0x19c163cd4 start_wqthread + 4 #16. Thread 0 libsystem_kernel.dylib 0x19c0d3148 semaphore_timedwait_trap + 8 1 libdispatch.dylib 0x19bf50a4c _dispatch_sema4_timedwait$VARIANT$armv81 + 64 2 libdispatch.dylib 0x19bf513a8 _dispatch_semaphore_wait_slow + 72 3 libdispatch.dylib 0x19bf647c8 _dispatch_worker_thread + 344 4 libsystem_pthread.dylib 0x19c1602c0 _pthread_body + 128 5 libsystem_pthread.dylib 0x19c160220 _pthread_start + 44 6 libsystem_pthread.dylib 0x19c163cdc thread_start + 4 #17. Thread 0 libsystem_kernel.dylib 0x19c0d3148 semaphore_timedwait_trap + 8 1 libdispatch.dylib 0x19bf50a4c _dispatch_sema4_timedwait$VARIANT$armv81 + 64 2 libdispatch.dylib 0x19bf513a8 _dispatch_semaphore_wait_slow + 72 3 libdispatch.dylib 0x19bf647c8 _dispatch_worker_thread + 344 4 libsystem_pthread.dylib 0x19c1602c0 _pthread_body + 128 5 libsystem_pthread.dylib 0x19c160220 _pthread_start + 44 6 libsystem_pthread.dylib 0x19c163cdc thread_start + 4 #18. Thread 0 libsystem_kernel.dylib 0x19c0d3148 semaphore_timedwait_trap + 8 1 libdispatch.dylib 0x19bf50a4c _dispatch_sema4_timedwait$VARIANT$armv81 + 64 2 libdispatch.dylib 0x19bf513a8 _dispatch_semaphore_wait_slow + 72 3 libdispatch.dylib 0x19bf647c8 _dispatch_worker_thread + 344 4 libsystem_pthread.dylib 0x19c1602c0 _pthread_body + 128 5 libsystem_pthread.dylib 0x19c160220 _pthread_start + 44 6 libsystem_pthread.dylib 0x19c163cdc thread_start + 4 #19. Crashed: AXSpeech 0 libsystem_pthread.dylib 0x19c15e5b8 pthread_mutex_lock$VARIANT$armv81 + 102 1 CoreFoundation 0x19c4cf84c CFRunLoopSourceSignal + 68 2 Foundation 0x19cfc7280 performQueueDequeue + 464 3 Foundation 0x19cfc680c __NSThreadPerformPerform + 136 4 CoreFoundation 0x19c4d22bc __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 24 5 CoreFoundation 0x19c4d223c __CFRunLoopDoSource0 + 88 6 CoreFoundation 0x19c4d1b74 __CFRunLoopDoSources0 + 256 7 CoreFoundation 0x19c4cca60 __CFRunLoopRun + 1004 8 CoreFoundation 0x19c4cc354 CFRunLoopRunSpecific + 436 9 Foundation 0x19ce99fcc -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 300 10 libAXSpeechManager.dylib 0x1ac16c94c -[AXSpeechThread main] + 264 11 Foundation 0x19cfc66e4 __NSThread__start__ + 984 12 libsystem_pthread.dylib 0x19c1602c0 _pthread_body + 128 13 libsystem_pthread.dylib 0x19c160220 _pthread_start + 44 14 libsystem_pthread.dylib 0x19c163cdc thread_start + 4 #20. AXSpeech 0 (Missing) 0x1071ba524 (Missing) 1 (Missing) 0x1071b3e7c (Missing) 2 (Missing) 0x10718fba4 (Missing) 3 (Missing) 0x107184bc8 (Missing) 4 libdyld.dylib 0x19bf95908 dlopen + 176 5 CoreFoundation 0x19c5483e8 _CFBundleDlfcnLoadBundle + 140 6 CoreFoundation 0x19c486918 _CFBundleLoadExecutableAndReturnError + 352 7 Foundation 0x19ced5734 -[NSBundle loadAndReturnError:] + 428 8 TextToSpeech 0x1abfff800 TTSSpeechUnitTestingMode + 1020 9 libdispatch.dylib 0x19bf817d4 _dispatch_client_callout + 16 10 libdispatch.dylib 0x19bf52040 _dispatch_once_callout + 28 11 TextToSpeech 0x1abfff478 TTSSpeechUnitTestingMode + 116 12 libobjc.A.dylib 0x19b7173cc CALLING_SOME_+initialize_METHOD + 24 13 libobjc.A.dylib 0x19b71cee0 initializeNonMetaClass + 296 14 libobjc.A.dylib 0x19b71e640 initializeAndMaybeRelock(objc_class*, objc_object*, mutex_tt<false>&, bool) + 260 15 libobjc.A.dylib 0x19b7265a4 lookUpImpOrForward + 244 16 libobjc.A.dylib 0x19b733858 _objc_msgSend_uncached + 56 17 libAXSpeechManager.dylib 0x1ac167324 -[AXSpeechManager _initialize] + 68 18 Foundation 0x19cfc68d4 __NSThreadPerformPerform + 336 19 CoreFoundation 0x19c4d22bc __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 24 20 CoreFoundation 0x19c4d223c __CFRunLoopDoSource0 + 88 21 CoreFoundation 0x19c4d1b74 __CFRunLoopDoSources0 + 256 22 CoreFoundation 0x19c4cca60 __CFRunLoopRun + 1004 23 CoreFoundation 0x19c4cc354 CFRunLoopRunSpecific + 436 24 Foundation 0x19ce99fcc -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 300 25 libAXSpeechManager.dylib 0x1ac16c94c -[AXSpeechThread main] + 264 26 Foundation 0x19cfc66e4 __NSThread__start__ + 984 27 libsystem_pthread.dylib 0x19c1602c0 _pthread_body + 128 28 libsystem_pthread.dylib 0x19c160220 _pthread_start + 44 29 libsystem_pthread.dylib 0x19c163cdc thread_start + 4I change my code like this, It still has the same problem- (void)stopSpeech { if (self.synthesizer != nil && [self.synthesizer isPaused]) { return; } // if ([self.synthesizer isSpeaking]) { // BOOL isSpeech = [self.synthesizer stopSpeakingAtBoundary:AVSpeechBoundaryImmediate]; // if (!isSpeech) { // [self.synthesizer stopSpeakingAtBoundary:AVSpeechBoundaryWord]; // } // } if (self.synthesizer != nil) { [self.synthesizer stopSpeakingAtBoundary:AVSpeechBoundaryImmediate]; // if (!isSpeech) { // [self.synthesizer stopSpeakingAtBoundary:AVSpeechBoundaryWord]; // } self.stopBlock ? self.stopBlock() : nil; } }

Hello, My app is crashing a lot with this issue. I can't reproduce the problem but I can see it occurs at the user's devices. The Crashlytics report shows the following lines:Crashed: AXSpeech 0 libsystem_pthread.dylib 0x1824386bc pthread_mutex_lock$VARIANT$mp + 278 1 CoreFoundation 0x1826d3a34 CFRunLoopSourceSignal + 68 2 Foundation 0x18319ec90 performQueueDequeue + 468 3 Foundation 0x18325a020 __NSThreadPerformPerform + 136 4 CoreFoundation 0x1827b7404 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 24 5 CoreFoundation 0x1827b6ce0 __CFRunLoopDoSources0 + 456 6 CoreFoundation 0x1827b479c __CFRunLoopRun + 1204 7 CoreFoundation 0x1826d4da8 CFRunLoopRunSpecific + 552 8 Foundation 0x183149674 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 304 9 libAXSpeechManager.dylib 0x192852830 -[AXSpeechThread main] + 284 10 Foundation 0x183259efc __NSThread__start__ + 1040 11 libsystem_pthread.dylib 0x182435220 _pthread_body + 272 12 libsystem_pthread.dylib 0x182435110 _pthread_body + 290 13 libsystem_pthread.dylib 0x182433b10 thread_start + 4The crash occurs in different threads (never at main thread)It is driving me crazy... Can anybody help me?Thanks a lot

Hello, My goal is to enable users to perform a freeform search request for any product I sell using a spoken phrase, for example, "Hey Siri, search GAMING CONSOLES on MyCatalogApp". The result would launch MyCatalogApp and navigate to a search results page displaying gaming consoles. I have defined a SearchIntent (using the .system.search schema) and a Shortcut to accomplish this. However, Siri doesn't seem to be able to correctly parse the spoken phrase, extract the search string, and provide it as the critiria term within SearchIntent. What am I doing wrong? Here is the SearchIntent. Note the print() statement outputs the search string--which in the scenario above would be "GAMING CONSOLES"--but it doesn't work. import AppIntents @available(iOS 17.2, *) @AppIntent(schema: .system.search) struct SearchIntent: ShowInAppSearchResultsIntent { static var searchScopes: [StringSearchScope] = [.general] @Parameter(title: "Criteria") var criteria: StringSearchCriteria static var title: LocalizedStringResource = "Search with MyCatalogApp" @MainActor func perform() async throws -> some IntentResult { let searchString = criteria.term print("**** Search String: \(searchString) ****") // tmp debugging try await MyCatalogSearchHelper.search(for: searchString) // fetch results via my async fetch API return .result() } } Here's the Shortcuts definition: import AppIntents @available(iOS 17.2, *) struct Shortcuts: AppShortcutsProvider { @AppShortcutsBuilder static var appShortcuts: [AppShortcut] { AppShortcut( intent: SearchIntent(), phrases: ["Search for \(\.$criteria) on \(.applicationName)."], shortTitle: "Search", systemImageName: "magnifyingglass" ) } } Thanks for any help!

Hi, I'm building ANF (Autonomous Native Forge) — a cloud-free, 4-agent autonomous software production pipeline running on local hardware with local LLM inference. No middleware, pure Node.js native. Currently running on NVIDIA Blackwell GB10 with vLLM + DeepSeek-R1-32B. Now porting to Apple Silicon. Three technical questions: How production-ready is mlx-lm's OpenAI-compatible API server for long context generation (32K tokens)? What's the recommended approach for KV Cache management with Unified Memory architecture — any specific flags or configurations for M4 Ultra? MLX vs GGUF (llama.cpp) for a multi-agent pipeline where 4 agents call the inference endpoint concurrently — which handles parallel requests better on Apple Silicon? GitHub: github.com/trgysvc/AutonomousNativeForge Any guidance appreciated.

I am developing a macOS application using SwiftUI (with an iOS version as well). One feature we are exploring is displaying an avatar that reads or speaks dynamically generated text produced by an AI service. The basic flow would be: Text generated by an AI service Text converted to speech using a TTS engine An avatar (2D or 3D) rendered in the app that animates lip movement synchronized with the speech Ideally the avatar would render locally on the device. Questions: What Apple frameworks would be most appropriate for implementing a speaking avatar? SceneKit RealityKit SpriteKit (for 2D avatars) Is there any recommended way to drive lip-sync animation from speech audio using Apple frameworks? Does AVSpeechSynthesizer expose phoneme or viseme timing information that could be used for avatar animation? If such timing information is not available, what is the recommended approach for synchronizing character mouth animation with speech audio on macOS/iOS? Are there examples of real-time character animation synchronized with speech on macOS/iOS? Any architectural guidance or references would be greatly appreciated.

I keep getting the error “An unsupported language or locale was used.” Is there any documentation that specifies the accepted languages or locales in Foundation model?

I am trying to use @AppIntent(schema: .system.search) to search in my app via a Siri voice command, but I want to be able to return a .result that does not open the app, yet still get the model training benefits from the schema. Very new to this, this is my first app, so I would appreciate some guidance. I haven't gotten to the voice part, I tested on Shortcuts. Do I need to do AppIntents without the schema and wait until there is a search schema that does not open the app, or should I be using a different schema? What am I missing?

I opened a new project, iOS app, in XCode and then tabbed into the system_search snippet and built the project and got a build error. I can't imagine this was intended, at least not for new developers to the ecosystem like me. I solved it by tweaking a configuration I don't really understand advised here: https://github.com/apple/swift-openapi-generator/issues/796, hopefully that's a valid workaround

I don't know if these forums are any good for rumors or plans, but does anybody know whether or not Apple plans to release a library for training reinforcement learning? It would be handy, implementing games in Swift, for example, to be able to train the computer players on the same code.

We get two error message in Xcode debug. apple.model.catalog we get 1 time at startup, and the nw_protocol_instance_set_output_handler Not calling remove_input_handler on 0x152ac3c00:udp we get on sartup and some time during running of the app. I have tested cutoff repos WS eg. But nothing helpss, thats for the nw_protocol. We have a fondationmodel in a repo but we check if it is available if not we do not touch it. Please help me? nw_protocol_instance_set_output_handler Not calling remove_input_handler on 0x152ac3c00:udp com.apple.modelcatalog.catalog sync: connection error during call: Error Domain=NSCocoaErrorDomain Code=4099 "The connection to service named com.apple.modelcatalog.catalog was invalidated: Connection init failed at lookup with error 159 - Sandbox restriction." UserInfo={NSDebugDescription=The connection to service named com.apple.modelcatalog.catalog was invalidated: Connection init failed at lookup with error 159 - Sandbox restriction.} reached max num connection attempts: 1 The function we have in the repo is this: public actor FoundationRepo: JobDescriptionChecker, SubskillSuggester { private var session: LanguageModelSession? private let isEnabled: Bool private let shouldUseLocalFoundation: Bool private let baseURLString = "https://xx.xx.xxx/xx" private let http: HTTPPac public init(http: HTTPPac, isEnabled: Bool = true) { self.http = http self.isEnabled = isEnabled self.session = nil guard isEnabled else { self.shouldUseLocalFoundation = false return } let model = SystemLanguageModel.default guard model.supportsLocale() else { self.shouldUseLocalFoundation = false return } switch model.availability { case .available: self.shouldUseLocalFoundation = true case .unavailable(.deviceNotEligible), .unavailable(.appleIntelligenceNotEnabled), .unavailable(.modelNotReady): self.shouldUseLocalFoundation = false @unknown default: self.shouldUseLocalFoundation = false } } So here we decide if we are going to use iPhone ML or my backend-remote?

Environment: Hardware: Mac M4 OS: macOS Sequoia 15.7.4 TensorFlow-macOS Version: 2.16.2 TensorFlow-metal Version: 1.2.0 Description: When using the tensorflow-metal plug-in for GPU acceleration on M4, the ReLU activation function (both as a layer and as an activation argument) fails to correctly clip negative values to zero. The same code works correctly when forced to run on the CPU. Reproduction Script: import os import numpy as np import tensorflow as tf # weights and biases = -1 weights = [np.ones((10, 5)) * -1, np.ones(5) * -1] # input = 1 data = np.ones((1, 10)) # comment this line => GPU => get negative values # uncomment this line => CPU => no negative values # tf.config.set_visible_devices([], 'GPU') # create model model = tf.keras.Sequential([ tf.keras.layers.Input(shape=(10,)), tf.keras.layers.Dense(5, activation='relu') ]) # set weights model.layers[0].set_weights(weights) # get output output = model.predict(data) # check if negative is present print(f"min value: {output.min()}") print(f"is negative present? {np.any(output < 0)}")

Subject: Technical Report: Float32 Precision Ceiling & Memory Fragmentation in JAX/Metal Workloads on M3 To: Metal Developer Relations Hello, I am reporting a repeatable numerical saturation point encountered during sustained recursive high-order differential workloads on the Apple M3 (16 GB unified memory) using the JAX Metal backend. Workload Characteristics: Large-scale vector projections across multi-dimensional industrial datasets Repeated high-order finite-difference calculations Heavy use of jax.grad and lax.cond inside long-running loops Observation: Under these conditions, the Metal/MPS backend consistently enters a terminal quantization lock where outputs saturate at a fixed scalar value (2.0000), followed by system-wide NaN propagation. This appears to be a precision-limited boundary in the JAX-Metal bridge when handling high-order operations with cubic time-scale denominators. have identified the specific threshold where recursive high-order tensor derivatives exceed the numerical resolution of 32-bit consumer architectures, necessitating a migration to a dedicated 64-bit industrial stack. I have prepared a minimal synthetic test script (randomized vectors only, no proprietary logic) that reliably reproduces the allocator fragmentation and saturation behavior. Let me know if your team would like the telemetry for XLA/MPS optimization purposes. Best regards, Alex Severson Architect, QuantumPulse AI

Looking for help with or to help with, due to the pending document enhancement, the Vibe Coders edition of cml editor. Also for more information on how to use the .mlkey whether or not my model is suppose to say IOs18 when I am planning to use it on Mac Apple Intelligence seems to think coreML is for iOS but are the capabilities extended when running NPU on the book? How to use this graph. coming in hot sorry. btw. there are 100s of feedback and crash reports sent in form me for additional info? I attached a image that might help with updating Tags

When doing some exploratory research into using Apple Intelligence in our aviation-focused application, I noticed that there were several times that key phases would be marked as inappropriate. I tried to stifle these using prompts and rules but couldn't get it to take hold. I was encouraged by an Apple employee to go ahead and post this so that the AI team can use the feedback. There were several terms that triggered this warning, but the two that were most prominent were: 'Tailwind' 'JFK' or 'KJFK' (NY airport ICAO/IATA codes)