Audio Q&A

Why voice processing enabled on AVAudioInputNode makes output audio noticable lower than without it and how to overcome it using voice processing enabled

In iOS 27, when a user has configured a custom AirPods EQ (three-band lows/mids/highs), does that EQ apply on top of audio output from AVAudioEngine in my app, or only to media playback through AVPlayer/MPMusicPlayerController? My app generates therapeutic frequencies (e.g. 528 Hz pure tones) where precise frequency output matters - is there any API to detect or opt out of the system AirPods EQ?

For a sleep app running 8–12 hours in background, is AVAudioSourceNode with a real-time render block more power-efficient than looping a pre-encoded audio file via AVAudioPlayerNode? I want to migrate from files to procedural synthesis but not at the cost of battery. What does Instruments / Energy Log show as the typical CPU overhead difference, and is there Apple guidance on this trade-off?

Is it possible to link a HFBluetooth input device with the output iPhone speaker while Live Listen is active?

Is it possible to port a bluetooth microphone (BluetoothHFP) to MFi hearing aids connected to an iPhone. When I try to do that the BluetoothHFP grabs input and output. When I pull the output away (BluetoothA2DP) it gives up the input.

We are working on a voice app that uses ASR/TTS on the backend and run into some difficulties in noisy environments. We have compiled the DeepFilterNet3 library into an XCFramework and are using that on the app, but it's sometimes a bit ambitious with trimming out noise and removes some of the voice. Is there any way to make use of Apple's on-device voice isolation mic mode? I see that we can detect the user's mic mode, but we cannot programmatically set it. While we could prompt the user to enable it manually, this adds a bit of friction to the user experience. Do you have any suggestions for enabling voice isolation, or for performing denoising in general?

We are using AEC in our voice app and it mostly works. However, when the experience begins we play a greeting through the speaker, and the initial few hundred milliseconds of the greeting are being captured by the inputNode. This is throwing off our ASR/TTS. For now, we've disabled audio capture while playing audio, but would prefer to be able to capture all audio with echo cancellation working. Below is some relevant code snippets. Do you have any suggestions to get AEC working more quickly? I've tried a few things like enabling voice processing before setting the audio session to active. public init() { recorderNode = engine.inputNode speakerNode = engine.outputNode mainMixerNode = engine.mainMixerNode engine.attach(audioPlayer) engine.connect( audioPlayer, to: mainMixerNode, format: nil ) playbackFormat = mainMixerNode.outputFormat(forBus: 0) } public func setupAudioSession() async throws(AudioError) { do { let audioSession = AVAudioSession.sharedInstance() try audioSession.setCategory( .playAndRecord, mode: .voiceChat, policy: .default, options: [ .defaultToSpeaker, .allowBluetoothHFP, ] ) try audioSession.setActive(true) } catch { throw .audioSessionSetupFailed(error) } do { try recorderNode.setVoiceProcessingEnabled(true) try speakerNode.setVoiceProcessingEnabled(true) } catch { throw .enableVoiceProcessingFailed(error) } }

Am I correct in my understanding that the new MusicUnderstanding is not intended to be able to support analysis of Apple Music streams via MusicKit or in other ways?

I have a voice app that is both playing and recording audio. I have enabled voice processing and am setting AVAudioSession.Category to .playAndRecord and AVAudioSession.Mode to .voiceChat. When the experience first launches, we play a greeting. The first few hundred milliseconds of that greeting are being captured by the inputNode before AEC seems to start working. Is there any way to get AEC working the entire time? For now we've had to disable recording while we're playing audio, but would prefer to both play and record simultaneously. Here's some code snippets: public init(denoiseModelPath: URL? = nil) { noiseReducer = denoiseModelPath.flatMap { NoiseReducer(modelPath: $0) } recorderNode = engine.inputNode speakerNode = engine.outputNode mainMixerNode = engine.mainMixerNode engine.attach(audioPlayer) engine.connect( audioPlayer, to: mainMixerNode, format: nil ) playbackFormat = mainMixerNode.outputFormat(forBus: 0) } public func setupAudioSession() async throws(AudioError) { do { let audioSession = AVAudioSession.sharedInstance() try audioSession.setCategory( .playAndRecord, mode: .voiceChat, policy: .default, options: [ .defaultToSpeaker, .allowBluetoothHFP, ] ) try audioSession.setActive(true) } catch { throw .audioSessionSetupFailed(error) } do { try recorderNode.setVoiceProcessingEnabled(true) try speakerNode.setVoiceProcessingEnabled(true) } catch { throw .enableVoiceProcessingFailed(error) } }

We are working on a voice app that uses ASR/TTS on the backend and run into some difficulties in noisy environments. We have compiled the DeepFilterNet3 library into an XCFramework and are using that on the app, but it's sometimes a bit ambitious with trimming out noise and removes some of the voice. Is there any way to make use of Apple's on-device voice isolation mic mode? I see that we can detect the user's mic mode, but we cannot programmatically set it. While we could prompt the user to enable it manually, this adds a bit of friction to the user experience. Do you have any suggestions for enabling voice isolation, or for performing denoising in general?

Why are none of my posts showing up in the Audio Q&A? There are no errors or any useful messaging... my posts just aren't showing.

My macOS app provides real-time microphone level monitoring, and I also have a macOS widget that shows recent audio level status. Since WidgetKit widgets are snapshot/timeline based rather than continuously live SwiftUI views, is there any new capability in macOS 27 that allows a widget to show more responsive or animated audio-level changes? If true real-time animation is still not supported in widgets, what is Apple’s recommended approach for displaying frequently changing signal levels, such as audio meters, in a macOS widget?

My app has a sleep timer (up to 12 hours). Can I drive a Live Activity countdown in the Dynamic Island purely from the foreground/background app process using Activity.update(), without requiring a push notification server? And does Text(timerInterval:) in SwiftUI continue counting down accurately when the app process is suspended by the OS?

Is it appropriate to use the Foundation Models framework on-device to power a natural language sound recommendation feature (e.g. user types 'I can't focus' → model returns a frequency/playlist recommendation from a predefined set)? Or is this use case better served by a custom Core ML model? What's the recommended on-device inference approach in iOS 27 for this type of structured recommendation with a small output space?

With SiriKit deprecated in iOS 27, I'm implementing App Intents from scratch for a frequency/ambient sound app. Should I conform to AudioPlaybackIntent from the Media framework or build a custom AppIntent? Specifically, I need intents for: play a named frequency, start sleep timer with duration, stop playback. Is there a recommended Intent entity type for audio content that isn't music (i.e. not MusicKit-backed)?

I am experimenting with SpeechTranscriber and am curious if I can get quicker results when using buffered audio, rather than a file. The use case is a voice ordering experience for a restaurant. When I've been playing with it, it takes about 3 seconds for faster results and 7-8 seconds for accurate results. Is there any way to bring this down a bit? In this WWDC demo, the results appear nearly instantaneously. I'm curious how to replicate this in my app. I presume DicationTranscriber is faster, but how is siri detecting when the user stops speaking? Is it custom code, or is it using SpeechDetector? I tried using SpeechDetector with SpeechTranscriber but the detector didn't emit any results and seemed to slow down the results of SpeechTranscriber. I also assumed SpeechTranscriber makes more sense than DictationTranscriber in this use case, but want to confirm.

Any chance of more modern framework for building AUv3's coming anytime soon?

When the host app of a host/AUv3 extension pair tries to find the audio extension it fails. The system should have scanned the bundle for the extension, but this appears to fail. This basically happens in SimplePlayEngine.initComponent. I've seen this intermittent problem for years. There are many sometimes-effective workarounds that I've gathered and used, but the problem still exists. It is not a problem of validation -- the same AU can validate in some run cycles and then not be found via the host app in others. I've got a Feedback Assistant ticket for the latest iteration of the problem (FB22860885), which has affected beta testers. Usually I only see this when developing with Xcode myself. So even though the testers' app lives in ~/Applications it is still failing for some of them.

This question overlaps with a newer post that contains additional implementation details To avoid splitting discussion, please see: “MusicKit.js / WKWebView: Are audible hitches during seek a known WebKit or Apple Media Stack limitation?” We are building a music practice application that uses MusicKit.js inside a WKWebView on iOS and macOS. Our application requires tight musical looping, where playback repeatedly seeks back to a loop start position while a song is playing. We have observed that: • Seeking to a nearby position within already-buffered audio using HTMLAudioElement.currentTime causes an audible interruption or hitch. • Changing HTMLAudioElement.playbackRate during playback also appears to interrupt playback briefly. • These interruptions make seamless looping difficult to achieve. For comparison, a very similar JavaScript architecture running in a Chromium-based WebView on Android exhibits significantly smoother seek behavior and supports near-seamless looping. Our questions are: Is it expected that changing currentTime or playbackRate on the underlying audio element causes the playback pipeline to be interrupted or rebuilt on Apple platforms? Are there any recommended WebKit, WKWebView, or MusicKit.js techniques that can reduce or eliminate these interruptions when seeking within already-buffered media? Would native MusicKit playback provide lower-latency or more seamless seeking than MusicKit.js, or do both approaches ultimately rely on the same playback pipeline? Our use case is musical practice, where even interruptions of a few tens of milliseconds are noticeable. Any guidance on achieving the smoothest possible loop-seeking behavior on Apple platforms would be greatly appreciated.

We are building a musician practice application that uses MusicKit.js inside a WKWebView on iOS and macOS. One of our core use cases is musical looping, where playback repeatedly seeks back to a loop start position while a song is playing. A typical loop length is 1-4 seconds, making interruptions that might be acceptable for general media playback very noticeable in our use case. For looping we currently use MusicKit’s seekToTime() API. We have also experimented with audioElement.fastSeek() and directly setting audioElement.currentTime. In all cases, seeking to a nearby position during playback produces an audible hitch or pause, even when the destination appears to already be buffered. We have observed similar interruptions when changing the playback rate during playback. For comparison, the same JavaScript bridge architecture running in a Chromium-based WebView on Android exhibits significantly smoother seek behavior and supports near-seamless looping. At this point, the WebView media stack appears to be the primary platform difference we have identified. We are currently using MusicKit.js v2. We are aware that v3 is available, but have not found anything suggesting this behavior has changed. As part of our investigation, we also experimented with Apple’s native MusicKit APIs. We observed that seeking and playback-rate changes can still produce audible interruptions, although playback-rate changes behave noticeably better than our MusicKit.js / HTML audio implementation. This led us to wonder whether the behavior originates from the underlying Apple media pipeline rather than MusicKit.js itself. Our questions are: Is it expected that seeking during playback, whether through seekToTime(), fastSeek(), or currentTime, causes the playback pipeline to be interrupted or rebuilt on Apple platforms? Are there known differences in WebKit’s media pipeline, buffering, or seek handling that would explain why the same implementation exhibits smoother seek behavior in a Chromium-based WebView? If so, are there recommended WebKit, WKWebView, or MusicKit.js techniques to achieve smoother, near-seamless looping? Given that we observe similar, though less pronounced, behavior when using native MusicKit APIs, is this a known characteristic or limitation of the underlying Apple media stack? If so, are there any APIs or implementation patterns recommended for applications that require tight musical looping and low-latency seeks? Any guidance would be greatly appreciated. Our goal is to provide musicians with seamless looping for practice and transcription, where even very small interruptions are noticeable.