As the title suggests I am using AVAudioEngine for SpeechRecognition input & AVAudioPlayer for sound output. Apple says in this talk https://developer.apple.com/videos/play/wwdc2019/510 that the setVoiceProcessingEnabled function very usefully cancels the output from speaker to the mic. I set voiceProcessing on the Input and output nodes. It seems to work however the volume is low, even when the system volume is turned up. Any solution to this would be much appreciated.

AVSpeechSynthesizer was not working. it was working perfect before. below is my code objective - c. -(void)playVoiceMemoforMessageEVO:(NSString*)msg { [[AVAudioSession sharedInstance] overrideOutputAudioPort:AVAudioSessionPortOverrideSpeaker error:nil]; AVSpeechSynthesizer *synthesizer = [[AVSpeechSynthesizer alloc]init]; AVSpeechUtterance *speechutt = [AVSpeechUtterance speechUtteranceWithString:msg]; speechutt.volume=90.0f; speechutt.rate=0.50f; speechutt.pitchMultiplier=0.80f; [speechutt setRate:0.3f]; speechutt.voice = [AVSpeechSynthesisVoice voiceWithLanguage:@"en-us"]; [synthesizer speakUtterance:speechutt]; } please help me to solve this issue.

Our app is a game written in Unity where we have most of our audio playback handled by Unity. However, one of our game experiences utilized microphone input for speech recognition, and so in order for us to perform echo cancellation (while the game has audio playback), we setup an audio stream from Unity to native Swift code that performs the mixing of the input/output nodes. We however found that by streaming the audio buffer to our AVAudioSession: The volume of the audio playback appears to output differently When capturing a screen recording of the app, the audio playback being played from AVAudioSession does not get captured at all. Looking to figure out what could be causing the discrepency in playback as well as capture behaviour during screen recordings. We setup the AVAudioSession with this configuration: AVAudioSession.sharedInstance().setCategory(AVAudioSession.Category.playAndRecord, options: .mixWithOthers) with inputNode.setVoiceProcessingEnabled(true) after connecting our IO and mixer nodes. Any suggestions or ideas on what to look out for would be appreciated!

I am using AVSpeechSynthesizer to get audio buffer and play, I am using AVAudioEngine and AVAudioPlayerNode to play the buffer. But I am getting error. [avae] AVAEInternal.h:76 required condition is false: [AVAudioPlayerNode.mm:734:ScheduleBuffer: (_outputFormat.channelCount == buffer.format.channelCount)] 2023-05-02 03:14:35.709020-0700 AudioPlayer[12525:308940] *** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'required condition is false: _outputFormat.channelCount == buffer.format.channelCount' Can anyone please help me to play the AVAudioBuffer from AVSpeechSynthesizer write method?

We are developing an app that uses external hardware to measure analogue hearing-loop performance . It uses audio jack on phone/iPad. With the new hardware on iPad using USB-C , we have noticed that the same input , one with lighting adapter and one with usb-C adapter - both produce way different input levels. The USB-C is ~23dB lower, with the same code and settings. That's almost 10x difference. Is there any way to control the USB-C adapter? am I missing something ? The code simply uses AVAudioInputNode and block attached to it via self.inputNode.installTap we do adjust gain to 1.0 let gain: Float = 1.0 try session.setInputGain(gain) But that still does not help. I wish there was an apple lab I could go to , to speak to engineers about it.

I built a simple Swift game in SpriteKit. I have audio on the simulator but no audio or sound effects on the device. Any suggestions?

I've noticed that enabling voice processing on AVAudioInputNode change the node's format - most noticeably channel count. let inputNode = avEngine.inputNode print("Format #1: \(inputNode.outputFormat(forBus: 0))") // Format #1: <AVAudioFormat 0x600002bb4be0:  1 ch,  44100 Hz, Float32> try! inputNode.setVoiceProcessingEnabled(true) print("Format #2: \(inputNode.outputFormat(forBus: 0))") // Format #2: <AVAudioFormat 0x600002b18f50:  3 ch,  44100 Hz, Float32, deinterleaved> Is this expected? How can I interpret these channels? My input device is an aggregate device where each channel comes from a different microphone. I then record each channels to separate files. But when voice processing messes up with the channels layout, I cannot rely on this anymore.

I have a timer in one of my apps. I now want to add audio that plays at the end of the timer. It's a workout app and the sound should remind the user that it is time for the next exercise. The audio should duck music playback (Apple Music / Spotify) and also work in background. Background audio is enabled for the app. I am not able to achieve everything at the same time. I set the audio session to category playback with options duckOthers. do { try AVAudioSession.sharedInstance().setCategory( .playback, options: .duckOthers ) } catch { print(error) } For playback I just use the AVAudioPlayer. When the user starts the timer, i schedule a timer in the future and play the sound. While this works perfectly in the foreground, the sound is not played back when going to background, as timers are not fired in the background, but rather when the user puts the app back in foreground. I have also tried using AVAudioEngine and AVAudioPlayerNode, as the latter can start playback delayed. The case from above works now, but the audio ducking begins immediately when initialising the AVAudioEngine, which is also not what i want. Is there any other approach that I am not aware of?

I am analysing sounds by tapping the mic on the Mac. All is working well, but it disrupts other (what I assume) are low priority sounds e.g. dragging an item off the dock, sending a message is messages, speaking something in Shortcuts or Terminal. Other sounds like music.app playing, Siri speaking are not disrupted. The disruption sounds like the last part of the sound being repeated two extra times, very noticeable. This is the code: import Cocoa import AVFAudio class AudioHelper: NSObject { let audioEngine = AVAudioEngine() func start() async throws { audioEngine.inputNode.installTap(onBus: 0, bufferSize: 8192, format: nil) { buffer, time in } try audioEngine.start() } } I have tried increasing the buffer, changing the qos to utility (in the hope the sound analysis would become less important than the disrupted sounds),running on a non-main thread, but no luck. MacOS 13.4.1 Any assistance would be appreciated.

I cannot seem to create an AVAudioFile from a URL to be played in an AVAudioEngine. Here is my complete code, following the documentation. import UIKit import AVKit import AVFoundation class ViewController: UIViewController { let audioEngine = AVAudioEngine() let audioPlayerNode = AVAudioPlayerNode() override func viewDidLoad() { super.viewDidLoad() streamAudioFromURL(urlString: "https://samplelib.com/lib/preview/mp3/sample-9s.mp3") } func streamAudioFromURL(urlString: String) { guard let url = URL(string: urlString) else { print("Invalid URL") return } let audioFile = try! AVAudioFile(forReading: url) let audioEngine = AVAudioEngine() let playerNode = AVAudioPlayerNode() audioEngine.attach(playerNode) audioEngine.connect(playerNode, to: audioEngine.outputNode, format: audioFile.processingFormat) playerNode.scheduleFile(audioFile, at: nil, completionCallbackType: .dataPlayedBack) { _ in /* Handle any work that's necessary after playback. */ } do { try audioEngine.start() playerNode.play() } catch { /* Handle the error. */ } } } I am getting the following error on let audioFile = try! AVAudioFile(forReading: url) Thread 1: Fatal error: 'try!' expression unexpectedly raised an error: Error Domain=com.apple.coreaudio.avfaudio Code=2003334207 "(null)" UserInfo={failed call=ExtAudioFileOpenURL((CFURLRef)fileURL, &_extAudioFile)} I have tried many other .mp3 file URLs as well as .wav and .m4a and none seem to work. The documentation makes this look so easy but I have been trying for hours to no avail. If you have any suggestions, they would be greatly appreciated!

Hi, I have multiple audio files I want to decide which channel goes to which output. For example, how to route four 2-channel audio files to an 8-channel output. Also If I have an AVAudioPlayerNode playing a 2-channel track through headphones, can I flip the channels on the output for playback, i.e flip left and right? I have read the following thread which seeks to do something similar, but it is from 2012 and I do not quite understand how it would work in modern day. Many thanks, I am a bit stumped.

I'm trying to change the audio input (microphone) between all the available devices from AVAudioSession.sharedInstance().availableInputs. I'm using AVAudioSession.routeChangeNotification to get automatic route changes when devices get connected/disconnected and change the preferred input with setPreferredInput, then I restart my audioEngine and it works fine. But when I try to change the preferred input programmatically It doesn't change the audio capture inputNode. But keeps the last connected device and capturing. Even the AVAudioSession.sharedInstance().currentRoute.inputs changes but the audioEngine?.inputNode doesn't change to setPreferredInput call. WhatsApp seems to have done that without any issues. Any suggestions or leads are highly appreciated. Thanks.

Hi After IOS 16 update users are experiencing audio distortion when attempting to change Pitch and/or Tempo in the app which uses AVAudiounitTimepitch to achieve this. Prior to IOS 16 it has worked quite admirably. I see suggestions to change the algorithm used but am able to achieve this. Any pointers are appreciated

According to the Apple docs, you should be able to connect audio nodes to an AVAudioEngine instance during runtime, but I'm getting a crash while trying to do so, in particular, when trying to connect instances of AVAudioUnitTimePitch or AVAudioUnitVarispeed to an AVAudioEngine with manual rendering mode enabled. The error message I get is: Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'player started when in a disconnected state' In my code, first, I configure the audio engine: let engine = AVAudioEngine() let format = AVAudioFormat(standardFormatWithSampleRate: 48000, channels: 2)! try! engine.enableManualRenderingMode(.offline, format: format, maximumFrameCount: 1024) try! engine.start() Then, I try to attach the player to the engine: let player = AVAudioPlayerNode() configureEngine(player: player, useVarispeed: true) player.play() // this is the line that causes the crash Finally, this is the function I use to configure the engine nodes graph: func configureEngine(player: AVAudioPlayerNode, useVarispeed: Bool) { engine.attach(player) guard useVarispeed else { engine.connect(player, to: engine.mainMixerNode, format: format) return } let varispeed = AVAudioUnitVarispeed() engine.attach(varispeed) engine.connect(player, to: varispeed, format: format) engine.connect(varispeed, to: engine.mainMixerNode, format: format) } If I pass false as the value for the useVarispeed parameter, the crash goes away. What is even more interesting is that if I add a dummy player node before starting the engine, the crash goes away too 🤷‍♂️ Could anyone please add clarity on what's going on here? Is this a bug or a limitation of the framework that I'm not aware of? Here's a simple project demonstrating the problem: https://github.com/rlaguilar/AVAudioEngineBug

I have the following code to connect inputNode to mainMixerNode of AVAudioEngine: public func setupAudioEngine() { self.engine = AVAudioEngine() let format = engine.inputNode.inputFormat(forBus: 0) //main mixer node is connected to output node by default engine.connect(self.engine.inputNode, to: self.engine.mainMixerNode, format: format) do { engine.prepare() try self.engine.start() } catch { print("error couldn't start engine") } engineRunning = true } But I am seeing a crash in Crashlytics dashboard (which I can't reproduce). Fatal Exception: com.apple.coreaudio.avfaudio required condition is false: IsFormatSampleRateAndChannelCountValid(format) Before calling the function setupAudioEngine I make sure the AVAudioSession category is not playback where mic is not available. The function is called where audio route change notification is handled and I check this condition specifically. Can someone tell me what I am doing wrong? Fatal Exception: com.apple.coreaudio.avfaudio 0 CoreFoundation 0x99288 __exceptionPreprocess 1 libobjc.A.dylib 0x16744 objc_exception_throw 2 CoreFoundation 0x17048c -[NSException initWithCoder:] 3 AVFAudio 0x9f64 AVAE_RaiseException(NSString*, ...) 4 AVFAudio 0x55738 AVAudioEngineGraph::_Connect(AVAudioNodeImplBase*, AVAudioNodeImplBase*, unsigned int, unsigned int, AVAudioFormat*) 5 AVFAudio 0x5cce0 AVAudioEngineGraph::Connect(AVAudioNode*, AVAudioNode*, unsigned long, unsigned long, AVAudioFormat*) 6 AVFAudio 0xdf1a8 AVAudioEngineImpl::Connect(AVAudioNode*, AVAudioNode*, unsigned long, unsigned long, AVAudioFormat*) 7 AVFAudio 0xe0fc8 -[AVAudioEngine connect:to:format:] 8 MyApp 0xa6af8 setupAudioEngine + 701 (MicrophoneOutput.swift:701) 9 MyApp 0xa46f0 handleRouteChange + 378 (MicrophoneOutput.swift:378) 10 MyApp 0xa4f50 @objc MicrophoneOutput.handleRouteChange(note:) 11 CoreFoundation 0x2a834 __CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__ 12 CoreFoundation 0xc6fd4 ___CFXRegistrationPost_block_invoke 13 CoreFoundation 0x9a1d0 _CFXRegistrationPost 14 CoreFoundation 0x408ac _CFXNotificationPost 15 Foundation 0x1b754 -[NSNotificationCenter postNotificationName:object:userInfo:] 16 AudioSession 0x56f0 (anonymous namespace)::HandleRouteChange(AVAudioSession*, NSDictionary*) 17 AudioSession 0x5cbc invocation function for block in avfaudio::AVAudioSessionPropertyListener(void*, unsigned int, unsigned int, void const*) 18 libdispatch.dylib 0x1e6c _dispatch_call_block_and_release 19 libdispatch.dylib 0x3a30 _dispatch_client_callout 20 libdispatch.dylib 0x11f48 _dispatch_main_queue_drain 21 libdispatch.dylib 0x11b98 _dispatch_main_queue_callback_4CF 22 CoreFoundation 0x51800 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ 23 CoreFoundation 0xb704 __CFRunLoopRun 24 CoreFoundation 0x1ebc8 CFRunLoopRunSpecific 25 GraphicsServices 0x1374 GSEventRunModal 26 UIKitCore 0x514648 -[UIApplication _run] 27 UIKitCore 0x295d90 UIApplicationMain 28 libswiftUIKit.dylib 0x30ecc UIApplicationMain(_:_:_:_:) 29 MyApp 0xc358 main (WhiteBalanceUI.swift) 30 ??? 0x104b1dce4 (Missing)

Hi. I'm audio/video player developer. I implemented player as AVAudioPlayerNode to handle audio buffer. Suddenly, I wondered how great it would be if AVPlayer could handle buffers. I would like to know if there is a way to handle the buffer in AVPlayer, and if not, can you disclose it in the future? Thanks.

I recently released my first ShazamKit app, but there is one thing that still bothers me. When I started I followed the steps as documented by Apple right here : https://developer.apple.com/documentation/shazamkit/shsession/matching_audio_using_the_built-in_microphone however when I was running this on iPad I receive a lot of high pitched feedback noise when I ran my app with this configuration. I got it to work by commenting out the output node and format and only use the input. But now I want to be able to recognise the song that’s playing from the device that has my app open and was wondering if I need the output nodes for that or if I can do something else to prevent the Mic. Feedback from happening. In short: What can I do to prevent feedback from happening Can I use the output of a device to recognise songs or do I just need to make sure that the microphone can run at the same time as playing music? Other than that I really love the ShazamKit API and can highly recommend to have a go with it! This is the code as documented in the above link (I just added the comments of what broke it for me) func configureAudioEngine() { // Get the native audio format of the engine's input bus. let inputFormat = audioEngine.inputNode.inputFormat(forBus: 0) // THIS CREATES FEEDBACK ON IPAD PRO let outputFormat = AVAudioFormat(standardFormatWithSampleRate: 48000, channels: 1) // Create a mixer node to convert the input. audioEngine.attach(mixerNode) // Attach the mixer to the microphone input and the output of the audio engine. audioEngine.connect(audioEngine.inputNode, to: mixerNode, format: inputFormat) // THIS CREATES FEEDBACK ON IPAD PRO audioEngine.connect(mixerNode, to: audioEngine.outputNode, format: outputFormat) // Install a tap on the mixer node to capture the microphone audio. mixerNode.installTap(onBus: 0, bufferSize: 8192, format: outputFormat) { buffer, audioTime in // Add captured audio to the buffer used for making a match. self.addAudio(buffer: buffer, audioTime: audioTime) } }

I'm working on an audio processing app and am creating an AVAudioUnit extension as a part of it. I need to train a small neural network in the app and use it to process audio in real-time in the AudioUnit. The network is mostly convolutions and is ideal for running on the GPU but it should run in real-time on the CPU. The problem that I'm currently facing is that none of the ML frameworks seem to be safe to use for inference within custom AVAudioUnit kernels. My understanding is that only C and C++ should be used in these kernels (in addition to the other rules of real-time computing). Objective-C and Swift are discouraged per the documentation. My background is primarily in ML so I'm newer to Apple development and especially new to real-time development in this ecosystem. I've investigated CoreML, MPS, BNNS/Accelerate, and MLCompute so far but I'm not certain that any of them are safe to use. Any feedback would be greatly appreciated!

I want to process audio from videos using AVAudioEngine, but I'm not sure how to read the audio from videos using AVAudioEngine. I have the following code: let videoURL = /// url pointing to a local video file. let file = try AVAudioFile(forReading: videoURL) It works fine on macOS but on iOS it fails with the error message: [default]          ExtAudioFile.cpp:193   about to throw 'typ?': open audio file [avae]            AVAEInternal.h:109   [AVAudioFile.mm:134:AVAudioFileImpl: (ExtAudioFileOpenURL((CFURLRef)fileURL, &_extAudioFile)): error 1954115647 Error Domain=com.apple.coreaudio.avfaudio Code=1954115647 "(null)" UserInfo={failed call=ExtAudioFileOpenURL((CFURLRef)fileURL, &_extAudioFile)} Reading through the header files it seems that 'typ?' corresponds to kAudioFileUnsupportedFileTypeError, so that tells me that the file type is supported on macOS but not on iOS. So my question is: How can I work with audio from video files in an AVAudioEngine based setup? I already know that I could extract the audio from videos using something like AVAssetExportSession, but that approach requires extra preprocessing time that I rather not spend.

Hi, I have been searching all over for a way to do this on macOS: playing different stereo files on different pairs of audio outputs of the same hardware device. I am currently using AVAudioEngine with two AVAudioPlayerNodes and I can mix them and change the mapping of the entire mix through the use of AudioUnitSetProperty on the engine output, but I cannot have multiple AVAudioPlayerNodes play on different outputs. Obviously, being on macOS I cannot use AVAudioSession... Thank you if anyone has any idea on how to achieve this !