My app plays music via AVAudioEngine. I'm using effects that prohibit me from using a built-in audio player. I'm using MPNowPlayingInfoCenter to provide information to the NowPlaying app on the Apple Watch. When I play a song, the Apple Watch displays the song metadata. When I hit pause on my phone, the Apple Watch shows that the track is no longer advancing via the progress wheel; however, it continues to display the pause button even though the music is paused. Ideally, it would switch to the play symbol when the music is paused. In the documentation it says you can use the playBackState var to set the playback state in macOS. There does not seem to be an equivalent in iOS. I figured maybe the watch would know to switch the pause symbol to the play symbol when the playback rate is 0, but it does not. All other functionality is working properly. The only thing that doesn't update on the watch is the pause / play symbol when the music toggles between play and pause by tapping my phone or from receiving remote command events. Thank you in advance for your help.

AVAudioUnitTimePitch.latency is 0.09s on my debug devices. It will have a little time delay during render audio using `AVAudioEngine. I just want to change the pitch during playing audio. So how can I avoid this this latency??

According to the Apple docs, you should be able to connect audio nodes to an AVAudioEngine instance during runtime, but I'm getting a crash while trying to do so, in particular, when trying to connect instances of AVAudioUnitTimePitch or AVAudioUnitVarispeed to an AVAudioEngine with manual rendering mode enabled. The error message I get is: Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'player started when in a disconnected state' In my code, first, I configure the audio engine: let engine = AVAudioEngine() let format = AVAudioFormat(standardFormatWithSampleRate: 48000, channels: 2)! try! engine.enableManualRenderingMode(.offline, format: format, maximumFrameCount: 1024) try! engine.start() Then, I try to attach the player to the engine: let player = AVAudioPlayerNode() configureEngine(player: player, useVarispeed: true) player.play() // this is the line that causes the crash Finally, this is the function I use to configure the engine nodes graph: func configureEngine(player: AVAudioPlayerNode, useVarispeed: Bool) { engine.attach(player) guard useVarispeed else { engine.connect(player, to: engine.mainMixerNode, format: format) return } let varispeed = AVAudioUnitVarispeed() engine.attach(varispeed) engine.connect(player, to: varispeed, format: format) engine.connect(varispeed, to: engine.mainMixerNode, format: format) } If I pass false as the value for the useVarispeed parameter, the crash goes away. What is even more interesting is that if I add a dummy player node before starting the engine, the crash goes away too 🤷‍♂️ Could anyone please add clarity on what's going on here? Is this a bug or a limitation of the framework that I'm not aware of? Here's a simple project demonstrating the problem: https://github.com/rlaguilar/AVAudioEngineBug

Adopting Picture in Picture in a Custom Player https://developer.apple.com/documentation/avkit/adopting_picture_in_picture_in_a_custom_player I have implemented PIP by following this link when I tap Pip button it works fine but when I am playing any video If I minimise app Picture in Picture does not start .In my case I have used AVPlayerLayer and I have set AVAudiosession category as playback .I am on iOS 14 beta 4

Hello My app record voice with Record_Engine class(AvaudioEngine). Problem: Even though i made my app go to background, app return to inactive state in few seconds(about 3 seconds after watch screen locked). Example: How can i leave my app background. like built-in record app. and my recorder class is here. //Recorder.swift class Record_Engine : ObservableObject {    @Published var recording_file : AVAudioFile!    private var engine: AVAudioEngine!    private var mixerNode: AVAudioMixerNode!        init() {      setupSession()      setupEngine()    }            fileprivate func setupSession() {      let session = AVAudioSession.sharedInstance()      do {        try session.setCategory(AVAudioSession.Category.playAndRecord, mode: .default)        try session.setActive(true)      } catch {        print(error.localizedDescription)      }    }        fileprivate func setupEngine() {      engine = AVAudioEngine()      mixerNode = AVAudioMixerNode()      mixerNode.volume = 0      engine.attach(mixerNode)      makeConnections()      engine.prepare()    }    fileprivate func makeConnections() {      let inputNode = engine.inputNode      let inputFormat = inputNode.outputFormat(forBus: 0)      engine.connect(inputNode, to: mixerNode, format: inputFormat)      let mainMixerNode = engine.mainMixerNode      let mixerFormat = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: inputFormat.sampleRate, channels: 1, interleaved: false)      engine.connect(mixerNode, to: mainMixerNode, format: mixerFormat)    }        func startRecording() throws {      let tapNode: AVAudioNode = mixerNode        let format = tapNode.outputFormat(forBus: 0)        self.recording_file = try AVAudioFile(forWriting: get_file_path(), settings: format.settings)        tapNode.installTap(onBus: 0, bufferSize: 8192, format: format, block: {          (buffer, time) in          do {try self.recording_file.write(from: buffer)}          catch {print(error.localizedDescription)}        })        try engine.start()    } }

I have the following code to connect inputNode to mainMixerNode of AVAudioEngine: public func setupAudioEngine() { self.engine = AVAudioEngine() let format = engine.inputNode.inputFormat(forBus: 0) //main mixer node is connected to output node by default engine.connect(self.engine.inputNode, to: self.engine.mainMixerNode, format: format) do { engine.prepare() try self.engine.start() } catch { print("error couldn't start engine") } engineRunning = true } But I am seeing a crash in Crashlytics dashboard (which I can't reproduce). Fatal Exception: com.apple.coreaudio.avfaudio required condition is false: IsFormatSampleRateAndChannelCountValid(format) Before calling the function setupAudioEngine I make sure the AVAudioSession category is not playback where mic is not available. The function is called where audio route change notification is handled and I check this condition specifically. Can someone tell me what I am doing wrong? Fatal Exception: com.apple.coreaudio.avfaudio 0 CoreFoundation 0x99288 __exceptionPreprocess 1 libobjc.A.dylib 0x16744 objc_exception_throw 2 CoreFoundation 0x17048c -[NSException initWithCoder:] 3 AVFAudio 0x9f64 AVAE_RaiseException(NSString*, ...) 4 AVFAudio 0x55738 AVAudioEngineGraph::_Connect(AVAudioNodeImplBase*, AVAudioNodeImplBase*, unsigned int, unsigned int, AVAudioFormat*) 5 AVFAudio 0x5cce0 AVAudioEngineGraph::Connect(AVAudioNode*, AVAudioNode*, unsigned long, unsigned long, AVAudioFormat*) 6 AVFAudio 0xdf1a8 AVAudioEngineImpl::Connect(AVAudioNode*, AVAudioNode*, unsigned long, unsigned long, AVAudioFormat*) 7 AVFAudio 0xe0fc8 -[AVAudioEngine connect:to:format:] 8 MyApp 0xa6af8 setupAudioEngine + 701 (MicrophoneOutput.swift:701) 9 MyApp 0xa46f0 handleRouteChange + 378 (MicrophoneOutput.swift:378) 10 MyApp 0xa4f50 @objc MicrophoneOutput.handleRouteChange(note:) 11 CoreFoundation 0x2a834 __CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__ 12 CoreFoundation 0xc6fd4 ___CFXRegistrationPost_block_invoke 13 CoreFoundation 0x9a1d0 _CFXRegistrationPost 14 CoreFoundation 0x408ac _CFXNotificationPost 15 Foundation 0x1b754 -[NSNotificationCenter postNotificationName:object:userInfo:] 16 AudioSession 0x56f0 (anonymous namespace)::HandleRouteChange(AVAudioSession*, NSDictionary*) 17 AudioSession 0x5cbc invocation function for block in avfaudio::AVAudioSessionPropertyListener(void*, unsigned int, unsigned int, void const*) 18 libdispatch.dylib 0x1e6c _dispatch_call_block_and_release 19 libdispatch.dylib 0x3a30 _dispatch_client_callout 20 libdispatch.dylib 0x11f48 _dispatch_main_queue_drain 21 libdispatch.dylib 0x11b98 _dispatch_main_queue_callback_4CF 22 CoreFoundation 0x51800 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ 23 CoreFoundation 0xb704 __CFRunLoopRun 24 CoreFoundation 0x1ebc8 CFRunLoopRunSpecific 25 GraphicsServices 0x1374 GSEventRunModal 26 UIKitCore 0x514648 -[UIApplication _run] 27 UIKitCore 0x295d90 UIApplicationMain 28 libswiftUIKit.dylib 0x30ecc UIApplicationMain(_:_:_:_:) 29 MyApp 0xc358 main (WhiteBalanceUI.swift) 30 ??? 0x104b1dce4 (Missing)

Using latest version of Ventura on an M1 iMac... 2 issues I've noticed when an Auv3 is loaded Out Of Process... 1 - When using tokenByAddingRenderObserver: The provided block is never called if an Auv3 is loaded out of process. 2: When loaded In process an Auv3 audio unit that is itself a host for other audio units can see all the audio units in the system. That is all v2 units and all v3 units. When loaded Out of Procress and the same v3 unit can only find the v2 units when querying. The system appears to be hiding the installed v3 units. This seems like the reverse of what I would expect. When sandboxed (out of process) why are other V3 units hidden from inspection? I've filed bug reports on these ages ago but had no response. I'm particularly interesting in issue #2 above as it makes a big difference when our AU host unit is running in Logic which only allows loading out of process. When loading into our own host app we can load the au in-process and it works nicely, able to host any other unit installed on the system.

Hi. I'm audio/video player developer. I implemented player as AVAudioPlayerNode to handle audio buffer. Suddenly, I wondered how great it would be if AVPlayer could handle buffers. I would like to know if there is a way to handle the buffer in AVPlayer, and if not, can you disclose it in the future? Thanks.

Hi, I want to record audio from the keyboard extension of my parent app.

I recently released my first ShazamKit app, but there is one thing that still bothers me. When I started I followed the steps as documented by Apple right here : https://developer.apple.com/documentation/shazamkit/shsession/matching_audio_using_the_built-in_microphone however when I was running this on iPad I receive a lot of high pitched feedback noise when I ran my app with this configuration. I got it to work by commenting out the output node and format and only use the input. But now I want to be able to recognise the song that’s playing from the device that has my app open and was wondering if I need the output nodes for that or if I can do something else to prevent the Mic. Feedback from happening. In short: What can I do to prevent feedback from happening Can I use the output of a device to recognise songs or do I just need to make sure that the microphone can run at the same time as playing music? Other than that I really love the ShazamKit API and can highly recommend to have a go with it! This is the code as documented in the above link (I just added the comments of what broke it for me) func configureAudioEngine() { // Get the native audio format of the engine's input bus. let inputFormat = audioEngine.inputNode.inputFormat(forBus: 0) // THIS CREATES FEEDBACK ON IPAD PRO let outputFormat = AVAudioFormat(standardFormatWithSampleRate: 48000, channels: 1) // Create a mixer node to convert the input. audioEngine.attach(mixerNode) // Attach the mixer to the microphone input and the output of the audio engine. audioEngine.connect(audioEngine.inputNode, to: mixerNode, format: inputFormat) // THIS CREATES FEEDBACK ON IPAD PRO audioEngine.connect(mixerNode, to: audioEngine.outputNode, format: outputFormat) // Install a tap on the mixer node to capture the microphone audio. mixerNode.installTap(onBus: 0, bufferSize: 8192, format: outputFormat) { buffer, audioTime in // Add captured audio to the buffer used for making a match. self.addAudio(buffer: buffer, audioTime: audioTime) } }

In watchos we can back to clock(homescreen) by two way. click dial cover screen with hand My app record audio with avaudioengine. But when i try to back to clock by two way written above. App didn't stay in background, but appear in screen with inactive state. For example, built-in voice recording app excuted in background when we go back to clock by clicking dial. And the app also go back to clock when covering screen with hand. How can i make my app stay in background when dial clicked like built-in recording app.

hi, im learning watchos app. a newbi i made a test app. this watch app records with avadioengine. but while i made app background with pressing dial or cover screen with hand, app return to foreground in few seconds(about 5 seconds). it doesn't when the ingine is not started, but only does when started. please help me .!!!

I'd like to understand the most robust way to record audio to disk using AVAudioNodes installTap(onBus:bufferSize:format:block:) method. Currently, I'm dispatching out the buffers I receive in my AVAudioNodeTapBlock to a serial dispatch queue then writing to disk using some Audio Toolbox methods, but my concern is that if I hold on to the buffer provided in the AVAudioNodeTapBlock for too long (due to disk I/O for example), I'll end up getting issues. What I'm considering is creating my own larger pool of preallocated AVAudioPCMBuffers (a few seconds worth) and copying across the data from the buffer provided by the tap into one of the buffers from this larger pool in the AVAudioNodeTapBlock directly (no dispatch queue). Is there a simpler way of handling this, or does this sound like the best route?

I want to process audio from videos using AVAudioEngine, but I'm not sure how to read the audio from videos using AVAudioEngine. I have the following code: let videoURL = /// url pointing to a local video file. let file = try AVAudioFile(forReading: videoURL) It works fine on macOS but on iOS it fails with the error message: [default]          ExtAudioFile.cpp:193   about to throw 'typ?': open audio file [avae]            AVAEInternal.h:109   [AVAudioFile.mm:134:AVAudioFileImpl: (ExtAudioFileOpenURL((CFURLRef)fileURL, &_extAudioFile)): error 1954115647 Error Domain=com.apple.coreaudio.avfaudio Code=1954115647 "(null)" UserInfo={failed call=ExtAudioFileOpenURL((CFURLRef)fileURL, &_extAudioFile)} Reading through the header files it seems that 'typ?' corresponds to kAudioFileUnsupportedFileTypeError, so that tells me that the file type is supported on macOS but not on iOS. So my question is: How can I work with audio from video files in an AVAudioEngine based setup? I already know that I could extract the audio from videos using something like AVAssetExportSession, but that approach requires extra preprocessing time that I rather not spend.

When using VoiceProcessingIO audio unit with voicechat audio session mode to have echo cancellation, I can't play audio in stereo, it only allows mono audio. How can I enable stereo playback with echo cancellation? Is it some kind of limitation? since it isn't mentioned anywhere in the documentation.

Hi, I have been searching all over for a way to do this on macOS: playing different stereo files on different pairs of audio outputs of the same hardware device. I am currently using AVAudioEngine with two AVAudioPlayerNodes and I can mix them and change the mapping of the entire mix through the use of AudioUnitSetProperty on the engine output, but I cannot have multiple AVAudioPlayerNodes play on different outputs. Obviously, being on macOS I cannot use AVAudioSession... Thank you if anyone has any idea on how to achieve this !

We are not able to find speech framework for apple tv. we have to implement Speech to Text in our application. When we are import speech framework in Apple Tv we get Error Like (No such module 'Speech' Please provide solution for this. )

I am using AVAudioEngine, I have a stereo wav file that I want to use with an AVAudioEnvironmentNode but it requires the input source is mono otherwise it will not spatialize the sounds. Is there some way I can convert the stereo input to mono dynamically, maybe using an avaudiomixernode or something else? It looks like maybe I can use AVAudioConverter to convert the original buffer, but I want to be able to just add a node in the AVAudioEngine graph that can convert it to mono if that is possible, rather than having to convert the input buffer explicitly. So something like: WAV -> AVAudioPlayerNode -> Stereo to mono node -> AVAudioEnvironmentNode

I am trying to build a project that will allow me to use an iphone USB connected to a Mac as an audio output device. There are options to do this wirelessly like "AirFoil" but they all introduce latency during playback. When connected via USB, the iphone will appear in Audio Midi Setup as a device. Currently it is only selectable as an input device. I believe it can be used as an external microphone or a midi input device. My hope is that through code, the grayed output selection could be overridden. Is this even possible? From what I've read the AVAudioEngine might provide a solution to this? I have read through posts about listing all available AudioDevices by ID. Listing all available audio devices From here would I be able to set the audio output device as the iphone? Or is this something that would require a companion app running on the phone to route the audio as well? Any information would be helpful, thanks!

Hi, I'm using the Speech library to get speech recognition from AVAudioRecorder and store the text to database. However, most of the time it is not accurate. Heck, even "test 123" gave me entirely different words (eg. "this one till three", "this one ticket", etc). The funny thing was, it was correct a few times when it was still progressing, but then it changed into the wrong final words. So, how do I increase the accuracy of it? This is what I got as the settings:           try audioSession.setCategory(.record, mode: .spokenAudio, options: .duckOthers)          try audioSession.setActive(true, options: .notifyOthersOnDeactivation)      let recorderSettings: [String:Any] = [       AVFormatIDKey: NSNumber(value: kAudioFormatAppleLossless),        AVSampleRateKey: 44100.0,        AVNumberOfChannelsKey: 1,        AVEncoderAudioQualityKey: AVAudioQuality.max.rawValue     ] Is there anything else to improve? Thank you.