I am trying to use AVAudioEngine for listening to mic samples and playing them simultaneously via external speaker or headphones (assuming they are attached to iOS device). I tried the following using AVAudioPlayerNode and it works, but there is too much delay in the audio playback. Is there a way to hear sound realtime without delay? Why the scheduleBuffer API has so much delay I wonder. var engine: AVAudioEngine! var playerNode: AVAudioPlayerNode! var mixer: AVAudioMixerNode! var audioEngineRunning = false public func setupAudioEngine() { self.engine = AVAudioEngine() let input = engine.inputNode let format = input.inputFormat(forBus: 0) playerNode = AVAudioPlayerNode() engine.attach(playerNode) self.mixer = engine.mainMixerNode engine.connect(self.playerNode, to: self.mixer, format: playerNode.outputFormat(forBus: 0)) engine.inputNode.installTap(onBus: 0, bufferSize: 4096, format: format, block: { buffer, time in self.playerNode.scheduleBuffer(buffer, completionHandler: nil) }) do { engine.prepare() try self.engine.start() audioEngineRunning = true self.playerNode.play() } catch { print("error couldn't start engine") audioEngineRunning = false } }

I am using AVAudioSession with playAndRecord category as follows: private func setupAudioSessionForRecording() { let audioSession = AVAudioSession.sharedInstance() do { try audioSession.setActive(false) try audioSession.setPreferredSampleRate(Double(48000)) } catch { NSLog("Unable to deactivate Audio session") } let options:AVAudioSession.CategoryOptions = [.allowAirPlay, .mixWithOthers] do { try AVAudioSession.sharedInstance().setCategory(AVAudioSession.Category.playAndRecord, mode: AVAudioSession.Mode.default, options: options) } catch { NSLog("Could not set audio session category \(error)") } do { try audioSession.setActive(true) } catch { NSLog("Unable to activate AudioSession") } } Next I use AVAudioEngine to repeat what I say in the microphone to external speakers (on the TV connected with iPhone using HDMI Cable). //MARK:- AudioEngine var engine: AVAudioEngine! var playerNode: AVAudioPlayerNode! var mixer: AVAudioMixerNode! var audioEngineRunning = false public func setupAudioEngine() { self.engine = AVAudioEngine() engine.connect(self.engine.inputNode, to: self.engine.outputNode, format: nil) do { engine.prepare() try self.engine.start() } catch { print("error couldn't start engine") } audioEngineRunning = true } public func stopAudioEngine() { engine.stop() audioEngineRunning = false } The issue is I hear some kind of reverb/humming noise after I speak for a few seconds that keeps getting amplified and repeated. If I use a RemoteIO unit instead, no such noise comes out of speakers. I am not sure if my setup of AVAudioEngine is correct. I have tried all kinds of AVAudioSession configuration but nothing changes. The link to sample audio with background speaker noise is posted [here] in the Stackoverflow forum (https://stackoverflow.com/questions/72170548/echo-when-using-avaudioengine-over-hdmi#comment127514327_72170548)

I have a RemoteIO unit that successfully playbacks the microphone samples in realtime via attached headphones. I need to get the same functionality ported using AVAudioEngine, but I can't seem to make a head start. Here is my code, all I do is connect inputNode to playerNode which crashes. var engine: AVAudioEngine! var playerNode: AVAudioPlayerNode! var mixer: AVAudioMixerNode! var engineRunning = false private func setupAudioSession() { var options:AVAudioSession.CategoryOptions = [.allowBluetooth, .allowBluetoothA2DP] do { try AVAudioSession.sharedInstance().setCategory(AVAudioSession.Category.playAndRecord, mode: AVAudioSession.Mode.default, options: options) try AVAudioSession.sharedInstance().setAllowHapticsAndSystemSoundsDuringRecording(true) } catch { MPLog("Could not set audio session category") } let audioSession = AVAudioSession.sharedInstance() do { try audioSession.setActive(false) try audioSession.setPreferredSampleRate(Double(44100)) } catch { print("Unable to deactivate Audio session") } do { try audioSession.setActive(true) } catch { print("Unable to activate AudioSession") } } private func setupAudioEngine() { self.engine = AVAudioEngine() self.playerNode = AVAudioPlayerNode() self.engine.attach(self.playerNode) engine.connect(self.engine.inputNode, to: self.playerNode, format: nil) do { try self.engine.start() } catch { print("error couldn't start engine") } engineRunning = true } But starting AVAudioEngine causes a crash: libc++abi: terminating with uncaught exception of type NSException *** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'required condition is false: inDestImpl->NumberInputs() > 0 || graphNodeDest->CanResizeNumberOfInputs()' terminating with uncaught exception of type NSException How do I get realtime record and playback of mic samples via headphones working?

I am working on music application where multiple wav files are scheduling within time frame. Everything is working perfect except one scenario where there is small beef is coming while scheduling player node again. For example - one.wav is playing on PlayerNode1 and now I am rescheduling to second.wav after 2 second then there is small beep is coming. I have tried to stop node by checking isPlaying condition. Still it is not working. Am I doing anything wrong here. if playerNode.isPlaying { playerNode.stop() } playerNode.scheduleFile(audioFile, at: nil, completionHandler: nil) playerNode.play() I am using same player node for performance as there are 24 wav files that needs to be played in 1 minute so there is no point to keeping all player nodes. How would stop beef while rescheduling new audio file for same player node? I have shared link below to check issue. https://drive.google.com/file/d/1FjZtLUj_wUp0LQPyjIwfJNy67HWUlt0I/view?usp=sharing Expected result should be song continuity

I'm developing a game that will use speech recognition to execute various commands. I am using code from Apple's Recognizing Speech in Live Audio documentation page. When I run this in a Swift Playground, it works just fine. However, when I make a SpriteKit game application (basic setup from Xcode's "New Project" menu option), I get the following error: required condition is false: IsFormatSampleRateAndChannelCountValid(hwFormat) Upon further research, it appears that my input node has no channels. The following is the relevant portion of my code, along with debug output: let inputNode = audioEngine.inputNode print("Number of inputs: \(inputNode.numberOfInputs)") // 1 print("Input Format: \(inputNode.inputFormat(forBus: 0))") // <AVAudioFormat 0x600001bcf200: 0 ch, 0 Hz, 'lpcm' (0x00000029) 32-bit little-endian float, deinterleaved> let channelCount = inputNode.inputFormat(forBus: 0).channelCount print("Channel Count: \(channelCount)") // 0 <== Agrees with the inputFormat output listed previously // Configure the microphone input. print("Number of outputs: \(inputNode.numberOfOutputs)") // 1 let recordingFormat = inputNode.outputFormat(forBus: 0) print("Output Format: \(recordingFormat)") // <AVAudioFormat 0x600001bf3160: 2 ch, 44100 Hz, Float32, non-inter> inputNode.installTap(onBus: 0, bufferSize: 256, format: recordingFormat, block: audioTap) // <== This is where the error occurs. // NOTE: 'audioTap' is a function defined in this class. Using this defined function instead of an inline, anonymous function. The code snippet is included in the game's AppDelegate class (which includes import statements for Cocoa, AVFoundation, and Speech), and executes during its applicationDidFinishLaunching function. I'm having trouble understanding why Playground works, but a game app doesn't work. Do I need to do something specific to get the application to recognize the microphone? NOTE: This if for MacOS, NOT iOS. While the "How To" documentation cited earlier indicates iOS, Apple stated at WWDC19 that it is now supported on the MacOS. NOTE: I have included the NSSpeechRecognitionUsageDescription key in the applications plist, and successfully acknowledged the authorization request for the microphone.

Hi all, I'm using AVAudioEngine to play multiple nodes at various times (like GarageBand for example). So far I managed to play the various files at the right time using this code : DispatchQueue.global(qos: .background).async {             AudioManager.instance.audioEngine.attach(AudioManager.instance.mixer)             AudioManager.instance.audioEngine.connect(AudioManager.instance.mixer, to: AudioManager.instance.audioEngine.outputNode, format: nil)           // !important - start the engine *before* setting up the player nodes           try! AudioManager.instance.audioEngine.start()                      for audioFile in data {             // Create and attach the audioPlayer node for this file             let audioPlayer = AVAudioPlayerNode()             AudioManager.instance.audioEngine.attach(audioPlayer)             AudioManager.instance.nodes.append(audioPlayer)             // Notice the output is the mixer in this case             AudioManager.instance.audioEngine.connect(audioPlayer, to: AudioManager.instance.mixer, format: nil)             let fileUrl = audioFile.audio.fileUrl             if let file : AVAudioFile = try? AVAudioFile.init(forReading: fileUrl) {                 let time = audioFile.start > 0 ? AudioManager.instance.secondsToAVAudioTime(hostTime: mach_absolute_time(), time: Double(audioFile.start / CGFloat.secondsToPoints)) : nil                 audioPlayer.scheduleFile(file, at: time, completionHandler: nil)                 audioPlayer.play(at: time)             }           }         } Basically my data object contains struct that have a reference to an audio fileURL and the startPosition at which it should begin. That works great. now I would like to export all these tracks mixed into a single file and save it to the Document's directory of the user. How can I achieve this? Thanks for your help.

I have an AVMutableAudioMix and use MTAudioProcessingTap to process the audio data.But After I pass the buffer to AVAudioEngine and to render it with renderOffline,the audio has no any effects...How can I do it? Any idea? Here is the code for MTAudioProcessingTapProcessCallback var callback = MTAudioProcessingTapCallbacks(version: kMTAudioProcessingTapCallbacksVersion_0, clientInfo:UnsafeMutableRawPointer(Unmanaged.passUnretained(self.engine).toOpaque()), init: tapInit, finalize: tapFinalize, prepare: tapPrepare, unprepare: tapUnprepare) { tap, numberFrames, flags, bufferListInOut, numberFramesOut, flagsOut in                       guard MTAudioProcessingTapGetSourceAudio(tap, numberFrames, bufferListInOut, flagsOut, nil, numberFramesOut) == noErr else {         preconditionFailure()       }       let storage = MTAudioProcessingTapGetStorage(tap)       let engine = Unmanaged<Engine>.fromOpaque(storage).takeUnretainedValue()       // render the audio with effect       engine.render(bufferPtr: bufferListInOut,numberOfFrames: numberFrames)     } And here is the Engine code class Engine {   let engine = AVAudioEngine()       let player = AVAudioPlayerNode()   let pitchEffect = AVAudioUnitTimePitch()   let reverbEffect = AVAudioUnitReverb()   let rateEffect = AVAudioUnitVarispeed()   let volumeEffect = AVAudioUnitEQ()   let format = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: 44100, channels: 2, interleaved: false)!   init() {     engine.attach(player)     engine.attach(pitchEffect)     engine.attach(reverbEffect)     engine.attach(rateEffect)     engine.attach(volumeEffect)           engine.connect(player, to: pitchEffect, format: format)     engine.connect(pitchEffect, to: reverbEffect, format: format)     engine.connect(reverbEffect, to: rateEffect, format: format)     engine.connect(rateEffect, to: volumeEffect, format: format)     engine.connect(volumeEffect, to: engine.mainMixerNode, format: format)           try! engine.enableManualRenderingMode(.offline, format: format, maximumFrameCount: 4096)           reverbEffect.loadFactoryPreset(AVAudioUnitReverbPreset.largeRoom2)     reverbEffect.wetDryMix = 100     pitchEffect.pitch = 2100           try! engine.start()     player.play()   }       func render(bufferPtr:UnsafeMutablePointer<AudioBufferList>,numberOfFrames:CMItemCount) {     let buffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: 4096)!     buffer.frameLength = AVAudioFrameCount(numberOfFrames)     buffer.mutableAudioBufferList.pointee = bufferPtr.pointee     self.player.scheduleBuffer(buffer) {       try! self.engine.renderOffline(AVAudioFrameCount(numberOfFrames), to: buffer)     }   } }

I have a kind of trivial request, I need to seek sound playback. The problem is I don't have a local sound file, I got a pointer to the sound instead (as well as other params), from (my internal) native lib. There is a method that I use in order to convert UnsafeRawPointer to AVAudioPCMBuffer ... var byteCount: Int32 = 0       var buffer: UnsafeMutableRawPointer?               defer {         buffer?.deallocate()         buffer = nil       }               if audioReader?.getAudioByteData(byteCount: &byteCount, data: &buffer) ?? false && buffer != nil {         let audioFormat = AVAudioFormat(standardFormatWithSampleRate: Double(audioSampleRate), channels: AVAudioChannelCount(audioChannels))!         if let pcmBuf = AVAudioPCMBuffer(pcmFormat: audioFormat, frameCapacity: AVAudioFrameCount(byteCount)) {           let monoChannel = pcmBuf.floatChannelData![0]           pcmFloatData = [Float](repeating: 0.0, count: Int(byteCount))                       //>>> Convert UnsafeMutableRawPointer to [Int8] array           let int16Ptr: UnsafeMutablePointer<Int16> = buffer!.bindMemory(to: Int16.self, capacity: Int(byteCount))           let int16Buffer: UnsafeBufferPointer<Int16> = UnsafeBufferPointer(start: int16Ptr, count: Int(byteCount) / MemoryLayout<Int16>.size)           let int16Arr: [Int16] = Array(int16Buffer)           //<<<                       // Int16 ranges from -32768 to 32767 -- we want to convert and scale these to Float values between -1.0 and 1.0           var scale = Float(Int16.max) + 1.0           vDSP_vflt16(int16Arr, 1, &pcmFloatData[0], 1, vDSP_Length(int16Arr.count)) // Int16 to Float           vDSP_vsdiv(pcmFloatData, 1, &scale, &pcmFloatData[0], 1, vDSP_Length(int16Arr.count)) // divide by scale              memcpy(monoChannel, pcmFloatData, MemoryLayout<Float>.size * Int(int16Arr.count))           pcmBuf.frameLength = UInt32(int16Arr.count)                       usagePlayer.setupAudioEngine(with: audioFormat)           audioClip = pcmBuf         }       } ... So, at the end of the method, you can see this line audioClip = pcmBuf, where the prepared pcmBuf is pass to the local variable. Then, what I need is just to start it like this ... /*player is AVAudioPlayerNode*/ player.scheduleBuffer(buf, at: nil, options: .loops) player.play() ... and that is it, now I can hear the sound. But let's say I need to seek forward in 10 sec, in order to do this I need stop() the player node, set a new AVAudioPCMBuffer but this time with an offset of 10 sec. The problem is there is no method for offset, nor from the player node side nor AVAudioPCMBuffer. For example, if I would work with a file (instead of a buffer), I could use this method ... player.scheduleSegment(         file,         startingFrame: seekFrame,         frameCount: frameCount,         at: nil       ) ... There at least you can use startingFrame: seekFrame and frameCount: frameCount params. But in my case I don't use file I use buffer and the problem is that - there is no such params for buffer implementation. Looks like I can't implement seek logic if I use AVAudioPCMBuffer. What am I doing wrong?

I want to create a sort of soundscape in surround sound. Imagine something along the lines of the user can place the sound of a waterfall to their front right and the sound of frogs croaking to their left etc. etc. I have an AVAudioEngine playing a number of AVAudioPlayerNodes. I'm using AVAudioEnvironmentNode to simulate the positioning of these. The position seems to work correctly. However, I'd like these to work with head tracking so if the user moves their head the sounds from the players move accordingly. I can't figure out for to do it or find any docs on the subject. Is it possible to make AVAudioEngine output surround sound and if it can would the tracking just work automagically the same as it does when playing surround sound content using AVPlayerItem. If not is the only way to achieve this effect to use CMHeadphonemotionmanager and manually move the listener AVAudioEnvironmentNode listener around?