I have an array of Float (representing audio samples) and I want to turn it into an AVAudioPCMBuffer so I can pass it to AVAudioFile's write(from:). There's an obvious way (actually not obvious at all, I cribbed it from this gist): var floats: [Float] = ... // this comes from somewhere else let audioBuffer = AudioBuffer(mNumberChannels: 1, mDataByteSize: UInt32(floats.count * MemoryLayout<Float>.size), mData: &floats) var bufferList = AudioBufferList(mNumberBuffers: 1, mBuffers: audioBuffer) let outputAudioBuffer = AVAudioPCMBuffer(pcmFormat: buffer.format, bufferListNoCopy: &bufferList)! try self.renderedAudioFile?.write(from: outputAudioBuffer) This works (I get the audio output I expect) but in Xcode 13.4.1 this gives me a warning on the &floats: Cannot use inout expression here; argument 'mData' must be a pointer that outlives the call to 'init(mNumberChannels:mDataByteSize:mData:)' Ok, scope the pointer then: var floats: [Float] = ... // this comes from somewhere else try withUnsafeMutablePointer(to: &floats) { bytes in let audioBuffer = AudioBuffer(mNumberChannels: 1, mDataByteSize: UInt32(bytes.pointee.count * MemoryLayout<Float>.size), mData: bytes) var bufferList = AudioBufferList(mNumberBuffers: 1, mBuffers: audioBuffer) let outputAudioBuffer = AVAudioPCMBuffer(pcmFormat: buffer.format, bufferListNoCopy: &bufferList)! try self.renderedAudioFile?.write(from: outputAudioBuffer) } The warning goes away, but now the output is garbage. I really don't understand this as floats.count and bytes.pointee.count are the same number. What am I doing wrong?

Voice isolation does a great job with noise suppression when the user is holding the phone in hand (facetime use case). But when the phone is about 4-feet away from the user, voice isolation quality substantially drops and we are seeing that it is better to not use it. Our use case demands that user mounts phone on tripod and sites approximately 4 feet away from camera. In this case we are seeing worst performance from voice isolation, presumably because of heavy signal processing and lower original signal to begin with.

For reasons too convoluted to go into, I have the need to ‘tap’ the final audio going to the output device (speaker, headphones, etc) from within the app I’m working on. (just need to grab my own audio) It would be really important to not know any structures in the app, and tie into them. It seems like AVAudioEngine / installTapOnBus is the way to go, but it’s unclear what exactly would be the input- I have implemented all of the AVAudioBufferList -> CMSampleBufferRef code through to the expected output, and all I get is silence. It seemed like just attaching to the mainMixerNode should have done it, but now it seems like a “device” input would be necessary. So the questions are: Is AVAudioEngine the SDK for this task? If not, then what? If yes, then what AVAudioNode would serve as an input? Is there an AVAudioUnit that is just the ‘current device output’? I’m not looking for a complete solution, just the last few lines that can complete this connection, or a pointer to the framework that could accomplish this.

Hello, In my iPadOS app I need to be able to apply AUv3 effects to individual input channels. However, AVAudioEngine's inputNode delivers one output bus with all input channels in that bus. How can I separate these input channels? For example, an audio interface with 6 input channels yields an inputNode with 6 channels on the input bus and output bus. How can I apply an effect to channel 1? AVAudioEngine only lets me connect busses between audio units. AVAudioConnectionPoint is an auxiliary class that allows me to establish a one-to-many routing. It seems that with the V3 API I cannot achieve my goal. What other solutions are there? Use the V2 API and install a callback... exactly where? Or is there some kind of audio format conversion foreseen?

I am trying to mix the audio from 2 different hardware audio devices together in real-time and record the results. Does anybody have any idea how to do this? This is on macOS. Things I have tried and why it didn't work: Adding 2 audio AVCaptureDevices to an AVCaptureMovieFileOutput or AVAssetWriter. This results in a file that has 2 audio tracks. This doesn't work for me for various reasons. Sure I can mix them together with an AVAssetExportSession, but it needs to be real-time. Programmatically creating an aggregate device and recording that as an AVCaptureDevice. This "sort of" works, but it always results in a recording with strange channel issues. For example, if I combine a 1 channel mic and a 2 channel device, I get a recording with 3 channel audio (L R C). If I make an aggregate out of 2 stereo devices, I get a recording with quadraphonic sound(L R Ls Rs), which won't even play back on some players. If I always force it to stereo, all stereo tracks get turned to mono for some reason. Programmatically creating an aggregate device and trying to use it in an AVAudioEngine. I've had multiple problems with this, but the main one is that when the aggregate device is an input node, it only reports the format of its main device, and no sub-devices. And I can't force it to be 3 or 4 channels without errors. Use an AVCaptureSession to output the sample buffers of both devices, then convert and put those samples into their own AVPlayerNodes. Then mix those AVPlayerNodes into an AVAudioEngine mixer. This actually works, but the resulting audio lags so far behind real-time, that it is unusable. If I record a webcam video along with the audio, the lip-sync is off by like half a second. I really need help with this. If anybody has a way to do this, let me know. Some caveats that have also been tripping me up: The hardware devices that need to be recorded might not be the default input device for the system. The MBP built in mic might be the default device, but I need to record 2 other devices and disclose the built in mic. The devices usually don't have the same audio format. I might be mixing an lpcm mono int16 interleaved with a lpcm stereo float32 non-interleaved. It absolutely has to be real-time and 1 single audio track. It shouldn't be this hard, right?

Hi, I'm trying to send audio data via UDP. I am using Network.framework with networking, so to use send method in NWConnection sending data must be Data type or confirm to DataProtocol. To satisfy those conditions, I have implemented a method to convert from AVAudioPCMBuffer type to Data type. func makeDataFromPCMBuffer(buffer: AVAudioPCMBuffer, time: AVAudioTime) -> Data {         let audioBuffer = buffer.audioBufferList.pointee.mBuffers         let data: Data!         data = .init(bytes: audioBuffer.mData!, count: Int(audioBuffer.mDataByteSize))         return data     } Implementation above is referenced from this post The problem is that the size of converted data is too big to fit in UDP datagram and error below occurs when I try to send data. I have found out that initial size of buffer is too big to fit in maximumDatagramSize. Below is code regarding to buffer.         let tapNode: AVAudioNode = mixerNode         let format = tapNode.outputFormat(forBus: 0)         tapNode.installTap(onBus: 0, bufferSize: 4096, format: format, block: { (buffer, time) in          // size of buffer: AVAudioPCMBuffer is 19200 already.             let bufferData = self.makeDataFromPCMBuffer(buffer: buffer, time: time)             sharedConnection?.sendRecordedBuffer(buffer: bufferData)         }) I need to reduce size of AVAudioPCMBuffer to fit in UDP datagram, But I can't find right way to do it. What would be best way to make data fit in datagram? I thought of dividing data in half, but this is UDP so I'm not sure how to handle those datas when one data has lost. So I'm trying to make AVAudioPCMBuffer fit in datagram. Any help would be very appreciated!

When I create an aggregate device with 2 hardware inputs and 1 output and I try to use it with AVAudioEngine, it fails to start. I get the error IsFormatSampleRateAndChannelCountValid(outputHWFormat) If I use an aggregate device with only 1 input/output, it works. The problem seems to stem from how aggregate devices handle channels. If I add a 2 channel device and a 1 channel device to the aggregate as inputs, I get an aggregate device with 3 channels. However, if I try and get the format of the input node, it only reports the format of the first device in the aggregate. So instead of saying the device has 3 channels, it will say it has 1 or 2 depending on which device is the main device. I've tried creating my own AVAudioFormat using channel layouts such as kAudioChannelLayoutTag_AAC_3_0, but this only works in very specific cases and is very unreliable. Can anybody help with this? It is driving me crazy. The main problem I am trying to solve is to combine/mix 2 hardware (or virtual hardware via HAL) audio devices in real-time for recording. An aggregate device alone doesn't work (see https://developer.apple.com/forums/thread/703258) Thanks for any help, you would save my day/week.

Hi, I am interested in decoding multichannel Higher-Order Ambisonics feeds to the Spatial Audio renderer discussed in the "Immerse your app in spatial audio" WWDC21 talk. However, I could not find any documentation about which multichannel audio formats are actually supported by the renderer, and a search for "Ambisonics" in the developer documentation only contains results pertaining to the Audio DriverKit. Can someone please enlighten me? Thank you!

Hi, Ever since MacOS 12.3 Apple Music stops playing cd's when AirPods is connected. The music just stops. It works using iMac Speakers och external sound card but now with AirPods. The Console states the following - now running macOS 12.4 - Console log - filtered on Music app AirPods works when streaming music from Apple Music, only when using Superdrive it doesn't work. Any help is appreciated.

Hi, I'm creating a process to read an existing audio file, add an effect using AVAudioEngine, and then save it as another audio file. However, with the following method using an AVAudioPlayerNode, the save process must wait until the end of playback. import UIKit import AVFoundation class ViewController: UIViewController {          let engine = AVAudioEngine()     let playerNode = AVAudioPlayerNode()     let reverbNode = AVAudioUnitReverb()          override func viewDidLoad() {         super.viewDidLoad()         do {                          let url = URL(fileURLWithPath: Bundle.main.path(forResource: "original", ofType: "mp3")!)             let file = try AVAudioFile(forReading: url)                          // playerNode             engine.attach(playerNode)             // reverbNode             reverbNode.loadFactoryPreset(.largeChamber)             reverbNode.wetDryMix = 5.0             engine.attach(reverbNode)                          engine.connect(playerNode, to: reverbNode, format: file.processingFormat)             engine.connect(reverbNode, to: engine.mainMixerNode, format: file.processingFormat)             playerNode.scheduleFile(file, at: nil, completionCallbackType: .dataPlayedBack){ [self] _ in                 reverbNode.removeTap(onBus: 0)             }             // start             try engine.start()             playerNode.play()                          let url2 = URL(fileURLWithPath: fileInDocumentsDirectory(filename: "changed.wav"))             let outputFile = try! AVAudioFile(forWriting: url2, settings: playerNode.outputFormat(forBus: 0).settings)             reverbNode.installTap(onBus: 0, bufferSize: AVAudioFrameCount(reverbNode.outputFormat(forBus: 0).sampleRate), format: reverbNode.outputFormat(forBus: 0)) { (buffer, when) in                 do {                     try outputFile.write(from: buffer)                 } catch let error {                     print(error)                 }             }         } catch {             print(error.localizedDescription)         }     }     func getDocumentsURL() -> NSURL {         let documentsURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)[0] as NSURL         return documentsURL     }          func fileInDocumentsDirectory(filename: String) -> String {         let fileURL = getDocumentsURL().appendingPathComponent(filename)         return fileURL!.path     } } Is there a way to complete the writing without waiting for the playback to complete? My ideal is to complete the write in the time required by CPU and storage performance. It seems that reverbNode.installTap(...) { (buffer, when) in ...} in the code is processed in parallel with the current playback position, so I would like to dramatically improve the processing speed. Best regards.

I am using AVAudioSession with playAndRecord category as follows: private func setupAudioSessionForRecording() { let audioSession = AVAudioSession.sharedInstance() do { try audioSession.setActive(false) try audioSession.setPreferredSampleRate(Double(48000)) } catch { NSLog("Unable to deactivate Audio session") } let options:AVAudioSession.CategoryOptions = [.allowAirPlay, .mixWithOthers] do { try AVAudioSession.sharedInstance().setCategory(AVAudioSession.Category.playAndRecord, mode: AVAudioSession.Mode.default, options: options) } catch { NSLog("Could not set audio session category \(error)") } do { try audioSession.setActive(true) } catch { NSLog("Unable to activate AudioSession") } } Next I use AVAudioEngine to repeat what I say in the microphone to external speakers (on the TV connected with iPhone using HDMI Cable). //MARK:- AudioEngine var engine: AVAudioEngine! var playerNode: AVAudioPlayerNode! var mixer: AVAudioMixerNode! var audioEngineRunning = false public func setupAudioEngine() { self.engine = AVAudioEngine() engine.connect(self.engine.inputNode, to: self.engine.outputNode, format: nil) do { engine.prepare() try self.engine.start() } catch { print("error couldn't start engine") } audioEngineRunning = true } public func stopAudioEngine() { engine.stop() audioEngineRunning = false } The issue is I hear some kind of reverb/humming noise after I speak for a few seconds that keeps getting amplified and repeated. If I use a RemoteIO unit instead, no such noise comes out of speakers. I am not sure if my setup of AVAudioEngine is correct. I have tried all kinds of AVAudioSession configuration but nothing changes. The link to sample audio with background speaker noise is posted [here] in the Stackoverflow forum (https://stackoverflow.com/questions/72170548/echo-when-using-avaudioengine-over-hdmi#comment127514327_72170548)

We start a voice recording via self.avAudioRecorder = try AVAudioRecorder( url: self.recordingFileUrl, settings: settings ) self.avAudioRecorder.record() At certain point, we will stop the recording via self.avAudioRecorder.stop() I was wondering, is it safe to perform file copy on self.recordingFileUrl immediately, after self.avAudioRecorder.stop()? Is all recording data has been flushed to self.recordingFileUrl and self.recordingFileUrl file is closed properly?

I create a process to save an effected audio file by AVAudioEngine. However, even if I change the left and right side volume by changing .pan parameter and save it, the effect is not reflected on the file. Sample code) let emvironmentNode = AVAudioEnvironmentNode() emvironmentNode.pan = 1.0 ... < omit > ... emvironmentNode.installTap(onBus: 0, bufferSize: AVAudioFrameCount(emvironmentNode.outputFormat(forBus: 0).sampleRate), format: emvironmentNode.outputFormat(forBus: 0)) { (buffer, when) in do { try outputFile.write(from: buffer) } catch let error { print(error) } } How do I save an audio file that reflects the .pan settings? Please let me know if there is another way to change the volume on the left and right side and save it to a file. Thank you.

I have a RemoteIO unit that successfully playbacks the microphone samples in realtime via attached headphones. I need to get the same functionality ported using AVAudioEngine, but I can't seem to make a head start. Here is my code, all I do is connect inputNode to playerNode which crashes. var engine: AVAudioEngine! var playerNode: AVAudioPlayerNode! var mixer: AVAudioMixerNode! var engineRunning = false private func setupAudioSession() { var options:AVAudioSession.CategoryOptions = [.allowBluetooth, .allowBluetoothA2DP] do { try AVAudioSession.sharedInstance().setCategory(AVAudioSession.Category.playAndRecord, mode: AVAudioSession.Mode.default, options: options) try AVAudioSession.sharedInstance().setAllowHapticsAndSystemSoundsDuringRecording(true) } catch { MPLog("Could not set audio session category") } let audioSession = AVAudioSession.sharedInstance() do { try audioSession.setActive(false) try audioSession.setPreferredSampleRate(Double(44100)) } catch { print("Unable to deactivate Audio session") } do { try audioSession.setActive(true) } catch { print("Unable to activate AudioSession") } } private func setupAudioEngine() { self.engine = AVAudioEngine() self.playerNode = AVAudioPlayerNode() self.engine.attach(self.playerNode) engine.connect(self.engine.inputNode, to: self.playerNode, format: nil) do { try self.engine.start() } catch { print("error couldn't start engine") } engineRunning = true } But starting AVAudioEngine causes a crash: libc++abi: terminating with uncaught exception of type NSException *** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'required condition is false: inDestImpl->NumberInputs() > 0 || graphNodeDest->CanResizeNumberOfInputs()' terminating with uncaught exception of type NSException How do I get realtime record and playback of mic samples via headphones working?

I have scheduled 2 player nodes with same file to audio engine but sound waves are not correctly visualised in audacity. It is creating noise or silence in between. I am not understanding what this about. If i scheduled 2 different file or single file then it is perfectly working. Use case - There are 2 player nodes connect to same audio engine and each player node scheduled with same audio file. Then it is not working as expected. Sound waves has error like SILENCE and mute for one of player node Below is code snippet for reference self.audioEngine = AVAudioEngine() self.mainMixerNode = AVAudioMixerNode() self.audioEngine.attach(self.mainMixerNode) self.audioEngine.connect(mainMixerNode, to: self.audioEngine.outputNode, format: nil)       self.audioEngine.prepare()       try! audioEngine.start() // Scheduling same file playerNode.prepare(withFrameCount: AVAudioFrameCount(segmentFrameCount)) playerNode.scheduleSegment(audioFile, startingFrame: 0, frameCount: AVAudioFrameCount(segmentFrameCount), at: playerTime, completionHandler: nil)

This is on a Mac Mini M1 with OSX Monterey. I am trying to write an audio network using AVAudioEngine as opposed to AUAudioGraph (which I understand is deprecated in favor of AVAudioEngine). My code works properly with AUAudioGraph. The input is a microphone which has a sample rate of 8 kHz. In the render proc, the data is written to a ring buffer. Debugging shows that the render proc is called every 0.064 seconds and writes 512 samples (8000 * 0x064 = 512). The program creates an AVAudioSourceNode. The render block for that node pulls data from the above ring buffer. But debugging shows that it is trying to take 512 samples about every 0.0107 seconds. That works out to 48000 samples per second, which is the output device sample rate. Obviously the ring buffer can't keep up. In the statement connecting the above source node to the AVEngine's mixer node, I specify (at least I think I am) a sample rate of 8000, but it still seems to be running at 48000. let inputFormat = AVAudioFormat( commonFormat: outputFormat.commonFormat, sampleRate: 8000, channels: 1, interleaved: outputFormat.isInterleaved) engine.connect(srcNode, to: mixerNode, fromBus: 0, toBus: 0, format: inputFormat) Also, looking at the microphone input using Audio MIDI Setup shows that microphone format is 8000 Hz, 1 channel 16-bit integer, but when I examine the input format of the AudioNode it is reported as 8000 Hz, 1 channel 32-bit float. The input node is using HAL. Obviously, somewhere in the internals of the node the samples are being converted from 16-bit ints to 32-bit floats. Is there a way to also have the sample rate changed? Am I doing this wrong? The HAL node was used with AUAudioGraph. Is there a different node that should be used with AVAudioEngine? I see that AVAudioEngine has an input node, but it seems if I connect it to the microphone, the input goes straight to the hardware output without going through the mixer node (where I want to mix in other audio sources). The original AUGraph code was modeled after the code in "Learning Core Audio" by Adamson & Avila, which, although it is old (pre-dating Swift and AVAudioEngine), is the only detailed reference on CoreAudio that I have been able to find. Is there a newer reference? Thanks, Mark

I updated Xcode to Xcode 13 and iPadOS to 15.0. Now my previously working application using SFSpeechRecognizer fails to start, regardless of whether I'm using on device mode or not. I use the delegate approach, and it looks like although the plist is set-up correctly (the authorization is successful and I get the orange circle indicating the microphone is on), the delegate method speechRecognitionTask(_:didFinishSuccessfully:) always returns false, but there is no particular error message to go along with this. I also downloaded the official example from Apple's documentation pages: SpokenWord SFSpeechRecognition example project page Unfortunately, it also does not work anymore. I'm working on a time-sensitive project and don't know where to go from here. How can we troubleshoot this? If it's an issue with Apple's API update or something has changed in the initial setup, I really need to know as soon as possible. Thanks.

I've currently received a task that requires to evaluate the possibility, as title, of recording via mic on BLE headset and play sound via built-in speaker at same time on iOS. I've done implementing forcing audio device set to built-in speaker whenever the BLE headset is connected/disconnected. It works if both mic/speaker need to be set to built-in one. But after days of search and try, I found that it is not possible to make mic/speaker set separately. Even specifying input device on AVAudioEngine is supported only on MacOS, not iOS. Can anyone or any technician give me a persuading answer about "Possibility of record via mic on BLE headset and play sound via built-in speaker at same time"?

How can you add a live audio player to Xcode where they will have a interactive UI to control the audio and they will be able to exit out of the app and or turn their device off and it will keep playing? Is their a framework or API that will work for this? Thanks! Really need help with this…. 🤩 I have looked everywhere and haven’t found something that works….

How can you add a live audio player to Xcode where they will have a interactive UI to control the audio and they will be able to exit out of the app and or turn their device off and it will keep playing? Is their a framework or API that will work for this? Thanks! Really need help with this…. 🤩