AVAudioNode

RSS for tag

Use the AVAudioNode abstract class for audio generation, processing, or I/O block.

Posts under AVAudioNode tag

89 Posts

Post

Replies

Boosts

Views

Activity

Audio Engine failure when attaching AVAudioUnitTimeEffect
According to the Apple docs, you should be able to connect audio nodes to an AVAudioEngine instance during runtime, but I'm getting a crash while trying to do so, in particular, when trying to connect instances of AVAudioUnitTimePitch or AVAudioUnitVarispeed to an AVAudioEngine with manual rendering mode enabled. The error message I get is: Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'player started when in a disconnected state' In my code, first, I configure the audio engine: let engine = AVAudioEngine() let format = AVAudioFormat(standardFormatWithSampleRate: 48000, channels: 2)! try! engine.enableManualRenderingMode(.offline, format: format, maximumFrameCount: 1024) try! engine.start() Then, I try to attach the player to the engine: let player = AVAudioPlayerNode() configureEngine(player: player, useVarispeed: true) player.play() // this is the line that causes the crash Finally, this is the function I use to configure the engine nodes graph: func configureEngine(player: AVAudioPlayerNode, useVarispeed: Bool) { engine.attach(player) guard useVarispeed else { engine.connect(player, to: engine.mainMixerNode, format: format) return } let varispeed = AVAudioUnitVarispeed() engine.attach(varispeed) engine.connect(player, to: varispeed, format: format) engine.connect(varispeed, to: engine.mainMixerNode, format: format) } If I pass false as the value for the useVarispeed parameter, the crash goes away. What is even more interesting is that if I add a dummy player node before starting the engine, the crash goes away too 🤷‍♂️ Could anyone please add clarity on what's going on here? Is this a bug or a limitation of the framework that I'm not aware of? Here's a simple project demonstrating the problem: https://github.com/rlaguilar/AVAudioEngineBug
0
0
1.6k
Mar ’23
AVAudioEngine crash when connecting inputNode to mainMixerNode
I have the following code to connect inputNode to mainMixerNode of AVAudioEngine: public func setupAudioEngine() { self.engine = AVAudioEngine() let format = engine.inputNode.inputFormat(forBus: 0) //main mixer node is connected to output node by default engine.connect(self.engine.inputNode, to: self.engine.mainMixerNode, format: format) do { engine.prepare() try self.engine.start() } catch { print("error couldn't start engine") } engineRunning = true } But I am seeing a crash in Crashlytics dashboard (which I can't reproduce). Fatal Exception: com.apple.coreaudio.avfaudio required condition is false: IsFormatSampleRateAndChannelCountValid(format) Before calling the function setupAudioEngine I make sure the AVAudioSession category is not playback where mic is not available. The function is called where audio route change notification is handled and I check this condition specifically. Can someone tell me what I am doing wrong? Fatal Exception: com.apple.coreaudio.avfaudio 0 CoreFoundation 0x99288 __exceptionPreprocess 1 libobjc.A.dylib 0x16744 objc_exception_throw 2 CoreFoundation 0x17048c -[NSException initWithCoder:] 3 AVFAudio 0x9f64 AVAE_RaiseException(NSString*, ...) 4 AVFAudio 0x55738 AVAudioEngineGraph::_Connect(AVAudioNodeImplBase*, AVAudioNodeImplBase*, unsigned int, unsigned int, AVAudioFormat*) 5 AVFAudio 0x5cce0 AVAudioEngineGraph::Connect(AVAudioNode*, AVAudioNode*, unsigned long, unsigned long, AVAudioFormat*) 6 AVFAudio 0xdf1a8 AVAudioEngineImpl::Connect(AVAudioNode*, AVAudioNode*, unsigned long, unsigned long, AVAudioFormat*) 7 AVFAudio 0xe0fc8 -[AVAudioEngine connect:to:format:] 8 MyApp 0xa6af8 setupAudioEngine + 701 (MicrophoneOutput.swift:701) 9 MyApp 0xa46f0 handleRouteChange + 378 (MicrophoneOutput.swift:378) 10 MyApp 0xa4f50 @objc MicrophoneOutput.handleRouteChange(note:) 11 CoreFoundation 0x2a834 __CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__ 12 CoreFoundation 0xc6fd4 ___CFXRegistrationPost_block_invoke 13 CoreFoundation 0x9a1d0 _CFXRegistrationPost 14 CoreFoundation 0x408ac _CFXNotificationPost 15 Foundation 0x1b754 -[NSNotificationCenter postNotificationName:object:userInfo:] 16 AudioSession 0x56f0 (anonymous namespace)::HandleRouteChange(AVAudioSession*, NSDictionary*) 17 AudioSession 0x5cbc invocation function for block in avfaudio::AVAudioSessionPropertyListener(void*, unsigned int, unsigned int, void const*) 18 libdispatch.dylib 0x1e6c _dispatch_call_block_and_release 19 libdispatch.dylib 0x3a30 _dispatch_client_callout 20 libdispatch.dylib 0x11f48 _dispatch_main_queue_drain 21 libdispatch.dylib 0x11b98 _dispatch_main_queue_callback_4CF 22 CoreFoundation 0x51800 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ 23 CoreFoundation 0xb704 __CFRunLoopRun 24 CoreFoundation 0x1ebc8 CFRunLoopRunSpecific 25 GraphicsServices 0x1374 GSEventRunModal 26 UIKitCore 0x514648 -[UIApplication _run] 27 UIKitCore 0x295d90 UIApplicationMain 28 libswiftUIKit.dylib 0x30ecc UIApplicationMain(_:_:_:_:) 29 MyApp 0xc358 main (WhiteBalanceUI.swift) 30 ??? 0x104b1dce4 (Missing)
1
0
2.5k
Mar ’23
Microphone feedback noise and can I use the output to recognise?
I recently released my first ShazamKit app, but there is one thing that still bothers me. When I started I followed the steps as documented by Apple right here : https://developer.apple.com/documentation/shazamkit/shsession/matching_audio_using_the_built-in_microphone however when I was running this on iPad I receive a lot of high pitched feedback noise when I ran my app with this configuration. I got it to work by commenting out the output node and format and only use the input. But now I want to be able to recognise the song that’s playing from the device that has my app open and was wondering if I need the output nodes for that or if I can do something else to prevent the Mic. Feedback from happening. In short: What can I do to prevent feedback from happening Can I use the output of a device to recognise songs or do I just need to make sure that the microphone can run at the same time as playing music? Other than that I really love the ShazamKit API and can highly recommend to have a go with it! This is the code as documented in the above link (I just added the comments of what broke it for me) func configureAudioEngine() { // Get the native audio format of the engine's input bus. let inputFormat = audioEngine.inputNode.inputFormat(forBus: 0) // THIS CREATES FEEDBACK ON IPAD PRO let outputFormat = AVAudioFormat(standardFormatWithSampleRate: 48000, channels: 1) // Create a mixer node to convert the input. audioEngine.attach(mixerNode) // Attach the mixer to the microphone input and the output of the audio engine. audioEngine.connect(audioEngine.inputNode, to: mixerNode, format: inputFormat) // THIS CREATES FEEDBACK ON IPAD PRO audioEngine.connect(mixerNode, to: audioEngine.outputNode, format: outputFormat) // Install a tap on the mixer node to capture the microphone audio. mixerNode.installTap(onBus: 0, bufferSize: 8192, format: outputFormat) { buffer, audioTime in // Add captured audio to the buffer used for making a match. self.addAudio(buffer: buffer, audioTime: audioTime) } }
3
0
3k
Feb ’23
Are any of the ML frameworks real-time safe for audio processing?
I'm working on an audio processing app and am creating an AVAudioUnit extension as a part of it. I need to train a small neural network in the app and use it to process audio in real-time in the AudioUnit. The network is mostly convolutions and is ideal for running on the GPU but it should run in real-time on the CPU. The problem that I'm currently facing is that none of the ML frameworks seem to be safe to use for inference within custom AVAudioUnit kernels. My understanding is that only C and C++ should be used in these kernels (in addition to the other rules of real-time computing). Objective-C and Swift are discouraged per the documentation. My background is primarily in ML so I'm newer to Apple development and especially new to real-time development in this ecosystem. I've investigated CoreML, MPS, BNNS/Accelerate, and MLCompute so far but I'm not certain that any of them are safe to use. Any feedback would be greatly appreciated!
0
1
2.0k
Feb ’23
Process video audio using AVAudioEngine in iOS
I want to process audio from videos using AVAudioEngine, but I'm not sure how to read the audio from videos using AVAudioEngine. I have the following code: let videoURL = /// url pointing to a local video file. let file = try AVAudioFile(forReading: videoURL) It works fine on macOS but on iOS it fails with the error message: [default]          ExtAudioFile.cpp:193   about to throw 'typ?': open audio file [avae]            AVAEInternal.h:109   [AVAudioFile.mm:134:AVAudioFileImpl: (ExtAudioFileOpenURL((CFURLRef)fileURL, &_extAudioFile)): error 1954115647 Error Domain=com.apple.coreaudio.avfaudio Code=1954115647 "(null)" UserInfo={failed call=ExtAudioFileOpenURL((CFURLRef)fileURL, &_extAudioFile)} Reading through the header files it seems that 'typ?' corresponds to kAudioFileUnsupportedFileTypeError, so that tells me that the file type is supported on macOS but not on iOS. So my question is: How can I work with audio from video files in an AVAudioEngine based setup? I already know that I could extract the audio from videos using something like AVAssetExportSession, but that approach requires extra preprocessing time that I rather not spend.
0
0
1.5k
Jan ’23
AVAudioEngine: routing different AVAudioPlayerNodes to different channels
Hi, I have been searching all over for a way to do this on macOS: playing different stereo files on different pairs of audio outputs of the same hardware device. I am currently using AVAudioEngine with two AVAudioPlayerNodes and I can mix them and change the mapping of the entire mix through the use of AudioUnitSetProperty on the engine output, but I cannot have multiple AVAudioPlayerNodes play on different outputs. Obviously, being on macOS I cannot use AVAudioSession... Thank you if anyone has any idea on how to achieve this !
0
1
1.2k
Jan ’23
If not AVMIDIRecorder, then what?
For AVAudioPlayer, there is corresponding AVAudioRecorder. For AVMIDIPLayer, I found nothing for recording from the system's active MIDI input device. Can I record midi events from the system’s active input midi device, without resolving to low level CoreMidi? After configuring AVAudioUnitSampler with just a few lines of code, import AVFoundation var engine = AVAudioEngine() let unit = AVAudioUnitSampler() engine.attach(unit) engine.connect(unit, to: engine.outputNode, format: engine.outputNode.outputFormat (forBus:0)) try! unit.loadInstrument(at:sndurl) //url to .sf2 file try! engine.start() I could send midi events programmatically. // feeding AVAudioUnitMIDIInstrument with midi data let range = (0..<100) let midiStart = range.map { _ in UInt8.random(in: 70...90) } let midiStop = [0] + midiStart let times = range.map { _ in TimeInterval.random(in: 0...100) * 0.3 } for i in range {     DispatchQueue.main.asyncAfter(deadline: .now()+TimeInterval(times[i])){         unit.stopNote(midiStop[i], onChannel: 1)         unit.startNote(midiStart[i], withVelocity: 127, onChannel: 1)     } } But instead, I need to send midi events from a midi instrument, and tap to them for recording.
1
0
1.8k
Dec ’22
AVAudioPlayerNode crashes on play
I have a very simple setup involving AVAudioEngine, and a player node, AVAudioPlayerNode.The player node is attached to the audio engine, then connected to the engine's main mixer node.Then, to start outputing some sound, I start the engine, and finally play the node :NSError* err = nil; BOOL started = [_audioEngine startAndReturnError:&amp;amp;err]; if(started) { [_playerNode play]; } else { // handle error }It happens sometimes, the app crashes (on the call to the play function) because, as the system says : "player started when engine not running".I can't understand how this can happen, at least in the code above.No error is returned in `err`.Does somebody knows what happen or faced such a situation ?
3
1
3.8k
Nov ’22
Apple.PHASE Unity Plug-in as AVAudioNode
Hi, my name is Jakob, we are working on sound creation apps with a visual and spatial approach in Unity. For iOS, we are using the "Unity as a library"-approach, so we have a native app part which is written in Swift and access to the AVAudioSession. Everything works great, except for one feature: We want to send the audio data generated with our app as binaural mix to another host app. To do so, we need to feed that data into an AVAudioUnit in our native iOS app. There are different approaches and we keep on making research and working on solutions. Could your new native spatial audio Unity plug-in Apple.PHASE might be a direct bridge feeding an AVAudioNode and combine the needs in between music and spatial audio content creators. Thanks a lot in advance & all the best!
0
0
1.7k
Sep ’22
Playing Audio
So I have successfully triggered a PTT notification, but when I try to play audio – any audio – it doesn't play. Seems to be an issue with initiating my AVAudioSession. If I do not initiate it, the sound plays (outside of the didActivateAudioSession; such as on view did load), so I know that it's not the audio playing code. For some reason, the AVAudioSession is not allowing me to play sound. Even when I put "PlayandRecord" and when I put "mix" in the options
2
0
1.8k
Sep ’22
AVAudioEngine exception - required condition is false format.sampleRate == hwFormat.sampleRate
I see in Crashlytics few users are getting this exception when connecting the inputNode to mainMixerNode in AVAudioEngine: Fatal Exception: com.apple.coreaudio.avfaudio required condition is false: format.sampleRate == hwFormat.sampleRate Here is my code: self.engine = AVAudioEngine() let format = engine.inputNode.inputFormat(forBus: 0) //main mixer node is connected to output node by default engine.connect(self.engine.inputNode, to: self.engine.mainMixerNode, format: format) Just want to understand how can this error occur and what is the right fix?
2
1
3.3k
Aug ’22
Why is internalRenderBlock fetched and used before allocateRenderResources() is called?
I have a working AUv3 AUAudioUnit app extension but I had to work around a strange issue: I found that the internalRenderBlock value is fetched and invoked before allocateRenderResources() is called. I have not found any documentation stating that this would be the case, and intuitively it does not make any sense. Is there something I am doing in my code that would be causing this to be the case? Should I *force* a call to allocateRenderResources() if it has not been called before internalRenderBlock is fetched? Thanks! Brad
6
1
3.1k
Jul ’22
How does toggling AVAudioSession active state affect the AVAudioPlayerNode
Every time the AVAudioSession category is re-activated ( after being inactivated) and the audioengine is restarted (calling stop() and play()), the output from audioplayer node seems to ignore the audio session category, until explicitly connecting the audio player node again using audioEngine.connect(playerNode, to: audioEngine.outputNode, format: audioFile.processingFormat) The documentation regarding this behavior is not clear and would like to clarify the following. Should audioEngine.connect be called everytime the AVAudioSession is activated? Should audioEngine.connect be called after the audio player engine is stopped (audioEngine.stop())?
0
0
950
Jun ’22
How do I mix 2 hardware audio devices and record them as 1 audio track?
I am trying to mix the audio from 2 different hardware audio devices together in real-time and record the results. Does anybody have any idea how to do this? This is on macOS. Things I have tried and why it didn't work: Adding 2 audio AVCaptureDevices to an AVCaptureMovieFileOutput or AVAssetWriter. This results in a file that has 2 audio tracks. This doesn't work for me for various reasons. Sure I can mix them together with an AVAssetExportSession, but it needs to be real-time. Programmatically creating an aggregate device and recording that as an AVCaptureDevice. This "sort of" works, but it always results in a recording with strange channel issues. For example, if I combine a 1 channel mic and a 2 channel device, I get a recording with 3 channel audio (L R C). If I make an aggregate out of 2 stereo devices, I get a recording with quadraphonic sound(L R Ls Rs), which won't even play back on some players. If I always force it to stereo, all stereo tracks get turned to mono for some reason. Programmatically creating an aggregate device and trying to use it in an AVAudioEngine. I've had multiple problems with this, but the main one is that when the aggregate device is an input node, it only reports the format of its main device, and no sub-devices. And I can't force it to be 3 or 4 channels without errors. Use an AVCaptureSession to output the sample buffers of both devices, then convert and put those samples into their own AVPlayerNodes. Then mix those AVPlayerNodes into an AVAudioEngine mixer. This actually works, but the resulting audio lags so far behind real-time, that it is unusable. If I record a webcam video along with the audio, the lip-sync is off by like half a second. I really need help with this. If anybody has a way to do this, let me know. Some caveats that have also been tripping me up: The hardware devices that need to be recorded might not be the default input device for the system. The MBP built in mic might be the default device, but I need to record 2 other devices and disclose the built in mic. The devices usually don't have the same audio format. I might be mixing an lpcm mono int16 interleaved with a lpcm stereo float32 non-interleaved. It absolutely has to be real-time and 1 single audio track. It shouldn't be this hard, right?
1
1
1.5k
May ’22
About reducing size of AVAudioPCMBuffer
Hi, I'm trying to send audio data via UDP. I am using Network.framework with networking, so to use send method in NWConnection sending data must be Data type or confirm to DataProtocol. To satisfy those conditions, I have implemented a method to convert from AVAudioPCMBuffer type to Data type. func makeDataFromPCMBuffer(buffer: AVAudioPCMBuffer, time: AVAudioTime) -> Data {         let audioBuffer = buffer.audioBufferList.pointee.mBuffers         let data: Data!         data = .init(bytes: audioBuffer.mData!, count: Int(audioBuffer.mDataByteSize))         return data     } Implementation above is referenced from this post The problem is that the size of converted data is too big to fit in UDP datagram and error below occurs when I try to send data. I have found out that initial size of buffer is too big to fit in maximumDatagramSize. Below is code regarding to buffer.         let tapNode: AVAudioNode = mixerNode         let format = tapNode.outputFormat(forBus: 0)         tapNode.installTap(onBus: 0, bufferSize: 4096, format: format, block: { (buffer, time) in          // size of buffer: AVAudioPCMBuffer is 19200 already.             let bufferData = self.makeDataFromPCMBuffer(buffer: buffer, time: time)             sharedConnection?.sendRecordedBuffer(buffer: bufferData)         }) I need to reduce size of AVAudioPCMBuffer to fit in UDP datagram, But I can't find right way to do it. What would be best way to make data fit in datagram? I thought of dividing data in half, but this is UDP so I'm not sure how to handle those datas when one data has lost. So I'm trying to make AVAudioPCMBuffer fit in datagram. Any help would be very appreciated!
0
0
1.1k
May ’22
I want to instantly save the sound with AVAudioEngine's effect.
Hi, I'm creating a process to read an existing audio file, add an effect using AVAudioEngine, and then save it as another audio file. However, with the following method using an AVAudioPlayerNode, the save process must wait until the end of playback. import UIKit import AVFoundation class ViewController: UIViewController {          let engine = AVAudioEngine()     let playerNode = AVAudioPlayerNode()     let reverbNode = AVAudioUnitReverb()          override func viewDidLoad() {         super.viewDidLoad()         do {                          let url = URL(fileURLWithPath: Bundle.main.path(forResource: "original", ofType: "mp3")!)             let file = try AVAudioFile(forReading: url)                          // playerNode             engine.attach(playerNode)             // reverbNode             reverbNode.loadFactoryPreset(.largeChamber)             reverbNode.wetDryMix = 5.0             engine.attach(reverbNode)                          engine.connect(playerNode, to: reverbNode, format: file.processingFormat)             engine.connect(reverbNode, to: engine.mainMixerNode, format: file.processingFormat)             playerNode.scheduleFile(file, at: nil, completionCallbackType: .dataPlayedBack){ [self] _ in                 reverbNode.removeTap(onBus: 0)             }             // start             try engine.start()             playerNode.play()                          let url2 = URL(fileURLWithPath: fileInDocumentsDirectory(filename: "changed.wav"))             let outputFile = try! AVAudioFile(forWriting: url2, settings: playerNode.outputFormat(forBus: 0).settings)             reverbNode.installTap(onBus: 0, bufferSize: AVAudioFrameCount(reverbNode.outputFormat(forBus: 0).sampleRate), format: reverbNode.outputFormat(forBus: 0)) { (buffer, when) in                 do {                     try outputFile.write(from: buffer)                 } catch let error {                     print(error)                 }             }         } catch {             print(error.localizedDescription)         }     }     func getDocumentsURL() -> NSURL {         let documentsURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)[0] as NSURL         return documentsURL     }          func fileInDocumentsDirectory(filename: String) -> String {         let fileURL = getDocumentsURL().appendingPathComponent(filename)         return fileURL!.path     } } Is there a way to complete the writing without waiting for the playback to complete? My ideal is to complete the write in the time required by CPU and storage performance. It seems that reverbNode.installTap(...) { (buffer, when) in ...} in the code is processed in parallel with the current playback position, so I would like to dramatically improve the processing speed. Best regards.
0
0
1.2k
May ’22
AVAudiounitTimepitch issue iOS 16
Hi After IOS 16 update users are experiencing audio distortion when attempting to change Pitch and/or Tempo in the app which uses AVAudiounitTimepitch to achieve this. Prior to IOS 16 it has worked quite admirably. I see suggestions to change the algorithm used but am able to achieve this. Any pointers are appreciated
Replies
2
Boosts
0
Views
1.6k
Activity
Apr ’23
Audio Engine failure when attaching AVAudioUnitTimeEffect
According to the Apple docs, you should be able to connect audio nodes to an AVAudioEngine instance during runtime, but I'm getting a crash while trying to do so, in particular, when trying to connect instances of AVAudioUnitTimePitch or AVAudioUnitVarispeed to an AVAudioEngine with manual rendering mode enabled. The error message I get is: Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'player started when in a disconnected state' In my code, first, I configure the audio engine: let engine = AVAudioEngine() let format = AVAudioFormat(standardFormatWithSampleRate: 48000, channels: 2)! try! engine.enableManualRenderingMode(.offline, format: format, maximumFrameCount: 1024) try! engine.start() Then, I try to attach the player to the engine: let player = AVAudioPlayerNode() configureEngine(player: player, useVarispeed: true) player.play() // this is the line that causes the crash Finally, this is the function I use to configure the engine nodes graph: func configureEngine(player: AVAudioPlayerNode, useVarispeed: Bool) { engine.attach(player) guard useVarispeed else { engine.connect(player, to: engine.mainMixerNode, format: format) return } let varispeed = AVAudioUnitVarispeed() engine.attach(varispeed) engine.connect(player, to: varispeed, format: format) engine.connect(varispeed, to: engine.mainMixerNode, format: format) } If I pass false as the value for the useVarispeed parameter, the crash goes away. What is even more interesting is that if I add a dummy player node before starting the engine, the crash goes away too 🤷‍♂️ Could anyone please add clarity on what's going on here? Is this a bug or a limitation of the framework that I'm not aware of? Here's a simple project demonstrating the problem: https://github.com/rlaguilar/AVAudioEngineBug
Replies
0
Boosts
0
Views
1.6k
Activity
Mar ’23
AVAudioEngine crash when connecting inputNode to mainMixerNode
I have the following code to connect inputNode to mainMixerNode of AVAudioEngine: public func setupAudioEngine() { self.engine = AVAudioEngine() let format = engine.inputNode.inputFormat(forBus: 0) //main mixer node is connected to output node by default engine.connect(self.engine.inputNode, to: self.engine.mainMixerNode, format: format) do { engine.prepare() try self.engine.start() } catch { print("error couldn't start engine") } engineRunning = true } But I am seeing a crash in Crashlytics dashboard (which I can't reproduce). Fatal Exception: com.apple.coreaudio.avfaudio required condition is false: IsFormatSampleRateAndChannelCountValid(format) Before calling the function setupAudioEngine I make sure the AVAudioSession category is not playback where mic is not available. The function is called where audio route change notification is handled and I check this condition specifically. Can someone tell me what I am doing wrong? Fatal Exception: com.apple.coreaudio.avfaudio 0 CoreFoundation 0x99288 __exceptionPreprocess 1 libobjc.A.dylib 0x16744 objc_exception_throw 2 CoreFoundation 0x17048c -[NSException initWithCoder:] 3 AVFAudio 0x9f64 AVAE_RaiseException(NSString*, ...) 4 AVFAudio 0x55738 AVAudioEngineGraph::_Connect(AVAudioNodeImplBase*, AVAudioNodeImplBase*, unsigned int, unsigned int, AVAudioFormat*) 5 AVFAudio 0x5cce0 AVAudioEngineGraph::Connect(AVAudioNode*, AVAudioNode*, unsigned long, unsigned long, AVAudioFormat*) 6 AVFAudio 0xdf1a8 AVAudioEngineImpl::Connect(AVAudioNode*, AVAudioNode*, unsigned long, unsigned long, AVAudioFormat*) 7 AVFAudio 0xe0fc8 -[AVAudioEngine connect:to:format:] 8 MyApp 0xa6af8 setupAudioEngine + 701 (MicrophoneOutput.swift:701) 9 MyApp 0xa46f0 handleRouteChange + 378 (MicrophoneOutput.swift:378) 10 MyApp 0xa4f50 @objc MicrophoneOutput.handleRouteChange(note:) 11 CoreFoundation 0x2a834 __CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__ 12 CoreFoundation 0xc6fd4 ___CFXRegistrationPost_block_invoke 13 CoreFoundation 0x9a1d0 _CFXRegistrationPost 14 CoreFoundation 0x408ac _CFXNotificationPost 15 Foundation 0x1b754 -[NSNotificationCenter postNotificationName:object:userInfo:] 16 AudioSession 0x56f0 (anonymous namespace)::HandleRouteChange(AVAudioSession*, NSDictionary*) 17 AudioSession 0x5cbc invocation function for block in avfaudio::AVAudioSessionPropertyListener(void*, unsigned int, unsigned int, void const*) 18 libdispatch.dylib 0x1e6c _dispatch_call_block_and_release 19 libdispatch.dylib 0x3a30 _dispatch_client_callout 20 libdispatch.dylib 0x11f48 _dispatch_main_queue_drain 21 libdispatch.dylib 0x11b98 _dispatch_main_queue_callback_4CF 22 CoreFoundation 0x51800 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ 23 CoreFoundation 0xb704 __CFRunLoopRun 24 CoreFoundation 0x1ebc8 CFRunLoopRunSpecific 25 GraphicsServices 0x1374 GSEventRunModal 26 UIKitCore 0x514648 -[UIApplication _run] 27 UIKitCore 0x295d90 UIApplicationMain 28 libswiftUIKit.dylib 0x30ecc UIApplicationMain(_:_:_:_:) 29 MyApp 0xc358 main (WhiteBalanceUI.swift) 30 ??? 0x104b1dce4 (Missing)
Replies
1
Boosts
0
Views
2.5k
Activity
Mar ’23
Is there any way to handle the AVPlayer buffer?
Hi. I'm audio/video player developer. I implemented player as AVAudioPlayerNode to handle audio buffer. Suddenly, I wondered how great it would be if AVPlayer could handle buffers. I would like to know if there is a way to handle the buffer in AVPlayer, and if not, can you disclose it in the future? Thanks.
Replies
0
Boosts
0
Views
1.3k
Activity
Mar ’23
Microphone feedback noise and can I use the output to recognise?
I recently released my first ShazamKit app, but there is one thing that still bothers me. When I started I followed the steps as documented by Apple right here : https://developer.apple.com/documentation/shazamkit/shsession/matching_audio_using_the_built-in_microphone however when I was running this on iPad I receive a lot of high pitched feedback noise when I ran my app with this configuration. I got it to work by commenting out the output node and format and only use the input. But now I want to be able to recognise the song that’s playing from the device that has my app open and was wondering if I need the output nodes for that or if I can do something else to prevent the Mic. Feedback from happening. In short: What can I do to prevent feedback from happening Can I use the output of a device to recognise songs or do I just need to make sure that the microphone can run at the same time as playing music? Other than that I really love the ShazamKit API and can highly recommend to have a go with it! This is the code as documented in the above link (I just added the comments of what broke it for me) func configureAudioEngine() { // Get the native audio format of the engine's input bus. let inputFormat = audioEngine.inputNode.inputFormat(forBus: 0) // THIS CREATES FEEDBACK ON IPAD PRO let outputFormat = AVAudioFormat(standardFormatWithSampleRate: 48000, channels: 1) // Create a mixer node to convert the input. audioEngine.attach(mixerNode) // Attach the mixer to the microphone input and the output of the audio engine. audioEngine.connect(audioEngine.inputNode, to: mixerNode, format: inputFormat) // THIS CREATES FEEDBACK ON IPAD PRO audioEngine.connect(mixerNode, to: audioEngine.outputNode, format: outputFormat) // Install a tap on the mixer node to capture the microphone audio. mixerNode.installTap(onBus: 0, bufferSize: 8192, format: outputFormat) { buffer, audioTime in // Add captured audio to the buffer used for making a match. self.addAudio(buffer: buffer, audioTime: audioTime) } }
Replies
3
Boosts
0
Views
3k
Activity
Feb ’23
Are any of the ML frameworks real-time safe for audio processing?
I'm working on an audio processing app and am creating an AVAudioUnit extension as a part of it. I need to train a small neural network in the app and use it to process audio in real-time in the AudioUnit. The network is mostly convolutions and is ideal for running on the GPU but it should run in real-time on the CPU. The problem that I'm currently facing is that none of the ML frameworks seem to be safe to use for inference within custom AVAudioUnit kernels. My understanding is that only C and C++ should be used in these kernels (in addition to the other rules of real-time computing). Objective-C and Swift are discouraged per the documentation. My background is primarily in ML so I'm newer to Apple development and especially new to real-time development in this ecosystem. I've investigated CoreML, MPS, BNNS/Accelerate, and MLCompute so far but I'm not certain that any of them are safe to use. Any feedback would be greatly appreciated!
Replies
0
Boosts
1
Views
2.0k
Activity
Feb ’23
Process video audio using AVAudioEngine in iOS
I want to process audio from videos using AVAudioEngine, but I'm not sure how to read the audio from videos using AVAudioEngine. I have the following code: let videoURL = /// url pointing to a local video file. let file = try AVAudioFile(forReading: videoURL) It works fine on macOS but on iOS it fails with the error message: [default]          ExtAudioFile.cpp:193   about to throw 'typ?': open audio file [avae]            AVAEInternal.h:109   [AVAudioFile.mm:134:AVAudioFileImpl: (ExtAudioFileOpenURL((CFURLRef)fileURL, &_extAudioFile)): error 1954115647 Error Domain=com.apple.coreaudio.avfaudio Code=1954115647 "(null)" UserInfo={failed call=ExtAudioFileOpenURL((CFURLRef)fileURL, &_extAudioFile)} Reading through the header files it seems that 'typ?' corresponds to kAudioFileUnsupportedFileTypeError, so that tells me that the file type is supported on macOS but not on iOS. So my question is: How can I work with audio from video files in an AVAudioEngine based setup? I already know that I could extract the audio from videos using something like AVAssetExportSession, but that approach requires extra preprocessing time that I rather not spend.
Replies
0
Boosts
0
Views
1.5k
Activity
Jan ’23
AVAudioEngine: routing different AVAudioPlayerNodes to different channels
Hi, I have been searching all over for a way to do this on macOS: playing different stereo files on different pairs of audio outputs of the same hardware device. I am currently using AVAudioEngine with two AVAudioPlayerNodes and I can mix them and change the mapping of the entire mix through the use of AudioUnitSetProperty on the engine output, but I cannot have multiple AVAudioPlayerNodes play on different outputs. Obviously, being on macOS I cannot use AVAudioSession... Thank you if anyone has any idea on how to achieve this !
Replies
0
Boosts
1
Views
1.2k
Activity
Jan ’23
If not AVMIDIRecorder, then what?
For AVAudioPlayer, there is corresponding AVAudioRecorder. For AVMIDIPLayer, I found nothing for recording from the system's active MIDI input device. Can I record midi events from the system’s active input midi device, without resolving to low level CoreMidi? After configuring AVAudioUnitSampler with just a few lines of code, import AVFoundation var engine = AVAudioEngine() let unit = AVAudioUnitSampler() engine.attach(unit) engine.connect(unit, to: engine.outputNode, format: engine.outputNode.outputFormat (forBus:0)) try! unit.loadInstrument(at:sndurl) //url to .sf2 file try! engine.start() I could send midi events programmatically. // feeding AVAudioUnitMIDIInstrument with midi data let range = (0..<100) let midiStart = range.map { _ in UInt8.random(in: 70...90) } let midiStop = [0] + midiStart let times = range.map { _ in TimeInterval.random(in: 0...100) * 0.3 } for i in range {     DispatchQueue.main.asyncAfter(deadline: .now()+TimeInterval(times[i])){         unit.stopNote(midiStop[i], onChannel: 1)         unit.startNote(midiStart[i], withVelocity: 127, onChannel: 1)     } } But instead, I need to send midi events from a midi instrument, and tap to them for recording.
Replies
1
Boosts
0
Views
1.8k
Activity
Dec ’22
AVAudioPlayerNode crashes on play
I have a very simple setup involving AVAudioEngine, and a player node, AVAudioPlayerNode.The player node is attached to the audio engine, then connected to the engine's main mixer node.Then, to start outputing some sound, I start the engine, and finally play the node :NSError* err = nil; BOOL started = [_audioEngine startAndReturnError:&amp;amp;err]; if(started) { [_playerNode play]; } else { // handle error }It happens sometimes, the app crashes (on the call to the play function) because, as the system says : "player started when engine not running".I can't understand how this can happen, at least in the code above.No error is returned in `err`.Does somebody knows what happen or faced such a situation ?
Replies
3
Boosts
1
Views
3.8k
Activity
Nov ’22
[AVAudioSession sharedInstance] cost long time to return in UE4 app
Our UE4 game is stuned when received notification from other apps. It will take 50ms-500ms to call [AVAudioSession sharedInstance]. This cause the main game loop can not to call FAppEntry::Tick() before the func returns.
Replies
1
Boosts
0
Views
1k
Activity
Sep ’22
Apple.PHASE Unity Plug-in as AVAudioNode
Hi, my name is Jakob, we are working on sound creation apps with a visual and spatial approach in Unity. For iOS, we are using the "Unity as a library"-approach, so we have a native app part which is written in Swift and access to the AVAudioSession. Everything works great, except for one feature: We want to send the audio data generated with our app as binaural mix to another host app. To do so, we need to feed that data into an AVAudioUnit in our native iOS app. There are different approaches and we keep on making research and working on solutions. Could your new native spatial audio Unity plug-in Apple.PHASE might be a direct bridge feeding an AVAudioNode and combine the needs in between music and spatial audio content creators. Thanks a lot in advance & all the best!
Replies
0
Boosts
0
Views
1.7k
Activity
Sep ’22
Playing Audio
So I have successfully triggered a PTT notification, but when I try to play audio – any audio – it doesn't play. Seems to be an issue with initiating my AVAudioSession. If I do not initiate it, the sound plays (outside of the didActivateAudioSession; such as on view did load), so I know that it's not the audio playing code. For some reason, the AVAudioSession is not allowing me to play sound. Even when I put "PlayandRecord" and when I put "mix" in the options
Replies
2
Boosts
0
Views
1.8k
Activity
Sep ’22
AVAudioEngine and c++?
Im guessing i already know the answer to this but is AVAudioEngine class accessible in c++? No matter what i try i can't seem to get a pointer to it.
Replies
1
Boosts
0
Views
1.3k
Activity
Sep ’22
AVAudioEngine exception - required condition is false format.sampleRate == hwFormat.sampleRate
I see in Crashlytics few users are getting this exception when connecting the inputNode to mainMixerNode in AVAudioEngine: Fatal Exception: com.apple.coreaudio.avfaudio required condition is false: format.sampleRate == hwFormat.sampleRate Here is my code: self.engine = AVAudioEngine() let format = engine.inputNode.inputFormat(forBus: 0) //main mixer node is connected to output node by default engine.connect(self.engine.inputNode, to: self.engine.mainMixerNode, format: format) Just want to understand how can this error occur and what is the right fix?
Replies
2
Boosts
1
Views
3.3k
Activity
Aug ’22
Why is internalRenderBlock fetched and used before allocateRenderResources() is called?
I have a working AUv3 AUAudioUnit app extension but I had to work around a strange issue: I found that the internalRenderBlock value is fetched and invoked before allocateRenderResources() is called. I have not found any documentation stating that this would be the case, and intuitively it does not make any sense. Is there something I am doing in my code that would be causing this to be the case? Should I *force* a call to allocateRenderResources() if it has not been called before internalRenderBlock is fetched? Thanks! Brad
Replies
6
Boosts
1
Views
3.1k
Activity
Jul ’22
How does toggling AVAudioSession active state affect the AVAudioPlayerNode
Every time the AVAudioSession category is re-activated ( after being inactivated) and the audioengine is restarted (calling stop() and play()), the output from audioplayer node seems to ignore the audio session category, until explicitly connecting the audio player node again using audioEngine.connect(playerNode, to: audioEngine.outputNode, format: audioFile.processingFormat) The documentation regarding this behavior is not clear and would like to clarify the following. Should audioEngine.connect be called everytime the AVAudioSession is activated? Should audioEngine.connect be called after the audio player engine is stopped (audioEngine.stop())?
Replies
0
Boosts
0
Views
950
Activity
Jun ’22
How do I mix 2 hardware audio devices and record them as 1 audio track?
I am trying to mix the audio from 2 different hardware audio devices together in real-time and record the results. Does anybody have any idea how to do this? This is on macOS. Things I have tried and why it didn't work: Adding 2 audio AVCaptureDevices to an AVCaptureMovieFileOutput or AVAssetWriter. This results in a file that has 2 audio tracks. This doesn't work for me for various reasons. Sure I can mix them together with an AVAssetExportSession, but it needs to be real-time. Programmatically creating an aggregate device and recording that as an AVCaptureDevice. This "sort of" works, but it always results in a recording with strange channel issues. For example, if I combine a 1 channel mic and a 2 channel device, I get a recording with 3 channel audio (L R C). If I make an aggregate out of 2 stereo devices, I get a recording with quadraphonic sound(L R Ls Rs), which won't even play back on some players. If I always force it to stereo, all stereo tracks get turned to mono for some reason. Programmatically creating an aggregate device and trying to use it in an AVAudioEngine. I've had multiple problems with this, but the main one is that when the aggregate device is an input node, it only reports the format of its main device, and no sub-devices. And I can't force it to be 3 or 4 channels without errors. Use an AVCaptureSession to output the sample buffers of both devices, then convert and put those samples into their own AVPlayerNodes. Then mix those AVPlayerNodes into an AVAudioEngine mixer. This actually works, but the resulting audio lags so far behind real-time, that it is unusable. If I record a webcam video along with the audio, the lip-sync is off by like half a second. I really need help with this. If anybody has a way to do this, let me know. Some caveats that have also been tripping me up: The hardware devices that need to be recorded might not be the default input device for the system. The MBP built in mic might be the default device, but I need to record 2 other devices and disclose the built in mic. The devices usually don't have the same audio format. I might be mixing an lpcm mono int16 interleaved with a lpcm stereo float32 non-interleaved. It absolutely has to be real-time and 1 single audio track. It shouldn't be this hard, right?
Replies
1
Boosts
1
Views
1.5k
Activity
May ’22
About reducing size of AVAudioPCMBuffer
Hi, I'm trying to send audio data via UDP. I am using Network.framework with networking, so to use send method in NWConnection sending data must be Data type or confirm to DataProtocol. To satisfy those conditions, I have implemented a method to convert from AVAudioPCMBuffer type to Data type. func makeDataFromPCMBuffer(buffer: AVAudioPCMBuffer, time: AVAudioTime) -> Data {         let audioBuffer = buffer.audioBufferList.pointee.mBuffers         let data: Data!         data = .init(bytes: audioBuffer.mData!, count: Int(audioBuffer.mDataByteSize))         return data     } Implementation above is referenced from this post The problem is that the size of converted data is too big to fit in UDP datagram and error below occurs when I try to send data. I have found out that initial size of buffer is too big to fit in maximumDatagramSize. Below is code regarding to buffer.         let tapNode: AVAudioNode = mixerNode         let format = tapNode.outputFormat(forBus: 0)         tapNode.installTap(onBus: 0, bufferSize: 4096, format: format, block: { (buffer, time) in          // size of buffer: AVAudioPCMBuffer is 19200 already.             let bufferData = self.makeDataFromPCMBuffer(buffer: buffer, time: time)             sharedConnection?.sendRecordedBuffer(buffer: bufferData)         }) I need to reduce size of AVAudioPCMBuffer to fit in UDP datagram, But I can't find right way to do it. What would be best way to make data fit in datagram? I thought of dividing data in half, but this is UDP so I'm not sure how to handle those datas when one data has lost. So I'm trying to make AVAudioPCMBuffer fit in datagram. Any help would be very appreciated!
Replies
0
Boosts
0
Views
1.1k
Activity
May ’22
I want to instantly save the sound with AVAudioEngine's effect.
Hi, I'm creating a process to read an existing audio file, add an effect using AVAudioEngine, and then save it as another audio file. However, with the following method using an AVAudioPlayerNode, the save process must wait until the end of playback. import UIKit import AVFoundation class ViewController: UIViewController {          let engine = AVAudioEngine()     let playerNode = AVAudioPlayerNode()     let reverbNode = AVAudioUnitReverb()          override func viewDidLoad() {         super.viewDidLoad()         do {                          let url = URL(fileURLWithPath: Bundle.main.path(forResource: "original", ofType: "mp3")!)             let file = try AVAudioFile(forReading: url)                          // playerNode             engine.attach(playerNode)             // reverbNode             reverbNode.loadFactoryPreset(.largeChamber)             reverbNode.wetDryMix = 5.0             engine.attach(reverbNode)                          engine.connect(playerNode, to: reverbNode, format: file.processingFormat)             engine.connect(reverbNode, to: engine.mainMixerNode, format: file.processingFormat)             playerNode.scheduleFile(file, at: nil, completionCallbackType: .dataPlayedBack){ [self] _ in                 reverbNode.removeTap(onBus: 0)             }             // start             try engine.start()             playerNode.play()                          let url2 = URL(fileURLWithPath: fileInDocumentsDirectory(filename: "changed.wav"))             let outputFile = try! AVAudioFile(forWriting: url2, settings: playerNode.outputFormat(forBus: 0).settings)             reverbNode.installTap(onBus: 0, bufferSize: AVAudioFrameCount(reverbNode.outputFormat(forBus: 0).sampleRate), format: reverbNode.outputFormat(forBus: 0)) { (buffer, when) in                 do {                     try outputFile.write(from: buffer)                 } catch let error {                     print(error)                 }             }         } catch {             print(error.localizedDescription)         }     }     func getDocumentsURL() -> NSURL {         let documentsURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)[0] as NSURL         return documentsURL     }          func fileInDocumentsDirectory(filename: String) -> String {         let fileURL = getDocumentsURL().appendingPathComponent(filename)         return fileURL!.path     } } Is there a way to complete the writing without waiting for the playback to complete? My ideal is to complete the write in the time required by CPU and storage performance. It seems that reverbNode.installTap(...) { (buffer, when) in ...} in the code is processed in parallel with the current playback position, so I would like to dramatically improve the processing speed. Best regards.
Replies
0
Boosts
0
Views
1.2k
Activity
May ’22