Low latency host code for AU v3 (remoteIO)?

Question

Created Oct ’16

Replies 4

Boosts 0

Views 1.9k

Participants 2

I have written a realtime audio processing app for iOS that uses the Superpowered framework, which in turn uses the v2 RemoteIO Audio component to achieve the lowest possible audio latency ADC -> DSP -> DAC. I would now like to modernize my app so that it uses AU v3 plumbing instead, which will make it more portable to Mac (among other things). But the Apple example app for hosting a v3 AU is based on AVAudioEngine, which I do not believe is intended for minimum audio latency.

I cannot seem to find any example host code for v3 AUs which is intended for low latency. I'm rather surprised by this - although it seems in general that the v3 AudioUnit architecture is underdocumented.

Can anyone point me to v3 AU host code which uses RemoteIO (or whatever the low latency replacement is?).

On a related note, the 2015 WWDC video on v3 AUs mentions that v3 AUs run out of process, but doesn't discuss the latency or overhead implications on iOS at all. Can anyone comment on this?

I'm beginning to wonder whether investing time in the v3 AU framework is worth the time.

- Andy

Answered by jandyman in 188846022

After a few hours of pain and some help from a friend, I realized that AVAudioEngine is actually targeted for low latency directly, so I don't need to use RemoteIO anymore. This test code works:

let session = AVAudioSession.sharedInstance()

do { try session.setCategory(AVAudioSessionCategoryPlayAndRecord) }

catch { fatalError("Can't set Audio Session category") }

do { try session.setPreferredIOBufferDuration(2e-3) }

catch { fatalError("Can't set preferred buffer size") }

let input = engine.inputNode

let inFormat = input!.inputFormat(forBus: 0)

engine.connect(input!, to: engine.outputNode, format: inFormat)

do { try engine.start() }

catch { fatalError("Can't start audio engine") }

let bufDur = session.ioBufferDuration

When I check bufDur with native HW, I get about 1.5 mSecs, which corresponds to 44.1 Khz at a buffer size of 64. When I play instrument through the iPad, I hear no noticeable delay.

I wish this was more clear in the Apple docs. Often I feel like throwing things at the wall when dealing with Apple documentation.

Boost

Answer 1

hotpaw2 OP

Oct ’16

Search on github dot com for hotpaw2. I have a short Github Gist posted there in Swift 3 for an Audio Unit v3 host for the iOS RemoteIO Audio Unit. Seems to permit extremely low latencies (callback buffers even shorter than 256 samples) on an iPhone 6s or 7.

0

Answer 2

jandyman OP

Oct ’16

Accepted Answer

After a few hours of pain and some help from a friend, I realized that AVAudioEngine is actually targeted for low latency directly, so I don't need to use RemoteIO anymore. This test code works:

let session = AVAudioSession.sharedInstance()

do { try session.setCategory(AVAudioSessionCategoryPlayAndRecord) }

catch { fatalError("Can't set Audio Session category") }

do { try session.setPreferredIOBufferDuration(2e-3) }

catch { fatalError("Can't set preferred buffer size") }

let input = engine.inputNode

let inFormat = input!.inputFormat(forBus: 0)

engine.connect(input!, to: engine.outputNode, format: inFormat)

do { try engine.start() }

catch { fatalError("Can't start audio engine") }

let bufDur = session.ioBufferDuration

When I check bufDur with native HW, I get about 1.5 mSecs, which corresponds to 44.1 Khz at a buffer size of 64. When I play instrument through the iPad, I hear no noticeable delay.

I wish this was more clear in the Apple docs. Often I feel like throwing things at the wall when dealing with Apple documentation.

0

Answer 3

jandyman OP

Oct ’16

Thanks for this! See my reply - I've figured out how to get AVAudioEngine to do this as well.

Note that in my world, a buffer size of 256 (at 44.1 or 48 Khz) is not low latency. I like to get down to the 64 sample range. I was able to do that with Superpowered (basically RemoteIO), and now I can do that with AVAudioEngine

0

Answer 4

hotpaw2 OP

Oct ’16

Note that the minimum audio buffer size that an app can successfully get might depend on the iOS device model (and possibly the OS version as well), as well as potentially whether the device is locked or not.

0