Using VoiceProcessingIO for echo cancellation on macOS

Question

Hi! I am making a voice chat application for macOS and having hard times getting echo cancellation to work.

I've made an audio unit graph with the following structure:

┌─────────────┐               ┌──────────────┐               ┌───────────────────────┐
│             │               │              │               │                       │
│  My Render  │   48000 kHz   │  Resampler   │   44100 kHz   │  Voice Processing IO  │
│  Callback   │──────────────▶│     Node     │──────────────▶│         Node          │
│             │               │              │               │                       │
└─────────────┘               └──────────────┘               └───────────────────────┘

The code is available on GitHub Gist (250 lines):

https://gist.github.com/konstantinpavlikhin/d08f98b14328baa5eddbdf98d0ab8b91

I receive samples in -[XXXSpeakerDevice myStreamFormat] format from the network. VPIO seems to like 44100 kHz, that's why I added a resampler node (otherwise I've got errors on initialization). The problem is that this thing doesn't work at all. I've carefully checked return code of every function call and there are no errors at all. Yet my render callback doesn't get called.

The CAShow() prints the following:

[aqme] 254: AQDefaultOutput (1): skipping input stream 0 0 0x0
AudioUnitGraph 0x1299086:
  Member Nodes:
  node 1: 'aufc' 'conv' 'appl', instance 0x81299087 O I
  node 2: 'auou' 'vpio' 'appl', instance 0x81299088 O I
  Connections:
  node   1 bus   0 => node   2 bus   0  [ 1 ch,  44100 Hz, 'lpcm' (0x00000029) 32-bit little-endian float, deinterleaved]
  Input Callbacks:
  {0x1001e0c00, 0x608000437000} => node   1 bus   0  [1 ch, 48000 Hz]
  CurrentState:
  mLastUpdateError=0, eventsToProcess=F, isInitialized=T, isRunning=T (1)

And I also get this:

[vp] 17:  >vp> SERIOUS AUDIO STREAM CORRUPTION ERROR DETECTED IN kVPSignalIndex_DlFev : stream has been all zeros for 10.00 seconds!!

When I change kAudioUnitSubType_VoiceProcessingIO to kAudioUnitSubType_HALOutput everything works, but obviosly without echo cancellation.

I would greatly appreciate any help on this problem!

Core Audio

3.8k

Posted by

konstantinpavlikhin

Reply

Add a Comment

Answer 1

Does your code enable input via kAudioOutputUnitProperty_EnableIO?

Here's some generic setup code that I would expect to work...error handling removed.

OSStatus SetupOutputUnit(AURenderCallbackStruct inInputProc,
  AURenderCallbackStruct inRenderProc,
                         AudioUnit &outUnit,
  const AudioStreamBasicDescription &voiceIOFormat)
{
  OSStatus result = noErr;

  // Open the output unit
    AudioComponentDescription desc = { kAudioUnitType_Output,               // type
                                       kAudioUnitSubType_VoiceProcessingIO, // subType
                                       kAudioUnitManufacturer_Apple,        // manufacturer
                                       0, 0 };                              // flags

/** error checking removed **/
  AudioComponent comp = AudioComponentFindNext(NULL, &desc);

  result = AudioComponentInstanceNew(comp, &outUnit);

  UInt32 one; one = 1;
  result = AudioUnitSetProperty(outUnit, kAudioOutputUnitProperty_EnableIO, kAudioUnitScope_Input, 1, &one, sizeof(one));

  result = AudioUnitSetProperty(outUnit, kAudioOutputUnitProperty_SetInputCallback, kAudioUnitScope_Global, 1, &inInputProc, sizeof(inInputProc));

  result = AudioUnitSetProperty(outUnit, kAudioUnitProperty_SetRenderCallback, kAudioUnitScope_Input, 0, &inRenderProc, sizeof(inRenderProc));

  result = AudioUnitSetProperty(outUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &voiceIOFormat, sizeof(voiceIOFormat));

  result = AudioUnitSetProperty(outUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Output, 1, &voiceIOFormat, sizeof(voiceIOFormat));

  result = AudioUnitInitialize(outUnit);
                             
 return result; 
}

Also this post discussing working with the VoiceProcessingIO on macOS https://forums.developer.apple.com/message/157883

Posted by

theanalogkid

Add a Comment

Answer 2

Thank you for your reply!

> Does your code enable input via kAudioOutputUnitProperty_EnableIO?

Every time I try to enable IO on either input or output elements I get error -10865, which stands for kAudioUnitErr_PropertyNotWritable.

UInt32 value = 1;

// Always returns -10865.
OSStatus status = AudioUnitSetProperty(_voiceProcessingUnit, kAudioOutputUnitProperty_EnableIO, kAudioUnitScope_Input, 1, &value, sizeof(value));

// Always returns -10865.
status = AudioUnitSetProperty(_voiceProcessingUnit, kAudioOutputUnitProperty_EnableIO, kAudioUnitScope_Output, 0, &value, sizeof(value));

In VPIO unit IO seems to be enabled by default on both input and output elements and there is no way to toggle it.

> some generic setup code that I would expect to work...error handling removed.

I finally managed to get some positive results with VPIO when I switched to callbacks way of combining things, similar to yours. I am still puzzled by the fact neither AUGraph* API, nor declarative unit connections via kAudioUnitProperty_MakeConnection work with VPIO. It cost me a huge amount of trial and error time to try every possible combination starting with a highest level approach (v2 graphs) down to the lowest level one (callbacks).

> Also this post discussing working with the VoiceProcessingIO on macOS

Thanks! I've seen this thread before. It's great that this specific details are documented at least on the forums, because in my app I also need to implement devices (mic/speaker) switching.

Posted by

konstantinpavlikhin

Add a Comment

Using VoiceProcessingIO for echo cancellation on macOS

Replies