Recording Microphone and Music at the same time with AVAudioEngine

Question

Created May ’16

Replies 5

Boosts 0

Views 3.8k

Participants 2

I'm currently trying to record a player node and input node at the same time but i'm having some difficulty. I can get them to record individually and at some point have set my code up to record both. The only problem is when I manage to do this the recorded content from the input node will play through the speakers as its recording creating an echo that eventually creates feedback that is too much to bear. So I've tried to shape my code as follows so the input node doesn't play through the speakers but i'm finding it difficult.

I'm trying to set up my code as follows:

Player Node

Input Node (mic)

    |                                  |

    |                                  |

    |                                  |

    |                                  |

    |                                  |

    v     AVAudioConnectionPoint       v

Main Mixer ----------------------------------------->> Second Mixer

| |

v Node Tap

Output

I've tried a number of combinations but I can't put them all here so i'll lay out some fundamental code and hope to Jesus someone has a little expertise in this area. First I create my engine and attach the nodes:

- (void)createEngineAndAttachNodes
{
    _engine = [[AVAudioEngine alloc] init];
    [_engine attachNode:_player];
    _inputOne = _engine.inputNode;
    _outputOne = _engine.outputNode;
}

Then I make the engine connections (this is where I need to know where i'm going wrong.)

- (void)makeEngineConnections
{

    _mainMixerOne = [_engine mainMixerNode];

    _secondaryMixerOne = [_engine mainMixerNode];

    _commonFormat = [[AVAudioFormat alloc] initWithCommonFormat:AVAudioPCMFormatFloat32 sampleRate:44100 channels:2 interleaved:false];

    AVAudioFormat *stereoFormat = [[AVAudioFormat alloc] initStandardFormatWithSampleRate:44100 channels:2];

    [_engine connect:_player to:_mainMixerOne format:stereoFormat];
    [_engine connect:_mainMixerOne to:_outputOne format:_commonFormat];

    [_engine connect:_inputOne to:_secondaryMixerOne format:_commonFormat];
    NSArray<AVAudioConnectionPoint *> *destinationNodes = [NSArray arrayWithObjects:[[AVAudioConnectionPoint alloc]
initWithNode:_mainMixerOne bus:1], [[AVAudioConnectionPoint alloc] initWithNode:_secondaryMixerOne bus:0], nil];
    [_engine connect:_player toConnectionPoints:destinationNodes fromBus:0 format:_commonFormat];

}

Whenever I try to directly connect one mixer to the other I get the error:

thread 1 exc_bad_access code=2

Finally we have where I try and put this all together in my record function. The function you see will record the player but not the input.

-(void)setupRecordOne
{



    NSError *error;
    if (!_mixerFileOne) _mixerFileOne = [NSURL URLWithString:[NSTemporaryDirectory() stringByAppendingString:@"mixerOutput.caf"]];

  /
    AVAudioFile *mixerOutputFile = [[AVAudioFile alloc] initForWriting:_mixerFileOne settings:_commonFormat.settings error:&error];
    NSAssert(mixerOutputFile != nil, @"mixerOutputFile is nil, %@", [error localizedDescription]);


    [self startEngine];
    [_secondaryMixerOne installTapOnBus:0 bufferSize:4096 format:_commonFormat block:^(AVAudioPCMBuffer *buffer, AVAudioTime *when) {
        NSError *error;
        BOOL success = NO;

        success = [mixerOutputFile writeFromBuffer:buffer error:&error];
        NSAssert(success, @"error writing buffer data to file, %@", [error localizedDescription]);
    }];
    _isRecording = YES;
}

The only way I can record the input is by setting the node tap as follows:

   [self startEngine];
[_inputOne installTapOnBus:0 bufferSize:4096 format:_commonFormat
block:^(AVAudioPCMBuffer *buffer, AVAudioTime *when) {
    NSError *error;
    BOOL success = NO;
    success = [mixerOutputFile writeFromBuffer:buffer error:&error];
    NSAssert(success, @"error writing buffer data to file, %@", [error localizedDescription]);
}];

But then it wont record the player.

Please tell me there is someone out there who knows what to do.

Thanks,

Joshua

Boost

Answer 1

theanalogkid OP

May ’16

Calling mainMixerNode on the engine the first time will create a "Main Mixer" and connect it to the Output while the second call will not create a new separate mixer as _secondaryMixerOne, all it will do is retrieve the previously created Main Mixer.

First call to mainMixerNode -> the engine will create a singleton mixer (Main) connect it to the output node and return it to you.

Second call to mainMixerNode -> the engine simply returns the mixer (Main) back to you.

From AVAudioEngine.h

@property mainMixerNode

@abstract

The engine's optional singleton main mixer node.

@discussion

The engine will construct a singleton main mixer and connect it to the outputNode on demand,

when this property is first accessed. You can then connect additional nodes to the mixer.

By default, the mixer's output format (sample rate and channel count) will track the format

of the output node. You may however make the connection explicitly with a different format.

@property (readonly, nonatomic) AVAudioMixerNode *mainMixerNode;

0

Answer 2

WaterMonkey OP

May ’16

Ok, so basically the whole concept of the second mixer is not the soultion.

Sure I can link the input node and player node to the same mixer and record them both. The only problem with that is the amount of feedback I get as the sound echoes and echoes through the speaker.

Connections:

    _mainMixerOne = [_engine mainMixerNode];
   
    _commonFormat = [[AVAudioFormat alloc] initWithCommonFormat:AVAudioPCMFormatFloat32
sampleRate:44100 channels:2 interleaved:false];

        [_engine connect:_player to:_mainMixerOne fromBus:0 toBus:0 format:_commonFormat];
        [_engine connect:_inputOne to:_mainMixerOne fromBus:0 toBus:2 format:_commonFormat];

Node Tap:

    AVAudioFile *mixerOutputFile = [[AVAudioFile alloc] initForWriting:_mixerFileOne settings:
     [[_mainMixerOne outputFormatForBus:0] settings] error:&error];
    NSAssert(mixerOutputFile != nil, @"mixerOutputFile is nil, %@", [error localizedDescription]);



    [self startEngine];
    [_mainMixerOne installTapOnBus:0 bufferSize:4096 format:_commonFormat
     block:^(AVAudioPCMBuffer *buffer, AVAudioTime *when) {
        NSError *error;
        BOOL success = NO;
   
        success = [mixerOutputFile writeFromBuffer:buffer error:&error];
        NSAssert(success, @"error writing buffer data to file, %@", [error localizedDescription]);
    }];

This code will record both the input microphone and the playernode but as mentioned, as its recording, the sound will play through the speaker and get louder and louder.

Any idea of how to get around this?

0

Answer 3

WaterMonkey OP

May ’16

Any advice or just a general point in the right direction would be greatly appreciated. This is the last complex thing I have to do to finish my app.

Thanks,

Joshua

0

Answer 4

WaterMonkey OP

May ’16

I've managed to get it all working rather well when headphones are plugged in. Although at the moment it is panning to the left for the voice recording.

When I record to avoid feedback the player routes to the top of the speaker while the mic records from the bottom. The only problem with this is, unless the headphones are plugged in, the player volume is very low as its not coming from the optimal speaker. If I override it to do so, the feedback is unbearable as the mic is recording so close to the player content.

I'm hoping that i'll be able to set a function with the devices microphone selection in order to record from a different input source to allow the player to play at full volume through the bottom speaker at least when the headphones are plugged in.

I'll keep you updated.

https://developer.apple.com/library/ios/qa/qa1799/_index.html

0

Answer 5

theanalogkid OP

May ’16

Not sure how to avoid the feedback loop if you're playing the input to the output since you are indeed creating the feedback loop. If you just wanted to record the input node and not have that audio routed though the mixer to the output that's doable - you can have your InputNode operate independently with a Tap without having the input also connected to the mixer. Take a look at AVAudioEngine in Practice, there's a discussion of using a Tap directly with an AVAudioInputNode.

0