Issues with AVRoutePickerView channel switching

We are developing a voice call application that uses AVRoutePickerView, allowing users to switch between an iPhone, a speaker, and Bluetooth earphones. However, we have encountered an intermittent issue: when switching from the speaker to Bluetooth earphones, the earphones remain in a loading state, and the audio channel subsequently switches back to the speaker. How can we resolve this issue? Here is the sample code:

let session = AVAudioSession.sharedInstance()
        do {
            try session.setCategory(.playAndRecord, mode: .default, options: [.allowBluetooth])
            try session.overrideOutputAudioPort(isBluetoothConnected ? .none : .speaker)
            try session.setPreferredSampleRate(48000)
            try session.setPreferredIOBufferDuration(0.02)
            try session.setActive(true, options: [])
            INFO(" >>> start: hasBluetooth=\(isBluetoothConnected), inputs: \(session.currentRoute.inputs.map { $0.portType }) outputs: \(session.currentRoute.outputs.map { $0.portType })")
        } catch {
            ERROR("(error.localizedDescription)")
        }
Answered by DTS Engineer in 905851022

We are developing a voice call application that uses AVRoutePickerView, allowing users to switch between an iPhone, a speaker, and Bluetooth earphones. However, we have encountered an intermittent issue: when switching from the speaker to Bluetooth earphones, the earphones remain in a loading state, and the audio channel subsequently switches back to the speaker. How can we resolve this issue?

This isn't really going to work, at least not the way you think/hope it will. The problem here is that your CallKit works by manipulating your AudioSession "behind your back", basically converting it from a standard "PlayAndRecord" session into a special "PhoneCall" audio session.

That conversion is REALLY important because the underlying reason why your phone call doesn't act like a "normal" audio session. For example, the standard audio session priority rules say that:

  • You can't activate a non-mixable playback or ANY recording session from the background -> CallKit does this.

  • A foreground app (like a music player) will interrupt the existing audio session owner -> except that's not what phone calls do.

You can actually hear the difference if you listen closely, as the maximum volume of the phone audio session is slightly higher than the standard audio session.

In any case, the problem with manipulating the audio session like you're trying to do here is that directly calling "setActive" is going to end up doing one of two things:

  1. Best case, the phone audio session is already active, so the setActive will fail and nothing happens, since you can't activate a session that's already active.

  2. Worst case, the phone audio session isn't active, so the setActive works. Your PlayAndRecord audio session is now active, which means the PhoneCall audio session may not be able to activate properly.

Once you're in state #2, things get weird. Sometimes everything seems to work fine, as you basically end up with a half-configured phone session that sort of works. Worst case, you have a true "PlayAndRecord" audio session, which opens up the door to all sorts of problems. Most notably, an incoming call can cause a complete session interruption, suspending your app immediately, effectively ending your call with minimal notice.

That last issue is actually the biggest issue that led us to create CallKit in the first place. CallKit apps cannot activate their own sessions like this.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

We are developing a voice call application that uses AVRoutePickerView, allowing users to switch between an iPhone, a speaker, and Bluetooth earphones. However, we have encountered an intermittent issue: when switching from the speaker to Bluetooth earphones, the earphones remain in a loading state, and the audio channel subsequently switches back to the speaker. How can we resolve this issue?

This isn't really going to work, at least not the way you think/hope it will. The problem here is that your CallKit works by manipulating your AudioSession "behind your back", basically converting it from a standard "PlayAndRecord" session into a special "PhoneCall" audio session.

That conversion is REALLY important because the underlying reason why your phone call doesn't act like a "normal" audio session. For example, the standard audio session priority rules say that:

  • You can't activate a non-mixable playback or ANY recording session from the background -> CallKit does this.

  • A foreground app (like a music player) will interrupt the existing audio session owner -> except that's not what phone calls do.

You can actually hear the difference if you listen closely, as the maximum volume of the phone audio session is slightly higher than the standard audio session.

In any case, the problem with manipulating the audio session like you're trying to do here is that directly calling "setActive" is going to end up doing one of two things:

  1. Best case, the phone audio session is already active, so the setActive will fail and nothing happens, since you can't activate a session that's already active.

  2. Worst case, the phone audio session isn't active, so the setActive works. Your PlayAndRecord audio session is now active, which means the PhoneCall audio session may not be able to activate properly.

Once you're in state #2, things get weird. Sometimes everything seems to work fine, as you basically end up with a half-configured phone session that sort of works. Worst case, you have a true "PlayAndRecord" audio session, which opens up the door to all sorts of problems. Most notably, an incoming call can cause a complete session interruption, suspending your app immediately, effectively ending your call with minimal notice.

That last issue is actually the biggest issue that led us to create CallKit in the first place. CallKit apps cannot activate their own sessions like this.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

The application we developed allows iPhones to call devices running non-iOS operating systems, so CallKit cannot be used.

The implementation relies on the .playAndRecord session to provide input and output capabilities. However, we have split this into two separate chains: a "pure recording engine" and a "pure playback engine," thereby avoiding the issue where the iOS full duplex engine locks the output path. This approach enables simultaneous recording and playback with switchable routing.

When users use AVRoutePickerView to switch between the iPhone, speakers, and Bluetooth earphones, there is approximately a 20% chance of failure. In such cases, AVRoutePickerView reverts to the previous selection, and we do not receive any failure logs. Could you provide us with some suggestions?

The application we developed allows iPhones to call devices running non-iOS operating systems, so CallKit cannot be used.

I think you've misunderstood what CallKit actually "is". CallKit doesn't make calls and, in fact, doesn't really "do" anything. Fundamentally, CallKit is best understood as a specialized interface framework that VoIP apps use to manage their on-call user experience. It has some connection with the audio system but doesn't actually know (or care) how your app transfers data or even really “works". Indeed, CallKit's primary sample project is in fact a "fake call" app[1] that mimics the entire call experience without having ANY network code at all.

More to the point, the VAST majority of VoIP apps are cross-platform, so I don't see why that would prevent using CallKit.

When users use AVRoutePickerView to switch between the iPhone, speakers, and Bluetooth earphones, there is approximately a 20% chance of failure. In such cases, AVRoutePickerView reverts to the previous selection, and we do not receive any failure logs. Could you provide us with some suggestions?

How are you determining that failure rate? Have you ever directly reproduced it under controlled conditions or at least "seen" it occur? Do you have any user reports that provide any more detail or context?

[1] It is, in fact, the best fake call app on the platform, since its calls are literally indistinguishable from "real" calls.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

The reasons why CallKit cannot be used are as follows: our call scenario involves the iPhone actively calling an electric vehicle, which is a one-sided call; the UI interface requires customization; and the main reason is that the application needs to be listed on the Apple Store in China, where legal policies do not permit it.

How are you determining that failure rate? Have you ever directly reproduced it under controlled conditions or at least "seen" it occur? Do you have any user reports that provide any more detail or context?

During the actual call process, when switching between iPhone, speaker, and Bluetooth earphones using AVRoutePickerView, if I continuously switch the route 10 times, it will inevitably fail 1 to 2 times. The code determines the success of the switch by listening to the "AVAudioSession.routeChangeNotification" notification. When a failure occurs, no logs can be captured, and the only indication is that the checkmark icon on the AVRoutePickerView UI reverts to the previous selection.

The reasons why CallKit cannot be used are as follows: our call scenario involves the iPhone actively calling an electric vehicle, which is a one-sided call;

OK. So this isn't a true "voip" app (you're not using the "voip" background category) as such, you’re just doing foreground only PlayAndRecord.

and the main reason is that the application needs to be listed on the Apple Store in China, where legal policies do not permit it.

LiveCommunicationKit would work for that.

During the actual call process, when switching between iPhone, speaker, and Bluetooth earphones using AVRoutePickerView, if I continuously switch the route 10 times, it will inevitably fail 1 to 2 times.

I see. In terms of diagnosing the specific issue, the next step would be to capture a sysdiagnose of the issue and file a bug so the audio team can take a look.

Having said that, one point to be aware of here is that switching audio routes can be relatively time-consuming, particularly on Bluetooth where an audio route change can potentially involve switching the device back and forth between a2dp and HFP. This is also exacerbated by the fact that there is VERY wide variance in chipset performance, even with "new" devices[1]. That can create a situation where a route change failed because the accessory hadn't finished cleaning up after the previous route change.

That does lead to one point I'd missed here:

Try session.setCategory(.playAndRecord, mode: .default, options: [.allowBluetooth])

The allowBluetooth option has been deprecated for many years, replaced by the more specific allowBluetoothA2DP, allowBluetoothHFP, and the new bluetoothHighQualityRecording. I don't think the older option would directly cause what you're seeing, but it certainly isn't helping.

[1] It's not unusual for a new device (even newly released devices) to be using an older chipset, in some cases even VERY old chipsets.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware