PushToTalk

What problem are you actually having?

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

In the above scenario, the sound recorded with AVAudioRecorder is all noise.

In the above scenario, the sound recorded with AVAudioRecorder is all noise.

Like most of our higher-level audio APIs, AVAudioRecorder will work with the PushToTalk (or CallKit) framework. The issue here is that the PushToTalk framework is tightly integrated into our audio infrastructure and, as part of that integration, the activation of your audio session MUST be triggered by the PushToTalk system, NOT your app.

Unfortunately, AVAudioRecorder does its own activation, which will then break the PTT system. You'll need to use a lower-level audio API, typically AVAudioEngine. For an example of how the audio side of this, please see the Speakerbox CallKit sample.

It may seem odd that I'm recommending a CallKit sample, but the thing to understand is that CallKit and the PushToTalk framework share exactly the same underlying implementation. Both frameworks are managed callservicesd and, internally, they actually share a common implementation, particularly when it comes to the audio system.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Firstly, I am using AVAudioEngine for recording implementation. We are not currently using CallKit. The audio recorded by AVAudioEngine still has noise in the above scenario (when PushToTalk is turned on and the app is switched to the background, the app is in a state of long-term survival in the background, and then AVAudioEngine recording is started by clicking the Bluetooth device button), but I found that the audio recorded after clicking the intercom button on the PushToTalk interface has no noise. The audio recorded by clicking the Bluetooth device button is noise. The method called by Bluetooth devices is (requestBeginTransmitting), which enables AVAudioEngine recording in the (channelManager: PTChannelManager, didActivate audioSession: AVAudioSession) method. Is there any additional setting for the intercom button on the PushToTalk interface?

The “Zello Walkie Talkie” app's intercom function is perfectly implemented

The audio recorded by clicking the Bluetooth device button is noise. The method called by Bluetooth devices is (requestBeginTransmitting), which enables AVAudioEngine recording in the (channelManager: PTChannelManager, didActivate audioSession: AVAudioSession) method.

First off, I want to clarify that possible background activation paths here:

1. The user pushes the "talk" button using the systems on screen UI.

2. The app is Bluetooth authorized and adds directly calls requestBeginTransmitting() because of a BLE event from a proprietary accessory.

3. The app has set setAccessoryButtonEventsEnabled(...) to true and the user sends non-HFP* remote control event to the PTT system. The standard example of this is pressing the "play" button on an A2DP/HFP Bluetooth accessory that's currently in playback mode.

*HFP events are automatically handled by the system through the standard delegate system.

Expanding on #3, this is how the PTT system provides its own support for the experience many PTT apps had previously implemented that allowed PTT apps to be controlled through "standard" A2DP/HFP headset. This user experience is reasonably straightforward, but the details of how it actually functions are more complicated than they look.

Based on the fact you're calling requestBeginTransmitting(), I'm going to assume you're referring to #2, but please let me know if I've misunderstood. That leads to here:

The audio recorded by AVAudioEngine still has noise in the above scenario (when PushToTalk is turned on and the app is switched to the background, the app is in a state of long-term survival in the background, and then AVAudioEngine recording is started by clicking the Bluetooth device button), but I found that the audio recorded after clicking the intercom button on the PushToTalk interface has no noise.

The short answer here is that I think there's an issue with the details of your activation code. The thing to understand here is that #2 above doesn't actually involve the audio system AT ALL. The system uses Bluetooth authorization to confirm that you're "allowed" to trigger requestBeginTransmitting from that background, after which activation proceeds exactly as it would in the foreground.

However, what tends to break down here goes back to this issue:

The issue here is that the PushToTalk framework is tightly integrated into our audio infrastructure and, as part of that integration, the activation of your audio session MUST be triggered by the PushToTalk system, NOT your app.

Expanding on that point, the problem here is that "setActive" will work in the foreground (because the foreground app always has control over the audio system) but will NOT work correctly in the background. That dynamic is why PTT/CallKit apps often have audio problems that only show up in the foreground.

That then leads to here...

I found that the audio recorded after clicking the intercom button on the PushToTalk interface has no noise.

The only difference between #2 and #1/#3 is that #2 starts with your app calling requestBeginTransmitting(), which #1/#3 both "start" with your app receiving channelManager(_:channelUUID:didBeginTransmittingFrom:). That means any difference your app is experiencing is probably tied to whatever your app is doing when it starts that process. Typically that means your either reconfiguring your audio objects at the wrong time, or you're directly calling setActive().

Is there any additional setting for the intercom button on the PushToTalk interface?

No.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

This is my PTT code 1: Click on the intercom button and Bluetooth device button on the PushToTalk interface. The recording all starts in this method (channelManager: PTChannelManager, didActivate AudioSession: AVAudioSession) 2: There is no problem with recording audio by clicking the intercom button on the PushToTalk interface. But there are problems with clicking on recorded audio on Bluetooth devices (all noise or no sound during playback) 3: The method called after the application switches to the background (setupChannelManager (channelUUID: UUID)) 4: The method called by clicking the Bluetooth button (requestBeginTransmitting()) 5: The above scenarios are all when the application switches to the background (the application has set the background to keep alive and Background Modes) 6: All functions are normal without any issues when the application switches to the frontend.

So, let me start here:

But there are problems with clicking on recorded audio on Bluetooth devices (all noise or no sound during playback).

The thing to keep in mind here is that Bluetooth has two entirely separate audio mechanisms, which have very different levels of audio quality and behavior:

• A2DP -> Playback only, higher quality, modeled as "music player".

• HFP -> Play and record, lower quality audio, modeled as a "phone".

Switching between those modes isn't immediate, so manipulating the audio session configuration can change how audio sounds and/or cause odd "glitches". These issues will also be "worse" in the background, where your app no longer has direct/less control over the audio session state.

In terms of your code, there are a few different issues:

(1)

AVAudioPlayer should not be used for exactly the same reasons you should not use AVAudioRecorder. All your playback and recording should be going through AVAudioEngine.

(2)

This code here is wrong and doesn't really work, at least not they way you'd want:

       func stopEngineRecording() {
           cleanupEngineRecording()
       }
       
       func cleanupEngineRecording(){
           inputNode.removeTap(onBus: bus)
           engine.stop()
           stopTransmitting()
           if let audioFilename = self.audioFilename {
               playRecording(url: audioFilename)
           }
           print("录音已停止")
       }
...
    func channelManager(_ channelManager: PTChannelManager, didDeactivate audioSession: AVAudioSession) {
        print("已停用")
        stopEngineRecording()
    }

You didn't include your audio session configuration, but I suspect it's been configured as "mixable". AVAudioPlayer is the switching you to "playback", which then allows you to activate from the background. However, this will cause audio issues as well as other issues with calls and the PTT system. The right approach here is to:

1. Do all playback and recording through AVAudioEngine.

2. Start playback by calling setActiveRemoteParticipant(...).

Since your full duplex, you could call "setActiveRemoteParticipant(...)" to immediately start "playback" or "chain" the events by:

• Calling stopTransmitting() to end transmission.

• Calling setActiveRemoteParticipant(...) in didDeactivate to tell the system you want to begin "playback".

• Start actual playback in didActivate, when the session goes active again.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

1: Turn on the PushToTalk status

2: The application switches to the background

3: Use a Bluetooth device to call up the microphone for audio recording and write the audio data to the file

Can you help me write the relevant code? ( I spent more than a month on this function, but it hasn't been solved)

Thank you.

3: Use a Bluetooth device to call up the microphone for audio recording and write the audio data to the file.

What is the “Bluetooth device" here? Is it:

(1) A standard, classic Bluetooth, consumer headset (A2DP/HFP)?

To support these, all you need to do is enable accessory events using "setAccessoryButtonEventsEnabled(...)", at which point the PTT framework will start delivering events through the standard delegate methods.

The main issue to be aware of here is that the entire process that makes this "work" is inherently a bit "weird" and can look somewhat "buggy". For example, many A2DP head units automatically send "play" on connect, which the PTT framework will convert to a "start transmission". The problem here is that the system doesn't really have any way to "know" exactly what kind of device it's dealing with (for example, differentiating between an headset and a head unit), which means any attempt to really "fix" one edge case ends up breaking other devices.

Two points to all of this:

1. Your app is going to need to do some amount of "user education" around this feature so that users know what to expect and aren't surprised by it.

2. While I'd expect most headset to work fine, I'm also sure there are headsets which simply do not work well, either because they use unexpected commands and/or their button interface makes it hard for the user to send the command the system is looking for.

(2) A 3rd party, Bluetooth Low Energy, ”PTT controller" which you're controlling through CoreBluetooth

I think this is the case you're dealing with, in which case I'm actually not sure what issue you'd really "have". An app that's Bluetooth authorized has very broad control over transmission, so all you really need to do is call "requestBeginTransmitting()" and let the flow proceed exactly as it would in the foreground. What's actually happening in your app when you do this? Does it fail when you're running the app through Xcode or only when you try and run the app on its own?

On that last point, one issue that can trip developers up is how background activation "chains" between different parts of the system. Take Bluetooth as an example, what's actually going on "under the hood" looks something like this:

1. Bluetooth event arrives.
2. bluetoothd takes an assertion to wake up your app.
3. Your app wakes up and processes the event from Bluetooth.
4. While processing that event, your app uses requestBeginTransmitting() to send a transmission request to callservicesd.
5. Your app finishes event processing.
6. bluetoothd ends its assertion (from #2)
7. callservicesd receives the transmission event (from #4).
8. callservicesd takes an assertion to wake up your app.
9. Your app processes the transmission request.

The problem here is that, in theory, there's a time gap between #6 and #7 where your app could suspend, disrupting the entire process. Even worse, the events from #4-> #8 are actually happening in parallel, which makes the actual behavior timing specific and hard to predict. Lastly, when EXACTLY a given daemon ends its assertion (#6) isn't actually defined and varies WILDLY between daemons and even features managed by the same daemon.

In any case, the solution to these issues is to manage your own lifetime. Call UIApplication.beginBackgroundTask(withName:...) when you receive the Bluetooth event (#3) and end the task when you're "done" (for example, once transmission has actually started). That will give you ~30s of background runtime, which is plenty of time to sort out anything you need to. For more information on these issues, see "Background Tasks Resources", particularly "UIApplication Background Task Notes" and "Testing and Debugging Code Running in the Background”.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

I found the problem.

1: Because the recording operation needs to press the Bluetooth button and hold it all the time to start recording. Let go and stop recording.

2: When you press and hold the Bluetooth device button. The mobile phone will always receive the data sent by the Bluetooth device. The recorded audio at this time is noise (the app is always running in the background).

3: But it is normal that there is no noise when the app is running in the foreground and repeats the recording operation.

4: Press and lift the Bluetooth device button to start recording. Wait for the recording. Press and lift again. Stop recording. At this time, the recording is normal and there is no noise.

5: So how to do it without changing the audio session mode when long pressing (long press will always receive the data sent by Bluetooth)

6: How to maintain the audio mode at the beginning