AVSpeechSynthesizer doesn't notifyOthersOnDeactivation

Hello, I am building a new iOS app which uses AVSpeechSynthesizer and should be able to mix audio nicely with audio from other apps. AVSpeechSynthesizer seems to handle setting the AVAudioSession to active on it's own, but does not deactivate the audio session. This leads to issues, namely that other audio sources remain "ducked" after AVSpeechSynthesizer is done speaking.

I have implemented deactivating the audio session myself, which "works", in that it allows other audio sources to become "un-ducked", but it throws this exception each time even though it appears successful.

Error Domain=NSOSStatusErrorDomain Code=560030580 "Session deactivation failed" UserInfo={NSLocalizedDescription=Session deactivation failed}

It appears to be a bug with how AVSpeechSynthesizer handles activating/deactivating the audio session.

Below is a minimal example which illustrates the problem. It has two buttons, one which manually deactivates the audio sessions, which throws the exception, but otherwise works, and another button which leaves audio session management to the AVSpeechSynthesizer but does not "un-duck" other audio.

If you play some audio from another app (ex: Music), you'll see the button which throws/catches an exception successfully ducks/un-ducks the audio, while the one without attempting to deactivate the session ducks but does not un-duck the audio.

import AVFoundation

struct ContentView: View {
    let workingSynthesizer = UnduckingSpeechSynthesizer()
    let brokenSynthesizer = BrokenSpeechSynthesizer()
    
    init() {
        let audioSession = AVAudioSession.sharedInstance()
        do {
            try audioSession.setCategory(.playback, mode: .voicePrompt, options: [.duckOthers])
        } catch {
            print("Setup error info: \(error)")
        }
    }
    
    var body: some View {
        VStack {
            Button("Works Correctly"){
                workingSynthesizer.speak(text: "Hello planet")
            }
            Text("-------")
            Button("Does not work"){
                brokenSynthesizer.speak(text: "Hello planet")
            }
        }
        .padding()
    }
}

class UnduckingSpeechSynthesizer: NSObject {
    var synth = AVSpeechSynthesizer()
    let audioSession = AVAudioSession.sharedInstance()

    override init(){
        super.init()
        synth.delegate = self
    }
    
    func speak(text: String) {
        let utterance = AVSpeechUtterance(string: text)
        synth.speak(utterance)
    }
}

extension UnduckingSpeechSynthesizer: AVSpeechSynthesizerDelegate {
    func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, didFinish utterance: AVSpeechUtterance) {
        do {
            try audioSession.setActive(false, options: .notifyOthersOnDeactivation)
        }
        catch {
            // always throws an error
            // Error Domain=NSOSStatusErrorDomain Code=560030580 "Session deactivation failed" UserInfo={NSLocalizedDescription=Session deactivation failed}
            print("Deactivate error info: \(error)")
        }
    }
}

class BrokenSpeechSynthesizer {
    var synth = AVSpeechSynthesizer()
    let audioSession = AVAudioSession.sharedInstance()

    func speak(text: String) {
        let utterance = AVSpeechUtterance(string: text)
        synth.speak(utterance)
    }
}

(I have a separate issue where the first speech attempt takes a few seconds but I don't think it's related)