Urgent Issue with SoundAnalysis in iOS 18 - Critical Background Permissions Error

Machine Learning & AI Core ML
Jaythaking OP
Created Oct ’24
Replies 12
Boosts 11
Views 1.4k
Participants 8

We are experiencing a major issue with the native .version1 of the SoundAnalysis framework in iOS 18, which has led to all our user not having recordings. Our core feature relies heavily on sound analysis in the background, and it previously worked flawlessly in prior iOS versions. However, in the new iOS 18, sound analysis stops working in the background, triggering a critical warning.

Details of the issue:

  • We are using SoundAnalysis to analyze background sounds and have enabled the necessary background permissions.
  • We are using the latest XCode
  • A warning now appears, and sound analysis fails in the background. Below is the warning message we are encountering:

Warning Message:

Execution of the command buffer was aborted due to an error during execution. Insufficient Permission (to submit GPU work from background)
[Espresso::handle_ex_plan] exception=Espresso exception: "Generic error": Insufficient Permission (to submit GPU work from background) (00000006:kIOGPUCommandBufferCallbackErrorBackgroundExecutionNotPermitted); code=7 status=-1
Unable to compute the prediction using a neural network model. It can be an invalid input data or broken/unsupported model (error code: -1).
CoreML prediction failed with Error Domain=com.apple.CoreML Code=0 "Failed to evaluate model 0 in pipeline" UserInfo={NSLocalizedDescription=Failed to evaluate model 0 in pipeline, NSUnderlyingError=0x30330e910 {Error Domain=com.apple.CoreML Code=0 "Failed to evaluate model 1 in pipeline" UserInfo={NSLocalizedDescription=Failed to evaluate model 1 in pipeline, NSUnderlyingError=0x303307840 {Error Domain=com.apple.CoreML Code=0 "Unable to compute the prediction using a neural network model. It can be an invalid input data or broken/unsupported model (error code: -1)." UserInfo={NSLocalizedDescription=Unable to compute the prediction using a neural network model. It can be an invalid input data or broken/unsupported model (error code: -1).}}}}}

We urgently need guidance or a fix for this, as our application’s main functionality is severely impacted by this background permission error. Please let us know the next steps or if this is a known issue with iOS 18.

Replies  12
Boosts  11
Views  1.4k
Participants  8
Ralph42 OP
Oct ’24

Hello,

I have exactly the same problem. It works in 15,16,17, but not in 18. Cannot be reproduced in the simulator. Only on real devices!

And yes my app 's main functionality is impacted by this issue!

3
Engineer OP
Apple
Oct ’24

Hello @Jaythaking, thank you for your post. If you haven't already, please use Feedback Assistant to submit a bug report, attach a sysdiagnose, and post here the FB number for my reference.

0
Ralph42 OP
Oct ’24

Hello @Engineer,

here are two FB's from other developers with the same issue:

FB15361195 FB15150861

Ralph

1
Poghos OP
Oct ’24

@here anyone aware if they fix this issue yet or not

0
Ralph42 OP
Nov ’24

Hi everyone,

I just tested under 18.2 Beta 2 and unfortunately the problem is still there.

Has anyone received any new information or feedback from Apple?

0
thinkingBanana OP
Nov ’24

Same issue, newest public release of iOS18 FB15747195

0
Borrecan OP
Nov ’24

I tested in iOS 18.2 Beta 3 is not working yet.

0
KMacTexas OP
Nov ’24

I just sent a highly detailed ticket to DTS using the WWDC21 session - 10036 sample code for audio analysis with background modes set. It does the same error we are all seeing with our own apps when the iPhone locks or the app goes to the background:

Insufficient Permission (to submit GPU work from background) (00000006:kIOGPUCommandBufferCallbackErrorBackgroundExecutionNotPermitted)

I'll reply back here as soon as I hear something.

2
KMacTexas OP
Nov ’24

Still have not heard back from Apple, but have discovered a few interesting things. It looks like:

SNClassifySoundRequest(classifierIdentifier: version1)

was never meant to run on the GPU in the background and it was a fluke it worked on iOS 17 and below. We are using the built-in model provided by the Sound Analysis framework - that ones seems to be optimized to run on the GPU only and thus causes the explosion we are seeing. If you look at:

SNClassifySoundRequest(mlModel: mlModel)

where you use your own model you can specify that it run on the CPU and this might work in the background. (We don't have our own model to test with since we are doing a proof of concept first using Apple's sounds). Leaving this here incase anyone wants to try it.

// create the configuration for the model
let configuration = MLModelConfiguration()
configuration.computeUnits = .cpuOnly // force computations to use only the CPU

do {
    // load your Core ML model with the configuration
    let mlModel = try MLModel(contentsOf: modelURL, configuration: configuration)
    
    // wrap the model into a Sound Analysis request
    let request = try SNClassifySoundRequest(mlModel: mlModel)
    
    // create your SNAudioStreamAnalyzer
    let analyzer = SNAudioStreamAnalyzer(format: audioFormat)
    
    // add the request to the analyzer
    try analyzer.add(request, withObserver: self)
} catch {
    print("Error setting up Sound Analysis: \(error)")
}
1
KMacTexas OP
Nov ’24

Apple DTS responded and asked me to submit the issue using FeedBack Assistant because they believe this issue would be best handled by the relevant engineering team directly, as it may require changes to the OS, tools, or the frameworks being used.

More as it occurs...

2
Borrecan OP
2w

This is the solution for background sound recognition.

Enable Background processing in Background Modes (Signing and Capabilities.)

1
MichaelNC OP
1w

Just tried on iOS 18.2 (22C151) and XCode 16.1 (16B40) and we still see the same error (Execution of the command buffer was aborted due to an error during execution. Insufficient Permission (to submit GPU work from background)). Did anyone manage to get SNClassifySoundRequest(classifierIdentifier: .version1) working in the background on iOS 18.2?

0
Urgent Issue with SoundAnalysis in iOS 18 - Critical Background Permissions Error
First post date Last post date
 
 
Q