iOS 27: ML Program pipeline with data-dependent shapes (NMS / gather / non_zero) fails on every compute unit and allocates ~2.8 GB at load

Question

Created 9h

Replies 1

Boosts 0

Participants 2

A Core ML model that we used in production on iOS 26 no longer works on iOS 27.0.1. Prediction now fails on every compute unit. Loading it with .all also increases the process footprint by about 2.8 GB, so on 6 GB devices jetsam kills the app shortly after launch (per-process-limit).

Environment

iPhone 13 Pro (iPhone14,2), iOS 27.0.1 (24A446): the memory spike and the prediction failure both reproduce. The app is killed about 1.5 s after launch.
iPhone 15 Pro, iOS 27.0.1 (24A446): the memory spike reproduces (about 3 GB). The app is not killed because this device's limit is higher.
macOS 27.0.1 host, small Swift CLI using Core ML: the prediction failure reproduces.
The same model and app build worked on iOS 26.x. On an iPhone 15 Pro running iOS 26.6.1, the app peaked at about 570 MiB.

Model

A compiled .mlmodelc (from an .mlpackage). It is a pipeline of two ML Programs. The input is a fixed-size 640×480 image.

model 0: a YOLO-style instance segmentation network converted from TensorFlow.
- It includes non_maximum_suppression over 1500 candidate boxes.
- The NMS is followed by range_1d → gather with data-dependent [?] shapes.
- There is a zero-sized intermediate tensor (tensor<fp16, [1500, 0]>).
- Results are written with scatter_nd into a fixed-size output [1, 300, 38], plus prototype masks [1, 120, 160, 32].
model 1: post-processing converted from PyTorch.
- It runs non_zero over the 300 candidates.
- After that it works on dynamic [?, 1, 480, 640] tensors (upsample_bilinear, greater, cast, mul), then uses tile and reduce_argmax.

Symptom 1: prediction fails on all compute units

The failure is the same with .all, .cpuAndNeuralEngine, .cpuAndGPU and .cpuOnly, with a real camera image as input:

Error Domain=com.apple.CoreML Code=0 "Failed to evaluate model 0 in pipeline"
  -> "Unable to compute the asynchronous prediction using ML Program. It can be an invalid input data or broken/unsupported model."
  -> "E5RT: Submit Async failed for [6:1]: Unable to successfully execute plan. @ ExecuteSync (11)"

On macOS 27, the following is logged when the model loads:

MIL program has non-constant (dynamic) shapes for external input but FlexibleShapeInformation attribute is missing. E5 function will be produced with unknown input shapes and cannot be run without reshaping first.
Got input with flexible dimension but no default shapes provided. Will use 1

Symptom 2: about 2.8 GB allocated at load

This is phys_footprint right after MLModel(contentsOf:configuration:). The model was loaded alone in a fresh process on an iPhone 13 Pro running iOS 27.0.1. The baseline before loading was 13 MiB.

.all: 2858 MiB
.cpuAndNeuralEngine: 2857 MiB
.cpuAndGPU: 2515 MiB
.cpuOnly: 62 MiB

The other models in the same app stay under about 520 MiB on iOS 27 (load plus one prediction each). They include object detection, pose estimation and a ViT that runs on the ANE. Another model in the app with data-dependent shapes, a keypoint detector, still loads and predicts correctly on iOS 27, so not every dynamic-shape model is affected.

In the field, the JetsamEvent report shows per-process-limit at 3072 MiB. A diskwrites_resource report shows the heaviest stack under Core ML model load → Espresso → MetalPerformanceShadersGraph.

Current workaround

Loading this model with .cpuOnly on iOS 27 avoids the memory spike, so the app no longer crashes. Prediction still fails, though, so the model can't be used on iOS 27.

Questions

Is this a known iOS 27 regression in how E5RT handles data-dependent shapes inside ML Program pipelines (NMS, gather, non_zero, zero-sized tensors)?
What is the recommended way to export a model like this for iOS 27? Would adding flexible-shape information help, or should NMS and mask post-processing be moved out of the model?
Is it expected that unbounded dynamic shapes cause a worst-case allocation of several GB at load time with GPU or ANE compute units?

I can't share the model itself, but I'm happy to provide more details (full MIL op listing, logs, sysdiagnose) through Feedback Assistant if that helps.

Answer 1

D44IEL OP

3h

If look into the iOS 27 release documentation, and a couple of changes stood out that might explain what you're experiencing.

The most interesting is issue 174796039, documented in Apple's iOS 27 release notes under Core AI. Apple states that Neural Engine memory usage is now attributed to the application process rather than the system.

This could potentially explain the substantial increase in reported memory consumption. It may not necessarily mean the model is allocating more memory than before, but rather that memory previously attributed to the system is now being counted against your application's limit.

However, that wouldn't explain why inference also fails with .cpuOnly.

I also noticed two documented issues involving dynamic tensor shapes:

176807213 – Inference failures involving dynamically shaped outputs.
177354777 – Inference failures or crashes involving control flow over dynamic-shaped tensors.

Although these relate to Core AI and are marked as resolved, I wonder whether there could be a related regression in Core ML's handling of dynamic shapes, particularly given your FlexibleShapeInformation warning and the use of non_zero, gather, and other operations producing variable-sized outputs.

One other consideration is that Core ML specialises models for the operating system and hardware, meaning the same model can undergo different optimisation and compilation steps following an OS update.

Have you tried loading and running the two ML Programs independently on iOS 27? It would be interesting to determine whether the memory spike originates from the segmentation model's NMS operations or the post-processing model's dynamically sized masks.

It might also be worth comparing identical builds produced with Xcode 26 and Xcode 27 to establish whether the behaviour is entirely OS-dependent.

The Neural Engine memory attribution change seems particularly relevant to the memory issue, but I suspect the CPU inference failure may point to a separate dynamic-shape regression.

Apple's release notes: https://developer.apple.com/documentation/ios-ipados-release-notes/ios-ipados-27-release-notes

Hope this helps narrow down the investigation.