Core AI: on-device specialization takes about 10 s per Neural Engine model because of ANELargeModelCompilerService respawn throttling

We ship a chunked Stable Diffusion XL pipeline as Core AI .aimodelc assets, compiled ahead of time with coreai-build compile for the device's architecture. On first launch, after an OS update, or after the system's model cache cleanup, each asset is specialized on the device with AIModel.specialize(contentsOf:options:cachePolicy:).

Every model with Neural Engine regions takes about 10 s to specialize, no matter how large it is. With 16 such models, users wait about 2.5 minutes.

The device log shows why. Each specialization launches ANELargeModelCompilerService, which compiles for 0.1–0.3 s and exits. launchd then delays the next launch until 10 s after the previous one:

launchd: Successfully spawned ANELargeModelCompilerService[…] because ipc (mach)   10:05:39.190
launchd: service inactive: com.apple.ANELargeModelCompilerService.…                10:05:39.312
launchd: Successfully spawned ANELargeModelCompilerService[…] because ipc (mach)   10:05:49.546
launchd: service inactive: com.apple.ANELargeModelCompilerService.…                10:05:49.816
launchd: Successfully spawned ANELargeModelCompilerService[…] because ipc (mach)   10:06:00.448

In some captures launchd also logs "Pushing respawn out by 10 seconds".

Calling specialize concurrently for all models doesn't help. The launches stay 10 s apart, the total time was unchanged, app memory roughly quadrupled, and some specializations failed with failedToSpecialize.

Signs of the issue:

  • Specializing a Neural Engine model takes about 10 s, while a GPU-only model takes milliseconds.
  • Console shows launches of ANELargeModelCompilerService about 10 s apart during specialization.

Seen on iOS 27.2.1 (iPhone with A20 Pro, h19p) and iOS 27.2 (A19 Pro, h18p).

Filed as FB25548599.

Has anyone found a way to get several models compiled in one launch of the service, for example with multifunction assets?