Core ML Model performance far lowe… | Apple Developer Forums

Hello,

I posted an issue on the coremltools GitHub about my Core ML models not performing as well on iOS 17 vs iOS 16 but I'm posting it here just in case.

TL;DR

The same model on the same device/chip performs far slower (doesn't use the Neural Engine) on iOS 17 compared to iOS 16.

Longer description

The following screenshots show the performance of the same model (a PyTorch computer vision model) on an iPhone SE 3rd gen and iPhone 13 Pro (both use the A15 Bionic).

iOS 16 - iPhone SE 3rd Gen (A15 Bioinc)

iOS 16 uses the ANE and results in fast prediction, load and compilation times.

iOS 17 - iPhone 13 Pro (A15 Bionic)

iOS 17 doesn't seem to use the ANE, thus the prediction, load and compilation times are all slower.

Code To Reproduce

The following is my code I'm using to export my PyTorch vision model (using coremltools).

I've used the same code for the past few months with sensational results on iOS 16.

# Convert to Core ML using the Unified Conversion API
coreml_model = ct.convert(
    model=traced_model,
    inputs=[image_input],
    outputs=[ct.TensorType(name="output")],
    classifier_config=ct.ClassifierConfig(class_names),
    convert_to="neuralnetwork",
    # compute_precision=ct.precision.FLOAT16,
    compute_units=ct.ComputeUnit.ALL
)

System environment:

Xcode version: 15.0
coremltools version: 7.0.0
OS (e.g. MacOS version or Linux type): Linux Ubuntu 20.04 (for exporting), macOS 13.6 (for testing on Xcode)
Any other relevant version information (e.g. PyTorch or TensorFlow version): PyTorch 2.0

Additional context

This happens across "neuralnetwork" and "mlprogram" type models, neither use the ANE on iOS 17 but both use the ANE on iOS 16

If anyone has a similar experience, I'd love to hear more.

Otherwise, if I'm doing something wrong for the exporting of models for iOS 17+, please let me know.

Thank you!

Core ML Model performance far lower on iOS 17 vs iOS 16 (iOS 17 not using Neural Engine)