How can I use a CreateML model in Swift Playgrounds

I believe this is a build issue in Xcode 15. For reference, I reported this as FB13521432.

When I add an ML model (I tried .mlmodel, .mlpackage or .mlmodelc) to my App Playground's Resources folder in Xcode 15, the build fails with 1-2 errors, even if I don't use the models within the playground's code.

The first error I get is related to multiple build files with the same name being produced. Here is what I get with an .mlmodel or .mlpackage file:

error: Unexpected duplicate tasks
    note: Target 'playground' (project 'playground') has write command with output /Users/[user]/Library/Developer/Xcode/DerivedData/playground-dzqpmayqqzdqoqebzuczjghhqqbk/Build/Intermediates.noindex/playground.build/Debug-iphoneos/playground.build/e1be5a3b551803f068478ad1867f7edd.sb
    note: Target 'playground' (project 'playground') has write command with output /Users/[user]/Library/Developer/Xcode/DerivedData/playground-dzqpmayqqzdqoqebzuczjghhqqbk/Build/Intermediates.noindex/playground.build/Debug-iphoneos/playground.build/e1be5a3b551803f068478ad1867f7edd.sb

A similar error is produced when using compiled models (.mlmodelc):

Multiple commands produce '/Users/[user]/Library/Developer/Xcode/DerivedData/playground-dzqpmayqqzdqoqebzuczjghhqqbk/Build/Products/Debug-iphoneos/playground.app/coremldata.bin'

The second error I get when using ML models that weren't compiled is related to COREML_CODEGEN_LANGUAGE:

No predominant language detected.  Set COREML_CODEGEN_LANGUAGE to preferred language.

Did you also get these build issues?

I think I found out why I couldn't build playgrounds with .mlmodelc files - it seems that the playground I used had a Resources folder but it didn't appear in the target's resources in Package.swift due to some reason - perhaps I manually played with it at some point and forgot about it. Editing this file / making a new playground solved the build issues for me.

However, I still get the same build problems with uncompiled models (.mlmodel) in Xcode 15.

I believe I finally found an overall solution that works for both compiled (.mlmodelc) and uncompiled models (.mlmodel, .mlpackage), plus an explanation of why .mlmodelc files may not be correctly copied in the bundle during the build process. I also sent a similar response in this recently opened similar thread.

Steps to fix the bundling issues

You will need to edit the Package.swift file manually. Go to the playground in Finder, control-click it and select Show Package Contents. Then, open the Package.swift file. At the bottom of this file, you'll find the following target section, which might look different for you:

    targets: [
        .executableTarget(
            name: "AppModule",
            path: ".",
            resources: [
                .process("Resources") // copy resources to the app bundle without keeping folder structure
            ]
        )
    ]

The .process("Resources") line indicates that the resources (in your case, your model) must be copied without keeping the original folder structure. Say you have 2 folders inside the Resources folder, and in each of these you have a few files. Only the files inside will be copied in the bundle, and the 2 original folders won't appear.

You will need to replace (or add, if you don't have the process() function) process() with copy(), which maintains the internal folder structure (I explained below why). You can either use .copy("Resources"), which require you to change the model's bundle URL inside your code, not this Package.swift file to this:

Bundle.main.url(forResource: "Resources/MobileNetV2", withExtension:"mlmodelc")

Notice this includes "Resources/" inside the path.

Or you might directly copy the model folder to the Bundle using .copy("Resources/MobileNetV2") and keep the URL without the "Resources/" part - note that this way, any other files in the Resources might not be available unless you add other process / copy functions to your Package.swift file.

For more information about process() and copy(), check the docs for bundling resources with Swift Packages (since a Swift Playground is a special type of a Swift Package).

Why compiled models are not correctly copied in the bundle using process()

Compiled ML models are essentially a folder that contains one or more coremldata.bin files (and maybe other files as well) - in fact, they appear as a folder on the filesystem (you can cd into them in Terminal). The reason why these .mlmodelc aren't correctly copied to your bundle when using .process("Resources"), or may give a build error with certain models that contain more coremldata.bin files, is that the internal folder structure is not copied, but only the files inside it. That's why you found that your bundle contained a coremldata.bin file only.

Using the previously mentioned .copy() method, you can keep the internal folder structure.

How do I use my ML models, now that I can bundle them with my App?

You can create your own code that reads the model from the bundle and compiles it if necessary, or you can copy the Model Class file that is usually generated in Xcode projects. You will need to make the necessary changes related to the file's Bundle URL.

Why do uncompiled models result in build issues?

Regarding why uncompiled models cannot be built in Xcode, something similar to the compiled models issue might be happening, but these do not appear as folders on the filesystem - so this should probably not happen and perhaps it is a build issue.

It also seems that .mlmodels are not compiled at runtime, which I believe used to happen in the previous Swift Playgrounds / Xcode versions from last year. So, if the bundling issue is fixed and .mlmodels are bundled with your app, you might need to insert additional code that compiles the model with something like this:

let compiledModelURL = try await MLModel.compileModel(at: url)
let model = try MLModel(contentsOf: compiledModelURL)

Let me know if this works!