Trouble Loading Precompiled Metal Shader (.metallib) into ShaderLibrary

I am currently finalizing my Swift Student Challenge submission, and Metal shaders are an essential part of my app. However, during submission, I noticed a note explaining: "Note: Xcode app playgrounds are run in Simulator", which is not possible for my app, as it also requires the camera of a physical device to function. So, I am currently transferring my app from Xcode into Swift Playgrounds, which I presume will run on physical devices.

However, I noticed that Swift Playgrounds do not yet support Metal shaders directly, so I am now pre-compiling my shaders to load them at runtime instead. Note that all the code below was run either in the terminal or in Xcode.

I have already compiled my Metal shaders with:

xcrun -sdk iphoneos metal -o Shaders.ir  -c Shaders.metal
xcrun -sdk iphoneos metallib Shaders.ir -o Shaders.metallib

Which seems to have run without any problems.

When I run:

let shaderPath = Bundle.main.path(forResource: "Shaders", ofType: "metallib")
let shaderURL = URL(fileURLWithPath: shaderPath!)
let shaderData = try! Data(contentsOf: shaderURL)

do {
    let device = MTLCreateSystemDefaultDevice()!

    let library = try shaderData.withUnsafeBytes { bytes -> MTLLibrary? in
        let dispatchData = DispatchData(bytes: bytes)
        return try device.makeLibrary(data: dispatchData as __DispatchData)
    }
    print(library!.functionNames)
} catch {
    print(error.localizedDescription)
}

My Metal shader functions are printed correctly in the console. However, based on my research, it seems like a MTLLibrary cannot be converted into a SwiftUI ShaderLibrary.


That is why I am now looking at these two initializers:

ShaderLibrary(url: URL)
ShaderLibrary(data: Data)

Which state: Creates a new Metal shader library from the contents of url/data, which must be the contents of precompiled Metal library. Functions compiled from the returned library will only be cached as long as the returned library exists., which I believe should work for my use case.

However, the problem arises when I run this code:

let shaderPath = Bundle.main.path(forResource: "Shaders", ofType: "metallib")
let shaderURL = URL(fileURLWithPath: shaderPath!)

let library = ShaderLibrary(url: shaderURL)

My app consistently seems to crash on the ShaderLibrary initialization, rendering the app unusable. Why does ShaderLibrary(url: shaderURL) cause a crash, even though my .metallib file is valid? Are there additional requirements for loading a ShaderLibrary that I may have missed?

Answered by MrKai77 in 825580022

Thank you!

Here's a test SwiftUI View similar to yours, but without the Dynamic Member enum

I encountered the same crash, even though I verified that the passthrough shader existed. However, I decided to test it in Swift Playgrounds instead of Xcode, and surprisingly, it worked! I’m not sure why the shader fails to load in Xcode but runs fine in Swift Playgrounds.

After confirming that it worked in Playgrounds, I reopened the project in Xcode, and the bug reappeared. This makes me fairly certain that the issue stems from differences between Xcode and Swift Playgrounds. Hopefully, this helps others who run into the same problem!

Small update:

This is my current code:

import SwiftUI

@dynamicMemberLookup
enum MyShaderLibrary {
    private static let library: ShaderLibrary = {
        let shaderPath = Bundle.main.path(forResource: "Shaders", ofType: "metallib")
        let shaderURL = URL(fileURLWithPath: shaderPath!)
        let shaderData = try! Data(contentsOf: shaderURL)
        let library = ShaderLibrary(data: shaderData)
        return library
    }()

    static subscript(dynamicMember name: String) -> ShaderFunction {
        let function = library[dynamicMember: name]

        print(function) // Important!

        return function
    }
}

And I have a passthrough Metal shader:

[[ stitchable ]]
half4 passthrough(float2 position, half4 color) {
    return color;
}

When my SwiftUI view calls the shader with:

View()
    .colorEffect(MyShaderLibrary.passthrough())

the print function shown above correctly prints this:

ShaderFunction(library: SwiftUI.ShaderLibrary(rbLibrary: <RBShaderLibrary: 0x60000000cf80>), name: "passthrough")

Which tells me that the shader library is being loaded correctly? What else could be going wrong? The code doesn't seem to crash when initializing the ShaderLibrary, but rather, when trying to render the shader?

I've been in the exact scenario you described, but thankfully I got mine working. In fact, I think I've done pretty much what you described already: compile the model separately(ince Playgrounds cannot compile it, similar to how CoreML models need to be compiled first) and include the metallib as a Resource.

My app consistently seems to crash on the ShaderLibrary initialization, rendering the app unusable. Why does ShaderLibrary(url: shaderURL) cause a crash, even though my .metallib file is valid? Are there additional requirements for loading a ShaderLibrary that I may have missed?

I don't think there's anything else you need. Here's a test SwiftUI View similar to yours, but without the Dynamic Member enum:

struct ContentView: View {
    var library: ShaderLibrary {
        // note that i'm force-unwrapping some optionals here,
        // but maybe you can make this an Optional type instead
        // (i.e. SharedLibrary? and return nil on error)
        // if you want to be safe in case something goes wrong.
        let shaderPath = Bundle.main.path(forResource: "Shader", ofType: "metallib")!
        let shaderURL = URL(fileURLWithPath: shaderPath)
        let library = ShaderLibrary(url: shaderURL)
        return library
    }

    var body: some View {
        View()
            .colorEffect(library.myEffect())
    }

I've ran this successfully without any issues. My Shader.metallib was placed inside the Resources folder, and displays as a resource inside Playgrounds, and you likely did this too for the reasons described below.

So we confirmed that the issue is not regarding the Metal shader itself. Now, looking back at your example, everything looks pretty normal, I'm not exactly sure what goes wrong, but here's what I would check:

ShaderFunction(library: SwiftUI.ShaderLibrary(rbLibrary: <RBShaderLibrary: 0x60000000cf80>), name: "passthrough")

This indeed confirms that your metallib url path is correct, and your library was loaded correctly. But there's a catch: you're using @dynamicMemberLookup here, which means you can subscript absolutely anything, including inexistent shaders. Meaning that, if there's no actual passtrough shader available, your print will be the same.

You can check that yourself: if you change passthrough to an inexistent, shadeer, something random, you'll see that your print is the same.

Note: this is also the case in my example above; ShaderLibrary is a @dynamicMemberLookup type itself, so it is prone to issues like this as well.

TL;DR Can you check that the passthrough shader actually exists (and was compiled correctly in your metallib)?

I have same problem, did you find any solution?

Accepted Answer

Thank you!

Here's a test SwiftUI View similar to yours, but without the Dynamic Member enum

I encountered the same crash, even though I verified that the passthrough shader existed. However, I decided to test it in Swift Playgrounds instead of Xcode, and surprisingly, it worked! I’m not sure why the shader fails to load in Xcode but runs fine in Swift Playgrounds.

After confirming that it worked in Playgrounds, I reopened the project in Xcode, and the bug reappeared. This makes me fairly certain that the issue stems from differences between Xcode and Swift Playgrounds. Hopefully, this helps others who run into the same problem!

Trouble Loading Precompiled Metal Shader (.metallib) into ShaderLibrary
 
 
Q