Create MTLLibrary from raw String for use within RealityKit

Hello,

I have a usecase where I need to to download and compile metal shaders on demand as strings or .metal files. These should then be used for CustomMaterials and/or postprocessing within RealityKit. Essentially this boils down to having raw source code that needs to be compiled at runtime.

My plan was to use the method makeLibrary(source:options:completionHandler:) to accomplish this.

The problem is that I get the following error during compilation:

RealityKitARExperienceAssetProvider: An error occured while trying to compile shader library »testShaderLibrary«

- Error Domain=MTLLibraryErrorDomain Code=3 "program_source:2:10: fatal error: 'RealityKit/RealityKit.h' file not found

#include <RealityKit/RealityKit.h>

My code for creating the library looks like this (simplified example):

let librarySourceString: String = """
#include <metal_stdlib>
#include <RealityKit/RealityKit.h>

using namespace metal;

[[visible]]
void mySurfaceShader(realitykit::surface_parameters params)
{
    params.surface().set_base_color(half3(1, 1, 1));
}
"""

mtlDevice.makeLibrary(source: librarySourceString, options: nil) { library, error in
    if let error = error {
        dump(error)
        return
    }

    // do something with library
}

So I'm wondering if there's a way to tell the metal compiler how to resolve this reference to the RealityKit header file? Would I need to replace that part of the source string maybe with an absolute path to the RealityKit framework (if so – how would I get this at runtime)?

Appreciate any hints - thanks!

Hi, the runtime metal compiler is unable to resolve header dependencies at runtime. The only other option would be to inline/embed the contents of RealityKit/RealityKit.h in your shader source. However, the Metal team does not recommend this since it can lead to UI hitches, you should try to compile your shaders offline if possible. You should be able to find the source in your SDK by command-clicking on a metal file containing #include <RealityKit/RealityKit.h> and choosing 'jump to definition'. Also, please file a feature request to the Metal team for this on Feedback Assistant. Include some info about your use case and post the feedback number here and I can send it to Metal directly.

Create MTLLibrary from raw String for use within RealityKit
 
 
Q