RealityKit Texture Coordinate Transformation In Metal

Question

Created Sep ’21

Replies 2

Boosts 0

Participants 3

I am new to metal, and am trying to move a material normal texture by an offset while also taking advantage of metal's geometry modifier. When I was using a PhysicallyBasedMaterial I was using this function in the session function in the ViewController:

waterMaterial.textureCoordinateTransform.offset.x += 0.0001

The normal is a png. This would move the texture every frame. Now that I'm using a CustomMaterial to take advantage of a geometryModifier this is no longer working. I can see the texture and am using the shader successfully but the texture itself is not moving. I assume I need to do this in my metal shader file. Possibly starting in this direction:

[[visible]]
void moveTexture(realitykit::geometry_parameters params)

{

    auto normal = params.textures().normal();

}

Any help replicating the above functionality in metal would be much appreciated.

Answered by DTS Engineer in 689641022

You can create a CustomMaterial with a geometry modifier (i.e. a shader that you can use to manipulate vertex data). In this case, it sounds like the geometry parameter that you wish to modify is the texture coordinates of the material.

You could create your CustomMaterial like this:

        do {
            let customMaterial = try CustomMaterial(from: originalMaterial, geometryModifier: .init(named: "yourShader", in: library))
        } catch {
            // handle the error.
        }

Then, you will need a .metal file with yourShader, which would look something like this:

[[visible]]
void yourShader(realitykit::geometry_parameters params)
{
    // Get the current texture coordinates.
    float2 uv0 = params.geometry().uv0();

    // Offset the x texture coordinate by some value.
    uv0.x += sin(params.uniforms().time());

    // Set the offset texture coordinates for the vertex.
    params.geometry().set_uv0(uv0);
}

You can either calculate your offset value in the shader (like this example), or you can utilize the custom property of the CustomMaterial to set your offset on the CPU, and then simply access that value in your shader through the geometry_parameters.

Boost

Answer 1

DTS Engineer OP

Apple

Sep ’21

Accepted Answer

You can create a CustomMaterial with a geometry modifier (i.e. a shader that you can use to manipulate vertex data). In this case, it sounds like the geometry parameter that you wish to modify is the texture coordinates of the material.

You could create your CustomMaterial like this:

        do {
            let customMaterial = try CustomMaterial(from: originalMaterial, geometryModifier: .init(named: "yourShader", in: library))
        } catch {
            // handle the error.
        }

Then, you will need a .metal file with yourShader, which would look something like this:

[[visible]]
void yourShader(realitykit::geometry_parameters params)
{
    // Get the current texture coordinates.
    float2 uv0 = params.geometry().uv0();

    // Offset the x texture coordinate by some value.
    uv0.x += sin(params.uniforms().time());

    // Set the offset texture coordinates for the vertex.
    params.geometry().set_uv0(uv0);
}

You can either calculate your offset value in the shader (like this example), or you can utilize the custom property of the CustomMaterial to set your offset on the CPU, and then simply access that value in your shader through the geometry_parameters.

0

Answer 2

Graphics and Games Engineer OP

Apple

Sep ’21

Hi, since you are writing a custom shader, you will have to sample from the texture yourself. you can access textureCoordinateTransform within your geometry modifier via params.uniforms().uv0_offset(). And you can access the uv coordinates in params.geometry().uv0(). You'll probably need to do something like

[[visible]]
void moveTexture(realitykit::geometry_parameters params)

{
    auto uv = params.geometry().uv0();
    uv.y = 1.0 - uv.y;
    uv.x += params.uniforms().uv0_offset().x;
    params.geometry().set_uv0(uv);
    ...
}

[[visible]]
void surfaceShader(realitykit::surface_parameters params)

{
    constexpr sampler samplerBilinear(metal::coord::normalized,
                                      metal::address::clamp_to_edge,
                                      metal::filter::linear,
                                      metal::mip_filter::linear);
    auto uv = params.geometry().uv0();
    auto normal = params.textures().normal().sample(sampleBilinear, uv).xyz;
    ...
}

Also see https://developer.apple.com/metal/Metal-RealityKit-APIs.pdf

0