RealityKit Texture Coordinate Transformation In Metal

Question

I am new to metal, and am trying to move a material normal texture by an offset while also taking advantage of metal's geometry modifier. When I was using a PhysicallyBasedMaterial I was using this function in the session function in the ViewController:

waterMaterial.textureCoordinateTransform.offset.x += 0.0001

The normal is a png. This would move the texture every frame. Now that I'm using a CustomMaterial to take advantage of a geometryModifier this is no longer working. I can see the texture and am using the shader successfully but the texture itself is not moving. I assume I need to do this in my metal shader file. Possibly starting in this direction:

[[visible]]
void moveTexture(realitykit::geometry_parameters params)

{

    auto normal = params.textures().normal();

}

Any help replicating the above functionality in metal would be much appreciated.

341

Posted by

gpetrochenkov

Reply

Add a Comment

Answer 1

You can create a CustomMaterial with a geometry modifier (i.e. a shader that you can use to manipulate vertex data). In this case, it sounds like the geometry parameter that you wish to modify is the texture coordinates of the material.

You could create your CustomMaterial like this:

        do {
            let customMaterial = try CustomMaterial(from: originalMaterial, geometryModifier: .init(named: "yourShader", in: library))
        } catch {
            // handle the error.
        }

Then, you will need a .metal file with yourShader, which would look something like this:

[[visible]]
void yourShader(realitykit::geometry_parameters params)
{
    // Get the current texture coordinates.
    float2 uv0 = params.geometry().uv0();

    // Offset the x texture coordinate by some value.
    uv0.x += sin(params.uniforms().time());

    // Set the offset texture coordinates for the vertex.
    params.geometry().set_uv0(uv0);
}

You can either calculate your offset value in the shader (like this example), or you can utilize the custom property of the CustomMaterial to set your offset on the CPU, and then simply access that value in your shader through the geometry_parameters.

Posted by

gchiste

Thanks so much for the explanation! This moved the textures as desired.

—
gpetrochenkov
No problem, as the other answer mentions, you should use the uv0_offset() uniform to get the texture coordinate offset, you don't need to use the custom property of the CustomMaterial for that.

—
gchiste

Add a Comment

Answer 2

You can create a CustomMaterial with a geometry modifier (i.e. a shader that you can use to manipulate vertex data). In this case, it sounds like the geometry parameter that you wish to modify is the texture coordinates of the material.

You could create your CustomMaterial like this:

        do {
            let customMaterial = try CustomMaterial(from: originalMaterial, geometryModifier: .init(named: "yourShader", in: library))
        } catch {
            // handle the error.
        }

Then, you will need a .metal file with yourShader, which would look something like this:

[[visible]]
void yourShader(realitykit::geometry_parameters params)
{
    // Get the current texture coordinates.
    float2 uv0 = params.geometry().uv0();

    // Offset the x texture coordinate by some value.
    uv0.x += sin(params.uniforms().time());

    // Set the offset texture coordinates for the vertex.
    params.geometry().set_uv0(uv0);
}

You can either calculate your offset value in the shader (like this example), or you can utilize the custom property of the CustomMaterial to set your offset on the CPU, and then simply access that value in your shader through the geometry_parameters.

Posted by

gchiste

Thanks so much for the explanation! This moved the textures as desired.

—
gpetrochenkov
No problem, as the other answer mentions, you should use the uv0_offset() uniform to get the texture coordinate offset, you don't need to use the custom property of the CustomMaterial for that.

—
gchiste

Add a Comment

Answer 3

Hi, since you are writing a custom shader, you will have to sample from the texture yourself. you can access textureCoordinateTransform within your geometry modifier via params.uniforms().uv0_offset(). And you can access the uv coordinates in params.geometry().uv0(). You'll probably need to do something like

[[visible]]
void moveTexture(realitykit::geometry_parameters params)

{
    auto uv = params.geometry().uv0();
    uv.y = 1.0 - uv.y;
    uv.x += params.uniforms().uv0_offset().x;
    params.geometry().set_uv0(uv);
    ...
}

[[visible]]
void surfaceShader(realitykit::surface_parameters params)

{
    constexpr sampler samplerBilinear(metal::coord::normalized,
                                      metal::address::clamp_to_edge,
                                      metal::filter::linear,
                                      metal::mip_filter::linear);
    auto uv = params.geometry().uv0();
    auto normal = params.textures().normal().sample(sampleBilinear, uv).xyz;
    ...
}

Also see https://developer.apple.com/metal/Metal-RealityKit-APIs.pdf

Posted by

Graphics and Games Engineer

Add a Comment

RealityKit Texture Coordinate Transformation In Metal

Accepted Reply

Replies