Animating a RealityComposerPro shader's uniform input value

I'm trying to build a Shader in "Reality Composer Pro" that updates from a start time. Initially I tried the following:

The idea was that when the startTime was 0, the output would be 0, but then I would set startTime from within code and this would be compared with the current GPU time, and difference used to drive another part of the shader graph:

		if
			let testEntity = root.findEntity(named: "Test"),
			var shaderGraphMaterial = testEntity.components[ModelComponent.self]?.materials.first as? ShaderGraphMaterial
		{
			let time = CFAbsoluteTimeGetCurrent()
			try! shaderGraphMaterial.setParameter(name: "StartTime", value: .float(Float(time)))
			testEntity.components[ModelComponent.self]?.materials[0] = shaderGraphMaterial
		}

However, I haven't found a reference to the time the shader would be using.

So now I am trying to write an EntityAction to achieve the same effect. Instead of comparing a start time to the GPU's time I'm trying to animate one of the shader's uniform input. However, I'm not sure how to specify the bind target. Here's my attempt so far:

import RealityKit

struct ShaderAction: EntityAction {
	
	let startValue: Float
	let targetValue: Float
	var animatedValueType: (any AnimatableData.Type)? { Float.self }
	
	static func registerEntityAction() {
		
		ShaderAction.subscribe(to: .updated) { event in
			guard let animationState = event.animationState else { return }
			let value = simd_mix(event.action.startValue, event.action.targetValue, Float(animationState.normalizedTime))
			animationState.storeAnimatedValue(value)
		}
	}
}

extension Entity {
		func updateShader(from startValue: Float, to targetValue: Float, duration: Double) {
		let fadeAction = ShaderAction(startValue: startValue, targetValue: targetValue)
		if let shaderAnimation = try? AnimationResource.makeActionAnimation(for: fadeAction, duration: duration, bindTarget: .material(0).customValue) {
			playAnimation(shaderAnimation)
		}
	}
}
'''

Currently when I run this I get an assertion failure: 'Index out of range (operator[]:line 797) index = 260, max = 8'

Furthermore, even if it didn't crash I don't understand how to pass a binding to the custom shader value "startValue".

Any clues of how to achieve this effect - even if it's a completely different way.
Answered by Vision Pro Engineer in 823933022

Hi @peggers123

ShaderGraphMaterial doesn't support parameter animation with EntityAction at the moment. If you'd like us to consider adding the necessary functionality, please file an enhancement request using Feedback Assistant.

That being said, you can still achieve your desired effect by utilizing RealityKit's Entity Component System (ECS) to animate your material's parameters.

Start by defining a custom shader parameter animator component:

struct ShaderParameterAnimatorComponent: Component {
    let parameterName: String
    let startValue: Float
    let endValue: Float
    let duration: Float
    var isAnimating: Bool = false
    var normalizedTime: Float = 0
    
    init(parameterName: String, startValue: Float, endValue: Float, duration: Float) {
        self.parameterName = parameterName
        self.startValue = startValue
        self.endValue = endValue
        self.duration = duration
    }
    
    public mutating func playAnimation() {
        normalizedTime = 0
        isAnimating = true
    }
}

This component stores all of the information your animation needs, such as its startValue, endValue, duration, and parameterName — the name of the uniform input shader parameter the animation acts upon.

Next, create a system that animates the shader parameter over time:

struct ShaderParameterAnimatorSystem: System {
    let query = EntityQuery(where: .has(ShaderParameterAnimatorComponent.self))

    public init(scene: RealityKit.Scene) { }
    
    public func update(context: SceneUpdateContext) {
        let deltaTime = Float(context.deltaTime)
        for entity in context.entities(matching: query, updatingSystemWhen: .rendering) {
            if var component = entity.components[ShaderParameterAnimatorComponent.self],
               var shaderGraphMaterial = entity.components[ModelComponent.self]?.materials.first as? ShaderGraphMaterial {
                
                // Update the animation when it's animating.
                if component.isAnimating {
                    // Update the normalized animation time, clamping it between 0 and 1.
                    component.normalizedTime = simd_clamp(component.normalizedTime + deltaTime / component.duration, 0, 1)
                    // Calculate the interpolated shader parameter value.
                    let value = simd_mix(component.startValue, component.endValue, component.normalizedTime)
                    // Pass the value to the shader graph material.
                    try? shaderGraphMaterial.setParameter(name: component.parameterName, value: .float(value))
                    // Apply the shader graph material to the model.
                    entity.components[ModelComponent.self]?.materials = [shaderGraphMaterial]
                    
                    // Stop the animation when it completes.
                    if component.normalizedTime >= 1 {
                        component.isAnimating = false
                    }
                }
                
                // Update the shader parameter animator component.
                entity.components.set(component)
            }
        }
    }
}

This system animates the shader parameter by interpolating its value from the startValue to the endValue over the duration of the animation.

Now, assuming you have an entity with a ModelComponent that references your custom ShaderGraphMaterial, you can animate a parameter of that material as follows.

myEntity.components.set(ShaderParameterAnimatorComponent(parameterName: "AnimationTime", startValue: 0, endValue: 1, duration: 5))
myEntity.components[ShaderParameterAnimatorComponent.self]?.playAnimation()

Here, feel free to replace "AnimationTime" with the name of the uniform input parameter you expose in your custom Shader Graph Material in RCP, such as "StartTime".

Finally, don't forget to register the ShaderParameterAnimatorSystem:

ShaderParameterAnimatorSystem.registerSystem()
Accepted Answer

Hi @peggers123

ShaderGraphMaterial doesn't support parameter animation with EntityAction at the moment. If you'd like us to consider adding the necessary functionality, please file an enhancement request using Feedback Assistant.

That being said, you can still achieve your desired effect by utilizing RealityKit's Entity Component System (ECS) to animate your material's parameters.

Start by defining a custom shader parameter animator component:

struct ShaderParameterAnimatorComponent: Component {
    let parameterName: String
    let startValue: Float
    let endValue: Float
    let duration: Float
    var isAnimating: Bool = false
    var normalizedTime: Float = 0
    
    init(parameterName: String, startValue: Float, endValue: Float, duration: Float) {
        self.parameterName = parameterName
        self.startValue = startValue
        self.endValue = endValue
        self.duration = duration
    }
    
    public mutating func playAnimation() {
        normalizedTime = 0
        isAnimating = true
    }
}

This component stores all of the information your animation needs, such as its startValue, endValue, duration, and parameterName — the name of the uniform input shader parameter the animation acts upon.

Next, create a system that animates the shader parameter over time:

struct ShaderParameterAnimatorSystem: System {
    let query = EntityQuery(where: .has(ShaderParameterAnimatorComponent.self))

    public init(scene: RealityKit.Scene) { }
    
    public func update(context: SceneUpdateContext) {
        let deltaTime = Float(context.deltaTime)
        for entity in context.entities(matching: query, updatingSystemWhen: .rendering) {
            if var component = entity.components[ShaderParameterAnimatorComponent.self],
               var shaderGraphMaterial = entity.components[ModelComponent.self]?.materials.first as? ShaderGraphMaterial {
                
                // Update the animation when it's animating.
                if component.isAnimating {
                    // Update the normalized animation time, clamping it between 0 and 1.
                    component.normalizedTime = simd_clamp(component.normalizedTime + deltaTime / component.duration, 0, 1)
                    // Calculate the interpolated shader parameter value.
                    let value = simd_mix(component.startValue, component.endValue, component.normalizedTime)
                    // Pass the value to the shader graph material.
                    try? shaderGraphMaterial.setParameter(name: component.parameterName, value: .float(value))
                    // Apply the shader graph material to the model.
                    entity.components[ModelComponent.self]?.materials = [shaderGraphMaterial]
                    
                    // Stop the animation when it completes.
                    if component.normalizedTime >= 1 {
                        component.isAnimating = false
                    }
                }
                
                // Update the shader parameter animator component.
                entity.components.set(component)
            }
        }
    }
}

This system animates the shader parameter by interpolating its value from the startValue to the endValue over the duration of the animation.

Now, assuming you have an entity with a ModelComponent that references your custom ShaderGraphMaterial, you can animate a parameter of that material as follows.

myEntity.components.set(ShaderParameterAnimatorComponent(parameterName: "AnimationTime", startValue: 0, endValue: 1, duration: 5))
myEntity.components[ShaderParameterAnimatorComponent.self]?.playAnimation()

Here, feel free to replace "AnimationTime" with the name of the uniform input parameter you expose in your custom Shader Graph Material in RCP, such as "StartTime".

Finally, don't forget to register the ShaderParameterAnimatorSystem:

ShaderParameterAnimatorSystem.registerSystem()
Animating a RealityComposerPro shader's uniform input value
 
 
Q