How to get grounding shadow to work in VisionOS?

Hi,

I'm trying to replicate ground shadow in this video. However, I couldn't get it to work in the simulator.

My scene looks like the following which is rendered as an immersive space:

The rocket object has the grounding shadow component with "cast shadow" set to true:

but I couldn't see any shadow on the plane beneath it.

Things I tried:

  • using code to add the grounding shadow component, didn't work
  • re-used the IBL from the helloworld project to get some lighting for the objects. Although the IBL worked, I still couldn't see the shadow
  • tried adding a DirectionalLight but got an error saying that directional lights are not supported in VisionOS (despite the docs saying the opposite)

A related question on lighting: I can see that the simulator definitely applies some scene lighting to objects. But it doesn't seem to do it perfectly. For example in the above screenshot I placed the objects under a transparent ceiling which is supposed to get a lot of lights. But everything is still quite dark.

Up vote post of tracyhenry Down vote post of tracyhenry
1.8k views
Posted by
tracyhenry
Reply
Add a Comment

Accepted Reply

@tracyhenry I think I figured it out.

I think I figured out the problem through experimentation, and GroundingShadowComponent actually does work.

I think when you read a model from a USDZ file, you can't simply assign the GroundingShadowComponent to the root entity of the entity hierarchy that's in the USDZ file. GroundingShadowComponent must be assigned to each ModelEntity that's in the USDZ file hierarchy.

This would explain why GroundingShadowComponent worked when directly assigned to sphereEntity in your code, because it's a ModelEntity.

Assuming the existence of Entity/enumerateHierarchy, defined below, you could write something like this after you load the entity from the USDZ file:

usdzFileEntity.enumerateHierarchy { entity, stop in
    if entity is ModelEntity {
        entity.components.set(GroundingShadowComponent(castsShadow: true))
    }
}

This worked for me when I tried it.

import RealityKit

extension Entity {
    
    /// Executes a closure for each of the entity's child and descendant
    /// entities, as well as for the entity itself.
    ///
    /// Set `stop` to true in the closure to abort further processing of the child entity subtree.
    func enumerateHierarchy(_ body: (Entity, UnsafeMutablePointer<Bool>) -> Void) {
        var stop = false

        func enumerate(_ body: (Entity, UnsafeMutablePointer<Bool>) -> Void) {
            guard !stop else {
                return
            }

            body(self, &stop)
            
            for child in children {
                guard !stop else {
                    break
                }
                child.enumerateHierarchy(body)
            }
        }
        
        enumerate(body)
    }
    
}
Posted by
hhjgjhghjghjghjg
Up vote reply of hhjgjhghjghjghjg Down vote reply of hhjgjhghjghjghjg
Post marked as solved

  • based on this, here's a twitter thread summarizing my exploration on shadows (together with hhjgjhghjghjghjg): https://twitter.com/tracy__henry/status/1690865514132946945?s=20

    —
    tracyhenry

  • This worked and it looks fantastic. I do wonder what the performance penalty is.

    —
    rcpdesigns

  • Thank you! This is simpler than I thought, it’s much appreciated!

    —
    mrnewell

Replies

Hi @tracyhenry did you ever make any progress on this issue?

I'm in the same situation. It appears to me that the visionOS simulator doesn't support grounding shadows. It's possible that they'll work on a physical headset, but there's no way to tell yet.

Also with respect to lighting, where do you see that the docs say that DirectionalLight is actually supported on visionOS? I first looked into this on Monday July 31 and the DirectionalLight doc page didn't include visionOS in the list of supported platforms for DirectionalLight (or any other light types).

I've also seen a WWDC Slack Q&A post that strongly implies that any sort of traditional lights or depth map shadows aren't supported on visionOS, only environment lighting and the grounding shadows.

I haven't yet seen evidence that grounding shadows will work in any context.

fwiw, even if grounding shadows work, I assume they're going to put a shadow of your ground plane square on the floor below it, and your rocket won't cast shadows onto your ground plane square.

Posted by
hhjgjhghjghjghjg
Up vote reply of hhjgjhghjghjghjg Down vote reply of hhjgjhghjghjghjg
Add a Comment

Never mind, I was trying ARKIt grounding shadows, not the RealityKit grounding shadows.

I will continue investigating...

Posted by
hhjgjhghjghjghjg
Up vote reply of hhjgjhghjghjghjg Down vote reply of hhjgjhghjghjghjg
Add a Comment

I still couldn't get shadows working for usdz models, but grounding shadow does work for procedural meshes generated using code like the following:

let sphereEntity = ModelEntity(mesh: MeshResource.generateSphere(radius: 0.05), materials: [SimpleMaterial(color: .blue, roughness: 0, isMetallic: true)])
sphereEntity.setPosition([0, 0.1, 0], relativeTo: root)
sphereEntity.components.set(GroundingShadowComponent(castsShadow: true))
sphereEntity.components.set(TraceComponent(anchor: root, width: 1))

Posted by
tracyhenry
Up vote reply of tracyhenry Down vote reply of tracyhenry
Add a Comment

I tried setting GroundingShadowComponent on a ModelEntity read from a USDZ file and it didn't work for me either in Xcode 15 beta 2.

Posted by
hhjgjhghjghjghjg
Up vote reply of hhjgjhghjghjghjg Down vote reply of hhjgjhghjghjghjg
Add a Comment

@tracyhenry I think I figured it out.

I think I figured out the problem through experimentation, and GroundingShadowComponent actually does work.

I think when you read a model from a USDZ file, you can't simply assign the GroundingShadowComponent to the root entity of the entity hierarchy that's in the USDZ file. GroundingShadowComponent must be assigned to each ModelEntity that's in the USDZ file hierarchy.

This would explain why GroundingShadowComponent worked when directly assigned to sphereEntity in your code, because it's a ModelEntity.

Assuming the existence of Entity/enumerateHierarchy, defined below, you could write something like this after you load the entity from the USDZ file:

usdzFileEntity.enumerateHierarchy { entity, stop in
    if entity is ModelEntity {
        entity.components.set(GroundingShadowComponent(castsShadow: true))
    }
}

This worked for me when I tried it.

import RealityKit

extension Entity {
    
    /// Executes a closure for each of the entity's child and descendant
    /// entities, as well as for the entity itself.
    ///
    /// Set `stop` to true in the closure to abort further processing of the child entity subtree.
    func enumerateHierarchy(_ body: (Entity, UnsafeMutablePointer<Bool>) -> Void) {
        var stop = false

        func enumerate(_ body: (Entity, UnsafeMutablePointer<Bool>) -> Void) {
            guard !stop else {
                return
            }

            body(self, &stop)
            
            for child in children {
                guard !stop else {
                    break
                }
                child.enumerateHierarchy(body)
            }
        }
        
        enumerate(body)
    }
    
}
Posted by
hhjgjhghjghjghjg
Up vote reply of hhjgjhghjghjghjg Down vote reply of hhjgjhghjghjghjg
Post marked as solved

  • based on this, here's a twitter thread summarizing my exploration on shadows (together with hhjgjhghjghjghjg): https://twitter.com/tracy__henry/status/1690865514132946945?s=20

    —
    tracyhenry

  • This worked and it looks fantastic. I do wonder what the performance penalty is.

    —
    rcpdesigns

  • Thank you! This is simpler than I thought, it’s much appreciated!

    —
    mrnewell