Issue with Hand Occlusion in a Metal CompositorLayer

I have an issue with hand occlusion in immersive mode. I have an entry view for the app and a Metal CompositorLayer (which is the immersive volume) where I have set .upperLimbVisibility(Visibility.hidden). The problem is that when I dismiss the entry view, sometimes it hides the hands and sometimes it doesn't (randomly).

@main struct AVPainterApp: App { @State var hand: Int32 = 0

var body: some Scene
{
    WindowGroup()
    {
        ContentView(hand: $hand)
    }
    .windowResizability(.contentSize)

    ImmersiveSpace(id: "ImmersiveSpace")
    {
        CompositorLayer(configuration: MetalLayerConfiguration())
        {
            layerRenderer in SpatialSceneRun(layerRenderer, hand)
        }
    }
    .upperLimbVisibility(Visibility.hidden)
    .immersionStyle(selection: .constant(.full), in: .full)
}

}