RealityKit gesture doesn't work on child entity

Hello all, RealityKit. visionOS.

I have a parent entity (surface in the code below) that I'm adding him a child entity with all the required components for gestures (collision & input target). I'm setting the TapGesture as expected (same for SpatialTapGesture).

Result: gesture is not working. Neither the hover effect. The entity is not recognized as a tappable element.

However, if I'm adding the child entity to the content and not to the parent entity - everything is working. Enclosed code below for both scenarios.

Any idea?

Many thanks, Dudi

Doesn't work -

RealityView { content, attachments  in
    let scene = clonedEntity.clone(recursive: true)

    ...
    
    surface.addChild(scene) // <- Doesn't work
} attachments: {
    ...
}
.gesture(SpatialTapGesture().targetedToAnyEntity().onEnded { value in
    openWindow(id: "detailed-window")
})
.hoverEffect()

Work -

RealityView { content, attachments  in
    let scene = clonedEntity.clone(recursive: true)

    ...
    
    content.add(scene) // <- Work
} attachments: {
    ...
}
.gesture(SpatialTapGesture().targetedToAnyEntity().onEnded { value in
    openWindow(id: "detailed-window")
})
.hoverEffect()

Replies

Hello @DudiSG,

Are you able to provide a focused sample project that reproduces the issue?

@gchiste You can easily recreate it by creating ModelEntity parentEntity (with InputTargetComponent), content.add(parentEntity), create another ModelEntity innerEntity (with InputTargetComponent), parentEntity.addChild(innerEntity). The tap value.entity will not give you innerEntity nor parent entity, if you add child entities dynamically and can't specify SpatialTapGesture().targetToEntity(innerEntity) it's impossible to recognize the tap on these children.

Hello @gchiste Indeed - as @Vlad_Baev explained - it is very easy to reproduce. Thanks a lot.

Hello @DudiSG,

I'm not able to reproduce the described behavior using a straightforward example:

import SwiftUI
import RealityKit
import RealityKitContent

struct ContentView: View {
    var body: some View {
        
        RealityView { content in
            
            let box = ModelEntity(mesh: .generateBox(size: 0.05))
            box.generateCollisionShapes(recursive: false)
            box.components.set(HoverEffectComponent())
            box.components.set(InputTargetComponent())
            
            // This works.
            let parent = Entity()
            parent.addChild(box)
            content.add(parent)
            
            // This also works.
//            content.add(box)
        }
        .gesture(tapGesture)
        
    }
    
    private var tapGesture: some Gesture {
        SpatialEventGesture()
            .targetedToAnyEntity()
            .onEnded { value in
                print("tap received at time: \(CACurrentMediaTime())")
            }
    }
}

Hey @gchiste, The problem is that the gesture is not recognized by the child entity. I have a surface and some child entities and when I try to capture the gesture on one of the Childs, it's detected only by the part child (the surface). If you check the tapped entity you will see that it's the parent and not the child, so you can't actually get any details from the child entity.

If you have one parent and 5 childs, you'll always get the value of the parent entity, regardless the child you actually tapped.

Hello @DudiSG,

My guess is that your "surface" has a collision shape that fully encompasses the children's collision shapes, and that it also has an InputTargetComponent. If this is the case, it is expected that you would receive hits on the "surface" and not its children, because the children cannot be hit, the parent would always be hit first.

If you provide a focused sample project I can verify my theory.

Hey @gchiste Thank you so much! I really tried all the options, but I'm always happy to learn :-)

I have a big surface (rectangle) with small bricks attached to it. I tried to add the collision and the input target components only to the bricks and it didn't help. I only need the surface to function as a board with the bricks attached to it - I don't need any gesture to be detected on the surface. So as for now, even if the surface is configured with or without the components, the gesture is not detected by the child. Is it possible to achieve this functionality? What should be the structure of the entities and the components?

I expect that this structure will work - Surface - no components Bricks - collision and input target components

But it doesn't.

Hey @gchiste

These are the objects. I load both during launch. Then I'm adding the surface to the content and in a loop, I clone the bricks through this original brick object and add each one of them as a child to the surface.

Hello @DudiSG,

In general, it is possible to tap specifically on a child Entity. Without an example project that reproduces the behavior you are describing, I won't be able to figure out where the issue is.