gesture control with groups

Hello,

after a lot of trial and error, I managed to move/scale/rotate a simple object with gestures. In retrospect, it was actually quite simple. If you know how to do it. But now I have a new problem. Maybe you would be so kind again and help me.

I took the Augmented Reality base file in Xcode (the one with the Experience.rcproject file in it). I entered this code in the ViewController:

import RealityKit
​
class ViewController: UIViewController {
    @IBOutlet var arView: ARView!
    override func viewDidLoad() {
        super.viewDidLoad()
        // Load the "Box" scene from the "Experience" Reality File
        let boxAnchor = try! Experience.loadBox()
        if let box = boxAnchor.steelBox?.children.first as? ModelEntity {
            box.generateCollisionShapes(recursive: true)
            arView.installGestures(.all, for: box)
        }
        // Add the box anchor to the scene
        arView.scene.anchors.append(boxAnchor)
    }
}

And it works. The object, that is the cube, can be controlled with gestures.

Then I created a second object (I duplicated the cube) in RealityComposer (from Xcode), renamed it and grouped it with the first cube. I named the group "boxGroup." Then I adjusted the code accordingly so that the reference no longer goes to "steelBox", but to "boxGroup". But now the gesture control doesn't work. Here is the code for it:

import RealityKit
​
class ViewController: UIViewController {
    @IBOutlet var arView: ARView!
    override func viewDidLoad() {
        super.viewDidLoad()
        // Load the "Box" scene from the "Experience" Reality File
        let boxAnchor = try! Experience.loadBox()
        if let box = boxAnchor.boxGroup?.children.first as? ModelEntity {
            box.generateCollisionShapes(recursive: true)
            arView.installGestures(.all, for: box)
        }
        // Add the box anchor to the scene
        arView.scene.anchors.append(boxAnchor)
    }
}

What am I doing wrong?

Thanks for your help.

Thomas

Answered by ovidus in 703446022

Hello,

I figured it out. Ok, with the help from Japan. Many thanks for this, D.... (Don't want to write his name here, to save his privacy, but he will know if he reads this)

For the example, the basic file that opens when you select "Augmented Reality" as the project is used. The RealityComposer file is called "Experience.rcproject". The scene in "Experience.rcproject" is called "Box". First, of course, we need a group. We create that in RealityComposer (from within Xcode). We open Experience.rcproject and click on "Open in RealityComposer" in the 3d window. A cube already exists. So we create another geometry and group (menu) both objects. We rename the group to "boxGroup" (to make the code work).

Then enter the code into the ViewController.

!! Please note: The transparent box created in the code can still be seen dimly. Here I am still looking for a solution, so that it is not visible at all. But you can make the box very big. !! The transparent box must not be scaled to zero, because then the gesture control does not work anymore.

Have fun with the rebuild.

import UIKit
import RealityKit

class ViewController: UIViewController {
   
  @IBOutlet var arView: ARView!
   
  override func viewDidLoad() {
    super.viewDidLoad()

// Load Scene   
    let sceneAnchor = try! Experience.loadBox()
     
// Create transparent box
    let transparentBox = ModelEntity(mesh: .generateBox(size: [1.0,1.0,1.0]),materials: [SimpleMaterial(color: .clear, isMetallic: false)])

// Load group of geometry
    let boxEntity = sceneAnchor.findEntity(named: "boxGroup")

// Add group of geometry to transparent box
    transparentBox.addChild(boxEntity!)
     
// Add collision and gestures to transparent box
    transparentBox.generateCollisionShapes(recursive: true)
    arView.installGestures(.all, for: transparentBox)
     
// Add plane Anchor
    let anchor = AnchorEntity(plane: .horizontal)

// Add transparent box to anchor
    anchor.addChild(transparentBox)

// Add anchor to scene
    arView.scene.anchors.append(anchor)
   
  }
}

Addition: I added the command "print (boxAnchor)" and now I get a listing. There I noticed that the cubes are defined as modelEntity, while the group is an entity. Does this have something to do with my problem?

Accepted Answer

Hello,

I figured it out. Ok, with the help from Japan. Many thanks for this, D.... (Don't want to write his name here, to save his privacy, but he will know if he reads this)

For the example, the basic file that opens when you select "Augmented Reality" as the project is used. The RealityComposer file is called "Experience.rcproject". The scene in "Experience.rcproject" is called "Box". First, of course, we need a group. We create that in RealityComposer (from within Xcode). We open Experience.rcproject and click on "Open in RealityComposer" in the 3d window. A cube already exists. So we create another geometry and group (menu) both objects. We rename the group to "boxGroup" (to make the code work).

Then enter the code into the ViewController.

!! Please note: The transparent box created in the code can still be seen dimly. Here I am still looking for a solution, so that it is not visible at all. But you can make the box very big. !! The transparent box must not be scaled to zero, because then the gesture control does not work anymore.

Have fun with the rebuild.

import UIKit
import RealityKit

class ViewController: UIViewController {
   
  @IBOutlet var arView: ARView!
   
  override func viewDidLoad() {
    super.viewDidLoad()

// Load Scene   
    let sceneAnchor = try! Experience.loadBox()
     
// Create transparent box
    let transparentBox = ModelEntity(mesh: .generateBox(size: [1.0,1.0,1.0]),materials: [SimpleMaterial(color: .clear, isMetallic: false)])

// Load group of geometry
    let boxEntity = sceneAnchor.findEntity(named: "boxGroup")

// Add group of geometry to transparent box
    transparentBox.addChild(boxEntity!)
     
// Add collision and gestures to transparent box
    transparentBox.generateCollisionShapes(recursive: true)
    arView.installGestures(.all, for: transparentBox)
     
// Add plane Anchor
    let anchor = AnchorEntity(plane: .horizontal)

// Add transparent box to anchor
    anchor.addChild(transparentBox)

// Add anchor to scene
    arView.scene.anchors.append(anchor)
   
  }
}
gesture control with groups
 
 
Q