Collision handling between the entity and the real environment objects and planes in RealityKit

Question

Created Sep ’23

Replies 0

Boosts 0

Participants 1

I'm trying to achieve a similar behaviour to the native AR preview app on iOS when we can place a model and once we move or rotate it, it automatically detects the obstacles and gives a haptic feedback, and doesn't go through the walls. I'm using the devices with LiDAR only.

Here is what I have so far:

Session setup

private func configureWorldTracking() {
    let configuration = ARWorldTrackingConfiguration()
    configuration.planeDetection = [.horizontal, .vertical]
    configuration.environmentTexturing = .automatic

if ARWorldTrackingConfiguration.supportsSceneReconstruction(.meshWithClassification) {
        configuration.sceneReconstruction = .meshWithClassification
    }

    let frameSemantics: ARConfiguration.FrameSemantics = [.smoothedSceneDepth, .sceneDepth]
    if ARWorldTrackingConfiguration.supportsFrameSemantics(frameSemantics) {
        configuration.frameSemantics.insert(frameSemantics)
    }

    session.run(configuration)
    session.delegate = self

    arView.debugOptions.insert(.showSceneUnderstanding)
    arView.renderOptions.insert(.disableMotionBlur)
    arView.environment.sceneUnderstanding.options.insert([.collision, .physics, .receivesLighting, .occlusion])
}

Custom entity:

class CustomEntity: Entity, HasModel, HasCollision, HasPhysics {
    var modelName: String = ""
    private var cancellable: AnyCancellable?

    init(modelName: String) {
        super.init()
        self.modelName = modelName
        self.name = modelName
        load()
    }

    required init() {
         fatalError("init() has not been implemented")
    }

    deinit {
         cancellable?.cancel()
    }

    func load() {
         cancellable = Entity.loadModelAsync(named: modelName + ".usdz")
             .sink(receiveCompletion: { result in
                 switch result {
                 case .finished:
                     break
                 case .failure(let failure):
                     debugPrint(failure.localizedDescription)
                  }
            }, receiveValue: { modelEntity in
                 modelEntity.generateCollisionShapes(recursive: true)
            
                 self.model = modelEntity.model
                 self.collision = modelEntity.collision
                 self.collision?.filter.mask.formUnion(.sceneUnderstanding)
                 self.physicsBody = modelEntity.physicsBody
                 self.physicsBody?.mode = .kinematic
            })
}

Entity loading and placing

let tapLocation = sender.location(in: arView)
guard let raycastResult = arView.raycast(from: tapLocation, allowing: .estimatedPlane, alignment: .horizontal).first else { return }

let entity = CustomEntity(modelName: modelName)
let anchor = AnchorEntity(world: raycastResult.worldTransform)
anchor.name = entity.name
anchor.addChild(entity)
arView.scene.addAnchor(anchor)
arView.installGestures([.rotation, .translation], for: entity)

This loads my model properly and allows me to move it and rotate as well, but I cannot figure out how to handle the collision handling with the real environment like walls and interrupt gestures once my model starts going thought it?

Boost