New Native Gaussian splats vanish up close

I'm building a MR teleoperation app for the Apple Vision Pro. Instead of 2D camera feeds, I render the robot's remote workspace as 3D Gaussian splats, so the operator can perceive depth and inspect objects from any angle by moving their head. That means the operator needs to lean in close to an object, and sometimes move the viewpoint right up to or through it, without the object dropping out of view.

Right now when I use the visionOS 27 splat sample out of the box, the splats disappear when the operator gets too close. As the viewpoint approaches a splatted object the entire splat will vanish. This can be solved partially with chunking the splats, but the operator still can't move inside the splat itself.

The entire splat vanishing, and chunking helping, makes me think the splat entity is culled once the camera enters its bounds, rather than individual Gaussians being clipped at the near plane. I don't know the renderer's internals, so I may be wrong about the cause. I believe the sample renders in an ImmersiveSpace, not a bounded volume.

Does GaussianSplatComponent expose any control over how splats are culled up close (near plane, clipping, or entity bounds), so the splat stays visible as the viewer moves close to or inside it? If there's no such control today, is there a recommended way to let the viewer move through a splat without it disappearing?

Hello @benyang , thank you for your question!

GuassianSplatComponent does not expose any control over how splats are culled. I highly recommend filing feedback using the Feedback Assistant app, and include as many details about your use case as you can. And please post your ticket number here in this thread, that will help us make sure we are able to follow up on your issue if there are updates in the future.

In visionOS 27, there isn't a way to allow the person wearing the device to enter the splat without the splat disappearing, as you have observed. A few recommendations:

  1. Chunk your large splats. You've already noticed this helps, but I understand it isn't a perfect solution.

  2. Keep chunk size modest. A larger splat has a larger bounding box, which is easier for the user to enter inadvertently.

  3. Trim low opacity gaussians from the edges of your splats during training/preprocessing. Transparent content around the edges of the shape will expand the bounding box further, and far away transparent splats can easily grow the bounding box unexpectedly.

I will make sure your feedback is delivered to the right people. Thank you!

Hitting the same limitation for cultural-heritage / environment captures where the viewer should be able to walk through the space. Filed FB23039893 covering this (camera-traversable splat volumes) plus runtime-mutable splat data.

New Native Gaussian splats vanish up close
 
 
Q