How to replicate ObjectCaptureSession's boundary restriction?

Question

Sebastian-Immersicom OP

Created Dec ’23

Replies 1

Boosts 0

Participants 2

Hello,

I want to use Apple's PhotogrammetrySession to scan a window. However, ObjectCaptureSession seems to be a monotasker and won't allow capture to occur with anything but a small object on a flat surface.

So, I need to manually feed data into PhotogrammetrySession. But when I do, it focuses way too much on the scene behind the window, sacrificing detail on the window itself. Screenshot 2023-12-07 at 11.09.18 AM.png

Is there a way for me to either coax ObjectCaptureSession into capturing an area on the wall, or for me to restrict PhotogrammetrySession's target bounding box manually? How does ObjectCaptureSession communicate the limited bounding box to PhotogrammetrySession?

Thanks, Sebastian

Boost

Answer 1

UltimateWalrus0 OP

Dec ’23

I can see that there is a way for PhotogrammetySession to accept object masks. I ensured isObjectMaskingEnabled was set to true, and then I put black-and-white images suffixed with _mask.TIF in the input directory, but they seem to have no effect vs. just not having them at all. I noticed the depth map is rotated 90 degrees, so I tried doing that to the mask, but it didn't make a difference either way.

Example of how I'm formatting it: https://drive.google.com/file/d/1-BZCmncAzHnLEGLTKC2W99tjhTIMvc6Z/view?usp=sharing

I also noticed there are functions within PhotogrammetrySession that allow input of a bounding box. For example, I found this signature under public struct Geometry:

            public init(orientedBounds: OrientedBoundingBox, transform: Transform = Transform.identity)
            /// Creates an instance with an optional bounding box and transform.
            ///
            /// - Parameters:
            ///
            ///   - bounds: An optional bounding box to specify the size of the generated 3D model.
            ///    If you don’t pass a value, the initializer sets the bounding box to .empty, which tells
            ///    RealityKit to calculate the object’s size.

But I'm not exactly sure what coordinate space this is using. Is it relative to the pose of the first image or something?

0