Reconstruction with Object Mask in PhotogrammetrySession Fails .

Question

Created 3w

Replies 3

Boosts 0

Participants 2

Reconstruction with Object Mask in PhotogrammetrySession Fails and doesn't complete because of alignment and object masks with thing segmentation even after increasing the mask region . Also the PhotogrammetrySample hangs with EXC_BREAKPOINTwhen I usePhotogrammetrySession(input: [PhotogrammetrySample])withobjectMask set and the Apple Developer Forums report crashes (“HelloPhotogrammetry” / sample+mask), and (EXC_BREAKPOINT` was being observed at ~4 GB) with freezing the execution and Xcode interface

Answered by Axuby in 889024022

Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I have created a feedback report as well like recommended with this ID: FB22823733 with more elaborate images of my implementation, also see here https://origin-devforums.apple.com/forums/thread/827311

Answer 1

Engineer OP

Apple

2w

It's hard to provide guidance without more information. For example, is this iOS or macOS? Which versions? Which device?

If you are seeing breakpoints at 4GB on an iPhone using the Sequence input to the session, you are likely facing memory issues by trying to hold all images in memory at once during the reconstruction, which will not work on iPhone since the algorithm needs significant memory itself. Since you mention that you're using [PhotogrammetrySample] as the input, I anticipate this is the case. You will need to use a lazy Sequence iterator so only one image is loaded at a time for this to work. Please see https://developer.apple.com/documentation/realitykit/photogrammetrysession/init(input:configuration:)-7glmh#discussion for more information.

You may also be having issues in alignment due to the masks not being the same orientation as the original images, or are in the wrong format. You can check the logs to see if any errors are reported.

Finally, my recommendation would be to capture a feedback report and submit it with the image and mask data so that our engineers can take a look and try to guide you. You can post the bug ID here for reference as well.

Answer 2

Axuby OP

2w

Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here https://origin-devforums.apple.com/forums/thread/827311

Answer 3

Axuby OP

2w

Accepted Answer

Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I have created a feedback report as well like recommended with this ID: FB22823733 with more elaborate images of my implementation, also see here https://origin-devforums.apple.com/forums/thread/827311