PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates

Question

Created 3w

Replies 4

Boosts 0

Views 1.5k

Participants 4

Xcode hangs when I call PhotogrammetrySession(input: [PhotogrammetrySample]) with objectMask set and traps on some devices, see the attached screenshot, it gets to the function and hangs. Even the folder reconstruction also doesn't complete as it can't find alignment and displays the CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. In this case it failed while object masking was ON, so RealityKit could not find enough consistent feature tracks inside the masked pixels across the image set.

What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally.

The ObjectMasks are actually segmentation masks from an ML algorithm .

To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing.

I will appreciate a timely response and willing to provide more clarity and informations, thank you so much for your understanding

Answered by Vision Pro Engineer in 893290022

Hello :

Thank you for filing this feedback.

You can see the status of your feedback in Feedback Assistant. There, you can track if the report is still being investigated, has a potential identifiable fix, or has been resolved in another way. The status appears beside the label "Resolution." We're unable to share any updates on specific reports on the forums.

For more details on when you'll see updates to your report, please see What to expect after submission.

Answer 1

DTS Engineer OP

Apple

3w

Thank you for your post. I recommend you provide a link to a simple and focused project that demonstrates the issue, allowing engineers to examine it in detail the issue you are reporting.

If so, please share a link to your test project. That'll help us better understand what's going on. If you're not familiar with preparing a test project, take a look at Creating a test project.

The little a know is a hard hang or trap when passing an array of PhotogrammetrySample objects usually points to a data format mismatch? Because masking removes the background features that Object Capture relies on to figure out where the camera is in 3D space.

What are the exact dimensions and pixel formats of the image and objectMask you are passing? Are you running this on macOS or iOS? What version of Xcode are you using? What is the specific trap error in the Xcode console?

Looking forward to see the focused sample.

Albert   Worldwide Developer Relations.

Answer 2

Engineer OP

Apple

3w

Please see the reply in https://developer.apple.com/forums/thread/827043 for more information on how to prepare data for the custom input since using a pre-populated array can cause out of memory traps on iOS devices. Also, if the orientations do not match, for example, or the pixel buffer format is incorrect, the data can be rejected. You can view the logs in Console.app to see any debug information near where these are sent to the system for processing.

Also we recommend starting with folder input, which you mentioned isn't working either. There could be multiple reasons for this failure (besides alignment) which cannot be debugged without more information. The best course of action is to use https://developer.apple.com/feedback-assistant/ and file a bug report with the device, operating system, and data for our engineers to investigate.

Thanks!

Answer 3

Axuby OP

2w

Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I have created a feedback report as well like recommended with this ID: FB22823733 with more elaborate images of my implementation, also see here https://origin-devforums.apple.com/forums/thread/827311

Answer 4

Vision Pro Engineer OP

Apple

4h

Recommended

Hello :

Thank you for filing this feedback.

You can see the status of your feedback in Feedback Assistant. There, you can track if the report is still being investigated, has a potential identifiable fix, or has been resolved in another way. The status appears beside the label "Resolution." We're unable to share any updates on specific reports on the forums.

For more details on when you'll see updates to your report, please see What to expect after submission.