PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates

Question

Created 5h

Replies 2

Boosts 0

Participants 3

Xcode hangs when I call PhotogrammetrySession(input: [PhotogrammetrySample]) with objectMask set and traps on some devices, see the attached screenshot, it gets to the function and hangs. Even the folder reconstruction also doesn't complete as it can't find alignment and displays the CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. In this case it failed while object masking was ON, so RealityKit could not find enough consistent feature tracks inside the masked pixels across the image set.

What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally.

The ObjectMasks are actually segmentation masks from an ML algorithm .

To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing.

I will appreciate a timely response and willing to provide more clarity and informations, thank you so much for your understanding

Boost

Answer 1

DTS Engineer OP

Apple

2h

Thank you for your post. I recommend you provide a link to a simple and focused project that demonstrates the issue, allowing engineers to examine it in detail the issue you are reporting.

If so, please share a link to your test project. That'll help us better understand what's going on. If you're not familiar with preparing a test project, take a look at Creating a test project.

The little a know is a hard hang or trap when passing an array of PhotogrammetrySample objects usually points to a data format mismatch? Because masking removes the background features that Object Capture relies on to figure out where the camera is in 3D space.

What are the exact dimensions and pixel formats of the image and objectMask you are passing? Are you running this on macOS or iOS? What version of Xcode are you using? What is the specific trap error in the Xcode console?

Looking forward to see the focused sample.

Albert   Worldwide Developer Relations.

0

Answer 2

Engineer OP

Apple

2h

Please see the reply in https://developer.apple.com/forums/thread/827043 for more information on how to prepare data for the custom input since using a pre-populated array can cause out of memory traps on iOS devices. Also, if the orientations do not match, for example, or the pixel buffer format is incorrect, the data can be rejected. You can view the logs in Console.app to see any debug information near where these are sent to the system for processing.

Also we recommend starting with folder input, which you mentioned isn't working either. There could be multiple reasons for this failure (besides alignment) which cannot be debugged without more information. The best course of action is to use https://developer.apple.com/feedback-assistant/ and file a bug report with the device, operating system, and data for our engineers to investigate.

Thanks!

0