Wrong TrueDepth (meta) data in AV and ARKit sessions on IPads

Hi,

while working with TrueDepth RGB-D data both in AVFoundation sessions and ARKit sessions, we stumbled across a significant issue with meta data and depth to RGB alignment on the following iPads: iPad 11'' 2gen, 11'' 3gen, 12.9'' 4gen and 12.9'' 5gen.

We compiled an elaborate report: https://arxiv.org/abs/2201.10865 that details the issues we faced: and how you can correct them in post processing. Actually there are actually 2 different problem

TL;DR; On 7Mpix TrueDepth modules (11'' 2gen and 12.9''4gen) the depth is not aligned to the RGB correctly in an ARKit session (this was remarked in this post over a year ago for some iPhones): https://developer.apple.com/forums/thread/660388

On 12MPix wide angle TrueDepth modules (11''3gen and 12.9''5gen) the issue is more severe. Here the depth map align in both ARKit and AV session, however the factory calibrated camera focal length that you get in both sessions is wrong. In AVSession it's off around 7.5% and in ARKit session about 15%. We verified this across several iPads with camera calibration and object measurements.

Especially the second issue makes ALL measurements derived with those iPad generations and the TrueDepth camera wrong.

Over a week ago we already posted a ticket using the feedback assistant (FB9848651). And we also opened a ticket using the TSI (Technical Support Incident) interface.

However so far no reaction :/ Does anyone have an idea what we could do?

Cheers Steffen

Post not yet marked as solved Up vote post of urbste Down vote post of urbste
835 views
  • Is there any update on this issue. It seems like the ticket FB9848651 doesn’t exist anymore or it is just not visible for everyone.

    The paper "On the Issues of TrueDepth Sensor Data" was very insightful in order to understand the origin of this problem. However it only covers Ipads until gen. 5. Does anyone know if Ipad pro gen. 6 is also affected?

Add a Comment

Replies

Does anyone has an update about this? The mentioned ticket FB9848651 on the feedback assistant seems to either no longer exist or not publicly accessible anymore.

The paper "On the Issues of TrueDepth Sensor Data" is very insightful and describes the origin of the problem. However it only covers the devices until Ipad Pro gen 5. Does anyone know if the Ipad Pro gen 6 is also affected?