Truedepth Camera on iPhone13 products - lower accuracy of depth data?

Hi,

just experienced using the Apple demo app for Truedepth images on different devices that there are significant differences in the provided data quality.

Data derived on iPhones before iPhone 13 lineup provide quite smooth surfaces - like you may know from one of the many different 3D scanner apps displaying data from the front facing Truedepth camera.

Data derived on e.g. iPhone13 Pro has now some kind of wavy overlaying structures and has - visually perceived - very low accuracy compared to older iPhones.

iPhone 12pro: data as pointcloud, object about 25cm from phone:

iPhone 13pro: data as pointcloud, same setup

Tried it out on different iPhone 13 devices, same result, all running on latest iOS. Images captured with the same code. Capturing by using some of the standard 3D scanner apps for the Truedepth camera are providing similar lower quality images or point clouds.  

Is this due to degraded hardware (smaller sized Truedepth camera) on new iPhone release or a software issue from within iOS, e.g. driver part of the Truedepth camera?

 

Are there any foreseen improvements or solutions already announced by Apple?

 

Best

Holger

Post not yet marked as solved Up vote post of r3176 Down vote post of r3176
3.2k views

Replies

I can confirm that we have experienced the exact same findings.

Furthermore, beyond the noisy depth data returned (likely due to switching to a ToF sensor, which then appears to be upsampled which might explain the "noise"), there seems to be a very concerning bug with regards to how depth data is handled on one side of the face. In all previous devices with old version of TrueDepth (prior to iPhone 13), there is clear separation of z-depth data. HOWEVER, on iPhone 13 variants, there is "blending" of the z-depth on the edge of one side of the face. This is a huge red flag (tested on iOS 15.x). This blending is apparent if you run unaltered apple sample code found here on an iPhone 13 variant (iPhone 13, 13 Pro, 13 Pro Max, 13 mini). https://developer.apple.com/documentation/avfoundation/cameras_and_media_capture/streaming_depth_data_from_the_truedepth_camera

Implementing filtering/smoothing of the data does NOT fix this concerning bug. It is temporal filters, so while it does aid in reducing the "bumpy noise" returned from the sensor, the incorrect Z-depth on one side of the face (or any object scanned) is not fixed. And temporal smoothing can be done by averaging across multiple depth buffers (its nice this is done for you), but I want clean data on a per-depth-frame returned.

Huge red flag bug in returned depth data from iPhone 13 devices (aside from noisy/bumpy/inaccurate depth). Highlighting issue in previous image posted.

Highlighting (in previous image posted) the bug I mentioned in returned depth data from any iPhone 13 variant device.

I’m not am apple engineer or anything but I’m pretty sure the issue is caused because of how close together the sensors she now, if you think about it the dot projector and IR cameras need to be squeezed closer, which would decrease the parallax off The dots and the camera, all I think it’sa hardware fault rather than software unfortunately

Any news from an Apple Developer?