Processing frames on ARKit

Hello everyone, we have developed an app using ARKit in iOS and MLKit in Android.

With normal light (lets say in a room with a single lightbulb) ARKit is able to detect the face of the user and is able to draw a mesh over it.

When light is very low (think a dark night) or inexistent (total darkness) ARKit is unable to detect the face and nothing is processed.

The original approach was to let the ARSCNView apply a filter but the filter is applied AFTER the calculations are done. So even if you increase the image contrast ARKit won't process the frame.

Then we tried inverting the control, handling ourselves the AVCaptureSession and feed an ARKit session each time a frame arrives with an imageAnchor containing the image, but adding anchors caused our UI to freeze or become unresponsive after a while.

Any ideas on what to test given that the AVCaptureSession is private and can't be used?

Processing frames on ARKit
 
 
Q