Is saving portrait effects matte with image data to PHPhotoLibrary possible?

Objective and steps

Use the device front true depth camera (iPhone 12 Pro Max) to capture image data, live photo data and metadata (e.g. depth data and portrait effects matte) using AVFoundation capture principles into an AVCapturePhoto object. Save this captured object with its metadata to PHPhotoLibrary using a PHAssetCreationRequest object API.

Result

Image data, live data, disparity depth data (640x480 px) and some metadata is stored with the image through the PHPhotoLibrary API but the high quality portrait effects matte is lost.

Notes

Upon receiving the AVCapturePhoto object from AVFoundation capture delegate API I can verify that AVCapturePhoto object contains a high quality portrait effects matte member object. Using object's fileDataRepresentation() to obtain Data blob, writing that to a test file URL and reading it back I can see that flattened data API writes and restores the portrait effects matte.

However, it gets stripped from the data when writing through the PHPhotoLibrary asset creation request. When later picking the image e.g. with PHPickerViewController + PHPickerResult and peeking into the object's data with CGImageSourceCopyAuxiliaryDataInfoAtIndex() I can see that there is data dictionary only for key kCGImageAuxiliaryDataTypeDisparity, and kCGImageAuxiliaryDataTypeDepth and kCGImageAuxiliaryDataTypePortraitEffectsMatte are both missing.

Please, anyone has more detailed information if this possible at all? Thanks!

This was finally resolved. Here are some notes in case someone makes the same mistakes when e.g. experimenting with AVCam source code.

It turned out the portrait matte was indeed written to PHPhotoLibrary with the image, but reading it back with PHImagePickerController needed some attention. When accessing the image data with NSItemProvider.loadDataRepresentation(forTypeIdentifier: representation) the representation identifier has to be the same format and type the file was saved. For example my photo container was of type .heic. Now, even though NSItemProvider.hasRepresentationConforming(toTypeIdentifier: representation...) listed type identifiers,

UTType.heic

UTType.jpeg

as supported, only the first could load the matte correctly. If I tried to access the data with .jpeg identifier portrait matte was not loaded, but strange error,

"Could not create a bookmark: NSError: Cocoa 257 "The file couldn’t be opened because you don’t have permission to view it"

kept appearing on Xcode console.

The second mistake was capturing and storing live photo data with portrait matte. It seems possible, but at least iOS Photos app can't handle those photos properly in edit mode and saving the edit always fails in error

"Photo could not be saved, please try again later."

So, don't enable live photo options in capture settings when capturing portrait matte.

Is saving portrait effects matte with image data to PHPhotoLibrary possible?
 
 
Q