Hello Developers,
I am working on an app where I need to capture 48MP high-resolution photos using the ultra-wide camera of the iPhone 16 Pro while an AR session is running. The goal is to take these photos without interrupting or impacting the AR session, which uses the main wide-angle camera. Despite extensive testing and various approaches, we have been unable to achieve the desired functionality.
What We Have Tried So Far
1. Using AVCaptureMultiCamSession:
• We attempted to leverage AVCaptureMultiCamSession to simultaneously use the wide-angle camera for ARKit and the ultra-wide camera for photo capture.
• However, this approach resulted in resource conflicts, with errors such as Cannot Record (OSStatus error -16409) and dropped frames.
Additionally, the ultra-wide camera feed would frequently freeze or stop.
2. Dedicated AVCaptureSession for the Ultra-Wide Camera:
• We separated the ultra-wide camera into its own AVCaptureSession while letting ARKit exclusively use the wide-angle camera.
• This setup showed initial promise, but the ultra-wide camera feed would still stop running after a very short time (under one second).
• Debugging logs indicated potential system-level interruptions, possibly due to resource prioritization by iOS.
3. Notification-Based Monitoring:
• We implemented monitoring for session interruptions (AVCaptureSession.wasInterruptedNotification), but this provided limited insights into the exact cause of the session stopping.
• We suspect iOS is de-prioritizing the ultra-wide camera session due to resource management policies or conflicts with ARKit.
4. Adjusting Camera Configurations:
• We attempted to simplify both ARKit and AVCaptureSession configurations by reducing features like depth data and by using lower session presets for video capture. However, the core issue persisted.
The Core Problem
• The ultra-wide camera session frequently stops or freezes when used alongside ARKit.
• Capturing high-resolution 48MP photos during the AR session is critical to the functionality of our app.
Question
Has anyone successfully implemented a similar setup? Specifically: • Capturing 48MP photos with the ultra-wide camera while ARKit is actively using the main camera. • Avoiding conflicts between ARKit and AVCaptureSession for the ultra-wide camera.
Any insights, suggestions, or alternative approaches would be greatly appreciated. Thank you in advance for your help! 😊