Photos & Camera

RSS for tag

Explore technical aspects of capturing high-quality photos and videos, including exposure control, focus modes, and RAW capture options.

Posts under Photos & Camera subtopic

Post

Replies

Boosts

Views

Activity

Camera app best practices
As a reminder, the recommended approach to monitor camera rotations, including on upcoming hardware, is to use the AVCaptureDevice.RotationCoordinator / AVCaptureDeviceRotationCoordinator API. In many common scenarios on iPhone Duo, proper use of the rotation coordinator is the only way to ensure that your camera previews and captures use the orientation expected by the user. AVCaptureVideoOrientation has been deprecated as of iOS 17.0 and iPadOS 17.0. When switching cameras in your app, make sure to create a rotation coordinator for that device. To ensure that your app has implemented the rotation coordinator API correctly, you should test different orientations and sizes of your app on an iPad with Stage Manager enabled. Additional information: Sample Code: Supporting device rotation in your camera app Tech Talk: Build a great camera experience for iPhone Duo WWDC23 session: Support external cameras in your iPadOS app
0
0
497
Oct ’26
IOSurface with System Extensions
Hi All, I'm working on a camera system extension where the main app is supposed to transfer a video stream using IOSurface memory sharing to the cam extension. I have built a sample app that does contains all the logic, but without a camera extension. So I'm essentially using IOSurface to render a video in one SwiftUI view and show the result in another SwiftUI view. Just for testing purposes. And everything works fine so far. Now, when moving the receiver code to the camera extensions, I'm having problems in accessing the IOSurface via ID. I am sharing the IOSurface ID via UserDefaults. I know from the logs the ID is correctly transferred. Here is the code that uses IOSurfaceLookup to get the IOSurface. But this fails with the given message. The error message prints the surface ID which is the correct one. I know this from the main app where I get the ID and print it as well. private var surfaceId: Int = -1 { didSet { logger.info("surfaceId has changed") if surfaceId == -1 { stopReceivingFrames() ioSurface = nil } else { guard let surface = IOSurfaceLookup(IOSurfaceID(surfaceId)) else { logger.error("failed to lookup IOSurface with ID: \(self.surfaceId)") return } self.ioSurface = surface logger.info("surface set, now starting receiving frames") startReceivingFrames() } } } My gut feeling says that this issue might be related to some missing entitlement, sandboxing. In general, I have a working camera extension. I'm just not able to render a video in the main app, and send it over to the camera extension to overlay another web cam. Both, the main app and camera extension are in the same XCode workspace and share the same AppGroup. In short, my actual questions are: Is there any entitlement required for using IOSurface between app and camera system extension? Is using IOSurface actually possible in system extensions? Is there any specific setting/requirement that I need to handle to make this work?
1
1
1.2k
2h
3D scanning and 3D model with texture mapping
I am new here and would appreciate help in coding or an explanation what to use in swift for an app which will be able to capture LiDAR scanning and RGB data from taken pictures, generate a 3D mesh, and create .OBJ, .MTL, and .JPEG file set for further manipulation of 3D model. I am able to create from LiDAR scanning 3D mesh and .OBJ file but can not generate .MTL and .JPEG for a texture of 3D model.
2
0
1.1k
2d
Camera preview rotated 90 degrees on initial presentation on iPadOS 27.2.1, corrected after returning from Control Center
Dear Apple Support Team, We are investigating a camera preview orientation issue on an iPad and would appreciate guidance on the relevant UIKit and AVFoundation behavior. Our app uses Flutter and the third-party Regula Document Reader SDK for passport MRZ scanning. We understand that third-party SDK internals are outside Apple’s support scope. Our question concerns the relationship between the active interface orientation, presentation lifecycle, and camera preview connection orientation. Environment Device: iPad (A16), hardware identifier iPad15,7 iPadOS: 27.2.1, as reported by the device and diagnostic export App distribution: TestFlight App version in the diagnostic log: 2.0.14, build 7 Flutter plugin: flutter_document_reader_api 9.8.1718 Runtime Regula SDK API version: 9.8.6944 Runtime Regula SDK Core version: 9.8.20492 Steps to reproduce Hold the affected iPad in landscape orientation and open the app. Open the passport scanner. The scanner interface appears in landscape, but the camera image appears rotated by 90 degrees. Open Control Center, then dismiss it and return to the app. The preview can then appear in the correct orientation. Close and reopen the scanner; the issue may recur. Physically rotating the device does not consistently resolve the issue. Expected: The preview should match the current interface orientation from initial presentation. Actual: The preview initially has an orientation mismatch and may recover only after leaving and returning to the app. Diagnostic observations We collected initial-entry and app-resume observations within the same scanner session: Before scanner presentation, the active UIWindowScene.interfaceOrientation is landscapeRight. The SDK’s presenting view controller belongs to the active application window. The scanner controller has landscape dimensions of approximately 1180 × 820. The third-party SDK’s supported interface orientation is configured for landscape right. However, the initial preview connection reports videoOrientation = portrait and videoRotationAngle = 90. Updating supported interface orientations and requesting layout updates did not resolve the initial mismatch in this test. A subsequent app-resume observation reports videoOrientation = landscapeRight and videoRotationAngle = 0. This change is recorded before our app’s delayed layout refresh. We recognize that interface orientation and capture rotation are different concepts, and that connection metadata alone does not establish the displayed pixel orientation. We are providing these observations alongside the visible symptom rather than treating a particular angle as universally correct. Scope and investigation status Our separate QR scanner and receipt camera work normally on the same device. The passport scanner also works normally on many other iPads. We have not established that this affects all devices running this iPadOS version. An app-side orientation enum comparison issue was corrected before collecting the build 7 log, but the preview mismatch remained. We have not isolated the cause to iPadOS, our presentation code, or the SDK’s capture initialization. We do not yet have a focused standalone Xcode reproduction project. We are also preparing an inquiry to the SDK vendor. Questions Are there known circumstances in which a camera preview initially uses portrait orientation despite being presented in a landscape UIWindowScene? What is the recommended lifecycle point for determining the initial interface orientation and configuring the preview connection, particularly while presentation or rotation transitions are in progress? Which supported APIs should be used to keep preview rotation synchronized with interface changes on this iPadOS version? Could the recovery after dismissing Control Center indicate an initialization or lifecycle ordering issue? What additional observations would help distinguish these possibilities? What should we include in a minimal native reproduction or system diagnostic collection to make this issue actionable? We can provide a symptom video and an application diagnostic export. These were collected in separate tests on the same affected device and are not time-synchronized. The application export is not an Apple sysdiagnose. Thank you for any guidance on how to investigate this further. With Best Regards, GTF_Japan_Service
0
0
27
2d
iOS 27.2: iPhone 13 Pro / 14 Pro heat up sooner during 4K camera capture than on iOS 27.0
We make a camera app (AVCaptureSession at 4K, live preview, recording with AVAssetWriter). Since updating to iOS 27.2, a user with an iPhone 13 Pro reports that the phone gets hot while using our app. Our own iPhone 14 Pro also runs warmer on 27.2 than it did on 27.0. To check, we reran on the 14 Pro the same automated test we ran there on iOS 27.0 in September. Each run lasts 300 seconds with the phone on power and fully charged, Low Power Mode off, and starts at ProcessInfo.thermalState == .nominal. We log thermalState every 5 seconds. Capture is 3840x2160 at 60 fps in all three runs: Idle preview, no processing: iOS 27.0: stayed Nominal for all 300 s iOS 27.2 (24B115): Fair at 278 s Idle preview with a Metal overlay: iOS 27.0: stayed Nominal for all 300 s iOS 27.2: Fair at 123 s Recording, HEVC 10-bit: iOS 27.0: Fair at 38 s, Serious at 108 s iOS 27.2: Fair at 33 s, Serious at 83 s Frame delivery held up. The 5-minute recording lost no frames, even at Serious. Questions: Are there known changes in iOS 27.2 to power use or thermal management during camera capture (AVCaptureSession at 4K) on A15 / A16 devices? Have the thresholds behind ProcessInfo.thermalState or AVCaptureDevice.SystemPressureState changed in 27.2? What data would be most useful in a Feedback report for this: a sysdiagnose taken right after a run, or a specific Instruments template?
1
0
545
1w
[PhotoKit] Significant Delay when Opening Photo/Video Gallery from Camera View
Experiencing a noticeable delay where opening the photo and video gallery from application's PTX camera interface takes approximately 3 to 4 seconds. This delay degrades the user experience and occurs consistently across both the latest iOS 27 RC and previous iOS 26 builds. There is a roughly 3-4 second delay between tapping the gallery button and the gallery interface actually rendering and becoming responsive. Steps Launch the application. Open the PTX Camera interface. Tap to select/open the Gallery. Observe the loading time.
0
0
51
1w
Public API for torch beam width (Adaptive True Tone flash)?
On iPhone 14 Pro and later, the system flashlight in Control Center can adjust beam width (narrow ↔ wide). As far as I can tell, AVCaptureDevice only exposes torchMode, torchLevel and setTorchModeOn(level:) I found nothing related to beam width Is there any supported way for a third-party app to control the torch beam width? If not, is this on the roadmap?
0
0
331
2w
Is activeLockedVideoFrameDuration supported on iPhone 16 Pro? Hardware or iOS limitation?
Hello Apple AVFoundation team, Could you clarify the supported iPhone models and OS versions for standalone locked video frame duration using AVCaptureDeviceInput.activeLockedVideoFrameDuration? The API is documented as available from iOS 26.0, but I understand that OS availability alone does not mean every camera device/configuration supports it. I have read the API documentation and the DTS reply in iPhone 17 Pro's Genlock feature, but could not find a model compatibility list for the standalone locked-duration API. Scope: I am investigating actual capture timing for multi-iPhone motion capture. This question is about locking each camera's frame period using its internal timing, without an external genlock accessory. I am not asking about encoding a constant-frame-rate file, ordinary equal min/max frame durations, timecode metadata, or synchronizing the first exposure across phones. Physical-device test iPhone 16 Pro, iOS 26.2 (23C55). Built with Xcode 26.3 and the iOS 26.2 SDK. One AVCaptureSession, one physical rear .builtInWideAngleCamera input, and one AVCaptureVideoDataOutput. 8-bit SDR 420v formats, tested at 1920 x 1080 and 3840 x 2160, each at 30 and 60 fps. Auto video frame rate disabled; no spatial video capture or multicam session. Manual exposure approximately 2 ms, shorter than half a frame at both tested rates. The session is running and delivering frames. Two separate probe runs produced the same results, including capability readbacks after a settling period. For all four tested configurations: input.isLockedVideoFrameDurationSupported = false device.minSupportedLockedVideoFrameDuration = kCMTimeInvalid input.activeLockedVideoFrameDuration = kCMTimeInvalid device.isVideoFrameDurationLocked = falseOrdinary activeVideoMinFrameDuration and activeVideoMaxFrameDuration can both be set to the requested CMTime(value: 1, timescale: 30) or CMTime(value: 1, timescale: 60), and frames are delivered. This is only a configuration readback, not a claim of exact capture timing in the ordinary mode. The app directly queries the public support properties; it has no model whitelist. It does not attempt to set a valid locked duration when support is false, as the documented setter contract would reject that. These tests cover the selected formats, not every camera/format on this phone. Questions Which iPhone models support this standalone API? In particular, is false expected on iPhone 16 Pro? Is there an official compatibility list covering iPhone 13/14/15/16 and iPhone 17 models? Is the iPhone 16 Pro result a hardware capability restriction, an iOS 26.2 limitation/known issue, or a configuration requirement? Is there a released iOS version that enables this on iPhone 16 Pro, or is newer hardware required? What is the minimum supported capture configuration and correct time to query support? Are there restrictions involving the physical versus virtual camera input, pixel format, SDR/HDR, or exposure mode that could explain this result? If so, which configuration should be tested before concluding that the device does not support it? On supported hardware, can this standalone lock be enabled without an external accessory or a special entitlement, independently of follow(_:videoFrameDuration:delegate:)? Relevant documentation: activeLockedVideoFrameDuration isLockedVideoFrameDurationSupported minSupportedLockedVideoFrameDuration I can provide a focused reproducer if needed. Thank you for clarifying the device requirements and whether an OS upgrade can change the result.
1
0
646
2w
iPhone 17 Pro: Is Native Apple Log 2 ISP Processing Available for 4224×3024/60fps RAW?
Hi everyone, I'm developing an iOS camera app that aims to record 4224×3024 Open Gate video at 60fps in 10-bit HEVC with Apple Log 2. Native Apple ISP-processed Apple Log 2 capture works at 3840×2160/60fps. However, at 4224×3024/60fps, the tested capture configuration only provides Bayer RAW (btp2). I've successfully decoded ProRes RAW using Apple's decoder into linear Rec.2020 and applied the published Apple Gamut and Apple Log 2 transforms. However, the resulting chroma is only approximately 72–74% of the native ISP output in controlled color-chart comparisons. I've also confirmed that VTRAWProcessingSession and related RAW processing controls are not publicly available to iOS apps in the iPhoneOS 27.0 SDK. My questions are: Is there a public iOS API that applies the same color and tone rendering as Apple's native camera ISP to captured Bayer RAW or ProRes RAW? Is there a supported configuration for obtaining full-resolution 4224×3024/60fps ISP-processed Apple Log 2 output (x420 or x422), without preview proxies or upscaling? If neither is supported, what is Apple's recommended approach? Is this a hardware limitation or a public API limitation? The goal is to use Apple's official color processing pipeline without empirically fitted matrices, LUTs, or arbitrary color corrections. Any guidance would be greatly appreciated. Thank you!
0
0
347
3w
iPhone Duo: continuous front-camera depth for measuring face distance
Hi, I’m building an app that counts push-ups with the phone on the floor. It currently uses ARKit and TrueDepth to measure the distance between the camera and the user’s face. I want to support iPhone Duo too, but I don’t have one. I only have an iPhone 17 Pro. I watched the camera presentation and saw that depth is supported when accessing the individual cameras. I’m trying to understand what that means for my app. Can either front camera provide continuous depth through AVCaptureDepthDataOutput, or is it only available when taking photos? Does it give actual distances in metres, or relative depth estimates? The face would usually be around 20 to 80 cm from the camera. Also, is there an equivalent camera mode on my iPhone 17 Pro that I could use to test the same approach, or would I need an actual Duo? It’s okay if counting is less accurate than with TrueDepth. I mainly need to detect the user moving down and back up reliably. Thank you!
1
0
346
3w
Radiometric interpretation of Apple ProRAW and Bayer RAW access via AVFoundation
I am working on a computational photography research project involving multi-exposure HDR reconstruction using Bayer RAW and Apple ProRAW captures. I would like to clarify the radiometric interpretation of Apple ProRAW and the availability of Bayer RAW capture through AVFoundation. My questions are: On current iPhone Pro devices, is it possible for third-party apps to capture and export true Bayer-pattern RAW DNG files through AVFoundation, rather than Apple ProRAW linear DNG files? If so, which availableRawPhotoPixelFormatTypes correspond to Bayer RAW, and what device or format restrictions apply? Apple ProRAW appears to be demosaiced and computationally processed, and may include multi-frame fusion. Is the decoded ProRAW image intended to be radiometrically linear and scene-referred? For a bracketed ProRAW sequence captured with fixed ISO, white balance, lens, and focus, but different exposure times, can one assume that the decoded linear pixel values Y_i(p) satisfy an exposure-proportional model in non-saturated regions, such as Y_i(p) ≈ t_i R(p), across brackets? This question is about radiometric consistency for algorithmic use, not about visual editing or tone mapping. Thank you for your help.
2
0
2.1k
3w
Radiometric interpretation of Apple ProRAW and Bayer RAW access via AVFoundation
I am working on a computational photography research project involving multi-exposure HDR reconstruction using Bayer RAW and Apple ProRAW captures. I would like to clarify the radiometric interpretation of Apple ProRAW and the availability of Bayer RAW capture through AVFoundation. My questions are: 1.On current iPhone Pro devices, is it possible for third-party apps to capture and export true Bayer-pattern RAW DNG files through AVFoundation, rather than Apple ProRAW linear DNG files? If so, which availableRawPhotoPixelFormatTypes correspond to Bayer RAW, and what device or format restrictions apply? 2.Apple ProRAW appears to be demosaiced and computationally processed, and may include multi-frame fusion. Is the decoded ProRAW image intended to be radiometrically linear and scene-referred? 3.For a bracketed ProRAW sequence captured with fixed ISO, white balance, lens, and focus, but different exposure times, can one assume that the decoded linear pixel values Y_i(p) satisfy an exposure-proportional model in non-saturated regions, such as Y_i(p) ≈ t_i R(p), across brackets? This question is about radiometric consistency for algorithmic use, not about visual editing or tone mapping. Thank you for your help.
2
0
1.7k
3w
AVFoundation: full-resolution Quad Bayer RAW capture path for third-party computational photography? (FB25352517)
At the WWDC26 Camera and Photo Technologies Group Lab (session 8018, ~29:55 to 32:46), the panel explained why ProRAW can produce 48 MP output from Quad Bayer sensors while native Bayer RAW is limited to the binned resolution. ProRAW is already demosaiced and linearized, and may incorporate Photonic Engine multi-frame processing. The native RAW path exposes the binned Bayer result. A true full-resolution RAW output would be Quad Bayer, not conventional Bayer, and supporting it properly needs a decoding path around it. The panel suggested filing Feedback Assistant requests if developers need this, so I filed FB25352517. My use case is computational photography that wants minimally processed sensor-domain samples on purpose — its own multi-frame alignment, denoising, HDR fusion, motion rejection, and super-resolution. Adobe Project Indigo is a public example of that kind of pipeline. It captures RAW bursts and uses measured sub-pixel variation between frames to reconstruct detail. Processed 48 MP photos and ProRAW don't cover that. They've already been through Apple's reconstruction or demosaic. Is filing enhancement requests still the intended path for a supported full-resolution Quad Bayer RAW or mosaic capture path? If so, what would make these reports more useful — mosaic metadata, DNG representation, CIRAWFilter support, AVCaptureDevice.Format capability discovery, something else? WWDC26 session: https://developer.apple.com/videos/play/wwdc2026/8018/ Project Indigo technical overview: https://research.adobe.com/articles/indigo/indigo.html
2
0
521
3w
Retain warning on cloudIdentifierMappings(forLocalIdentifiers: [String] )
Whenever I call this method on the PHPhotoLibrary class I get a warning which I've included below. I've tried looking in the Apple forums, Stack Overflow etc. and nobody seems to mention this issue that I can find. Is this warning unavoidable or am I using the API wrong? I can reproduce this trivially in a simple untitled project. Warning: Retain cycle risk: client-side managed object fetch on long-lived context for entity Asset, on context named: PHPhotoLibrary (inode=19657147) .Generic, consider using -[PHPhotoLibrary performAndWaitOnAutoResettingLibraryWithName:workBlock:] or +[PLDatabaseContext newShortLivedLibraryWithName:bundle:]...work, or using a dictionary result type, or suppressing this warning using 'clientSideLongLivedFetchWarningsSuppressed'). STEPS TO REPRODUCE Create a new untitled App project in Xcode. Set a photo library usage privacy description in the Info for the project target Replace ContentView.swift with the code included below. Make sure there is at least one photo in the photo library on the target device. A simulator is fine. Run the app Click the Get Cloud ID button, respond to the authorization dialog Click the Get Cloud ID button again Look in Xcode and see the warning import SwiftUI import Playgrounds import Photos struct ContentView: View { var body: some View { Button( action: getCloudID ) { Text("Get Cloud ID") } .padding() } private func getCloudID() { var fetchOptions = PHFetchOptions() fetchOptions.fetchLimit = 1 let assetResult = PHAsset.fetchAssets(with: .image, options: fetchOptions) var localIds = [String]() assetResult.enumerateObjects() { asset, _, _ in localIds.append(asset.localIdentifier) } let mappings = PHPhotoLibrary.shared().cloudIdentifierMappings(forLocalIdentifiers: localIds ) for mapping in mappings { debugPrint("cloud ID \(mapping.value)") } } } `
0
0
69
3w
FaceGroupAnalyzer: one unreadable asset makes `insertOrUpdateAssets` deliver zero elements, discarding results already computed for the valid assets in the batch
If a single asset in the array passed to insertOrUpdateAssets cannot be read, the returned AsyncSequence throws and delivers zero elements — including for the valid assets that appear before the offending one in the array. In my measurement a batch of 5 — two valid photographs, one zero-byte .jpg, then two more valid photographs — delivered 0 of 5 elements and reported 0 faces. The framework's own stdout log shows it had already processed the valid photographs before failing, so the work was done and then thrown away. This is the behaviour I would expect from a function returning [Result] after processing everything, not from a streaming AsyncSequence. The whole reason to expose an async sequence is to deliver results as they are produced; here the sequence produces nothing at all, which makes the streaming shape actively misleading. Why this matters at library scale. I am cataloguing 97 photo libraries, many of them archives of scanned family photographs going back to the 1970s, where a handful of damaged files is normal rather than exceptional. As the API stands, the batch size I choose for throughput is also the amount of work a single corrupt file destroys — with a batch of 100, one bad file costs 100 images. The only safe strategy is to catch the failure and re-submit the batch one asset at a time to find the culprit, which turns a rare bad file into a full re-run of that batch and makes the worst case quadratic in the number of bad files. Note also that the error gives no indication of which asset failed (see the related enhancement request on error taxonomy), so isolating the culprit by re-submission is the only option available. Feedback: FB24174733
1
0
550
Oct ’26
FaceGroupAnalyzer: a truncated image file reports zero faces with no error, indistinguishable from a photograph that contains no people
An image file whose data is cut in half is accepted by insertOrUpdateAssets, produces no error, and is reported as an image containing zero faces. That result is indistinguishable from a photograph that genuinely has no people in it. In my measurement the intact file reported 4 faces and a copy truncated to half its bytes reported 0 faces — both without throwing. Why this is worse than an error. A client that receives "zero faces, no error" will record the asset as analysed, no people present. That is a permanent, silent, incorrect result: the file is never revisited, and the people in it are lost from the catalogue with no diagnostic anywhere. An error would have been recorded as a failure and retried later. Silence is the one outcome that cannot be recovered from. The failure mode is also inconsistent. Other kinds of damaged input do throw — a zero-byte file, a text file with a .jpg extension, and a file whose interior bytes have been destroyed all raise MediaIntelligenceError.faceGroupProcessing. Truncation is the case that silently succeeds, and truncation is precisely the damage that partial downloads, interrupted copies and failing disks produce — the most common form of corruption in a real photo archive, and the one I have to handle in 97 libraries of mixed provenance. My current workaround is to fully decode every image myself before handing it to the framework, purely to detect truncation. That means every file is decoded twice, which roughly doubles the I/O of the analysis pass. Feedback: FB24174749
1
0
568
Oct ’26
iPhone Duo: Does CameraCaptureAccessory accept touch, what counts as an active camera session, and can both front cameras run at once?
After watching "Leverage multiple displays and scenes on iPhone Duo" and "Build a great camera experience for iPhone Duo", we have four questions about the outer display while the device is open: Does a view hosted in CameraCaptureAccessory receive touch input on the outer display, or is it display-only? The accessory is described as available when the app "has an active camera session." Does that require a running preview layer or an attached output (photo, movie or data), or is a running AVCaptureSession with an input enough? Is a session whose main purpose is to host the accessory a supported use? Can AVCaptureMultiCamSession run the inner (under-display) front camera and the outer front camera at the same time on iPhone Duo? If so, which device-type combinations are supported, and are there thermal or power limits? Is CameraCaptureAccessory the only way for an app to show content on the outer display while the inner display is live, or are scene accessories that don't depend on the camera planned? Our use case is not photography. We're exploring a companion view for a second person sitting across from the device owner, so we want to understand whether the camera accessory is the right tool before designing around it.
5
1
401
Oct ’26
Why does Apple prefer Quad Bayer over traditional Bayer sensors?
Hi everyone, I've been reading about iPhone camera sensors recently, and there's something I've been wondering about. Apple has explained the benefits of Quad Bayer, especially pixel binning for low-light photography and high-resolution capture. But why not use a traditional Bayer sensor with Dual Pixel PDAF instead? As far as I understand, a native Bayer arrangement could potentially preserve more fine detail without requiring remosaicing. And if both sensors have the same size and pixel count, couldn't the Bayer sensor also produce a cleaner 12MP image by downsampling? I'm also curious about the microlens design. Some sensors use one microlens shared by four pixels for phase detection. Wouldn't individual microlenses potentially offer better spatial resolution? So my main question is: What hardware-level advantages make Quad Bayer worth choosing over traditional Bayer, despite the additional image reconstruction involved? Is it mainly about read noise, analog binning, power efficiency, autofocus, or something else? I'm not a camera engineer, just someone interested in understanding the technology behind iPhone cameras. I'd really appreciate any technical insight. Thanks!
0
0
147
Oct ’26
Ultra Wide camera: frame-to-frame jitter below 2.0x and sharpness step at 2.0x during native zoom ramp (iPhone 17 Pro, 4K/60)
I am developing CineZoom, part of CameraControl, a remote camera-direction system (macOS app controlling an iPhone capture engine). Its core requirement is a smooth, repeatable cinematic zoom. On iPhone 17 Pro (iOS 26.5.1), using the physical Back Ultra Wide camera at 3840x2160, 60 fps, a single native ramp(toVideoZoomFactor:withRate:) from 1.7x to 2.1x over 5 seconds shows a visible stutter. I isolated it with a minimal diagnostic app (no networking) and analysed four tripod recordings frame by frame, in two locations with different lighting. The results are consistent across all four: Jitter below 2.0x: while the zoom factor is below 2.0, the image centre shifts about 2.8 px RMS between consecutive frames (peaks ~5 px), alternating almost every frame (26-28 Hz). Above 2.0x, during the same ramp at the same speed, it drops to 1.2-1.5 px. Sharpness step at 2.0x: centre-crop sharpness drops by 23-31% within a few frames exactly when the zoom factor crosses 2.0. There is no consistent effect at the reported secondaryNativeResolutionZoomFactors value (1.7838). The zoom command path itself is correct: the zoom measured from the image matches videoZoomFactor throughout, with no dropped or duplicated frames. Questions: Is this expected behaviour for the Ultra Wide camera in this zoom range? Is the jitter related to video stabilization or to how the sensor crop is positioned in this readout mode? Is there a public API that reports or controls the zoom factor at which this transition happens, so an app can plan around it? What configuration do you recommend for a smooth continuous video zoom across 1.7x-2.1x on this camera (format, stabilization mode, constituent-camera switching)? Are there existing APIs that would let an app detect or compensate this (per-frame stabilization/crop metadata, consistent processing during zoom, or a reduced-processing video stream)? I have a full technical report, per-frame measurements, charts and the original recordings, and I am happy to share them. Thank you!
1
0
723
Oct ’26
Camera app best practices
As a reminder, the recommended approach to monitor camera rotations, including on upcoming hardware, is to use the AVCaptureDevice.RotationCoordinator / AVCaptureDeviceRotationCoordinator API. In many common scenarios on iPhone Duo, proper use of the rotation coordinator is the only way to ensure that your camera previews and captures use the orientation expected by the user. AVCaptureVideoOrientation has been deprecated as of iOS 17.0 and iPadOS 17.0. When switching cameras in your app, make sure to create a rotation coordinator for that device. To ensure that your app has implemented the rotation coordinator API correctly, you should test different orientations and sizes of your app on an iPad with Stage Manager enabled. Additional information: Sample Code: Supporting device rotation in your camera app Tech Talk: Build a great camera experience for iPhone Duo WWDC23 session: Support external cameras in your iPadOS app
Replies
0
Boosts
0
Views
497
Activity
Oct ’26
IOSurface with System Extensions
Hi All, I'm working on a camera system extension where the main app is supposed to transfer a video stream using IOSurface memory sharing to the cam extension. I have built a sample app that does contains all the logic, but without a camera extension. So I'm essentially using IOSurface to render a video in one SwiftUI view and show the result in another SwiftUI view. Just for testing purposes. And everything works fine so far. Now, when moving the receiver code to the camera extensions, I'm having problems in accessing the IOSurface via ID. I am sharing the IOSurface ID via UserDefaults. I know from the logs the ID is correctly transferred. Here is the code that uses IOSurfaceLookup to get the IOSurface. But this fails with the given message. The error message prints the surface ID which is the correct one. I know this from the main app where I get the ID and print it as well. private var surfaceId: Int = -1 { didSet { logger.info("surfaceId has changed") if surfaceId == -1 { stopReceivingFrames() ioSurface = nil } else { guard let surface = IOSurfaceLookup(IOSurfaceID(surfaceId)) else { logger.error("failed to lookup IOSurface with ID: \(self.surfaceId)") return } self.ioSurface = surface logger.info("surface set, now starting receiving frames") startReceivingFrames() } } } My gut feeling says that this issue might be related to some missing entitlement, sandboxing. In general, I have a working camera extension. I'm just not able to render a video in the main app, and send it over to the camera extension to overlay another web cam. Both, the main app and camera extension are in the same XCode workspace and share the same AppGroup. In short, my actual questions are: Is there any entitlement required for using IOSurface between app and camera system extension? Is using IOSurface actually possible in system extensions? Is there any specific setting/requirement that I need to handle to make this work?
Replies
1
Boosts
1
Views
1.2k
Activity
2h
3D scanning and 3D model with texture mapping
I am new here and would appreciate help in coding or an explanation what to use in swift for an app which will be able to capture LiDAR scanning and RGB data from taken pictures, generate a 3D mesh, and create .OBJ, .MTL, and .JPEG file set for further manipulation of 3D model. I am able to create from LiDAR scanning 3D mesh and .OBJ file but can not generate .MTL and .JPEG for a texture of 3D model.
Replies
2
Boosts
0
Views
1.1k
Activity
2d
Camera preview rotated 90 degrees on initial presentation on iPadOS 27.2.1, corrected after returning from Control Center
Dear Apple Support Team, We are investigating a camera preview orientation issue on an iPad and would appreciate guidance on the relevant UIKit and AVFoundation behavior. Our app uses Flutter and the third-party Regula Document Reader SDK for passport MRZ scanning. We understand that third-party SDK internals are outside Apple’s support scope. Our question concerns the relationship between the active interface orientation, presentation lifecycle, and camera preview connection orientation. Environment Device: iPad (A16), hardware identifier iPad15,7 iPadOS: 27.2.1, as reported by the device and diagnostic export App distribution: TestFlight App version in the diagnostic log: 2.0.14, build 7 Flutter plugin: flutter_document_reader_api 9.8.1718 Runtime Regula SDK API version: 9.8.6944 Runtime Regula SDK Core version: 9.8.20492 Steps to reproduce Hold the affected iPad in landscape orientation and open the app. Open the passport scanner. The scanner interface appears in landscape, but the camera image appears rotated by 90 degrees. Open Control Center, then dismiss it and return to the app. The preview can then appear in the correct orientation. Close and reopen the scanner; the issue may recur. Physically rotating the device does not consistently resolve the issue. Expected: The preview should match the current interface orientation from initial presentation. Actual: The preview initially has an orientation mismatch and may recover only after leaving and returning to the app. Diagnostic observations We collected initial-entry and app-resume observations within the same scanner session: Before scanner presentation, the active UIWindowScene.interfaceOrientation is landscapeRight. The SDK’s presenting view controller belongs to the active application window. The scanner controller has landscape dimensions of approximately 1180 × 820. The third-party SDK’s supported interface orientation is configured for landscape right. However, the initial preview connection reports videoOrientation = portrait and videoRotationAngle = 90. Updating supported interface orientations and requesting layout updates did not resolve the initial mismatch in this test. A subsequent app-resume observation reports videoOrientation = landscapeRight and videoRotationAngle = 0. This change is recorded before our app’s delayed layout refresh. We recognize that interface orientation and capture rotation are different concepts, and that connection metadata alone does not establish the displayed pixel orientation. We are providing these observations alongside the visible symptom rather than treating a particular angle as universally correct. Scope and investigation status Our separate QR scanner and receipt camera work normally on the same device. The passport scanner also works normally on many other iPads. We have not established that this affects all devices running this iPadOS version. An app-side orientation enum comparison issue was corrected before collecting the build 7 log, but the preview mismatch remained. We have not isolated the cause to iPadOS, our presentation code, or the SDK’s capture initialization. We do not yet have a focused standalone Xcode reproduction project. We are also preparing an inquiry to the SDK vendor. Questions Are there known circumstances in which a camera preview initially uses portrait orientation despite being presented in a landscape UIWindowScene? What is the recommended lifecycle point for determining the initial interface orientation and configuring the preview connection, particularly while presentation or rotation transitions are in progress? Which supported APIs should be used to keep preview rotation synchronized with interface changes on this iPadOS version? Could the recovery after dismissing Control Center indicate an initialization or lifecycle ordering issue? What additional observations would help distinguish these possibilities? What should we include in a minimal native reproduction or system diagnostic collection to make this issue actionable? We can provide a symptom video and an application diagnostic export. These were collected in separate tests on the same affected device and are not time-synchronized. The application export is not an Apple sysdiagnose. Thank you for any guidance on how to investigate this further. With Best Regards, GTF_Japan_Service
Replies
0
Boosts
0
Views
27
Activity
2d
iOS 27.2: iPhone 13 Pro / 14 Pro heat up sooner during 4K camera capture than on iOS 27.0
We make a camera app (AVCaptureSession at 4K, live preview, recording with AVAssetWriter). Since updating to iOS 27.2, a user with an iPhone 13 Pro reports that the phone gets hot while using our app. Our own iPhone 14 Pro also runs warmer on 27.2 than it did on 27.0. To check, we reran on the 14 Pro the same automated test we ran there on iOS 27.0 in September. Each run lasts 300 seconds with the phone on power and fully charged, Low Power Mode off, and starts at ProcessInfo.thermalState == .nominal. We log thermalState every 5 seconds. Capture is 3840x2160 at 60 fps in all three runs: Idle preview, no processing: iOS 27.0: stayed Nominal for all 300 s iOS 27.2 (24B115): Fair at 278 s Idle preview with a Metal overlay: iOS 27.0: stayed Nominal for all 300 s iOS 27.2: Fair at 123 s Recording, HEVC 10-bit: iOS 27.0: Fair at 38 s, Serious at 108 s iOS 27.2: Fair at 33 s, Serious at 83 s Frame delivery held up. The 5-minute recording lost no frames, even at Serious. Questions: Are there known changes in iOS 27.2 to power use or thermal management during camera capture (AVCaptureSession at 4K) on A15 / A16 devices? Have the thresholds behind ProcessInfo.thermalState or AVCaptureDevice.SystemPressureState changed in 27.2? What data would be most useful in a Feedback report for this: a sysdiagnose taken right after a run, or a specific Instruments template?
Replies
1
Boosts
0
Views
545
Activity
1w
[PhotoKit] Significant Delay when Opening Photo/Video Gallery from Camera View
Experiencing a noticeable delay where opening the photo and video gallery from application's PTX camera interface takes approximately 3 to 4 seconds. This delay degrades the user experience and occurs consistently across both the latest iOS 27 RC and previous iOS 26 builds. There is a roughly 3-4 second delay between tapping the gallery button and the gallery interface actually rendering and becoming responsive. Steps Launch the application. Open the PTX Camera interface. Tap to select/open the Gallery. Observe the loading time.
Replies
0
Boosts
0
Views
51
Activity
1w
Public API for torch beam width (Adaptive True Tone flash)?
On iPhone 14 Pro and later, the system flashlight in Control Center can adjust beam width (narrow ↔ wide). As far as I can tell, AVCaptureDevice only exposes torchMode, torchLevel and setTorchModeOn(level:) I found nothing related to beam width Is there any supported way for a third-party app to control the torch beam width? If not, is this on the roadmap?
Replies
0
Boosts
0
Views
331
Activity
2w
Is activeLockedVideoFrameDuration supported on iPhone 16 Pro? Hardware or iOS limitation?
Hello Apple AVFoundation team, Could you clarify the supported iPhone models and OS versions for standalone locked video frame duration using AVCaptureDeviceInput.activeLockedVideoFrameDuration? The API is documented as available from iOS 26.0, but I understand that OS availability alone does not mean every camera device/configuration supports it. I have read the API documentation and the DTS reply in iPhone 17 Pro's Genlock feature, but could not find a model compatibility list for the standalone locked-duration API. Scope: I am investigating actual capture timing for multi-iPhone motion capture. This question is about locking each camera's frame period using its internal timing, without an external genlock accessory. I am not asking about encoding a constant-frame-rate file, ordinary equal min/max frame durations, timecode metadata, or synchronizing the first exposure across phones. Physical-device test iPhone 16 Pro, iOS 26.2 (23C55). Built with Xcode 26.3 and the iOS 26.2 SDK. One AVCaptureSession, one physical rear .builtInWideAngleCamera input, and one AVCaptureVideoDataOutput. 8-bit SDR 420v formats, tested at 1920 x 1080 and 3840 x 2160, each at 30 and 60 fps. Auto video frame rate disabled; no spatial video capture or multicam session. Manual exposure approximately 2 ms, shorter than half a frame at both tested rates. The session is running and delivering frames. Two separate probe runs produced the same results, including capability readbacks after a settling period. For all four tested configurations: input.isLockedVideoFrameDurationSupported = false device.minSupportedLockedVideoFrameDuration = kCMTimeInvalid input.activeLockedVideoFrameDuration = kCMTimeInvalid device.isVideoFrameDurationLocked = falseOrdinary activeVideoMinFrameDuration and activeVideoMaxFrameDuration can both be set to the requested CMTime(value: 1, timescale: 30) or CMTime(value: 1, timescale: 60), and frames are delivered. This is only a configuration readback, not a claim of exact capture timing in the ordinary mode. The app directly queries the public support properties; it has no model whitelist. It does not attempt to set a valid locked duration when support is false, as the documented setter contract would reject that. These tests cover the selected formats, not every camera/format on this phone. Questions Which iPhone models support this standalone API? In particular, is false expected on iPhone 16 Pro? Is there an official compatibility list covering iPhone 13/14/15/16 and iPhone 17 models? Is the iPhone 16 Pro result a hardware capability restriction, an iOS 26.2 limitation/known issue, or a configuration requirement? Is there a released iOS version that enables this on iPhone 16 Pro, or is newer hardware required? What is the minimum supported capture configuration and correct time to query support? Are there restrictions involving the physical versus virtual camera input, pixel format, SDR/HDR, or exposure mode that could explain this result? If so, which configuration should be tested before concluding that the device does not support it? On supported hardware, can this standalone lock be enabled without an external accessory or a special entitlement, independently of follow(_:videoFrameDuration:delegate:)? Relevant documentation: activeLockedVideoFrameDuration isLockedVideoFrameDurationSupported minSupportedLockedVideoFrameDuration I can provide a focused reproducer if needed. Thank you for clarifying the device requirements and whether an OS upgrade can change the result.
Replies
1
Boosts
0
Views
646
Activity
2w
iPhone 17 Pro: Is Native Apple Log 2 ISP Processing Available for 4224×3024/60fps RAW?
Hi everyone, I'm developing an iOS camera app that aims to record 4224×3024 Open Gate video at 60fps in 10-bit HEVC with Apple Log 2. Native Apple ISP-processed Apple Log 2 capture works at 3840×2160/60fps. However, at 4224×3024/60fps, the tested capture configuration only provides Bayer RAW (btp2). I've successfully decoded ProRes RAW using Apple's decoder into linear Rec.2020 and applied the published Apple Gamut and Apple Log 2 transforms. However, the resulting chroma is only approximately 72–74% of the native ISP output in controlled color-chart comparisons. I've also confirmed that VTRAWProcessingSession and related RAW processing controls are not publicly available to iOS apps in the iPhoneOS 27.0 SDK. My questions are: Is there a public iOS API that applies the same color and tone rendering as Apple's native camera ISP to captured Bayer RAW or ProRes RAW? Is there a supported configuration for obtaining full-resolution 4224×3024/60fps ISP-processed Apple Log 2 output (x420 or x422), without preview proxies or upscaling? If neither is supported, what is Apple's recommended approach? Is this a hardware limitation or a public API limitation? The goal is to use Apple's official color processing pipeline without empirically fitted matrices, LUTs, or arbitrary color corrections. Any guidance would be greatly appreciated. Thank you!
Replies
0
Boosts
0
Views
347
Activity
3w
iPhone Duo: continuous front-camera depth for measuring face distance
Hi, I’m building an app that counts push-ups with the phone on the floor. It currently uses ARKit and TrueDepth to measure the distance between the camera and the user’s face. I want to support iPhone Duo too, but I don’t have one. I only have an iPhone 17 Pro. I watched the camera presentation and saw that depth is supported when accessing the individual cameras. I’m trying to understand what that means for my app. Can either front camera provide continuous depth through AVCaptureDepthDataOutput, or is it only available when taking photos? Does it give actual distances in metres, or relative depth estimates? The face would usually be around 20 to 80 cm from the camera. Also, is there an equivalent camera mode on my iPhone 17 Pro that I could use to test the same approach, or would I need an actual Duo? It’s okay if counting is less accurate than with TrueDepth. I mainly need to detect the user moving down and back up reliably. Thank you!
Replies
1
Boosts
0
Views
346
Activity
3w
Radiometric interpretation of Apple ProRAW and Bayer RAW access via AVFoundation
I am working on a computational photography research project involving multi-exposure HDR reconstruction using Bayer RAW and Apple ProRAW captures. I would like to clarify the radiometric interpretation of Apple ProRAW and the availability of Bayer RAW capture through AVFoundation. My questions are: On current iPhone Pro devices, is it possible for third-party apps to capture and export true Bayer-pattern RAW DNG files through AVFoundation, rather than Apple ProRAW linear DNG files? If so, which availableRawPhotoPixelFormatTypes correspond to Bayer RAW, and what device or format restrictions apply? Apple ProRAW appears to be demosaiced and computationally processed, and may include multi-frame fusion. Is the decoded ProRAW image intended to be radiometrically linear and scene-referred? For a bracketed ProRAW sequence captured with fixed ISO, white balance, lens, and focus, but different exposure times, can one assume that the decoded linear pixel values Y_i(p) satisfy an exposure-proportional model in non-saturated regions, such as Y_i(p) ≈ t_i R(p), across brackets? This question is about radiometric consistency for algorithmic use, not about visual editing or tone mapping. Thank you for your help.
Replies
2
Boosts
0
Views
2.1k
Activity
3w
Radiometric interpretation of Apple ProRAW and Bayer RAW access via AVFoundation
I am working on a computational photography research project involving multi-exposure HDR reconstruction using Bayer RAW and Apple ProRAW captures. I would like to clarify the radiometric interpretation of Apple ProRAW and the availability of Bayer RAW capture through AVFoundation. My questions are: 1.On current iPhone Pro devices, is it possible for third-party apps to capture and export true Bayer-pattern RAW DNG files through AVFoundation, rather than Apple ProRAW linear DNG files? If so, which availableRawPhotoPixelFormatTypes correspond to Bayer RAW, and what device or format restrictions apply? 2.Apple ProRAW appears to be demosaiced and computationally processed, and may include multi-frame fusion. Is the decoded ProRAW image intended to be radiometrically linear and scene-referred? 3.For a bracketed ProRAW sequence captured with fixed ISO, white balance, lens, and focus, but different exposure times, can one assume that the decoded linear pixel values Y_i(p) satisfy an exposure-proportional model in non-saturated regions, such as Y_i(p) ≈ t_i R(p), across brackets? This question is about radiometric consistency for algorithmic use, not about visual editing or tone mapping. Thank you for your help.
Replies
2
Boosts
0
Views
1.7k
Activity
3w
AVFoundation: full-resolution Quad Bayer RAW capture path for third-party computational photography? (FB25352517)
At the WWDC26 Camera and Photo Technologies Group Lab (session 8018, ~29:55 to 32:46), the panel explained why ProRAW can produce 48 MP output from Quad Bayer sensors while native Bayer RAW is limited to the binned resolution. ProRAW is already demosaiced and linearized, and may incorporate Photonic Engine multi-frame processing. The native RAW path exposes the binned Bayer result. A true full-resolution RAW output would be Quad Bayer, not conventional Bayer, and supporting it properly needs a decoding path around it. The panel suggested filing Feedback Assistant requests if developers need this, so I filed FB25352517. My use case is computational photography that wants minimally processed sensor-domain samples on purpose — its own multi-frame alignment, denoising, HDR fusion, motion rejection, and super-resolution. Adobe Project Indigo is a public example of that kind of pipeline. It captures RAW bursts and uses measured sub-pixel variation between frames to reconstruct detail. Processed 48 MP photos and ProRAW don't cover that. They've already been through Apple's reconstruction or demosaic. Is filing enhancement requests still the intended path for a supported full-resolution Quad Bayer RAW or mosaic capture path? If so, what would make these reports more useful — mosaic metadata, DNG representation, CIRAWFilter support, AVCaptureDevice.Format capability discovery, something else? WWDC26 session: https://developer.apple.com/videos/play/wwdc2026/8018/ Project Indigo technical overview: https://research.adobe.com/articles/indigo/indigo.html
Replies
2
Boosts
0
Views
521
Activity
3w
Retain warning on cloudIdentifierMappings(forLocalIdentifiers: [String] )
Whenever I call this method on the PHPhotoLibrary class I get a warning which I've included below. I've tried looking in the Apple forums, Stack Overflow etc. and nobody seems to mention this issue that I can find. Is this warning unavoidable or am I using the API wrong? I can reproduce this trivially in a simple untitled project. Warning: Retain cycle risk: client-side managed object fetch on long-lived context for entity Asset, on context named: PHPhotoLibrary (inode=19657147) .Generic, consider using -[PHPhotoLibrary performAndWaitOnAutoResettingLibraryWithName:workBlock:] or +[PLDatabaseContext newShortLivedLibraryWithName:bundle:]...work, or using a dictionary result type, or suppressing this warning using 'clientSideLongLivedFetchWarningsSuppressed'). STEPS TO REPRODUCE Create a new untitled App project in Xcode. Set a photo library usage privacy description in the Info for the project target Replace ContentView.swift with the code included below. Make sure there is at least one photo in the photo library on the target device. A simulator is fine. Run the app Click the Get Cloud ID button, respond to the authorization dialog Click the Get Cloud ID button again Look in Xcode and see the warning import SwiftUI import Playgrounds import Photos struct ContentView: View { var body: some View { Button( action: getCloudID ) { Text("Get Cloud ID") } .padding() } private func getCloudID() { var fetchOptions = PHFetchOptions() fetchOptions.fetchLimit = 1 let assetResult = PHAsset.fetchAssets(with: .image, options: fetchOptions) var localIds = [String]() assetResult.enumerateObjects() { asset, _, _ in localIds.append(asset.localIdentifier) } let mappings = PHPhotoLibrary.shared().cloudIdentifierMappings(forLocalIdentifiers: localIds ) for mapping in mappings { debugPrint("cloud ID \(mapping.value)") } } } `
Replies
0
Boosts
0
Views
69
Activity
3w
Supported approach for live camera PiP during ReplayKit screen recording in a non-calling iPhone app?
For an iPhone recording utility with no calling functionality, what is the supported way to keep the front camera live in Picture in Picture while a ReplayKit broadcast records another foreground app? Is using AVPictureInPictureVideoCallViewController with the voip background mode supported for this purpose?
Replies
0
Boosts
0
Views
168
Activity
3w
FaceGroupAnalyzer: one unreadable asset makes `insertOrUpdateAssets` deliver zero elements, discarding results already computed for the valid assets in the batch
If a single asset in the array passed to insertOrUpdateAssets cannot be read, the returned AsyncSequence throws and delivers zero elements — including for the valid assets that appear before the offending one in the array. In my measurement a batch of 5 — two valid photographs, one zero-byte .jpg, then two more valid photographs — delivered 0 of 5 elements and reported 0 faces. The framework's own stdout log shows it had already processed the valid photographs before failing, so the work was done and then thrown away. This is the behaviour I would expect from a function returning [Result] after processing everything, not from a streaming AsyncSequence. The whole reason to expose an async sequence is to deliver results as they are produced; here the sequence produces nothing at all, which makes the streaming shape actively misleading. Why this matters at library scale. I am cataloguing 97 photo libraries, many of them archives of scanned family photographs going back to the 1970s, where a handful of damaged files is normal rather than exceptional. As the API stands, the batch size I choose for throughput is also the amount of work a single corrupt file destroys — with a batch of 100, one bad file costs 100 images. The only safe strategy is to catch the failure and re-submit the batch one asset at a time to find the culprit, which turns a rare bad file into a full re-run of that batch and makes the worst case quadratic in the number of bad files. Note also that the error gives no indication of which asset failed (see the related enhancement request on error taxonomy), so isolating the culprit by re-submission is the only option available. Feedback: FB24174733
Replies
1
Boosts
0
Views
550
Activity
Oct ’26
FaceGroupAnalyzer: a truncated image file reports zero faces with no error, indistinguishable from a photograph that contains no people
An image file whose data is cut in half is accepted by insertOrUpdateAssets, produces no error, and is reported as an image containing zero faces. That result is indistinguishable from a photograph that genuinely has no people in it. In my measurement the intact file reported 4 faces and a copy truncated to half its bytes reported 0 faces — both without throwing. Why this is worse than an error. A client that receives "zero faces, no error" will record the asset as analysed, no people present. That is a permanent, silent, incorrect result: the file is never revisited, and the people in it are lost from the catalogue with no diagnostic anywhere. An error would have been recorded as a failure and retried later. Silence is the one outcome that cannot be recovered from. The failure mode is also inconsistent. Other kinds of damaged input do throw — a zero-byte file, a text file with a .jpg extension, and a file whose interior bytes have been destroyed all raise MediaIntelligenceError.faceGroupProcessing. Truncation is the case that silently succeeds, and truncation is precisely the damage that partial downloads, interrupted copies and failing disks produce — the most common form of corruption in a real photo archive, and the one I have to handle in 97 libraries of mixed provenance. My current workaround is to fully decode every image myself before handing it to the framework, purely to detect truncation. That means every file is decoded twice, which roughly doubles the I/O of the analysis pass. Feedback: FB24174749
Replies
1
Boosts
0
Views
568
Activity
Oct ’26
iPhone Duo: Does CameraCaptureAccessory accept touch, what counts as an active camera session, and can both front cameras run at once?
After watching "Leverage multiple displays and scenes on iPhone Duo" and "Build a great camera experience for iPhone Duo", we have four questions about the outer display while the device is open: Does a view hosted in CameraCaptureAccessory receive touch input on the outer display, or is it display-only? The accessory is described as available when the app "has an active camera session." Does that require a running preview layer or an attached output (photo, movie or data), or is a running AVCaptureSession with an input enough? Is a session whose main purpose is to host the accessory a supported use? Can AVCaptureMultiCamSession run the inner (under-display) front camera and the outer front camera at the same time on iPhone Duo? If so, which device-type combinations are supported, and are there thermal or power limits? Is CameraCaptureAccessory the only way for an app to show content on the outer display while the inner display is live, or are scene accessories that don't depend on the camera planned? Our use case is not photography. We're exploring a companion view for a second person sitting across from the device owner, so we want to understand whether the camera accessory is the right tool before designing around it.
Replies
5
Boosts
1
Views
401
Activity
Oct ’26
Why does Apple prefer Quad Bayer over traditional Bayer sensors?
Hi everyone, I've been reading about iPhone camera sensors recently, and there's something I've been wondering about. Apple has explained the benefits of Quad Bayer, especially pixel binning for low-light photography and high-resolution capture. But why not use a traditional Bayer sensor with Dual Pixel PDAF instead? As far as I understand, a native Bayer arrangement could potentially preserve more fine detail without requiring remosaicing. And if both sensors have the same size and pixel count, couldn't the Bayer sensor also produce a cleaner 12MP image by downsampling? I'm also curious about the microlens design. Some sensors use one microlens shared by four pixels for phase detection. Wouldn't individual microlenses potentially offer better spatial resolution? So my main question is: What hardware-level advantages make Quad Bayer worth choosing over traditional Bayer, despite the additional image reconstruction involved? Is it mainly about read noise, analog binning, power efficiency, autofocus, or something else? I'm not a camera engineer, just someone interested in understanding the technology behind iPhone cameras. I'd really appreciate any technical insight. Thanks!
Replies
0
Boosts
0
Views
147
Activity
Oct ’26
Ultra Wide camera: frame-to-frame jitter below 2.0x and sharpness step at 2.0x during native zoom ramp (iPhone 17 Pro, 4K/60)
I am developing CineZoom, part of CameraControl, a remote camera-direction system (macOS app controlling an iPhone capture engine). Its core requirement is a smooth, repeatable cinematic zoom. On iPhone 17 Pro (iOS 26.5.1), using the physical Back Ultra Wide camera at 3840x2160, 60 fps, a single native ramp(toVideoZoomFactor:withRate:) from 1.7x to 2.1x over 5 seconds shows a visible stutter. I isolated it with a minimal diagnostic app (no networking) and analysed four tripod recordings frame by frame, in two locations with different lighting. The results are consistent across all four: Jitter below 2.0x: while the zoom factor is below 2.0, the image centre shifts about 2.8 px RMS between consecutive frames (peaks ~5 px), alternating almost every frame (26-28 Hz). Above 2.0x, during the same ramp at the same speed, it drops to 1.2-1.5 px. Sharpness step at 2.0x: centre-crop sharpness drops by 23-31% within a few frames exactly when the zoom factor crosses 2.0. There is no consistent effect at the reported secondaryNativeResolutionZoomFactors value (1.7838). The zoom command path itself is correct: the zoom measured from the image matches videoZoomFactor throughout, with no dropped or duplicated frames. Questions: Is this expected behaviour for the Ultra Wide camera in this zoom range? Is the jitter related to video stabilization or to how the sensor crop is positioned in this readout mode? Is there a public API that reports or controls the zoom factor at which this transition happens, so an app can plan around it? What configuration do you recommend for a smooth continuous video zoom across 1.7x-2.1x on this camera (format, stabilization mode, constituent-camera switching)? Are there existing APIs that would let an app detect or compensate this (per-frame stabilization/crop metadata, consistent processing during zoom, or a reduced-processing video stream)? I have a full technical report, per-frame measurements, charts and the original recordings, and I am happy to share them. Thank you!
Replies
1
Boosts
0
Views
723
Activity
Oct ’26