I am a private individual working on a personal panorama viewing and reprojection project. I have been investigating the geometry of panoramas produced by the iPhone Camera app, with extensive analytical and research assistance from OpenAI's ChatGPT.
My goal is to determine the physical viewing direction (longitude and latitude, or equivalently a 3-D viewing ray) corresponding to each pixel in a finished iPhone panorama.
I have been able to recover much of this geometry from information contained in the panorama itself. Controlled tests on several iPhone models indicate that I can accurately determine the effective focal length of the perspective source camera, camera rotation during capture, endpoint geometry, horizontal field of view, and gravity-relative camera orientation.
The remaining problem is the absolute vertical/latitude mapping in the finished panorama.
The gravity/orientation information allows me to determine where a known physical latitude—for example, the true horizontal plane (0° latitude)—falls in the individual source-camera frames. However, I have not found a documented way to determine exactly where those source-frame viewing rays are placed vertically in the final panorama bitmap.
My investigation suggests that the Camera app performs additional geometric processing during panorama assembly, including geometric/perspective correction and the selection, cropping and/or rotation of image slices before they are blended into the panorama. What I have not been able to determine is whether the information needed to reconstruct that source-ray-to-finished-pixel mapping is retained anywhere in the finished panorama or is available through a supported Apple API.
My primary question is therefore:
Is there a documented or supported way to determine the physical viewing ray corresponding to each pixel in a panorama produced by the iPhone Camera app, including its absolute vertical/latitude mapping?
More specifically:
-
Is the projection used for the finished Camera panorama documented?
-
Is the gravity-relative orientation or vertical reference of that projection available through supported metadata or an API?
-
Is sufficient information retained in the panorama to reconstruct the geometric correction and placement applied to source images/slices during panorama assembly?
-
If that information is not retained in the finished image, is there another supported framework or API that provides it during or after panorama capture?
I am not trying to reproduce Apple's image blending or other proprietary image-processing algorithms. I simply want to interpret the geometry of the finished image accurately enough to reproject it—for example, to cylindrical or equirectangular coordinates—and assign physical longitude and latitude to its pixels.
I have controlled test panoramas from multiple iPhone generations and considerably more technical detail about the investigation. I would be happy to provide example files, measurements, or additional information if that would be helpful.
Even confirmation that the required vertical-placement information is not retained or exposed would be very useful.