I want to: Run ARKit on the main rear camera, and while it's running shoot high resolution pictures on the wide camera, without disturbing the AR tracking. Is this possible?

I’ve seen, mainly in discussions with AIs, that ARFaceTrackingConfiguration uses the same technology as Face ID and therefore should work in complete darkness. However, I haven’t been able to achieve this. Does anyone know if this is actually true? I'm using an iPhone 16 to test, and the Face ID works well in darkness.

As the title says. I am developing an app for IOS 18, which involves adding a building relatively far away from the user's place (around 10-20m.), and I need to building to show contact shadows with the floor for added realism. Now, I know that RealityKit has automatic contact shadows using GroundingShadowComponent, but this only works if the object is placed in a plane that has been detected by ARKit... and ARKIt doesn't detect floors so far away, so I need to add my own shadows: add an invisible plane acting as a floor below my building, and have it receive shadows and show only the shadows. The problem is that, from what I see, RealityKit has no materials that can do this: UnlitMaterial ignores all lighting, including shadows. SimpleMaterial does display the shadows, but the floor is displayed too. I can make the material almost transparent (setting the opacity to 0.01 or something), but the floor is still visible. OcclusionMaterial with receiveDynamicLighting is a solution, but it doesn't work either. If I declare it using OcclusionMaterial(receivesDynamicLighting: true), all I get is an invisible plane with no contact shadows in it. What do other people do for this? Do people just bake in the contact shadows in the 3D model?

Is there a supported route to automated regression testing of ARKit apps? Reality Composer's "Record AR Session" plus Xcode's "ARKit Replay data" scheme option work well for manual debugging, but replay isn't wired into XCUITest, doesn't run in CI, and ARKit doesn't exist in the simulator — so every AR regression today needs to be run by a human holding a device. Even replay-driven ARSession in the simulator, or an XCTest hook for selecting replay files, would unlock some automated coverage.

Currently VisionOS has WorldAnchors which are a great, privacy-preserving method to affix and localize entities into physical spaces. These are great for apps that work in Mixed immersive mode as well as augmented layer or virtual overlays onto physical spaces. However, they have some fundamental limitations when compared to iOS’s ARWorldMap which limit their capabilities as truly persistent WorldAnchors, which the platform would greatly benefit from. The problem - Impermanence: WorldAnchors are on-device only, stored in data at a system level and expose their ID for collection and usa within apps. If the app which created them is removed, or if the device is rese, those saved World Anchors are removed and lost forever. Their ID will no longer point to an anchor in physical space and it is as if the anchor has been deleted or never existed. This means that anchors effectively only temporarily exist on a single device while that specific device is in an undisturbed state. A second device using that same app with interest in utilizing that anchor is not available. With no way to export this Anchor, save it, re-load it either inside an app bundle, from a server, on-site data storage, etc. the anchor is 'trapped' on the device which creates it. ARWorldMap can be saved to a file, and acquired in many ways which circumvents this problem, and allows it to exist as a single source of truth for that particular physical environment that any iOS or iPadOS device can localize to as long as they have the ARWorldMap. Imperfect Workaround #1 - SharedAnchors: After setting a WorldAnchor on Device #1, start a local S harePlay session with Device #1 & #2 in the same physical space. Then, use a 'Shared Anchor' with transform offset data matching the WorldAnchor. On Device #2, save that offset as a new WorldAnchor. This new WorldAnchor will have the same transform effectively matching the original, though it will have a new ID and is entirely separate. Problem with Workaround #1: This requires two VisionOS devices to be in the same physical location simultaneously, have an active SharePlay session between the two devices, then conduct a sharing operation. This means that if Device #1 does not happen to be available in the physical space at the same time as Device #2 for any reason, Device #2 will never have access to this anchor. In cases such as a public or industrial space, it is not realistically viable to always have a person with Device #1 available at all times. There are many situations where Device #2 would want to enter the space and observe the anchor while they are alone, and would never have access to this anchor. Workaround #2 - iOS / iPadOS Middleman: Once a visionOS WorldAnchor is created on Device #1, that same user can manually co-locate an iOS or iPadOS device, with some shared session (for example, using ImageTrigger on the iOS/iPadOS device's display which the visionOS device reads, then determines an offset for). Then, create an ARWorldMap on that co-located iOS/iPadOS device, and save it, then serve that ARWorldMap. Then to localize visionOS Device #2, enter the space and first localize an iOS / iPadOS device with the ARWorldMap. Then, manually co-locate visionOS Device #2 with the iOS/iPadOS device and access the offset. Then save this offset as WorldAnchor on Device #2, at this point the iOS/iPadOS device is no longer needed. Problems With Workaround #2: At a minimum, this requires 3 devices, or more realistically 4 if Device #1 and #2 are not in the space at the same time. This is not only a very poor user experience to have to operate an iOS/iPadOS device simultaneously while wearing visionOS device, but it is also very complicated and a several step process that is not intuitive to most users. The accuracy of this process is extremely poor, as tracking an image from the screen of an iOS/iPadOS device will never be as precise as the internal tracking system on VisionOS. It will always have SOME margin for drift and error, which can result in the anchor being very far offset from the intended anchor position. This is antithetical to the purpose of WorldAnchors, as the delta can be so inconsistent that Device #2 will observe content attached to the anchor at an incorrect location, which could lead to unintended user behaviour when interacting with this content potentially moving attached content to look correct for them, which would negatively impact all other devices viewing the content. Desired Solution: An optimal upgrade to visionOS WorldAnchors would; Use the same underlying tracking & relocalization system that is currently in use Allow for exporting of the WorldAnchor's data in an encrypted, privacy preserving way (not simply a point cloud) which could be saved and shared as a file similar to ARWorldMap, then used for relocalization on ANY device with access to that file and application. Be interoperable across platforms where iOS, iPadOS, visionOS and any other spatial-capable platform can use that single file, and localize themselves in the physical space. This would unify ARKit WorldAnchors for all platforms, ensuring that the same physical space can be localized to by all devices, anchors can be created by all devices, and the content would exist in the same position on all devices. Allowing an iOS or iPadOS device to create a WorldAnchor that is then identical on visionOS, or visa-versa. These features unlock true persistence in anchoring content to real-world spaces, a critical component of a Spatial Computing platform that maximizes the unique capabilities and benefits of the medium. This creates an upwards path for users to start on iOS / iPadOS today, and upgrade to visionOS in the future. Accessibility: Not all users are able to use visionOS for various reasons, including physical, regional and financial circumstances among many many others. There is no reason why someone who is unable to use visionOS for any one of these reasons should be LOCKED-OUT from Spatial Computing applications. Spatial computing is the most human computing medium ever created, and applications need to allow all humans to engage with Spatial Computing experiences regardless of their level of access to visionOS devices. We as developers want to build everlasting Spatial Computing applications that accentuate the medium, maximize the benefits of Spatial Computing, include / invite all humans regardless of their Accessibility level, and establish virtual content that can outlive the individuals who create it. Please, take this request into serious consideration, as the feature as described would contribute to the Apple Ecosystem being the single greatest Spatial Computing platform of all time (and space), enabling permanent layering of physical spaces, preserving privacy for sensitive data, and maximizing accessibility across the spectrum of humans and devices. Thank you.

This is a very general and high level question. I am working on an app that scans buildings using on device LiDAR. One of my biggest issues is drift over time. What is the recommended approach by Apple to successfully scan for a long period of time and reduce drift error? I am currently storing frames at 20 fps with pose data and then trying to correct for pose errors off device. Maybe my approach is incorrect and would like to understand how Apple recommends to do about this. I am being overly general on purpose. Thank you for your help.

We are building an ARKit application targeting iPhone Pro devices on iOS 26.4+. During testing, we consistently observe the following log message at a specific point in our AR session (approximately 90° into a walking arc around a vehicle): ARWorldTrackingTechnique: resource constraints [33] What specifically does resource constraint code [33] represent? Is it a VIO constraint type, a sensor fusion budget limit, or something else entirely? How does it differ from code [1] (which appears to be a general CPU starvation signal for the ARKit sensor fusion thread)? Is there a public API to detect or respond to this specific constraint condition, short of waiting for a full ARCamera.TrackingState downgrade? Was this constraint type introduced or changed in iOS 26.4?

I'm trying to track identified objects in realtime with bounding rects, with no 3D integration, but still has poor update performance. I'm trying to understand how to optimize frame updates. (I'm a new programmer) Using: Foundation, AVFoundation, ARKit, CoreVideo, Vision, CoreImage, CoreVideo with YOLOE-11s object detection currently throttled to 2fps. (target iOS, testing on 16pro) YOLOE-11S CoreML model detects objects with class labels + bounding boxes Labels are matched against ObjectCatalog.json for relevance Matched objects are promoted from blue (detected) to green (identified) Log warnings: ARSession <0x110afdb80>: The delegate of ARSession is retaining 13 ARFrames. The camera will stop delivering camera images if the delegate keeps holding on to too many ARFrames. This could be a threading or memory management issue in the delegate and should be fixed. Skipping integration due to poor slam at time: 619447.208339 vio_initialized(1) map_size(0) tracking_state_is_nominal(0) is_3dof(0) reinitialize_attempts(6) slam_mode(RegularSLAM)

Film crews record "takes" in my app: camera feed plus virtual AR content with timecode. On SceneKit I rely on a third-party Metal-layer hook (SCNRecorder), because ReplayKit captures overlay UI and adds a permission prompt — and RealityKit appears to have no capture API at all, making recording a regression in any SceneKit→RealityKit migration. Is there a recommended first-party way to record an AR view to a movie file, or should I file feedback?

My users import their own 3D assets at runtime — OBJ, FBX, DAE, STL, PLY, often with skeletal animation — which we convert in-app to SceneKit scenes via Assimp. RealityKit only loads USD/.reality at runtime. Is the recommended path to hand-build MeshResource, skeletons, and animations from parsed geometry, or is there any supported on-device conversion to USD? This is one of my biggest blockers to leaving SceneKit.

My app is an 8-year-old film-previsualization tool built on ARSCNView, with SceneKit rendering over an ARKit world-tracking session. Given SceneKit's deprecation with critical-bug-only maintenance: does that commitment, and the "ample notice" promise, explicitly cover ARSCNView? Should I plan on existing SceneKit apps continuing to work on new hardware and OS releases for several more years? I have a few blockers stopping me from migrating to RealityKit and I've noticed the lack of big updates to ARKit over the recent years.

On iOS I colocate multiple iPhones/iPads by sharing ARWorldMap over MultipeerConnectivity. visionOS 26 added the conceptual equivalent (shared coordinate spaces via SharedCoordinateSpaceProvider) but as far as I can tell it's Vision Pro-to-Vision Pro only, and visionOS has no ARWorldMap, so there's no bridge. Is there any supported way today to put an iPhone and a Vision Pro into one shared coordinate space or should I file feedback for cross-platform colocation? It would be useful to my users, for example, to have someone controlling an AR scene layout on a Vision Pro while using iOS devices to preview and record the scene from various specific angles.

Is simultaneous ProRes Log 422 capture and ARKit VIO tracking supported on iPhone? If not today, is this a capability Apple is considering for future SDK releases? I’m interested in capturing high-quality ProRes Log footage and, at the same time, using ARKit for real-time camera pose tracking for synchronization and post-processing workflows.

My app previews cinema lenses by cropping the AR feed, so I can only simulate lenses tighter than the main camera's ~24mm-equivalent, meaning wider cinema lenses are impossible to accurately preview. ARConfiguration.VideoFormat exposes captureDeviceType, but world-tracking formats only ever list the wide-angle module. Is running world tracking on (or alongside) the ultra-wide camera fundamentally infeasible? Or just a case of not enough Feedback Assistant requests?

Dear Apple Developer Team, I would like to ask a few questions related to RoomPlan, Room API, USDZ export, and possible future spatial scanning workflows: Can the RoomPlan or Room API load an existing USDZ room model and compare it in real time with a new live scan of the same space? Is Apple considering extending RoomPlan beyond indoor rooms, for example to scan outdoor areas, house exteriors, terrain, and simple building volumes with rough dimensions? Will RoomPlan support multi-floor continuous scanning, custom object detection elements, and reliable export of the scanned result to USDZ for further CAD/BIM workflows? Is it possible to create a real-time comparator between a reference object in USDZ or STEP format and a physical object being scanned live, so that deviations in geometry or dimensions can be detected during scanning? Best regards, Ivo Saina

Hello, I’m developing an Apple Vision Pro app designed to help people with mobility and speech challenges communicate more effectively with their caregivers. The app requires camera access in order to function as intended. I previously submitted a request for the required camera access entitlement by email and received an initial response indicating that the request would be reviewed with a colleague, but I have not received any follow-up since then. My Apple Developer account is registered under an LLC, and I currently have other visionOS apps available on the App Store. Additional documentation for this app is available here: https://revivrstudios.com/stareandshare.html My company is focused on developing Apple Vision Pro apps that improve quality of life for people living with mobility and speech challenges. I believe this app has the potential to provide meaningful support for those users and their caregivers. Could someone advise on the best way to follow up on the camera access request, or confirm whether there is an additional process I should complete? Thank you.

Hi, We're building a mixed-immersive visionOS app in which the user holds a thin metal tool — a needle — near virtual content. We need the real needle to correctly occlude virtual content: when the needle (and especially its tip) is physically in front of a virtual object, the virtual object must not draw over it. We are solution-agnostic. Whether the right answer is LiDAR / scene-depth based, object-tracking based, hand-tracking based, or something else entirely — we just need needle occlusion to work. Here is what we've ruled out so far: ObjectTrackingProvider — blocked at enrolment. We can't even produce a usable reference object: the object-capture / scanning step doesn't pick up the needle at all (≈1 mm shaft, specular/metallic, almost no surface features). So object tracking never gets off the ground — it fails before runtime. SceneReconstructionProvider + OcclusionMaterial — only usable for large planar surfaces. Occlusion against a wall or floor is clean. But the mesh is too coarse for anything with finer geometry: a chair already gives very rough, ragged occlusion edges, and the needle is far below the mesh resolution — its shaft never appears in the mesh, so it never occludes at all. So this isn't only a needle problem: it's a mesh resolution problem that goes from rough (chair) to nonexistent (needle). Automatic hand occlusion — doesn't cover the tip. The hand is composited in front correctly, but the needle tip protrudes a few centimetres beyond the fingers, and virtual content draws over exactly that part. The needle tip is the part that most needs to occlude correctly, so this is blocking for us. The full Xcode sample project and the screen recording are in this GitHub repo: https://github.com/JerryNee/occlusion-demo (the video is at media/occlusion-demo.mp4). The screenshots below are stills from that recording, which has four stages, all with the same virtual cube: Occlusion off — baseline; the cube draws over everything in the room. Occlusion on, against a wall — the cube is occluded cleanly. Occlusion on, against a chair — the cube is occluded, but with rough, ragged edges. Occlusion on, with a needle — the needle does not occlude the cube at all. The same occlusion path therefore degrades from clean (wall) → rough (chair) → nonexistent (needle) as the real geometry gets finer. Questions / avenues we'd welcome guidance on (any one would unblock us): What is the recommended way to occlude an object this thin on visionOS — and is it possible at all today? Is there app-accessible per-pixel scene depth (analogous to ARFrame.sceneDepth on iOS) that we could use to occlude geometry the reconstruction mesh misses? Can the system's automatic hand / upper-limb occlusion be extended to include a thin instrument held in the hand — i.e. treat a held tool as part of the hand region? The tool has known, rigid geometry, so we considered attaching an OcclusionMaterial proxy mesh and positioning it from hand-tracking joints — but the needle's orientation within the grip is not determined by the hand pose (the same grip can hold the needle at many different angles), so a hand-joint proxy would be misaligned. Is there any supported way to recover the actual 6-DoF pose of a thin tool held in the hand (i.e. something that senses the tool itself, since the hand pose doesn't constrain it)? Is there any way to make object capture / scanning succeed for thin, specular objects, or an alternative enrolment path? Environment: Apple Vision Pro, visionOS 26.2. Full Xcode sample + screen recording: https://github.com/JerryNee/occlusion-demo Even a "not currently possible / known limitation" answer would help us plan our roadmap. Thank you!

Hi all: Novice iOS programmer here. I am developing an iPhone app (target: iOS 18) where I need to place a large building a bit far away from the user: around 10-20 meters away. Now, the problem I am having is that ARKit's floor detection usually doesn't reach that far. I solved it by using "existingPlaneInfinite" in the raycast: let raycastResults = arView.session.raycast( ARRaycastQuery(origin: rayOrigin, direction: rayDirection, allowing: .existingPlaneInfinite, alignment: .horizontal) But the problem is that the building appears... but without grounding shadows. I tried enabling GroundingShadowComponent, but without success. After some more testing, I found out that if I place an object in a plane that ARKit is actually detecting, then it will have grounding shadows... but if I place it further away, in the part of the floor that ARKit doesn't detect, then it won't add shadows. It will still place the building at the right height (because it extends the detected plane to the infinite), but it won't add contact shadows with the floor. I can solve this problem using the usual tricks (adding an invisible plane as a shadow catcher, etc.), but I just wanted some confirmation that this is indeed the correct behaviour. Also, are there any changes in this regard in IOS 26?

My cousin implanted an itch I'd like to scratch. She does some academic psycho-neurological research that I don't fully understand. But in talking it sounds like mixed reality in the AVP might facilitate a type of research that she's pursuing. I just don't know if such a thing is currently possible with the AVP given how it treats physically moving frames of reference. The idea is to have subjects in a motion vehicle simulator rig, able to operate it and see the interior of the vehicle/gauges/etc. But at times and in specific ways, obscure the views out of the "windows" of the vehicle. The obvious problem here is that the AVP doesn't like being in a dynamically moving frame of reference. This seems to be beyond what "travel mode" is intended for. They need to bump and turn the subjects to get the responses they're studying, so not a smooth ride like on a plane or train. The interior of the simulator rig is a "known object" and can be modeled. I just watched the new video about training/tracking a hand-held object via Create ML. Could a similar approach from a hand-held object be applied to the user's surroundings - the mocked up vehicle interior and its window frames? We'd then apply an obscuring blur or even just an opaque polygon to that "window" while the simulator rig (and thus the user) is in motion? The alignment of the blur/polygon doesn't need to have perfect tracking and registration. Update rate does not need to be low millisecond, but full second updates might make motion sickness worse. Also, am I correct in inferring that object tracking should be free from major drift over the course of tens of minutes? A stretch: would it be possible to counteract the illumination from room light sources so that the movement of light/dark is reduced on the interior of the "vehicle" while it is moving? I also noticed in the new "Explore enhancements to visionOS object tracking" video that some of the effect of overlaying MR elements onto something that is visually passed through is being demonstrated on an iPhone. Could a proof of concept of this app(?) be "mocked up" using an iPhone as a basis to justify the expense of buying the AVP? Hold the phone while the rig is moving, track the interior and blur/"open" the windows as seen on the phone screen?

With the recent announcement that VisionOS 27 would support custom actively tracked tools, via arrays of embedded LEDs, I would just like to confirm if the demo given in this video here: https://developer.apple.com/videos/play/wwdc2026/283/ at minute 1:00 is indeed an actively tracked tool, or if it is passive IR. if it's passive IR (reflectance), are there any examples of how this could be achieved, i.e. can we access the onboard IR cameras directly?