Integrate iOS device camera and motion features to produce augmented reality experiences in your app or game using ARKit.

Posts under ARKit tag

200 Posts

Post

Replies

Boosts

Views

Activity

iPad Compass Calibration Issues: Impact of Magnetic Cover
We have developed an iPad application using the ARCL (AR + CoreLocation) library to render Point of Interest (POI) annotations in both an AR view and a standard MapView. The application performs as expected on standard devices. However, we have some iPad covered with strong magnet. This creates significant magnetic interference, resulting in a 90° to 180° heading offset, rendering the AR POI placement and MapView orientation unusable. Technical Challenges & Constraints: Hardware Lock: The magnetic cover is a mandatory business requirement and cannot be removed during field use. Sensor Failure: The internal magnetometer cannot provide an accurate North reference due to the proximity of the cover’s magnets. While CoreLocation and CoreMotion use sensor fusion, the magnetometer remains the primary source for absolute heading. Alternative Orientation Tracking: Is there a documented method to bypass the magnetometer and derive device orientation using only the Gyroscope and Accelerometer (e.g., relative tracking) while still maintaining alignment with geographic coordinates in CoreLocation? Programmatic Offsets: Are there known APIs or mathematical workarounds to programmatically "nullify" or offset a constant magnetic bias once the device is inside the cover? so we can use that offset for ARView and in Mapview as well.
2
0
1.4k
Jun ’26
ARKit Face Tracking works in total darkness?
I’ve seen, mainly in discussions with AIs, that ARFaceTrackingConfiguration uses the same technology as Face ID and therefore should work in complete darkness. However, I haven’t been able to achieve this. Does anyone know if this is actually true? I'm using an iPhone 16 to test, and the Face ID works well in darkness.
1
1
705
Jun ’26
What does `ARWorldTrackingTechnique: resource constraints [33]` mean?
We are building an ARKit application targeting iPhone Pro devices on iOS 26.4+. During testing, we consistently observe the following log message at a specific point in our AR session (approximately 90° into a walking arc around a vehicle): ARWorldTrackingTechnique: resource constraints [33] What specifically does resource constraint code [33] represent? Is it a VIO constraint type, a sensor fusion budget limit, or something else entirely? How does it differ from code [1] (which appears to be a general CPU starvation signal for the ARKit sensor fusion thread)? Is there a public API to detect or respond to this specific constraint condition, short of waiting for a full ARCamera.TrackingState downgrade? Was this constraint type introduced or changed in iOS 26.4?
1
0
1.5k
Jun ’26
ARKit object tracking performance
I'm trying to track identified objects in realtime with bounding rects, with no 3D integration, but still has poor update performance. I'm trying to understand how to optimize frame updates. (I'm a new programmer) Using: Foundation, AVFoundation, ARKit, CoreVideo, Vision, CoreImage, CoreVideo with YOLOE-11s object detection currently throttled to 2fps. (target iOS, testing on 16pro) YOLOE-11S CoreML model detects objects with class labels + bounding boxes Labels are matched against ObjectCatalog.json for relevance Matched objects are promoted from blue (detected) to green (identified) Log warnings: ARSession <0x110afdb80>: The delegate of ARSession is retaining 13 ARFrames. The camera will stop delivering camera images if the delegate keeps holding on to too many ARFrames. This could be a threading or memory management issue in the delegate and should be fixed. Skipping integration due to poor slam at time: 619447.208339 vio_initialized(1) map_size(0) tracking_state_is_nominal(0) is_3dof(0) reinitialize_attempts(6) slam_mode(RegularSLAM)
2
0
808
Jun ’26
Recommended visionOS architecture for opt-in nearby exchange triggered by a physical gesture
I’m exploring a visionOS interaction pattern where nearby Apple Vision Pro users have opted in ahead of time, such as when entering a shared venue, and a physical gesture like a handshake triggers a lightweight exchange of user-approved information between their devices. I’m not asking about identifying strangers, accessing raw camera data, or tracking another person without consent. I’m trying to understand the most automatic Apple-supported architecture for this kind of privacy-preserving nearby interaction. What is the recommended approach on visionOS? Specifically: Can a visionOS app use ARKit hand tracking to detect the current user’s own gesture, then combine that with nearby peer discovery or ranging through Nearby Interaction, Network framework, Multipeer Connectivity, Bluetooth, Wi-Fi, or another supported API? Is there a supported way for two nearby Vision Pro devices to exchange a small user-approved payload after prior opt-in, without requiring users to manually start SharePlay or confirm every individual exchange? If SharePlay or Group Activities is the recommended path for shared spatial context, is there a supported alternative for venue-scale or multi-user interactions that should not be limited to a small active SharePlay group? What are the privacy and App Review boundaries for this pattern? Should developers assume the app cannot identify nearby people, observe another person’s body or hands, or trigger an exchange unless both users have explicit opt-in and clear awareness? If a mostly passive gesture-triggered exchange is not supported today, what is the closest Apple-recommended design pattern?
2
0
932
Jun ’26
How can we occlude a thin metal needle (held in the hand) with virtual content on visionOS?
Hi, We're building a mixed-immersive visionOS app in which the user holds a thin metal tool — a needle — near virtual content. We need the real needle to correctly occlude virtual content: when the needle (and especially its tip) is physically in front of a virtual object, the virtual object must not draw over it. We are solution-agnostic. Whether the right answer is LiDAR / scene-depth based, object-tracking based, hand-tracking based, or something else entirely — we just need needle occlusion to work. Here is what we've ruled out so far: ObjectTrackingProvider — blocked at enrolment. We can't even produce a usable reference object: the object-capture / scanning step doesn't pick up the needle at all (≈1 mm shaft, specular/metallic, almost no surface features). So object tracking never gets off the ground — it fails before runtime. SceneReconstructionProvider + OcclusionMaterial — only usable for large planar surfaces. Occlusion against a wall or floor is clean. But the mesh is too coarse for anything with finer geometry: a chair already gives very rough, ragged occlusion edges, and the needle is far below the mesh resolution — its shaft never appears in the mesh, so it never occludes at all. So this isn't only a needle problem: it's a mesh resolution problem that goes from rough (chair) to nonexistent (needle). Automatic hand occlusion — doesn't cover the tip. The hand is composited in front correctly, but the needle tip protrudes a few centimetres beyond the fingers, and virtual content draws over exactly that part. The needle tip is the part that most needs to occlude correctly, so this is blocking for us. The full Xcode sample project and the screen recording are in this GitHub repo: https://github.com/JerryNee/occlusion-demo (the video is at media/occlusion-demo.mp4). The screenshots below are stills from that recording, which has four stages, all with the same virtual cube: Occlusion off — baseline; the cube draws over everything in the room. Occlusion on, against a wall — the cube is occluded cleanly. Occlusion on, against a chair — the cube is occluded, but with rough, ragged edges. Occlusion on, with a needle — the needle does not occlude the cube at all. The same occlusion path therefore degrades from clean (wall) → rough (chair) → nonexistent (needle) as the real geometry gets finer. Questions / avenues we'd welcome guidance on (any one would unblock us): What is the recommended way to occlude an object this thin on visionOS — and is it possible at all today? Is there app-accessible per-pixel scene depth (analogous to ARFrame.sceneDepth on iOS) that we could use to occlude geometry the reconstruction mesh misses? Can the system's automatic hand / upper-limb occlusion be extended to include a thin instrument held in the hand — i.e. treat a held tool as part of the hand region? The tool has known, rigid geometry, so we considered attaching an OcclusionMaterial proxy mesh and positioning it from hand-tracking joints — but the needle's orientation within the grip is not determined by the hand pose (the same grip can hold the needle at many different angles), so a hand-joint proxy would be misaligned. Is there any supported way to recover the actual 6-DoF pose of a thin tool held in the hand (i.e. something that senses the tool itself, since the hand pose doesn't constrain it)? Is there any way to make object capture / scanning succeed for thin, specular objects, or an alternative enrolment path? Environment: Apple Vision Pro, visionOS 26.2. Full Xcode sample + screen recording: https://github.com/JerryNee/occlusion-demo Even a "not currently possible / known limitation" answer would help us plan our roadmap. Thank you!
2
0
610
Jun ’26
Grounding shadows appear only in an actually tracked plane: is that how it's supposed to work?
Hi all: Novice iOS programmer here. I am developing an iPhone app (target: iOS 18) where I need to place a large building a bit far away from the user: around 10-20 meters away. Now, the problem I am having is that ARKit's floor detection usually doesn't reach that far. I solved it by using "existingPlaneInfinite" in the raycast: let raycastResults = arView.session.raycast( ARRaycastQuery(origin: rayOrigin, direction: rayDirection, allowing: .existingPlaneInfinite, alignment: .horizontal) But the problem is that the building appears... but without grounding shadows. I tried enabling GroundingShadowComponent, but without success. After some more testing, I found out that if I place an object in a plane that ARKit is actually detecting, then it will have grounding shadows... but if I place it further away, in the part of the floor that ARKit doesn't detect, then it won't add shadows. It will still place the building at the right height (because it extends the detected plane to the infinite), but it won't add contact shadows with the floor. I can solve this problem using the usual tricks (adding an invisible plane as a shadow catcher, etc.), but I just wanted some confirmation that this is indeed the correct behaviour. Also, are there any changes in this regard in IOS 26?
1
0
203
Jun ’26
PhotogrammetrySession(input: [PhotogrammetrySample])
Please I would really appreciate a timely response to this: Providing a response and feedback to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing. Please I would really appreciate a timely response to this
1
0
1.2k
Jun ’26
Adding Images larger than 640px to an AR Ressources Group crashes XCode 26.5
As soon as I add an image with more than 640px (longest side) to an AR Resource Group as an AR Reference Image, Xcode crashes immediately. I narrowed it down to that value. It does not seem to matter whether jpeg or png and the file size seems to be irrelevant, too. The problem came to my attention when I tried to run a new Unity build of an AR App I created a year ago (for testing) in Xcode. This used to work flawlessly. Now It doesn’t; the build produces an error „Command CompileAssetCatalog failed with a nonzero exit code“ … and when I try to click on the Asset Catalogue inside of Xcode, Xcode crashes also immediately.
1
0
80
Jun ’26
Object Capture broken on iPhone 17 Pro (iOS 26.x): ObjectCaptureSession sensorFailed + PhotogrammetrySession cv3dapi 4004
The full RealityKit Object Capture stack fails device-specifically on the iPhone 17 Pro family (iPhone18,1 / iPhone18,2) under iOS 26.x. Both the live and offline paths break. The same code and images work on iPhone 12 Pro (iPhone13,3) at the same iOS version. Both ObjectCaptureSession.isSupported and PhotogrammetrySession.isSupported return true on the 17 Pro, so there's no API signal that the operation will fail — it only fails at runtime on real hardware. Defect 1 — ObjectCaptureSession.Error.sensorFailed A guided capture session fails. On iOS 26.4.2 it fails at scan start. On iOS 26.5 it reaches .capturing and renders the live session for several seconds, but auto-capture never fires; attempting a manual capture then fails. The Swift enum (.sensorFailed) hides the cause; the underlying framework log is: <<<< FigXPCUtilities >>>> signalled err=-17281 at <>:308 <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:569) - (err=-17281) ObjectCaptureSession.takeStillImageCapture(isAutoCapture:): Failed to capture high resolution frame! ARSession <0x…>: did fail with error: Error Domain=com.apple.arkit.error Code=102 "Sensor failure." UserInfo={NSUnderlyingError=0x… {Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot complete action" UserInfo={NSLocalizedRecoverySuggestion=Try again later.}} NSLocalizedFailureReason=The sensor did not provide the required data.} Capturing failed with error: objectCaptureSessionError(CoreOC.ObjectCaptureSession.Error.sensorFailed) A second, uncatchable variant aborts instead of returning the error: Abort: …/AppleCVHWA/library/VIO/HWFeatureDetection/src/HwGeneralProcessingAPI.cpp:711 : Original error reported by request utility function: kVisionHWAReturnGeneralProcessingFailed Appears to be mediaserverd resetting (−11819) underneath the live AVCaptureSession, tearing it out from under ARKit (ARError 102). Configuration-independent: tested with isOverCaptureEnabled = false and the identical chain still occurs. Defect 2 — PhotogrammetrySession reconstruction (cv3dapi 4004) Folder-mode reconstruction starts but fails after a few seconds; SfM builds no pose map: ERROR cv3dapi.pg: Internal error codes (1): 4004 // "No SfM map found" WARN cv3dapi.pg: Internal warning codes (2): 2502 3502 Output error with code = -15 requestError: CoreOC.PhotogrammetrySession.Error.processError The input is just a folder of images handed to PhotogrammetrySession in folder mode — no other configuration. The same folder reconstructs fine on the 12 Pro. Affected config Failing: iPhone 17 Pro (iPhone18,1), 17 Pro Max (iPhone18,2), iOS 26.4.2 and 26.5 Working (same code/OS): iPhone 12 Pro (iPhone13,3) Open question Does a 17-Pro image set fail PhotogrammetrySession everywhere, or only on the 17 Pro? Fails everywhere → 17 Pro images / HEIC metadata confuse SfM. Works elsewhere → the on-device reconstruction engine itself is the defect. Sample image set + sysdiagnose available on request. References cv3dapi 4004 = "No SfM map found" (683300) · -15/processError wrapper (794169) · iOS 26 Object Capture regression (810496) · 17 Pro Max sensorFailed (827216)
1
0
1.4k
Jun ’26
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Providing clarity and response to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing.
0
0
1.4k
May ’26
ARKit ARCamera.intrinsics changes between frames: what can affect fx/fy/cx/cy?
Hello, I am doing a personal research project around ARKit camera frames and camera calibration data. For each accepted ARFrame, I export: ARFrame.timestamp ARFrame.camera.imageResolution ARFrame.camera.intrinsics ARFrame.capturedImage The target video mode is 4K / 30 FPS. In my test captures, I noticed two things: ARFrame.camera.intrinsics is not constant across the recording. Some neighboring frames from ARFrame.capturedImage have noticeably different sharpness: one frame can look sharp, while the next frame can look blurred. Regarding intrinsics, I observe small per-frame changes in fx/fy/cx/cy. For example, in one session: fx/fy changed by approximately 36 px cx changed by approximately 1.5 px cy changed by approximately 2.3 px I understand the standard meaning of the intrinsic matrix: fx / fy = focal length in pixels cx / cy = principal point in pixels relative to the image/reference frame My main question is not about the definition of these values, but about how to correctly interpret their changes over time. In particular, I am interested in cx/cy, because the principal point is important for measurements. I would like to understand whether a change in cx/cy corresponds to a meaningful update of the camera model for the current frame, and what physical or camera-pipeline process may cause this change. Questions about ARFrame.camera.intrinsics: Are ARFrame.camera.intrinsics values expected to change between frames during a single ARWorldTrackingConfiguration session? If fx/fy changes over time, can this reflect autofocus / focus breathing, internal camera calibration updates, digital crop/scaling, stabilization-related mapping, or other camera pipeline changes? If cx/cy changes over time, should I interpret this as an updated principal point for the current ARFrame image/reference frame? Does ARKit update cx/cy to account for any internal crop, scaling, stabilization, virtual camera behavior, lens movement, or calibration changes? Is there a known physical interpretation for small cx/cy changes reported by ARFrame.camera.intrinsics, or should these values simply be treated as ARKit’s best current camera model for that frame? In an ARKit-first pipeline where focus/exposure/stabilization are managed by ARKit/camera pipeline, is the recommended approach to store ARFrame.camera.intrinsics per frame together with ARFrame.timestamp and ARFrame.camera.imageResolution? Is there any public API that exposes the reason for intrinsics changes, such as focus position, stabilization transform, crop transform, active camera constituent, lens movement, or per-frame calibration update reason? I also have a related question about ARFrame.capturedImage frame quality. In some recordings, neighboring frames can have noticeably different sharpness. For example, one frame can look sharp, while the next frame can be visibly blurred, even though the scene and camera movement are continuous. Questions about frame sharpness / camera pipeline behavior: Is frame-to-frame sharpness variation expected when using ARFrame.capturedImage as the video source? Can ARKit change focus, exposure duration, ISO, white balance, crop/scaling, or other camera pipeline parameters between frames during an ARWorldTrackingConfiguration session? Is there any public API to inspect per-frame exposure duration, ISO, focus position, lens position, stabilization state, crop/scaling transform, or other capture parameters for ARFrame.capturedImage? If some frames are sharp and neighboring frames are blurred, should this usually be interpreted as motion blur / exposure behavior / autofocus behavior, or can ARKit internal processing also affect this? Is there any recommended way in an ARKit-first pipeline to reduce frame-to-frame sharpness inconsistency, or is the only reliable approach to use an AVFoundation-first capture pipeline with locked focus/exposure/ISO/white balance? The main goal of my research is to understand how to correctly interpret ARKit-provided intrinsics over time and how much of the camera pipeline behavior is observable through public ARKit APIs. Since fx/fy/cx/cy are important for measurements, I want to know whether treating ARFrame.camera.intrinsics as the Apple-delivered per-frame camera model is the safest approach. Thank you.
0
0
609
May ’26
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Providing a response and feedback to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 I have a created a feedback/bug report with ID FB22823733 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing.
0
0
1.8k
May ’26
ARSession Error: Required sensor failed
Hi everyone, I’m currently using the RoomPlan API, which has been working reliably until recently. However, I’ve started encountering an intermittent error and I’m trying to understand what might be causing it. The error is triggered in the ARSession observer method: session(_ session: ARSession, didFailWithError error: Error) It has occurred on at least two devices: iPhone 14 Pro iPhone 17 Pro Here’s the full error message: ARSession failed domain=com.apple.arkit.error code=102 desc=Required sensor failed. userInfo=["NSLocalizedFailureReason": A sensor failed to deliver the required input., "NSUnderlyingError": Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot Complete Action" UserInfo={NSLocalizedDescription=Cannot Complete Action, NSLocalizedRecoverySuggestion=Try again later.}, "NSLocalizedDescription": Required sensor failed.] This seems to indicate that a required sensor (likely LiDAR or camera) failed to provide input, but I’m not sure what’s causing it or why it happens only occasionally. Has anyone experienced something similar or has insight into possible causes or fixes? Thanks in advance!
1
0
1.3k
May ’26
`ARCamera.unprojectPoint` and `ARCamera.TrackingState` behavior changes between iOS 26.3 and 26.4 under AR resource pressure
ARCamera.TrackingState questions: Did the threshold or sensitivity for transitioning ARCamera.TrackingState from .normal to .limited(.excessiveMotion) or .limited(.insufficientFeatures) change between iOS 26.3 and iOS 26.4? What does "ARWorldTrackingTechnique: resource constraints [33]" mean, and is it new in iOS 26.4? Does it correspond to a tracking state degradation? Is there a way for the client to detect or respond to ARKit entering a resource-constrained mode short of the full tracking state transition — for example, a lower-level notification or a flag on ARFrame — so that apps can take protective action without interpreting it as a full tracking failure? ARCamera.unprojectPoint questions: Did the behavior of ARCamera.unprojectPoint(_:ontoPlane:orientation:viewportSize:) change between iOS 26.3 and iOS 26.4 for near-parallel geometry? Specifically, on iOS 26.3 this method returns nil when the camera ray is nearly parallel to the target plane (denominator of the ray-plane intersection → 0 at ~90° of camera rotation). On iOS 26.4, with identical code and environment, it returns a large finite value instead — we observed z = −12.27m. Since the method's optional return type implies nil is the documented signal for no valid intersection, this reads as a behavioral regression rather than an intentional change. If returning the computed value for near-parallel geometry is now the intended behavior, is there a recommended way for the caller to guard against it? For example, should we check abs(dot(rayDirection, planeNormal)) against a threshold before calling, and if so, is there a documented epsilon Apple uses internally? Alternatively, is there a newer API we should prefer over unprojectPoint(:ontoPlane:) for this use case that handles degenerate geometry more gracefully — such as ARSession.raycast(:)? Are there any other ARKit API adjustments between OS 26.3 and 26.4? We are using the same codebase, but it behaves differently in between these 2 OS versions now. Thanks!
0
0
617
May ’26
Real world anchors
I’m trying to build a persistent world map of my college campus using ARKit, but it’s not very reliable. Anchors don’t consistently appear in the same place across sessions. I’ve tried using image anchors, but they didn’t improve accuracy much. How can I create a stable world map for a larger area and reliably relocalize anchors? Are there better approaches or recommended resources for this?
1
0
1.7k
Apr ’26
ParticleEmitterComponent Position Offset Issue After iOS 26.1 Update – Seeking Solutions & Workarounds
Problem Summary After upgrading to iOS 26.1 and 26.2, I'm experiencing a particle positioning bug in RealityKit where ParticleEmitterComponent particles render at an incorrect offset relative to their parent entity. This behavior does not occur on iOS 18.6.2 or earlier versions, suggesting a regression introduced in the newer OS builds. Environment Details Operating System: iOS 26.1 & iOS 26.2 Framework: RealityKit Xcode Version: 16.2 (16C5032a) Expected vs. Actual Behavior Expected: Particles should render at the position of the entity to which the ParticleEmitterComponent is attached, matching the behavior on iOS 18.6.2 and earlier. Actual: Particles appear away from their parent entity, creating a visual misalignment that breaks the intended AR experience. Steps to Reproduce Create or open an AR application with RealityKit that uses particle components Attach a ParticleEmitterComponent to an entity via a custom system Run the application on iOS 26.1 or iOS 26.2 Observe that particles render at an offset position away from the entity Minimal Code Example Here's the setup from my test case: Custom Component & System: struct SparkleComponent4: Component {} class SparkleSystem4: System { static let query = EntityQuery(where: .has(SparkleComponent4.self)) required init(scene: Scene) {} func update(context: SceneUpdateContext) { for entity in context.scene.performQuery(Self.query) { // Only add once if entity.components.has(ParticleEmitterComponent.self) { continue } var newEmitter = ParticleEmitterComponent() newEmitter.mainEmitter.color = .constant(.single(.red)) entity.components.set(newEmitter) } } } AR Setup: let material = SimpleMaterial(color: .gray, roughness: 0.15, isMetallic: true) let model = Entity() model.components.set(ModelComponent(mesh: boxMesh, materials: [material])) model.components.set(SparkleComponent4()) model.position = [0, 0.05, 0] model.name = "MyCube" let anchor = AnchorEntity(.plane(.horizontal, classification: .any, minimumBounds: [0.2, 0.2])) anchor.addChild(model) arView.scene.addAnchor(anchor) Questions for the Community Has anyone else encountered this particle positioning issue after updating to iOS 26.1/26.2? Are there known workarounds or configuration changes to ParticleEmitterComponent that restore correct positioning? Is this a confirmed bug, or could there be a change in coordinate system handling or transform inheritance that I'm missing? Additional Information I've already submitted this issue via Feedback Assistant(FB21346746)
3
1
2.1k
Apr ’26
RealityView camera feed not shown
I have two RealityView: ParentView and When click the button in ParentView, ChildView will be shown as full screen cover, but the camera feed in ChildView will not be shown, only black screen. If I show ChildView directly, it works with camera feed. Please help me on this issue? Thanks. import RealityKit import SwiftUI struct ParentView: View{ @State private var showIt = false var body: some View{ ZStack{ RealityView{content in content.camera = .virtual let box = ModelEntity(mesh: MeshResource.generateSphere(radius: 0.2),materials: [createSimpleMaterial(color: .red)]) content.add(box) } Button("Click here"){ showIt = true } } .fullScreenCover(isPresented: $showIt){ ChildView() .overlay( Button("Close"){ showIt = false }.padding(20), alignment: .bottomLeading ) } .ignoresSafeArea(.all) } } import ARKit import RealityKit import SwiftUI struct ChildView: View{ var body: some View{ RealityView{content in content.camera = .spatialTracking } } }
5
1
2.9k
Apr ’26
iPad Compass Calibration Issues: Impact of Magnetic Cover
We have developed an iPad application using the ARCL (AR + CoreLocation) library to render Point of Interest (POI) annotations in both an AR view and a standard MapView. The application performs as expected on standard devices. However, we have some iPad covered with strong magnet. This creates significant magnetic interference, resulting in a 90° to 180° heading offset, rendering the AR POI placement and MapView orientation unusable. Technical Challenges & Constraints: Hardware Lock: The magnetic cover is a mandatory business requirement and cannot be removed during field use. Sensor Failure: The internal magnetometer cannot provide an accurate North reference due to the proximity of the cover’s magnets. While CoreLocation and CoreMotion use sensor fusion, the magnetometer remains the primary source for absolute heading. Alternative Orientation Tracking: Is there a documented method to bypass the magnetometer and derive device orientation using only the Gyroscope and Accelerometer (e.g., relative tracking) while still maintaining alignment with geographic coordinates in CoreLocation? Programmatic Offsets: Are there known APIs or mathematical workarounds to programmatically "nullify" or offset a constant magnetic bias once the device is inside the cover? so we can use that offset for ARView and in Mapview as well.
Replies
2
Boosts
0
Views
1.4k
Activity
Jun ’26
ARKit Face Tracking works in total darkness?
I’ve seen, mainly in discussions with AIs, that ARFaceTrackingConfiguration uses the same technology as Face ID and therefore should work in complete darkness. However, I haven’t been able to achieve this. Does anyone know if this is actually true? I'm using an iPhone 16 to test, and the Face ID works well in darkness.
Replies
1
Boosts
1
Views
705
Activity
Jun ’26
What does `ARWorldTrackingTechnique: resource constraints [33]` mean?
We are building an ARKit application targeting iPhone Pro devices on iOS 26.4+. During testing, we consistently observe the following log message at a specific point in our AR session (approximately 90° into a walking arc around a vehicle): ARWorldTrackingTechnique: resource constraints [33] What specifically does resource constraint code [33] represent? Is it a VIO constraint type, a sensor fusion budget limit, or something else entirely? How does it differ from code [1] (which appears to be a general CPU starvation signal for the ARKit sensor fusion thread)? Is there a public API to detect or respond to this specific constraint condition, short of waiting for a full ARCamera.TrackingState downgrade? Was this constraint type introduced or changed in iOS 26.4?
Replies
1
Boosts
0
Views
1.5k
Activity
Jun ’26
ARKit object tracking performance
I'm trying to track identified objects in realtime with bounding rects, with no 3D integration, but still has poor update performance. I'm trying to understand how to optimize frame updates. (I'm a new programmer) Using: Foundation, AVFoundation, ARKit, CoreVideo, Vision, CoreImage, CoreVideo with YOLOE-11s object detection currently throttled to 2fps. (target iOS, testing on 16pro) YOLOE-11S CoreML model detects objects with class labels + bounding boxes Labels are matched against ObjectCatalog.json for relevance Matched objects are promoted from blue (detected) to green (identified) Log warnings: ARSession <0x110afdb80>: The delegate of ARSession is retaining 13 ARFrames. The camera will stop delivering camera images if the delegate keeps holding on to too many ARFrames. This could be a threading or memory management issue in the delegate and should be fixed. Skipping integration due to poor slam at time: 619447.208339 vio_initialized(1) map_size(0) tracking_state_is_nominal(0) is_3dof(0) reinitialize_attempts(6) slam_mode(RegularSLAM)
Replies
2
Boosts
0
Views
808
Activity
Jun ’26
Recommended visionOS architecture for opt-in nearby exchange triggered by a physical gesture
I’m exploring a visionOS interaction pattern where nearby Apple Vision Pro users have opted in ahead of time, such as when entering a shared venue, and a physical gesture like a handshake triggers a lightweight exchange of user-approved information between their devices. I’m not asking about identifying strangers, accessing raw camera data, or tracking another person without consent. I’m trying to understand the most automatic Apple-supported architecture for this kind of privacy-preserving nearby interaction. What is the recommended approach on visionOS? Specifically: Can a visionOS app use ARKit hand tracking to detect the current user’s own gesture, then combine that with nearby peer discovery or ranging through Nearby Interaction, Network framework, Multipeer Connectivity, Bluetooth, Wi-Fi, or another supported API? Is there a supported way for two nearby Vision Pro devices to exchange a small user-approved payload after prior opt-in, without requiring users to manually start SharePlay or confirm every individual exchange? If SharePlay or Group Activities is the recommended path for shared spatial context, is there a supported alternative for venue-scale or multi-user interactions that should not be limited to a small active SharePlay group? What are the privacy and App Review boundaries for this pattern? Should developers assume the app cannot identify nearby people, observe another person’s body or hands, or trigger an exchange unless both users have explicit opt-in and clear awareness? If a mostly passive gesture-triggered exchange is not supported today, what is the closest Apple-recommended design pattern?
Replies
2
Boosts
0
Views
932
Activity
Jun ’26
How can we occlude a thin metal needle (held in the hand) with virtual content on visionOS?
Hi, We're building a mixed-immersive visionOS app in which the user holds a thin metal tool — a needle — near virtual content. We need the real needle to correctly occlude virtual content: when the needle (and especially its tip) is physically in front of a virtual object, the virtual object must not draw over it. We are solution-agnostic. Whether the right answer is LiDAR / scene-depth based, object-tracking based, hand-tracking based, or something else entirely — we just need needle occlusion to work. Here is what we've ruled out so far: ObjectTrackingProvider — blocked at enrolment. We can't even produce a usable reference object: the object-capture / scanning step doesn't pick up the needle at all (≈1 mm shaft, specular/metallic, almost no surface features). So object tracking never gets off the ground — it fails before runtime. SceneReconstructionProvider + OcclusionMaterial — only usable for large planar surfaces. Occlusion against a wall or floor is clean. But the mesh is too coarse for anything with finer geometry: a chair already gives very rough, ragged occlusion edges, and the needle is far below the mesh resolution — its shaft never appears in the mesh, so it never occludes at all. So this isn't only a needle problem: it's a mesh resolution problem that goes from rough (chair) to nonexistent (needle). Automatic hand occlusion — doesn't cover the tip. The hand is composited in front correctly, but the needle tip protrudes a few centimetres beyond the fingers, and virtual content draws over exactly that part. The needle tip is the part that most needs to occlude correctly, so this is blocking for us. The full Xcode sample project and the screen recording are in this GitHub repo: https://github.com/JerryNee/occlusion-demo (the video is at media/occlusion-demo.mp4). The screenshots below are stills from that recording, which has four stages, all with the same virtual cube: Occlusion off — baseline; the cube draws over everything in the room. Occlusion on, against a wall — the cube is occluded cleanly. Occlusion on, against a chair — the cube is occluded, but with rough, ragged edges. Occlusion on, with a needle — the needle does not occlude the cube at all. The same occlusion path therefore degrades from clean (wall) → rough (chair) → nonexistent (needle) as the real geometry gets finer. Questions / avenues we'd welcome guidance on (any one would unblock us): What is the recommended way to occlude an object this thin on visionOS — and is it possible at all today? Is there app-accessible per-pixel scene depth (analogous to ARFrame.sceneDepth on iOS) that we could use to occlude geometry the reconstruction mesh misses? Can the system's automatic hand / upper-limb occlusion be extended to include a thin instrument held in the hand — i.e. treat a held tool as part of the hand region? The tool has known, rigid geometry, so we considered attaching an OcclusionMaterial proxy mesh and positioning it from hand-tracking joints — but the needle's orientation within the grip is not determined by the hand pose (the same grip can hold the needle at many different angles), so a hand-joint proxy would be misaligned. Is there any supported way to recover the actual 6-DoF pose of a thin tool held in the hand (i.e. something that senses the tool itself, since the hand pose doesn't constrain it)? Is there any way to make object capture / scanning succeed for thin, specular objects, or an alternative enrolment path? Environment: Apple Vision Pro, visionOS 26.2. Full Xcode sample + screen recording: https://github.com/JerryNee/occlusion-demo Even a "not currently possible / known limitation" answer would help us plan our roadmap. Thank you!
Replies
2
Boosts
0
Views
610
Activity
Jun ’26
Grounding shadows appear only in an actually tracked plane: is that how it's supposed to work?
Hi all: Novice iOS programmer here. I am developing an iPhone app (target: iOS 18) where I need to place a large building a bit far away from the user: around 10-20 meters away. Now, the problem I am having is that ARKit's floor detection usually doesn't reach that far. I solved it by using "existingPlaneInfinite" in the raycast: let raycastResults = arView.session.raycast( ARRaycastQuery(origin: rayOrigin, direction: rayDirection, allowing: .existingPlaneInfinite, alignment: .horizontal) But the problem is that the building appears... but without grounding shadows. I tried enabling GroundingShadowComponent, but without success. After some more testing, I found out that if I place an object in a plane that ARKit is actually detecting, then it will have grounding shadows... but if I place it further away, in the part of the floor that ARKit doesn't detect, then it won't add shadows. It will still place the building at the right height (because it extends the detected plane to the infinite), but it won't add contact shadows with the floor. I can solve this problem using the usual tricks (adding an invisible plane as a shadow catcher, etc.), but I just wanted some confirmation that this is indeed the correct behaviour. Also, are there any changes in this regard in IOS 26?
Replies
1
Boosts
0
Views
203
Activity
Jun ’26
PhotogrammetrySession(input: [PhotogrammetrySample])
Please I would really appreciate a timely response to this: Providing a response and feedback to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing. Please I would really appreciate a timely response to this
Replies
1
Boosts
0
Views
1.2k
Activity
Jun ’26
Adding Images larger than 640px to an AR Ressources Group crashes XCode 26.5
As soon as I add an image with more than 640px (longest side) to an AR Resource Group as an AR Reference Image, Xcode crashes immediately. I narrowed it down to that value. It does not seem to matter whether jpeg or png and the file size seems to be irrelevant, too. The problem came to my attention when I tried to run a new Unity build of an AR App I created a year ago (for testing) in Xcode. This used to work flawlessly. Now It doesn’t; the build produces an error „Command CompileAssetCatalog failed with a nonzero exit code“ … and when I try to click on the Asset Catalogue inside of Xcode, Xcode crashes also immediately.
Replies
1
Boosts
0
Views
80
Activity
Jun ’26
Object Capture broken on iPhone 17 Pro (iOS 26.x): ObjectCaptureSession sensorFailed + PhotogrammetrySession cv3dapi 4004
The full RealityKit Object Capture stack fails device-specifically on the iPhone 17 Pro family (iPhone18,1 / iPhone18,2) under iOS 26.x. Both the live and offline paths break. The same code and images work on iPhone 12 Pro (iPhone13,3) at the same iOS version. Both ObjectCaptureSession.isSupported and PhotogrammetrySession.isSupported return true on the 17 Pro, so there's no API signal that the operation will fail — it only fails at runtime on real hardware. Defect 1 — ObjectCaptureSession.Error.sensorFailed A guided capture session fails. On iOS 26.4.2 it fails at scan start. On iOS 26.5 it reaches .capturing and renders the live session for several seconds, but auto-capture never fires; attempting a manual capture then fails. The Swift enum (.sensorFailed) hides the cause; the underlying framework log is: <<<< FigXPCUtilities >>>> signalled err=-17281 at <>:308 <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:569) - (err=-17281) ObjectCaptureSession.takeStillImageCapture(isAutoCapture:): Failed to capture high resolution frame! ARSession <0x…>: did fail with error: Error Domain=com.apple.arkit.error Code=102 "Sensor failure." UserInfo={NSUnderlyingError=0x… {Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot complete action" UserInfo={NSLocalizedRecoverySuggestion=Try again later.}} NSLocalizedFailureReason=The sensor did not provide the required data.} Capturing failed with error: objectCaptureSessionError(CoreOC.ObjectCaptureSession.Error.sensorFailed) A second, uncatchable variant aborts instead of returning the error: Abort: …/AppleCVHWA/library/VIO/HWFeatureDetection/src/HwGeneralProcessingAPI.cpp:711 : Original error reported by request utility function: kVisionHWAReturnGeneralProcessingFailed Appears to be mediaserverd resetting (−11819) underneath the live AVCaptureSession, tearing it out from under ARKit (ARError 102). Configuration-independent: tested with isOverCaptureEnabled = false and the identical chain still occurs. Defect 2 — PhotogrammetrySession reconstruction (cv3dapi 4004) Folder-mode reconstruction starts but fails after a few seconds; SfM builds no pose map: ERROR cv3dapi.pg: Internal error codes (1): 4004 // "No SfM map found" WARN cv3dapi.pg: Internal warning codes (2): 2502 3502 Output error with code = -15 requestError: CoreOC.PhotogrammetrySession.Error.processError The input is just a folder of images handed to PhotogrammetrySession in folder mode — no other configuration. The same folder reconstructs fine on the 12 Pro. Affected config Failing: iPhone 17 Pro (iPhone18,1), 17 Pro Max (iPhone18,2), iOS 26.4.2 and 26.5 Working (same code/OS): iPhone 12 Pro (iPhone13,3) Open question Does a 17-Pro image set fail PhotogrammetrySession everywhere, or only on the 17 Pro? Fails everywhere → 17 Pro images / HEIC metadata confuse SfM. Works elsewhere → the on-device reconstruction engine itself is the defect. Sample image set + sysdiagnose available on request. References cv3dapi 4004 = "No SfM map found" (683300) · -15/processError wrapper (794169) · iOS 26 Object Capture regression (810496) · 17 Pro Max sensorFailed (827216)
Replies
1
Boosts
0
Views
1.4k
Activity
Jun ’26
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Providing clarity and response to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing.
Replies
0
Boosts
0
Views
1.4k
Activity
May ’26
ARKit ARCamera.intrinsics changes between frames: what can affect fx/fy/cx/cy?
Hello, I am doing a personal research project around ARKit camera frames and camera calibration data. For each accepted ARFrame, I export: ARFrame.timestamp ARFrame.camera.imageResolution ARFrame.camera.intrinsics ARFrame.capturedImage The target video mode is 4K / 30 FPS. In my test captures, I noticed two things: ARFrame.camera.intrinsics is not constant across the recording. Some neighboring frames from ARFrame.capturedImage have noticeably different sharpness: one frame can look sharp, while the next frame can look blurred. Regarding intrinsics, I observe small per-frame changes in fx/fy/cx/cy. For example, in one session: fx/fy changed by approximately 36 px cx changed by approximately 1.5 px cy changed by approximately 2.3 px I understand the standard meaning of the intrinsic matrix: fx / fy = focal length in pixels cx / cy = principal point in pixels relative to the image/reference frame My main question is not about the definition of these values, but about how to correctly interpret their changes over time. In particular, I am interested in cx/cy, because the principal point is important for measurements. I would like to understand whether a change in cx/cy corresponds to a meaningful update of the camera model for the current frame, and what physical or camera-pipeline process may cause this change. Questions about ARFrame.camera.intrinsics: Are ARFrame.camera.intrinsics values expected to change between frames during a single ARWorldTrackingConfiguration session? If fx/fy changes over time, can this reflect autofocus / focus breathing, internal camera calibration updates, digital crop/scaling, stabilization-related mapping, or other camera pipeline changes? If cx/cy changes over time, should I interpret this as an updated principal point for the current ARFrame image/reference frame? Does ARKit update cx/cy to account for any internal crop, scaling, stabilization, virtual camera behavior, lens movement, or calibration changes? Is there a known physical interpretation for small cx/cy changes reported by ARFrame.camera.intrinsics, or should these values simply be treated as ARKit’s best current camera model for that frame? In an ARKit-first pipeline where focus/exposure/stabilization are managed by ARKit/camera pipeline, is the recommended approach to store ARFrame.camera.intrinsics per frame together with ARFrame.timestamp and ARFrame.camera.imageResolution? Is there any public API that exposes the reason for intrinsics changes, such as focus position, stabilization transform, crop transform, active camera constituent, lens movement, or per-frame calibration update reason? I also have a related question about ARFrame.capturedImage frame quality. In some recordings, neighboring frames can have noticeably different sharpness. For example, one frame can look sharp, while the next frame can be visibly blurred, even though the scene and camera movement are continuous. Questions about frame sharpness / camera pipeline behavior: Is frame-to-frame sharpness variation expected when using ARFrame.capturedImage as the video source? Can ARKit change focus, exposure duration, ISO, white balance, crop/scaling, or other camera pipeline parameters between frames during an ARWorldTrackingConfiguration session? Is there any public API to inspect per-frame exposure duration, ISO, focus position, lens position, stabilization state, crop/scaling transform, or other capture parameters for ARFrame.capturedImage? If some frames are sharp and neighboring frames are blurred, should this usually be interpreted as motion blur / exposure behavior / autofocus behavior, or can ARKit internal processing also affect this? Is there any recommended way in an ARKit-first pipeline to reduce frame-to-frame sharpness inconsistency, or is the only reliable approach to use an AVFoundation-first capture pipeline with locked focus/exposure/ISO/white balance? The main goal of my research is to understand how to correctly interpret ARKit-provided intrinsics over time and how much of the camera pipeline behavior is observable through public ARKit APIs. Since fx/fy/cx/cy are important for measurements, I want to know whether treating ARFrame.camera.intrinsics as the Apple-delivered per-frame camera model is the safest approach. Thank you.
Replies
0
Boosts
0
Views
609
Activity
May ’26
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Providing a response and feedback to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 I have a created a feedback/bug report with ID FB22823733 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing.
Replies
0
Boosts
0
Views
1.8k
Activity
May ’26
ARSession Error: Required sensor failed
Hi everyone, I’m currently using the RoomPlan API, which has been working reliably until recently. However, I’ve started encountering an intermittent error and I’m trying to understand what might be causing it. The error is triggered in the ARSession observer method: session(_ session: ARSession, didFailWithError error: Error) It has occurred on at least two devices: iPhone 14 Pro iPhone 17 Pro Here’s the full error message: ARSession failed domain=com.apple.arkit.error code=102 desc=Required sensor failed. userInfo=["NSLocalizedFailureReason": A sensor failed to deliver the required input., "NSUnderlyingError": Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot Complete Action" UserInfo={NSLocalizedDescription=Cannot Complete Action, NSLocalizedRecoverySuggestion=Try again later.}, "NSLocalizedDescription": Required sensor failed.] This seems to indicate that a required sensor (likely LiDAR or camera) failed to provide input, but I’m not sure what’s causing it or why it happens only occasionally. Has anyone experienced something similar or has insight into possible causes or fixes? Thanks in advance!
Replies
1
Boosts
0
Views
1.3k
Activity
May ’26
`ARCamera.unprojectPoint` and `ARCamera.TrackingState` behavior changes between iOS 26.3 and 26.4 under AR resource pressure
ARCamera.TrackingState questions: Did the threshold or sensitivity for transitioning ARCamera.TrackingState from .normal to .limited(.excessiveMotion) or .limited(.insufficientFeatures) change between iOS 26.3 and iOS 26.4? What does "ARWorldTrackingTechnique: resource constraints [33]" mean, and is it new in iOS 26.4? Does it correspond to a tracking state degradation? Is there a way for the client to detect or respond to ARKit entering a resource-constrained mode short of the full tracking state transition — for example, a lower-level notification or a flag on ARFrame — so that apps can take protective action without interpreting it as a full tracking failure? ARCamera.unprojectPoint questions: Did the behavior of ARCamera.unprojectPoint(_:ontoPlane:orientation:viewportSize:) change between iOS 26.3 and iOS 26.4 for near-parallel geometry? Specifically, on iOS 26.3 this method returns nil when the camera ray is nearly parallel to the target plane (denominator of the ray-plane intersection → 0 at ~90° of camera rotation). On iOS 26.4, with identical code and environment, it returns a large finite value instead — we observed z = −12.27m. Since the method's optional return type implies nil is the documented signal for no valid intersection, this reads as a behavioral regression rather than an intentional change. If returning the computed value for near-parallel geometry is now the intended behavior, is there a recommended way for the caller to guard against it? For example, should we check abs(dot(rayDirection, planeNormal)) against a threshold before calling, and if so, is there a documented epsilon Apple uses internally? Alternatively, is there a newer API we should prefer over unprojectPoint(:ontoPlane:) for this use case that handles degenerate geometry more gracefully — such as ARSession.raycast(:)? Are there any other ARKit API adjustments between OS 26.3 and 26.4? We are using the same codebase, but it behaves differently in between these 2 OS versions now. Thanks!
Replies
0
Boosts
0
Views
617
Activity
May ’26
Real world anchors
I’m trying to build a persistent world map of my college campus using ARKit, but it’s not very reliable. Anchors don’t consistently appear in the same place across sessions. I’ve tried using image anchors, but they didn’t improve accuracy much. How can I create a stable world map for a larger area and reliably relocalize anchors? Are there better approaches or recommended resources for this?
Replies
1
Boosts
0
Views
1.7k
Activity
Apr ’26
ParticleEmitterComponent Position Offset Issue After iOS 26.1 Update – Seeking Solutions & Workarounds
Problem Summary After upgrading to iOS 26.1 and 26.2, I'm experiencing a particle positioning bug in RealityKit where ParticleEmitterComponent particles render at an incorrect offset relative to their parent entity. This behavior does not occur on iOS 18.6.2 or earlier versions, suggesting a regression introduced in the newer OS builds. Environment Details Operating System: iOS 26.1 & iOS 26.2 Framework: RealityKit Xcode Version: 16.2 (16C5032a) Expected vs. Actual Behavior Expected: Particles should render at the position of the entity to which the ParticleEmitterComponent is attached, matching the behavior on iOS 18.6.2 and earlier. Actual: Particles appear away from their parent entity, creating a visual misalignment that breaks the intended AR experience. Steps to Reproduce Create or open an AR application with RealityKit that uses particle components Attach a ParticleEmitterComponent to an entity via a custom system Run the application on iOS 26.1 or iOS 26.2 Observe that particles render at an offset position away from the entity Minimal Code Example Here's the setup from my test case: Custom Component & System: struct SparkleComponent4: Component {} class SparkleSystem4: System { static let query = EntityQuery(where: .has(SparkleComponent4.self)) required init(scene: Scene) {} func update(context: SceneUpdateContext) { for entity in context.scene.performQuery(Self.query) { // Only add once if entity.components.has(ParticleEmitterComponent.self) { continue } var newEmitter = ParticleEmitterComponent() newEmitter.mainEmitter.color = .constant(.single(.red)) entity.components.set(newEmitter) } } } AR Setup: let material = SimpleMaterial(color: .gray, roughness: 0.15, isMetallic: true) let model = Entity() model.components.set(ModelComponent(mesh: boxMesh, materials: [material])) model.components.set(SparkleComponent4()) model.position = [0, 0.05, 0] model.name = "MyCube" let anchor = AnchorEntity(.plane(.horizontal, classification: .any, minimumBounds: [0.2, 0.2])) anchor.addChild(model) arView.scene.addAnchor(anchor) Questions for the Community Has anyone else encountered this particle positioning issue after updating to iOS 26.1/26.2? Are there known workarounds or configuration changes to ParticleEmitterComponent that restore correct positioning? Is this a confirmed bug, or could there be a change in coordinate system handling or transform inheritance that I'm missing? Additional Information I've already submitted this issue via Feedback Assistant(FB21346746)
Replies
3
Boosts
1
Views
2.1k
Activity
Apr ’26
RealityView camera feed not shown
I have two RealityView: ParentView and When click the button in ParentView, ChildView will be shown as full screen cover, but the camera feed in ChildView will not be shown, only black screen. If I show ChildView directly, it works with camera feed. Please help me on this issue? Thanks. import RealityKit import SwiftUI struct ParentView: View{ @State private var showIt = false var body: some View{ ZStack{ RealityView{content in content.camera = .virtual let box = ModelEntity(mesh: MeshResource.generateSphere(radius: 0.2),materials: [createSimpleMaterial(color: .red)]) content.add(box) } Button("Click here"){ showIt = true } } .fullScreenCover(isPresented: $showIt){ ChildView() .overlay( Button("Close"){ showIt = false }.padding(20), alignment: .bottomLeading ) } .ignoresSafeArea(.all) } } import ARKit import RealityKit import SwiftUI struct ChildView: View{ var body: some View{ RealityView{content in content.camera = .spatialTracking } } }
Replies
5
Boosts
1
Views
2.9k
Activity
Apr ’26