I work on a game where I use timeline animations in Reality Composer Pro.
The game runs in an immersive space, but can be paused where I then move the whole level root entity from the immersive space to another RealityView in a Window Group. When the player continues I do it exactly the other way around to move the level root from the window group back to my immersive space RealityView.
And it seems like all animations get automatically stopped and restarted when the scene gets changed. The problem is, it does not resume where it stopped before, it completely starts again from where it stopped and therefore, has for example a wrong y offset as visible in the picture.
For example in the picture, the yellow sphere loops the following animation:
0 to 100
100 to -100
-100 to 0
If I now pause the game (and basically switch scenes), the previous animation gets stopped and restarted at position y = 100. So now it loops:
100 to 200
200 to 0
0 to 100
I already tried all kind of setups - like:
Setting the animations relative to root, parent, local
Using behaviors (on Added to Scene, on Notification)
And finally even by accessing the availableAnimations directly and saving the playback controller of the animation
There I saw, if I manually trigger the following code before switching the scene, everything works as expected:
Button("Reset") {
animationPlaybackController.time = 0
animationPlaybackController.pause()
animationPlaybackController.stop(blendOutDuration: 0.00001)
}
But if I use time = 0 with .stop() directly, the time = 0 seems to be ignored and I get the same behavior as before that it stops in a wrong y offset, hence my assumption that animations get stopped and invalidated once they change the scene.
I tried to call the code manually on ImmersiveSpace.onDisappear, WindowGroup.onAppear and different kind of SceneEvents subscriptions, but unfortunately nothing worked.
So am I doing something wrong in general or is there a way to fix this?
Discuss spatial computing on Apple platforms and how to design and build an entirely new universe of apps and games for Apple Vision Pro.
Selecting any option will automatically load the page
Post
Replies
Boosts
Views
Activity
I'm working on a project that uses imageTrackingProvider through ARKit on VisionPro, and I want to detect multiple images(about 5) and show info at the same time.
However, I found that it seems only 1 image could be detected by device at one time.
And the api of maximumNumberOfTrackedImages doing this seems not available for visionOS but only iOS.
Anyone knows possible ways to detect multiple images at the same time on VisionPro?
Topic:
Spatial Computing
SubTopic:
ARKit
When rendering an equirectangular video on a sphere using VideoMaterial and MeshResource.generateSphere(), there is a visible black seam line running vertically on the sphere. This appears to be at the UV seam where the texture coordinates wrap from 1.0 back to 0.0.
The same video file plays without any visible seam in other 360° video players on Vision Pro, so the issue is not with the video content itself.
Here is the relevant code:
private func createVideoSphere(content: RealityViewContent, player: AVPlayer) {
let sphere = MeshResource.generateSphere(radius: 1000)
let material = VideoMaterial(avPlayer: player)
let entity = ModelEntity(mesh: sphere, materials: [material])
entity.scale *= .init(x: -1, y: 1, z: 1) // Flip to render on inside
content.add(entity)
player.play()
}
The setup is straightforward: MeshResource.generateSphere(radius: 1000) generates the sphere mesh VideoMaterial(avPlayer:) provides the video texture X scale is flipped to -1 so the texture renders on the inside of the sphere The video is a standard equirectangular 360° MP4 file
What I've tried:
I attempted to create a custom sphere mesh using MeshDescriptor with duplicate vertices at the UV seam (longitude 0°/360°) to ensure proper UV continuity. However, VideoMaterial did not render any video on the custom mesh (only audio played), and the app eventually crashed. It seems VideoMaterial may have specific mesh requirements.
Questions:
Is the black seam a known limitation of MeshResource.generateSphere() when used with VideoMaterial for 360° video?
Is there a recommended way to eliminate this UV seam — for example, a texture addressing mode or a specific mesh configuration that works with VideoMaterial?
Is there an official sample project or code example for playing 360° equirectangular video in a fully immersive space on visionOS? That would be extremely helpful as a reference.
Any guidance would be greatly appreciated. Thank you!
This is a very exciting feature in 26.4 beta. But from the document, it seems can only integrate with NVIDIA CloudXR™ SDK.
I'm wondering if it's possible to use this tool to stream immersive video from Mac to Vision Pro?
Hi there
I'm using Reality Composer Pro to anchor virtual content to a .referenceobject. However by moving the referenceobject quickly, it causes tracking to stop. (I know this is a limitation so im trying to make it a feature)
IS there a way to play a USDZ animation at the last known location, after detecting that reference object is no longer being tracked? is it possible to set this up in Reality Composer pro?
Nearly everything is set up in Reality Composer pro with my immersive.scene just anchoring virtual content to the Reference object in the RCP Scene, so my immersive view just does this -
if let immersiveContentEntity = try? await Entity(named: "Immersive", in: realityKitContentBundle) {
content.add(immersiveContentEntity)
& this
.onAppear {
appModel.immersiveSpaceState = .open
}
.onDisappear {
appModel.immersiveSpaceState = .closed
}
I have tried Using SpatialTracking & WorldTrackProvider, but I'm still quite new to Swift and coding in general so im unsure how to implement in conjunction with my RCP scene and if this is actually the right way to do it.
Apologies for my lack of knowledge.
Topic:
Spatial Computing
SubTopic:
Reality Composer Pro
Tags:
ARKit
RealityKit
Reality Composer Pro
visionOS
I’ve seen, mainly in discussions with AIs, that ARFaceTrackingConfiguration uses the same technology as Face ID and therefore should work in complete darkness. However, I haven’t been able to achieve this. Does anyone know if this is actually true?
I'm using an iPhone 16 to test, and the Face ID works well in darkness.
佩戴者头部自然晃动时,设备拍摄的画面会出现明显抖动,导致观看直播的用户产生眩晕感,严重影响直播沉浸体验和购物决策效率。
希望
优化设备内置防抖算法,降低头部常规晃动对画面稳定性的影响,提升直播画面的流畅度。
Hello, I am trying to build an AVP app for real-time "zero-latency" spatial video streaming. I am trying to figure out, on a high level, the best way to do this.
Currently this is my method:
Server sends stereo images via a WebRTC service (ie, livekit)
The WebRTC stream is converted to a CVPixelBuffer, writes them to file, plays via AVPlayer, and applies a VideoMaterial to a plane entity.
However, this is a bit hacky and it seems like this won't be compatible with Apple's spatial experinces. To my understanding, Apple supports HLS streaming for spatial experiences and APMP content. However, HLS (and even Low Latency HLS) introduces a second or more of latency, likely do to the segmentation nature of HLS. Thus, HLS will not work for us.
Some other alternatives I've thought of are streaming the live stream video via webrtc from the server to a local computer in the AVP's network, and then using LL-HLS to stream from the local computer to the vision pro. Still, it seems like this would introduce latency on the order of seconds.
Is my current approach the best way to implement this? Or could anyone suggest a better way, perhaps something compatible with AVP's spatial experiences
Topic:
Spatial Computing
SubTopic:
General
Hi,
after upgrading to 2.4.1 (from 1.0) my vision stucks on "Retrieving configuration" screen. Apple Store didn't support my case since it has been sold in USA and the product isn't still present in italian market. I don't have dev strap, how can I manage the issue?
Thank you
Topic:
Spatial Computing
SubTopic:
General
画面亮度存在无规律动态波动(时亮时暗),且无手动控制入口,导致商品颜色还原失真、主播面部曝光异常(过曝 / 欠曝),严重影响直播展示效果。
期望
"· 优化直播模式的自动曝光算法,提升复杂光线环境下的亮度稳定性;
· 增加 “直播模式” 专属亮度锁定功能,支持手动设定亮度参数并锁定,满足直播场景下的画质可控需求。
"
I'm working on an iOS app using ARKit and RealityKit where I scan QR codes and want to place 3D models at the exact position of the QR code in the real world.
Is it possible to accurately place a 3D model at the exact position of a QR code in AR using ARKit and RealityKit? Specifically, I want the model to appear at the precise location where the QR code is detected, rather than just somewhere in the AR space.
If this is possible, could you point me in the right direction or recommend the best approach to achieve this?
Thank you for your help!
Visionos26 Enterprise api has the new feature: Shared Coordinate Space, participants exchange their coordinate data by SharedCoordinateSpaceProvider through their own network, when shared coordinate space established with nearby participants, the event: connectedParticipantIdentifiers(participants: [UUID]) will be received.
But the Event.participantIdentifier still be an invalid default value(00000000-0000-0000-FFFF-FFFFFFFF) in this time, I wonder when or how I can get a valid event.participantIdentifier, or is there some other way to get the local participantIdentifier?
Or If it's a bug, please fix it in later beta release version, thank you.
Summary
After updating to visionOS 26, we’ve encountered severe transparency rendering issues in RealityKit that did not exist in visionOS 2.6 and earlier.
These regressions affect applications that dynamically control scene opacity (via OpacityComponent).
Our app renders ultra-realistic apartment environments in real time, where users can walk or teleport inside 3D spaces. When the user moves above a speed threshold, we apply a global transparency effect to prevent physical collisions with real-world objects.
Everything worked perfectly in visionOS 2.6 — the problems appeared only after upgrading to 26.
Scene Setup Overview
The environment consists of multiple USDZ models (e.g., architecture, rooms, furniture).
We manage LODs manually for performance (e.g., walls and floors always visible in full-res, while rooms swap between low/high-res versions based on user position and field of view).
Transparency is achieved using OpacityComponent, applied dynamically when the user moves.
Some meshes (e.g., portals to skyboxes, glass windows) use alpha materials
We also use OcclusionMaterials to prevent things to be seen through walls when scene is transparent
Observed Behavior by Scenario
(I can share a video showing the results of each scenario if needed.)
Scenario 1 — Severe Flickering (Root Opacity)
Setup:
OpacityComponent applied to the root entity
NO ModelSortGroupComponent used
Symptoms:
Strong flickering when transparency is active
Triangles within the same mesh render at inconsistent opacity levels
Appears as if per-triangle alpha sorting is broken
Workaround:
Moving the OpacityComponent from the root to each individual USDZ entity removes the per-triangle flicker
Pros:
No conflicts with portals or alpha materials
Scenario 2 — Partially Stable, But Alpha Conflicts
Setup:
OpacityComponent applied per USDZ entity
ModelSortGroupComponent(planarUIAlwaysBehind) applied to portal meshes
Other entities have NO ModelSortGroupComponent
Symptoms:
Frequent alpha blending conflicts:
Transparent surfaces behind other transparent surfaces flicker or disappear
Example: Wine glasses behind glass doors — sometimes neither is rendered, or only one
Even opaque meshes behind glass flicker due to depth buffer confusion
Alpha materials sometimes render portals or the real world behind them, ignoring other geometry entirely
Analysis:
Appears related to internal changes in alpha sorting or depth pre-pass behavior introduced in visionOS 26
Pros:
Most stable setup so far
Cons:
Still unreliable when OpacityComponent is active
Scenario 3 — Layer Separation Attempt (Regression)
Setup:
Same as Scenario 2, but:
Entities with alpha materials moved to separate USDZs
Explicit ModelSortGroupComponent order set (alpha surfaces rendered last)
Symptoms:
Transparent surfaces behind other transparent surfaces flicker or disappear
Depth is completely broken when there's a large transparent surface
Alpha materials sometimes render portals or the real world behind them, ignoring other geometry entirely
Workaround Attempt:
Re-ordering and further separating models did not solve it
Pros:
None — this setup makes transparency unusable
Conclusion
There appears to be a regression in RealityKit’s handling of transparency and sorting in visionOS 26, particularly when:
OpacityComponent is applied dynamically, and
Scenes rely on multiple overlapping transparent materials.
These issues did not exist prior to 26, and the same project (no code changes) behaves correctly on previous versions.
Request
We’d appreciate any insight or confirmation from Apple engineers regarding:
Whether alpha sorting or opacity blending behavior changed in visionOS 26
If there are new recommended practices for combining OpacityComponent with transparent materials
If a bug report already exists for this regression
Thanks in advance!
After updating to visionOS 26.2 Beta 2 (and Beta 3), I'm unable to establish a spatial connection to Vision Pro. This was working fine before the update.
To test, I've created a fresh spatialApp project from the Xcode template with zero modifications, but I'm hitting the same issue - the Vision Pro is discovered but won't connect.
Am I forgetting to update the config somewhere? Any ideas what might be causing this and how to fix it?
Thanks!
Warning: -[NSWindow makeKeyWindow] called on <NSWindow: 0xa1f811900> windowNumber=1b9 which returned NO from -[NSWindow canBecomeKeyWindow].
((processConfiguration != nil && configuration != nil) || (processConfiguration == nil && configuration == nil)) - /AppleInternal/Library/BuildRoots/4~CBS0ugAIF7BrQZjLe6r0lhPXO4GJmNDTovxYoV0/Library/Caches/com.apple.xbs/Sources/ExtensionKit/ExtensionKit/Source/HostViewController/Internal/EXHostSessionDriver.m:80: `processConfiguration` and `configuration` must be both non-nil or both nil
Unable to obtain a task name port right for pid 415: (os/kern) failure (0x5)
CCContextDeviceGroup.mm(291):+[CCContextDeviceGroup checkBinaryArchivesForDevice:withBundle:]:
Failed to find any binary shader archive
Topic:
Spatial Computing
SubTopic:
General
I like to compose an APN message. (using FCM)
what shall I do for it?
Topic:
Spatial Computing
SubTopic:
ARKit
In Reality Composer Pro, why is the Sky Sphere so much larger than the Sky Dome?
By my estimate, the Sky Sphere has a radius of 100m, while the Sky only has a radius of only 12m.
Spatial photo in RealityView has a default corner radius. I made a parallel effect with spatial photos in ScrollView(like Spatial Gallery), but the corner radius disappeared on left and right spatial photos. I've tried .clipShape and .mask modifiers, but they did't work. How to clip or mask spatial photo with corner radius effect?
Anyone could share ideas or nodes setup to implement a gaussian blur on shader graph material, with a blur size parameter? Thanks!
I'm developing a custom gesture-based visionOS project that uses hand tracking with collision detection spheres on fingers to register user interactions through collision components. I'm experiencing a critical occlusion issue where collision detection spheres are intermittently occluded by the background/depth buffer, causing fingers to pass through the 3D model entities without registering interactions.
Detailed Description:
I have added 3D entities in an immersive scene with collision spheres attached to fingers for detecting user interactions.
Each sphere has:
CollisionComponent with sphere shape
Proper collision masks and groups configured
Real-time position updates from hand joint transforms
Each entity has:
InputTarget components to register collisions
The Issue:
When users move their fingers to the entity to interact, some collision spheres (particularly on the pinkie and ring fingers) become occluded and pass directly through the 3D model without triggering collision events.
Meanwhile, other fingers (like the index finger) continue to work correctly.
This appears to be a depth perception/z-buffer issue between the model entity and the hand tracking collision spheres
Questions:
Is there a recommended approach for maintaining consistent depth ordering between hand-tracking entities and 3D models in immersive spaces to prevent occlusion issues?
Should I be using AnchorEntities to anchor the entity to a plane or world position to establish a more stable depth reference?
Are there specific RenderingComponent or material settings that could help ensure collision entities maintain their depth priority and don't get occluded?
Could this be related to z-fighting when collision spheres and entity geometry occupy similar depth ranges? If so, what's the recommended depth bias approach?
Is there a better architectural approach for implementing interactions with custom hand gesture tracking that avoids these depth perception issues?
What Would Help:
Implementation guidance for ensuring reliable collision detection between hand-tracked entities through custom gestures and 3D models.
Best practices for depth management in immersive spaces with custom hand gesture tracking.
Sample code demonstrating stable hand-to-object interaction patterns.
Information about whether this is a known limitation or if there are specific APIs I should be leveraging
This issue is significantly impacting the reliability of our app experience, as users cannot consistently interact with all model components. Any guidance from Apple engineers or developers who have solved similar depth/occlusion challenges would be greatly appreciated.
Additional Context:
This is for a productivity-focused application where accuracy and reliability are critical.
Thank you for any assistance!
Topic:
Spatial Computing
SubTopic:
Reality Composer Pro
Tags:
ARKit
Reality Composer
AR / VR
visionOS
I'm currently implementing 180° / 360° immersive video for my app.
I easily implemented 360° by just applying VideoMaterial to flipped sphere.
But I'm stuck at 180°. I'm trying to implement by applying VideoMaterial to hemisphere (half sphere). I want to make VideoMaterial to be visible half front sphere and half back sphere transparent / clear.
Would there be any advice / information / idea to implement this? Your help would be grateful.