We can add ornaments to popovers shown by PresentationComponent, but I’m not sure if we should.
While working on the editor for entities in a Volume-based app, I had the idea to add ornaments to the presented views. The entire app exists inside a volume. A user can tap a item to present a popoverUI to edit it. This is displayed using the new PresentationComponent in visionOS 26.
Ornaments have a new attachment anchor option this year: .parent().
.ornament(attachmentAnchor: .parent(.top), ornament: {...})
This works well in the Simulator. We can add ornaments around this popover view just like we would with a window.
Unfortunately, when I run this on device I get a different experience. Any part of the ornament that overlaps with the popover content isn’t rendered correctly. Sometimes it entirely disappears, other times it becomes partially transparent.
We could use content alignment to try to make sure the ornament doesn’t overlap the popover content.
.ornament(attachmentAnchor: .parent(.top), contentAlignment: .bottom, ornament: {...})
This works sometimes–but not all the time. It’s not clear if this is a bug or not, because I’m not sure if we are even supposed to be able to use ornaments in this way. Here is my hierarchy:
An app opens as a Volume
Volume presenting a RealityView, with its own ornament using .scene() anchor
Multiple Entities with Presentation Component show an edit view
The view uses .parent() anchor to add ornaments.
What makes me unsure is that other methods for drawing UI in RealityView don’t seem to work with ornaments. For example, if I add an attachment to show a view with the ornament–even when I use the .parent() anchor–the ornament is anchor to the volume, not the attachment view.
So what do we think? Is this a rendering bug? Are ornaments intended to work with attachments and presentations?
General
RSS for tagDiscuss Spatial Computing on Apple Platforms.
Selecting any option will automatically load the page
Post
Replies
Boosts
Views
Activity
Is it possible to use a local wifi router connecting Vision Pro and Mac for developing? I tried from Unity and Xcode.
From Unity, the host app wouldn't open without WIFI (internet connection)
From Xcode, I can see the Vision Pro paired, but while try to run there's no device listed.
Any suggestions? Thanks a lot, /Ruiying
Topic:
Spatial Computing
SubTopic:
General
Hi I know it's possible to play equirectangular VR180 video either SBS or MV-HEVC. And for fisheye video, the only way I know is to convert it into an AIVU for playback.
Is there any way to directly play fisheye video using AVPlayer? Thanks a lot!
Hi everyone,
I’m building a visualization app for VisionPro that uses SharePlay and GroupActivities to explore datasets collaboratively.
I’ve successfully implemented the new SharedWorldAnchor feature, and everything works well with nearby, local participants.
However, I’m stuck on one point:
How can I share a world anchor with remote participants who join via FaceTime as spatial personas?
Apple’s demo app (where multiple users move a plane model around) seems to suggest that this is possible.
For context, I’m building an immersive app with Metal rendering.
Any guidance or examples would be greatly appreciated!
Thanks,
Jens
I want to open the control center in Vision Pro’s Xcode simulator. Can I open it? If I can, please tell me how to do it. Thank you.
So I am exporting a .usdc file from blender that already has some morph animations. The animations play well in blender but when I export I cannot seem to play them in RealityKit or RCP.
Entity.availableAnimations is an empty array.
Not of the child objects in the entity hierarchy has an animation library component with it.
Maybe I am exporting it wrong but I tried multiple combinations but doesn't seem to work.
Here are my export settings in blender
The original file I purchased is an FBX file that has the animation but when I try to directly get it in RealityConverter it doesn't seem to play animations.
Topic:
Spatial Computing
SubTopic:
General
Tags:
Reality Converter
RealityKit
Reality Composer Pro
visionOS
Here is my code in visionOS 2.3
NavigationSplitView {
List {
}
.navigationTitle("Passwords")
} detail: {
Text("Hello")
.navigationTitle("All")
}
The font size of "Passwords" and "All" are smaller than the ones in Passwords app.
I've been struggling with this for far too long so I've decided to finally come here and see if anyone can point me to the documentation that I'm missing. I'm sure it's something so simple but I just can't figure it out.
I can SharePlay our test app with my brother (device to device) but when I open a volumetric window, it says "not shared" under it. I assume this will likely fix the video sharing problem we have as well. Everything else works so smooth but SharePlay has just been such a struggle for me. It's the last piece to the puzzle before we can put it on the App Store.
Hey all,
I'm working on a visionOS app that captures live frames from the left and right cameras of Apple Vision Pro using cameraFrame.sample(for: .left/.right).
Apple provides documentation on encoding side-by-side frames into MV-HEVC spatial video using CMTaggedBuffer:
Converting Side-by-Side 3D Video to MV-HEVC
My question:
Is there any way to render tagged frames (e.g. CMTaggedBuffer with .stereoView(.leftEye/.rightEye)) live, directly to a surface in RealityKit or Metal, without saving them to a file?
I’d like to create a true stereoscopic (spatial) live video preview, not just render two images side-by-side.
Any advice or insights would be greatly appreciated!
Hi,
I am creating an ECS. With this ECS I will need to register several DragGesture.
Question: Is it possible to define DragGestures in ECS? If yes, how do we do that? If not, what is the best way to do that?
Question: Is there a "gesture" method that takes an array of gestures as a parameter?
I am interested in any information that can help me, if possible with an example of code.
Regards
Tof
Hi!
I'm currently trying to render another XR scene in front of a RealityKit one.
Actually, I'm anchoring a plane to the head with a shader to display for left/right eye side-by-side images. By default, the camera has a near plane so I can directly draw at z=0.
Is there a way to change the camera near plane? Or maybe there is a better solution to overlay image/texture for left/right eyes?
Ideally, I would layer some kind of CompositorLayer on RealityKit, but that's sadly not possible from what I know.
Thanks in advance and have a good day!
Hi !
I'm new on this forum, so if I need to update this post to have more info, or anything else, please let me know.
I'm using the Apple Vision Pro to develop some app (with unity). To demonstrate what the user see on the headset, I would like to mirror the view on a device (an iPad in this case). I managed to do this without any issue.
My problem is that, in the Vision Pro, I have an interface that the user can interact with. But I would like to be able to manage myself the interface on the iPad. What I mean is that the user can (or can't, doesn't matter) see the interface in the headset, and the interface is controlled by myself on the iPad.
Is there any way to do this ? Is this a question I should ask on unity's forum ? (I don't think so, because it should be related to the mirroring function non ?)
Topic:
Spatial Computing
SubTopic:
General
In WWDC25 session What’s new for the spatial web, the presenter showed creating an immersive environment for a web page by adding to the page's HEAD section
<link rel="spatial-backdrop" href="office.usdz" environmentmap="lighting.hdr">
My first attempt failed, and I am trying to track down why.
Before I search all the potential failure paths, I wanted to ask the community,
Is this feature available in the latest visionOS 26 beta?
I haven't seen anyone talk about their use of the feature yet.
Topic:
Spatial Computing
SubTopic:
General
Hi ,
I'm struggling with visionOS window management and need help with closing child windows programmatically.
App Structure
My app has a Main-Sub window hierarchy:
AWindow (Home/Main)
BWindow (Main feature window)
CWindow (Tool window - child of BWindow)
Navigation flow:
AWindow → BWindow (switch, 1 window on screen)
BWindow → CWindow (opens child, 2 windows on screen)
I want BWindow and CWindow to be separate movable windows (not sheet/popover) so users can position them independently in space.
The Problem
CWindow doesn't close when BWindow closes by tapping the X button below the app (next to the window bar)
User clicks X on BWindow → BWindow closes but CWindow remains
CWindow becomes orphaned on screen
Can close CWindow programmatically when switching BWindow back to AWindow
App launch issue
After closing both windows, CWindow is remembered as last window
Reopening app shows only CWindow instead of BWindow
User gets stuck in CWindow with no way back to BWindow
I've Tried Environment dismissWindow in cleanup but its not working.
// In BWindow.swift
.onDisappear {
if windowManager.isWindowOpen("cWindow") {
dismissWindow(id: "cWindow")
}
}
My App Structure Code Now
// in MyNameApp.swift
@main
struct MyNameApp: App {
var body: some Scene {
WindowGroup(id: "aWindow") {
AWindow()
}
WindowGroup(id: "bWindow") {
BWindow()
}
WindowGroup(id: "cWindow") {
CWindow()
}
}
}
// WindowStateManager.swift
class WindowStateManager: ObservableObject {
static let shared = WindowStateManager()
@Published private var openWindows: Set<String> = []
@Published private var windowDependencies: [String: String] = [:]
private init() {}
func markWindowAsOpen(_ id: String) {
markWindowAsOpen(id, parent: nil)
}
func markWindowAsClosed(_ id: String) {
openWindows.remove(id)
windowDependencies[id] = nil
}
func isWindowOpen(_ id: String) -> Bool {
let isOpen = openWindows.contains(id)
return isOpen
}
func markWindowAsOpen(_ id: String, parent: String? = nil) {
openWindows.insert(id)
if let parentId = parent {
windowDependencies[id] = parentId
}
}
func getParentWindow(of childId: String) -> String? {
let parent = windowDependencies[childId]
return parent
}
func getChildWindows(of parentId: String) -> [String] {
let children = windowDependencies.compactMap { key, value in
value == parentId ? key : nil
}
return children
}
func setNextWindowParent(_ parentId: String) {
UserDefaults.standard.set(parentId, forKey: "nextWindowParent")
}
func getAndClearNextWindowParent() -> String? {
let parent = UserDefaults.standard.string(forKey: "nextWindowParent")
UserDefaults.standard.removeObject(forKey: "nextWindowParent")
return parent
}
func forceCloseChildWindows(of parentId: String) {
let children = getChildWindows(of: parentId)
for child in children {
markWindowAsClosed(child)
NotificationCenter.default.post(
name: Notification.Name("ForceCloseWindow"),
object: nil,
userInfo: ["windowId": child]
)
forceCloseChildWindows(of: child)
}
}
func hasMainWindowOpen() -> Bool {
let mainWindows = ["main", "bWindow"]
return mainWindows.contains { isWindowOpen($0) }
}
func cleanupOrphanWindows() {
for (child, parent) in windowDependencies {
if isWindowOpen(child) && !isWindowOpen(parent) {
NotificationCenter.default.post(
name: Notification.Name("ForceCloseWindow"),
object: nil,
userInfo: ["windowId": child]
)
markWindowAsClosed(child)
}
}
}
}
// BWindow.swift
struct BWindow: View {
@Environment(\.dismissWindow) private var dismissWindow
@ObservedObject private var windowManager = WindowStateManager.shared
var body: some View {
VStack {
Button("Open C Window") {
windowManager.setNextWindowParent("bWindow")
openWindow(id: "cWindow")
}
}
.onAppear {
windowManager.markWindowAsOpen("bWindow")
}
.onDisappear {
windowManager.markWindowAsClosed("bWindow")
windowManager.forceCloseChildWindows(of: "bWindow")
}
.onChange(of: scenePhase) { oldValue, newValue in
if newValue == .background || newValue == .inactive {
windowManager.forceCloseChildWindows(of: "bWindow")
}
}
}
}
// CWindow.swift
import SwiftUI
struct cWindow: View {
@ObservedObject private var windowManager = WindowStateManager.shared
@State private var shouldClose = false
var body: some View {
// Content
}
.onDisappear {
windowManager.markWindowAsClosed("cWindow")
NotificationCenter.default.removeObserver(
self,
name: Notification.Name("ForceCloseWindow"),
object: nil
)
}
.onChange(of: scenePhase) { oldValue, newValue in
if newValue == .background {
}
}
.onAppear {
let parent = windowManager.getAndClearNextWindowParent()
windowManager.markWindowAsOpen("cWindow", parent: parent)
NotificationCenter.default.addObserver(
forName: Notification.Name("ForceCloseWindow"),
object: nil, queue: .main) { notification in
if let windowId = notification.userInfo?["windowId"] as? String, windowId == "cWindow" {
shouldClose = true
}
}
}
.onChange(of: shouldClose) { _, newValue in
if newValue {
dismissWindow()
}
}
}
The logs show everything executes correctly, but CWindow remains visible on screen.
Questions
Why doesn't dismissWindow(id:) work in cleanup scenarios?
Is there a proper way to create a window relationships like parent-child relationships in visionOS?
How can I ensure main windows open on app launch instead of tool windows?
What's the recommended pattern for dependent windows in visionOS?
Environment: Xcode 16.2, visionOS 2.0, SwiftUI
Hi,
I'm trying to correct the lens distortion in frames provided by Enterprise API camera frame provider. The frames provided seem to have only in/extrinsics info, but not the distortion lookup table.
Is there some magic setting, or function to do that (I can't seem to find anything like this)? Or is there a way to use AVCameraCalibrationData together with provider?
Hello experts, and question seekers,
I have been trying to get Gaussian splats working with RealityKit, however it seems not to work out for me.
The library I use for Gaussian splatting: https://github.com/scier/MetalSplatter
My idea was to use the renderers provided by RealityKit (aka RealityRenderer) https://developer.apple.com/documentation/realitykit/realityrenderer and the renderer provided by MetalSplatter (aka. SplatRenderer) https://github.com/scier/MetalSplatter/blob/main/MetalSplatter/Sources/SplatRenderer.swift
Then with a custom render pipeline, I would be able to compose the outputs of the renderers, enabling the possibility, for example to build immersive scenery with realistic environment scans, as Gaussian splats, and RealityKit to provide the necessary features to build extra scenery around Gaussian splats, eg. dynamic 3D models inside Gaussian splats.
However the problem is, as of now I am not able to do that with the current implementation of RealityRenderer.
It seems to be, that first RealityRenderer is supposed to be an API, just to render colour information onto a texture, which in first glance might be useful, but misses important information, such as for example depth, and stencil information.
Second issue is, even with that in mind, currently I am not able to execute RealityRenderer.updateAndRender, due to the following error messages:
Could not resolve material name 'engine:BuiltinRenderGraphResources/Common/realityRendererBackground.rematerial' in bundle at '/Users//Library/Developer/CoreSimulator/Devices//data/Containers/Bundle/Application//.app'. Loading via asset path.
exiting spatial tracking service update thread because wait returned 37”
I was able to build a custom Metal view with UIViewRepresentable, MTKView, and MTKViewDelegate, enabling me to build a custom rendering pipeline, by utilising some of the Metal developer workflows.
Reference: https://developer.apple.com/documentation/xcode/metal-developer-workflows/
Inside draw(in view: MTKView), in a class derived by MTKViewDelegate:
guard let currentDrawable = view.currentDrawable else {
return
}
let realityRenderer = try! RealityRenderer()
try! realityRenderer.updateAndRender(deltaTime: 0.0, cameraOutput: .init(.singleProjection(colorTexture: currentDrawable.texture)), whenScheduled: { realityRenderer in
print("Rendering scheduled")
}, onComplete: { RealityRenderer in
print("Rendering completed")
})
Can you please tell me, what I am doing wrong?
Is there any solution, that enables me to use RealityKit with for example Gaussian splats?
Any help is greatly appreciated.
All the best,
Ethem Kurt
I've tried following apple's documentation to apply a video material on a Model Entity, but I have encountered a compile error while attempting to specify the Spatial Audio type.
It is a 360 video on a Sphere which plays just fine, but the audio is too quiet compared to the volume I get when I preview the video on Xcode. So I tried tried to configure audio playback mode on the material but it gives me a compile error:
"audioInputMode' is unavailable in visionOS
audioInputMode' has been explicitly marked unavailable here
RealityFoundation.VideoPlaybackController.audioInputMode)"
https://developer.apple.com/documentation/realitykit/videomaterial/
Code:
let player = AVPlayer(url: url)
// Instantiate and configure the video material.
let material = VideoMaterial(avPlayer: player)
// Configure audio playback mode.
material.controller.audioInputMode = .spatial // this line won’t compile.
VisionOS 2.4, Xcode 16.4, also tried Xcode 26 beta 2.
The videos are HEVC MPEG-4 codecs.
Is there any other way to do this, or is there a workaround available?
Thank you.
Hello, I'm adding a CollisionComponent to an entity in RealityView. CollisionComponent requires that a Mesh must be provided as a reference for collision detection. However, in order to achieve more accurate detection, I hope that this Mesh resource is a geometric shape of a USDZ model. Is there any way to make it happen? Thank you!
Hi team I don't know from this error has popped up any suggestions please
Thanks
Zippy Games
I am experiencing an issue where USDZ files exported from Blender do not display textures when opened in Apple Vision Pro Quick Look. Instead of the expected materials, the model appears completely white, as if the textures are missing or not being recognized by the rendering engine.
Topic:
Spatial Computing
SubTopic:
General