Posts under Spatial Computing topic

Post

Replies

Boosts

Views

Activity

Improving metric scale consistency of ARWorldTrackingConfiguration on devices without LiDAR
We build a medical scanning app. A practitioner walks a slow orbit (~1 m radius, two laps, 25–35 s) around a standing subject while an ARSession runs, and we capture ~50 high-resolution stills with captureHighResolutionFrame. The ARKit camera poses for those frames are later used to set the metric scale of a photogrammetry reconstruction. We need the resulting scale to be within about 2 % of true. What we observe, measured against a printed A4 sheet on the floor in every scan: Devices with LiDAR (iPhone 16 Pro Max, iPad Pro), using the sceneDepth frame semantic as a check: scale within ±0.8 % across 11 sessions. Devices without LiDAR (iPhone 13, iPad 9th and 11th gen, plus non-LiDAR runs on the Pro devices): the world-frame metre is on average 2–3 % short of true, and varies by a further ±2 % from one session to the next, across 24 sessions in four rooms. Within a session the scale is stable: lap-to-lap variation is under 0.4 %. The error appears to be set during initialisation and held. Nothing we record about capture correlates with the error: time spent before capture, distance walked, vertical camera movement, orbit radius, rotation speed, tracking state, or the residual of our pose fit. Questions: Is there any session setup, initialisation practice, or motion during the first seconds of tracking that is known to improve the metric scale estimate on devices without LiDAR? Does the choice of video format, resolution or frame rate affect scale estimation? Is there any signal, beyond ARCamera.trackingState, that indicates how well scale has been determined for a session, so we could flag sessions rather than accept them? Does anything in the configuration above, in particular locking exposure on the capture device or using the high-resolution video format, work against tracking quality or scale?
1
0
532
1m
Presenting the system share sheet from a full ImmersiveSpace on visionOS 26
We are building a visionOS 26 app (deployment target visionOS 26.0). Inside a full immersive space, the user takes a spatial photo. The photo is shown on a RealityView attachment. We need the system share sheet( AirDrop, Messages, Mail, Copy, Save Image, and the item preview with Done) to appear when the user taps Share on that attachment. The immersive space is: ImmersiveSpace(id: "ImmersiveView") { ImmersiveView() } .immersionStyle(selection: $panoImageView, in: .full) .upperLimbVisibility(.visible) The review UI is a RealityView attachment, not a window: Attachment(id: "panoramaFrameCaptureReview") { PanoramaFrameCaptureReviewView(model: frameCapture) { guard let path = frameCapture.shareURL?.path else { return } dismissWindow(id: "PanoramaPhotoShare") Task { @MainActor in try? await Task.sleep(nanoseconds: 180_000_000) openWindow(id: "PanoramaPhotoShare", value: path) } } } The shared item is a stereo HEIC written with Image I/O (kCGImagePropertyGroupTypeStereoPair, left/right images, HEIF camera extrinsics and SimplifiedPinhole intrinsics). The same data displays correctly with ImagePresentationComponent and saves to Photos as a spatial photo. Share is the only part that fails. Attempt 1. ShareLink on the attachment Tapping Share does nothing. On entry to the immersive space the system logs: Presentations are not currently supported in Immersive contexts. Attempt 2. UIActivityViewController from the attachment’s hosting controller let activity = UIActivityViewController(activityItems: [fileURL], applicationActivities: nil) var presenter: UIViewController = self while let parent = presenter.parent { presenter = parent } presenter.present(activity, animated: true) presenter is UIHostingController<RootView>. The console then says: Skipping popover presentations transition, missing window or scene for <UIActivityViewController>. Failed to request default share mode for fileURL:...heic error:Error Domain=NSOSStatusErrorDomain Code=-10814 No share UI appears. presentedViewController is non-nil, so a “did it present?” check does not detect the failure. Attempt 3. A real WindowGroup whose only job is to present UIActivityViewController WindowGroup(id: "PanoramaPhotoShare", for: String.self) { path in if let path = path.wrappedValue { PanoramaPhotoShareWindow(path: path) } } .windowStyle(.plain) .defaultSize(width: 20, height: 20) .defaultWindowPlacement { _, _ in WindowPlacement(.utilityPanel) } struct PanoramaPhotoShareWindow: View { let path: String @Environment(.dismissWindow) private var dismissWindow var body: some View { PanoramaPhotoSharePresenter(url: URL(fileURLWithPath: path)) { dismissWindow(id: "PanoramaPhotoShare") } .frame(width: 8, height: 8) .background(.clear) } } final class PanoramaPhotoShareController: UIViewController { override func viewDidAppear(_ animated: Bool) { super.viewDidAppear(animated) guard view.window != nil, let url else { return } let activity = UIActivityViewController(activityItems: [url], applicationActivities: nil) activity.completionWithItemsHandler = { _, _, _, _ in self.onFinish?() } present(activity, animated: true) } } Tapping Share opens this window and the app crashes immediately. We removed the feature so users are not crashed. An earlier build that also put a ShareLink in that window logged: Attempt to present <UIActivityViewController> on <UIHostingController<...PanoramaPhotoShareWindow...>> which is already presenting <UIActivityViewController>. Question What is the supported way, on visionOS 26, to present the system share sheet while a .full ImmersiveSpace stays open? If a window is required, how should that window present the sheet without crashing, and without showing a second window that only contains another Share button?
0
0
246
1d
visionOS 27 beta 4 (24M5326g) stuck: update fails with MSU 806 MADownloadDaemonNotReady, all Apple services hang
My Apple Vision Pro is still on visionOS 27.0 beta 4 (24M5326g) and can't update or use any Apple service. Safari works. Software Update offers visionOS 27.2 (24N6120) but stays at "Update Requested". The OTAUpdate log shows: MobileSoftwareUpdateErrorDomain 806 "Failed to download update brain asset, MADownLoadResult:MADownloadDaemonNotReady". App Store can't connect, Apple TV and Games say a network connection is required, iCloud doesn't work, and Media & Purchases hangs at "Signing in to Apple Media Services". Other devices on the same network work normally. Reset All Settings didn't help. devicectl device sysdiagnose fails with CoreDeviceCLISupport.DiagnoseError 0. I don't have a Developer Strap. Has anyone recovered a Vision Pro stuck on an early beta like this without one? Filed as FB25386556.
0
0
370
2d
Reality kit Entities Appearing to Lag in a Full or Progressive Style Immersive Space When Opened with Environment Turned On
PLATFORM AND VERSION Vision OS Development environment: Xcode 16.2, macOS 15.2 Run-time configuration: visionOS 2.3 (On Real Device, Not simulator) Please someone confirm I'm not crazy and this issue is actually out of my control. Spent hours trying to fix my app and running profiles because thought it was an issue related to my apps performance. Finally considered chance it was issue with API itself and made sample app to isolate problem, and it still existed in it. The issue is when a model entity moves around in a full space that was launched when the system environment immersion was turned up before opening it, the entities looks very choppy as they move around. If you take off the headset while still in the space, and put it back on, this fixes it and then they move smoothly as they should. In addition, you can also leave the space, and then turn the system environment immersion all the way down before launching the full space again, this will also make the entity moves smoothly as it should. If you launch a mixed immersion style instead of a full immersion style, this issue never arrises. The issue only arrises if you launch the space with either a full style, or progressive style, while the system immersion level is turned on. STEPS TO REPRODUCE https://github.com/nathan-707/ChoppyEntitySample Open my test project, its a small, modified vision os project template that shows it clearly. otherwise: create immersive space with either full or progressive immersion style. setup a entity in kinematic mode, apply a velocity to it to make it pass over your head when the space appears. if you opened the space while the Apple Vision Pros system environment was turned up, the entity will look choppy. if you take the headset off while in the same space, and put it back on, it will fix the issue and it will look smooth. alternatively if you open the space with the system immersion environment all the way down, you will also not run into the issue. Again, issue also does not happen if space launched is in mixed style.
2
1
1.1k
2d
Immersive space sometimes opens with a broken world origin (world anchors fail with error 200, recenter times out)
Hi all, We're seeing an intermittent issue in a native SwiftUI + RealityKit app on Vision Pro (visionOS 27.0.1, still there on 27.2). When we open a mixed ImmersiveSpace, roughly 1 in 10–30 opens, everything ends up offset from where it should be. Sometimes it's 50 cm, sometimes several meters, sometimes it's rotated. Holding the Digital Crown to recenter fixes it immediately. We are attaching things to the camera, like we have a window following the user and that is offset in these rare cases which makes the app not usable. In the device logs, a good open looks like this: backboardd (recenter) Setting new world origin with anchor ... backboardd ARWorldTrackingProcessor: Added world anchor ... successfully (x4) backboardd (recenter) Finished (error: (null)) setting set new world origin ... A bad open looks like this: backboardd (recenter) Setting new world origin with anchor ... arkitd SLAMAnchorUpdate: Failed to add anchor request ... NSPOSIXErrorDomain Code=4 backboardd ARWorldTrackingProcessor: Failed to add world anchor ... com.apple.arkit Code=200 "Adding an object failed." (x4) backboardd (recenter) timeout exceeded. Forcing fade in. Like whenever the bad state occurs it is because adding a world anchor seems to fail. I haven't tested with visionOS26 but the same app didnt run into that problem ever. So the system seems to fail adding its own origin anchors, gives up after 2 seconds and shows the space anyway. Has anyone else run into this? And is there any way to detect it from the app or trigger a recenter in code? Thanks!
2
0
624
5d
High fidelity gaming on Apple Vision Pro
I'm looking to find the best setup for a game on AVP that gives the best quality environment with gameplay but there seems to be a mix of possible solutions each with their own drawback. My ideal scenario is a high fidelity room/environment while making use of the PS VR2 Sense controllers to track the user's hands and give them more inputs via the controllers. Eg they hold a weapon tracked to the controllers with the buttons able to use the weapon, shooting and hitting enemies in the room. I want to aim more for highest qualtiy gaming than cartoony so opted for foveated streaming from the Mac as my ideal, but this seems limited to only having some base inputs from the hands and no controller inputs for hand and positional tracking from the controller movement (as they need the data from the Vision Pro camera but the game is technically running on the Mac and streamed to the VP) The only real solution for what I want here is to go via RealityKit as a native Vision Pro app but this reduces the game quality I had in mind as the entire thing is now rendering on the VP instead of the Mac. Am I missing something here or is this a real limitation today? To reitarate I'm looking for: highest fidelity game quality (rendering as much as possible on the Mac) tracking the VR2 Sense controller's positions (to track a weapon) in 3D space
1
0
1k
6d
Async Entity(contentsOf:) is 15 to 20x slower while an ARView is rendering
While a non-AR ARView is on screen and rendering, await Entity(contentsOf:) takes about 26 s to load a 4.5 MB USDZ with 300 textured meshes. The same file loads in about 1.5 s with the synchronous Entity.load(contentsOf:), or with the async API while the view is hidden. iPhone 16 Pro, iOS 27.2Mac Catalyst, macOS 27.2await Entity(contentsOf:), view rendering22 to 26 s26 sEntity.load(contentsOf:), view rendering1.6 s1.1 to 1.7 sawait Entity(contentsOf:), view hidden1.4 s1.3 to 1.5 sModelEntity(contentsOf:) and Entity.loadAsync(contentsOf:) behave the same. A .reality file loads fast either way, and the iOS Simulator doesn't show it. A Time Profiler trace shows the main thread blocked in re::RenderThread::waitForFramePacing on every frame, so the @MainActor import only advances between frames. Neither workaround fits loading content into a scene the user is looking at: the sync load blocks the main thread, and hiding the view blanks it. Filed as FB25340264, with a sample project.
0
0
155
6d
Reality Composer Pro Assistant: "Authorization failed" when adding custom Azure AI Foundry OpenAI provider
Hello everyone, I am trying to configure a custom model provider inside the new Reality Composer Pro Assistant settings panel using an LLM hosted via Azure AI Foundry. Despite verifying my credentials and testing multiple base URL structures, I am persistently encountering the following error: Authorization failed — check your API key My Configuration Details: Target Model Deployment: gpt-5.1-codex-mini (GlobalStandard deployment type) Azure AI Foundry Endpoint Pattern: https://azure.com API Key Header Used: api-key What I Have Tried: Utilizing the Azure OpenAI format: https://<project-name>.services.ai.azure.com/openai/v1/responses Verifying the API key explicitly with headers like api-key and Authorization (testing both raw keys and Bearer <key> prefixes). It appears that Reality Composer Pro's validation step might be calling a standard GET /v1/models route or structurally modifying the outbound request in a way that causes Azure's gateway architecture to reject it with a 401 response code. Has anyone successfully mapped an Azure AI Foundry endpoint directly to the Assistant interface? If so, what is the exact base URL syntax and header structure expected by the desktop client to pass authorization? Thanks in advance for any insights!
0
0
433
1w
Ultra-wide camera in world tracking
My app previews cinema lenses by cropping the AR feed, so I can only simulate lenses tighter than the main camera's ~24mm-equivalent, meaning wider cinema lenses are impossible to accurately preview. ARConfiguration.VideoFormat exposes captureDeviceType, but world-tracking formats only ever list the wide-angle module. Is running world tracking on (or alongside) the ultra-wide camera fundamentally infeasible? Or just a case of not enough Feedback Assistant requests?
3
2
1.8k
1w
visionOS : any non-nil preferredSurroundingsEffect enables a light-spill halo around VideoPlayerComponent video
Hi all — sharing a platform behavior I isolated while building a custom immersive video player, in case others hit it (and to ask whether it's intended). Finding On visionOS 27 (Xcode 27), an entity playing video via VideoPlayerComponent(videoRenderer:) gets a light-spill "screen glow" halo around its edges whenever the immersive space declares any non-nil preferredSurroundingsEffect. The halo follows the video's colors frame by frame. With no surroundings effect (nil), there is no halo. Quick repro Progressive immersive space + VideoPlayerComponent playing. Add .preferredSurroundingsEffect(.dim(intensity: 0.0)) → halo appears (even though the effect is visually a no-op). Remove it → halo disappears. Same with .colorMultiply(_:) and preset effects. Things I ruled out on device (Vision Pro) ImmersiveContentBrightness (.custom(1.0) vs .automatic): no difference. VideoPlayerComponent configuration (.stereo, spatial video mode, isPassthroughTintingEnabled) and component attach timing: no difference. Presenting the same video via VideoMaterial: no halo. Bright white SwiftUI content: no halo. Questions Is this coupling intended? The preferredSurroundingsEffect docs don't mention any effect on video playback. Is there a public way to keep a surroundings effect (e.g. room dimming) while avoiding the video light spill? I found none on VideoPlayerComponent. Current workaround: don't declare any surroundings effect in the modes where the halo is undesirable. Also filed via Feedback Assistant(FB25130748). Curious if anyone else has seen this or knows a switch.
2
1
2.9k
1w
Foveated streaming QR code Authentication alternative
I was wondering whether there is a way to authenticate a CloudXR connection without using a QR code in a local deployment scenario. If the server is running in a headless environment, QR code-based authentication would not be practical. I reviewed the documentation but could not find any information on alternative authentication methods. Is a non-QR authentication workflow similar to the remote case supported?
2
0
1.5k
1w
Reality Composer Pro exporting larger and larger .reality files for each reference usage
We've noticed something kind of odd related to the exported .reality file in Reality Composer Pro 3. Our exported .reality file seems to be increasing a lot in size just by referencing an imported asset or another entity in the scene. This contradicts everything we previously knew about references. Essentially it seems like our assets are being copied or duplicated on each export. Before you dive into the description of the issue, yes, I've already created feedback about this. I think it's a bug, but am willing to be corrected if this is an expected, if somewhat odd, behavior. The feedback number is FB25253483 I did a quick test with a single model (Earth.usdz from the RCP2 content library. Project A: RCPExportDupeIssues Phase 01 Created new project in RCP 3 Imported the file and added it to the default world entity Generated an Xcode linked project and used RCP3 to run the app in the simulator (this runs through the entire .reality export process.) Result: 48.4 MB Phase 02 Created AnotherScene entity Added the imported earth model to it Used RCP3 to run the app in the simulator Result: 64.5 MB Phase 03 I thought maybe if I added the earth import to a single entity and added that entity to the other two scenes, I might get some savings. This also increased the overall export size. Created EarthEntity Added the imported earth model to it Used RCP3 to run the app in the simulator Result: 80.7 MB Overall result: an RCP3 project with a single imported Earth asset, used in three otherwise empty scenes, ends up with a massive 80 MB. I even tried deleting the Earth import reference in the first two scenes. Even with only the Earth entity present, the export was still 80.7 MB. Project B: RCPExportNoDupes I tried a second project without ever adding the Earth import file to the first two scenes. I created a dedicated EarthEntity and only added that to the other two scenes. Created new project in RCP 3 Created EarthEntity Imported the file and added it to EarthEntity Added `EarthEntity to the default world entity Created AnotherScene entity and added `EarthEntity to it Generated an Xcode linked project and used RCP3 to run the app in the simulator (this runs through the entire .reality export process.) Result: 80.7 MB Apple: I provided zipped versions of the exports in the feedback item (FB25253483)
1
0
857
1w
RCP .reality file in iPadOS with behavior tree
I am having troubles with a .reality file in iPadOS 27.0.1. I have a Reality Composer Pro 3 project with an Animation Graph and Behavior Tree. The animation simply switches between a character walking and running, and the Behavior Tree simply changes the boolean to trigger the running state, waits, flips the boolean back, waits, and repeats. When I run the Simulate in RCP 3, it behaves as I expected - walks, runs, walks, runs, etc. I export the World scene as a .reality file, and open the .reality file in Xcode on macOS 27.2, and it behaves as I expected. But when I load the .reality file in an iPadOS app (iPadOS 27.0.1), the character does not switch between states. It starts the walking animation, but it never switches to the running animation. It appears the Behavior Tree isn't working as expected. Any idea what is wrong? Should I switch the iPad to the iPadOS 27 betas? Should I set the minimum OS target for the app to iPadOS 27? Is the Behavior Tree a known limitation with iOS or iPadOS 27?
1
0
596
1w
Metal Compositor Service & Persona (VisionOS)
Hello, I'm currently trying to make a collaborative app. But it just works only on Reality View, when I tried to use Compositor Layer like below, the personas disappeared. ImmersiveSpace(id: "ImmersiveSpace-Metal") { CompositorLayer(configuration: MetalLayerConfiguration()) { layerRenderer in SpatialRenderer_InitAndRun(layerRenderer) } } Is there any potential solution too see Personas in Metal view? Thanks in advance!
3
0
2k
2w
Permitted research logging of latestFocusRegion in Foveated Streaming provider?
We are evaluating a consented research study in which a participant freely looks at a painting streamed from a desktop Unity/OpenXR application to Apple Vision Pro. A custom Foveated Streaming provider extension would read FoveatedStreamingProviderContext.latestFocusRegion during an active streaming session and classify the approximate focus direction into a painting-region identifier. No pinch or other action would be required. Is an approved provider permitted to export and retain timestamped painting-region identifiers for research analysis? If so, may it also retain the focus region’s direction, distance, and timestamp, or only the derived region? Is there a supported path for sending the derived data to the paired app or remote endpoint? What entitlement-review, consent, privacy, data-use, and distribution conditions apply? We understand the provider entitlement requires Apple approval and that ordinary visionOS apps do not receive passive gaze data. We would request the entitlement only if this research use is allowed. We are seeking guidance on permitted use before implementing or collecting data.
0
0
901
2w
Main Camera development request blocked by Individual-to-Organization conversion
We are developing MadnessBot as a business-only Vision Pro automotive inspection app for professional mechanics and repair shops, intended for private business distribution. Main Camera access is essential for analyzing the vehicle and component in front of the technician. A native development app is running on our physical headset, and chat plus OBDLink CX diagnostics have been tested. Our existing Apple Developer Program membership is enrolled as Individual. We contacted Developer Support on October 6, 2026, with a follow-up on October 9, for assistance converting it to Organization. The conversion form asks for the type of tax ID historically associated with the Individual membership; the Account Holder does not know that record and has asked Support to verify it before selecting an answer. The enterprise API request form remains inaccessible until organization enrollment is confirmed. We have not submitted a camera entitlement request yet. Our planned initial request is Development Only Main Camera access for registered test devices. Feedback reference: FB25132939. Could the Enterprise APIs for visionOS team help route this prerequisite blocker to membership support, or advise the correct escalation path so we can complete conversion and apply for Development Only access?
0
0
1.6k
3w
RoomPlan: placed RealityKit content does not align with the scanned room
I scan a room with RoomPlan API, then keep the same ARSession running (stop(pauseARSession: false)) and place RealityKit entities using the CapturedRoom wall transforms (wall.transform, wall.dimensions), parented to an anchor at the world origin. The camera position stays continuous, so tracking was not reset. But the entities do not sit on the real walls. The whole room appears shifted. What is the correct way to place content in the live AR session so it aligns with the scanned room?
1
0
1.9k
3w
RealityKit VideoMaterial renders 8-bit HEVC HLG/BT.2020 360° video with a purple tint in the visionOS Simulator (device is fine)
We paint a looping 360° video on the inside of a sphere with VideoMaterial(avPlayer:): an AVQueuePlayer plus AVPlayerLooper, a ModelEntity with an inward-facing equirectangular mesh, inside a .mixed ImmersiveSpace. No shaders, no VideoPlayerComponent. We could not find an official Apple sample that puts a VideoMaterial on a 360° sphere; this is the plain pattern from the VideoMaterial(avPlayer:) reference and the one used in threads like 734081. On Apple Vision Pro hardware the video renders correctly. In the visionOS Simulator (27.0, 26.5 and earlier 26.x, Xcode 27.0) the whole frame has a strong purple/magenta cast. Audio and timing are fine. The video (which we cannot share, so here is its metadata, image sample of issue on simulator provided below): MP4, HEVC Main, 8-bit 4:2:0, 3840x1920, tagged BT.2020 primaries, HLG transfer (ARIB STD-B67), BT.2020 non-constant matrix, both in the colr atom and in the SPS VUI. ffprobe -show_format -show_streams (trimmed): [STREAM] index=0 codec_name=hevc profile=Main codec_type=video codec_tag_string=hvc1 width=3840 height=1920 pix_fmt=yuv420p color_range=tv color_space=bt2020nc color_transfer=arib-std-b67 color_primaries=bt2020 r_frame_rate=30000/1001 avg_frame_rate=679200/22663 duration=37.771667 bit_rate=25030607 [SIDE_DATA] side_data_type=Stereo 3D type=2D horizontal_field_of_view=360000/1000 [/SIDE_DATA] [/STREAM] [STREAM] index=1 codec_name=aac profile=LC codec_type=audio codec_tag_string=mp4a sample_fmt=fltp sample_rate=48000 channels=2 channel_layout=stereo duration=37.758333 bit_rate=128176 [/STREAM] [FORMAT] nb_streams=2 format_name=mov,mp4,m4a,3gp,3g2,mj2 duration=37.778333 bit_rate=25160348 TAG:major_brand=mp42 TAG:minor_version=1 TAG:compatible_brands=isommp41mp42 [/FORMAT] AVFoundation's view of the track, from CMFormatDescriptionGetExtension on the loaded video track: primaries=ITU_R_2020 transfer=ITU_R_2100_HLG matrix=ITU_R_2020 bitsPerComponent=8 The playback code, reduced to the essentials: let item = AVPlayerItem(url: fileURL) let looper = AVPlayerLooper(player: player, templateItem: item) // retained player.actionAtItemEnd = .advance let material = VideoMaterial(avPlayer: player) let sphere = ModelEntity( mesh: try equirectSphere(radius: 12), // lat/long grid, normals inward, materials: [material] // UVs (u, 1 - v), inward winding ) sphere.position = [0, 1.5, 0] rootEntity.addChild(sphere) // rootEntity was added in RealityView { } player.play() What we have ruled out in a minimal sample project: Apple's own 360° HLS example (360 Lighthouse, HEVC Main 10, 10-bit, BT.709 SDR) plays with correct colours in the same sphere, same code, same simulator session. Rewriting our file's colr atom to BT.709 (three bytes, no re-encode) is honoured by AVFoundation (the track's CMFormatDescription then reports ITU_R_709_2 for primaries, transfer and matrix) but the tint stays. So the remaining differences between the failing file and Apple's working stream are 8-bit Main vs 10-bit Main 10, HLG/BT.2020 in the bitstream VUI vs BT.709, and local MP4 vs HLS. The minimal sample project (without the video) is available on request. Is this a known limitation of the simulator's VideoMaterial path with 8-bit HLG content, and is there a recommended way to get device-equivalent colour in the simulator? Feedback filed as FB24997147. Here is a sample of the result on the simulator of a cement column, painted white, but rendered purple on simulator:
1
0
3.3k
Oct ’26
Mac spatial preview feature: Supported with Blender?
Hello! I am relatively new to dev for Apple Vision Pro, however I have owned one since launch and love it. When I heard that we were going to be able to preview real-time, 3D models from the Mac I couldn’t believe it! This is one of my major workflows and I have been using 3rd party workarounds (that don’t work well). My question is, is there any word on supported 3D modeling apps like Blender or Cinema 4D? Or will there be a way to implement this preview feature with any app via a protocol? Any documentation would be really helpfu! Thank you, and continue the great additions to Apple Vision Pro!
2
0
2.8k
Oct ’26
Revealed keyboard drifts in progressive immersion, and no immersion style gives windows-in-front + Crown + passthrough cutouts
Environment: Apple Vision Pro, visionOS 27.0.1 (24M372). Magic Keyboard, with Settings › Awareness & Safety › Reveal Keyboards set to Always. Our app puts the user in an opaque virtual room while they work at their real desk, using app windows and Mac Virtual Display. A virtual desk is placed on the real desk. It is world-locked: positioned once from a PlaneAnchor (PlaneDetectionProvider), then a plain entity at fixed ImmersiveSpace coordinates. No AnchorEntity or WorldAnchor is involved. We offer two modes: .mixed, and .progressive for Digital Crown control. Each gives us only half of what desk work needs. 1. The revealed keyboard drifts in progressive immersion (FB25006789) In .progressive, the physical keyboard shown by keyboard awareness does not stay registered with the virtual desk. After the keyboard has been out of view for a few seconds (for example, while looking at a Mac Virtual Display window), it looks "floaty" when you look back: as the head turns, it slides against the desk. Turning right, the keyboard moves left farther than the desk does. What we checked: It happens in .progressive at any Crown amount, and in .full. With identical content in .mixed (no app passthrough cutout, so again only keyboard awareness shows the keyboard), it is much steadier. On 27.0.1, PlaneAnchor and DeviceAnchor coordinateSpace(correction: .rendered) are exactly equal to their .none transforms: 0 mm, 0°, scale 1. That held over 835 per-frame samples with ±50° yaw and up to 36 cm of head movement. So the rendered correction gives the app nothing to compensate with. The app's desk transform is constant while the drift is visible. upperLimbVisibility(.automatic) vs .visible and a .portrait progressive aspect ratio make no difference. Possibly related: https://developer.apple.com/forums/thread/796557 (FB19666209). 2. No immersion style combines what desk work needs (FB25006837) We need, at the same time: windows (our own and Mac Virtual Display) never covered by the room's walls, furniture or trees; the Digital Crown to bring the real surroundings back in; a passthrough cutout over the real desk, and a well-registered keyboard. .mixed: ✓ OcclusionMaterial cutouts reveal passthrough. ✓ The keyboard stays registered. ✗ RealityKit entities can render in front of windows, including Mac Virtual Display and window chrome. breakthroughEffect doesn't cover whole windows or Mac Virtual Display. ✗ The Crown can't adjust immersion. .progressive: ✓ Windows always stay in front of immersive content (as an Apple engineer confirmed in https://developer.apple.com/forums/thread/764556). ✓ The Crown adjusts immersion. ✗ OcclusionMaterial reveals nothing inside the portal, so there is no desk cutout. ✗ The keyboard drifts (issue 1). Because of issue 1 we have switched our default back to .mixed, and lost windows-in-front. Questions Is keyboard awareness in .progressive/.full expected to be registered less accurately than in .mixed? Is there a supported way to keep it registered with world-locked content, or to get the pose at which the system draws it? In .mixed, is there a supported way to keep windows (including Mac Virtual Display) in front of an app's immersive content? Or, in .progressive, a supported way to show passthrough in an app-defined region inside the portal? Any pointers or workarounds are appreciated.
0
0
1.6k
Sep ’26
Improving metric scale consistency of ARWorldTrackingConfiguration on devices without LiDAR
We build a medical scanning app. A practitioner walks a slow orbit (~1 m radius, two laps, 25–35 s) around a standing subject while an ARSession runs, and we capture ~50 high-resolution stills with captureHighResolutionFrame. The ARKit camera poses for those frames are later used to set the metric scale of a photogrammetry reconstruction. We need the resulting scale to be within about 2 % of true. What we observe, measured against a printed A4 sheet on the floor in every scan: Devices with LiDAR (iPhone 16 Pro Max, iPad Pro), using the sceneDepth frame semantic as a check: scale within ±0.8 % across 11 sessions. Devices without LiDAR (iPhone 13, iPad 9th and 11th gen, plus non-LiDAR runs on the Pro devices): the world-frame metre is on average 2–3 % short of true, and varies by a further ±2 % from one session to the next, across 24 sessions in four rooms. Within a session the scale is stable: lap-to-lap variation is under 0.4 %. The error appears to be set during initialisation and held. Nothing we record about capture correlates with the error: time spent before capture, distance walked, vertical camera movement, orbit radius, rotation speed, tracking state, or the residual of our pose fit. Questions: Is there any session setup, initialisation practice, or motion during the first seconds of tracking that is known to improve the metric scale estimate on devices without LiDAR? Does the choice of video format, resolution or frame rate affect scale estimation? Is there any signal, beyond ARCamera.trackingState, that indicates how well scale has been determined for a session, so we could flag sessions rather than accept them? Does anything in the configuration above, in particular locking exposure on the capture device or using the high-resolution video format, work against tracking quality or scale?
Replies
1
Boosts
0
Views
532
Activity
1m
Presenting the system share sheet from a full ImmersiveSpace on visionOS 26
We are building a visionOS 26 app (deployment target visionOS 26.0). Inside a full immersive space, the user takes a spatial photo. The photo is shown on a RealityView attachment. We need the system share sheet( AirDrop, Messages, Mail, Copy, Save Image, and the item preview with Done) to appear when the user taps Share on that attachment. The immersive space is: ImmersiveSpace(id: "ImmersiveView") { ImmersiveView() } .immersionStyle(selection: $panoImageView, in: .full) .upperLimbVisibility(.visible) The review UI is a RealityView attachment, not a window: Attachment(id: "panoramaFrameCaptureReview") { PanoramaFrameCaptureReviewView(model: frameCapture) { guard let path = frameCapture.shareURL?.path else { return } dismissWindow(id: "PanoramaPhotoShare") Task { @MainActor in try? await Task.sleep(nanoseconds: 180_000_000) openWindow(id: "PanoramaPhotoShare", value: path) } } } The shared item is a stereo HEIC written with Image I/O (kCGImagePropertyGroupTypeStereoPair, left/right images, HEIF camera extrinsics and SimplifiedPinhole intrinsics). The same data displays correctly with ImagePresentationComponent and saves to Photos as a spatial photo. Share is the only part that fails. Attempt 1. ShareLink on the attachment Tapping Share does nothing. On entry to the immersive space the system logs: Presentations are not currently supported in Immersive contexts. Attempt 2. UIActivityViewController from the attachment’s hosting controller let activity = UIActivityViewController(activityItems: [fileURL], applicationActivities: nil) var presenter: UIViewController = self while let parent = presenter.parent { presenter = parent } presenter.present(activity, animated: true) presenter is UIHostingController<RootView>. The console then says: Skipping popover presentations transition, missing window or scene for <UIActivityViewController>. Failed to request default share mode for fileURL:...heic error:Error Domain=NSOSStatusErrorDomain Code=-10814 No share UI appears. presentedViewController is non-nil, so a “did it present?” check does not detect the failure. Attempt 3. A real WindowGroup whose only job is to present UIActivityViewController WindowGroup(id: "PanoramaPhotoShare", for: String.self) { path in if let path = path.wrappedValue { PanoramaPhotoShareWindow(path: path) } } .windowStyle(.plain) .defaultSize(width: 20, height: 20) .defaultWindowPlacement { _, _ in WindowPlacement(.utilityPanel) } struct PanoramaPhotoShareWindow: View { let path: String @Environment(.dismissWindow) private var dismissWindow var body: some View { PanoramaPhotoSharePresenter(url: URL(fileURLWithPath: path)) { dismissWindow(id: "PanoramaPhotoShare") } .frame(width: 8, height: 8) .background(.clear) } } final class PanoramaPhotoShareController: UIViewController { override func viewDidAppear(_ animated: Bool) { super.viewDidAppear(animated) guard view.window != nil, let url else { return } let activity = UIActivityViewController(activityItems: [url], applicationActivities: nil) activity.completionWithItemsHandler = { _, _, _, _ in self.onFinish?() } present(activity, animated: true) } } Tapping Share opens this window and the app crashes immediately. We removed the feature so users are not crashed. An earlier build that also put a ShareLink in that window logged: Attempt to present <UIActivityViewController> on <UIHostingController<...PanoramaPhotoShareWindow...>> which is already presenting <UIActivityViewController>. Question What is the supported way, on visionOS 26, to present the system share sheet while a .full ImmersiveSpace stays open? If a window is required, how should that window present the sheet without crashing, and without showing a second window that only contains another Share button?
Replies
0
Boosts
0
Views
246
Activity
1d
visionOS 27 beta 4 (24M5326g) stuck: update fails with MSU 806 MADownloadDaemonNotReady, all Apple services hang
My Apple Vision Pro is still on visionOS 27.0 beta 4 (24M5326g) and can't update or use any Apple service. Safari works. Software Update offers visionOS 27.2 (24N6120) but stays at "Update Requested". The OTAUpdate log shows: MobileSoftwareUpdateErrorDomain 806 "Failed to download update brain asset, MADownLoadResult:MADownloadDaemonNotReady". App Store can't connect, Apple TV and Games say a network connection is required, iCloud doesn't work, and Media & Purchases hangs at "Signing in to Apple Media Services". Other devices on the same network work normally. Reset All Settings didn't help. devicectl device sysdiagnose fails with CoreDeviceCLISupport.DiagnoseError 0. I don't have a Developer Strap. Has anyone recovered a Vision Pro stuck on an early beta like this without one? Filed as FB25386556.
Replies
0
Boosts
0
Views
370
Activity
2d
Reality kit Entities Appearing to Lag in a Full or Progressive Style Immersive Space When Opened with Environment Turned On
PLATFORM AND VERSION Vision OS Development environment: Xcode 16.2, macOS 15.2 Run-time configuration: visionOS 2.3 (On Real Device, Not simulator) Please someone confirm I'm not crazy and this issue is actually out of my control. Spent hours trying to fix my app and running profiles because thought it was an issue related to my apps performance. Finally considered chance it was issue with API itself and made sample app to isolate problem, and it still existed in it. The issue is when a model entity moves around in a full space that was launched when the system environment immersion was turned up before opening it, the entities looks very choppy as they move around. If you take off the headset while still in the space, and put it back on, this fixes it and then they move smoothly as they should. In addition, you can also leave the space, and then turn the system environment immersion all the way down before launching the full space again, this will also make the entity moves smoothly as it should. If you launch a mixed immersion style instead of a full immersion style, this issue never arrises. The issue only arrises if you launch the space with either a full style, or progressive style, while the system immersion level is turned on. STEPS TO REPRODUCE https://github.com/nathan-707/ChoppyEntitySample Open my test project, its a small, modified vision os project template that shows it clearly. otherwise: create immersive space with either full or progressive immersion style. setup a entity in kinematic mode, apply a velocity to it to make it pass over your head when the space appears. if you opened the space while the Apple Vision Pros system environment was turned up, the entity will look choppy. if you take the headset off while in the same space, and put it back on, it will fix the issue and it will look smooth. alternatively if you open the space with the system immersion environment all the way down, you will also not run into the issue. Again, issue also does not happen if space launched is in mixed style.
Replies
2
Boosts
1
Views
1.1k
Activity
2d
Immersive space sometimes opens with a broken world origin (world anchors fail with error 200, recenter times out)
Hi all, We're seeing an intermittent issue in a native SwiftUI + RealityKit app on Vision Pro (visionOS 27.0.1, still there on 27.2). When we open a mixed ImmersiveSpace, roughly 1 in 10–30 opens, everything ends up offset from where it should be. Sometimes it's 50 cm, sometimes several meters, sometimes it's rotated. Holding the Digital Crown to recenter fixes it immediately. We are attaching things to the camera, like we have a window following the user and that is offset in these rare cases which makes the app not usable. In the device logs, a good open looks like this: backboardd (recenter) Setting new world origin with anchor ... backboardd ARWorldTrackingProcessor: Added world anchor ... successfully (x4) backboardd (recenter) Finished (error: (null)) setting set new world origin ... A bad open looks like this: backboardd (recenter) Setting new world origin with anchor ... arkitd SLAMAnchorUpdate: Failed to add anchor request ... NSPOSIXErrorDomain Code=4 backboardd ARWorldTrackingProcessor: Failed to add world anchor ... com.apple.arkit Code=200 "Adding an object failed." (x4) backboardd (recenter) timeout exceeded. Forcing fade in. Like whenever the bad state occurs it is because adding a world anchor seems to fail. I haven't tested with visionOS26 but the same app didnt run into that problem ever. So the system seems to fail adding its own origin anchors, gives up after 2 seconds and shows the space anyway. Has anyone else run into this? And is there any way to detect it from the app or trigger a recenter in code? Thanks!
Replies
2
Boosts
0
Views
624
Activity
5d
High fidelity gaming on Apple Vision Pro
I'm looking to find the best setup for a game on AVP that gives the best quality environment with gameplay but there seems to be a mix of possible solutions each with their own drawback. My ideal scenario is a high fidelity room/environment while making use of the PS VR2 Sense controllers to track the user's hands and give them more inputs via the controllers. Eg they hold a weapon tracked to the controllers with the buttons able to use the weapon, shooting and hitting enemies in the room. I want to aim more for highest qualtiy gaming than cartoony so opted for foveated streaming from the Mac as my ideal, but this seems limited to only having some base inputs from the hands and no controller inputs for hand and positional tracking from the controller movement (as they need the data from the Vision Pro camera but the game is technically running on the Mac and streamed to the VP) The only real solution for what I want here is to go via RealityKit as a native Vision Pro app but this reduces the game quality I had in mind as the entire thing is now rendering on the VP instead of the Mac. Am I missing something here or is this a real limitation today? To reitarate I'm looking for: highest fidelity game quality (rendering as much as possible on the Mac) tracking the VR2 Sense controller's positions (to track a weapon) in 3D space
Replies
1
Boosts
0
Views
1k
Activity
6d
Async Entity(contentsOf:) is 15 to 20x slower while an ARView is rendering
While a non-AR ARView is on screen and rendering, await Entity(contentsOf:) takes about 26 s to load a 4.5 MB USDZ with 300 textured meshes. The same file loads in about 1.5 s with the synchronous Entity.load(contentsOf:), or with the async API while the view is hidden. iPhone 16 Pro, iOS 27.2Mac Catalyst, macOS 27.2await Entity(contentsOf:), view rendering22 to 26 s26 sEntity.load(contentsOf:), view rendering1.6 s1.1 to 1.7 sawait Entity(contentsOf:), view hidden1.4 s1.3 to 1.5 sModelEntity(contentsOf:) and Entity.loadAsync(contentsOf:) behave the same. A .reality file loads fast either way, and the iOS Simulator doesn't show it. A Time Profiler trace shows the main thread blocked in re::RenderThread::waitForFramePacing on every frame, so the @MainActor import only advances between frames. Neither workaround fits loading content into a scene the user is looking at: the sync load blocks the main thread, and hiding the view blanks it. Filed as FB25340264, with a sample project.
Replies
0
Boosts
0
Views
155
Activity
6d
Reality Composer Pro Assistant: "Authorization failed" when adding custom Azure AI Foundry OpenAI provider
Hello everyone, I am trying to configure a custom model provider inside the new Reality Composer Pro Assistant settings panel using an LLM hosted via Azure AI Foundry. Despite verifying my credentials and testing multiple base URL structures, I am persistently encountering the following error: Authorization failed — check your API key My Configuration Details: Target Model Deployment: gpt-5.1-codex-mini (GlobalStandard deployment type) Azure AI Foundry Endpoint Pattern: https://azure.com API Key Header Used: api-key What I Have Tried: Utilizing the Azure OpenAI format: https://<project-name>.services.ai.azure.com/openai/v1/responses Verifying the API key explicitly with headers like api-key and Authorization (testing both raw keys and Bearer <key> prefixes). It appears that Reality Composer Pro's validation step might be calling a standard GET /v1/models route or structurally modifying the outbound request in a way that causes Azure's gateway architecture to reject it with a 401 response code. Has anyone successfully mapped an Azure AI Foundry endpoint directly to the Assistant interface? If so, what is the exact base URL syntax and header structure expected by the desktop client to pass authorization? Thanks in advance for any insights!
Replies
0
Boosts
0
Views
433
Activity
1w
Ultra-wide camera in world tracking
My app previews cinema lenses by cropping the AR feed, so I can only simulate lenses tighter than the main camera's ~24mm-equivalent, meaning wider cinema lenses are impossible to accurately preview. ARConfiguration.VideoFormat exposes captureDeviceType, but world-tracking formats only ever list the wide-angle module. Is running world tracking on (or alongside) the ultra-wide camera fundamentally infeasible? Or just a case of not enough Feedback Assistant requests?
Replies
3
Boosts
2
Views
1.8k
Activity
1w
visionOS : any non-nil preferredSurroundingsEffect enables a light-spill halo around VideoPlayerComponent video
Hi all — sharing a platform behavior I isolated while building a custom immersive video player, in case others hit it (and to ask whether it's intended). Finding On visionOS 27 (Xcode 27), an entity playing video via VideoPlayerComponent(videoRenderer:) gets a light-spill "screen glow" halo around its edges whenever the immersive space declares any non-nil preferredSurroundingsEffect. The halo follows the video's colors frame by frame. With no surroundings effect (nil), there is no halo. Quick repro Progressive immersive space + VideoPlayerComponent playing. Add .preferredSurroundingsEffect(.dim(intensity: 0.0)) → halo appears (even though the effect is visually a no-op). Remove it → halo disappears. Same with .colorMultiply(_:) and preset effects. Things I ruled out on device (Vision Pro) ImmersiveContentBrightness (.custom(1.0) vs .automatic): no difference. VideoPlayerComponent configuration (.stereo, spatial video mode, isPassthroughTintingEnabled) and component attach timing: no difference. Presenting the same video via VideoMaterial: no halo. Bright white SwiftUI content: no halo. Questions Is this coupling intended? The preferredSurroundingsEffect docs don't mention any effect on video playback. Is there a public way to keep a surroundings effect (e.g. room dimming) while avoiding the video light spill? I found none on VideoPlayerComponent. Current workaround: don't declare any surroundings effect in the modes where the halo is undesirable. Also filed via Feedback Assistant(FB25130748). Curious if anyone else has seen this or knows a switch.
Replies
2
Boosts
1
Views
2.9k
Activity
1w
Foveated streaming QR code Authentication alternative
I was wondering whether there is a way to authenticate a CloudXR connection without using a QR code in a local deployment scenario. If the server is running in a headless environment, QR code-based authentication would not be practical. I reviewed the documentation but could not find any information on alternative authentication methods. Is a non-QR authentication workflow similar to the remote case supported?
Replies
2
Boosts
0
Views
1.5k
Activity
1w
Reality Composer Pro exporting larger and larger .reality files for each reference usage
We've noticed something kind of odd related to the exported .reality file in Reality Composer Pro 3. Our exported .reality file seems to be increasing a lot in size just by referencing an imported asset or another entity in the scene. This contradicts everything we previously knew about references. Essentially it seems like our assets are being copied or duplicated on each export. Before you dive into the description of the issue, yes, I've already created feedback about this. I think it's a bug, but am willing to be corrected if this is an expected, if somewhat odd, behavior. The feedback number is FB25253483 I did a quick test with a single model (Earth.usdz from the RCP2 content library. Project A: RCPExportDupeIssues Phase 01 Created new project in RCP 3 Imported the file and added it to the default world entity Generated an Xcode linked project and used RCP3 to run the app in the simulator (this runs through the entire .reality export process.) Result: 48.4 MB Phase 02 Created AnotherScene entity Added the imported earth model to it Used RCP3 to run the app in the simulator Result: 64.5 MB Phase 03 I thought maybe if I added the earth import to a single entity and added that entity to the other two scenes, I might get some savings. This also increased the overall export size. Created EarthEntity Added the imported earth model to it Used RCP3 to run the app in the simulator Result: 80.7 MB Overall result: an RCP3 project with a single imported Earth asset, used in three otherwise empty scenes, ends up with a massive 80 MB. I even tried deleting the Earth import reference in the first two scenes. Even with only the Earth entity present, the export was still 80.7 MB. Project B: RCPExportNoDupes I tried a second project without ever adding the Earth import file to the first two scenes. I created a dedicated EarthEntity and only added that to the other two scenes. Created new project in RCP 3 Created EarthEntity Imported the file and added it to EarthEntity Added `EarthEntity to the default world entity Created AnotherScene entity and added `EarthEntity to it Generated an Xcode linked project and used RCP3 to run the app in the simulator (this runs through the entire .reality export process.) Result: 80.7 MB Apple: I provided zipped versions of the exports in the feedback item (FB25253483)
Replies
1
Boosts
0
Views
857
Activity
1w
RCP .reality file in iPadOS with behavior tree
I am having troubles with a .reality file in iPadOS 27.0.1. I have a Reality Composer Pro 3 project with an Animation Graph and Behavior Tree. The animation simply switches between a character walking and running, and the Behavior Tree simply changes the boolean to trigger the running state, waits, flips the boolean back, waits, and repeats. When I run the Simulate in RCP 3, it behaves as I expected - walks, runs, walks, runs, etc. I export the World scene as a .reality file, and open the .reality file in Xcode on macOS 27.2, and it behaves as I expected. But when I load the .reality file in an iPadOS app (iPadOS 27.0.1), the character does not switch between states. It starts the walking animation, but it never switches to the running animation. It appears the Behavior Tree isn't working as expected. Any idea what is wrong? Should I switch the iPad to the iPadOS 27 betas? Should I set the minimum OS target for the app to iPadOS 27? Is the Behavior Tree a known limitation with iOS or iPadOS 27?
Replies
1
Boosts
0
Views
596
Activity
1w
Metal Compositor Service & Persona (VisionOS)
Hello, I'm currently trying to make a collaborative app. But it just works only on Reality View, when I tried to use Compositor Layer like below, the personas disappeared. ImmersiveSpace(id: "ImmersiveSpace-Metal") { CompositorLayer(configuration: MetalLayerConfiguration()) { layerRenderer in SpatialRenderer_InitAndRun(layerRenderer) } } Is there any potential solution too see Personas in Metal view? Thanks in advance!
Replies
3
Boosts
0
Views
2k
Activity
2w
Permitted research logging of latestFocusRegion in Foveated Streaming provider?
We are evaluating a consented research study in which a participant freely looks at a painting streamed from a desktop Unity/OpenXR application to Apple Vision Pro. A custom Foveated Streaming provider extension would read FoveatedStreamingProviderContext.latestFocusRegion during an active streaming session and classify the approximate focus direction into a painting-region identifier. No pinch or other action would be required. Is an approved provider permitted to export and retain timestamped painting-region identifiers for research analysis? If so, may it also retain the focus region’s direction, distance, and timestamp, or only the derived region? Is there a supported path for sending the derived data to the paired app or remote endpoint? What entitlement-review, consent, privacy, data-use, and distribution conditions apply? We understand the provider entitlement requires Apple approval and that ordinary visionOS apps do not receive passive gaze data. We would request the entitlement only if this research use is allowed. We are seeking guidance on permitted use before implementing or collecting data.
Replies
0
Boosts
0
Views
901
Activity
2w
Main Camera development request blocked by Individual-to-Organization conversion
We are developing MadnessBot as a business-only Vision Pro automotive inspection app for professional mechanics and repair shops, intended for private business distribution. Main Camera access is essential for analyzing the vehicle and component in front of the technician. A native development app is running on our physical headset, and chat plus OBDLink CX diagnostics have been tested. Our existing Apple Developer Program membership is enrolled as Individual. We contacted Developer Support on October 6, 2026, with a follow-up on October 9, for assistance converting it to Organization. The conversion form asks for the type of tax ID historically associated with the Individual membership; the Account Holder does not know that record and has asked Support to verify it before selecting an answer. The enterprise API request form remains inaccessible until organization enrollment is confirmed. We have not submitted a camera entitlement request yet. Our planned initial request is Development Only Main Camera access for registered test devices. Feedback reference: FB25132939. Could the Enterprise APIs for visionOS team help route this prerequisite blocker to membership support, or advise the correct escalation path so we can complete conversion and apply for Development Only access?
Replies
0
Boosts
0
Views
1.6k
Activity
3w
RoomPlan: placed RealityKit content does not align with the scanned room
I scan a room with RoomPlan API, then keep the same ARSession running (stop(pauseARSession: false)) and place RealityKit entities using the CapturedRoom wall transforms (wall.transform, wall.dimensions), parented to an anchor at the world origin. The camera position stays continuous, so tracking was not reset. But the entities do not sit on the real walls. The whole room appears shifted. What is the correct way to place content in the live AR session so it aligns with the scanned room?
Replies
1
Boosts
0
Views
1.9k
Activity
3w
RealityKit VideoMaterial renders 8-bit HEVC HLG/BT.2020 360° video with a purple tint in the visionOS Simulator (device is fine)
We paint a looping 360° video on the inside of a sphere with VideoMaterial(avPlayer:): an AVQueuePlayer plus AVPlayerLooper, a ModelEntity with an inward-facing equirectangular mesh, inside a .mixed ImmersiveSpace. No shaders, no VideoPlayerComponent. We could not find an official Apple sample that puts a VideoMaterial on a 360° sphere; this is the plain pattern from the VideoMaterial(avPlayer:) reference and the one used in threads like 734081. On Apple Vision Pro hardware the video renders correctly. In the visionOS Simulator (27.0, 26.5 and earlier 26.x, Xcode 27.0) the whole frame has a strong purple/magenta cast. Audio and timing are fine. The video (which we cannot share, so here is its metadata, image sample of issue on simulator provided below): MP4, HEVC Main, 8-bit 4:2:0, 3840x1920, tagged BT.2020 primaries, HLG transfer (ARIB STD-B67), BT.2020 non-constant matrix, both in the colr atom and in the SPS VUI. ffprobe -show_format -show_streams (trimmed): [STREAM] index=0 codec_name=hevc profile=Main codec_type=video codec_tag_string=hvc1 width=3840 height=1920 pix_fmt=yuv420p color_range=tv color_space=bt2020nc color_transfer=arib-std-b67 color_primaries=bt2020 r_frame_rate=30000/1001 avg_frame_rate=679200/22663 duration=37.771667 bit_rate=25030607 [SIDE_DATA] side_data_type=Stereo 3D type=2D horizontal_field_of_view=360000/1000 [/SIDE_DATA] [/STREAM] [STREAM] index=1 codec_name=aac profile=LC codec_type=audio codec_tag_string=mp4a sample_fmt=fltp sample_rate=48000 channels=2 channel_layout=stereo duration=37.758333 bit_rate=128176 [/STREAM] [FORMAT] nb_streams=2 format_name=mov,mp4,m4a,3gp,3g2,mj2 duration=37.778333 bit_rate=25160348 TAG:major_brand=mp42 TAG:minor_version=1 TAG:compatible_brands=isommp41mp42 [/FORMAT] AVFoundation's view of the track, from CMFormatDescriptionGetExtension on the loaded video track: primaries=ITU_R_2020 transfer=ITU_R_2100_HLG matrix=ITU_R_2020 bitsPerComponent=8 The playback code, reduced to the essentials: let item = AVPlayerItem(url: fileURL) let looper = AVPlayerLooper(player: player, templateItem: item) // retained player.actionAtItemEnd = .advance let material = VideoMaterial(avPlayer: player) let sphere = ModelEntity( mesh: try equirectSphere(radius: 12), // lat/long grid, normals inward, materials: [material] // UVs (u, 1 - v), inward winding ) sphere.position = [0, 1.5, 0] rootEntity.addChild(sphere) // rootEntity was added in RealityView { } player.play() What we have ruled out in a minimal sample project: Apple's own 360° HLS example (360 Lighthouse, HEVC Main 10, 10-bit, BT.709 SDR) plays with correct colours in the same sphere, same code, same simulator session. Rewriting our file's colr atom to BT.709 (three bytes, no re-encode) is honoured by AVFoundation (the track's CMFormatDescription then reports ITU_R_709_2 for primaries, transfer and matrix) but the tint stays. So the remaining differences between the failing file and Apple's working stream are 8-bit Main vs 10-bit Main 10, HLG/BT.2020 in the bitstream VUI vs BT.709, and local MP4 vs HLS. The minimal sample project (without the video) is available on request. Is this a known limitation of the simulator's VideoMaterial path with 8-bit HLG content, and is there a recommended way to get device-equivalent colour in the simulator? Feedback filed as FB24997147. Here is a sample of the result on the simulator of a cement column, painted white, but rendered purple on simulator:
Replies
1
Boosts
0
Views
3.3k
Activity
Oct ’26
Mac spatial preview feature: Supported with Blender?
Hello! I am relatively new to dev for Apple Vision Pro, however I have owned one since launch and love it. When I heard that we were going to be able to preview real-time, 3D models from the Mac I couldn’t believe it! This is one of my major workflows and I have been using 3rd party workarounds (that don’t work well). My question is, is there any word on supported 3D modeling apps like Blender or Cinema 4D? Or will there be a way to implement this preview feature with any app via a protocol? Any documentation would be really helpfu! Thank you, and continue the great additions to Apple Vision Pro!
Replies
2
Boosts
0
Views
2.8k
Activity
Oct ’26
Revealed keyboard drifts in progressive immersion, and no immersion style gives windows-in-front + Crown + passthrough cutouts
Environment: Apple Vision Pro, visionOS 27.0.1 (24M372). Magic Keyboard, with Settings › Awareness & Safety › Reveal Keyboards set to Always. Our app puts the user in an opaque virtual room while they work at their real desk, using app windows and Mac Virtual Display. A virtual desk is placed on the real desk. It is world-locked: positioned once from a PlaneAnchor (PlaneDetectionProvider), then a plain entity at fixed ImmersiveSpace coordinates. No AnchorEntity or WorldAnchor is involved. We offer two modes: .mixed, and .progressive for Digital Crown control. Each gives us only half of what desk work needs. 1. The revealed keyboard drifts in progressive immersion (FB25006789) In .progressive, the physical keyboard shown by keyboard awareness does not stay registered with the virtual desk. After the keyboard has been out of view for a few seconds (for example, while looking at a Mac Virtual Display window), it looks "floaty" when you look back: as the head turns, it slides against the desk. Turning right, the keyboard moves left farther than the desk does. What we checked: It happens in .progressive at any Crown amount, and in .full. With identical content in .mixed (no app passthrough cutout, so again only keyboard awareness shows the keyboard), it is much steadier. On 27.0.1, PlaneAnchor and DeviceAnchor coordinateSpace(correction: .rendered) are exactly equal to their .none transforms: 0 mm, 0°, scale 1. That held over 835 per-frame samples with ±50° yaw and up to 36 cm of head movement. So the rendered correction gives the app nothing to compensate with. The app's desk transform is constant while the drift is visible. upperLimbVisibility(.automatic) vs .visible and a .portrait progressive aspect ratio make no difference. Possibly related: https://developer.apple.com/forums/thread/796557 (FB19666209). 2. No immersion style combines what desk work needs (FB25006837) We need, at the same time: windows (our own and Mac Virtual Display) never covered by the room's walls, furniture or trees; the Digital Crown to bring the real surroundings back in; a passthrough cutout over the real desk, and a well-registered keyboard. .mixed: ✓ OcclusionMaterial cutouts reveal passthrough. ✓ The keyboard stays registered. ✗ RealityKit entities can render in front of windows, including Mac Virtual Display and window chrome. breakthroughEffect doesn't cover whole windows or Mac Virtual Display. ✗ The Crown can't adjust immersion. .progressive: ✓ Windows always stay in front of immersive content (as an Apple engineer confirmed in https://developer.apple.com/forums/thread/764556). ✓ The Crown adjusts immersion. ✗ OcclusionMaterial reveals nothing inside the portal, so there is no desk cutout. ✗ The keyboard drifts (issue 1). Because of issue 1 we have switched our default back to .mixed, and lost windows-in-front. Questions Is keyboard awareness in .progressive/.full expected to be registered less accurately than in .mixed? Is there a supported way to keep it registered with world-locked content, or to get the pose at which the system draws it? In .mixed, is there a supported way to keep windows (including Mac Virtual Display) in front of an app's immersive content? Or, in .progressive, a supported way to show passthrough in an app-defined region inside the portal? Any pointers or workarounds are appreciated.
Replies
0
Boosts
0
Views
1.6k
Activity
Sep ’26