In RealityView, physical components are only applicable to certain solids. How can I simulate the physical effects of water and cloth? enity.components.set(PhysicalComponent)

I’d like to use ARKit world tracking and display both the back camera feed and the front camera feeds, using the front feed as as a PIP. This would work great for an internet streaming use case. However, it’s impossible. As soon as ARKit is told to use one mode, the camera for the other side freezes/doesn’t work. This page also says you have to pick one camera to show: https://developer.apple.com/documentation/arkit/arkit_in_ios/choosing_which_camera_feed_to_augment?language=objc A question to the developers: why is this limitation in-place? Are there any work-arounds for the use case of ARKit world tracking + displaying the back camera feed + displaying the front camera feed as an overlay? It’s possible to do this with plain camera initialization without ARKit. (There’s an official example.) With ARKit, it no longer works. It’s strange that I cannot access the front feed via one of the other frameworks, but I guess that ARKit blocks that.

Hi everyone, I’m having trouble with image anchoring when working on a project in Reality Composer and Reality Composer Pro. Here’s the issue: 1. What I’m Trying to Achieve: I want to create an AR scene where an object anchors to an image I provide. I don't want to create an app for this but just use the USDZ File the Scene creates. The USDZ File then should be viewable via the various integrations of AR Quick Look across the Apple Ecosystem. The image anchoring works perfectly when I preview the scene inside Reality Composer using AR mode. 2. The Problem: When I export the project (tried both USDZ and Reality formats) and open it on my iPhone using the Files app (which uses AR Quick Look), the image anchoring no longer works. The object doesn’t anchor to the provided image as expected. It just anchors to the first plane it recognizes and not the image. 3. What I’ve Tried: Exporting the scene in USDZ format. Exporting the scene in Reality format. Both formats result in the same issue: no image anchoring outside of the Reality Composer environment. Trying different images but all resulting in same manor that the image anchoring is not working Tried different iOS Version but resulting in the same issue 4. Current Setup: Reality Composer Pro version: 2.0 iPhone model: iPhone 13 Pro iOS version: 18.1. 5. What I Need Help With: Is there a way to ensure image anchoring works in exported files when opened via AR Quick Look? Do I need to configure something specific during the export process? Are there limitations in AR Quick Look that prevent image anchoring from functioning correctly? Do i need to create an app to make this work? I’d appreciate any advice or insights from the community. If anyone has experience with similar issues or knows of a workaround, please let me know! Thanks in advance, Mav

Hello everyone, Since last night, the Object Capture feature in my app has stopped working. Whenever I try to use it, a blank screen is displayed instead of the expected functionality. I’ve also tested several other apps that rely on Object Capture, and they are experiencing the same issue. This makes me think it might not be a problem specific to my device or app. I’ve already tried restarting my device and ensuring all apps are up to date, but the issue persists. Does anyone have more information about this issue? If so, is there any update on when it might be resolved? Thank you in advance for your help! Best regards

Hello Developers, I am working on an app where I need to capture 48MP high-resolution photos using the ultra-wide camera of the iPhone 16 Pro while an AR session is running. The goal is to take these photos without interrupting or impacting the AR session, which uses the main wide-angle camera. Despite extensive testing and various approaches, we have been unable to achieve the desired functionality. What We Have Tried So Far 1. Using AVCaptureMultiCamSession: • We attempted to leverage AVCaptureMultiCamSession to simultaneously use the wide-angle camera for ARKit and the ultra-wide camera for photo capture. • However, this approach resulted in resource conflicts, with errors such as Cannot Record (OSStatus error -16409) and dropped frames. Additionally, the ultra-wide camera feed would frequently freeze or stop. 2. Dedicated AVCaptureSession for the Ultra-Wide Camera: • We separated the ultra-wide camera into its own AVCaptureSession while letting ARKit exclusively use the wide-angle camera. • This setup showed initial promise, but the ultra-wide camera feed would still stop running after a very short time (under one second). • Debugging logs indicated potential system-level interruptions, possibly due to resource prioritization by iOS. 3. Notification-Based Monitoring: • We implemented monitoring for session interruptions (AVCaptureSession.wasInterruptedNotification), but this provided limited insights into the exact cause of the session stopping. • We suspect iOS is de-prioritizing the ultra-wide camera session due to resource management policies or conflicts with ARKit. 4. Adjusting Camera Configurations: • We attempted to simplify both ARKit and AVCaptureSession configurations by reducing features like depth data and by using lower session presets for video capture. However, the core issue persisted. The Core Problem • The ultra-wide camera session frequently stops or freezes when used alongside ARKit. • Capturing high-resolution 48MP photos during the AR session is critical to the functionality of our app. Question Has anyone successfully implemented a similar setup? Specifically: • Capturing 48MP photos with the ultra-wide camera while ARKit is actively using the main camera. • Avoiding conflicts between ARKit and AVCaptureSession for the ultra-wide camera. Any insights, suggestions, or alternative approaches would be greatly appreciated. Thank you in advance for your help! 😊

Hi, in visionOS2, I wonder if there is a method to add/remove referenceObjects of ObjectTrackingProvider after ARSession started without stop it. Now I must stop it and run it again. The current tracking is stopped. Thanks.

Hello, i've recently received the entitlements to access the main camera stream for a project on the Apple Vision Pro. What happens : When executing code from this WWDC tutorial , i'm getting this error when trying to use a Camera Frame Provider : ar_camera_frame_provider_t <0x300d58870>: Failed to start camera stream with error: <ar_error_t: 0x303fcc4c0 Error Domain=com.apple.arkit Code=100 "App not authorized." UserInfo={NSLocalizedFailureReason=Using camera frame provider requires an entitlement., NSLocalizedRecoverySuggestion=, NSLocalizedDescription=App not authorized.} What I've tried : I followed the instructions given by mail, by : adding the .license file at the root of my project, adding the .entitlements file by adding capabilities in the project (Main Camera Access & Passthrough in screen capture are there). I've added NSCameraDescription, NSEnterpriseMCAMUsageDescription and NSWorldSensingUsageDescription (they all have a value assigned). I've also followed those post & post advices. When checking on the Account settings, i do see the capabilities in the "additional capabilities" On first launch, I'm also getting prompted to accept the NSEnterpriseMCAMUsageDescription, so I assume the info.plist file is valid? What did i missed to get the entitlements working ? Here's the code : import ARKit import SwiftUI import Vision import RealityKit class MainCameraAccess { var arKitSession = ARKitSession() var cameraFrameProvider = CameraFrameProvider() var pixelBuffer: CVPixelBuffer? func startCameraSession() async { let formats = CameraVideoFormat.supportedVideoFormats(for: .main, cameraPositions: [.left]) // Request authorization await arKitSession.requestAuthorization(for: [.cameraAccess]) // Start the session do { try await arKitSession.run([cameraFrameProvider]) } catch { print("Failed to start ARKit session: \(error)") return } // Get camera frame updates guard let cameraFrameUpdates = cameraFrameProvider.cameraFrameUpdates(for: formats[0]) else { return } // Process frames for await cameraFrame in cameraFrameUpdates { guard let mainCameraSample = cameraFrame.sample(for: .left) else { continue } self.pixelBuffer = mainCameraSample.pixelBuffer } } func saveLatestImage() { guard let pixelBuffer = self.pixelBuffer else { print("No image available to save.") return } // Convert CVPixelBuffer to UIImage let ciImage = CIImage(cvPixelBuffer: pixelBuffer) let context = CIContext() guard let cgImage = context.createCGImage(ciImage, from: ciImage.extent) else { print("Failed to create CGImage.") return } let uiImage = UIImage(cgImage: cgImage) // Save UIImage to Photos Album UIImageWriteToSavedPhotosAlbum(uiImage, nil, nil, nil) print("Image saved to photo library.") } } Thanks in advance for the help, Jeremy

I wanted to know if I could create an app that involves some AR scenes using ARKit. I have a Figma prototype and was thinking of making some AR scenes that involve object tracking on ARKit. I wanted to know if I would be able to incorporate these AR scenes into my app somehow. I want these experiences to be triggered by a button from my app design. Also would this be easy to do and involve lots of coding? Thank you!!!

Lately i got a lot of crashes on iOS 18 devices (mostly 13 Pro devices but also a 16 Pro Max), has anyone encountered similar issues and is there more information about offlineFloorPlanGeneration? com.apple.RoomScanCore.offlineFloorPlanGeneration EXC_BAD_ACCESS KERN_INVALID_ADDRESS 0x0000f5cab18e03c0 RoomScanCore RSFrameFromDictionary + 110992

How do we author a Reality File like the ones under Examples with animations at https://developer.apple.com/augmented-reality/quick-look/ ?? For example, "The Hab" : https://developer.apple.com/augmented-reality/quick-look/models/hab/hab_en.reality Tapping on various buttons in this experience triggers various complex animations. I don't see any way to accomplish this in Reality Composer. And I don't see any way to export/compile to a "reality file" from within Xcode. How can I use multiple animations within a single GLTF file? How can I set up multiple "tap target" on a single object, where each one triggers a different action? How do we author something similar? What tools do we use? Thanks

I have created a simple scene in reality composer (composer not composer pro). It contains just a cube and text item. I convert this to usdz file and load it into a Arkit swift app. Since ios 18/xcode 16 - the "text" element is not displayed at all. The cube is displayed, anchors correctly and can be moved etc.... The output from usdchecker ➜ Desktop usdchecker GKTUHR1.6.3.usdz -v --arkit Opening GKTUHR1.6.3.usdz Checking layer <GKTUHR1.6.3.usdz>. Checking package <GKTUHR1.6.3.usdz> Checking prim </Root>. Checking prim </Root/Scenes>. Checking prim </Root/Scenes/Scene>. Checking prim </Root/Scenes/Scene/Gravity>. Checking prim </Root/Scenes/Scene/sceneGroundPlane>. Checking prim </Root/Scenes/Scene/sceneGroundPlane/physicsMaterial>. Checking prim </Root/Scenes/Scene/Children>. Checking prim </Root/Scenes/Scene/Children/hello>. Checking prim </Root/Scenes/Scene/Children/hello/Generated>. Checking prim </Root/Scenes/Scene/Children/hello/Generated/Text>. Checking prim </Root/Scenes/Scene/Children/hello/Generated/Text/Material>. Checking prim </Root/Scenes/Scene/Children/hello/Generated/Text/Material/PBRShader>. Checking shader </Root/Scenes/Scene/Children/hello/Generated/Text/Material/PBRShader>. Checking prim </Root/Scenes/Scene/Children/hello/Children>. Checking prim </Root/Scenes/Scene/Children/Box>. Checking prim </Root/Scenes/Scene/Children/Box/Generated>. Checking prim </Root/Scenes/Scene/Children/Box/Generated/Mesh0>. Checking prim </Root/Scenes/Scene/Children/Box/Generated/Mesh0/Mesh0>. Checking prim </Root/Scenes/Scene/Children/Box/Generated/Mesh0/Material>. Checking prim </Root/Scenes/Scene/Children/Box/Generated/Mesh0/Material/PBRShader>. Checking shader </Root/Scenes/Scene/Children/Box/Generated/Mesh0/Material/PBRShader>. Checking prim </Root/Scenes/Scene/Children/Box/Children>. Checking prim </Root/Scenes/Scene/Children/Box/PhysicsMaterial_Box>. Found material bindings but no MaterialBindingAPI applied on the prim </Root/Scenes/Scene/sceneGroundPlane>. (fails 'MaterialBindingAPIAppliedChecker') Found material bindings but no MaterialBindingAPI applied on the prim </Root/Scenes/Scene/Children/hello/Generated/Text>. (fails 'MaterialBindingAPIAppliedChecker') Found material bindings but no MaterialBindingAPI applied on the prim </Root/Scenes/Scene/Children/Box>. (fails 'MaterialBindingAPIAppliedChecker') Found material bindings but no MaterialBindingAPI applied on the prim </Root/Scenes/Scene/Children/Box/Generated/Mesh0>. (fails 'MaterialBindingAPIAppliedChecker') Failed!

Hi, every one! I'm trying to bind timeline(animation + audio) and behaviors on an entity in reality composer pro. In xcode, I need to clone this entity and use the behavior, but I found that the behaviors are not clone(send notification but not received by reality composer pro and not execute the timeline). How can I solve this problem? Thanks!

Hi, I would like to know how predicted hand tracking in ARKit works. Does it use the Kalman filter, or does it have another approach?

Hi, I heve a problem with an visionOS app and I couldn't find a solution. I have 3D carousel with cards and when I use the drag gesture and I drag to the left I want the carousel to rotate clockwise and when I drag to the right I want the carousel to rotate counter clockwise. My problem is when I rotate my body more than 90 degrees to the left or to the right the drag gesture changes it's value and the carousel rotates in the opposite direction? Do you know how can I maintain the right sense taking into account that the user can rotate his body? I've tried to take the user orientation with device tracking and check if rotation on Y axis is greater than 90 degrees in both direction but It is a very small area bettween 70-110 degrees when it still rotates in the opposite direction. I think that's because the device traker doesn't update at the same rate as drag gesture or it doesn't have the same acurracy.

Hi! I'm creating an app like this: Using Image Tracking to set world anchor in real world first. The timeline in Reality Composer Pro scene needs to be played in same time(for the people in same place using the app). People using the app will see the same contents in same position in same time in same place. I already made Image Tracking feature worked. But the big problem is "Synchronization". I found Group Activities and TabletopKit to solve the problem. But I don't know if this are the right modules for this project. How do I solve this problem technically? If you have ideas, please let me know. I really need help for this.

Hello, I am currently working on an app that features multiple environments in which I combine Reality Composer Pro scenes with objects managed at runtime as well as make heavy use of RealityView attachments that modify the appearance of certain objects. Is it possible to keep track of an AR anchor when transitioning between immersive spaces? About my app: There are two main contexts/scenes in the app that the user progresses through. The first takes place in AR and is non-interactive and driven by a timeline animation. The second is in VR and allows the user to change materials of select models. Both scenes need to be placed relative to a real-life object that functions as an image anchor. Anchoring is necessary for visual purposes in AR context and it would be nice to use it in the VR context as well in order to provide passive haptics to the user. If the user doesn't have access to the physical object, we make use of plane-based anchoring. Either way, we would like to keep the anchor's position across the scenes.

I am seeking a comprehensive pathway to learning Metal programming on VisionOS. The official documentation’s Pathway on Metal is insufficient in this regard. I kindly request that someone create a detailed pathway to assist me in this endeavor. The pathway should encompass the following key areas: Knowledge Base: Understand the fundamental principles of Metal and other frameworks, as well as basic concepts, to prepare for future learning. Metal3 (very important) : Gain a deep understanding of Metal itself, the programming language used to communicate with the GPU on the device to render graphics. This knowledge forms the foundation for all Metal-related tasks. Compositor Services and ARKit (important) : Learn how to display Metal scenes within the Vision device’s space and enable augmented reality (AR) and hand interaction. This knowledge is essential for creating interactive and immersive experiences. Metal Performance Shaders: Acquire expertise in optimizing material rendering to enhance performance. MetalKit: Simplifies the tasks that display your Metal content onscreen. MetalFX: Develop proficiency in using MetalFX to improve rendering efficiency and achieve visually stunning effects. I would appreciate it if you could provide me with a detailed and comprehensive pathway, including the URLs of relevant documents, to guide my learning journey. Thank you for your assistance.

Hi, would anyone be so kind and try to guide me, which technologies, Kits, APIs, approaches etc. are useful for creating a horizontal window with map (preferrably MapKit) on visionOS using SwiftUI? I was hoping to achieve scenario: User can walk around and interact with horizontal map window and also interact with (3D) pins on the map. Similar thing was done by SAP in their "SAP Analytics Cloud" app (second image from top). Since I am complete beginner in this area, I was looking for a clean, simple solution. I need to know, if AR/RealityKit is necessary or is this achievable only using native SwiftUI? I tried using just Map() with .rotation3DEffect() which actually makes the map horizontal, but gestures on the map are out of sync and I really dont know, if this approach is valid or complete rubbish. Any feedback appreciated.

Hi all, Our app allows a user to scan a room and then save that scan on a separate view, followed by additional scans. We're looking into allowing room combining via CapturedStructure, so we need rooms to be scanned in the same ARWorldMap without necessarily needing to re-localize in the same session. This should fit within the first scenario that Apple described. The only way I have found that allows our requirements is to save RoomCaptureView and to re-use that RoomCaptureView whenever we need to start a session again. This creates a number of other issues, and ideally, we wouldn't need to save a View in something like a singleton. We are using captureSession.stop(pauseARSession: false). Additionally, if we use the same RoomCaptureView and an error occurs during the scanning process, we can't get the instructions overlay to appear again if we reuse this view (specifically, the instructions in the middle of the view that state "Move device to start"). It's as if the instructions are completely removed and scanning is stuck on an error state if an error occurs. These instructions also seem to be separate from the instructions we can grab from RoomCaptureViewDelegate via didProvide instruction: RoomCaptureSession.Instruction), so we can't use that either. There's a couple subviews that seem relevant to this: RoomCaptureCoachingOverlayView and ARGlyphView - but both are not public, so we can't force them to appear. Also attempted a number of other things to try to get these subviews to appear, such as layoutIfNeeded(). Saving the ARSession and using it in let roomCaptureView = RoomCaptureView(frame: viewBounds, arSession: arSession) where we're creating a new view with the same ARSession seems much more ideal as that solves the above issues, but we run into another issue: world tracking seems to be completely lost when a new RoomCaptureView (and thus a new RoomCaptureSession) is started, even with the same already started ARSession, almost as if captureSession.stop(pauseARSession: false) doesn't work as described. Is there any way around needing to use the same RoomCaptureView or RoomCaptureSession for subsequent scans in the same session without needing to re-localize via ARWorldMap loading? Is there a way to force the guiding instructions to appear?

Ever since updating to Xcode 16 my AR app doesn't compile, because Xcode doesn't recognize the .rcproject files used to load the AR experiences in iOS app. The .rcproject files were authored in Reality Composer on iPadOS. The expected behavior is described in this official Apple documentation article: https://developer.apple.com/documentation/realitykit/loading-entities-from-a-file How do I submit a ticket to Apple?