The light of RealityView can only be effective on virtual objects. I hope it can be projected into the real world. What API can be implemented?

My goal is to pin an attachment view precisely at the point where I tap on an entity using SpatialTapGesture. However, the current code doesn't pin the attachment view accurately to the tapped point. Instead, it often appears in space rather than on the entity itself. The issue might be due to an incorrect conversion of coordinates or values. My code: struct ImmersiveView: View { @State private var location: GlobeLocation? var body: some View { RealityView { content, attachments in guard let rootEnity = try? await Entity(named: "Scene", in: realityKitContentBundle) else { return } content.add(rootEnity) }update: { content, attachments in if let earth = content.entities.first?.findEntity(named: "Earth"),let desView = attachments.entity(for: "1") { let pinTransform = computeTransform(for: location ?? GlobeLocation(latitude: 0, longitude: 0)) earth.addChild(desView) // desView.transform = desView.setPosition(pinTransform, relativeTo: earth) } } attachments: { Attachment(id: "1") { DescriptionView(location: location) } } .gesture(DragGesture().targetedToAnyEntity().onChanged({ value in value.entity.position = value.convert(value.location3D, from: .local, to: .scene) })) .gesture(SpatialTapGesture().targetedToAnyEntity().onEnded({ value in })) } func lookUpLocation(at value: CGPoint) -> GlobeLocation? { return GlobeLocation(latitude: value.x, longitude: value.y) } func computeTransform(for location: GlobeLocation) -> SIMD3<Float> { // Constants for Earth's radius. Adjust this to match the scale of your 3D model. let earthRadius: Float = 1.0 // Convert latitude and longitude from degrees to radians let latitude = Float(location.latitude) * .pi / 180 let longitude = Float(location.longitude) * .pi / 180 // Calculate the position in Cartesian coordinates let x = earthRadius * cos(latitude) * cos(longitude) let y = earthRadius * sin(latitude) let z = earthRadius * cos(latitude) * sin(longitude) return position } } struct GlobeLocation { var latitude: Double var longitude: Double }

Hi everyone, I'm developing a visionOS app using SwiftUI and RealityKit, and I'm encountering an issue with the pivot points of primitive shapes created in Reality Composer Pro. Scenario: When I use Reality Composer Pro within Xcode to add primitive shapes (such as cubes, capsules, etc.) to my scene, the pivot points for these objects seem to be set incorrectly. The pivot is located far from the actual object, which affects transformations and positioning. Question: Is there a way to correct or adjust the pivot point for primitive shapes created in Reality Composer Pro? Additional Information: I’ve attached a screenshot illustrating the issue with the pivot point being misaligned. Any guidance on how to resolve this would be greatly appreciated. Thanks in advance for your help! Best, Siddharth

I am using Model3D to display an RCP scene/model in my UI. How can I get to the entities so I can set material properties to adjust the appearance? I looked at interfaces for Model3D and ResolvedModel3D and could not find a way to get access to the RCP scene or RealityKit entity.

My visionOS app has over 150 3D models that work flawlessly via firebase URL. I also have some RealityKitContent scenes (stored locally) that are getting pretty large so I'm looking to move those into firebase as well and call on them as needed. I can't find any documentation that has worked for this and can't find anyone that's talked about it. Does anyone know if this possible? I tried exporting the Reality Composer Pro scenes as USDZ's and then importing them in but kept getting material errors. Maybe there's a way to call on each 3D model separately and have RealityKit build them into the scene? I'm not really sure. Any help would be much appreciated !

In Reality View, I want to move an entity A to the position of entity B, but I can't determine the coordinates of entity B (for example, entity B is tracking the hand). What's the solution?

RealityContentKit bundle resource issue Recently I always encounter weird loading bugs from RealityKitContent bundle. When I was trying to load audio resource as AudioFileResource or AudioFileGroupResource by loading from *.usda from RealityKitContent bundle, with this method. My code is nothing complicated but simple as below: let primPath: String = "/SampleAudios/SE_bounce_audio" guard let resource = try? AudioFileGroupResource.load(named: primPath, from: "MyScene.usda", in: realityKitContentBundle) else { return } And the runtime program "sometimes"(whenever I change something RCP it somethings work again but the behavior is unpredictable) reports that it "Cannot find MyScene.usda:/SampleAudios/SE_bounce_audio in RealityKitContent.bundle". I put MyScene.usda under the root folder of RealityKitContent package because I found that RealityKit just cannot find any *.usda scene if you didn't put that on the root level (could be a bug because of the way it indexes its files). I even double checked my .usda file with usdview, the primPath is absolutely correct. I think there are some unknown issues when RealityKitContent copy resources and build the package. I tried to play with the package Package.swift file a bit to see if I could manually copy my resources (everything) and let the package carry my resources but it just didn't work. So right now I just keep this file untouched below (just upgrade the swift-tools-version to 6.0 as only that can supports .visionOS(.v2)): // swift-tools-version:6.0 // The swift-tools-version declares the minimum version of Swift required to build this package. import PackageDescription let package = Package( name: "RealityKitContent", platforms: [ .visionOS(.v2) ], products: [ // Products define the executables and libraries a package produces, and make them visible to other packages. .library( name: "RealityKitContent", targets: ["RealityKitContent"]), ], dependencies: [ // Dependencies declare other packages that this package depends on. // .package(url: /* package url */, from: "1.0.0"), ], targets: [ // Targets are the basic building blocks of a package. A target can define a module or a test suite. // Targets can depend on other targets in this package, and on products in packages this package depends on. .target( name: "RealityKitContent" ), ] ) That is just issue one, RealityKitContent package build issue. Audio file format issue Another is about Audio File Format RCP supports. I remember is a place (WWDC?) saying .wav and .mp4 are supported to be used as audio source. But when I try to set up Spatial Audio, I find sometimes *.wav or *.mp3 can also be imported as AudioSourceFile. But the behavior is unpredictable. With two *.wav files SE_ball_hit_01.wav and SE_ball_hit_02.wav, only SE_ball_hit_01.wav is supported, 02 is reported as the format is not supported/ Check out my screenshots to see the details of two files. Two files have almost the same format (same sample rate or channel). I understand there might be different requirements for a source file to be used as Spatial or Ambient audio. But I haven't figured that out or there is nothing I can find helpful on Apple Documentation. So what is the rules? Thanks for reading and any thought is welcomed.

I have been digging into learning shader graphs by watching Unity shader graph content, cause lots of the same concepts apply. One thing I noticed was that in Unity, each node in the shader graph has a little preview. I don't think this exists in Reality Composer Pro, but is there anyway to mimic it (like can I hook up a node that allows me to debug the graph at that point?) If not, I'm happy to just file a feedback about it, but just thought I'd ask!

From my early testing it seems like the object tracking works best for static objects. For example, if I am holding something in my hand the object tracker is slow to update. Is there anything that can be modified to decrease the tracking latency? I noticed that the Enterprise API has some override features is this something that can only be done using Enterprise?

In RealityView I have two entities that contain tracking components and collision components, which are used to follow the hand and detect collisions. In the Behaviors component of one of the entities, there is an instruction to execute action through onCollision. However, when I test, they cannot execute action after collisions. Why is this?

Hello, I am developing a VisionOS based application, that uses the various data providers like Image Tracking, Plane Detection, Scene Reconstruction but these are not supported on VisionOS Simulator. What is the Work Around for this issue ?

I have a model entity (from Reality Composer Pro) I want to change the material of the model entity inside swift. The material is also imported in reality composer pro. I am copying the USDZ file of the material in the same directory as the script. This is the code I am using to reference the Material. do { // Load the file data if let materialURL = Bundle.main.url(forResource: "BlackABSPlastic", withExtension: "usdz") { let materialData = try Data(contentsOf: materialURL) // Check the first few bytes of the data to see if it matches expected types let headerBytes = materialData.prefix(4) let headerString = String(decoding: headerBytes, as: UTF8.self) // Print out the header information for debugging print("File header: \(headerString)") // Attempt to load the ShaderGraphMaterial let ScratchedMetallicPaint = try await ShaderGraphMaterial( named: "BlackABSPlastic", from: materialData ) print(ScratchedMetallicPaint) } else { print("BlackABSPlastic.usdz file not found.") } } catch { // Catch the error and print it print("BlackABSPlastic load failed: \(error)") // Attempt to infer file type based on the error or file content if let error = error as? DecodingError { switch error { case .typeMismatch(let type, _): print("Type mismatch: Expected \(type)") case .dataCorrupted(let context): print("Data corrupted: \(context.debugDescription)") default: print("Decoding error: \(error)") } } else { print("Unexpected error: \(error)") } } I am receiving these errors: File header: PK TBB Global TLS count is not == 1, instead it is: 2 Unable to create stage from in-memory buffer. BlackABSPlastic load failed: internalImportError Unexpected error: internalImportError am I doing anything wrong? I am able to access the materials of the model entity easily but this seems to something different. How can this be resolved? Thanks.

When using the sample application of object tracking with apple Vision Pro, the left eye display shows the same position of the original object's target object, but the right eye is misaligned. Is there any way to resolve this issue?

Hello Dev team, 3 weeks I'm looking for how I can export a static Cinema 4D objects WITH TEXTURES to Reality Composer Pro ! I can export it directly on USDA format and it works well for the 3D model in Reality Composer Pro, BUT, I can't have the textures on my model. My model is simple not colored ! Of course I expect to have textures applied on the good place and same appearance I've in Cinema 4D. Could you give me a process to do that please ? I'm using Cinema 4D R25 and Last XCode and Reality Composer Pro beta versions. Big big thanks to the one could help me on this. It will unblock many things to me!!!! Cheers Mathis

I’m developing an app for Vision Pro and have encountered an issue related to the UI layout and model display. Here's a summary of the problem: I created an anchor window to display text and models in the hand menu UI. While testing on my Vision Pro, everything works as expected; the text and models do not overlap and appear correctly. However, after pushing the changes to GitHub and having my client test it, the text and models are overlapping. Details: I’m using Reality Composer Pro to load models and set them in the hand menu UI. All pins are attached to attachmentHandManu, and attachmentHandManu is set to track the hand and show the elements in the hand menu. I ensure that the attachmentHandManu tracks the hand properly and displays the UI components correctly in my local tests. Question: What could be causing the text and models to overlap in the client’s environment but not in mine? Are there any specific settings or configurations I should verify to ensure consistent behavior across different environments? Additionally, what troubleshooting steps can I take to resolve this issue?

I can execute an action by allowing Xcode to send a notification to Reality Composer Pro via NotificationCenter, or I can send notifications to Xcode through the Notification Action in Reality Composer Pro. However, I discovered that they were unable to accept notifications from both parties within my project. To ascertain whether there was an error in my code, I created a simple Demo project. I utilized the same code and determined that it functioned normally within the Demo project. It is perplexing that I am unable to resolve this issue. Do I require additional modifications?

Is there any action that can clone the entity in RealityView to the number I want? If there is, please let me know. Thank you!

I created a simple Timeline animation with only a "Play Audio" action in RCP. Also a Behaviors Component setting an "OnTap" trigger to fire this Timeline animation. In my code, I simply run Entity.applyTapForBehaviors() when something happened. The audio can be normally played on the simulator but cannot be played on the device. Any potential bug leads this behavior? Env below: Simulator Version: visionOS 2.0 (22N5286g) XCode Version: Version 16.0 beta 4 (16A5211f) Device Version: visionOS 2.0 beta (latest)

Using Xcode 15.4 and macOS Sonoma 14.6.1 and Reality Composer Pro 1.0. Lately (I believe since updating to Xcode 15.4, not sure..), I'm getting build errors that I didn't get before. The error is saying that "the package manifest at /Package.swift cannot be accessed (/Package.swift doesn't exist in file system)". When I click for more information and bring up the Resolve Packages - Log, it gives a red X error "Resolving Package Graph Failed" I'm not sure why I'm recently getting this error. My Reality Composer Pro Package is part of my project and listed in my project navigator as it has been. The Package.swift file is in the same file location as it has always been and I believe it is correct. I tried removing the package from my project navigator then dragging and dropping it back into the project navigator, but I'm still getting the error. Note: this error is only appearing on a working branch for a new version I'm working on. The error does not appear on my production (main) branch even though it is the same package and location in Finder and appears the same way in the main branch project navigator. I get these same errors on Xcode 16 Beta 5.

In my Volume, there is a RealityView that includes lighting effects. However, when the user drags the position of the window back and forth, the farther the distance between the volume and the user, the greater the brightness of the light effect. ( I believe this may be a Beta version of a bug.) Note: The volume windowGroup has the .defaultWorldScaling(.dynamic) property.