VisionOS integrates SwiftUI interface in Unity, but the system drag bar at the bottom of the interface is too far away from the interface. How to solve this problem?

My visionOS app has over 150 3D models that work flawlessly via firebase URL. I also have some RealityKitContent scenes (stored locally) that are getting pretty large so I'm looking to move those into firebase as well and call on them as needed. I can't find any documentation that has worked for this and can't find anyone that's talked about it. Does anyone know if this possible? I tried exporting the Reality Composer Pro scenes as USDZ's and then importing them in but kept getting material errors. Maybe there's a way to call on each 3D model separately and have RealityKit build them into the scene? I'm not really sure. Any help would be much appreciated !

RealityContentKit bundle resource issue Recently I always encounter weird loading bugs from RealityKitContent bundle. When I was trying to load audio resource as AudioFileResource or AudioFileGroupResource by loading from *.usda from RealityKitContent bundle, with this method. My code is nothing complicated but simple as below: let primPath: String = "/SampleAudios/SE_bounce_audio" guard let resource = try? AudioFileGroupResource.load(named: primPath, from: "MyScene.usda", in: realityKitContentBundle) else { return } And the runtime program "sometimes"(whenever I change something RCP it somethings work again but the behavior is unpredictable) reports that it "Cannot find MyScene.usda:/SampleAudios/SE_bounce_audio in RealityKitContent.bundle". I put MyScene.usda under the root folder of RealityKitContent package because I found that RealityKit just cannot find any *.usda scene if you didn't put that on the root level (could be a bug because of the way it indexes its files). I even double checked my .usda file with usdview, the primPath is absolutely correct. I think there are some unknown issues when RealityKitContent copy resources and build the package. I tried to play with the package Package.swift file a bit to see if I could manually copy my resources (everything) and let the package carry my resources but it just didn't work. So right now I just keep this file untouched below (just upgrade the swift-tools-version to 6.0 as only that can supports .visionOS(.v2)): // swift-tools-version:6.0 // The swift-tools-version declares the minimum version of Swift required to build this package. import PackageDescription let package = Package( name: "RealityKitContent", platforms: [ .visionOS(.v2) ], products: [ // Products define the executables and libraries a package produces, and make them visible to other packages. .library( name: "RealityKitContent", targets: ["RealityKitContent"]), ], dependencies: [ // Dependencies declare other packages that this package depends on. // .package(url: /* package url */, from: "1.0.0"), ], targets: [ // Targets are the basic building blocks of a package. A target can define a module or a test suite. // Targets can depend on other targets in this package, and on products in packages this package depends on. .target( name: "RealityKitContent" ), ] ) That is just issue one, RealityKitContent package build issue. Audio file format issue Another is about Audio File Format RCP supports. I remember is a place (WWDC?) saying .wav and .mp4 are supported to be used as audio source. But when I try to set up Spatial Audio, I find sometimes *.wav or *.mp3 can also be imported as AudioSourceFile. But the behavior is unpredictable. With two *.wav files SE_ball_hit_01.wav and SE_ball_hit_02.wav, only SE_ball_hit_01.wav is supported, 02 is reported as the format is not supported/ Check out my screenshots to see the details of two files. Two files have almost the same format (same sample rate or channel). I understand there might be different requirements for a source file to be used as Spatial or Ambient audio. But I haven't figured that out or there is nothing I can find helpful on Apple Documentation. So what is the rules? Thanks for reading and any thought is welcomed.

Hello! I have the great fortune to be working on a joint Apple/(unnamed brand) app. I am teamed up with a photogrammetry vendor, and will be assembling a scene in Maya, using a UDIM workflow from retouched assets in Substance Painter. This will be for an immersive environment, similar to Joshua Tree and the Hawaiian environments on AVP, which are great. We did a first pass, bringing USDZ files into Reality Composer Pro and sending to headset - very cool. However the models have a specular component that is washing everything out. After further researching, I learned that I need to not be using a Physically based shader, but instead, an Unlit shader. I had to manually create this in RCP, then it viewed properly. The issue is, every time I need to add a new asset, will I now have to manually create numerous custom shaders? Ideally, I am wanting to either figure out how to get an Unlit shader specified in Substance Painter that will get exported out properly, in addition to getting it to export from Maya. I tried using a Maya SurfaceShader - which is essentially unlit - but that does not export properly. I found that a Maya StandardSurface will export properly, however, it still imports as Physically Based shader. The conundrum is, I'll be working with a UDIM workflow, and would like to avoid having to manually create and hookup what could be between 10-40 textures per USDZ file. I guess what I'm trying to ask is - what is the preferred shader to use in Maya when exporting to USDZ that will import as Unlit? Or, is there a way to easily switch from Physically Based to Unlit inside RCP? And NOT doing it in Xcode/Swift, because the files I need to deliver need to be USDZ, the the developer will be assembling themselves. I'm using Maya because I need to work on my PC for the heavy GPU lifting, and also, it's just easier to assemble everything there VS Reality Composer Pro, which is like the iMovie of game engines. ;) (please take that constructively, it really needs to be more industry standard). I just need to make sure I can use Maya with the proper shaders to export my pieces, quickly send to RCP (with proper Unlit specification) then over to headset so I can check stuff in realtime with my photogrammetry vendor. Any help or advice is greatly appreciated! I'm really excited to be working on my first Vision Pro app. Thx!

From my early testing it seems like the object tracking works best for static objects. For example, if I am holding something in my hand the object tracker is slow to update. Is there anything that can be modified to decrease the tracking latency? I noticed that the Enterprise API has some override features is this something that can only be done using Enterprise?

My goal is to pin an attachment view precisely at the point where I tap on an entity using SpatialTapGesture. However, the current code doesn't pin the attachment view accurately to the tapped point. Instead, it often appears in space rather than on the entity itself. The issue might be due to an incorrect conversion of coordinates or values. My code: struct ImmersiveView: View { @State private var location: GlobeLocation? var body: some View { RealityView { content, attachments in guard let rootEnity = try? await Entity(named: "Scene", in: realityKitContentBundle) else { return } content.add(rootEnity) }update: { content, attachments in if let earth = content.entities.first?.findEntity(named: "Earth"),let desView = attachments.entity(for: "1") { let pinTransform = computeTransform(for: location ?? GlobeLocation(latitude: 0, longitude: 0)) earth.addChild(desView) // desView.transform = desView.setPosition(pinTransform, relativeTo: earth) } } attachments: { Attachment(id: "1") { DescriptionView(location: location) } } .gesture(DragGesture().targetedToAnyEntity().onChanged({ value in value.entity.position = value.convert(value.location3D, from: .local, to: .scene) })) .gesture(SpatialTapGesture().targetedToAnyEntity().onEnded({ value in })) } func lookUpLocation(at value: CGPoint) -> GlobeLocation? { return GlobeLocation(latitude: value.x, longitude: value.y) } func computeTransform(for location: GlobeLocation) -> SIMD3<Float> { // Constants for Earth's radius. Adjust this to match the scale of your 3D model. let earthRadius: Float = 1.0 // Convert latitude and longitude from degrees to radians let latitude = Float(location.latitude) * .pi / 180 let longitude = Float(location.longitude) * .pi / 180 // Calculate the position in Cartesian coordinates let x = earthRadius * cos(latitude) * cos(longitude) let y = earthRadius * sin(latitude) let z = earthRadius * cos(latitude) * sin(longitude) return position } } struct GlobeLocation { var latitude: Double var longitude: Double }

In RealityView I have two entities that contain tracking components and collision components, which are used to follow the hand and detect collisions. In the Behaviors component of one of the entities, there is an instruction to execute action through onCollision. However, when I test, they cannot execute action after collisions. Why is this?

I have a model entity (from Reality Composer Pro) I want to change the material of the model entity inside swift. The material is also imported in reality composer pro. I am copying the USDZ file of the material in the same directory as the script. This is the code I am using to reference the Material. do { // Load the file data if let materialURL = Bundle.main.url(forResource: "BlackABSPlastic", withExtension: "usdz") { let materialData = try Data(contentsOf: materialURL) // Check the first few bytes of the data to see if it matches expected types let headerBytes = materialData.prefix(4) let headerString = String(decoding: headerBytes, as: UTF8.self) // Print out the header information for debugging print("File header: \(headerString)") // Attempt to load the ShaderGraphMaterial let ScratchedMetallicPaint = try await ShaderGraphMaterial( named: "BlackABSPlastic", from: materialData ) print(ScratchedMetallicPaint) } else { print("BlackABSPlastic.usdz file not found.") } } catch { // Catch the error and print it print("BlackABSPlastic load failed: \(error)") // Attempt to infer file type based on the error or file content if let error = error as? DecodingError { switch error { case .typeMismatch(let type, _): print("Type mismatch: Expected \(type)") case .dataCorrupted(let context): print("Data corrupted: \(context.debugDescription)") default: print("Decoding error: \(error)") } } else { print("Unexpected error: \(error)") } } I am receiving these errors: File header: PK TBB Global TLS count is not == 1, instead it is: 2 Unable to create stage from in-memory buffer. BlackABSPlastic load failed: internalImportError Unexpected error: internalImportError am I doing anything wrong? I am able to access the materials of the model entity easily but this seems to something different. How can this be resolved? Thanks.

Dear Apple Developer Forums, I am just starting out developing in Swift, using RealityKit and Reality Composer Pro, as a project I'm working on is transitioning from using Unity to native only. I am trying to attach a particle system to the user's right hand, emitting from a single point, showing a 'spatial trail' of sorts, basically acting as a visualizer of the hand's spatial history. However, in Reality Composer Pro, when I anchor my particle emitter's parent entity using an Anchor component, even though the "Particles Inherit Transform" option is unticked (false), all of the spawned particles will also be anchored to the specified anchor position, as opposed to the expected behavior, which is that the emitter itself is anchored, but the spawned particles retain their spawn position in worldspace. Am I missing something, or does anchoring simply behave this way in relation to particle systems? Thank you! RCP 1.0, Xcode 15.4, visionOS 1.2

func createEnvironmentResource(image:UIImage) -> EnvironmentResource? { do { let cube = try TextureResource( cubeFromEquirectangular: image.cgImage!, quality: .normal, options: TextureResource.CreateOptions(semantic: .hdrColor) ) let environment = try EnvironmentResource( cube: cube, options: EnvironmentResource.CreateOptions( samplingQuality: .normal, specularCubeDimension: cube.width/2 // compression: .astc(blockSize: .block4x4, quality: .high) ) ) return environment }catch{ print("error: \(error)") } return nil } When I put this code in the project, it can run normally on the visionOS 2.0 simulator. When it is run on the real machine, an error is reported at startup: dyld[987]: Symbol not found: _$s10RealityKit19EnvironmentResourceC4cube7optionsAcA07TextureD0C_AC0A10FoundationE13CreateOptionsVtKcfC Referenced from: <DEC8652C-109C-3B32-BE6B-FE634EC0D6D5> /private/var/containers/Bundle/Application/CD2FAAE0-415A-4534-9700-37D325DFA845/HomePreviewDEV.app/HomePreviewDEV.debug.dylib Expected in: <403FB960-8688-34E4-824C-26E21A7F18BC> /System/Library/Frameworks/RealityFoundation.framework/RealityFoundation What is the reason and how to solve it ?

Compilation of the project for the WWDC 2024 session title Compose interactive 3D content in Reality Composer Pro fails. After applying the fix mentioned here (https://developer.apple.com/forums/thread/762030?login=true), the project still won't compile. Using Xcode 16 beta 7, I get these errors: error: [xrsimulator] Component Compatibility: EnvironmentLightingConfiguration not available for 'xros 1.0', please update 'platforms' array in Package.swift error: [xrsimulator] Component Compatibility: AudioLibrary not available for 'xros 1.0', please update 'platforms' array in Package.swift error: [xrsimulator] Component Compatibility: BlendShapeWeights not available for 'xros 1.0', please update 'platforms' array in Package.swift error: [xrsimulator] Exception thrown during compile: compileFailedBecause(reason: "compatibility faults") error: Tool exited with code 1

Hello everyone, I am a developer working on the Apple Vision Pro platform, currently developing an application that relies heavily on the Vision Pro LiDAR sensor. To ensure the accuracy and performance of my application, I would like to gather more detailed information about the technical specifications of the LiDAR sensor, particularly in the following areas: 1. Distance Accuracy: How accurate is the LiDAR sensor at different distances? 2. Spatial Resolution: What is the smallest object size that the sensor can detect? 3. Environmental Impact: How does the performance of the LiDAR sensor vary under different lighting conditions or environmental factors (e.g., reflective surfaces, fog)? I would greatly appreciate any detailed information or technical documentation regarding these questions. If there are any developers or Apple staff members who have insights on this, your input would be highly valued. Thank you in advance for your assistance!

My App will dynamically load different immersive furniture design scenes. After each scene is loaded, I need to set the HDR image as ImageBasedLight. How can I load EnvironmentResource dynamically? This way I can set the ImageBasedLightComponent dynamically

Hello Dev team, 3 weeks I'm looking for how I can export a static Cinema 4D objects WITH TEXTURES to Reality Composer Pro ! I can export it directly on USDA format and it works well for the 3D model in Reality Composer Pro, BUT, I can't have the textures on my model. My model is simple not colored ! Of course I expect to have textures applied on the good place and same appearance I've in Cinema 4D. Could you give me a process to do that please ? I'm using Cinema 4D R25 and Last XCode and Reality Composer Pro beta versions. Big big thanks to the one could help me on this. It will unblock many things to me!!!! Cheers Mathis

Is there any action that can clone the entity in RealityView to the number I want? If there is, please let me know. Thank you!

I created a simple Timeline animation with only a "Play Audio" action in RCP. Also a Behaviors Component setting an "OnTap" trigger to fire this Timeline animation. In my code, I simply run Entity.applyTapForBehaviors() when something happened. The audio can be normally played on the simulator but cannot be played on the device. Any potential bug leads this behavior? Env below: Simulator Version: visionOS 2.0 (22N5286g) XCode Version: Version 16.0 beta 4 (16A5211f) Device Version: visionOS 2.0 beta (latest)

When using the sample application of object tracking with apple Vision Pro, the left eye display shows the same position of the original object's target object, but the right eye is misaligned. Is there any way to resolve this issue?

I can execute an action by allowing Xcode to send a notification to Reality Composer Pro via NotificationCenter, or I can send notifications to Xcode through the Notification Action in Reality Composer Pro. However, I discovered that they were unable to accept notifications from both parties within my project. To ascertain whether there was an error in my code, I created a simple Demo project. I utilized the same code and determined that it functioned normally within the Demo project. It is perplexing that I am unable to resolve this issue. Do I require additional modifications?

It's all about notifications to trigger actions from RCP's new Timeline system. From Compose interactive 3D content in Reality Composer Pro I am actually starting to confuse why there was need to use Entity.applyTapForBehaviors in code to trigger content in Behaviors Component. Simply because in Behaviors Component, we have chosen OnTap to allow a "Tap Notification" to trigger our action (on a selected target object). Then I guess by selecting OnCollision this trigger, I should write something like CollisionEvent.entityA.applyCollisionForBehaviors, which we don't have. And ofc the collision on my object won't trigger this action (because I only did things in RCP not in code). Ignoring this post has pointed out we could use Behaviors Component's OnNotification to trigger something for now. I found that I could still use OnTap trigger but actually put my code Entity.applyTapForBehaviors under my subscribed collision's begin event. That actually works better than OnCollision So what is the design principles here? And how could I trigger a collision notification to let my Behaviors Component's OnCollision actually works?

The entity in My RealityView contains tracking components and allows them to track different places of the hand. However, I found that except for the fingertip of the index finger, the fingertip of the thumb, the palm and the wrist, all other positions cannot be tracked normally (such as the fingertip of the middle finger). How can I solve it (I think it may be a beta version of the bug)