Hello, I'm here. I am posting this in the hope that you can give me some advice on what I would like to achieve. What I would like to achieve is to download the USDZ 3D model from the web server within the visionOS app and display it with the Shared Space volume (volumetric) size set to fit the downloaded USDZ model. Currently, after downloading USDZ and generating it as a Model Entity, Using openWindow, The Model Entity is created as a volumetric WindowGroup in the RealityViewContent of the RealityView using openWindow. The Model Entity generated by downloading USDZ is added to the RealityViewContent of the RealityView in the View called by openWindow. The USDZ downloaded by the above method appears in the volume on visionOS without any problems. However, the size of the USDZ model to be downloaded is not uniform, so it may not fit in the volume. I am trying to generate a WindowGroup with openWindow using Binding with the appropriate size value set to defaultSize, but I am not sure which property of ModelEntity can be set to the appropriate value for defaultSize. The attached image does not have the correct position and I would like to place the position down if possible. I would appreciate your advice on sizing and positioning the downloaded USDZ to fit in the volume. Incidentally, I tried a plane style window and found that it displayed a USDZ Model Entity that was much larger in scale compared to the volume, so I have decided not to support a plane style window. If there is any information on how to properly set the position and size of the USDZ files created by visionOS and RealityKit, I would appreciate it if you could also provide it. Best regards. Sadao Tokuyama https://twitter.com/tokufxug https://1planet.co.jp/tech-blog/category/applevisionpro

The Apple documentation seems to say RealityKit should obey the autoplay metadata, but it doesn't seem to work. Is the problem with my (hand coded) USDA files, the Swift, or something else? Thanks in advance. I can make the animations run with an explicit call to run, but what have I done wrong to get the one cube to autoplay? https://github.com/carlynorama/ExploreVisionPro_AnimationTests import SwiftUI import RealityKit import RealityKitContent struct ContentView: View { @State var enlarge = false var body: some View { VStack { //A ModelEntity, not expected to autoplay Model3D(named: "cube_purple_autoplay", bundle: realityKitContentBundle) //An Entity, actually expected this to autoplay RealityView { content in if let cube = try? await Entity(named: "cube_purple_autoplay", in: realityKitContentBundle) { print(cube.components) content.add(cube) } } //Scene has one cube that should auto play, one that should not. //Neither do, but both will start (as expected) with click. RealityView { content in // Add the initial RealityKit content if let scene = try? await Entity(named: "Scene", in: realityKitContentBundle) { content.add(scene) } } update: { content in // Update the RealityKit content when SwiftUI state changes if let scene = content.entities.first { if enlarge { for animation in scene.availableAnimations { scene.playAnimation(animation.repeat()) } } else { scene.stopAllAnimations() } let uniformScale: Float = enlarge ? 1.4 : 1.0 scene.transform.scale = [uniformScale, uniformScale, uniformScale] } } .gesture(TapGesture().targetedToAnyEntity().onEnded { _ in enlarge.toggle() }) VStack { Toggle("Enlarge RealityView Content", isOn: $enlarge) .toggleStyle(.button) }.padding().glassBackgroundEffect() } } } No autospin meta data #usda 1.0 ( defaultPrim = "transformAnimation" endTimeCode = 89 startTimeCode = 0 timeCodesPerSecond = 24 upAxis = "Y" ) def Xform "transformAnimation" () { def Scope "Geom" { def Xform "xform1" { float xformOp:rotateY.timeSamples = { ... } double3 xformOp:translate = (0, 0, 0) uniform token[] xformOpOrder = ["xformOp:translate", "xformOp:rotateY"] over "cube_1" ( prepend references = @./cube_base_with_purple_linked.usd@ ) { double3 xformOp:translate = (0, 0, 0) uniform token[] xformOpOrder = ["xformOp:translate"] } } With autoplay metadata #usda 1.0 ( defaultPrim = "autoAnimation" endTimeCode = 89 startTimeCode = 0 timeCodesPerSecond = 24 autoPlay = true playbackMode = "loop" upAxis = "Y" ) def Xform "autoAnimation" { def Scope "Geom" { def Xform "xform1" { float xformOp:rotateY.timeSamples = { ... } double3 xformOp:translate = (0, 0, 0) uniform token[] xformOpOrder = ["xformOp:translate", "xformOp:rotateY"] over "cube_1" ( prepend references = @./cube_base_with_purple_linked.usd@ ) { quatf xformOp:orient = (1, 0, 0, 0) float3 xformOp:scale = (2, 2, 2) double3 xformOp:translate = (0, 0, 0) uniform token[] xformOpOrder = ["xformOp:translate", "xformOp:orient", "xformOp:scale"] } } } }

I wanted to scan the person's face via a true depth camera generate a 3d face model and display the scene I didn't find any solutions. Has anybody integrated this type of functionality? Please help me out or provide a sample code to scan and preview the scanned face into sceneview or ARview

We are trying to save usdz file in file manager some time its saved but some time we are getting the error. Like: path.absoluteURL file:///var/mobile/Containers/Data/Application/6D14A430-47B4-45F2-9D0D-6C31588A6A03/Documents/2896837C-C7E0-4FA8-BFE2-21A59B26D801.usdz Warning: in SdfPath at line 151 of sdf/path.cpp -- Ill-formed SdfPath </2896837CC7E04FA8BFE221A59B26D801>: syntax error Coding Error: in _IsValidPathForCreatingPrim at line 3338 of usd/stage.cpp -- Path must be an absolute path: <> cannotCreateNode(path: "/2896837CC7E04FA8BFE221A59B26D801") func saveFileLocal() { if let finalResult { let fm = FileManager.default var path = fm.urls(for: .documentDirectory, in: .userDomainMask).first! let fileName = "(UUID().uuidString).usdz" path.appendPathComponent(fileName) do { try finalResult.export(to: path.absoluteURL) } catch{ print(error) } } } func removeFiles() { var filePath = "" let fm = FileManager.default let path = fm.urls(for: .documentDirectory, in: .userDomainMask).first! do{ let content = try fm.contentsOfDirectory(atPath: path.path) for c in content{ filePath = path.appendingPathComponent(c).absoluteString if let url = URL(string: filePath){ try fm.removeItem(at: url) } } } catch{ print(error) } }

With the latest software upgrades such Sonoma 14.0, Xcode 15.0 Beta 8 and Reality Composer Pro, what is the alternative to behaviours for a web based AR app for iOS 17? Any example, documentation or tutorial about how to use custom components to provide user interactivity such click events and be able to export the scene as USDZ?

I am using the RoomPlanExampleApp to export a USDZ. When I open the USDZ on Reality Converter, I get this error Invalid Structure in USDZ file The root layer of the package must be a usdc file and must not include any external dependencies that participate in stage composition.

Anybody has noticed pivot issue in constructed model through object capture. Ideally pivot of object should be centre of bounding box but with new macOS changes now pivot is at 0,0,0 (below the bounding box) Here is a quick comparison Old v/s new

Does anyone know where I can find quality assets in USDZ format? For Unity and Unreal Engine, I just use the built-in asset stores. There seem to be a number of third-party 3D model stores like Laughing Squid, but they tend not to have models in USD format. In particular, I'm looking for some nice-looking explosions for a RealityKit-based visionOS game I'm writing. Some nice boulders would also be useful. Thanks in advance!

Hi, I've been exploring a project with visionOS, and have been quite confused on capabilities and workflows for using custom materials in RealityKit & RealityComposerPro for visionOS. Ideally I would be able to create / load / modify a model and its materials in Blender, export to openUSD and have it load in fully in RCP, but this hasn't been the case. Instead different aspects of the material don't seem to be exported correctly and that has lead me to investigate more into understanding MaterialX, openUSD and Metal, and how they work in visionOS, RealityKit and RealityComposer. MaterialX was announced as a primary format for working with 3D materials, but the .mtlx file format doesn't appear to be compatible in RCP directly - specifically trying materials provided in the AMD OpenGPU MaterialX Library. (note: AFAIK, Blender does not currently support MaterialX) When downloading a material, this provides a folder with the textures and corresponding .mtlx, but currently in RCP (Xcode 15.6 beta) this file is ignored. Similarly, trying to load it using ShaderGraphMaterial fails with 'Error in prim' and no other details that I can see. It also appears that there is a way of bundling MaterialX files within an openUSD file (especially implied by the error about Prims), but I haven't been able to understand how this can be done, or if this is the correct approach. Unpacking the Apple-provided materials in RCP from usdz to usda, these appear to define the shaders in openUSD and reference the RCP MaterialX Preview Shader (presumably created using the Shader Graph). There is also reference however from the official MaterialX.org and OpenUSD around using a USD / MaterialX Plugin to enable compatibility. I've also tried, and followed along with the introductory tutorial on the in-built ShaderGraph, and find it difficult to understand and quite different from Blender's Shader Nodes, but it currently appears that this is the primary way promoted to create and work with materials. Finally, I had expected that CustomMaterials using Metal Shaders would be available, as Metal was mentioned for Fully Immersive Spaces, and 'Explore Advanced Rendering with RealityKit 2' from WWDC21 covers using custom shaders, but this is not listed as included in visionOS and according to an answer here, it's not currently planned. (although the documentation here still mentions Metal briefly) Overall, what are the suggestions for workflows with materials for RealityKit on visionOS? Is there a fully compatible path from Blender -> openUSD -> RealtyComposerPro? Do I need to export materials and models from Blender individually and rebuild them in RCP using the ShaderGraph? Can I utilise existing MaterialX materials in RealityComposerPro, and if so, how? Are there any other good resources for getting comfortable and understanding the nodes within the ShaderGraph? what WWDC talks would be good to revise on this? Really appreciate any guidance!

Is there support for using multiple UV channels in AR QuickLook in iOS17? One important use case would be to put a tiling texture in an overlapping tiling UV set while mapping Ambient Occlusion to a separate unwrapped non-overlapping UV set. This is very important to author 3D content combining high-resolution surface detail and high-quality Ambient Occlusion data while keeping file size to a minimum.

I have a USDZ file with a 3d object that is animated. Loading the 3d object into an ARView on iPad the animation shows and plays as intended. Loading, the same 3d object in a RealityView in VisionOS, results in only part of the animation playing. The part of the animation that doesn't play is a mesh animation. This animation doesn't play correctly when viewing in Reality Composer Pro either. Is this a limitation of the beta software? Will mesh animations be supported on VisionOS and in RealityViews? ARView is unavailable on VisionOS. What other options do I have to support the existing animation?

When I create an USDZ file from the original Reality Composer (non pro) and view it in the Vision OS simulator, the transforms and rotations don't look similar. For example a simple Tap and Flip behaviour does not rotate similar in Vision OS. Should we regard RC as discontinued sw and only work with RC-pro? Hopefully Apple will combine the features from the original RC into the new RC pro !

If you have a scene with a simple custom .usda material applied to a primitive like a cube, the exported (.usdz) material definition is unknown for tools like Reality Converter Version 1.0 (53) or Blender Version 3.6.1. Reality Converter shows up some warnings "Missing references in USD file", "Invalid USD shader node in USD file". Even Reality Composer Pro is unable to recreate the material correct with it's own exported .usdz files. Feedback: FB12699421

Hello, I have created a material in RCP and applied it via "Material Binding" to 3D geometry within my scene... however, when I export it out as a USDz file and open in Xcode... the material changes have not persisted. Any suggestions on why this is so?

Hi! I created manually USDZ with one cube to anchoring on wall (vertical plane) #usda 1.0 ( defaultPrim = "Root" metersPerUnit = 1 upAxis = "Y" ) def Xform "Root" ( assetInfo = { string name = "Root" } kind = "component" ) { def Xform "Geom" ( prepend apiSchemas = [ "Preliminary_AnchoringAPI" ] ) { # token preliminary:anchoring:type = "plane" # token preliminary:planeAnchoring:alignment = "vertical" matrix4d xformOp:transform = ( (0, 0, 0, 0), (0, 0, 0, 0), (0, 0, 0, 0), (0, 0, 0.2, 1) ) def Xform "Group" { def Cube "cube_0" { float3[] extent = [(-1, -1, -1), (1, 1, 1)] uniform bool doubleSided = 1 rel material:binding = </Root/Materials/material_0> matrix4d xformOp:transform = ( (0.1, 0, 0, 0), (0, 0.1, 0, 0), (0, 0, 0.01, 0), (0, 0, 0, 1) ) uniform token[] xformOpOrder = ["xformOp:transform"] } } } } It's displayed correct in ARQuickLook When I add second cube to USD(z) #usda 1.0 ( defaultPrim = "Root" metersPerUnit = 1 upAxis = "Y" ) def Xform "Root" ( assetInfo = { string name = "Root" } kind = "component" ) { def Xform "Geom" ( prepend apiSchemas = [ "Preliminary_AnchoringAPI" ] ) { # token preliminary:anchoring:type = "plane" # token preliminary:planeAnchoring:alignment = "vertical" matrix4d xformOp:transform = ( (0, 0, 0, 0), (0, 0, 0, 0), (0, 0, 0, 0), (0, 0, 0.2, 1) ) def Xform "Group" { def Cube "cube_0" { float3[] extent = [(-1, -1, -1), (1, 1, 1)] uniform bool doubleSided = 1 rel material:binding = </Root/Materials/material_0> matrix4d xformOp:transform = ( (0.1, 0, 0, 0), (0, 0.1, 0, 0), (0, 0, 0.01, 0), (0, 0, 0, 1) ) uniform token[] xformOpOrder = ["xformOp:transform"] } def Cube "cube_1" { float3[] extent = [(-1, -1, -1), (1, 1, 1)] uniform bool doubleSided = 1 rel material:binding = </Root/Materials/material_0> matrix4d xformOp:transform = ( (0.1, 0, 0, 0), (0, 0.1, 0, 0), (0, 0, 0.01, 0), (0.3, 0, 0, 1) ) uniform token[] xformOpOrder = ["xformOp:transform"] } } } } ARQuickLook display scene ~ 10 cm from wall, and then scene have more cubes we see increased distance from wall. Here for two cubes I also tryed recreate scene in Reality Composer on iPhone. Everything ok for one cube, and ok when preview in App(ARKit?) for two cubes, but when export scene in RealityComposer macOS to USDZ we again see wrong distance for two cubes and more. For tests I use iPhone 13 Pro Max with iOS 16.3.1

I am trying to follow the WWDC 2022 video Explore USD tools and rendering: https://developer.apple.com/videos/play/wwdc2022/10141/ I followed the steps here to create an Xcode project that uses OpenUSD to load scenes. https://developer.apple.com/documentation/metal/metal_sample_code_library/creating_a_3d_application_with_hydra_rendering?language=objc After installing OpenUSD and generating an Xcode project, I opened Xcode, set the scheme to hydraplayer and clicked the build button. The code compiles but fails to link with a bunch of undefined symbols errors like this one: Undefined symbol: pxrInternal_v0_23__pxrReserved__::GfMatrix4d::SetDiagonal(double) I tried to tag this post wwdc2022-10141, but the tag was not found so I tagged a related session.

Hi, I have this pesky issue where whenever I open an fbx or usdc file in Reality Converter it fails to load the image texture due to lack of permissions. I then have to click on each individual one to open a file dialog and manually open it. This gets boring very quickly. I have granted full disk access to Reality Converter in my Privacy & Security Preferences but this made no difference. Does anyone know how to get around this issue?

Hi all. I am new to swift and AR. I'm trying a project on AR and ran into a problem that I can't change the material on the models. With geometry such as a sphere or a cube, everything is simple. Tell me what am I doing wrong? My simple code: @IBOutlet var sceneView: ARSCNView! var modelNode: SCNNode! override func viewDidLoad() { super.viewDidLoad() sceneView.delegate = self sceneView.showsStatistics = true let scene = SCNScene(named: "art.scnassets/jacket.usdz")! modelNode = scene.rootNode.childNode(withName: "jacket", recursively: true) let material = SCNMaterial() material.diffuse.contents = UIImage(named: "art.scnassets/58.png") modelNode.childNodes[0].geometry?.materials = [material] sceneView.scene = scene

such as a human model, use code to let him walk to a certain position?

USD added draco compression support in 19.11. is this something that apple is also considering to adopt?