I am fairly new to 3D model rendering and do not know where to start. I am trying to, ideally with ARKit & RealityKit or SceneKit, do a scan of an environment. This includes: Applying realistic textures to the model. Being able to save it as a .usdz file (to be able to open it within the App itself) Once it is save do post-processing measurements within the model. I would prefer to accomplish this feature by using a mesh, instead of the pointCloud that is used in the sample project of apple. Would this be doable using Apple's APIs and on a mobile device or would it be necessary to use a third party program? I have managed to create a USDZ file using SceneKit's .scene.write(to:,delegate:) method. However the saved file is a "single object" and it is not possible to use raycasting to do post-processing measurements in the model.

I am generating a USD file with RealityKit and ARKit. I want to, within the same app, open the USD file and display it as if it was a USDZ file. For the user to see, without having to send it to another device. Is this possible? Thanks

I wanted to show a progress of a certain part of the game using an entity that looks like a "pie chart", basically cylinder with a cut-out. And as progress is changed (0-100) the entity would be fuller. Is there a way to create this kind of model entity? I know there are ways to animated entities, warp them between meshes, but I was wondering if somebody knows how to achieve it in a simplest way possible? Maybe some kind of custom shader that would just change how the material is rendered? I do not need its physics body, just to show it. I know how to do it in UIKit and classic 2d UI Apple frameworks but here working with model entities it gets a bit tricky for me. Here is example of how it would look, examples are in 2d but you can imagine it being 3d cylinders with a cut-out. Thank you!

I am trying to implement a way to rotate a 3D model around its y axis, but this doesn't seem to work. What am I missing? The scene only contains one model entity. @State private var rotateBy:Double = 0.0 RealityView { content in do { let entity = try await Entity.init(named: "VinylScene", in: realityKitContentBundle) entity.scale = SIMD3<Float>(repeating: 0.6) content.add(entity) } catch { ProgressView() } } .gesture( DragGesture(minimumDistance: 0.0) .targetedToAnyEntity() .onChanged { value in let location3d = value.convert(value.location3D, from: .local, to: .scene) let startLocation = value.convert(value.startLocation3D, from: .local, to: .scene) let delta = location3d - startLocation rotateBy = Double(atan(delta.x * 200)) } )

Hi. Can anyone who's into cosmology give hints as to how I might depict and animate dark matter for the VisionPro?

I've got a couple 2D PNG assets that I want to add to a scene made of a couple other udsz files in RCP (picture adding a couple 2D videogame characters to a simple 3D diorama). When I try to drag the PNGs to the workspace or the file tree…nothing happens. I found a walkthrough on Medium (called "Importing and Exporting Personalized Objects for Augmented Reality: Reality Composer and SwiftUI" for those curious as I can't link to Medium posts here) that makes it look like users could do this with simple drag-and-drop. The Medium post is from June 2023, and in the screenshots RCP visually looks a lot more like Reality Composer on iPad, so I'm assuming it's changed a lot since then? Is there still a way to do this? I've tried adding the 2D elements to a scene with Blenders "import images as planes," but I'm getting weird halos around them and was hoping RCP could make the process a bit easier/cleaner.

Hello, I've been trying to render these models in a VisionOS app using RealityKit's Model3D API. The heart seem to appear dark all the time. Any thoughts on why this would happen? Color.clear .overlay { Model3D(named: modelName, bundle: realityKitContentBundle) { model in model.resizable() .scaledToFit() .rotation3DEffect( Rotation3D( eulerAngles: .init(angles: orientation, order: .xyz) ) ) .frame(depth: modelDepth) .offset(z: -modelDepth / 2) .accessibilitySortPriority(1) } placeholder: { ProgressView() .offset(z: -modelDepth * 0.75) } } .dragRotation(yawLimit: .degrees(120), pitchLimit: .degrees(20)) .offset(z: modelDepth)

I'm working on a project wherein RealityKit for iOS will be used to display 3D files (USDZ) in a real-world environment. The model will also need to animate differently depending on which button is pressed. When using models that are downloaded from various websites or via Apple QuickLook, the code functions well. I can hold the animation in place and click a button to play it. Unfortunately, although the model (through blender) my team provided is animating in SceneKit, it does not play at all when left in the real world, not even when a button is pressed. I checked RealityKit USDZ tool, and found usdz file is not valid, they are not figure out whats wrong. Could you please help me figure out what's wrong with my USDZ file? Working USDZ: https://developer.apple.com/augmented-reality/quick-look/models/drummertoy/toy_drummer_idle.usdz My file: https://drive.google.com/file/d/1UibIKBy2fx4q0XxSNodOwQZMLgktKiKF/view?usp=sharing

Basically the title. Is it all done in Reality Composer Pro or need some coding in swift.?

I have a single fbx model with several animations(idle, walk, run, eat......), but after I convert it to usdz format, I can only play 1 animation, where can I find other animations and how can I play them, or USDZ doesn't support it yet? Thank you. Cyan

Let's say I've created a scene with 3 models inside side by side. Now upon user interaction, I'd like to change these models to another model (that is also in the same reality composer pro project). Is that possible? How can one do that? One way I can think of is to just load all the individual models in RealityView and then just toggle the opacity to show/hide the models. But this doesn't seem like the right way for performance/memory reasons. How do you swap in and out usdz models?

Hi I would like to manipulate blend shapes of a usdz file in VisionOS, but I can not figure out how I could do that. In SceneKit it was a piece of cake with SCNNode and morphs, but with Entity class I got stuck. is it even possible? (I would be very disappointed if not)

I captured my office using 3D Scanner and get a USDZ file. The file contains a 3-D Model and a Physically based material. I can view the file correctly with texture on Xcode and Reality Composer Pro. But when using RealityView to present the model in immersive space. I got the model in whole black. My guess is my Material doesn't have a shader graph? Does anyone caught into similar issue? How to solve it?

Hello! I have a few questions about using RealityKit on iPadOS and Mac Catalyst. I currently have a model in Cinema 4D that has no materials added to it. I need to add a couple of simple materials to everything. My goal is to move my model from Cinema 4D to my RealityKit app. The model can load fine in the app. However, I'm having issues with the materials. First, when I export from Cinema 4D as a USD, the materials don't seem to come with it. I thought materials came with the model. I just get a pink striped model (no materials), which is not ideal. So, that crossed out making materials in Cinema 4D for me. Here's a test model with materials added: Here's what it looks like when exported as a USDA in Reality Composer Pro. It looks the same when exported as a USDZ: I checked materials when exporting from Cinema 4D, so I don't know what I am doing wrong. So, I thought of another idea instead of making materials in Cinema 4D. What if I used Apple's new Reality Composer Pro app to add materials to my model? You can add physically based materials or even custom shader materials with nodes. I thought that would work, and it does. When I export the model as a USDZ with physically based materials, they appear fine and work in my app. However, what about custom shader materials? Whenever I play with a custom shader material and apply it to my model, I am left with problems. Look at this image. This is my model in Reality Composer Pro with two types of materials added from the app. The water and sand on the beach are created with physically based materials in the app. No nodes. The gold/metal ball is created with a custom shader material with nodes. Looks fine, right? When I drag an exported USDZ of this into my Xcode project, it even looks good in the Xcode preview of the file under Resources. (Note I am not adding the USDZ as an . rkassets as Apple suggests as .rkassets folders are only available for visionOS. This is an iPadOS + Catalyst app.) However, when I run the actual app, only the physically based materials actually display correctly: Besides the lighting in the scene, the physically based materials look good. However, the metal ball that used a custom shader material? It looks gray. Why? How can I fix this? I guarantee this is not a problem with my app or its lighting setup etc. I tried loading a custom shader material in a visionOS simulator and it worked! But, not for this. I know Reality Composer Pro seems to be very focused on visionOS right now, but this is still just a USDZ file. Why aren't these custom shaders working? I've been working on this problem for at least 24+ hours, so I've tried as much as I could imagine. I thought Reality Composer Pro would be good to do textures in as it would be less error prone when moving the model over to Xcode, compared to moving materials from Cinema 4D to Xcode, and I kind of proved that with my second photo. For RealityKit on iPadOS + Catalyst, how should I be applying materials? What am I doing wrong? P.S. Yes, this is a nonAR project with a virtual camera, which is possible for RealityKit. Thanks for the help! :)

Hello All - I'm receiving .usdz files from a client. When previewing the .usdz file in Reality Converter - the materials show up as expected. But when I load the .usdz in Reality Composer Pro, all the materials show up as grey. I've attached an image of the errors I'm get inside Reality Converter, to help trouble shoot. What steps can I take to get these materials working in my Reality Composer Pro project? Thanks!

Hi guys, I thought I make a visionOS test app with the Apple's native robot.usdz file. My plan was to rotate limbs of the robot programatically, but while I can see the bones in previous Xcode versions and in Blender, somehow I can not reach them in Xcode 15.3 or Reality Composer Pro. Has anyone any experience with that?

USDZ is not getting the job done for AR on ios. Are there plans to utilize WebXR in future versions of ios for phones and ipads, so that developers might leverage all the capabilities that the GLB file format provides? By embracinng WebXR, Android is providing a much better environment to build AR experiences. As content creators, we would like to support all of our users with a common code stack and workflow. Thanks for any insights.

Load usdz file with realitykit, change pbr to customaterial like describe here: https://developer.apple.com/documentation/realitykit/modifying-realitykit-rendering-using-custom-materials -> Model texture look smaller when preview and on iOS device than use preview on Mac or Blender Step to reproduce: clone repo: https://github.com/ngocdv95/realitykit-demo-scale-bug Open project bellow View preview usdz file in usdz folder Open ContentView file, open SwiftUI preview Look preview: first it load usdz file, everything correct, wait 3 second (code line 74), material change and texture look smaller.

Hello, I'm exporting a 3D model with a shader created in Reality Composer Pro's Shader Graph to a USDZ file for viewing in QuickLook. Once exported, the USDZ file in QuickLook appears without the shader material. When I import the file back into Xcode, the material renders properly. Is it possible to publish MaterialX shaders to be viewed in QuickLook? Hopefully it is, because it seems the push to MaterialX and USDZ is supposed to be for universal support :) Any guidance is appreciated!

I have a USDZ model called 'GooseNModel' in the visionOS App project. I'm sure that this model contains an animation, so I wrote the following code to display the model with animation: import SwiftUI import RealityKit RealityView{ content in if let GooseNModel = try? await Entity(named: "GooseNModel"), let animation = GooseNModel.availableAnimations.first { GooseNModel.playAnimation(animation) content.add(GooseNModel) } } But when I ran it, I found that it did not contain animation as I imagined, but a static model. How to solve it?