I need a MeshResource from ModelEntity to generate a box collider, but ModelEntity fails to load USDZ files from the Reality Composer Pro (RCP) bundle. This code works for loading an Entity: // Successfully loads as generic Entity previewEntity = try await Entity(named: fileName, in: realityKitContentBundle) But this fails when trying to load as ModelEntity: // Fails to load as ModelEntity modelEntity = try await ModelEntity(named: fileName, in: realityKitContentBundle) I found this thread mentioning: "You'll likely go from USDZ to Entity which contains a MeshResource when you load/init the USDZ file." But it doesn't explain ​how to actually extract the MeshResource. Could anyone advise: How to properly load USDZ files as ModelEntity from RCP bundles? How to extract MeshResource from a successfully loaded Entity? Alternative approaches to generate box colliders if direct extraction isn't possible? Sample code for extraction or workarounds would be greatly appreciated!

I'm writing a swift app that uses metal to render textures to the main view. I currently use a NSViewRepresentable to place a MTKView into the window and a MTKViewDelegate to perform the metal operations. It's running well and I see my metal view being updated. However, when I close the window (either through the user clicking the close button or by programatically using the appropriate @Environment(\.dismissWindow) private var dismissWindow) and then reopen the window, I no longer receive calls to MTKViewDelegate draw(in mtkView: MTView). If I manually call the MTKView::draw() function my view updates it's content as expected, so it seems to be still be correctly setup / alive. As best as I can tell the CVDisplayLink created by MTKView is no longer active (or at least that's my understanding of how the MTKView::draw() function is called). I've setup the MTKView like this let mtkView = MTKView() mtkView.delegate = context.coordinator // My custom delegate mtkView.device = context.coordinator.device // The default metal device mtkView.preferredFramesPerSecond = 60 mtkView.enableSetNeedsDisplay = false mtkView.isPaused = false which I was hoping would call the draw function at 60fps while the view is visible. I've also verified the values don't change while running. Does anyone have any ideas on how I could restart the CVDisplayLink or anyother methods to avoid this problem?? Cheers Jack

Hello! I'm currently building an app where I feed images into a Photogrammetry session to create a USDZ. Pretty straightforward, works great. We've recently started some testing on older devices, and have discovered that Photogrammetry is requiring devices that have LIDAR (we've seen some console logs referencing LIDAR if we stumble through a photogrammetry process without checking isSupported first) Judging from @swredcam's posting about ReefScan from November 24 (https://developer.apple.com/forums/thread/769221) it looks like Photogrammetry did work on those non-LIDAR devices. In my own testing on an iPhone 12 mini with iOS 17, PhotogrammetrySession says it's not supported. Since we're only feeding in a sequence of photos that have never had depth data, and they process fine on pro/max devices, we're curious why this would require a LIDAR sensor to work, when it seems like it did work without LIDAR in the past. Or is there some other limitation of non-pro devices that is causing photogrammetry to not be supported (especially on today's really powerful hardware) Thanks! ++md

The solo Leveling:arise is a game but the game mode is not switching on and game crashing everything time while playing

I notice some metal-cpp classes have static funtion like static URL* fileURLWithPath(const class String* pPath); static class ComputePassDescriptor* computePassDescriptor(); static class AccelerationStructurePassDescriptor* accelerationStructurePassDescriptor(); which return a new object. these classes also provide 'alloc' and 'init' function to create object by default. for object created by 'alloc' and 'init', I use something like NS::Shaderd_Ptr or call release directly to free memory. Because 'alloc' and 'init' not explicit call on these static function. I wonder how to correctly free object created by these static function? did they managed by autorelease pool?

I have used SceneKit for several years but recently have a problem where a scene with fewer than 50 nodes is partially drawn, i.e., some nodes are, some aren't, and greater than 50 nodes are always draw correctly. This seems to have happened since concurrency was introduced. (w.r.t. concurrency, I had been using DispatchQueue successfully before then.) Since all nodes (few or many) are constructed and implemented by the same functions etc. I'm baffled. When I print the node hierarchy all nodes are present whether few or many. SceneView() has [.rendersContinually] option selected. Every node created (few or many) has .opacity = 1.0, .isHidden = false I haven't tried setting-back the compiler version as that is not a long term solution, and I know the same code worked fine then.

After many former OS and Xcode updates, my Game Controller Swift code generates a "DIS-CONNECTED" MESSAGE. Mac Sequoia 15.2 Xcode 16.2 Tried to update PlayStation controller firmware on my Mac. Still no luck with Xcode and its use of a game controller with tvOS.

Helle there Currently, I’m attempting to create an interactive learning application with a 3D view. I’ve discovered the framework SceneKit, but I lack the necessary knowledge to animate, load and moving objects. Could someone kindly suggest some good articles or tutorials on this topic?

Hey all! I'm got my hands on a refurbished mac mini m1 and already diving into metal. At the moment, i'm currently studying graphics programming with opengl and got to a point where I can almost create a 3d cube. However, I noticed there aren't many tutorials for metal cpp but rather demos. One thing I love about graphic programming, is skinning/skeletal animation. At the moment, I can't find any sources or tutorials on how to load skeletal animations into metal-cpp. So, if I create my character in blender and had all types of animations all loaded into a .FBX or maybe .DAE and load this into metal api with metal-cpp, how can I go on about how this works?

I'm trying to build a Shader in "Reality Composer Pro" that updates from a start time. Initially I tried the following: The idea was that when the startTime was 0, the output would be 0, but then I would set startTime from within code and this would be compared with the current GPU time, and difference used to drive another part of the shader graph: if let testEntity = root.findEntity(named: "Test"), var shaderGraphMaterial = testEntity.components[ModelComponent.self]?.materials.first as? ShaderGraphMaterial { let time = CFAbsoluteTimeGetCurrent() try! shaderGraphMaterial.setParameter(name: "StartTime", value: .float(Float(time))) testEntity.components[ModelComponent.self]?.materials[0] = shaderGraphMaterial } However, I haven't found a reference to the time the shader would be using. So now I am trying to write an EntityAction to achieve the same effect. Instead of comparing a start time to the GPU's time I'm trying to animate one of the shader's uniform input. However, I'm not sure how to specify the bind target. Here's my attempt so far: import RealityKit struct ShaderAction: EntityAction { let startValue: Float let targetValue: Float var animatedValueType: (any AnimatableData.Type)? { Float.self } static func registerEntityAction() { ShaderAction.subscribe(to: .updated) { event in guard let animationState = event.animationState else { return } let value = simd_mix(event.action.startValue, event.action.targetValue, Float(animationState.normalizedTime)) animationState.storeAnimatedValue(value) } } } extension Entity { func updateShader(from startValue: Float, to targetValue: Float, duration: Double) { let fadeAction = ShaderAction(startValue: startValue, targetValue: targetValue) if let shaderAnimation = try? AnimationResource.makeActionAnimation(for: fadeAction, duration: duration, bindTarget: .material(0).customValue) { playAnimation(shaderAnimation) } } } ''' Currently when I run this I get an assertion failure: 'Index out of range (operator[]:line 797) index = 260, max = 8' Furthermore, even if it didn't crash I don't understand how to pass a binding to the custom shader value "startValue". Any clues of how to achieve this effect - even if it's a completely different way.

When inspecting the geometry in Xcode's metal debugger, I noticed that the shown "frustrum box" didn't make sense. Since Metal uses depth range 0,1 in NDC space, I would expect a vertex that is projected to z:0 to be on the front clipping plane of the frustrum shown in the geometry inspector. This is however not the case. A vertex with ndc z:0 is shown halfway inside the frustrum. Vertices with ndc z less than 0 are correctly culled during rendering, while the geometry inspector's frustrum shows that the vertex is stil inside the frustrum. The image shows vertices that are visually in the middle of the frustrum on z axis, but at the same time the out position shows that they are projected to z:0. How is this possible, unless there's a bug in the geometry inspector?

How many 32-bit variables can I use concurrently in a single thread of a Metal compute kernel without worrying about the variables getting spilled into the device memory? Alternatively: how many 32-bit registers does a single thread have available for itself? Let's say that each thread of my compute kernel needs to store and work with its own array of N float variables, where N can be 128, 256, 512 or more. To achieve maximum possible performance, I do not want to the local thread variables to get spilled into the slow device memory. I want all N variables to be stored "on-chip", in the thread memory space. To make my question more concrete, let's say there is an array thread float localArray[N]. Assuming an unrealistic hypothetical scenario where localArray is the only variable in the whole kernel, what is the maximum value of N for which no portion of localArray would get spilled into the device memory? I searched in the Metal feature set tables, but I could not find any details.

Following the post on https://developer.apple.com/documentation/realitykit/custommaterial it's simple to use shader for materials and get uniforms and params from each vertex. However it's not available for visionOS. Any alternative to use in this case? I want to write shader to fill material by myself. (I have shader experience from web, familiar with fragment shader)

Hi everyone, I’m experiencing a critical issue with USDZ files created in Reality Composer on an iPad 9th Generation (iPadOS 18.3). The files work perfectly on iPads from the 10th Generation onwards and on iPad Pros. However, on older devices like the iPad 9th Generation and older iPhones, QuickLook (file preview) crashes when opening them. This is a major issue because these USDZ files are part of an exhibition where artworks are extended with AR elements via a web page. If some visitors cannot view the 3D content, it significantly impacts the experience. What’s puzzling is that two years ago, we exported USDZ files from Reality Composer, made them available via a website, and they worked flawlessly on all devices, including older iPads and iPhones. Now, with the latest iPadOS, they consistently crash on older devices. Has anyone encountered a similar issue? Are there known limitations with QuickLook on older devices, or is there a way to optimize the USDZ files to prevent crashes? Could this be related to changes in iPadOS or RealityKit? Any advice or workaround would be greatly appreciated! Thanks in advance!

I have two apps released -- ReefScan and ReefBuild -- that are based on the WWDC21 sample photogrammetry apps for iOS and MacOS. Those run fine without LiDAR and are used mostly for underwater models where LiDAR does not work at all. It now appears that the updated photogrammetry session requires LiDAR data, and building my app on current xcode results in a non-working app. Has the "old" version of photgrammetry session been broken by this update? It worked very well previously so I would hate to see this regression to needing LiDAR. Most of my users do not have that.

Hey folks, I have a legacy game that is running OpenGL ES - and it no longer works on the simulators that are running Apple Silicon, ie iPhone 15 Pro, or the 13" iPads. And yes, i'm also running on Apple Silicon (M1 Max). The apps work fine on the actual devices, but the simulator crashes on any glDrawElements with a stack that looks like the following: I have not yet seen an announcement about this not working but i've seen mention in other apps of stopping to support GL (https://github.com/maplibre/maplibre-native/issues/2351) Can anyone shed some light? I'm obviously going to try to fix it, or find a recent sample app from which to start to see what might be up. Or move to metal, but i hadn't bargained for that level of effort atm ;) Any suggestions appreciated!

I have this minimum repro code: import SpriteKit import GameplayKit class MyGameScene3D: SCNScene { weak var node3D: MyNode3D! override init() { super.init() background.contents = UIColor.green let playground = SCNNode() playground.boundingBox = ( min: SCNVector3(x: 0, y: 0, z: 0), max: SCNVector3(x: 10, y: 10, z: 10)) let box = SCNNode(geometry: SCNBox(width: 1, height: 1, length: 1, chamferRadius: 0)) box.position = SCNVector3(x: 5, y: 5, z: 5) playground.addChildNode(box) playground.position = SCNVector3(x: 0, y: 0, z: 0) rootNode.addChildNode(playground) let light = SCNLight() light.type = .ambient let lightNode = SCNNode() lightNode.light = light rootNode.addChildNode(lightNode) let camera = SCNCamera() let cameraNode = SCNNode() cameraNode.camera = camera cameraNode.eulerAngles = SCNVector3(x: -3.14/2, y: 0, z: 0) cameraNode.position = SCNVector3(x: 5, y: 11, z: 5) rootNode.addChildNode(cameraNode) } required init?(coder: NSCoder) { fatalError("init(coder:) has not been implemented") } func handleTouchBegan(_ location: CGPoint) { let res = node3D.hitTest(location) print(res) } } class MyNode3D: SK3DNode { override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) { let touch = touches.first! let scene = scnScene as! MyGameScene3D let location = touch.location(in: self) print(location) scene.handleTouchBegan(location) } } class GameScene: SKScene { init() { super.init(size: CGSize(width: 500, height: 1000)) self.backgroundColor = .red let node3D = MyNode3D() let scene3D = MyGameScene3D() node3D.scnScene = scene3D scene3D.node3D = node3D node3D.isUserInteractionEnabled = true node3D.viewportSize = CGSize(width: 100, height: 200) node3D.position = CGPoint(x: 50, y: 100) addChild(node3D) let up = SKSpriteNode(color: .blue, size: CGSize(width: 500, height: 10)) up.anchorPoint = CGPoint(x: 0, y:0) up.position = CGPoint(x:0, y:200) addChild(up) let right = SKSpriteNode(color: .gray, size: CGSize(width: 10, height: 500)) right.anchorPoint = CGPoint(x:0,y: 0) right.position = CGPoint(x:100, y:0) addChild(right) } required init?(coder aDecoder: NSCoder) { fatalError("init(coder:) has not been implemented") } } Basically, I have a SK3DNode of size 100x200, positioned at lower left corner of the screen (see screenshot below). Then in this SK3DNode, I have a SCNScene, where I put a 10x10x10 Playground node at position (0, 0, 0). Then I put a camera node right at the top of the Playground at position (5, 11, 5), and the camera looks down along the -y axis, with euler angle = (-90, 0, 0). Then in this Playground, I put a small box of size 1x1x1, at the center of the Playground at (5, 5, 5). The 2 long bars (gray & blue) are just there to indicate the boundary of the SK3DNode. The result rendering is correct (see screenshot below). However, I can't get the hit test working. I tap on the center 1x1x1 box on screen, get the right coordinate printed out, but the hit test result is empty. I want to be get the center 1x1x1 box when hitting there. How can I do so? Update: I tried to loop through all the pixels from -2000 to 2000, and still no hit: func handleTouchBegan(_ location: CGPoint) { for x in -2000...2000 { print("handling x: \(x)") for y in -2000...2000 { let res = node3D.hitTest(location) if !res.isEmpty { print("\(x), \(y), \(res)") } } } print("Done") }

Now the examples of metal-cpp are target on desktop and using AppKit which is not supported on iOS. Is there any tips for developing with metal-cpp on mobile device?

Hi, The metal-cpp distribution appears to only contain headers for Foundation and Quartzcore. The LearnMetalCPP download [1] provides a ZIP with an metal-cpp-extensions directory containing AppKit.hpp and MetalKit.hpp headers. First question: Are these headers distributed anywhere else more publicly? Without these headers only the renderer can be fully written in C++ as far as I can tell, i.e. no complete C++ NSApplication. Second question: Will these headers, if needed, be maintained (e.g. updated and/or extended) by Apple along side metal-cpp? [1] https://developer.apple.com/metal/cpp/ Thank you and regards.

Hi, I’m creating a game and I’m just wondering if I can integrate GCVirtualController in my SwiftUI app.