RealityKit

RSS for tag

Simulate and render 3D content for use in your augmented reality apps using RealityKit.

RealityKit Documentation

Posts under RealityKit tag

273 results found
Sort by:
Post not yet marked as solved
1.3k Views

USDZ Animations with RealityKit

Has anyone been able to access multiple animations stored in a USDZ file through the RealityKit APIs?
Asked Last updated
.
Post marked as solved
744 Views

Reality Converter fails to convert FBX on M1 Mac

Does Apple have any documentation on using Reality Converter to convert FBX to USDZ on an M1 Max? I'm trying to convert an .fbx file to USDZ with Apple's Reality Converter on an M1 Mac (macOS 12.3 Beta), but everything I've tried so far has failed. When I try to convert .fbx files on my Intel-based iMac Pro, it succeeds. Following some advice on these forums, I tried to install all packages from Autodesk https://www.autodesk.com/developer-network/platform-technologies/fbx-sdk-2020-0 FBX SDK 2020.0.1 Clang FBX Python SDK Mac FBX SDK 2020.0.1 Python Mac FBX Extensions SDK 2020.0.1 Mac Still no joy. I have a work around - I still have my Intel-based iMac. But I'd like to switch over to my M1 Mac for all my development. Any pointers? Note: I couldn't get the usdzconvert command line tool to work on my M1 Mac either. /usr/bin/python isn't there.
Asked
by Todd2.
Last updated
.
Post not yet marked as solved
28 Views

Crash on launching framework which uses RealityKit in iOS 12 devices.

I have a framework which imports RealityKit. Minimum deployment target for the framework is iOS 13 and the app which uses the framework has iOS 12 as minimum deployment. RealityKit and RealityFoundation is set to optional in the build phases of the framework and the framework is optional in the app as well. But on launching the app in iOS 12 simulator I keep getting the following crash. DYLD, can't resolve symbol _$s10RealityKit10HasPhysicsMp If I change the minimum deployment of the framework to iOS 15, the same code works in iOS 12 simulator. No issues. Also if I try with any other module like SwiftUI, which has min iOS 13 deployment, I don't find the issue and app launches without any issue. Similarly, if I use ARKit instead of RealityKit, it works fine. No issues. Although ARKit min deployment is iOS 11 but it is also not available in simulators. So why does weak linking RealityKit not work when the parent framework targets iOS 13 and iOS 14, but works alright when targeting iOS 15? * * X code version: 13.3.1**
Asked
by ommkar23.
Last updated
.
Post not yet marked as solved
192 Views

Reality Composer crashes (text element)

Hi,when i select or try to edit a text element RC crashes. all other functions works fine.anybody know this problem?#######Application Specific Information:*** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Invalid parameter not satisfying: index <= [_itemArray count]'terminating with uncaught exception of type NSExceptionabort() called#######My system:Reality Composer Version 1.4macOS 10.15.4iMac (Retina 5K, 27-inch, 2017)3,8 GHz Quad-Core Intel Core i5Radeon Pro 580regardsmichael
Asked
by kunstwerk.
Last updated
.
Post not yet marked as solved
43 Views

Dynamic Mesh Updates

Dear all, In "Explore advanced rendering with RealityKit 2," Courtland presents how one can efficiently leverage dynamic meshes in RealityKit and update them at runtime. My question is quite practical: Say, I have a model of fixed topology and a set of animations (coordinates of each vertex per frame, finite duration) that I can only generate at runtime. How do I drive the mesh updates at 60FPS? Can I define a reusable Animation Resource for every animation once at startup and then schedule their playback like simple transform animations? Any helpful reply pointing me in the right direction is appreciated. Thank you. ~ Alexander
Asked
by AlexLike.
Last updated
.
Post not yet marked as solved
48 Views

Face tracking

Anyone could help me for face tracking with rear camera to apply a 3d object?
Asked
by Zeuma.
Last updated
.
Post not yet marked as solved
1k Views

Loading of older .reality files is broken on iOS 15 (works on iOS 14)

Hello, in our app we are downloading some user generated content (.reality files and USDZs) and displaying it within the app. This worked without issues in iOS 14 but with iOS 15 (release version) there have been a lot of issues with certain .reality files. As far as I can see USDZ files still work. I've created a little test project and the error message log is not really helpful. 2021-10-01 19:42:30.207645+0100 RealityKitAssetTest-iOS15[3239:827718] [Assets] Failed to load asset of type 'RealityFileAsset', error:Could not find archive entry named assets/Scéna17_9dfa3d0.compiledscene. 2021-10-01 19:42:30.208097+0100 RealityKitAssetTest-iOS15[3239:827598] [Assets] Failed to load asset path '#18094855536753608259' 2021-10-01 19:42:30.208117+0100 RealityKitAssetTest-iOS15[3239:827598] [Assets] AssetLoadRequest failed because asset failed to load '#18094855536753608259' 2021-10-01 19:42:30.307040+0100 RealityKitAssetTest-iOS15[3239:827598] throwing -10878 2021-10-01 19:42:30.307608+0100 RealityKitAssetTest-iOS15[3239:827598] throwing -10878 2021-10-01 19:42:30.307712+0100 RealityKitAssetTest-iOS15[3239:827598] throwing -10878 2021-10-01 19:42:30.307753+0100 RealityKitAssetTest-iOS15[3239:827598] throwing -10878 2021-10-01 19:42:30.307790+0100 RealityKitAssetTest-iOS15[3239:827598] throwing -10878 2021-10-01 19:42:30.307907+0100 RealityKitAssetTest-iOS15[3239:827598] throwing -10878 2021-10-01 19:42:30.307955+0100 RealityKitAssetTest-iOS15[3239:827598] throwing -10878 2021-10-01 19:42:30.308155+0100 RealityKitAssetTest-iOS15[3239:827598] throwing -10878 2021-10-01 19:42:30.308194+0100 RealityKitAssetTest-iOS15[3239:827598] throwing -10878 ▿ Failed to load loadRequest.   - generic: "Failed to load loadRequest." Basic code structure that is used for loading: cancellable = Entity.loadAsync(named: entityName, in: .main)     .sink { completion in         switch completion {         case .failure(let error):             dump(error)             print("Done")         case .finished:             print("Finished loading")         }     } receiveValue: { entity in         print("Entity: \(entity)")     } Is there anyway to force it to load in a mode that enforces compatibility? As mentioned this only happens on iOS 15. Even ARQuickLook can't display the files anymore (no issues on iOS 14). Thanks for any help!
Asked Last updated
.
Post marked as solved
696 Views

iOS 15 RealityKit AnchorEntity possible bug

The context is: ARSessionDelegate func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {} Upon Image Detection, place an overlay of a RC Entity at the ImageAnchor location. Using IOS 14, I have been reliably using: mySceneAnchorEntity = AnchorEntity(anchor: imageAnchor) to render additive RC scene content. Additive model entities render correctly in the field of view. When I upgraded to IOS 15 using the same code that had been working for many (~12) months on all versions of IOS 14, my code failed to render any RC scene content. I get a ghosting of all of the render content in the proper location, but visible only for a moment, then it disappears. So, I finally found the root cause of the issue. It appears that IOS 15 only renders correctly in my application using: mySceneAnchorEntity = AnchorEntity(world: imageAnchor.transform). This led to many frustrating days of debugging to find the root cause. As a side note, IOS 14 renders RC scene entities correctly using both variants of AnchorEntity: mySceneAnchorEntity = AnchorEntity(anchor: imageAnchor) and mySceneAnchorEntity = AnchorEntity(world: imageAnchor.transform) So, this leads me to believe there is an issue with the behavior of IOS 15 with the following version of AnchorEntity: mySceneAnchorEntity = AnchorEntity(anchor: imageAnchor)
Asked
by MikeNunez.
Last updated
.
Post not yet marked as solved
182 Views

AR Quick Look not always opening file

I've noticed sometimes that even on iOS devices with AR Quick Look, the browser displays the file as text instead of opening as 3D.I'm curious to know how widespread this is. Have tested on several devices of different sizes, but there seems to be no correlation between phone (iphone5, 6, 7, X) and when the player does this.There is a test page here, but is appears to be regardless of usdz file. Also on the page an image of the unexpected output.https://lifetime2.unicaster.net/ar/kalinka.htmlAnyone that has time to test, a solution, theory or other experiences, it would be much appreciated.I'm wondering if it could be a memory issue, or a MIME type, although that would consistently cause the error, right?The model is a convertion from GLB in the Reality Converter.
Asked Last updated
.
Post marked as solved
850 Views

Matching Virtual Object Depth with ARFrame Estimated Depth Data

I am trying to do a hit test of sorts between a person in my ARFrame and a RealityKit Entity. So far I have been able to use the position value of my entity and project it to a CGPoint which I can match up with the ARFrame's segmentationBuffer to determine whether a person intersects with that entity. Now I want to find out if that person is at the same depth as that entity. How do I relate the SIMD3 position value for the entity, which is in meters I think, to the estimatedDepthData value?
Asked Last updated
.
Post marked as solved
146 Views

Screen Space Coordinates to CVPixelBuffer Coordinates

I would like to extract depth data for a given point in ARSession.currentFrame.smoothedSceneDepth. Optimally this would end up looking something like: ARView.depth(at point: CGPoint) With the point being in UIKit coordinates just like the points passed to the raycasting methods. I ultimately to use this depth data to convert a 2D normalized landmark from a Vision image request into a 3D world space coordinate in the 3D scene - I only lack the accurate depth data for a given 2D point. What I have available is: The normalized landmark from the Vision request. Ability to convert this^ to AVFoundation coordinates. Ability to convert this^ to screen-space/display coordinates. When the depth data is provided correctly I can combine the 2D position in UIKit/screen-space coordinates with the depth (in meters) to produce an accurate 3D world position with the use of ARView.ray(through:) What I have not been able to figure out is how to get this depth value for this coordinate on screen. I can index the pixel buffer like this: extension CVPixelBuffer {     func value(for point: CGPoint) -> Float32 {         CVPixelBufferLockBaseAddress(self, .readOnly)         let width = CVPixelBufferGetWidth(self)         let height = CVPixelBufferGetHeight(self)          //Something potentially going wrong here.         let pixelX: Int = width * point.x         let pixelY: Int = height * point.y            let bytesPerRow = CVPixelBufferGetBytesPerRow(self)         let baseAddress = CVPixelBufferGetBaseAddress(self)! assert(kCVPixelFormatType_DepthFloat32 == CVPixelBufferGetPixelFormatType(depthDataMap))         let rowData = baseAddress + pixelY * bytesPerRow         let distanceAtXYPoint = rowData.assumingMemoryBound(to: Float32.self)[pixelX]                CVPixelBufferUnlockBaseAddress(self, .readOnly)         return distanceAtXYPoint     } } And then try to use this method like so:         guard let depthMap = (currentFrame.smoothedSceneDepth ?? currentFrame.sceneDepth)?.depthMap else { return nil } //The depth at this coordinate, in meters. let depthValue = depthMap.value(for: myGivenPoint) The frame semantics [.smoothedSceneDepth, .sceneDepth] have been set properly on my ARConfiguration. The depth data is available. If I hard-code the width and height values like so:         let pixelX: Int = width / 2         let pixelY: Int = height / 2 I get the correct depth value for the center of the screen. I have only been testing in portrait mode. But I do not know how to index the depth data for any given point.
Asked
by CodeName.
Last updated
.
Post not yet marked as solved
180 Views

RealityKit - how to move entity to other entity smoothly

I want to move toy_drummer entity with rotating toward other model entity. as you can see video, firstly, it move very well to left one then, I touch downward box entity but it move weird. video is youtu.be/3rb9W334Uwg modelentity2 is the entity that I want to move modelentityleft & modelentityright are the entity that modelentity2 should go to , so I can call objective entity *code when I touch left box entity to move, if(entity?.parent?.name=="left"){ self.parent.value4="leftarea" if(oktoadd){ self.counting = self.counting + 1 if(counting==1){ var position = firstResult.position let currentMatrix = self.modelentityleft!.transform.matrix let rotation = simd_float4x4(SCNMatrix4MakeRotation( -90.0 * .pi/180, 0,1,0)) let transform = simd_mul(currentMatrix, rotation) self.modelentity2!.move(to: transform, relativeTo: self.me, duration: 1, timingFunction: .linear) DispatchQueue.main.asyncAfter(deadline: .now() + 1.1) { counting=0 } } } } when click downward box entity, if(ent ity?.parent?.name=="down"){ self.parent.value4="downarea" if(oktoadd){ self.counting = self.counting + 1 if(counting==1){ self.parent.value4="animating" var position = firstResult.position let currentMatrix = self.modelentitydown!.transform.matrix let rotation = simd_float4x4(SCNMatrix4MakeRotation( 90.0 * .pi/180, 0,1,0)) let transform = simd_mul(currentMatrix, rotation) print("added moving ...") self.modelentity2!.move(to: transform, relativeTo: self.me, duration: 1, timingFunction: .linear) DispatchQueue.main.asyncAfter(deadline: .now() + 1.1) { counting=0 } } } } in my code, I firstly change rotation toward objective entity then move is there any way that I can do it more simply? (also wonder I hardcoded radian by 90 or -90 , any way to fix it by getting objective entity's position? )
Asked
by kotran.
Last updated
.
Post not yet marked as solved
71 Views

Ultra-large texture atlases in SpriteKit

I'm looking to use high-quality and large images as animated sprites via SpriteKit. Right now the default maximum size of a texture atlas is 4096*4096, which is at least an order of magnitude below what I need. There's an option to create custom maximum sizes, but the tiny default and the age of the SpriteKit framework is giving me second thoughts even though I'd very much like to stick with an Apple framework and not have to rely on another party like Unity. Before I invest my time and energy into SpriteKit I'd like to know whether the decade-old framework, running on modern hardware (A11 and newer), can support massive texture atlases while maintaining decent performance (~30 FPS). If not, is it possible to implement something similar in less antiquated frameworks, such as RealityKit (non-AR). Thanks in advance for your time.
Asked
by wmk.
Last updated
.
Post not yet marked as solved
469 Views

Is there a way to load RealityKit Custom entities?

I've been watching a 2019 Developer Video "Building Apps with RealityKit" and working along with it. It shows how to create a custom entity. It shows how to load entities from .usd files. How do you either load a custom entity, convert an Entity to a custom entity, or maybe move model hierarchy from an Entity to a custom entity? I assume there's a way to do this.
Asked
by ronm.
Last updated
.
Post not yet marked as solved
103 Views

How to limit RealityKit's framerate?

I'm currently weighing moving my project from SpriteKit to RealityKit as a way to future proof it. An important part of the project involves limiting the framerate sprites are displayed at. While RealityKit seems to be set at 60FPS, my project demands a much lower framerate. I should note that I'm not using RealityKit for AR, so there's no camera passthrough to worry about. Is there a way to limit RealityKit's rendering framerate?
Asked
by wmk.
Last updated
.