If I place the .usdz file in the project directory alongside other .swift files, ModelEntity loads it perfectly. However, if I try to load the same file from Reality Composer Pro under RealityKitContent.rkassets, I get the error: resourceNotFound("heart").
Could someone help me with this? Thank you so much
Code:
//
// TestttttttApp.swift
// Testtttttt
//
// Created by Zhendong Chen on 2/17/25.
//
import SwiftUI
@main
struct TestttttttApp: App {
var body: some Scene {
WindowGroup {
ContentView()
}
.windowStyle(.volumetric)
}
}
//
// ContentView.swift
// Testtttttt
//
// Created by Zhendong Chen on 2/17/25.
//
import SwiftUI
import RealityKit
import RealityKitContent
struct ContentView: View {
@State private var enlarge = false
var body: some View {
RealityView { content in
do {
// MARK: Work
let scene = try await ModelEntity(named: "heart")
content.add(scene)
// MARK: Doesn't work
// let scene = try await ModelEntity(named: "heart", in: realityKitContentBundle)
// content.add(scene)
} catch {
print(error)
}
}
}
}
#Preview(windowStyle: .volumetric) {
ContentView()
}
RealityKit
RSS for tagSimulate and render 3D content for use in your augmented reality apps using RealityKit.
Posts under RealityKit tag
200 Posts
Sort by:
Post
Replies
Boosts
Views
Activity
Apple published a set of examples for using system gestures to interact with RealityKit entities. I've been using DragGesture a lot in my apps and noticed an issue when using it in an immersive space.
When dragging an entity, if I turn my body to face another direction, the dragged entity does not stay relative to my hand. This can lead to situations where the entity is pulled very close to me, or pushed far way, or even ends up behind me.
In the examples linked above, there are two versions of how they use drag.
handleFixedDrag: This is similar to what I'm doing now. It uses the value from value.gestureValue.translation3D as the basis for the drag
handlePivotDrag: This version aims to solve the problem I described above by using value.inputDevicePose3D as the basis of the gesture.
I've tried the example from handlePivotDrag, but it has one limitation. Using this version, I can move the entity around me as if it were on the inside of an arc or sphere. However, I can no longer move the entity further or closer. It stays within a similar (though not exact) distance relative to me while I drag.
Is there a way to combine these concepts? Ideally, I would like to use a gesture that behaves the same way that visionOS windows do. When we drag windows, I can move them around relative to myself, pull them closer, push them further, all while avoiding the issues described above.
Example from handleFixedDrag
mutating private func handleFixedDrag(value: EntityTargetValue<DragGesture.Value>) {
let state = EntityGestureState.shared
guard let entity = state.targetedEntity else { fatalError("Gesture contained no entity") }
if !state.isDragging {
state.isDragging = true
state.dragStartPosition = entity.scenePosition
}
let translation3D = value.convert(value.gestureValue.translation3D, from: .local, to: .scene)
let offset = SIMD3<Float>(x: Float(translation3D.x),
y: Float(translation3D.y),
z: Float(translation3D.z))
entity.scenePosition = state.dragStartPosition + offset
if let initialOrientation = state.initialOrientation {
state.targetedEntity?.setOrientation(initialOrientation, relativeTo: nil)
}
}
Example from handlePivotDrag
mutating private func handlePivotDrag(value: EntityTargetValue<DragGesture.Value>) {
let state = EntityGestureState.shared
guard let entity = state.targetedEntity else { fatalError("Gesture contained no entity") }
// The transform that the pivot will be moved to.
var targetPivotTransform = Transform()
// Set the target pivot transform depending on the input source.
if let inputDevicePose = value.inputDevicePose3D {
// If there is an input device pose, use it for positioning and rotating the pivot.
targetPivotTransform.scale = .one
targetPivotTransform.translation = value.convert(inputDevicePose.position, from: .local, to: .scene)
targetPivotTransform.rotation = value.convert(AffineTransform3D(rotation: inputDevicePose.rotation), from: .local, to: .scene).rotation
} else {
// If there is not an input device pose, use the location of the drag for positioning the pivot.
targetPivotTransform.translation = value.convert(value.location3D, from: .local, to: .scene)
}
if !state.isDragging {
// If this drag just started, create the pivot entity.
let pivotEntity = Entity()
guard let parent = entity.parent else { fatalError("Non-root entity is missing a parent.") }
// Add the pivot entity into the scene.
parent.addChild(pivotEntity)
// Move the pivot entity to the target transform.
pivotEntity.move(to: targetPivotTransform, relativeTo: nil)
// Add the targeted entity as a child of the pivot without changing the targeted entity's world transform.
pivotEntity.addChild(entity, preservingWorldTransform: true)
// Store the pivot entity.
state.pivotEntity = pivotEntity
// Indicate that a drag has started.
state.isDragging = true
} else {
// If this drag is ongoing, move the pivot entity to the target transform.
// The animation duration smooths the noise in the target transform across frames.
state.pivotEntity?.move(to: targetPivotTransform, relativeTo: nil, duration: 0.2)
}
if preserveOrientationOnPivotDrag, let initialOrientation = state.initialOrientation {
state.targetedEntity?.setOrientation(initialOrientation, relativeTo: nil)
}
}
How do you call the effect where the edges around the central image gradually become transparent? This effect is also seen when viewing immersive mode of spatial photos in Vision Pro. How can I achieve this effect using SwiftUI or ShaderGraph? I want to use this effect when displaying images in my app.
Still don't understand why no one is clarifying about this Apple Video https://developer.apple.com/videos/play/wwdc2023/10111
At the end of this video, there's an incomplete tutorial about connecting a USDZ with mesh and Skeleton structure to the hand tracking system. No example project is linked, and no one is giving the community any clarification. Please can you help us to understand how to proceed?
I am using ARKit with RealityKit to scan objects using LiDAR on iOS. I can generate an OBJ file from ARMeshAnchors, but I am missing the texture export (JPG + MTL).
What I Have So Far:
Successfully capturing mesh using ARMeshAnchor.
Converting mesh into MDLAsset and exporting .obj.
I need help generating the .jpg texture and linking it to the .mtl file.
private func exportScannedObject() {
guard
let camera = arView.session.currentFrame?.camera
else { return }
func convertToAsset(meshAnchors: [ARMeshAnchor]) -> MDLAsset? {
guard let device = MTLCreateSystemDefaultDevice() else {return nil}
let asset = MDLAsset()
for anchor in meshAnchors {
let mdlMesh = anchor.geometry.toMDLMesh(device: device, camera: camera, modelMatrix: anchor.transform)
// Apply a gray material to the mesh
let material = MDLMaterial(name: "GrayMaterial", scatteringFunction: MDLScatteringFunction())
material.setProperty(MDLMaterialProperty(name: "baseColor", semantic: .baseColor, float3: SIMD3(0.5, 0.5, 0.5))) // Gray color
if let submeshes = mdlMesh.submeshes as? [MDLSubmesh] {
for submesh in submeshes {
submesh.material = material
}
}
asset.add(mdlMesh)
}
return asset
}
func export(asset: MDLAsset) throws -> URL {
let directory = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first!
let url = directory.appendingPathComponent("scaned.obj")
if MDLAsset.canExportFileExtension("obj") {
do {
try asset.export(to: url)
return url
} catch let error {
fatalError(error.localizedDescription)
}
} else {
fatalError("Can't export USD")
}
}
if let meshAnchors = arView.session.currentFrame?.anchors.compactMap({ $0 as? ARMeshAnchor }),
let asset = convertToAsset(meshAnchors: meshAnchors) {
do {
let url = try export(asset: asset)
showScanPreview(url)
} catch {
print("export error")
}
}
}
extension ARMeshGeometry {
func vertex(at index: UInt32) -> SIMD3<Float> {
assert(vertices.format == MTLVertexFormat.float3, "Expected three floats (twelve bytes) per vertex.")
let vertexPointer = vertices.buffer.contents().advanced(by: vertices.offset + (vertices.stride * Int(index)))
let vertex = vertexPointer.assumingMemoryBound(to: SIMD3<Float>.self).pointee
return vertex
}
// helps from StackOverflow:
// https://stackoverflow.com/questions/61063571/arkit-3-5-how-to-export-obj-from-new-ipad-pro-with-lidar
func toMDLMesh(device: MTLDevice, camera: ARCamera, modelMatrix: simd_float4x4) -> MDLMesh {
func convertVertexLocalToWorld() {
let verticesPointer = vertices.buffer.contents()
for vertexIndex in 0..<vertices.count {
let vertex = self.vertex(at: UInt32(vertexIndex))
var vertexLocalTransform = matrix_identity_float4x4
vertexLocalTransform.columns.3 = SIMD4<Float>(x: vertex.x, y: vertex.y, z: vertex.z, w: 1)
let vertexWorldPosition = (modelMatrix * vertexLocalTransform).columns.3
let vertexOffset = vertices.offset + vertices.stride * vertexIndex
let componentStride = vertices.stride / 3
verticesPointer.storeBytes(of: vertexWorldPosition.x, toByteOffset: vertexOffset, as: Float.self)
verticesPointer.storeBytes(of: vertexWorldPosition.y, toByteOffset: vertexOffset + componentStride, as: Float.self)
verticesPointer.storeBytes(of: vertexWorldPosition.z, toByteOffset: vertexOffset + (2 * componentStride), as: Float.self)
}
}
convertVertexLocalToWorld()
let allocator = MTKMeshBufferAllocator(device: device);
let data = Data.init(bytes: vertices.buffer.contents(), count: vertices.stride * vertices.count);
let vertexBuffer = allocator.newBuffer(with: data, type: .vertex);
let indexData = Data.init(bytes: faces.buffer.contents(), count: faces.bytesPerIndex * faces.count * faces.indexCountPerPrimitive);
let indexBuffer = allocator.newBuffer(with: indexData, type: .index);
let submesh = MDLSubmesh(indexBuffer: indexBuffer,
indexCount: faces.count * faces.indexCountPerPrimitive,
indexType: .uInt32,
geometryType: .triangles,
material: nil);
let vertexDescriptor = MDLVertexDescriptor();
vertexDescriptor.attributes[0] = MDLVertexAttribute(name: MDLVertexAttributePosition,
format: .float3,
offset: 0,
bufferIndex: 0);
vertexDescriptor.layouts[0] = MDLVertexBufferLayout(stride: vertices.stride);
let mesh = MDLMesh(vertexBuffer: vertexBuffer,
vertexCount: vertices.count,
descriptor: vertexDescriptor,
submeshes: [submesh])
return mesh
}
}
What I Need Help With:
How do I generate the JPG texture from the AR scene?
How do I save an MTL file linking the OBJ model to the texture?
How can I correctly apply the texture when viewing the OBJ in an external 3D viewer?
I appreciate any guidance, including sample code or resources! If you have a complete working solution, I’d love to discuss further via private channels.
I have a visionOS app that I’m adding support for IOS and will like to keep using RealityView.
I know there are the following modifiers to add some navigation
.realityViewCameraControls(.orbit)
.realityViewCameraControls(.dolly)
.realityViewCameraControls(.pan)
But how can I add more than one? For example I would like to orbit with one finger, Pan with 2 fingers and dolly by pinching. Is this possible and if so can someone share some sample code on how to achieve that?
Thanks,
Guillermo
In Reality Composer, it is possible to create child components and manipulate them within the hierarchy of a ModelEntity. Is there a way to create child components in other 3D modeling programs, such as Blender?
Hello,
I am trying to use the subdivision mesh rendering option.
I can see it working in RealityComposerPro:
But not when loading asset and displaying in Simulator:
Using this code:
import SwiftUI
import RealityKit
import RealityKitContent
struct AirspaceView: View {
// MARK: - VIEW BODY
var body: some View {
RealityView { content in
if let a = try? await Entity(named: "Models/Test/Test.usdc", in: realityKitContentBundle) {
content.add(a)
}
}
}
}
Any ideas why?
Hi folks, I’m new to Vision Pro stack, still trying to learn all the nuances. Here is a problem I can’t seem to find an answer.
I placed entity A( a small .02 radius sphere) inside entity B( size:.1 box). Both entities have HoverEffectComponent, and both inputcomponent is set to .direct. Entity A is NOT a child of Entity B. When I direct touch Entity B, I noticed that Entity A’s hover effect is fired as well. This only happens if Entity A‘s position is inside Entity B. The gesture that is only targeted at Entity A doesn’t work either. I double checked Entity A collider which sits inside entity B collider, my direct touch shouldn’t have trigger its hove effect. Having one collider inside another seems to produce unpredictable behavior? Thanks in advance 🙏🙏🙏
Context: I’m trying to create an invisible bound around Entity A, so when my hand approaches the bound to grab Entity A, a nice spotlight hover effect would fire first on the bound before hand reaching entity A.
I am following this example to create a stereoscopic image: https://developer.apple.com/documentation/visionos/creating-stereoscopic-image-in-visionos
I would also like to add corner radius to the stereoscopic RealityView. With ordinary SwiftUI views, we typically just use .clipShape(RoundedRectangle(cornerRadius: 32)):
struct StereoImage: View {
var body: some View {
let spacing: CGFloat = 10.0
let padding: CGFloat = 40.0
VStack(spacing: spacing) {
Text("Stereoscopic Image Example")
.font(.largeTitle)
RealityView { content in
let creator = StereoImageCreator()
guard let entity = await creator.createImageEntity() else {
print("Failed to create the stereoscopic image entity.")
return
}
content.add(entity)
}
.frame(depth: .zero)
}
.padding(padding)
.clipShape(RoundedRectangle(cornerRadius: 32)) // <= HERE!
}
}
This doesn't seem to actually clip the RealityView shown in the sample above. I am guessing this is due to the fact that the box in the RealityView has a non-zero z scale, which means it isn't on the same "layer" as its SwiftUI containers, and thus isn't clipped by the modifiers apply to the containers.
How can I properly apply a clipshape to RealityViews like this? Thanks!
Hi,
I have used the template code for Plane Detection and placing models on them from here https://developer.apple.com/documentation/visionos/placing-content-on-detected-planes
This source code did not copy the animations in the preview model to the PlacedModel and hence I modified it to do a manual copy of animations and textures. There is a function called materialize() that does this and I was able to modify it to get it working where the placed models are now animating. The issue is when I apply gestures on them like drag or rotate. For those models that go through this logic I'm unable to add gestures even though I'm making sure that Collision and Input Target is set on the Placed Models. Has anyone been able to get this working or is it even a possibility?
My materialize function
func materialize() -> PlacedObject {
let shapes = previewEntity.components[CollisionComponent.self]!.shapes
// Clone render content first as we need its materials
let clonedRenderContent = renderContent.clone(recursive: true)
print("To be finding main model: \(descriptor.displayName)")
// Find the main model in preview hierarchy
func findMainModel(_ entity: Entity) -> Entity? {
if entity.name == descriptor.displayName.replacingOccurrences(of: " ", with: "_") {
print("Found main model: \(entity.name)")
return entity
}
for child in entity.children {
if child.name == descriptor.displayName.replacingOccurrences(of: " ", with: "_") {
print("Found main model in children: \(child.name)")
return child
}
}
return nil
}
// Clone hierarchy preserving structure, names, and materials
func cloneHierarchy(_ entity: Entity) -> Entity {
print("Cloning: \(entity.name)")
let cloned: Entity
if let model = entity as? ModelEntity {
// Clone with recursive false to handle children manually
cloned = model.clone(recursive: false)
if let clonedModel = cloned as? ModelEntity,
let originalMaterials = model.model?.materials {
// Preserve the original model's materials
clonedModel.model?.materials = originalMaterials
}
} else {
cloned = Entity()
}
// Preserve name and transform
cloned.name = entity.name
cloned.transform = entity.transform
// Clone children
for child in entity.children {
let clonedChild = cloneHierarchy(child)
cloned.addChild(clonedChild)
}
return cloned
}
print("=== Cloning Preview Structure ===")
// Clone the preview hierarchy with proper structure
let clonedStructure = cloneHierarchy(previewEntity)
// Find and use the main model
if let mainModel = findMainModel(clonedStructure) {
print("Using main model for PlacedObject")
let modelEntity: ModelEntity
if let asModel = mainModel as? ModelEntity {
print("Using asModel ")
modelEntity = asModel
} else {
modelEntity = ModelEntity()
modelEntity.name = mainModel.name
// Copy children and transforms
for child in mainModel.children {
modelEntity.addChild(child)
}
modelEntity.transform = mainModel.transform
}
// Add collision component here
let collisionComponent = CollisionComponent(shapes: shapes, isStatic: false,
filter: CollisionFilter(group: PlacedObject.collisionGroup, mask: .all))
modelEntity.components.set(collisionComponent)
// Create the placed object
let placedObject = PlacedObject(descriptor: descriptor, renderContentToClone: modelEntity, shapes: shapes)
// Set input target on the placed object itself
placedObject.components.set(InputTargetComponent(allowedInputTypes: [.direct, .indirect]))
return placedObject
} else {
print("Fallback to original render content")
let placedObject = PlacedObject(descriptor: descriptor, renderContentToClone: clonedRenderContent, shapes: shapes)
placedObject.components.set(InputTargetComponent(allowedInputTypes: [.direct, .indirect]))
return placedObject
}
}
My PlacedObject class where the init has the recursive cloning removed because it is handled in materialize
class PlacedObject: Entity {
let fileName: String
// The 3D model displayed for this object.
private let renderContent: ModelEntity
static let collisionGroup = CollisionGroup(rawValue: 1 << 29)
// The origin of the UI attached to this object.
// The UI is gravity aligned and oriented towards the user.
let uiOrigin = Entity()
var affectedByPhysics = false {
didSet {
guard affectedByPhysics != oldValue else { return }
if affectedByPhysics {
components[PhysicsBodyComponent.self]!.mode = .static
} else {
components[PhysicsBodyComponent.self]!.mode = .static
}
}
}
var isBeingDragged = false {
didSet {
affectedByPhysics = !isBeingDragged
}
}
var positionAtLastReanchoringCheck: SIMD3<Float>?
var atRest = false
init(descriptor: ModelDescriptor, renderContentToClone: ModelEntity, shapes: [ShapeResource]) {
fileName = descriptor.fileName
// renderContent = renderContentToClone.clone(recursive: true)
renderContent = renderContentToClone
super.init()
name = renderContent.name
// Apply the rendered content’s scale to this parent entity to ensure
// that the scale of the collision shape and physics body are correct.
scale = renderContent.scale
renderContent.scale = .one
// Make the object respond to gravity.
let physicsMaterial = PhysicsMaterialResource.generate(restitution: 0.0)
let physicsBodyComponent = PhysicsBodyComponent(shapes: shapes, mass: 1.0, material: physicsMaterial, mode: .static)
components.set(physicsBodyComponent)
components.set(CollisionComponent(shapes: shapes, isStatic: false,
filter: CollisionFilter(group: PlacedObject.collisionGroup, mask: .all)))
addChild(renderContent)
addChild(uiOrigin)
uiOrigin.position.y = extents.y / 2 // Position the UI origin in the object’s center.
// Allow direct and indirect manipulation of placed objects.
components.set(InputTargetComponent(allowedInputTypes: [.direct, .indirect]))
// Add a grounding shadow to placed objects.
renderContent.components.set(GroundingShadowComponent(castsShadow: true))
}
required init() {
fatalError("`init` is unimplemented.")
}
}
Thanks
Hello,
we have a RealityKit app that also runs on macOS via Catalyst.
For specific USD assets containing particle systems we have observed a reproducible crash.
Steps to reproduce:
Open Reality Composer Pro
Create new file
Create simple particle system (default one is fine)
export as USDZ
Create project in Xcode
Call Entity.load(… and pass in your USD
Running this on an Intel iMac with macOS Sequoia 15.3 will lead to a crash with the following console log:
validateWithDevice:4704: failed assertion `Render Pipeline DescrvalidateWithDevice:4704: failed assertion `Render Pipeline Descriptor Validation
depthAttachmentPixelFormat (MTLPixelFormatDepthvalidateWithDevice:4704: failed assertion `Render Pipeline Descriptor Validation
depthAttachmentPixelFormat (MTLPixelFormatDepthvalidateWithDevice:4704: failed assertion `Render Pipeline Descriptor Validation
depthAttachmentPixelFormat (MTLPixelFormatDepth32Float) and stencilAttachmentPixelFormat (MTLPixelFormatStencil8) must match.
'
iptor Validation
depthAttachmentPixelFormat (MTLPixelFormatDepth32Float) and stencilAttachmentPixelFormat (MTLPixelFormatStencil8) must match.
'
32Float) and stencilAttachmentPixelFormat (MTLPixelFormatStencil32Float) and stencilAttachmentPixelFormat (MTLPixelFormatStencil8) must match.
'
8) must match.
'
Xcode version: 16.2.0
iMac 2020 3,8GHz Intel Core i7
macOS Sequoia 15.3
FB16477373
It would be great if this could be fixed quickly or a workaround provided since it affects or production app. Thank you!
Hi
Looking at the documentation for screenSpaceAmbientOcclusionIntensity, I noticed that it says this is supported on visionOS 1.0+: https://developer.apple.com/documentation/scenekit/scncamera/screenspaceambientocclusionintensity
Could someone enlighten me as to how that would work? As far as I know, we don't use an SCNCamera on visionOS. So, what's the idea here? Can we activate SSAO on visionOS?
I am at a loss. I have looked at examples, and I have used chat/cursor. I cannot figure out how to target the transform/position of a Reality Composer Pro project when adding it to an ARView in iOS.
I have a test red sphere working perfectly for raycast positioning. When I pass the same variables (tested with print out) to the Entity or Anchor position/transform nothing changes.
It seems that, no matter what, the content of the Reality Composer Pro project is placed where the camera view initialized.
How do I actually interact with its position? I just want to be able to tap the screen and place the RCP wherever I want.
I submitted feedback as FB16463501 -- posting here for others to see, or maybe for Apple to share any help if there are workarounds, etc.:
Targets below iOS 18.x fail to launch app due to dyld[xxxxx]: Symbol not found: errors when referencing:
MeshResource.init(from:) async - https://developer.apple.com/documentation/realitykit/meshresource/init(from:)-b7hb
i.e. dyld[61511]: Symbol not found: _$s10RealityKit12MeshResourceC0A10FoundationE4fromACSayAD0C10DescriptorVG_tYaKcfC
MeshResource.replace(with:) async - https://developer.apple.com/documentation/realitykit/meshresource/replace(with:)-8uvri
i.e. dyld[78830]: Symbol not found: _$s10RealityKit12MeshResourceC0A10FoundationE7replace4withyAcDE8ContentsV_tYaKF
Targets tested that exhibit issue: (DYLD errors)
Device: iOS 17.7.2, iPhone 14 Pro Max
Simulator: iOS 17.5 (21F79), iPhone 15
System Information:
macOS Version 15.3 (Build 24D60)
Xcode 16.2 (23507) (Build 16C5032a)
MRE -- include this code in your app: (no need to invoke, just reference)
static func addOrUpdateEntityModel_MRE(_ entity: ModelEntity) async {
let descriptor = MeshDescriptor(name: "MyDescriptor")
do {
if let modelComponent = entity.model { // update existing ModelComponent
if let model = try? MeshResource.Model(id: "MyModelId", descriptors: [descriptor]) {
var contents = MeshResource.Contents()
contents.models = .init([model])
try await modelComponent.mesh.replace(with: contents) /// `dyld[78830]: Symbol not found: _$s10RealityKit12MeshResourceC0A10FoundationE7replace4withyAcDE8ContentsV_tYaKF`
}
} else { //create new ModelComponent
/// Comment-out the 2 lines below == dyld error for above `MeshResource.replace(with:)`
let meshRes = try await MeshResource(from: [descriptor]) /// `dyld[61511]: Symbol not found: _$s10RealityKit12MeshResourceC0A10FoundationE4fromACSayAD0C10DescriptorVG_tYaKcfC`
entity.model = .init(mesh: meshRes, materials: [SimpleMaterial()])
}
} catch {
fatalError()
}
}
my coworkers and i are guessing at what data defines an anchor. i tried searching but struggled to find anything helpful.
our best guess was a combination Triangular Irregular Networks (TIN), gps, magnetic compass direction and maybe elevation sensors.
is this documented anywhere? if not, can a definition or description be provided?
I'm trying to develop an immersive visionOS app, which you can move an Entity having a PerspectiveCamera as its child in immersive space, and render the camera view on 2D window.
According to this thread, this seems to can be achieved using RealityRenderer. But when I added the scene entity loaded from realityKitContentBundle to realityRenderer.entities, I needed to clone all entities of the scene, otherwise all entities in the immersive space will disappear.
@Observable
@MainActor
final class OffscreenRenderModel {
private let renderer: RealityRenderer
private let colorTexture: MTLTexture
init(scene: Entity) throws {
renderer = try RealityRenderer()
// If not clone entities in the scene, all entities in the immersive space will disappear
renderer.entities.append(scene.clone(recursive: true))
let camera = PerspectiveCamera()
renderer.activeCamera = camera
renderer.entities.append(camera)
...
}
}
Is this the expected behavior? Or is there any other way to do this (move camera in immersive space and render its output on 2D window)?
Here is my sample code:
https://github.com/TAATHub/RealityKitPerspectiveCamera
I'm preparing my submission for the Swift Student Challenge, and I have a couple of questions regarding the development environment.
Is it allowed to use Xcode to program my scene, or do I have to use Swift Playgrounds?
Can I use iPadOS 18 for development? I noticed that Swift Playgrounds currently only supports up to iPadOS 17.5, but I would like to use RealityView, which is only available starting from iPadOS 18.
I appreciate any clarification on this. Thanks in advance!
I am submitting a challenge to the Swift Student Challenge. I have created a RealityContent folder using Reality Composer Pro. How can I import this folder into the Swift Package Manager (.swiftpm) project hosted on Playground to ensure that it becomes a usable package?
We use SceneReconstructionProvider to detect meshes in the surrounding environment and apply an OcclusionMaterial to them.
// Assuming `entity` represents one of the detected mesh in the environment
entity.components.set(ModelComponent(
mesh: mesh,
materials: [OcclusionMaterial()]
))
While this correctly occludes entities placed in the immersive space, it also occludes system windows. This becomes problematic when a window is dragged into an occluded area (before or after entering the immersive space), preventing interaction with its elements. In some cases, it also makes it impossible to focus on the window’s drag handle, since this might become occluded as well after moving the window nearby. More generally, system windows can be occluded when they come into proximity with a model that has OcclusionMaterial applied.
I'm aware of a change introduced in visionOS 2 regarding how occlusions interact with UI elements (as noted in the release notes). I believe this change was intended to ensure windows do not remain visible when opened in another room. However, this also introduces some challenges, as described in the scenario above.
Is there a way to prevent system window occlusion while still allowing entities to be occluded by environmental features? Perhaps not using OcclusionMaterial at all?
Development environment: Xcode 16.2, macOS 15.2
Run-time configuration: visionOS 2.2 and 2.3