My experience has been that ModelEntity(named:in:) can be used to load a USD file with a simple structure consisting of entities and model entities, and, critically, it will flatten the entity hierarchy down to a single ModelEntity, presumably reducing the number of draw calls.
However, can anyone verify that the following is true?
If ModelEntity(named:in:) is used to load a USD file from a RealityKit content bundle, it may fail when the USD file contains more complex data, such as shader graph material definitions, or perhaps for some other reason. I am not sure.
AND the error that ModelEntity(named:in:) throws in this case is
Cannot load RealityKitContent entity: Failed to find resource with name "<name>" in bundle
which would literally suggest that the file does not exist, instead of what I assume the error actually is, which is "the file exists but its entity hierarchy could not be flattened to a single ModelEntity" ?
Is that an accurate description of the known behavior of ModelEntity:named:in:)?
I understand that I could use Entity(named:in:) instead, without the flattening feature. My question is really more about the seemingly misleading error message.
Thank you for any clarification you can provide.
Delve into the world of graphics and game development. Discuss creating stunning visuals, optimizing game mechanics, and share resources for game developers.
Selecting any option will automatically load the page
Post
Replies
Boosts
Views
Activity
I am trying to implement a ChacterControllerComponent using the following URL.
https://developer.apple.com/documentation/realitykit/charactercontrollercomponent
I have written sample code, but PhysicsSimulationEvents.WillSimulate is not executed and nothing happens.
import SwiftUI
import RealityKit
import RealityKitContent
struct ImmersiveView: View {
let gravity: SIMD3<Float> = [0, -50, 0]
let jumpSpeed: Float = 10
enum PlayerInput {
case none, jump
}
@State private var testCharacter: Entity = Entity()
@State private var myPlayerInput = PlayerInput.none
var body: some View {
RealityView { content in
// Add the initial RealityKit content
if let immersiveContentEntity = try? await Entity(named: "Immersive", in: realityKitContentBundle) {
content.add(immersiveContentEntity)
testCharacter = immersiveContentEntity.findEntity(named: "Capsule")!
testCharacter.components.set(CharacterControllerComponent())
let _ = content.subscribe(to: PhysicsSimulationEvents.WillSimulate.self, on: testCharacter) {
event in
print("subscribe run")
let deltaTime: Float = Float(event.deltaTime)
var velocity: SIMD3<Float> = .zero
var isOnGround: Bool = false
// RealityKit automatically adds `CharacterControllerStateComponent` after moving the character for the first time.
if let ccState = testCharacter.components[CharacterControllerStateComponent.self] {
velocity = ccState.velocity
isOnGround = ccState.isOnGround
}
if !isOnGround {
// Gravity is a force, so you need to accumulate it for each frame.
velocity += gravity * deltaTime
} else if myPlayerInput == .jump {
// Set the character's velocity directly to launch it in the air when the player jumps.
velocity.y = jumpSpeed
}
testCharacter.moveCharacter(by: velocity * deltaTime, deltaTime: deltaTime, relativeTo: nil) {
event in
print("playerEntity collided with \(event.hitEntity.name)")
}
}
}
}
}
}
The scene is loaded from RCP. It is simple, just a capsule on a pedestal.
Do I need a separate code to run testCharacter from this state?
I’m building an app that uses RealityKit and specifically ARConfiguration.FrameSemantics.personSegmentationWithDepth.
The goal is to insert an AR object into the scene behind a person, and an additional AR object in front of the person, while being as photo realistic as possible.
Through testing, I’ve noticed that many times, the edges of the person segmentation mask are not well matched to the actual person, and parts of the person are transparent, with the AR object bleeding through. It’s sort of like a “bad green screen” effect, which I’d expect to see a little bit, but not to this extent. I’ve been testing on iPhone 16, iPhone 14 Pro, iPad Pro 12.9 inch 6th Generation, and iPhone 12 Pro, with similar results across all devices.
I’m wondering what else I can do to improve this… either code changes, platform (like different iPhone models), or environment (like lighting, distance, etc).
Attaching some example screen grabs and a minimum reproducible code sample. Appreciate any insights!
import ARKit
import SwiftUI
import RealityKit
struct RealityViewContainer: UIViewRepresentable {
func makeUIView(context: Context) -> ARView {
let arView = ARView(frame: .zero)
arView.environment.sceneUnderstanding.options.insert(.occlusion)
arView.renderOptions.insert(.disableMotionBlur)
arView.renderOptions.insert(.disableDepthOfField)
let configuration = ARWorldTrackingConfiguration()
configuration.planeDetection = [.horizontal]
if ARWorldTrackingConfiguration.supportsFrameSemantics(.personSegmentationWithDepth) {
configuration.frameSemantics.insert(.personSegmentationWithDepth)
}
arView.session.run(configuration)
arView.session.delegate = context.coordinator
context.coordinator.arView = arView
}
func makeCoordinator() -> Coordinator {
Coordinator(self)
}
class Coordinator: NSObject, ARSessionDelegate {
var parent: RealityViewContainer
var floorAnchor: ARPlaneAnchor?
init(_ parent: RealityViewContainer) {
self.parent = parent
}
func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
if let arView,floorAnchor == nil {
for anchor in anchors {
if let horizontalPlaneAnchor = anchor as? ARPlaneAnchor,
horizontalPlaneAnchor.alignment == .horizontal,
horizontalPlaneAnchor.transform.columns.3.y < arView.cameraTransform.translation.y { // filter out ceiling
floorAnchor = horizontalPlaneAnchor
let backgroundEntity = BackgroundEntity()
let anchorEntity = AnchorEntity(anchor: horizontalPlaneAnchor)
anchorEntity.addChild(background)
let foregroundEntity = ForegroundEntity()
backgroundEntity.addChild(foregroundEntity)
arView.scene.addAnchor(anchorEntity)
arView.installGestures([.rotation, .translation], for: backgroundEntity)
break // Stop after adding the first horizontal plane (floor)
}
}
}
}
}
}
Topic:
Graphics & Games
SubTopic:
RealityKit
I recently needed to develop an application to obtain the window list, which requires Screen Recording permissions. Apple's official documentation mentions using the two functions CGPreflightScreenCaptureAccess and CGRequestScreenCaptureAccess to request permissions. These functions are stated to be available since version 10.15. However, when I used these two functions on a device running macOS 10.15.7, I encountered the errors shown in the attached screenshot. I used the nm tool to inspect the symbols in the CoreGraphics.framework and found that these two functions were not present. Could you help me understand why this is happening?
Topic:
Graphics & Games
SubTopic:
General
Hello,
Im trying to install it following these steps https://www.applegamingwiki.com/wiki/Game_Porting_Toolkit but i get an error with 'brew install apple/apple/game-porting-toolkit'
==> tar -xf crossover-sources-22.1.1.tar.gz --include=sources/clang/* --strip-components=2
==> cmake -G Ninja -DCMAKE_VERBOSE_MAKEFILE=Off -DCMAKE_MAKE_PROGRAM=ninja -DCMAKE_VERBOSE_MAKEFILE=On -DCMAKE_OSX_ARCHITECTUR
Last 15 lines from /Users/rafael/Library/Logs/Homebrew/game-porting-toolkit-compiler/02.cmake:
-DLLVM_INSTALL_TOOLCHAIN_ONLY=On
-DLLVM_ENABLE_PROJECTS=clang
/private/tmp/game-porting-toolkit-compiler-20250519-44600-qwrjgl/llvm
CMake Error at CMakeLists.txt:3 (cmake_minimum_required):
Compatibility with CMake < 3.5 has been removed from CMake.
Update the VERSION argument <min> value. Or, use the <min>...<max> syntax
to tell CMake that the project requires at least <min> but has been updated
to work with policies introduced by <max> or earlier.
Or, add -DCMAKE_POLICY_VERSION_MINIMUM=3.5 to try configuring anyway.
-- Configuring incomplete, errors occurred!
If reporting this issue please do so to (not Homebrew/* repositories):
apple/apple
MacOS 15.3.1
Thank you in advanced
Regards
I am currently using RealityKit (perspective camera) to render a character in my swiftUI app.
The character has customization such as clothing items and hair and all objects are properly weighted to the rig.
The way the model is setup in Blender is like so: Groups of objects that will be swapped (ex: Shoes -> Shoes objects) and an armature. I then export it to usdc with all objects active. This is the resulting hierarchy:
Before exporting for the animation (armature modifier applied), I simply had to store the Model entities and swap them in but now when I export with the Armature Modifier applied, so that animations get exported, the ModelComponent gets flattened to the armature and swapping entities and applying new materials to them is no longer as simple.
Here's a demo blend file and usdc export with a setup like mine, having an animated bone to swing a cube and sphere, to be swapped so that only one is visible https://www.dropbox.com/scl/fo/be2q6qcztc83z7c4gj1w0/AMapxWc_ip2KZ8oTOYDUMv8?rlkey=rcdaggcxq06dyen09mw5mqmem&st=bnc0d7j0&dl=0
This is how I'm loading the entity and removing a part, with the demo files
import SwiftUI
import RealityKit
struct SwapDemoView: View {
var body: some View {
RealityView { content in
let camera = PerspectiveCamera()
camera.transform.translation = SIMD3(x: 0, y: 0.1, z: 3)
guard let root = try? await Entity(named: "simpleSwapDemo") else { fatalError("simpleSwapDemo.usdc is not present") }
print(root) // Get initial hierarchy
guard let cube = root.findEntity(named: "Cube") else { fatalError("Entity cube doesn't exist") }
cube.removeFromParent() // <-- Cube is still visible after removal
print(root) // Get hierarchy to confirm removal of cube
let resource = root.availableAnimations[0]
root.playAnimation(resource.repeat())
content.add(root)
content.add(camera)
}
.background(.white)
}
}
And this is what the entity hierarchy looks like in RealityKit before cube removal
▿ 'root' : Entity, children: 1
⟐ SynchronizationComponent
⟐ AnimationLibraryComponent
⟐ Transform
▿ 'Armature' : ModelEntity, children: 2
⟐ SynchronizationComponent
⟐ ModelComponent
⟐ SkeletalPosesComponent
⟐ AnimationLibraryComponent
⟐ Transform
▿ 'Armature' : Entity
⟐ SynchronizationComponent
⟐ Transform
▿ 'Primitives' : Entity, children: 2
⟐ SynchronizationComponent
⟐ Transform
▿ 'Sphere' : Entity, children: 1
⟐ SynchronizationComponent
⟐ Transform
▿ 'Sphere' : Entity
⟐ SynchronizationComponent
⟐ Transform
▿ 'Cube' : Entity, children: 1
⟐ SynchronizationComponent
⟐ Transform
▿ 'Cube' : Entity
⟐ SynchronizationComponent
⟐ Transform
And here's the hierarchy after removal
▿ 'root' : Entity, children: 1
⟐ SynchronizationComponent
⟐ AnimationLibraryComponent
⟐ Transform
▿ 'Armature' : ModelEntity, children: 2
⟐ SynchronizationComponent
⟐ ModelComponent
⟐ SkeletalPosesComponent
⟐ AnimationLibraryComponent
⟐ Transform
▿ 'Armature' : Entity
⟐ SynchronizationComponent
⟐ Transform
▿ 'Primitives' : Entity, children: 1
⟐ SynchronizationComponent
⟐ Transform
▿ 'Sphere' : Entity, children: 1
⟐ SynchronizationComponent
⟐ Transform
▿ 'Sphere' : Entity
⟐ SynchronizationComponent
⟐ Transform
And this is the result:
What's the best practice here? Should animation be exported separately and then applied to the skeleton? If so, how is that achieved? I'm not really sure how to proceed here.
I am currently using RealityKit (perspective camera) to render a character in my swiftUI app.
The character has customization such as clothing items and hair and all objects are properly weighted to the rig.
The way the model is setup in Blender is like so: Groups of objects that will be swapped (ex: Shoes -> Shoes objects) and an armature. I then export it to usdc with all objects active. This is the resulting entity hierarchy, viewed in Reality Composer Pro:
My problem is that when I export with the Armature Modifier applied to the objects, so that animations get exported, the ModelComponent gets flattened to the armature and swapping entities is no longer as simple as removing the entity with the corresponding name.
What's the best practice here? Should animation be exported separately and then applied to the skeleton? If so, how is that achieved? I'm not really sure how to proceed here.
I'm trying to build an MDLMesh then add normals
let mdlMesh = MDLMesh.newBox(withDimensions: SIMD3<Float>(1, 1, 1),
segments: SIMD3<UInt32>(2, 2, 2),
geometryType: MDLGeometryType.triangles,
inwardNormals:false,
allocator: allocator)
mdlMesh.addNormals(withAttributeNamed: MDLVertexAttributeNormal, creaseThreshold: 0)
When I render the mesh, some normals are (0,0,0). I don't know if the problem is in the mesh, or in the conversion to MTKMesh. Is there a way to examine an MDLMesh with the geometry viewer?
When I look at the variable values for my mdlMesh I get this:
Not too useful. I don't know how to track down the normals.
What's the best way to find out where the normals getting broken?
I work on a team that provides an SDK for another game to handle various tasks like authentication. They are experiencing a case where devices using iOS 17 are failing to authenticate with GameCenter, receiving the message "The requested operation could not be completed because local player has not been authenticated." We imagine this is because they still have some setup to finish regarding GameCenter itself, and we're working with them to take care of that.
However, on iOS 18, their app ends up waiting indefinitely for GameCenter authentication messages that it never receives. That's where we're puzzled. We expect them to have the same outcome regardless of OS version.
We initiate GameCenter authentication by setting an authenticateHandler after some initial application setup. The handler has code to account for UI, errors, and successful authentication.
On iOS 17, it's clear that it's getting called as expected because they receive an indication that the player isn't authenticated. But on iOS 18, it looks like the same handler code on iOS 18 isn't being called at all. Are there differences in how iOS 18 interacts with the authenticationHandler that we somehow aren't accounting for? Or is there potentially something else that we're doing incorrectly that is manifesting only on iOS 18?
Here's a simplified version of our login function code (in Obj-C++). There is no OS-specific code, and the job that owns this function does stay in scope until after authentication is complete.
void beginLogin()
{
// Snip: Check if the user is already logged in.
// Snip: Prevent multiple concurrent calls to this function.
auto authenticateHandler = ^(UIViewController* gcViewController, NSError* error)
{
if (gcViewController != nil)
{
// Snip: Display the UI
}
else if (error != nil)
{
// Snip: Handle the error.
}
else
{
if ([[GKLocalPlayer localPlayer] isAuthenticated])
{
// Snip: Handle successful authentication.
}
else
{
// Snip: Handle other case.
}
}
};
[[GKLocalPlayer localPlayer] setAuthenticateHandler: authenticateHandler];
}
I have a SPFx React application where I am printing the HTML page content using the javascript default window.print() functionality. Once I save the page as pdf from the print preview window and open it using Adobe Acrobat, the links(for eg -> Google) within the content are not clickable and appearing as plain text.
I have tried to print random pages post searching with any keywords in Google and saved the files as pdfs, but, unfortunately, the links are still not clickable there as well.
To check whether it is an Adobe Acrobat issue, I have performed the same print functionality from Android devices and shared the pdf file across the iOS devices and in that case, when opened using Adobe Acrobat, the links are appearing to be clickable.
I am wondering whether it is something related to how the default print functionality works for iPadOS and iOS devices. Any insights on this would be really helpful. Thanks!!!
Note: The links are clickable for MacOS as well as for Windows.
#ios #ipados #javascript #spfx #react
hi
When analyzing our game using Instruments, I've always been confused about the two items "Drawable Present" and "Drawable Presented" in the GPU column. The timing of Drawable Present seems to be when the CPU layer calls commandbuffer:present, rather than when the actual encoding is completed on the GPU. Also, what does drawable presented specifically mean? In our case, when a CPU stall occurs, it appears that the vsync interval changes in the next frame, and a surface that has already been calculated is not displayed. Why is this happening?
Hello Everyone I am new here,
I am testing game center integration and using a development build of my IOS game. I have set up a couple of achievements in app store connect, but when I trigger them in the game then they do not unlock or show up.
Okay so i am signed into the game center with a sandbox account on a test advice. Is there anything else I need to configure, or do achievements usually only work after the game is released?.
I will appreciate any guidance…
Thanks in Advance!!!
So, I'm done with GPTK and decided to delete it. The only thing I installed was brew -v install apple/apple/game-porting-toolkit and the external libraries from the ditto command. Now, I tried to remove it, but even after
brew remove game-porting-toolkit
brew autoremove
all of the dependencies installed with brew are still there. The most obvious was game-porting-toolkit-compiler, but even after removing this there are so many libraries that are now orphaned and it's just impossible to manually identify those. Is there a way or is the easiest way to simply uninstall Homebrew completely and reinstall it again?
I use unity 2020.3.48f1 to develop a game; trying to implement Apple Services integration I use Apple unity plugins(https://github.com/apple/unityplugins) Using latest version of unity plugins I getting error in Unity project after plugin import It say "Not allowed platform VisionOS" When I tryed to use older version of the plugins I getting error on runtime when calling "var fetchItemsResponse = await GKLocalPlayer.Local.FetchItems();" in line 42 it drop EXC_BAD_ACCESS(code=257, address=0x0000...) error I tryed to use different commits from official repositorys and even custom branches of apple unity plugins like (https://github.com/muZZkat/unityplugins/tree/muzzkat/fix-fetch-items) but it did not help
There is whole my script which trying to use apple unuity plugins
using System.Threading.Tasks;
using UnityEngine;
using System.Collections;
using System;
using Apple.GameKit;
using UnityEngine.UI;
public class TheScript : MonoBehaviour
{
[SerializeField]
InputField otp;
string Signature;
string TeamPlayerID;
string Salt;
string PublicKeyUrl;
string Timestamp;
void Start()
{
StartCoroutine(Call());
}
private IEnumerator Call()
{
yield return new WaitForSeconds(5);
Login();
}
public async Task Login()
{
otp.text += $"Loginig... ";
if (!Apple.GameKit.GKLocalPlayer.Local.IsAuthenticated)
{
try
{
var player = await GKLocalPlayer.Authenticate();
var localPlayer = GKLocalPlayer.Local;
TeamPlayerID = localPlayer.TeamPlayerId;
var fetchItemsResponse = await GKLocalPlayer.Local.FetchItems();
Signature = Convert.ToBase64String(fetchItemsResponse.GetSignature());
PublicKeyUrl = fetchItemsResponse.PublicKeyUrl;
otp.text += $"Team Player ID: {TeamPlayerID} ";
otp.text += $"PublicKeyUrl: {PublicKeyUrl} ";
}
catch(Exception e)
{
otp.text += $"Error: " + e.Message;
}
}
else
{
Debug.Log("AppleGameCenter player already logged in.");
}
}
async Task SignInWithAppleGameCenterAsync(string signature, string teamPlayerId, string publicKeyURL, string salt, ulong timestamp)
{
}
}
Due to the release of ProMotion devices, the system may switch frame rates in certain scenarios, resulting in the loss of reference value for data collected through CADisplayLink callbacks at a fixed 60Hz frame rate. We cannot distinguish whether the slow callback of CADisplayLink is due to a stutter or a system switch in frame rate.
I know Hitch Time Ratio, but I can't use this scheme for some reasons.
How can I distinguish between stuck and frame rate gear shift in CADisplaylink callback?
In iOS 15, CADisplayLink.preferredFrameRateRange.preferred always returns 0, while minimum and maximum do change. Can I use these minimum and maximum range values as criteria to distinguish between frame rate switching and stuttering?
When running on my iPhone SE3 under IOS 18.4.1, achievement banners show as expected. The same code running on my iPad Air2 under IOS 15.8.4, achievement banners do not show, but they are accepted (as shown in the GameCenterViewController). The banners also don't show when running the simulator under iPhone 16 Pro Max under IOS 18.2 or simulator under iPhone SE3 under IOS 18.3. I haven't tried others. [Note that I clear the achievements each run during test so that I can duplicate this]
Topic:
Graphics & Games
SubTopic:
GameKit
Added achievements to my approved app. Added them for the next release version, which I am running in simulator. When I look at the Achievements page, I can see that there are 17 Achievements available (correct), but they all show as hidden, despite checking the "No" box in App Store Connect.
Topic:
Graphics & Games
SubTopic:
GameKit
New to achievements. I added them in App Store Connect and want to test them.
Apple Documentation says this:
"Before you begin testing your GameKit configuration, you need to enable Debug Mode. In Xcode, choose Product > Scheme > Edit Scheme. From the Run configuration, select Options and toggle Enable Debug Mode.
To begin testing your achievement configuration, open the game Progress Manager. In Xcode, choose Debug > GameKit > Manage Game Progress."
Sounds easy enough, but there is no "GameKit" under Debug in Xcode 16.2.
Topic:
Graphics & Games
SubTopic:
GameKit
This is the first time I have tried adding achievements to my games. The online documentation says:
"Before you can access achievements in your code, you can configure them in Xcode and sync the configuration updates you make with App Store Connect. Begin configuring achievements by creating a GameKit configuration file. In Xcode, choose File > New > File from Template. Select GameKit File, and click Next. In the sheet that appears, enter a name for the configuration and click Create."
Sounds easy - except Xcode 16.2 does not have a GameKit File as one of the Templates. Please advise on how to proceed and it would be nice if the documentation were updated.
Topic:
Graphics & Games
SubTopic:
GameKit
Hello!
I'm a developer working on a plugin for the Elgato Stream Deck, called GPU Metrics. The plugin currently only works on Windows but I'd like to bring it to macOS. However, based on forum posts I've read (and StackOverflow) there isn't a very clear path to query GPU metrics like usage, temperature, used GPU memory, and power consumption. There are some tools out there that do similar things, but I wanted to see what would be the recommendation from Apple's engineering team to get this data via a public API.
Requirements:
Access GPU utilization, temperature, memory usage, power usage
C/C++ based API for querying the metrics so I can expose the data to JavaScript via Node Addon
No need to compatibile with Intel-based Macs, as Apple silicon will be fine for now
Plugin GitHub
Thank you!
Noah