I want to know why there is no video frame data when RePlaykit enters the background and then enters the foreground?

I would like to receive some guidance and discussion on the ideas implemented with RealityKit.

__builtin_ia32_cvtb2mask512() is the GNU C builtin for vpmovb2m k, zmm. The Intel intrinsic for it is _mm512_movepi8_mask. It extracts the most-significant bit from each byte, producing an integer mask. The SSE2 and AVX2 instructions pmovmskb and vpmovmskb do the same thing for 16 or 32-byte vectors, producing the mask in a GPR instead of an AVX-512 mask register. (_mm_movemask_epi8 and _mm256_movemask_epi8). I would like an implementation for ARM that is faster than below I would like an implementation for ARM NEON I would like an implementation for ARM SVE I have attached a basic scalar implementation in C. For those trying to implement this in ARM, we care about the high bit, but each byte's high bit (in a 128bit vector), can be easily shifted to the low bit using the ARM NEON intrinsic: vshrq_n_u8(). Note that I would prefer not to store the bitmap to memory, it should just be the return value of the function similar to the following function. #define _(n) __attribute((vector_size(1<<n),aligned(1))) typedef char V _(6); // 64 bytes, 512 bits typedef unsigned long U; #undef _ U generic_cvtb2mask512(V v) { U mask=0;int i=0; while(i<64){ // shift mask by 1 and OR with MSB of v[i] byte mask=(mask<<1)|((v[i]&0x80)>>7); i++;} return mask; } This is also a dup of : https://stackoverflow.com/questions/79225312

I would like to implement zoom functionality in my SceneKit game: when the user performs the pinch gesture on a point on the screen, the scene zooms in to make that point larger. Until now I simply changed SCNCamera.focalLength, but this simply zooms in to the center of what is currently visible on screen. Is it somehow possible to implement the zoom functionality described above by perhaps interactively rotating the camera at the same time towards the pinched point? Is there a formula for this? I would like to avoid suddenly rotating the camera to face the pinched point when the pinch gesture begins and then zoom in while the pinch is in progress.

I've been working on Swift game which is not yet launched or available for preview. The game works in such a way that it has idle CPU while the user is thinking and sustained max CPU and GPU on as many cores as possible when he makes a move. Rarely, due to OS activity or something else outside of my control (for example when dropping the OS curtain even if for just a bit then remove it), the game or some of its threads are moved to efficiency cores which results in major stuttering which persists precisely until the game is idle again at which point the game is moved back on performance cores - but if the player keeps making moves the stuttering simply won't go away and so I guess compuptation is locked onto efficiency cores. The issue does not reproduce on MacCatalyst on Intel. How do I tell Swift to avoid efficiency cores? BTW Swift and SceneKIT have AMAZING performance especially when compared to others.

So recently I have been trying to install game-porting-toolkit, and when I try to run brew -v install apple/apple/game-porting-toolkit Im getting an error: Error: apple/apple/game-porting-toolkit 1.1 did not build Logs: /Users/mateusz/Library/Logs/Homebrew/game-porting-toolkit/00.options.out /Users/mateusz/Library/Logs/Homebrew/game-porting-toolkit/01.configure /Users/mateusz/Library/Logs/Homebrew/game-porting-toolkit/01.configure.cc /Users/mateusz/Library/Logs/Homebrew/game-porting-toolkit/wine64-build If reporting this issue please do so to (not Homebrew/brew or Homebrew/homebrew-core): apple/apple I searched on this forum some threads but I couldn't take any advice from them. I would really appreciate any help.

I've noticed a major third-party app has the following flag set to 1/true in its Info.plist: CAMetalLayerWantsCompositingDependencies Does anyone know if it’s recognized by Core Animation / Metal, and what it’s supposed to do? It might obviously have zero relationship to the OS, defined by that app and for that app... but since it looks very much like an unofficial/undocumented environment setting, it might be great to know what problem it solves. I happen to have issues related to compositing other CALayers over a CAMetalLayer in my app... so this definitely stood out as interesting. Thank you!

Hi Apple, In VisionOS, for real-time streaming of large 3D scenes, I plan to create Metal buffers and textures in multiple threads and then use a compute shader on the main thread to copy the Metal resources into RealityKit, minimizing main thread usage. Given that most of RealityKit's default APIs require execution on the main actor (main thread), it is not ideal for streaming data. Is this approach the best way to handle streaming data and real-time rendering? Thank you very much.

Hey everyone, I am currently developing an app in visionOS and using RealityComposerPro create scenes in put in my app. I have a humanoid model with hair strands, and each strand of hair has an opacity map. However, some reflections are still visible even though the opacity is zero. There are also some weird culling among hair strands (in the left circle) and weird reflections in hair cards (in the right circle). Here's my settings for the materials. Since all the hair strands are interconnected with each other, it is hard to decide the drawing order in Xcode, so I am wondering if there's an easier way to handle transparency objects. Please let me know if you know anything helpful, much appreciated!

Hello! I need to "draw" a set of particles into the texture. It would be trivial in render encoder of course. However, I would like to implement the task in compute kernel. Every particle draw operation is expected to set 5 texels - "center" one and left/right/upper/lower. Particles can and will overlap, so concurrent draws are to be expected. I tried using texture atomics - atomic_store() to be more precise. This worked, albeit pretty slowly - too slow for my purpose. Just to test what would happen, I tried using normal texture write(). I was expecting to see some kind of visual artefacts, but to my surprise, it worked very well (and much faster). My question: is it safe? I understand that calling write() doesn't guarantee any ordering of the operations, so if multiple threads write to the same texel, the final value may come from any of those threads. But suppose all the threads were to write the very same color? Can I assume that the texel in question will have said color after the compute kernel finishes? I am using M2 Pro MacBook, but ideally I would love to get the answer for the all Apple Silicon devices. My texture format is R32Int (so as to be able to use atomics), but I could do with any single-channel format, the purpose of the texture is to be binary mask of sorts. Thanks!

I am trying to use the SVGF denoiser to denoise my ray traced shadows (and also other textures later). I do get a smoothed image, but with wonky denoising. I need the depth-normal textures and motion textures for the SVGF and assume that these are badly filled in my case. However, neither in the above linked documentation nor in the WWDC19 video I find how they should be defined. I am looking to answers to: Is depth in red or alpha channel for the depth-normal texture? Are the normals in screen space? Is depth linear? Is it distance or z coordinate in view space? Or even logarithmically scaled or something else? Are the motion vectors supposed to be in pixels per frame? What is the orientation of the axis? Is y up or down? Are there are other restrictions on the formats? Also the linked code did not help me (I have not found any SVGF so far; also all the code is in Objective-C++, not Swift, but that's a different topic). So how should I fill these textures. Can someone point me to the documentation where these kinds of questions are answered?

Hello If you add a ModelEntity to a world inside a portal, the drawing of the model will be occluded properly to the portal bounds. However the invisible shape of the InputTargetComponent and CollisionComponent are not occluded. They are able to cross the portal, and if you have gestures on your ModelEntity you can trigger them in areas outside the portal bounds. This happens even if the ModelEntity has no PortalCrossingComponent.

RealityKit spatial audio crackles and pops on iOS 26.0 beta 5. It works correctly on iOS 18.6 and visionOS 26.0 beta 5. The APIs used are AudioPlaybackController, Entity.prepareAudio, Entity.play Videos of the expected and observed behavior are attached to the feedback FB19423059. The audio should be a consistent, repeating sound, but it seems oddly abbreviated and the volume varies unexpectedly. Thank you for investigating this issue.

I'm trying to display a right-aligned timecode in my game. I had expected that digits would all have the same width, but this doesn't seem to be the case in SpriteKit, even though it seems to be the case in AppKit. In SpriteKit, with the default font there is a noticeable difference in width between the digit 1 and the rest (1 is thinner), so whenever displaying a number with the least significant digit 1 all preceding digits shift slightly to the right. This happens even when setting a NSAttributedString with a font that has a fixedAdvance attribute. class GameScene: SKScene { override func didMove(to view: SKView) { let label = SKLabelNode(text: "") view.scene!.addChild(label) // label.horizontalAlignmentMode = .left label.horizontalAlignmentMode = .right var i = 11 Timer.scheduledTimer(withTimeInterval: 0.5, repeats: true) { _ in label.text = "\(i)" // let font = NSFont(descriptor: NSFontDescriptor(fontAttributes: [.name: "HelveticaNeue-UltraLight", .fixedAdvance: 20]), size: 30)! // let paragraphStyle = NSMutableParagraphStyle() // paragraphStyle.alignment = .right // label.attributedText = NSAttributedString(string: "\(i)", attributes: [.font: font, .foregroundColor: SKColor.labelColor, .paragraphStyle: paragraphStyle]) i += 5 } } } With AppKit, when using SpriteKit's default font HelveticaNeue-UltraLight, this issue doesn't exist, regardless whether I set the fixedAdvance font attribute. class ViewController: NSViewController { override func viewDidLoad() { super.viewDidLoad() let font = NSFont(descriptor: NSFontDescriptor(fontAttributes: [.name: "HelveticaNeue-UltraLight"]), size: 30)! // let font = NSFont(descriptor: NSFontDescriptor(fontAttributes: [.name: "HelveticaNeue-Light", .fixedAdvance: 20]), size: 30)! let paragraphStyle = NSMutableParagraphStyle() paragraphStyle.alignment = .right let textField = NSTextField(labelWithString: "") textField.font = font textField.alignment = .right // textField.alignment = .left textField.frame = CGRect(x: 100, y: 100, width: 100, height: 100) view.addSubview(textField) var i = 11 Timer.scheduledTimer(withTimeInterval: 0.5, repeats: true) { _ in textField.stringValue = "\(i)" // textField.attributedStringValue = NSAttributedString(string: "\(i)", attributes: [.font: font, .paragraphStyle: paragraphStyle]) i += 5 } } } Is there a solution to this problem? I filed FB15553700.

Hi, I’m facing an issue with SceneKit. I’m developing a 3D mobile game. I have a character 3D model and several skeletal animations CAAnimation. I import both the model and the animations from Maya in *.dae format. The character’s animations play continuously one after the other, with each new animation being triggered randomly. The transition between animations makes smoothly by setting the fadeInDuration and fadeOutDuration properties. Here’s an example of the code: import UIKit import QuartzCore import SceneKit //---------------------- class TestAnimationController: UIViewController { var bodyNode: SCNNode? override func viewDidLoad() { super.viewDidLoad() let scnView = SCNView(frame: self.view.bounds) scnView.backgroundColor = .black // Set your desired background color scnView.autoresizingMask = [.flexibleWidth, .flexibleHeight] let scene = SCNScene(named: "art.scnassets/scene/Base_room/ROOM5.scn")! bodyNode = collada2SCNNode(filepath: "art.scnassets/female/girl_body_races.dae")! bodyNode?.renderingOrder = 10 scene.rootNode.addChildNode(bodyNode!) playIdleAnimation() scnView.scene = scene // Assign the scene to the SCNView self.view.addSubview(scnView) // Add the SCNView to your main view) } func collada2SCNNode(filepath:String) -> SCNNode? { if let scene = SCNScene(named: filepath) { let node = scene.rootNode.childNodes[0] return node } else { return nil } } func playIdleAnimation() { let array = [ "art.scnassets/female/animations/idle/girl_idle_4.dae", "art.scnassets/female/animations/idle/girl_idle1.dae", "art.scnassets/female/animations/idle/girl_idle2.dae", "art.scnassets/female/animations/idle/Girl_idle3.dae", ] let animation = CAAnimation.animationWithSceneNamed(array.randomElement() ?? "")! self.setAnimationAdd( fadeInDuration: 1.0, fadeOutDuration: 1.0, keyTime: 0.99, animation, isLooped: false ) { [weak self] in guard let self = self else { return } try? self.playBoringAnimations() } } func playBoringAnimations() { let array = [ "art.scnassets/female/animations/boring/girl_boring1.dae", "art.scnassets/female/animations/boring/girl_boring2.dae", "art.scnassets/female/animations/boring/girl_boring3.dae", "art.scnassets/female/animations/boring/girl_boring4.dae", "art.scnassets/female/animations/boring/girl_boring5.dae", "art.scnassets/female/animations/boring/girl_boring6.dae", "art.scnassets/female/animations/boring/girl_boring8.dae" ] let animation = CAAnimation.animationWithSceneNamed(array.randomElement() ?? "")! self.setAnimationAdd( fadeInDuration: 1.0, fadeOutDuration: 1.0, keyTime: 0.99, animation, isLooped: false ) { [weak self] in guard let self = self else { return } try? self.playIdleAnimation() } } func setAnimationAdd(fadeInDuration : CGFloat, fadeOutDuration : CGFloat, keyTime : CGFloat, _ animation: CAAnimation, isLooped: Bool, completion: (() -> Void)?) { animation.fadeInDuration = fadeInDuration animation.fadeOutDuration = fadeOutDuration if !isLooped { animation.repeatCount = 1 } else { animation.repeatCount = Float.greatestFiniteMagnitude } animation.animationEvents = [ SCNAnimationEvent(keyTime: keyTime, block: { _, _, _ in completion?() }) ] bodyNode?.addAnimation(animation, forKey: "avatarAnimation") } } Everything worked perfectly until I updated to iOS 18. On a physical device, the animations now transition abruptly without the smooth blending that was present in earlier iOS versions. The switch between them is very noticeable, as if the fadeInDuration and fadeOutDuration parameters are being ignored. However, in the iOS 18 simulator, the animations still transition smoothly as before. Here two example videos - IOS 17.5 and IOS 18 https://youtube.com/shorts/jzoMRF4skAQ - IOS 17,5 smooth https://youtube.com/shorts/VJXrZzO9wl0 - IOS 18 not smooth

Hi Friends! I’m facing an issue with SceneKit. I’m developing a 3D mobile game. I have a character 3D model and several skeletal animations CAAnimation. I import both the model and the animations from Maya in *.dae format. The character’s animations play continuously one after the other, with each new animation being triggered randomly. The transition between animations makes smoothly by setting the fadeInDuration and fadeOutDuration properties. Here’s an example of the code: import UIKit import QuartzCore import SceneKit class TestAnimationController: UIViewController { var bodyNode: SCNNode? override func viewDidLoad() { super.viewDidLoad() let scnView = SCNView(frame: self.view.bounds) scnView.backgroundColor = .black // Set your desired background color scnView.autoresizingMask = [.flexibleWidth, .flexibleHeight] let scene = SCNScene(named: "art.scnassets/scene/Base_room/ROOM5.scn")! bodyNode = collada2SCNNode(filepath: "art.scnassets/female/girl_body_races.dae")! bodyNode?.renderingOrder = 10 scene.rootNode.addChildNode(bodyNode!) playIdleAnimation() scnView.scene = scene // Assign the scene to the SCNView self.view.addSubview(scnView) // Add the SCNView to your main view) } func collada2SCNNode(filepath:String) -> SCNNode? { if let scene = SCNScene(named: filepath) { let node = scene.rootNode.childNodes[0] return node } else { return nil } } func playIdleAnimation() { let array = [ "art.scnassets/female/animations/idle/girl_idle_4.dae", "art.scnassets/female/animations/idle/girl_idle1.dae", "art.scnassets/female/animations/idle/girl_idle2.dae", "art.scnassets/female/animations/idle/Girl_idle3.dae", ] let animation = CAAnimation.animationWithSceneNamed(array.randomElement() ?? "")! self.setAnimationAdd( fadeInDuration: 1.0, fadeOutDuration: 1.0, keyTime: 0.99, animation, isLooped: false ) { [weak self] in guard let self = self else { return } try? self.playBoringAnimations() } } func playBoringAnimations() { let array = [ "art.scnassets/female/animations/boring/girl_boring1.dae", "art.scnassets/female/animations/boring/girl_boring2.dae", "art.scnassets/female/animations/boring/girl_boring3.dae", "art.scnassets/female/animations/boring/girl_boring4.dae", "art.scnassets/female/animations/boring/girl_boring5.dae", "art.scnassets/female/animations/boring/girl_boring6.dae", "art.scnassets/female/animations/boring/girl_boring8.dae" ] let animation = CAAnimation.animationWithSceneNamed(array.randomElement() ?? "")! self.setAnimationAdd( fadeInDuration: 1.0, fadeOutDuration: 1.0, keyTime: 0.99, animation, isLooped: false ) { [weak self] in guard let self = self else { return } try? self.playIdleAnimation() } } func setAnimationAdd(fadeInDuration : CGFloat, fadeOutDuration : CGFloat, keyTime : CGFloat, _ animation: CAAnimation, isLooped: Bool, completion: (() -> Void)?) { animation.fadeInDuration = fadeInDuration animation.fadeOutDuration = fadeOutDuration if !isLooped { animation.repeatCount = 1 } else { animation.repeatCount = Float.greatestFiniteMagnitude } animation.animationEvents = [ SCNAnimationEvent(keyTime: keyTime, block: { _, _, _ in completion?() }) ] bodyNode?.addAnimation(animation, forKey: "avatarAnimation") } } Everything worked perfectly until I updated to iOS 18. On a physical device, the animations now transition abruptly without the smooth blending that was present in earlier iOS versions. The switch between them is very noticeable, as if the fadeInDuration and fadeOutDuration parameters are being ignored. However, in the iOS 18 simulator, the animations still transition smoothly as before. Here two example videos - IOS 17.5 and IOS 18 https://youtube.com/shorts/jzoMRF4skAQ - IOS 17,5 smooth https://youtube.com/shorts/VJXrZzO9wl0 - IOS 18 not smooth I try this code in IOS 17.5, everything works fine Does anyone have any ideas on how to fix this issue?

I am trying to implement a ChacterControllerComponent using the following URL. https://developer.apple.com/documentation/realitykit/charactercontrollercomponent I have written sample code, but PhysicsSimulationEvents.WillSimulate is not executed and nothing happens. import SwiftUI import RealityKit import RealityKitContent struct ImmersiveView: View { let gravity: SIMD3<Float> = [0, -50, 0] let jumpSpeed: Float = 10 enum PlayerInput { case none, jump } @State private var testCharacter: Entity = Entity() @State private var myPlayerInput = PlayerInput.none var body: some View { RealityView { content in // Add the initial RealityKit content if let immersiveContentEntity = try? await Entity(named: "Immersive", in: realityKitContentBundle) { content.add(immersiveContentEntity) testCharacter = immersiveContentEntity.findEntity(named: "Capsule")! testCharacter.components.set(CharacterControllerComponent()) let _ = content.subscribe(to: PhysicsSimulationEvents.WillSimulate.self, on: testCharacter) { event in print("subscribe run") let deltaTime: Float = Float(event.deltaTime) var velocity: SIMD3<Float> = .zero var isOnGround: Bool = false // RealityKit automatically adds `CharacterControllerStateComponent` after moving the character for the first time. if let ccState = testCharacter.components[CharacterControllerStateComponent.self] { velocity = ccState.velocity isOnGround = ccState.isOnGround } if !isOnGround { // Gravity is a force, so you need to accumulate it for each frame. velocity += gravity * deltaTime } else if myPlayerInput == .jump { // Set the character's velocity directly to launch it in the air when the player jumps. velocity.y = jumpSpeed } testCharacter.moveCharacter(by: velocity * deltaTime, deltaTime: deltaTime, relativeTo: nil) { event in print("playerEntity collided with \(event.hitEntity.name)") } } } } } } The scene is loaded from RCP. It is simple, just a capsule on a pedestal. Do I need a separate code to run testCharacter from this state?

I created an empty binary archive and try to save it on disk, but it returned false without giving any error message. The code is from https://developer.apple.com/documentation/metal/shader_libraries/metal_binary_archives/creating_binary_archives_from_device-built_pipeline_state_objects?language=objc

I want to implement the ability to apply Lightroom Preset (.xmp file) to an image in my app, but am running into difficulties. How can I configure things like color grading, curve, etc. in Swift?

I would like to take YCbCr CVPixelBuffers from AVCaptureVideoDataOutput, apply some processing in RGB space, render to an MTKView, and pass to AVAssetWriter for recording. Right now, I'm doing this all manually – deswing the incoming data if necessary, choose the right matrix to convert to RGB, apply processing, etc. I also have to convert back to YCbCr before feeding the frames to AVAssetWriter because encoding performs much better if I do. Is there any efficient, built-in way to achieve the same? I can't use AVCaptureVideoPreviewLayer, since I need to do some further processing before display. I can't use AVCaptureVideoDataOutput's videoSettings to get automatic BGRA conversion because that would lose bit depth for 10 bit video formats (and isn't available on all formats anyway). I see these Accelerate functions, but they seemingly don't use the GPU, nor do they support all the formats and bit depths I'd need. I found reference to some undocumented MTLPixelFormats that seem to do exactly what I want, but I don't want to rely on something like this unless it's explicitly endorsed. This would also incur an RGB/YCbCr conversion on every texture read and write, right? Is there anything I'm missing here?