I would like to take YCbCr CVPixelBuffers from AVCaptureVideoDataOutput, apply some processing in RGB space, render to an MTKView, and pass to AVAssetWriter for recording. Right now, I'm doing this all manually – deswing the incoming data if necessary, choose the right matrix to convert to RGB, apply processing, etc. I also have to convert back to YCbCr before feeding the frames to AVAssetWriter because encoding performs much better if I do. Is there any efficient, built-in way to achieve the same? I can't use AVCaptureVideoPreviewLayer, since I need to do some further processing before display. I can't use AVCaptureVideoDataOutput's videoSettings to get automatic BGRA conversion because that would lose bit depth for 10 bit video formats (and isn't available on all formats anyway). I see these Accelerate functions, but they seemingly don't use the GPU, nor do they support all the formats and bit depths I'd need. I found reference to some undocumented MTLPixelFormats that seem to do exactly what I want, but I don't want to rely on something like this unless it's explicitly endorsed. This would also incur an RGB/YCbCr conversion on every texture read and write, right? Is there anything I'm missing here?

Dear Apple Support Team, I recently purchased an iPad Pro 2022 and updated it to iOS 18.2. However, I am experiencing an issue while using Call of Duty Mobile. The Game Mode activates randomly and sometimes does not activate at all. Additionally, when the Game Mode is on, the game crashes unexpectedly, causing an unstable experience. I kindly request that you address this issue in upcoming iOS updates. Thank you for your attention and support. Best regards, [samadBg]

I want to implement the ability to apply Lightroom Preset (.xmp file) to an image in my app, but am running into difficulties. How can I configure things like color grading, curve, etc. in Swift?

I have an oval UIBezierPath with a moving SKSpriteNode, I stop its motion and record the stopped position. I then restart this motion and want it to restart where it initially stopped. Works great if motion is not stopped. Movement is great around entire oval Path. Also works great as long as this stop-restart sequence occurs along the top half of the oval UIBezierPath. However, I have problems along the bottom half of this Path -- it stops okay, but the restart position is not where it previously stopped. My method to create this oval UIBezierePath is as follows: func createTrainPath() { trainRect = CGRect(x: tracksPosX - tracksWidth/2, y: tracksPosY - tracksHeight/2, width: tracksWidth, height: tracksHeight) // these methods come from @DonMag trainPoints = generatePoints(inRect: trainRect, withNumberOfPoints: nbrPathPoints) trainPath = generatePathFromPoints(trainPoints!, startingAtIDX: savedTrainIndex) } // createTrainPath My method to stop this motion is as follows: func stopFollowTrainPath() { guard (myTrain != nil) else { return } myTrain.isPaused = true savedTrainPosition = myTrain.position // also from @DonMag savedTrainIndex = closestIndexInPath( trainPath, toPoint: savedTrainPosition) ?? 0 } // stopFollowTrainPath Finally, I call this to re-start this motion: func startFollowTrainPath() { var trainAction = SKAction.follow(trainPath.cgPath, asOffset: false, orientToPath: true, speed: thisSpeed) trainAction = SKAction.repeatForever(trainAction) myTrain.run(trainAction, withKey: runTrainKey) myTrain.isPaused = false } // startFollowTrainPath Again, great if motion is not stopped. Movement is great around entire oval Path. Again, no problem for stopping and then restarting along top half of oval .. the ohoh occurs along bottom half. Is there something I need to do within GameScene's update method that I am missing? For example, do I need to reconstruct my UIBezierPath? every time my node moves between the top half and the bottom half and therein account for the fact that the node is traveling in the opposite direction from the top half?

In macOS project with RealityKit and SwiftUI, adding OrthographicCameraComponent causes crashes in both Xcode Preview and at runtime. import SwiftUI import RealityKit struct ContentView: View { var body: some View { RealityView { content in var camera = Entity() var component = OrthographicCameraComponent() component.scale = 5 camera.position = [0, 0, 5] camera.components.set(component) content.add(camera) content.add(ModelEntity(mesh: .generateSphere(radius: 1))) } } } #Preview { ContentView() } Has anyone faced this issue or knows a fix?

Ever since the release of iOS18, we've been seeing a new crash related to calling jpegData(compressionQuality:). From reports, this isn't crashing during foreground usage of the app, but instead will prompt the user about a background app crash upon foregrounding. The stacks from crash reports show this crash is happening from a variety of callers, but all go through jpegData(compressionQuality:), down through [HDRImageConverter_Metal init] and end up in apthread_mutex_local call when it crashes. Attached is a sample crash report from 18.2(22C5125e), but we've been seeing this since the first iOS18 release. Did something change with around these calls in iOS18 that prohibits their use in the background? crash.txt

Hi, I tried to save the game progress using the official Apple plugin for Unity but the crash happened when I active the "iCloud Documents" inside capabilities and when deactivated it this error message appeared: Code=27 Domain=GKErrorDomain Description=The requested operation could not be completed because you are not signed in to iCloud or have not enabled iCloud Drive. (UbiquityContainerUnavailable) Authentication with Game Center works fine using the Core plugin, but nothing works correctly when I use the GameKit plugin. Note: I already active iCloud for app Identifier. Tichnecal informations: Unity version: 2022.3.47f1 LTS XCode 16 Swift 6 GameKit-3.0.0 (Apply unity plugin) Core-3.1.5 (Apply unity plugin)

I am working on adding synchronized physical properties to EntityEquipment in TableTopKit, allowing seamless coordination during GroupActivities sessions between players. Treating EntityEquipment's state to DieState is not a way, because it doesn't support custom collision shapes. I have also tried adding PhysicsBodyComponent and CollisionComponent to EntityEquipment's Entity. However, the main issue is that the position of EntityEquipment itself does not synchronize with the Entity's physics body, resulting in two separate instances of one object. struct PlayerPawn: EntityEquipment { let id: ID let entity: Entity var initialState: BaseEquipmentState init(id: ID, entity: Entity) { self.id = id let massProperties = PhysicsMassProperties(mass: 1.0) let material = PhysicsMaterialResource.generate(friction: 0.5, restitution: 0.5) let shape = ShapeResource.generateBox(size: [0.4, 0.2, 0.2]) let physicsBody = PhysicsBodyComponent(massProperties: massProperties, material: material, mode: .dynamic) let collisionComponent = CollisionComponent(shapes: [shape]) entity.components.set(physicsBody) entity.components.set(collisionComponent) self.entity = entity initialState = .init(parentID: .tableID, pose: .init(position: .init(), rotation: .zero), entity: self.entity) } } I’d appreciate any guidance on the recommended approach to adding synchronized physical properties to EntityEquipment.

Hi guys! Is there any way to get a frame at certain time? I'm writing plug-in and want to use 2 frames before and 2 frames after current frame in order to render final image.

I am writing an app to create 3D objects with curved surfaces such as a metal cabinet knob using SceneKit and Model I/O. I want the surfaces to be smooth so that edges between adjacent polygon faces are not visible. According to the documentation for MDLMesh.addNormals(withAttributeNamed: creaseThreshold:), a positive creaseThreshold value lower than 1.0 will interpolate sharper angles between faces into smooth surfaces. I have not been able to get this to work, and I need help with it. The lines of code where the problem occurs are shown here. let mesh = MDLMesh(scnGeometry: surfaceGeometry) // mesh.addNormals(withAttributeNamed: "MDLVertexAttributeNormal", creaseThreshold: 0.9) surfaceGeometry = SCNGeometry(mdlMesh: mesh) When the code is executed with middle line commented out, the knob object is rendered as shown in the screenshot. When that line is not commented out, mesh is altered and the SCNNode for the knob is created with no errors, but the node is not rendered. The questions I have are: (1) What changes do I need the make to the code so that the node will be rendered with a smooth surface?, and (2) what is the recommended way of smoothing a curved surface so that edges between faces are not visible? The full code for the function and a screenshot of the faceted knob object are attached. ![]("https://developer.apple.com/forums/content/attachment/a17feca7-ed6f-440c-add6-760a1cbf8778" "title=Screenshot cabinet knob with faceted surface.png;width=790;height=568") code-block func cabinetKnob() -> SCNNode { let controlPoints: [(x: Float, y: Float)] = [ (0.728,-0.237), (0.176,-0.06), (0.202,0.475), (0.989,0.842), (-0.066,1.093), (-0.726,0.787) ] let pairs = bsplinePath(controlPoints) var knobProfile = [SCNVector3]() for (x,y) in pairs { knobProfile += [ SCNVector3(x: CGFloat(x), y: CGFloat(y), z: 0)] } let nProfiles = 64 // create knob by rotating knobProfile about y-axis let aIncrement: CGFloat = 2 * CGFloat.pi / CGFloat(nProfiles) // ~6 degrees var angle: CGFloat = 0 var knobVertices = knobProfile.map( { $0 } ) angle = 0 for _ in 1...nProfiles { angle += aIncrement // rotate knobProfile about y-axis knobVertices += knobProfile.map( { $0.rotate(about: .y, by: angle) } ) } let source = SCNGeometrySource(vertices: knobVertices) var indices = [[UInt16]]() var i: UInt16 = 0 var j: UInt16 = UInt16(knobProfile.count) // 1st vertex of next profile for k in 0...nProfiles { var stripIndices = [UInt16]() if k == nProfiles { j = 0 } for _ in 0...knobProfile.count-1 { stripIndices += [i, j] i += 1; j += 1 } indices += [stripIndices] } let elements: [SCNGeometryElement] = indices.map( { SCNGeometryElement(indices: $0, primitiveType: .triangleStrip) } ) var surfaceGeometry = SCNGeometry(sources: [source], elements: elements) let mesh = MDLMesh(scnGeometry: surfaceGeometry) // mesh.addNormals(withAttributeNamed: "MDLVertexAttributeNormal", creaseThreshold: 0.9) surfaceGeometry = SCNGeometry(mdlMesh: mesh) let aluminum = SCNMaterial() aluminum.lightingModel = SCNMaterial.LightingModel.physicallyBased aluminum.diffuse.contents = NSColor(srgbRed: 0.95, green: 0.95, blue: 0.95, alpha: 1.0) aluminum.roughness.contents = 0.2 aluminum.metalness.contents = 0.9 aluminum.isDoubleSided = true surfaceGeometry.materials = [ aluminum ] let node = SCNNode(geometry: surfaceGeometry) return node }

I am trying to install the game-porting-toolkit using brew -v install apple/apple/game-porting-toolkit but this fails each time because of a dependency on a deprecated openssl version: Fetching dependencies for apple/apple/game-porting-toolkit: cmake, ninja, apple/apple/game-porting-toolkit-compiler, openssl1.1 ... ... Error: openssl@1.1 has been disabled because it is not supported upstream! It was disabled on 2024-10-24. Is there a way to override this dependency or use a newer version of openssl for the check?

SceneKit SCNScene MacOS 15.1 Xcode 16.0 SceneView(scene: , options:[autoenablesDefaultLighting, allowsCameraContol] there is: .rootNode.cameraNode .rootNode.camera .rootNode.nestedChildNodes each with its own animation when the object is animated and dragged by mouse to change the view point, I can't return the view to the previous view. I have reinstated a clone of the original cameraNode, positions of all childNodes, removed and re-activated all animations... in vain. I have also cloned, removed and replaced .rootNode.camera, in vain. The documentation states the camera is "attached" to an SCNNode but does not say how. I make no declaration to associate .rootNode.cameraNode to .rootNode.camera yet if either is absent there is no scene to view. What am I missing? Thanks

I've noticed a major third-party app has the following flag set to 1/true in its Info.plist: CAMetalLayerWantsCompositingDependencies Does anyone know if it’s recognized by Core Animation / Metal, and what it’s supposed to do? It might obviously have zero relationship to the OS, defined by that app and for that app... but since it looks very much like an unofficial/undocumented environment setting, it might be great to know what problem it solves. I happen to have issues related to compositing other CALayers over a CAMetalLayer in my app... so this definitely stood out as interesting. Thank you!

I am building a MacOS desktop app (https://anukari.com) that is using Metal compute to do real-time audio/DSP processing, as I have a problem that is highly parallelizable and too computationally expensive for the CPU. However it seems that the way in which I am using the GPU, even when my app is fully compute-limited, the OS never increases the power/performance state. Because this is a real-time audio synthesis application, it's a huge problem to not be able to take advantage of the full clock speeds that the GPU is capable of, because the app can't keep up with real-time. I discovered this issue while profiling the app using Instrument's Metal tracing (and Game tracing) modes. In the profiling configuration under "Metal Application" there is a drop-down to select the "Performance State." If I run the application under Instruments with Performance State set to Maximum, it runs amazingly well, and all my problems go away. For comparison, when I run the app on its own, outside of Instruments, the expensive GPU computation it's doing takes around 2x as long to complete, meaning that the app performs half as well. I've done a ton of work to micro-optimize my Metal compute code, based on every scrap of information from the WWDC videos, etc. A problem I'm running into is that I think that the more efficient I make my code, the less it signals to the OS that I want high GPU clock speeds! I think part of why the OS is confused is that in most use cases, my computation can be done using only a small number of Metal threadgroups. I'm guessing that the OS heuristics see that only a small fraction of the GPU is saturated and fail to scale up the power/clock state. I'm not sure what to do here; I'm in a bit of a bind. One possibility is that I intentionally schedule busy work -- spin threadgroups just to waste energy and signal to the OS that I need higher clock speeds. This is obviously a really bad idea, but it might work. Is there any other (better) way for my app to signal to the OS that it is doing real-time latency-sensitive computation on the GPU and needs the clock speeds to be scaled up? Note that game mode is not really an option, as my app also runs as an AU plugin inside hosts like Garageband, so it can't be made fullscreen, etc.

I'm displaying a GKGameCenterViewController after successfully authenticating and on iOS 18.0 and 18.1, I get a black screen. As a sanity check GKLocalPlayer.local.isAuthenticated is also returning true. The same code works just fine on iOS 17. Is there something that needs to be done on iOS 18 and above?

I have multiple CAMetalLayers that I render content to and noticed that the graphics overview HUD does not function properly when I have more than one CAMetalLayer. The values reported will be very strange. For example, FPS may report 999 or some large negative value. It the HUD simply not designed to work with multiple CAMetalLayers or MTKViews? When I disable all but one of my CAMetalLayers, the HUD works as expected.

I'm sure this question was asked many times before but I cannot find a good answer. In my case it just doesn't work. Here's what I did Created a couple of test user accounts on appstore connect under the sandbox section. Launched two different simulators via xcode. On each simulator I logged in into test icloud account and game center accordingly. Deployed and launched my code via xcode. What happens is that GameKit code fails to find any peers no matter what I do. Sending Invite doesn't work either because the simulator displays an error message saying "you need to log-in into icloud account first" although it's already logged in. I tried the same code on two different physical devices with two real icloud accounts and it works as expected but it's not a viable path to develop and debug an app. I'm using the latest Xcode 16.1 running on 15.1. Anybody has any clue how to solve this ?

I want use SwiftUI views as RealityKit entities to display AR Labels within a RealityKit scene, and the labels could be more complicated than just text and window as they might include images, dynamic texts, animations, WebViews, etc. Vision OS enables this through RealityView attachments, and there is a RealityView support on iOS 18. Tried running RealityView attachments code samples from VisionOS on iOS 18. However, the code below gives errors on iOS 18: import SwiftUI import RealityKit struct PassportRealityView: View { let qrCodeCenter: SIMD3<Float> let assetID: String var body: some View { RealityView { content, attachments in // Setup your AR content, such as markers or 3D models if let qrAnchor = try? await Entity(named: "QRAnchor") { qrAnchor.position = qrCodeCenter content.add(qrAnchor) } } attachments: { Attachment(id: "passportTextAttachment") { Text(assetID) .font(.title3) .foregroundColor(.white) .background(Color.black.opacity(0.7)) .padding(5) .cornerRadius(5) } } .frame(width: 300, height: 400) } } When I remove "attachments" keyword and the block, the errors are kind of gone. That does not help me as I want to attach SwiftUI views to Anchor Entities in RealityKit. As I understand, RealityView attachments are not supported on iOS 18. I wonder if there is any way of showing SwiftUI views as entities on iOS 18 at this point. Or am I forced to use the text meshes and 3d planes to build the UI? I checked out the RealityUI plugin, but it's too simple for my use case of building complex AR labels. Any advice would be appreciated. Thanks!

Hello, We are experimenting with Metal to accelerate some peculiar numerical computation. Our workloads are relatively small, so the ability to avoid moving data to and from the GPU's memory is very appealing. However, we are observing higher overhead compared to CUDA, which negates the benefits of avoiding data transfer. In our tests using an empty kernel, CUDA completes in 0.001 ms (Intel i7 10700K, RTX 3080), while Metal's waitUntilCompleted takes 0.12 ms (M2 Max). As we do not have prior experience with Metal, we are wondering if we are using the APIs just fine and this timing is expected, or if there is a way to reduce it. Thank you in advance for any comment! test-metal.cpp

Once GKAccessPoint is active, then enter an arview page, the arview will lose camera feed. OSVersion: iOS 18.0.1, iOS 18.1