I'd like to use the eye tracking feature in the latest iPadOS 18 update as more than an accessibility feature. i.e. another input modality that can be detected by event + enum checks similar to how we can detect and distinguish between touches and Apple pencil inputs. This might make it a lot easier to control and interact with iPad-based AR experiences that involve walking around, regardless of whether eye-tracking is enabled for accessibility. When walking, it's challenging to hold the device and interact with the screen with touch or pencil at all. Eye tracking + speech as input modalities could assist here. Also, this would help us create non-immersive AR experiences that parallel visionOS experiences that use eye tracking. I propose an API option for enabling eye-tracking (and an optional calibration dialogue within-app), as well as a specific UIControl class that simply detect when the eye looks at the control using the standard (begin/changed/end) events. My specific use case is that I'd like to treat eye-tracking-enabled UI elements or game objects differently depending on whether something is looked at with the eyes. For example, to select game objects while using speech recognition, suppose we have 4 buttons with the same name in 4 corners of the screen. Call them "corner" buttons. If I have my proposed invisible UI element for gaze detection, I can create 4 large rectangular regions on the screen. Then if the user says "select the corner" the system could parse this command and disambiguate between the 4 corners by checking which of the rectangular regions I'm currently looking at. (Note: the idea would be to make the gaze regions rather large to compensate for error.) The above is just a simple example, but the advantage over other methods like dwell is that it could be a lot faster. Another simple example: Using the same rectangular regions, instead of speech input, I could hold a button placed in just one spot on the screen, and look around the screen with my gaze to produce a laser beam for some kind of game, or draw curves (that I might smooth-out to reduce inaccuracy). OR maybe someone who does not have their hands available. This would require us to have the ability to get the coordinates of the eye gaze, but otherwise the other approach of just opting to trigger uicontrol elements might work for coarse selection. Would other developers find this useful as well? I'd like to propose this feature in feedback assistant, but I'm also opening-up a little discussion if by chance someone sees this. In short, I propose: a formal eye-tracking API for iPadOS 18+ that allows for turning on/off the tracking within the app, with the necessary user permissions the API should produce begin/changed/ended events similar to the existing events in UIKit, including screen coordinates. There should be a way to identify that an event came from eye-tracking. alternatively, we should have at minimum an invisible UIControl subclass that can detect when the eyes enter/leave the region.

Let's say the type of project doesn't work well by using the LSApplicationCategoryType property in Info.plist. How can I trigger it via code in MacOS?

I rewrite an old project since OpenGL is deprecated and the downgrade to High Sierra goes with the lost of the old project. Here is the drawRect: - (void)drawRect:(NSRect)dirtyRect { [super drawRect:dirtyRect]; if(backgroundColor == 1) glClearColor(0.95f, 1.0, 1.0f, 1.0); else glClearColor(0.0f, 0.0f, 0.0f, 1.0f); glClear (GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); glLoadIdentity(); glTranslatef(0.0 ,0.0, -10.0); // After mistake I wrote PushMatrix() glTranslatef(-0.01 ,-0.01, -10.0); // Screen Shot 0 // glTranslatef(-0.25 ,-0.25, -10.0); Screen Shot 1 glLineWidth(1.0); glRotated(rotationX, 1, 0, 0); glRotated(rotationY, 0, 1, 0); glCallList(axes); // After mistake I wrote PopMatrix() // Axes Vertices of xx,y,z are 1.0 // With vertices +/-0.99 nothing is drawn [self glError]; [self.openGLContext flushBuffer]; } I hope you Accept my Text and hope you can help me ! Mit freundlichen Grüßen Uwe Screen Shot 0 Screen Shot 1

Working on a vision OS app. I've noticed that even when castsShadow is false, performance goes down the drain when there are more than a few dozen entities that have GroundingShadowComponent. I managed to hard crash the Vision Pro with about 200 or so entities that each had two ModelEntities with GroundingShadowComponent attached but set to castShadows = false. My solution is to add and remove the GroundingShadowComponent from entities as needed, but I thought maybe someone at Apple might want to look into this. I don't expect great performance with that many entities casting shadows, but I'd think turning the shadow off would effectively disable the component and not incur a performance penalty.

When developing VR experiences for the Apple Vision Pro, using Unreal Engine 5.4. Can I deploy the project with deferred rendering? Or do I still need to use forward rendering? (for mobile devices) Thanks for any information on that, appreicate it!

What is the most efficient way to use a MTLTexture (created procedurally at run-time) as a RealityKit TextureResource? I update the MTLTexture per-frame using regular Metal rendering, so it’s not something I can do offline. Is there a way to wrap it without doing a copy? A specific example would be great. Thank you!

I'm using DrawableQueue to create textures that I apply to my ShaderGraphMaterial texture. My metal render is using a range of alpha values as a test. My objects displayed with the DrawableQueue texture are working as expected, but the alpha component is not working. Is this an issue with my DrawableQueue descriptor? My ShaderGraphMaterial? A missing setting on my scene objects? or some limitation in visionOS? DrawableQueue descriptor let descriptor = await TextureResource.DrawableQueue.Descriptor( pixelFormat: .rgba8Unorm, width: textureResource!.width, height: textureResource!.height, usage: [.renderTarget, .shaderRead, .shaderWrite], // Usage should match the requirements for how the texture will be used //usage: [.renderTarget], // Usage should match the requirements for how the texture will be used mipmapsMode: .none // Assuming no mipmaps are needed for the text texture ) let queue = try await TextureResource.DrawableQueue(descriptor) queue.allowsNextDrawableTimeout = true await textureResource!.replace(withDrawables: queue) Draw frame: guard let drawable = try? drawableQueue!.nextDrawable(), let commandBuffer = commandQueue?.makeCommandBuffer()//, //let renderPipelineState = renderPipelineState else { return } let renderPassDescriptor = MTLRenderPassDescriptor() renderPassDescriptor.colorAttachments[0].texture = drawable.texture renderPassDescriptor.colorAttachments[0].loadAction = .clear renderPassDescriptor.colorAttachments[0].storeAction = .store renderPassDescriptor.colorAttachments[0].clearColor = clearColor /*renderPassDescriptor.colorAttachments[0].clearColor = MTLClearColor( red: clearColor.red, green: clearColor.green, blue: clearColor.blue, alpha: 0.5 )*/ renderPassDescriptor.renderTargetHeight = drawable.texture.height renderPassDescriptor.renderTargetWidth = drawable.texture.width guard let renderEncoder = commandBuffer.makeRenderCommandEncoder(descriptor: renderPassDescriptor) else { return } renderEncoder.pushDebugGroup("DrawNextFrameWithColor") //renderEncoder.setRenderPipelineState(renderPipelineState) // No need to create a render command encoder with shaders, as we are only clearing the drawable. // Since we are just clearing the drawable to a solid color, no need to draw primitives renderEncoder.endEncoding() commandBuffer.commit() commandBuffer.waitUntilCompleted() drawable.present() }

Hi, I'm displaying linear gray by CAMetalLayer with the shader below. fragment float4 fragmentShader(VertexOut in [[stage_in]], texture2d<float, access::sample> BGRATexture [[ texture(0) ]]) { float color = in.texCoordinates.x; return float4(float3(color), 1.0); } And my CAMetalLayer has been set to linearSRGB. metalLayer.colorspace = CGColorSpace(name: CGColorSpace.linearSRGB) metalLayer.pixelFormat = .bgra8Unorm Why the display seem add gamma? Apparently the middle gray is 187 but not 128.

Hi Apple and Swift friends. I'm not really a fluent Swift programmer (or any language) just knowledgable, I just a vague logic of what's going on. This is a gamepad controller remapper similar to DS4Windows on the PC and DSX on Steam gaming. On macOS Sonoma, it successfully compiled and connected the amazing Sony Playstation DualSense but because it has 33 yellow warning, the Setting doesn't show. It says something about outdated something and needs to be replaced by a newer framework.network. It also says it can't use self: It should look like these: What could be the syntax changes that won't produce the 30+ yellow warnings? The file can be had here: [https://github.com/marcowindt/ds4macos) God bless.

Hello. When I try to profile a frame capture of an app running on iPad, I get the following error: Unexpected Replayer Termination Abort trap: 6 This is my configuration: macOS 14.0 Xcode 15.0 iPad Pro (10.5-inch) - A10X iPadOS 17.2 Is this expected? Is there a way to fix the issue? Thanks

I have my Xbox Controller plugged into my Macbook, but my controller won't stay on (no batteries are needed for this controller, and the plug I am using works). Also in system settings when I scroll to Xbox Controllers it says no devices found. How do I fix this?

Hello community this post is for show my complete unsatisfied with Apple specially on developing games for Apples platforms there is lack of support for it for example some new gaming technologies and still that there is no profit or worth from all the work and money invested to develop for it I will close the journey with Apple very unsatisfied I'm going to give opportunities with my business to other platforms that are really worth it and give support to all new technologies in gaming and yes Apple destroyed other gaming makers with their new services like arcade and seems no future for gaming in Apples platforms. Quit goodbye and good luck to everyone.

Hello everyone! After some time to think about I proceed with graphics api, I figured opengl will be my first since I'm completely new to graphics programming. As in my last post you may find, I was speaking on moltenvk and might just use metal instead, along with the demos I found using metal. So for now, and I know this is said MANY TIMES, apple deprecated opengl but wish to use it because I'm new to graphics programming and want to develop an app(a rendering engine really) for the iPhone 14 Pro Max and macOS Ventura 13.2(I think this is the latest). So what do you guys think? Can I still use opengl es on the 14 max, along with opengl 4+ on latest macOS even though is deprecated?

The touch input stutter issue that exists since iOS 16 on devices with Pro Motion Displays has not been fixed yet. I filed a bug report in July but there isn't any progress since months. I see the problem in all games I tried. My game is fast paced so the stutters are quite obvious and I receive a lot of complaining emails. My game did run smoothly on Pro Motion devices with iOS 15. Is there a known workaround? I am seeing other developers having the same issue but I can't find any solutions. Other threads about this issue: IPhone 14 Pro stuttering in most games when using touch controls FPS drops when tapping the screen on iPhone 13 Pro Max

I am trying to control the orientation of a box in Scene Kit (iOS) using gestures. I am using the translation in x and y to update the x and y rotation of the SCNNode. After a long search I have realised that x and y rotation will always lead to z rotation, thanks to this excellent post: [https://gamedev.stackexchange.com/questions/136174/im-rotating-an-object-on-two-axes-so-why-does-it-keep-twisting-around-the-thir?newreg=130c66c673f848a7be2873bf675573a9) So I am trying to get the z rotation causes, and then remove this from my object by applying the inverse quaternion however when I rotate the object 90 deg around x, and then 90 deg around Y it behaves VERY weirdly. It is almost behaving as it is in gimbal lock, but I did not think that using quaternion in the way that I am would cause gimbal lock in this way. I am sure it is something I am missing, or perhaps I am not able to remove the z rotation in this way. Thanks! I have added a video of the strange behaviour here [https://github.com/marcusraty/RotationExample/blob/main/Example.MP4) And the code example is here [https://github.com/marcusraty/RotationExample)

Hello everyone. I wonder what conditions should be met to let Sonoma activate Game Mode for my App? The app is a 3D application with high demands on GPU, so gaining additional performance from OS would be great. But simply going full screen does not activate Game Mode like e.g. games from Steam do. Are there any settings that one could influence to let the app trigger Game Mode? Best regards, Alex

I've got a Mac Mini running as a CI/CD server; it's a game project, and every time we push a change to the repo, the server creates a build & uploads it to TestFlight. I've noticed that after running the server for around a week or so, the main drive is getting filled to the point of the machine becoming unresponsive (and our builds failing). After some investigation, I've determined that /private/var/folders/ is getting filled up with... something, related to the Xcode build/archive. It seems like we absolutely do not need this information, so I've periodically run rm -rf /private/var/folders/* to clean it out – some folders are permission-protected, but regardless, I'm able to recover >100GB with this method. Although this workaround does suffice, I'm just wondering what kinds of files Xcode is creating here, and whether there's a cleaner/best-practice method to remove them.

How is this a valid stack trace with mangled symbol names and file/line information? I've already demangled the name, even though Windows does this for me. The recommended approach to get file/line seems to be to proc "atos" process repeatedly on the symbols to turn them all into file/line. But shouldn't there just be a function call for this, or option to backtrace_symbols() since it's looking up the symbol anyways. I don't get how this external process call would work for iOS, and it seems slow for macOS as well. Compare this with Window CaptureStackBackTrace, and then there is a simple function call via DbgHelp.lib to retrieve the file/line. Am I supposed to somehow use the CoreSymbolicate framework on macOS/iOS?

The macOS screen recording tool doesn't appear to support recording HDR content (f.e. in QuickTime player). This tool can record from the camera using various YCbCr 422 and 420 formats needed for HVEC and ProRes HDR10 recording, but doesn't offer any options for screen recording HDR. So that leaves in-game screen recording with AVFoundation. Without any YCbCr formats exposed in Metal api, how do we use CVPixelBuffer with Metal, and then send these formats off to the video codes directly? Can we send Rec2020 RGB10A2Unorm data directly? I'd like the fewest conversions possible.

I am facing an issue while playing pubg on 14 pro max with guided access on or while screen recording. Whenever one of these feature is on, there is a lot of fps drops. It doesn't matter if I just started playing or if phone's temprature is high, but there's no 90 fps. With both these features off, I get 90 fps. Can someone help me with this issue?