I have attempted to use VideoMaterial with HDR HLS stream, and also a TextureResource.DrawableQueue with rgba16Float in a ShaderGraphMaterial. I'm capturing to 64RGBAHalf with AVPlayerItemVideoOutput and converting that to rgba16Float. I don't believe it's displaying HDR properly or behaving like a raw AVPlayer. Since we can't configure any EDR metadata or color space for a RealityView, how do we display HDR video? Is using rgba16Float supposed to be enough? Is expecting the 64RGBAHalf capture to handle HDR properly a mistake and should I capture YUV and do the conversion myself? Thank you

Why do I get this error almost immediately on starting my rendering pass? Multiline BlockQuote. 2024-05-29 20:02:22.744035-0500 RoomPlanExampleApp[491:10341] [] <<<< AVPointCloudData >>>> Fig assert: "_dataBuffer" at bail (AVPointCloudData.m:217) - (err=0) 2024-05-29 20:02:22.744455-0500 RoomPlanExampleApp[491:10341] [] <<<< AVPointCloudData >>>> Fig assert: "_dataBuffer" at bail (AVPointCloudData.m:217) - (err=0) 2024-05-29 20:05:54.079981-0500 RoomPlanExampleApp[491:10025] [CAMetalLayer nextDrawable] returning nil because allocation failed. 2024-05-29 20:05:54.080144-0500 RoomPlanExampleApp[491:10341] [] <<<< AVPointCloudData >>>> Fig assert: "_dataBuffer" at bail (AVPointCloudData.m:217) - (err=0)

I've got a full-screen animation of a bunch of circles filled with gradients, with plenty of (careless) overdraw, plus real-time audio processing driving the animation, plus the overhead of SwiftUI's dependency analysis, and that app uses less energy (on iPhone 13) than the Xcode "Metal Game" template which is a rotating textured cube (a trivial GPU workload). Why is that? How can I investigate further? Does CoreAnimation have access to a compositor fast-path that a Metal app cannot access? Maybe another data point: when I do the same circles animation using SwiftUI's Canvas, the energy use is "Very High" and GPU utilization is also quite high. Eventually the phone's thermal state goes "Serious" and I get a message on the device that "Charging will resume when iPhone returns to normal temperature".

Hello, This exact question was already asked in this forum (8 years ago) but I can't find a definitive answer: Does Metal allow using the same color texture as both an input and output (color attachment) of a fragment shader? Is the behavior defined somewhere? I believe this results in undefined behavior under both DirectX and OpenGL, so I'd assume the same for Metal, but then why doesn't Metal warn me about this as it does on some many other "misconfigurations"? It also seems to work correctly in my case, as I found out by accident. Would love to get a clarification! Thanks ahead!

I feel like I'm missing something really simple. I've got the simplest possible CIKernel, it looks like this: extern "C" float4 Simple(coreimage::sampler s) { float2 current = s.coord(); float2 anotherCoord = float2(current.x + 1.0, current.y); float4 sample = s.sample(anotherCoord); // s.sample(current) works fine return sample; } It's (in my mind) incrementing the x position of the sampler by 1 and sampling the neighboring pixel. What I get in practice is a bunch of banded garbage (pictured below.) The sampler seems to be pretty much undocumented, so I have no idea whether I'm incrementing by the right amount to advance one pixel. The weird banding is still present if I clamp anootherCoord to s.extent() but it behaves normally if I sample s.coord() unchanged. I'm trying to write a box blur that samples / averages neighboring pixels and am completely blocked by this. What am I missing?

I'm writing a Vision Pro app that's fully immersive and rendered using Metal. Occasionally, some users of this app would benefit from being able to use a physical keyboard (or other accessory like a game controller). It seems very straightforward to capture and handle spatial gesture events, but I cannot find an interface that allows the detection, capture, or handling of keyboard events in any of the objects associated with fully immersive metal rendering: CompositorServices, LayerRenderer, and its associated .frame, .drawable, and .drawable.view don't seem to have any accessory awareness. Can you help me handle a keyboard event?

Hi everyone, This happens with Xcode 15.3 (15E204a) and visionOS 1.1.2 (21O231). To reproduce this issue, simply create a new VisionOS app with Metal (see below). Then simply change the following piece of code in Renderer.swift: func renderFrame() { [...] // Set the clear color red channel to 1.0 instead of 0.0. renderPassDescriptor.colorAttachments[0].clearColor = MTLClearColor(red: 1.0, green: 0.0, blue: 0.0, alpha: 0.0) [...] } On the simulator it works as expected while on device it will show a black background with red jagged edges (see below).

I use ScreenCaptureKit, CoreVideo, CoreImage, CoreMedia frameworks to capture screenshots on macOS 14.0 and higher. Example of creating CGImageRef: CVImageBufferRef cvImageBufferRef = ..; CIImage* temporaryImage = [CIImage imageWithCVPixelBuffer:cvImageBufferRef]; CIContext* temporaryContext = [CIContext context]; CGImageRef imageRef = [temporaryContext createCGImage:temporaryImage fromRect:CGRectMake(0, 0, CVPixelBufferGetWidth(cvImageBufferRef), CVPixelBufferGetHeight(cvImageBufferRef))]; I have the next results of profiling with XCode Instruments Memory Leaks & Allocations: there is constantly increasing memory usage, but no memory leaks are detected, and there are many calls to create IOSurface objects, that have been never released. The most part of memory - All Anonymous VM - VM: IOSurface. The heaviest stack trace: [RPIOSurfaceObject initWithCoder:] [IOSurface initWithMachPort:] IOSurfaceClientLookupFromMachPort I don't have any of IOSurface objects created by myself. There are low-level calls to it. In Allocation List I can see many allocations of IOSurface objects, but there are no info about releasing it. Due to this info, how can I release them to avoid permanent increasing memory consumption?

Hi all, I am trying to measure the performce of my video game in iOS platform. After launching the debugger and caputre GPU workload (with "frame" selected as scope), I can not do performance profiling. The error message is: "Failed to enable shader profiler. (516)". And if I export the GPU trace and reopen it, the Xcode can not find any compatible devices connected. However, the same device used for capturing is connected. Device informations are: Mac device: MacBook Air M2, 2022 Mac OS version: Sonoma 14.4 XCode version: 15.2 (15C500b) mobile device: iPhone 13 Pro Max iOS version: 15.0 GPU performance counter can be performed using the same mobile device in my colleague's Mac. So I think there might be something wrong with my Xcode.

Looking for info on the best way to convert a MTLTexture to SKTexture. I am migrating a project where a SpriteKit scene is rendered in an SKView - and relies heavily on SKView.texture(from: SKNode) API I am trying to migrate to an SKRenderer based SpriteKit scene - and wondering the best way to grab an SKTexture from an SKNode when rendered in a MTKView instead of SKView?

Hi, in this WebGPU example: https://skal65535.github.io/curl/index_bug_safari.html the lighting is wrong compared to Chrome's reference version. I narrowed the problem to the uniform value 'params.specular' at line 515 not being equal to the expected value 1.2f. The value is set a line at line 1078 in the uniform buffer. Platform: MacBook M1 Pro Sonoma 14.4.1 (23E224) Safari Technology Preview: Release 194 (Safari 17.4, WebKit 19619.1.11.111.2) Works ok with Chrome 124.0.6367.156 (Official Build) (arm64).

Hi, Since iOS 17, when setting weight on a SCNMorpher, the normals become completely wrong. As you can see below it only happens when there are vertices along an edge. Has anyone encountered that problem and found a solution? Thanks Reported: FB13798652

Hi! I was trying to port our sdk for visionOS. I was going through the documentation and saw this video: https://developer.apple.com/videos/play/wwdc2023/10089/ Is there any working code sample for it, same goes for arkit c api ? Couldn't find any links. Thanks in advance. Sahil

with the latest Xcode that runs with Mac OS 14.5 Developer Beta has messages with a time and date in them There are also some other fields of an indeterminate origin/type. "2024-05-06 15:37:32.383996-0500 RoomPlanExampleApp[24190:1708576] [CAMetalLayerDrawable texture] should not be called after already presenting this drawable. Get a nextDrawable instead." specifically I need to know how the string [24190:1708576] relates to a location in my application so I can act on the message. I certainly can't find the text in the "[CAMetalLayerDrawable texture]". field anywhere in the user documentation OR the Development documentation. In order for a diagnostic message to be Actionable and remedied by a user it must identify the module and source line of the initiating code and there must be accessible documentation for users to access to get an explanation of potential remedies.. This interface fails to supply enough information to diagnose the problem. The label in [CAMetalLayerDrawable texture] cannot even be found in a search of the package information attached to the Xcode Release paired with the IOS and Mac OS system releases.

I am wanting to create a 3D video game in Xcode for macOS, iOS, iPadOS, tvOS, and visionOS. I have heard that there are a few different ways to go about this such as MetalKit or SceneKit. These libraries seem to have little examples and documentation so I am wondering: Are they still be developed/supported? Which platform should I make a game in? Where are some resources to learn how to use these platforms? Are there other better platforms that I am just not aware of? Thanks!

What best to pass to the options parameter of: MTLDevice.makeBuffer(length:options:) MTLDevice.makeBuffer(bytes:length:options:) MTLDevice.makeBuffer(bytesNoCopy:length:options:deallocator:) Basically I'm looking for a "plain English" explanation of MTLResourceOptions doc page.

Hello. I am trying to enable Metal to take advantage of SwiftUI+Shaders in an existing app. In my target, I have added metal.framework, added a .metal file and confirmed I see the Metal Compiler option in my Build Settings. When I add a colorEffect modifier using the ShaderLibrary, the object renders black. I confirmed this is working in a new project so I suspect there are more steps required to get this working in an existing application.

I have a glb model that is loading absolutely fine, repeatedly, in safari or chrome. There is only one texture that is 8192x8192 it never has a problem when loading in browser. when we embed the url into an app, the model loads the first few times (exiting the model and going back to the main menu and then reloading the model) but, after a few attempts, the texture fails to load. The model and all data is visible but the texture, itself, is black. why could this be happening? Is there something in the iOS code that is breaking it? Is the iOS code trying to automatically cache the texture and it’s running out of memory? anyone who can provide the help and support that we require will be much appreciated. thank you advance.

Hi, just got an Apple M3 Pro to try it out on some Jax operations. I see the development is actively ongoing so maybe this error can help. This is the environment: Metal device set to: Apple M3 Pro systemMemory: 18.00 GB maxCacheSize: 6.00 GB jax: 0.4.26 jaxlib: 0.4.23 numpy: 1.26.4 python: 3.11.8 | packaged by conda-forge | (main, Feb 16 2024, 20:49:36) [Clang 16.0.6 ] jax.devices (1 total, 1 local): [METAL(id=0)] process_count: 1 platform: uname_result(system='Darwin', node='MKFL96VR9YT', release='23.4.0', version='Darwin Kernel Version 23.4.0: Wed Feb 21 21:44:54 PST 2024; root:xnu-10063.101.15~2/RELEASE_ARM64_T6030', machine='arm64') This is a minimal example which produces an error, I think due to the fft part: from jax import numpy as np array = np.ones((16, 16)) np.fft.fft2(array) This is the full traceback: Traceback (most recent call last): File "/Users/user/Downloads/wow.py", line 5, in &lt;module&gt; np.fft.fft2(array) File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/numpy/fft.py", line 216, in fft2 return _fft_core_2d('fft2', xla_client.FftType.FFT, a, s=s, axes=axes, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/numpy/fft.py", line 210, in _fft_core_2d return _fft_core(func_name, fft_type, a, s, axes, norm) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/numpy/fft.py", line 102, in _fft_core transformed = lax.fft(arr, fft_type, tuple(s)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/traceback_util.py", line 179, in reraise_with_filtered_traceback return fun(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/pjit.py", line 298, in cache_miss outs, out_flat, out_tree, args_flat, jaxpr, attrs_tracked = _python_pjit_helper( ^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/pjit.py", line 176, in _python_pjit_helper out_flat = pjit_p.bind(*args_flat, **params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/core.py", line 2788, in bind return self.bind_with_trace(top_trace, args, params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/core.py", line 425, in bind_with_trace out = trace.process_primitive(self, map(trace.full_raise, args), params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/core.py", line 913, in process_primitive return primitive.impl(*tracers, **params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/pjit.py", line 1494, in _pjit_call_impl return xc._xla.pjit(name, f, call_impl_cache_miss, [], [], donated_argnums, # type: ignore ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/pjit.py", line 1471, in call_impl_cache_miss out_flat, compiled = _pjit_call_impl_python( ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/pjit.py", line 1406, in _pjit_call_impl_python lowering_parameters=mlir.LoweringParameters()).compile() ^^^^^^^^^ File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/interpreters/pxla.py", line 2369, in compile executable = UnloadedMeshExecutable.from_hlo( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/interpreters/pxla.py", line 2908, in from_hlo xla_executable, compile_options = _cached_compilation( ^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/interpreters/pxla.py", line 2718, in _cached_compilation xla_executable = compiler.compile_or_get_cached( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/compiler.py", line 266, in compile_or_get_cached return backend_compile(backend, computation, compile_options, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/profiler.py", line 335, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/compiler.py", line 238, in backend_compile return backend.compile(built_c, compile_options=options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ jaxlib.xla_extension.XlaRuntimeError: UNKNOWN: &lt;unknown&gt;:0: error: 'func.func' op One or more function input/output data types are not supported. &lt;unknown&gt;:0: note: see current operation: "func.func"() &lt;{arg_attrs = [{mhlo.layout_mode = "default", mhlo.sharding = "{replicated}"}], function_type = (tensor&lt;16x16xf32&gt;) -&gt; tensor&lt;16x16xcomplex&lt;f32&gt;&gt;, res_attrs = [{jax.result_info = "", mhlo.layout_mode = "default"}], sym_name = "main", sym_visibility = "public"}&gt; ({ ^bb0(%arg0: tensor&lt;16x16xf32&gt;): %0 = "mhlo.convert"(%arg0) : (tensor&lt;16x16xf32&gt;) -&gt; tensor&lt;16x16xcomplex&lt;f32&gt;&gt; %1 = "mhlo.fft"(%0) {fft_length = dense&lt;16&gt; : tensor&lt;2xi64&gt;, fft_type = #mhlo&lt;fft_type FFT&gt;} : (tensor&lt;16x16xcomplex&lt;f32&gt;&gt;) -&gt; tensor&lt;16x16xcomplex&lt;f32&gt;&gt; "func.return"(%1) : (tensor&lt;16x16xcomplex&lt;f32&gt;&gt;) -&gt; () }) : () -&gt; () &lt;unknown&gt;:0: error: failed to legalize operation 'func.func' &lt;unknown&gt;:0: note: see current operation: "func.func"() &lt;{arg_attrs = [{mhlo.layout_mode = "default", mhlo.sharding = "{replicated}"}], function_type = (tensor&lt;16x16xf32&gt;) -&gt; tensor&lt;16x16xcomplex&lt;f32&gt;&gt;, res_attrs = [{jax.result_info = "", mhlo.layout_mode = "default"}], sym_name = "main", sym_visibility = "public"}&gt; ({ ^bb0(%arg0: tensor&lt;16x16xf32&gt;): %0 = "mhlo.convert"(%arg0) : (tensor&lt;16x16xf32&gt;) -&gt; tensor&lt;16x16xcomplex&lt;f32&gt;&gt; %1 = "mhlo.fft"(%0) {fft_length = dense&lt;16&gt; : tensor&lt;2xi64&gt;, fft_type = #mhlo&lt;fft_type FFT&gt;} : (tensor&lt;16x16xcomplex&lt;f32&gt;&gt;) -&gt; tensor&lt;16x16xcomplex&lt;f32&gt;&gt; "func.return"(%1) : (tensor&lt;16x16xcomplex&lt;f32&gt;&gt;) -&gt; () }) : () -&gt; () I'd be happy running more tests should you need them, I'm new to this, so not sure which just yet. Many thanks!!

Hello! I run into, what seem to be compiler issue. The shader source given to Metal is: https://shader-playground.timjones.io/1bcf3ffbb313878ccd594ddbb27b746e This shader is generated by spirv-cross, from GLSL source, so for readability here is original source: https://github.com/Try/OpenGothic/blob/master/shader/hiz/hiz_mip.comp (shader variant uses SSBO counter, not atomic-image) Here is relevant path of application log: 2024-04-21 16:27:13.621218+0200 Gothic2Notr[23992:2003969] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED 2024-04-21 16:27:13.656559+0200 Gothic2Notr[23992:2003969] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED 2024-04-21 16:27:13.701323+0200 Gothic2Notr[23992:2003969] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED 2024-04-21 16:27:13.701477+0200 Gothic2Notr[23992:2003969] MTLCompiler: Compilation failed with XPC_ERROR_CONNECTION_INTERRUPTED on 3 try 2024-04-21 16:27:13.701817+0200 Gothic2Notr[23992:2003969] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED iOS version: 15.8.2 MTL::CompileOptions::languageVersion: 2.4 (also tested other version - same result) Offended part of shader: void store(int mip, ivec2 uv, float z) { // NOTE: replacing this function to NOP, avoid the crash // NOTE2: this switch-case is crude emulation of bindless storage-image switch(mip) { case 1: imageStore(mip1, uv, vec4(z)); break; case 2: imageStore(mip2, uv, vec4(z)); break; case 3: imageStore(mip3, uv, vec4(z)); break; case 4: imageStore(mip4, uv, vec4(z)); break; case 5: imageStore(mip5, uv, vec4(z)); break; case 6: imageStore(mip6, uv, vec4(z)); break; case 7: imageStore(mip7, uv, vec4(z)); break; case 8: imageStore(mip8, uv, vec4(z)); break; } } Some extra info: The shader is simplified single-pass mip-map generator. The same shader is know to work on mac M1 laptop without any issues Please have a look and looking forward for driver-fix. Thanks!