I was wondering if anyone had any insight into why MTKTextureLoader might very rarely return a texture which is just fully opaque magenta (each pixel is #ff00ff). What I know: I'm using MTKTextureLoader.newTexture(URL:options:) to synchronously load the texture within a standard, synchronous dispatch queue (though the single texture loader itself is created on the main thread). No error is thrown in the above call and nothing is printed to the console, and a texture is returned. The URL of the texture resides on the local filesystem and points to a fairly unremarkable 512x512 JPEG. The resulting texture returned by the loader is the correct resolution (but every pixel is magenta). The majority of launches of the app load all the textures without any issues (I think at least 90%), but if a texture does fail to load, many others fail to load as well (especially textures which are loaded immediately after a failed one). The exact same files which load incorrectly in one run of the app will load correctly in another run. For completeness, the texture loader options: textureLoaderOptions = [ .allocateMipmaps: false, .generateMipmaps: false, .textureUsage: NSNumber(value: MTLTextureUsage.shaderRead.rawValue), .textureStorageMode: NSNumber(value: MTLStorageMode.`private`.rawValue) ]

Is it possible to render an MTKView in different color spaces? For example, if I want to render it in CMYK, is there a way to adjust the colors on every frame to that colorspace before presenting it on the screen, or is that something that needs to be handled in a shader?

I'm trying to render text with Metal for my (2D) game. I'm using the system fonts, e.g. the SF Pro family for English texts. I render the glyphs onto an atlas texture, and then sample from this texture. My questions: I assume that, for copyright reasons, I'm not allowed to include a pre-rendered font atlas in my app. Is my assumption correct? I can, however, have the app generate the atlas when it's first opened, and then use it within the app, right? If 2. is true, then can the app save the atlas somewhere in the app's private storage, so that it would not need to re-generate the atlas the next time? Thanks!

I have started learning Metal and the source text I was reading mentioned that interactions with images rendered to the screen is handled by Core Animation layers (CAMetalLayer) in iOS, but had no mention for any other platform. I am guessing for the other "mobile" platforms the same thing applies as iOS, but what handles this interaction for macOS, or is it also the same?

Is it possible to pass MTLTexture to Metal Core Image Kernel? How can Metal resources be shared with Core Image?

I working on this project last 1 month I have gone through all the possibilities but have not found the expected output. I have the go through the following article and link: https://ixtli.unam.mx/6280-2/ https://developer.apple.com/documentation/arkit/content_anchors/scanning_and_detecting_3d_objects?language=objc https://github.com/TokyoYoshida/ExampleOfiOSLiDAR https://stackoverflow.com/questions/61063571/arkit-3-5-how-to-export-obj-from-new-ipad-pro-with-lidar/61104855#61104855 I am getting normal 3d objects with gray color, I am expecting objects with texture, if you guys know some way to find the solution then please let me know. Thank You!

Hello Everybody. I'm trying to port graphic code written cg in unity to metal. And, one more thing I don't want to manually implement scene graph, so I gonna use SceneKit. So I should use SCNProgram or SCNNodeRendererDelegate, and I think SCNProgram is more comfort. And real my question is how convert this code, in cg Cull Front ZTest LEqual ZWrite On Blend SrcAlpha OneMinusSrcAlpha I know source alpha blending in MTLPipelineDescriptor, zbuffer in RenderCommandEncoder and Cull Face also. But When I use SCNProgram or SCNSceneRendererNode, can't find these options... how I change these. Help me.

Hello, Everyone. I try to use a metal kit with a scene kit. Because, the scene kits scene graph is great, I want to implement a low-level metal shader. I want to use SCNNodeRenderDelegate, without SCNProgram. Because I want low-level implement for example passing custom extra MTLBuffer, or multi-pass-rendering. So I pass model view projection matrix like that, in metal shader struct NodeBuffer {   float4x4 modelTransform;   float4x4 modelViewProjectionTransform;   float4x4 modelViewTransform;   float4x4 normalTransform;   float2x3 boundingBox; }; in Swift code struct NodeMatrix: sizeable {     var modelTransform = float4x4()     var modelViewProjectionTransform = float4x4()     var modelViewTransform = float4x4()     var normalTransform = float4x4()     var boundingBox = float2x3()   } ...    private func updateNodeMatrix(_ camNode: SCNNode) {     guard let camera = camNode.camera else {       return     }           let modelMatrix = transform     let viewMatrix = camNode.transform     let projectionMatrix = camera.projectionTransform           let viewProjection = SCNMatrix4Mult(viewMatrix, projectionMatrix)     let modelViewProjection = SCNMatrix4Mult(modelMatrix, viewProjection)     nodeMatrix.modelViewProjectionTransform = float4x4(modelViewProjectionMatrix)   } ... public func renderNode(_ node: SCNNode,               renderer: SCNRenderer,               arguments: [String: Any])   {     guard let renderTexturePipelineState = renderTexturePipelineState,        let renderCommandEncoder = renderer.currentRenderCommandEncoder,        let camNode = renderer.pointOfView,        let texture = texture     else { return }           updateNodeMatrix(camNode)     guard let nodeBuffer       = renderer.device?.makeBuffer(bytes: &nodeMatrix,                      length: NodeMatrix.stride,                      options: [])     else { return }           renderCommandEncoder.setDepthStencilState(depthState)     renderCommandEncoder.setRenderPipelineState(renderTexturePipelineState)     renderCommandEncoder.setFragmentTexture(texture, index: 0)     renderCommandEncoder.setVertexBuffer(vertexBuffer, offset: 0, index: 0)     renderCommandEncoder.setVertexBuffer(nodeBuffer, offset: 0, index: 1)     renderCommandEncoder.drawIndexedPrimitives(type: .triangle,                           indexCount: indexCount,                           indexType: .uint16,                           indexBuffer: indexBuffer,                           indexBufferOffset: 0)   } But I got the wrong model view projection matrix in the shader. I think scene kit has modify intermediate transform hiding. I can't know, help me...

I'm using Xcode 13 after recently updating to MacOS Monterey, and only after updating am I getting this error: [MTLDebugCommandBuffer lockPurgeableObjects]:2103: failed assertion `MTLResource 0x14a8a8cc0 (label: null), referenced in cmd buffer 0x149091400 (label: null) is in volatile or empty purgeable state at commit' I haven't changed my code at all between updating to the latest OS, and it worked perfectly before. How can I fix this? I don't think there should be any reason that I can't use a command buffer on a texture resource with a volatile/empty purgeable state.

I am getting this error while running the app with Xcode. especially when I am moving things in view like swiping with my finger, rotating the screen etc.. I have not been able to reproduce it when running the app like normal user. I am just wondering if Xcode is doing something weird in debugging and is hitting the apps animations.

Hey everyone :-) I'm trying access the information of my ARFrame so I can colorize different parts of my scene depending of their classifications. In other words, I'm looking for a way to transform XYZ World Coordinates or XY Screen Coordinates to what is being classified by my AR Session in that position. All the floor in the scene being colorize in red, the tables in blue, the walls in green, etc. How could I embed in real-time these classifications from my reconstructed ARMeshAnchor into my coordinates so I can access it on my Metal Script in real-time? As of today... I've been able to get the XYZ World Coordinate related to a XY Screen Coordinate reading the following example: Building an Immersive Experience with RealityKit Here is a scene painting it with different shades of purple depending on the Z world-coordinate: P.S. I want to be able to do it within RealityKit. Thanks in advance.

Hello everyone! how to put a custom shader in depth only on the floor? I'm trying to use the depth of the scene to put the shader exclusively on the floor, but apparently I'm doing something wrong Links https://www.dropbox.com/s/4ghun92frlcg7hz/IMG_9960.PNG?dl=0 https://www.dropbox.com/home?preview=-2362988581602429186.MP4 PostProcess.metal float linearizeDepth(float sampleDepth, float4x4 viewMatrix) {     constexpr float kDepthEpsilon = 1e-5f;     float d = max(kDepthEpsilon, sampleDepth);     d = abs(-viewMatrix[3][2] / d);     return d; } constexpr sampler textureSampler(address::clamp_to_edge, filter::bicubic); float getDepth(float2 coords, constant InputArgs *args, texture2d<float, access::sample> inDepth,depth2d<float, access::sample> arDepth) {     float2 arDepthCoords = args->orientationTransform * coords + args->orientationOffset;     float realDepth = arDepth.sample(textureSampler, arDepthCoords);     float virtualDepth = linearizeDepth(inDepth.sample(textureSampler, coords)[0], args->viewMatrix);     bool realFragment = (virtualDepth <= FLT_EPSILON);     if (realFragment) { virtualDepth = realDepth; }     return min(virtualDepth, realDepth); } float3 getDirection(float2 screenCoord, constant InputArgs *args) {     float3 top = mix(args->topLeft.xyz, args->topRight.xyz, screenCoord.x);     float3 bottom = mix(args->bottomLeft.xyz, args->bottomRight.xyz, screenCoord.x);     return normalize(mix(bottom, top, screenCoord.y)); } float3 worldCoordsForDepth(float depth, float2 screenCords, constant InputArgs *args) {     float3 centerDirection = getDirection(float2(0.5, 0.5), args);     float3 direction = getDirection(screenCords, args);     float depth2 = depth / dot(direction, centerDirection);     return direction * depth2 + args->viewTranslation.xyz; } [[kernel]] void postProcess(uint2 gid [[thread_position_in_grid]],                  texture2d<half, access::read> inputTexture [[texture(0)]],                  texture2d<float, access::sample> inDepth [[texture(1)]],                  texture2d<half, access::write> outputTexture [[texture(2)]],                  depth2d<float, access::sample> arDepth [[texture(3)]],                  constant InputArgs *args [[buffer(0)]]) {  float2 screenCoords = float2(float(gid[0]) / float(outputTexture.get_width()),                                  float(gid[1]) / float(outputTexture.get_height()));     float rawDepth = getDepth(screenCoords, args, inDepth, arDepth);     float3 worldCoords = worldCoordsForDepth(rawDepth, screenCoords, args);     float depth = rawDepth;     depth = 1 - pow(1 / (pow(depth, args->intensity) + 1), args->falloff);     depth = clamp(depth, 0.0, 1.0);     half4 nearColor = inputTexture.read(gid);     float blend = pow(1 - depth, args->exponent);     half4 color = half4(0.0);  float2 frag = worldCoords.xz;         frag *= 1.0 - 0.2 * cos (frag) * sin (3.14159 * 0.5 * inDepth.sample(textureSampler, float2(0.0)).x);         frag *= 5.0;         float random = rand (floor(frag));         float2 black = smoothstep(1.0, 0.8, cos(frag * 3.14159 * 2.0));         float3 finalColor = hsv2rgb (float3 (random, 1.0, 1.0));         finalColor *= black.x * black.y * smoothstep (1.0, 0.0, length (fract (frag) - 0.5));         finalColor *= 0.5 + 0.5 * cos (random + random * args->time + args->time + 3.14159 * 0.5 * inDepth.sample(textureSampler, float2(0.7)).x); color = blend * nearColor + (1.0 - blend) * half4(half3(finalColor), 1.0); } I really hope for help of understanding this one

There is a write function documented in the CoreImage Metal shader reference here: https://developer.apple.com/metal/MetalCIKLReference6.pdf But I'm not sure how to use it. I assumed one would be able to use it on the destination parameter i.e. dest.write(...) but I get the error, "no member named 'write' in 'coreimage::destination'" How do I use this function?

I've created a custom BoxBlur kernel that produces identical results to Apple's built-in box blur (CIBoxBlur) kernel but my custom kernel is orders of magnitude slower. So naturally I am wondering what I'm doing wrong to get such poor performance. Below is my custom kernel in the Metal shading language. Can you spot why it's so slow? The built-in filter performs well so I can only assume it's something I'm doing wrong. #include <CoreImage/CoreImage.h> #import <simd/simd.h> extern "C" { namespace coreimage { float4 customBoxBlurFilterKernel(sampler src) { float2 crd = src.coord(); int edge = 100; int minx = crd.x - edge; int maxx = crd.x + edge; int miny = crd.y - edge; int maxy = crd.y + edge; float4 sums = float4(0,0,0,0); float cnt = 0; // compute average of surrounding rgb values for(int row=miny; row < maxy; row++) { for(int col=minx; col < maxx; col++) { float4 samp = src.sample(float2(col, row)); sums[0] += samp[0]; sums[1] += samp[1]; sums[2] += samp[2]; cnt += 1.; } } return float4(sums[0]/cnt, sums[1]/cnt, sums[2]/cnt, 1); } } }

The documentation says the pixel format of the environmentTexture in an AREnvironmentProbeAnchor is bgra8Unorm_srgb. https://developer.apple.com/documentation/arkit/arenvironmentprobeanchor/2977511-environmenttexture However, when I inspect the pixelFormat property of the MTLTexture it says it's rgba16Float. I'm trying to read the texture out as a PNG, and because it's a 16-bit float image, I'm assuming its color space is CGColorSpace.displayP3, but I'm not 100% sure. The texture looks darker than what I expected. Could it be that the color space is sRGB, but it's 16-bit because it's actually an HDR texture stored as linear RGB? (Tested on iPhone 12, iOS 15)

I’m trying to acquire a point cloud and then capture an image (the capture would stop the point cloud acquisition session). The point cloud acquisition seems easy. This is a well discussed topic: https://developer.apple.com/forums/thread/658109 Furthermore, it would seem possible to store camera pose information from the point cloud with the photo so that data could be used to find a pose. Later, I’d align the 3D point cloud to that 2d image (preferably not using depth images, interpolation, or NNs). I am surprised, but seems really hard to find out the feasibility of such an approach.

I am working on rendering multiple objects in Metalkit. While doing it I can see objects are flickering if they are overlaying each-other. I'm passing the currentRenderPassDescriptor and current drawable in draw method which is calling every time. Please refer screenshot. Please help me out to render objects without flickering.

For running my app on devices with P3 screens, to take advantage of the P3 colors, which MTLPixelFormat is the proper one to use for my MTKView? My app is for iOS, but it’s nice to be able to debug on the simulator or as a Catalyst app. Right now, I’ve been using MTLPixelFormatBGR10_XR_sRGB and MTLPixelFormatBGRA10_XR_sRGB per the recommendation in WWDC 2016 session 605 for iOS support of P3 color. This appears to work fine on iOS devices, but on my Apple silicon Mac, when running as a Catalyst app, the colors sometimes look lighter than they should, but it doesn’t always happen. When I run in the simulator on this Mac, my app crashes. When I was using an Intel Mac (with an sRGB screen), I was having my app render in sRGB-only with MTLPixelFormat.bgra8Unorm. MTLPixelFormat.bgra8Unorm also works fine on my Apple silicon Mac, but of course, I’m not getting the P3 color space.  For Apple Silicon Macs, as the GPU hardware is more like an iOS device than like an Intel Mac, should the same pixel formats work on an Apple silicon Mac as on iOS devices? Or are the pixel formats that work with Apple Silicon Macs the same as those on Intel Macs?

Hi, i have a SceneKit/ARKit (with my own metal shader) app and I need to change the main texture format from .bgra8unorm_srgb to .bgra8unorm. Since it is srgb, anything I draw is again converted to srgb and shown too bright. Is there a way to change it/how can it be changed? I didn't find anything useful in docs or examples. Thx