Hello guys. With the release of the M1 Pro and M1 Max in particular, the Mac has become a platform that could become very interesting for games in the future. However, since some features are still missing in Metal, it could be problematic for some developers to port their games to Metal. Especially with the Unreal Engine 5 you can already see a tendency in this direction, since e.g. Nanite and Lumen are unfortunately not available on the Mac. As a Vulkan developer I wanted to inquire about some features that are not yet available in Metal at the moment. These features are very interesting if you want to write a GPU driven renderer for modern game engines. Furthermore, these features could be used to emulate D3D12 on the Mac via MoltenVK, which would result in more games being available on the Mac. Buffer device address: This feature allows the application to query a 64-bit buffer device address value for a buffer. It is very useful for D3D12 emulation and for compatibility with Vulkan, e.g. to implement ray tracing on MoltenVK. DrawIndirectCount: This feature allows an application to source the number of draws for indirect drawing calls from a buffer. Also very useful in many gpu driven situations Only 500000 resources per argument buffer Metal has a limit of 500000 resources per argument buffer. To be equivalent to D3D12 Resource Binding Tear 2, you would need 1 million. This is also very important as so many DirectX12 game engines could be ported to Metal more easily. Mesh shader / Task shader: Two interesting new shader stages to optimize the rendering pipeline Are there any plans to implement this features in future? Is there a roadmap for metal? Is there a website where I can suggest features to the metal developers? I hope to see at least the first 3 features in metal in the future and I think that many developers feel the same way. Best regards, Marlon

I am trying to develop tone curves filter using Metal or Core Image as I find CIToneCurve filter is having limitations (number of points are atmost 5, spline curve it is using is not documented, and sometimes output is a black image even with 4 points). Moreover it's not straightforward to have separate R,G,B curves independently. I decided to explore other libraries that implement tone curve and the only one that I know is GPUImage (few others borrow code from the same library). But the source code is too cryptic to understand and I have [doubts] about the manner in which it is generating look up texture (https://stackoverflow.com/questions/70516363/gpuimage-tone-curve-rgbcomposite-filter). Can someone explain how to correctly implement R,G,B, and RGB composite curves filter like in Mac Photos App?

Hello everyone! how to put a custom shader in depth only on the floor? I'm trying to use the depth of the scene to put the shader exclusively on the floor, but apparently I'm doing something wrong Links https://www.dropbox.com/s/4ghun92frlcg7hz/IMG_9960.PNG?dl=0 https://www.dropbox.com/home?preview=-2362988581602429186.MP4 PostProcess.metal float linearizeDepth(float sampleDepth, float4x4 viewMatrix) {     constexpr float kDepthEpsilon = 1e-5f;     float d = max(kDepthEpsilon, sampleDepth);     d = abs(-viewMatrix[3][2] / d);     return d; } constexpr sampler textureSampler(address::clamp_to_edge, filter::bicubic); float getDepth(float2 coords, constant InputArgs *args, texture2d<float, access::sample> inDepth,depth2d<float, access::sample> arDepth) {     float2 arDepthCoords = args->orientationTransform * coords + args->orientationOffset;     float realDepth = arDepth.sample(textureSampler, arDepthCoords);     float virtualDepth = linearizeDepth(inDepth.sample(textureSampler, coords)[0], args->viewMatrix);     bool realFragment = (virtualDepth <= FLT_EPSILON);     if (realFragment) { virtualDepth = realDepth; }     return min(virtualDepth, realDepth); } float3 getDirection(float2 screenCoord, constant InputArgs *args) {     float3 top = mix(args->topLeft.xyz, args->topRight.xyz, screenCoord.x);     float3 bottom = mix(args->bottomLeft.xyz, args->bottomRight.xyz, screenCoord.x);     return normalize(mix(bottom, top, screenCoord.y)); } float3 worldCoordsForDepth(float depth, float2 screenCords, constant InputArgs *args) {     float3 centerDirection = getDirection(float2(0.5, 0.5), args);     float3 direction = getDirection(screenCords, args);     float depth2 = depth / dot(direction, centerDirection);     return direction * depth2 + args->viewTranslation.xyz; } [[kernel]] void postProcess(uint2 gid [[thread_position_in_grid]],                  texture2d<half, access::read> inputTexture [[texture(0)]],                  texture2d<float, access::sample> inDepth [[texture(1)]],                  texture2d<half, access::write> outputTexture [[texture(2)]],                  depth2d<float, access::sample> arDepth [[texture(3)]],                  constant InputArgs *args [[buffer(0)]]) {  float2 screenCoords = float2(float(gid[0]) / float(outputTexture.get_width()),                                  float(gid[1]) / float(outputTexture.get_height()));     float rawDepth = getDepth(screenCoords, args, inDepth, arDepth);     float3 worldCoords = worldCoordsForDepth(rawDepth, screenCoords, args);     float depth = rawDepth;     depth = 1 - pow(1 / (pow(depth, args->intensity) + 1), args->falloff);     depth = clamp(depth, 0.0, 1.0);     half4 nearColor = inputTexture.read(gid);     float blend = pow(1 - depth, args->exponent);     half4 color = half4(0.0);  float2 frag = worldCoords.xz;         frag *= 1.0 - 0.2 * cos (frag) * sin (3.14159 * 0.5 * inDepth.sample(textureSampler, float2(0.0)).x);         frag *= 5.0;         float random = rand (floor(frag));         float2 black = smoothstep(1.0, 0.8, cos(frag * 3.14159 * 2.0));         float3 finalColor = hsv2rgb (float3 (random, 1.0, 1.0));         finalColor *= black.x * black.y * smoothstep (1.0, 0.0, length (fract (frag) - 0.5));         finalColor *= 0.5 + 0.5 * cos (random + random * args->time + args->time + 3.14159 * 0.5 * inDepth.sample(textureSampler, float2(0.7)).x); color = blend * nearColor + (1.0 - blend) * half4(half3(finalColor), 1.0); } I really hope for help of understanding this one

I've created a custom BoxBlur kernel that produces identical results to Apple's built-in box blur (CIBoxBlur) kernel but my custom kernel is orders of magnitude slower. So naturally I am wondering what I'm doing wrong to get such poor performance. Below is my custom kernel in the Metal shading language. Can you spot why it's so slow? The built-in filter performs well so I can only assume it's something I'm doing wrong. #include <CoreImage/CoreImage.h> #import <simd/simd.h> extern "C" { namespace coreimage { float4 customBoxBlurFilterKernel(sampler src) { float2 crd = src.coord(); int edge = 100; int minx = crd.x - edge; int maxx = crd.x + edge; int miny = crd.y - edge; int maxy = crd.y + edge; float4 sums = float4(0,0,0,0); float cnt = 0; // compute average of surrounding rgb values for(int row=miny; row < maxy; row++) { for(int col=minx; col < maxx; col++) { float4 samp = src.sample(float2(col, row)); sums[0] += samp[0]; sums[1] += samp[1]; sums[2] += samp[2]; cnt += 1.; } } return float4(sums[0]/cnt, sums[1]/cnt, sums[2]/cnt, 1); } } }

So I wanted to render video from .mov file in MTKView. Short algorithm: Read CMSampleBuffer from .mov using AVAssetReader and AVAssetReaderTrackOutput. Convert CMSampleBuffer from step 1 to MTLTexture and pass it to renderer I did those 2 steps and got the picture with twitches and I don’t know why. Link to .mov https://www.dropbox.com/s/gmzxd8j94pjhc1q/2.MOV?dl=0. Link to result https://www.dropbox.com/s/exgf1tk7oqvon25/result.mov?dl=0. The code. ViewController.swift VideoReader.swift SampleConverter.swift TextureModel.swift Renderer.swift MyMetal.metal

Hey everyone :-) I'm trying access the information of my ARFrame so I can colorize different parts of my scene depending of their classifications. In other words, I'm looking for a way to transform XYZ World Coordinates or XY Screen Coordinates to what is being classified by my AR Session in that position. All the floor in the scene being colorize in red, the tables in blue, the walls in green, etc. How could I embed in real-time these classifications from my reconstructed ARMeshAnchor into my coordinates so I can access it on my Metal Script in real-time? As of today... I've been able to get the XYZ World Coordinate related to a XY Screen Coordinate reading the following example: Building an Immersive Experience with RealityKit Here is a scene painting it with different shades of purple depending on the Z world-coordinate: P.S. I want to be able to do it within RealityKit. Thanks in advance.

I’m trying to acquire a point cloud and then capture an image (the capture would stop the point cloud acquisition session). The point cloud acquisition seems easy. This is a well discussed topic: https://developer.apple.com/forums/thread/658109 Furthermore, it would seem possible to store camera pose information from the point cloud with the photo so that data could be used to find a pose. Later, I’d align the 3D point cloud to that 2d image (preferably not using depth images, interpolation, or NNs). I am surprised, but seems really hard to find out the feasibility of such an approach.

I downloaded this sample: https://developer.apple.com/documentation/metal/basic_tasks_and_concepts/using_metal_to_draw_a_view_s_contents?preferredLanguage=occ I commented out this line in AAPLViewController.mm //    _view.enableSetNeedsDisplay = YES; I modified the presentDrawable line in AAPLRenderer.mm to add afterMinimumDuration:     [commandBuffer presentDrawable:drawable afterMinimumDuration:1.0/60]; I then added a presentedHandler before the above line that records the time between successive presents. Most of the time it correctly reports 0.166667s. However, about every dozen or so frames (it varies) it seems to present a frame early with an internal of 0.0083333333s followed by the next frame after around 0.24s. Is this expected behaviour, I was hoping that afterMinimumDuration would specifically make things consistent. Why would it present a frame early? This is on a new MacBook Pro 16 running latest macOS Monterrey, and the sample project upgraded to have a minimum deployment target of 11.0. Xcode latest public release 13.1.

I am trying to use a CIColorKernel or CIBlendKernel with sampler arguments but the program crashes. Here is my shader code which compiles successfully. extern "C" float4 wipeLinear(coreimage::sampler t1, coreimage::sampler t2, float time) { float2 coord1 = t1.coord(); float2 coord2 = t2.coord(); float4 innerRect = t2.extent(); float minX = innerRect.x + time*innerRect.z; float minY = innerRect.y + time*innerRect.w; float cropWidth = (1 - time) * innerRect.w; float cropHeight = (1 - time) * innerRect.z; float4 s1 = t1.sample(coord1); float4 s2 = t2.sample(coord2); if ( coord1.x > minX && coord1.x < minX + cropWidth && coord1.y > minY && coord1.y <= minY + cropHeight) { return s1; } else { return s2; } } And it crashes on initialization. class CIWipeRenderer: CIFilter { var backgroundImage:CIImage? var foregroundImage:CIImage? var inputTime: Float = 0.0 static var kernel:CIColorKernel = { () -> CIColorKernel in let url = Bundle.main.url(forResource: "AppCIKernels", withExtension: "ci.metallib")! let data = try! Data(contentsOf: url) return try! CIColorKernel(functionName: "wipeLinear", fromMetalLibraryData: data) //Crashes here!!!! }() override var outputImage: CIImage? { guard let backgroundImage = backgroundImage else { return nil } guard let foregroundImage = foregroundImage else { return nil } return CIWipeRenderer.kernel.apply(extent: backgroundImage.extent, arguments: [backgroundImage, foregroundImage, inputTime]) } } It crashes in the try line with the following error: Fatal error: 'try!' expression unexpectedly raised an error: Foundation._GenericObjCError.nilError If I replace the kernel code with the following, it works like a charm: extern "C" float4 wipeLinear(coreimage::sample_t s1, coreimage::sample_t s2, float time) { return mix(s1, s2, time); }

I am working on rendering multiple objects in Metalkit. While doing it I can see objects are flickering if they are overlaying each-other. I'm passing the currentRenderPassDescriptor and current drawable in draw method which is calling every time. Please refer screenshot. Please help me out to render objects without flickering.

For running my app on devices with P3 screens, to take advantage of the P3 colors, which MTLPixelFormat is the proper one to use for my MTKView? My app is for iOS, but it’s nice to be able to debug on the simulator or as a Catalyst app. Right now, I’ve been using MTLPixelFormatBGR10_XR_sRGB and MTLPixelFormatBGRA10_XR_sRGB per the recommendation in WWDC 2016 session 605 for iOS support of P3 color. This appears to work fine on iOS devices, but on my Apple silicon Mac, when running as a Catalyst app, the colors sometimes look lighter than they should, but it doesn’t always happen. When I run in the simulator on this Mac, my app crashes. When I was using an Intel Mac (with an sRGB screen), I was having my app render in sRGB-only with MTLPixelFormat.bgra8Unorm. MTLPixelFormat.bgra8Unorm also works fine on my Apple silicon Mac, but of course, I’m not getting the P3 color space.  For Apple Silicon Macs, as the GPU hardware is more like an iOS device than like an Intel Mac, should the same pixel formats work on an Apple silicon Mac as on iOS devices? Or are the pixel formats that work with Apple Silicon Macs the same as those on Intel Macs?

Friendly greetings ! I'm on a MacBook Air M1, using Xcode 13.1, Swift. The full source code is below, as a MaOS Console Application import Foundation import Metal import MetalKit import simd guard let device : MTLDevice = MTLCreateSystemDefaultDevice() else { fatalError("could not create metal default device") } let queue = device.makeCommandQueue() print("Hello, World!") The full output, running in debug from XCode is : 2021-10-29 06:00:16.567089+0200 FraCompute[71702:11572598] Metal API Validation Enabled 2021-10-29 06:00:16.621413+0200 FraCompute[71702:11572598] flock failed to lock list file (/var/folders/s7/7m1rt8kx3jq7c02mwclmnvf80000gn/C//com.apple.metal/16777235_322/functions.list): errno = 35 2021-10-29 06:00:16.628053+0200 FraCompute[71702:11572598] +[MTLIOAccelDevice registerDevices]: Zero Metal services found Hello, World! Program ended with exit code: 0 Running the sample "MacOS Game" default code (the one with the rotating cube) also give this "Zero Metal services found". What's a "metal service" anyway ? Googling for it obviously give me unrelated smithing service offer =^_^= It's not even "Metal Service not found" but, "I found Zero Metal Services", whatever that's supposed to mean ? is it "I didn't find any metal service" or is there a thing called "Zero Metal" ? I can query the metal device name and it get me "Apple M1". No problem, everything seems to work fine (so far). But what's the meaning of this message please ?

I'm using Xcode 13 after recently updating to MacOS Monterey, and only after updating am I getting this error: [MTLDebugCommandBuffer lockPurgeableObjects]:2103: failed assertion `MTLResource 0x14a8a8cc0 (label: null), referenced in cmd buffer 0x149091400 (label: null) is in volatile or empty purgeable state at commit' I haven't changed my code at all between updating to the latest OS, and it worked perfectly before. How can I fix this? I don't think there should be any reason that I can't use a command buffer on a texture resource with a volatile/empty purgeable state.

Hi, I have been getting the following crash in my app which has a Custom Metal Render Engine CoreFoundation                   -[__NSSetM clumpingFactor] + 264 libobjc.A.dylib                  __objc_empty_cache + 888 CoreAutoLayout                   DA979160-E330-3C35-BF6F-D3248DCC3246 + 67536 CoreAutoLayout                   DA979160-E330-3C35-BF6F-D3248DCC3246 + 68272 UIKitCore                        __OBJC_$_INSTANCE_METHODS__UIDatePickerCalendarTimeLabel + 600 UIKitCore                        __OBJC_$_INSTANCE_METHODS__UINavigationBarStarkVisualStyle + 100 UIKitCore                        ___79+[UISwitchModernVisualElement _modernThumbImageWithColor:mask:traitCollection:]_block_invoke_2 + 204 UIKitCore                        -[UISwitchModernVisualElement _switchTrackPositionAnimationWithFromValue:toValue:on:] + 388 UIKitCore                        -[UISwitchModernVisualElement _effectiveGradientImage] + 128 UIKitCore                        __OBJC_$_INSTANCE_METHODS__UISearchBarVisualProviderLegacy + 1924 QuartzCore                       CA::Layer::add_animation(CAAnimation*, __CFString const*) + 72 QuartzCore                       CA::Layer::remove_sublayer(CA::Transaction*, CALayer*) + 272 QuartzCore                       CA::OGL::Context::draw_elements(CA::OGL::PrimitiveMode, unsigned int, unsigned short const*, CA::OGL::Vertex const*, unsigned int, unsigned int, CA::OGL::ClipPlane const*) + 60 QuartzCore                       CAML::cgcolor_end(CAML::Context*, CAML::State*, char*, unsigned long) + 1252 QuartzCore                       native_window_swap(_EAGLNativeWindowObject*, unsigned int, double) + 712 QuartzCore                       -[CAStateControllerAnimation initWithLayer:key:] + 52 CoreFoundation                   ___CFSocketSetSocketReadBufferAttrs + 444 CoreFoundation                   __CFNonObjCEqual + 8 CoreFoundation                   __CFRelease + 952 Foundation                       4E7D1FF6-6B64-3833-9E60-CC662AFE2647 + 36236 danmu                            -[DMEngineBase runMetalThread] (in DMEngineBase.mm:148) Foundation                       4E7D1FF6-6B64-3833-9E60-CC662AFE2647 + 1549068 libsystem_pthread.dylib          _pthread_rwlock_unlock$VARIANT$armv81 + 160 libsystem_pthread.dylib          __pthread_create + 1196 this crash seems caused by the Metal Thread Rendering which triggered a CoreAnimation drawing and finally crashed at CoreAutoLayout internal method : Crashed View : appEnterForeground Exception Name : NSInternalInconsistencyException Exception Reason : Modifications to the layout engine must not be performed from a background thread after it has been accessed from the main thread. In the latest Version I have been trying solved this crash by add some protected like this : - (void)runMetalThread {   NSRunLoop *runLoop = [NSRunLoop currentRunLoop];   [_displayLink addToRunLoop:runLoop forMode:DMMetalRunLoopModelFrame];   BOOL continueRunLoop = YES;   while (continueRunLoop)   {     @autoreleasepool     {       [runLoop runMode:DMMetalRunLoopModelFrame beforeDate:[NSDate distantFuture]];     }     continueRunLoop = _continueRunLoop;   } } - (void)onBulletDraw:(CADisplayLink*)displayLink {   self.renderer.stoped = !_isActive || _stoped;   self.renderer.paused = _isPaused;   [self.renderer onDanmuDraw:displayLink]; } #pragma mark - Notification - (void)willResignActive {   self.isActive = NO;   MTLog(@"metal# engine %@ resign active", self); } - (void)didBecomeActive {   //Protected by delay active the metal engine after app state didBecomeActive   dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)(0.7 * NSEC_PER_SEC)), dispatch_get_main_queue(), ^{     if (UIApplication.sharedApplication.applicationState == UIApplicationStateActive) {       self.isActive = YES;       MTLog(@"metal# engine %@ become active", self);     }else{       MTLog(@"metal# engine still inactive after 0.7s");     }   }); } this protected has make sure to STOPed the Metal Render after the app resign active, and seems has some effect which reduced 84% crashes on my released app . but it did not completely solve this problem still has 16% crashes in some unknown scene . I have been checked all my Metal Rendering thread code , I can be guaranteed that no method will trigger CoreAnimation drawing in this metal thread. So is there has an Perfective Solution of this bug ?

Hello all, I need my texture to be not premultiplied alpha, because is use the alpha for some additional calculations in the fragment shader. At the moment I load my texture like this let textureLoader = MTKTextureLoader(device: sceneView.device!) myTexture = try textureLoader.newTexture(URL: URL(fileURLWithPath: path!), options: [MTKTextureLoader.Option.SRGB: false]) because I need the raw RGBA values. But of course it gets premultiplied. How can I turn off this behavior?