I'm trying to write vertex descriptor for my vertex shader which takes following struct as stage_in input. struct VertexIn { float2x2 foo; } vertex float4 vertexShader(const VertexIn in [[stage_in]]) {...} Now when defining vertex descriptor's attribute what should be the MTLVertexFormat? vertexDescriptor.attributes[0].format = ??? I went through the documentation, I didn't find any enum case for Matrices. Is it fine if say set format as float2 and give size 2 * size of float2?

Hi everyone I am trying to take the texture from a previous draw call that renders some objects and copy them to the next draw call that renders different objects. So far I have tried the blit commander way and I tried to manually copy the textures, but both ways seem to completely overwrite the current texture while not taking into account alpha information. I am doing this after my initial command encoder that binds the buffers. I noticed that the previous texture's alpha information is not processed so it overwrites the current layer with the _view.clearColor value. I confirmed that both layers get presented as one layer is shown the majority of the time while the other layer flashes every now and then. Layer 1 completely overwrites layer 0 so all I see if layer 1 instead of the two layers being combined, the green is the view's clearColor and seems to be part of the texture which I think is the problem: Layer 1: Layer 0: The blit commander way, my data is split into layers and if the first layer is getting rendered in the batch then that one does not get copied over since it starts from the top to render the layers in the correct order on top of each other:     if(previousDrawable && currentDrawable && previousTexture != currentDrawable.texture){         [blitCommandEncoder copyFromTexture:previousTexture toTexture:currentDrawable.texture];     }     [blitCommandEncoder endEncoding]; if(currentDrawable && layerId > 0){         self.previousDrawable = currentDrawable;         self.previousTexture = currentDrawable.texture;     }else{         self.previousDrawable = nil;     } The manual texture copying way: if(previousDrawable && currentDrawable && previousTexture != currentDrawable.texture){ int textureSize = currentDrawable.texture.width * currentDrawable.texture.height * 4;         unsigned char* previousTextureData = new unsigned char[textureSize];         unsigned char* currentTextureData = new unsigned char[textureSize];         [previousDrawable.texture getBytes:previousTextureData bytesPerRow:previousDrawable.texture.width * 4 fromRegion:MTLRegionMake2D(0, 0, previousDrawable.texture.width, previousDrawable.texture.height) mipmapLevel:0];         [currentDrawable.texture getBytes:currentTextureData bytesPerRow:currentDrawable.texture.width * 4 fromRegion:MTLRegionMake2D(0, 0, currentDrawable.texture.width, currentDrawable.texture.height) mipmapLevel:0];         for(int i = 0; i < textureSize; i+=4){             if((int)previousTextureData[i + 3] > 0)             {                 currentTextureData[i] = previousTextureData[i];                 currentTextureData[i + 1] = previousTextureData[i + 1];                 currentTextureData[i + 2] = previousTextureData[i + 2];                 currentTextureData[i + 3] = previousTextureData[i + 3];             }         }         [currentDrawable.texture replaceRegion:MTLRegionMake2D(0, 0, currentDrawable.texture.width, currentDrawable.texture.height) mipmapLevel:0 withBytes:currentTextureData bytesPerRow:currentDrawable.texture.width * 4];         delete[] previousTextureData;         delete[] currentTextureData; } if(currentDrawable && layerId > 0){         self.previousDrawable = currentDrawable;         self.previousTexture = currentDrawable.texture;     }else{         self.previousDrawable = nil;     }

Hello Metal programmers, i would like to start programming with metal, but I can not see any place to start from! I searched on YouTube, google, etc.. but I did not see any thing, Please help me to start with "Metal programming". I think this site is the good place to ask. And I would like to know what language that metal use? Is it have its own language ? Or its use swift or obj-c ?

I'm trying to calculate that the rotation becomes smoother when rotating the model, their movement from fast to slow steadily. I read the documentation and found it related to calculating Angular Speed and Angular Acceleration. Has anyone done it before can share me how to do it in metal ? Something like example here: https://threejs.org/examples/?q=rotation#misc_controls_arcball Thanks all.

I want to apply a given CIFilter but instead of the effect showing up instantly, I want to animate it: e.g., a color image desaturating to grey scale over 2 seconds, a blocky image depixellating to a full-resolution image using an EaseInOut animation curve over 0.8 seconds. If you're using one of the built in SwiftUI view modifiers like .blur(), you're golden. Just append some .animate() and you're done. But given that you have to jump through hoops whether you go the UIImage, CGImage, CIImage route, or the MTLView, CIRenderDestination, ContentView example from the WWDC 2022 sample code, I'm a bit confused. Ideally I guess I'd just like to write View Modifiers for each effect I want to do, so that they're as usable as the SwiftUI built-in ones, but I don't know if that's possible. Does anyone have any ideas?

It is possible with SceneKit, but I haven’t found any way for RealityKit.

Good day I am developing XRKit framework, which contains the pipeline logic for rendering using Metal, in manifest it has two targets - framework itself in Swift and XRKitDefenitions in C++ and MSL (since Apple forbids us multilingualism in one package). Both targets have Resources folders open in their manifest. When I try to access the test files hello01.txt (Resources for XRKit) and hello2.txt (Resources for XRKitDefenitions) via Bundle.module, I only see hello01.txt and it doesn't read hello2.txt because it's in a different target. How do I properly organize my code with SPM to access the Resources of XRKitDefenitions target? PS When trying to organize XRKitDefenitions as a remote package on GitHub and defining it as a dependency, situation does not change. I understand now that Bundle.module only refers to its Resources. Is there a way to refer to resources that provided other targets or dependencies in the same package?

I want to crop the usdz model in runtime. I use ModelIO for this. Before: [https://i.stack.imgur.com/yDXXF.jpg) After: [https://i.stack.imgur.com/m9ryg.jpg) First of all, get file from bundle let url = URL(fileURLWithPath: file) } else { print("Object not found in Bundle") } And then I need to access asset let asset = MDLAsset(url: url) What should I do after this step? How am I supposed to use SCNGeometrySource and SCNGeometryElement or MDLVoxelArray classes?

I am trying to develop tone curves filter using Metal or Core Image as I find CIToneCurve filter is having limitations (number of points are atmost 5, spline curve it is using is not documented, and sometimes output is a black image even with 4 points). Moreover it's not straightforward to have separate R,G,B curves independently. I decided to explore other libraries that implement tone curve and the only one that I know is GPUImage (few others borrow code from the same library). But the source code is too cryptic to understand and I have [doubts] about the manner in which it is generating look up texture (https://stackoverflow.com/questions/70516363/gpuimage-tone-curve-rgbcomposite-filter). Can someone explain how to correctly implement R,G,B, and RGB composite curves filter like in Mac Photos App?

I have seen this question come up a few times here on Apple Developer forums (recently noted here - https://developer.apple.com/forums/thread/655505), though I tend to find myself having a misunderstanding of what technology and steps are required to achieve a goal. In general, my colleague and I are try to use Apple's Visualizing a Point Cloud Using Scene Depth - https://developer.apple.com/documentation/arkit/visualizing_a_point_cloud_using_scene_depth sample project from WWDC 2020, and save the rendered point cloud as a 3D model. I've seen this achieved (there are quite a few samples of the final exports available on popular 3D modeling websites), but remain unsure how to do so. From what I can ascertain, using Model I/O seems like an ideal framework choice, by creating an empty MDLAsset and appending a MDLObject for each point to, finally, end up with a model ready for export. How would one go about converting each "point" to a MDLObject to append to the MDLAsset? Or am I going down the wrong path?

I want to remove unnecessary materials or textures in order to reduce the size of the USDZ model I have. How can I manipulate this model with swift? or, I can try any advice to reduce the size of the USDZ model

How can I crop a 3D model as seen in the photos? Should I use MetalKit or can I handle it with sceneKit and modelIO? I couldn't find any code examples on this topic. Can you share the code snippet Before: [https://i.stack.imgur.com/yDXXF.jpg) After: [https://i.stack.imgur.com/m9ryg.jpg)

Hi there, I am trying to create a CoreML Custom layer that runs on the GPU, using Objective-C for CoreML setup and Metal for GPU programming. I have created the CoreML model with the custom layer and can successfully execute on the GPU, I wish to create an MTLBuffer from an input MTLTexture in my setup actual GPU execution, although I can't seem to do so, or get access to the memory address to the MTLTexture memory. When defining a custom layer in CoreML to run on the GPU, the following function needs to be defined, with the given prototype; (BOOL) encodeToCommandBuffer:(id<MTLCommandBuffer>)commandBuffer inputs:(NSArray<id<MTLTexture>> *)inputs outputs:(NSArray<id<MTLTexture>> *)outputs error:(NSError *__autoreleasing _Nullable *)error{ // GPU Setup, moving data, encoding, execution and so on here } Here, the inputs are passed as an NSArray of MTLTexture's, we then pass these texture's on to the Metal Shader for computation. My problem is that I want to pass an MTLBuffer to the Metal Shader, which points to the input data, say inputs[0], but I am having troubling copying the input MTLTexture to an MTLBuffer. I have tried using the MTLBlitCommandEncoder to copy the data from the MTLTexture to an MTLBuffer like so; id<MTLBuffer> test_buffer = [command_PSO.device newBufferWithLength:(8) options:MTLResourceStorageModeShared]; id <MTLBlitCommandEncoder> blitCommandEncoder = [commandBuffer blitCommandEncoder]; [blitCommandEncoder copyFromTexture:inputs[0] sourceSlice:0 sourceLevel:0 sourceOrigin:MTLOriginMake(0, 0, 0) sourceSize:MTLSizeMake(1, 1, 1) toBuffer:test_buffer destinationOffset:0 destinationBytesPerRow:8 destinationBytesPerImage:8]; [blitCommandEncoder endEncoding]; The above example should copy a single pixel from the MTLTexture, inputs[0], to the MTLBuffer, test_buffer, but this is not the case. MTLTextures, getBytes also doesn't work as the inputs have MTLResourceStorageModePrivate set. When I inspect the input MTLTexture I note that the attribute buffer = <null> and I'm wondering if this could be an issue since the texture was not created from a buffer, and perhaps doesn't store the address to memory easily, but surely we should be able to get the memory address somewhere? For further reference, here is the input MTLTexture definition; <CaptureMTLTexture: 0x282469500> -> <AGXA14FamilyTexture: 0x133d9bb00> label = <none> textureType = MTLTextureType2DArray pixelFormat = MTLPixelFormatRGBA16Float width = 8 height = 1 depth = 1 arrayLength = 1 mipmapLevelCount = 1 sampleCount = 1 cpuCacheMode = MTLCPUCacheModeDefaultCache storageMode = MTLStorageModePrivate hazardTrackingMode = MTLHazardTrackingModeTracked resourceOptions = MTLResourceCPUCacheModeDefaultCache MTLResourceStorageModePrivate MTLResourceHazardTrackingModeTracked usage = MTLTextureUsageShaderRead MTLTextureUsageShaderWrite shareable = 0 framebufferOnly = 0 purgeableState = MTLPurgeableStateNonVolatile swizzle = [MTLTextureSwizzleRed, MTLTextureSwizzleGreen, MTLTextureSwizzleBlue, MTLTextureSwizzleAlpha] isCompressed = 0 parentTexture = <null> parentRelativeLevel = 0 parentRelativeSlice = 0 buffer = <null> bufferOffset = 0 bufferBytesPerRow = 0 iosurface = 0x0 iosurfacePlane = 0 allowGPUOptimizedContents = YES label = <none>

Hi What's the standard/recommended way to set the MTKView's drawableSize to a size lower than the view/window and, at the same time, be able to update the drawableSize whenever the app's window is resized? Thanks

Hello, developers, I'm implementing slice rendering of 3d volume. And then, I have a simple question... I use a simple vertex buffer type both in swift code and in metal code. Firstly, I defined uv to float2 but it's not working. It has weird texture coordinates when I use float2... public struct VertexIn: sizeable {     var position = float3()     var normal = float3()     var uv = float3()   } struct VertexIn {   float3 position [[ attribute(0) ]];   float3 normal [[ attribute(1) ]];   float3 uv [[ attribute(2) ]]; }; like this float2. float3. It has just difference at the uv type. And I have same issue at passing uniform to shader. When I pass uniform that includes float or short types it doesn't work. So I change type to float3... So I inquire that metal data type is so difference compared with swift type??? Or what types are same and supported from metal.

Hi, MacOS version: 12.2.1 Xcode version: 13.2.1 I'm using MTKView and setting a drawableSize = 1024,768 and autoResizeDrawable = NO, however, drawableSizeWillChange() is being called on app startup, just once, and never again, with a different drawableSize. Maybe it has to do with incorrect drawable size? Thanks

I was wondering if anyone had any insight into why MTKTextureLoader might very rarely return a texture which is just fully opaque magenta (each pixel is #ff00ff). What I know: I'm using MTKTextureLoader.newTexture(URL:options:) to synchronously load the texture within a standard, synchronous dispatch queue (though the single texture loader itself is created on the main thread). No error is thrown in the above call and nothing is printed to the console, and a texture is returned. The URL of the texture resides on the local filesystem and points to a fairly unremarkable 512x512 JPEG. The resulting texture returned by the loader is the correct resolution (but every pixel is magenta). The majority of launches of the app load all the textures without any issues (I think at least 90%), but if a texture does fail to load, many others fail to load as well (especially textures which are loaded immediately after a failed one). The exact same files which load incorrectly in one run of the app will load correctly in another run. For completeness, the texture loader options: textureLoaderOptions = [ .allocateMipmaps: false, .generateMipmaps: false, .textureUsage: NSNumber(value: MTLTextureUsage.shaderRead.rawValue), .textureStorageMode: NSNumber(value: MTLStorageMode.`private`.rawValue) ]

Is it possible to render an MTKView in different color spaces? For example, if I want to render it in CMYK, is there a way to adjust the colors on every frame to that colorspace before presenting it on the screen, or is that something that needs to be handled in a shader?

I'm trying to render text with Metal for my (2D) game. I'm using the system fonts, e.g. the SF Pro family for English texts. I render the glyphs onto an atlas texture, and then sample from this texture. My questions: I assume that, for copyright reasons, I'm not allowed to include a pre-rendered font atlas in my app. Is my assumption correct? I can, however, have the app generate the atlas when it's first opened, and then use it within the app, right? If 2. is true, then can the app save the atlas somewhere in the app's private storage, so that it would not need to re-generate the atlas the next time? Thanks!

I have started learning Metal and the source text I was reading mentioned that interactions with images rendered to the screen is handled by Core Animation layers (CAMetalLayer) in iOS, but had no mention for any other platform. I am guessing for the other "mobile" platforms the same thing applies as iOS, but what handles this interaction for macOS, or is it also the same?