Render advanced 3D graphics and perform data-parallel computations using graphics processors using Metal.

Metal Documentation

Posts under Metal tag

289 results found
Sort by:
Post not yet marked as solved
32 Views

Upcoming Developments with Metal Ray Tracing

I have watched the Enhance Your App with Metal Ray Tracing video ( wwdc21-10149 ) a few times and am very interested in many of the upcoming features described. It seems as though there will be some tweaks ( ie. Instance IDs ) and some major additions ( ie. Keyframe Animation ). I am wondering if there is any sense of the timing of these features becoming available ( beta or otherwise ) and also, if some of the sample code might be provided in Swift. Thanks so much for the work that went into the video and explanations.
Asked
by John Lobe.
Last updated
.
Post not yet marked as solved
52 Views

Texture Pixel layout in Metal

I'm working on an app in MacOS in which I need to do mouse picking in a MTKView. To do this I render a Hit Triangle map of the mouse active areas into a MTLTexture. Since these areas do not change at run time the map only has to be rendered once. The metal debugger noticed this and suggested the texture resources be made purgeable. So I copied the pixel data out of the texture and marked the the texture as purgeable. I then read the pixel data from the C array and the mouse picking was screwed up. The Hit Triangles were not where they were supposed to be. I fussed around a bit and by trial and error I found out the pixels were copied out into the C array in column major order rather than row major order. I looked in the Metal documentation and couldn't find anywhere where this was pointed out. Is there anywhere in the Metal docs where it specifies how the pixels are laid out in a texture and how that relates to the viewable coordinates? This is not first time I've had problems related to this. Little details like this can really hang you up. // Since the hit map only has to be made once the data is copied out // and the texture is made purgeable. This to stop the debugger // from nagging about it. [pickTexture getBytes: hitTriangleMap bytesPerRow: PICK_TEXTURE_SIZE * sizeof(uint8 [4]) fromRegion: MTLRegionMake2D( 0 , 0 , PICK_TEXTURE_SIZE, PICK_TEXTURE_SIZE) mipmapLevel: 0]; [pickTexture setPurgeableState: MTLPurgeableStateVolatile]; [pickDepth setPurgeableState: MTLPurgeableStateVolatile]; // // * // * // * // // Previous inefficient version // // [pickTexture getBytes: hit // bytesPerRow: PICK_TEXTURE_SIZE * sizeof(uint8 [4]) // fromRegion: MTLRegionMake2D( hitPoint.x , hitPoint.y , 1, 1) // mipmapLevel: 0]; // New Version Column major arrays?? In any event it works this way. x = hitPoint.x; y = hitPoint.y; hit = hitTriangleMap[y][x]; //Inverted subscripts!!! if( hit[0] <= Zs) { cursorHit = hit[0]; if( cursorOverHitArea == NO) [[NSCursor pointingHandCursor] set]; cursorOverHitArea = YES; }```
Asked
by Bruce D M.
Last updated
.
Post not yet marked as solved
44 Views

SCNTechnique: Accessing A Node's Textures in DRAW_SCENE Pass

Is there any way to access the Textures of a material for a Node in a SCNTechnique DRAW_SCENE pass? I am implementing a custom SCNTechnique. This is primarily for practice with different rendering techniques, so performance is not my goal. Ideally, if I had full control of the pipeline I (think I) could attach multiple targets and get the albedo, normal, depth in a single pass. Right now, I am attempting to implement SSAO and (eventually) Deferred Lighting with SCNTechniques. My SSAO passes will eventually look like the following: Render Albedo to Texture, Depth to Texture Render Normal to Texture Compute Occlusion, combine with Albedo Right now I am attempting to debug, and am doing the following: (attempting to) Render Albedo to Texture Render Normal to Texture Combining the two for visual confirmation My Technique has the following definition: "passes": [ "albedoPass": [ "draw": "DRAW_SCENE", "metalVertexShader": "albedoVertex", "metalFragmentShader": "albedoFragment", "outputs": [ "color": "AlbedoMap", ], "colorStates": [ "clear": "1", ], "depthStates": [ "clear": "1", ], ], "normalPass": [ "draw": "DRAW_SCENE", "metalVertexShader": "normalsVertex", "metalFragmentShader": "normalsFragment", "inputs": [ ], "outputs": [ "color": "ViewSpaceNormalMap", ], "colorStates": [ "clear": "1", ], "depthStates": [ "clear": "1", ], ], "showNormal": [ "draw": "DRAW_QUAD", "metalVertexShader": "normalsResultVertex", "metalFragmentShader": "normalsResultFragment", "inputs": [ "quadOverlayPosition": "quadOverlayPosition-Symbol", "viewSpaceNormalMapTexture": "ViewSpaceNormalMap", "albedoMapTexture": "AlbedoMap", ], "outputs": [ "color": "COLOR", ], "colorStates": [ "clear": "1", ], ] ], "sequence": [ "albedoPass", "normalPass", "showNormalPass", ], "targets": [ "AlbedoMap": [ "type": "color", ], "ViewSpaceNormalMap": [ "type": "color", ], ], "symbols": [ "quadOverlayPosition-Symbol": [ "semantic": "vertex", ], ], ] For the albedo pass, would it be possible to acquire the diffuse material in the fragment/vertex shader given to the pass? Like so: #include <metal_stdlib> using namespace metal; #include <SceneKit/scn_metal> struct VertexIn { float3 position  [[attribute(SCNVertexSemanticPosition)]]; float2 uv [[attribute(SCNVertexSemanticTexcoord0)]]; }; struct VertexOut { float4 position [[position]]; float2 uv; }; struct NodeBuffer { float4x4 modelTransform; float4x4 inverseModelTransform; float4x4 modelViewTransform; float4x4 inverseModelViewTransform; float4x4 normalTransform; // Inverse transpose of modelViewTransform float4x4 modelViewProjectionTransform; float4x4 inverseModelViewProjectionTransform; float2x3 boundingBox; float2x3 worldBoundingBox; }; vertex VertexOut albedoVertex(VertexIn in [[ stage_in ]], constant SCNSceneBuffer& scn_frame [[buffer(0)]], constant NodeBuffer& scn_node [[buffer(1)]]) { VertexOut out; out.position = scn_node.modelViewProjectionTransform * float4(in.position, 1.0); out.uv = in.uv; return out; } fragment float4 albedoFragment(VertexOut out [[ stage_in ]], texture2d<float, access::sample> diffuse [[texture(0)]]) { constexpr sampler textureSampler(coord::normalized, filter::linear, address::repeat); float4 color = diffuse.sample(textureSampler, out.uv); return color; } Any help would be appreciated, thanks!
Asked
by pprovins.
Last updated
.
Post not yet marked as solved
110 Views

Metal draw indirect missing draw count

Why is there no count to any of these draw indirect directives? I am appending draws to a single MTLBuffer on the cpu, but can't limit how many are drawn out of the buffer. An offset isn't enough to specify a range. Can this be supplied in some bind call? - (void)drawIndexedPrimitives:(MTLPrimitiveType)primitiveType indexType:(MTLIndexType)indexType indexBuffer:(id <MTLBuffer>)indexBuffer indexBufferOffset:(NSUInteger)indexBufferOffset indirectBuffer:(id <MTLBuffer>)indirectBuffer indirectBufferOffset:(NSUInteger)indirectBufferOffset API_AVAILABLE(macos(10.11), ios(9.0)); Contrast this with the Vulkan call which as an offset and count. vkCmdDrawIndexedIndirect( m_encoder, indirectBuffer, drawBufferOffset, drawCount, sizeof( vkCmdDrawIndexedIndirect ) );
Asked Last updated
.
Post marked as solved
78 Views

Does setVisibilityResultMode works for instanced rendering?

lets say if I set setVisibilityResultMode:MTLVisibilityResultModeBoolean offset:0 and then call draw with instances: drawIndexedPrimitives: ... instanceCount:10 Is it going to write one value into visibilityResultBuffer or 10 values?
Asked
by vladimirb.
Last updated
.
Post not yet marked as solved
131 Views

iOS Simulator running metal hangs entire OS when using discrete GPU

I am working on an app that uses some Metal compute shaders and trying to get this to work on the simulator. I am having an issue where the simulator freezes the entire OS (which eventually results in either a panic or WindowServer being killed and dumping me on the login screen). After some trial and error, and many reboots, I isolated the problem to a combination of 2 things: Using an atomic_fetch_add_explicit on an MTLBuffer from a compute kernel. Having the simulator set to prefer the discrete GPU. Without the atomic operation, everything runs fine on either GPU. With the atomic operation, it runs fine on the integrated GPU, but if I select the discrete GPU it freezes WindowServer, the GUI is completely unresponsive and frozen, but I can SSH into the machine (until it decides to panic). When it's frozen there aren't any processes using significant amounts of CPU time. I can reproduce this 100% of the time. I'm using a MacBook Pro 15" (2018, radeon 560X 4GB). It doesn't matter wether or not the discrete GPU is used by the OS.
Asked
by Aaargh.
Last updated
.
Post not yet marked as solved
411 Views

vkCmdDrawIndexedIndirectCount functionality under Metal

Hello, It looks like my previous question was closed without being resolved. https://developer.apple.com/forums/thread/668171 There are FPS values from our new benchmark. Indirect command buffers are not working properly. So there is no way to emulate multi-draw indirect count functionality other than a loop of draw indirect commands. As you can see below, the same hardware is working three times slower under Metal because of it. And Apple M1 performance is worse than AMD integrated graphics performance. We have a buffer with multiple draw commands. How should we render it efficiently under Metal? AMD Vega 56 eGPU: Direct3D12: 94.0 Direct3D11: 87.2 Vulkan: 91.1 Metal: 35.8 AMD Ryzen™ 7 4800H: Direct3D12: 21.1 Direct3D11: 19.4 Vulkan: 20.5 Apple M1: Metal: 16.9 Thank you
Asked
by frustumo.
Last updated
.
Post not yet marked as solved
848 Views

Slow performance on iPhone 12/Pro/Max when copying pixel data from metal texture

Hello, We recently noticed that copying pixel data from a meta texture to memory is a lot slower on the new iPhones equipped with the A14 Bionic. We tracked down the guilty function on MTLTexture and found that getBytes(_:bytesPerRow:from:mipmapLevel: runs 8 to 20 times slower than 2 years old iPhones (iPhone XR). To measure how long it takes, we used signposts. We've created a dummy demo project where we convert a MTLTexture to a CVPixelBuffer in this project: https://github.com/alikaragoz/UsingARenderPipelineToRenderPrimitives The interesting part is located at this line: https://github.com/alikaragoz/UsingARenderPipelineToRenderPrimitives/blob/41f7f4385a490e889b94ee2c8913ce532a43aacb/Renderer/MetalUtils.swift#L40 Do you guys have an idea about what could be the issue?
Asked
by ali.
Last updated
.
Post not yet marked as solved
689 Views

DrawIndexedIndirectCount functionality under Metal

Hello everybody, I have a situation here. I cannot realize vkCmdDrawIndexedIndirectCount functionality by using argument buffers (actually, they are useless and buggy). I tried to reach developer support with those issues, but nobody is answering. So maybe somebody has an idea of how to execute multiple indirect draw calls based on GPU-generated count? Moreover, it is impossible to use indirect command buffers for that: "Fragment shader cannot be used with indirect command buffers". Current issues with indirect command buffers: Intel UHD Graphics 630 is not rendering all elements from the buffer. eGPU RX Vega 56 hangs the whole system for 5-6 seconds when command generation is performed by the vertex shader. "Compiler encountered an internal error" on Intel Iris Plus Graphics. Apple M1 renders a magenta screen when the generation is performed on compute shader. Apple M1 renders a magenta screen with a 20% chance of success rendering when the generation is performed on vertex shader. Thank you!
Asked
by frustumo.
Last updated
.
Post not yet marked as solved
35 Views

console fills up with "[default] invalid display identifier Main"

Since I installed the beta version of macOS Monterey, I get flooded with entries in the console which all say the same line: [default] invalid display identifier Main I have no idea why and what to look for. Any suggestions?
Asked Last updated
.
Post not yet marked as solved
3.2k Views

How to disable Metal compiler warnings for SceneKit / ARKit?

The Metal compiler when invoked by SceneKit is generating a lot of spurious warnings in iOS / iPadOS 14 Beta 7 & 8. This seems to be causing a significant degradation in performance for our SceneKit/ARKit-based app. Is there any way to disable this unnecessary Metal compiler logging? I tried making a MTLCOMPILERFLAGS = -w environment variable, but it didn't seem to have any effect. Feedback ID FB8618939. Logging looks like this: 2020-09-09 14:23:33.700122-0700 App[4672:1638397] [Metal Compiler Warning] Warning: Compilation succeeded with:  programsource:95:26: warning: unused function 'reduceop'     static inline float4 reduceop(float4 d0, float4 d1)                          ^ programsource:581:26: warning: unused variable 'scnshadowsamplerordz' static constexpr sampler scnshadowsamplerordz = sampler(coord::normalized, filter::linear, mipfilter::none, address::clamptoedge, comparefunc::greaterequal);                          ^ 2020-09-09 14:23:33.962519-0700 App[4672:1638397] [Metal Compiler Warning] Warning: Compilation succeeded with:  programsource:95:26: warning: unused function 'reduceop'     static inline float4 reduceop(float4 d0, float4 d1)                          ^ programsource:581:26: warning: unused variable 'scnshadowsamplerordz' static constexpr sampler scnshadowsamplerordz = sampler(coord::normalized, filter::linear, mipfilter::none, address::clamptoedge, comparefunc::greaterequal);
Asked
by lenk.
Last updated
.
Post not yet marked as solved
96 Views

Using threadgroup memory for image convolution

In the talk „Create image processing apps powered by Apple Silicon“ Harsh Patil mentioned that one should use threadgroup memory to load a chunk of image containing all the required pixels to run a convolution kernel. Unfortunately there was no code example and I have difficulty figuring out how something like that would be set up. I can imagine using imageblocks, but how would one load/store them in the shader? Could anyone offer some guidance (ideally with a code snippet)?
Asked
by jcookie.
Last updated
.
Post not yet marked as solved
57 Views

Constant address space buffer and 64KB limits

I’m using constant address space MTLBuffer to implement instanced rendering using -[MTLRenderCommandEncoder drawIndexedPrimitives:indexCount:indexType:indexBuffer:indexBufferOffset:instanceCount:]. When I tested on the NVIDIA GPU (Geforce GT 650M) the buffer’s contents was truncated over 64KB in the shader program. However the content was not truncated on AMD or Intel GPUs. (I tested on Intel HD Graphics 4000, Intel UHD 630, Radeon Pro 5500M) I found a specification on Metal Feature Set Tables document “Maximum function memory allocation for a buffer in the constant address space” is 64KB in GPUFamilyMac1/2. It looks like why the buffer is truncated on NVIDIA GPU. When I target my app to Intel Mac, should I always limit constant address space buffer size maximum to 64KB even if running on AMD or Intel GPU? When I use device address space instead of constant address space, Is there any performance difference between constant and device address space? (I heard that there's constant buffer cache on DirectX GPUs) Thanks!
Asked
by Prin_E.
Last updated
.
Post not yet marked as solved
272 Views

xcode12.3 cannot debug IOS game on M1 macmini

I can build and run a IOS game in xcode 12.3, but the breakpoints are not work, and capture gpu frame pops up an error: Replayer terminated unexpectedly with error code 512. Please export the frame capture file then file a radar
Asked
by xoyojank.
Last updated
.
Post not yet marked as solved
91 Views

Prevent MTKView camera feed rotation, but allow other on-screen VCs to rotate

Goal With an MTKView, replicate the gravity of the AVCaptureVideoPreviewLayer or Apple's Camera app. Video device orientation does not change. The camera feed's edges do not budge a pixel, never revealing the screen background. Other on-screen VCs rotate normally. Observed Applying Tech QA 1890's transform during viewWillTransition, the MTKView does counter-rotate... BUT that rotation is still uncomfortably visible. The edges of the view come unpinned during the animation, masking some camera pixels and showing a white background set for the VC holding the MTKView. Question How can I make those edges stick to screen bounds like a scared clam? I assume my error is in constraints, but I'm open to being wrong in other ways. :) View Hierarchy A tiny camera filter app has an overlay of camera controls (VC #1) atop an MTKView (in VC #2) pinned to the screen's edges. UINavigationController │ └─ CameraScreenVC │ ├── CameraControlsVC <- Please rotate subviews │ └── MetalCameraFeedVC └── MTKView <- Please no rotation edges Code Buildable demo repo Relevant snippets below. MetalCameraVC.swift final class MetalCameraVC: UIViewController { let mtkView = MTKView() // This VC's only view /// Called in viewDidAppear func setupMetal(){ metalDevice = MTLCreateSystemDefaultDevice() mtkView.device = metalDevice mtkView.isPaused = true mtkView.enableSetNeedsDisplay = false metalCommandQueue = metalDevice.makeCommandQueue() mtkView.delegate = self mtkView.framebufferOnly = false ciContext = CIContext( mtlDevice: metalDevice, options: [.workingColorSpace: CGColorSpace(name: CGColorSpace.sRGB)!]) } ... func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) { // blank } func draw(in mtkview: MTKView) { image = image.transformed(by: scaleToScreenBounds) image = image.cropped(to: mtkview.drawableSize.zeroOriginRect()) guard let buffer = metalCommandQueue.makeCommandBuffer(), let currentDrawable = mtkview.currentDrawable else { return } ciContext.render(image, to: currentDrawable.texture, commandBuffer: buffer, bounds: mtkview.drawableSize.zeroOriginRect(), colorSpace: CGColorSpaceCreateDeviceRGB()) buffer.present(currentDrawable) buffer.commit() } } extension MetalCameraVC { override func viewDidLoad() { super.viewDidLoad() view.addSubview(mtkView) } override func viewWillAppear(_ animated: Bool) { super.viewWillAppear(animated) mtkView.frame = view.frame if let orientation = AVCaptureVideoOrientation.fromCurrentDeviceOrientation() { lastOrientation = orientation } } } +Rotation /// Apple Technical QA 1890 Prevent View From Rotating extension MetalCameraVC { override func viewWillLayoutSubviews() { super.viewWillLayoutSubviews() mtkView.center = CGPoint(x: view.bounds.midX, y: view.bounds.midY) } override func viewWillTransition(to size: CGSize, with coordinator: UIViewControllerTransitionCoordinator) { super.viewWillTransition(to: size, with: coordinator) coordinator.animate { [self] context in let delta = coordinator.targetTransform let deltaAngle = atan2(delta.b, delta.a) var currentAngle = mtkView.layer.value(forKeyPath: "transform.rotation.z") as? CGFloat ?? 0 currentAngle += -1 * deltaAngle + 0.1 mtkView.layer.setValue(currentAngle, forKeyPath: "transform.rotation.z") } completion: { [self] context in var rounded = mtkView.transform rounded.a = round(rounded.a) rounded.b = round(rounded.b) rounded.c = round(rounded.c) rounded.d = round(rounded.d) mtkView.transform = rounded } } }
Asked
by wingover.
Last updated
.