Hi!
I was trying to port our sdk for visionOS.
I was going through the documentation and saw this video: https://developer.apple.com/videos/play/wwdc2023/10089/
Is there any working code sample for it, same goes for arkit c api ?
Couldn't find any links. Thanks in advance.
Sahil
Metal
RSS for tagRender advanced 3D graphics and perform data-parallel computations using graphics processors using Metal.
Posts under Metal tag
200 Posts
Sort by:
Post
Replies
Boosts
Views
Activity
with the latest Xcode that runs with Mac OS 14.5 Developer Beta has messages with a time and date in them There are also some other fields of an indeterminate origin/type.
"2024-05-06 15:37:32.383996-0500 RoomPlanExampleApp[24190:1708576] [CAMetalLayerDrawable texture] should not be called after already presenting this drawable. Get a nextDrawable instead."
specifically I need to know how the string [24190:1708576] relates to a location in my application so I can act on the message. I certainly can't find the text in the "[CAMetalLayerDrawable texture]". field anywhere in the user documentation OR the Development documentation. In order for a diagnostic message to be Actionable and remedied by a user it must identify the module and source line of the initiating code and there must be accessible documentation for users to access to get an explanation of potential remedies.. This interface fails to supply enough information to diagnose the problem. The label in [CAMetalLayerDrawable texture] cannot even be found in a search of the package information attached to the Xcode Release paired with the IOS and Mac OS system releases.
I am wanting to create a 3D video game in Xcode for macOS, iOS, iPadOS, tvOS, and visionOS. I have heard that there are a few different ways to go about this such as MetalKit or SceneKit. These libraries seem to have little examples and documentation so I am wondering:
Are they still be developed/supported?
Which platform should I make a game in?
Where are some resources to learn how to use these platforms?
Are there other better platforms that I am just not aware of?
Thanks!
What best to pass to the options parameter of:
MTLDevice.makeBuffer(length:options:)
MTLDevice.makeBuffer(bytes:length:options:)
MTLDevice.makeBuffer(bytesNoCopy:length:options:deallocator:)
Basically I'm looking for a "plain English" explanation of MTLResourceOptions doc page.
Hello. I am trying to enable Metal to take advantage of SwiftUI+Shaders in an existing app. In my target, I have added metal.framework, added a .metal file and confirmed I see the Metal Compiler option in my Build Settings. When I add a colorEffect modifier using the ShaderLibrary, the object renders black. I confirmed this is working in a new project so I suspect there are more steps required to get this working in an existing application.
I have a glb model that is loading absolutely fine, repeatedly, in safari or chrome.
There is only one texture that is 8192x8192
it never has a problem when loading in browser.
when we embed the url into an app, the model loads the first few times (exiting the model and going back to the main menu and then reloading the model) but, after a few attempts, the texture fails to load. The model and all data is visible but the texture, itself, is black.
why could this be happening? Is there something in the iOS code that is breaking it? Is the iOS code trying to automatically cache the texture and it’s running out of memory?
anyone who can provide the help and support that we require will be much appreciated.
thank you advance.
Hi, just got an Apple M3 Pro to try it out on some Jax operations. I see the development is actively ongoing so maybe this error can help.
This is the environment:
Metal device set to: Apple M3 Pro
systemMemory: 18.00 GB
maxCacheSize: 6.00 GB
jax: 0.4.26
jaxlib: 0.4.23
numpy: 1.26.4
python: 3.11.8 | packaged by conda-forge | (main, Feb 16 2024, 20:49:36) [Clang 16.0.6 ]
jax.devices (1 total, 1 local): [METAL(id=0)]
process_count: 1
platform: uname_result(system='Darwin', node='MKFL96VR9YT', release='23.4.0', version='Darwin Kernel Version 23.4.0: Wed Feb 21 21:44:54 PST 2024; root:xnu-10063.101.15~2/RELEASE_ARM64_T6030', machine='arm64')
This is a minimal example which produces an error, I think due to the fft part:
from jax import numpy as np
array = np.ones((16, 16))
np.fft.fft2(array)
This is the full traceback:
Traceback (most recent call last):
File "/Users/user/Downloads/wow.py", line 5, in <module>
np.fft.fft2(array)
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/numpy/fft.py", line 216, in fft2
return _fft_core_2d('fft2', xla_client.FftType.FFT, a, s=s, axes=axes,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/numpy/fft.py", line 210, in _fft_core_2d
return _fft_core(func_name, fft_type, a, s, axes, norm)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/numpy/fft.py", line 102, in _fft_core
transformed = lax.fft(arr, fft_type, tuple(s))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/traceback_util.py", line 179, in reraise_with_filtered_traceback
return fun(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/pjit.py", line 298, in cache_miss
outs, out_flat, out_tree, args_flat, jaxpr, attrs_tracked = _python_pjit_helper(
^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/pjit.py", line 176, in _python_pjit_helper
out_flat = pjit_p.bind(*args_flat, **params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/core.py", line 2788, in bind
return self.bind_with_trace(top_trace, args, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/core.py", line 425, in bind_with_trace
out = trace.process_primitive(self, map(trace.full_raise, args), params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/core.py", line 913, in process_primitive
return primitive.impl(*tracers, **params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/pjit.py", line 1494, in _pjit_call_impl
return xc._xla.pjit(name, f, call_impl_cache_miss, [], [], donated_argnums, # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/pjit.py", line 1471, in call_impl_cache_miss
out_flat, compiled = _pjit_call_impl_python(
^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/pjit.py", line 1406, in _pjit_call_impl_python
lowering_parameters=mlir.LoweringParameters()).compile()
^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/interpreters/pxla.py", line 2369, in compile
executable = UnloadedMeshExecutable.from_hlo(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/interpreters/pxla.py", line 2908, in from_hlo
xla_executable, compile_options = _cached_compilation(
^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/interpreters/pxla.py", line 2718, in _cached_compilation
xla_executable = compiler.compile_or_get_cached(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/compiler.py", line 266, in compile_or_get_cached
return backend_compile(backend, computation, compile_options,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/profiler.py", line 335, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/compiler.py", line 238, in backend_compile
return backend.compile(built_c, compile_options=options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
jaxlib.xla_extension.XlaRuntimeError: UNKNOWN: <unknown>:0: error: 'func.func' op One or more function input/output data types are not supported.
<unknown>:0: note: see current operation:
"func.func"() <{arg_attrs = [{mhlo.layout_mode = "default", mhlo.sharding = "{replicated}"}], function_type = (tensor<16x16xf32>) -> tensor<16x16xcomplex<f32>>, res_attrs = [{jax.result_info = "", mhlo.layout_mode = "default"}], sym_name = "main", sym_visibility = "public"}> ({
^bb0(%arg0: tensor<16x16xf32>):
%0 = "mhlo.convert"(%arg0) : (tensor<16x16xf32>) -> tensor<16x16xcomplex<f32>>
%1 = "mhlo.fft"(%0) {fft_length = dense<16> : tensor<2xi64>, fft_type = #mhlo<fft_type FFT>} : (tensor<16x16xcomplex<f32>>) -> tensor<16x16xcomplex<f32>>
"func.return"(%1) : (tensor<16x16xcomplex<f32>>) -> ()
}) : () -> ()
<unknown>:0: error: failed to legalize operation 'func.func'
<unknown>:0: note: see current operation:
"func.func"() <{arg_attrs = [{mhlo.layout_mode = "default", mhlo.sharding = "{replicated}"}], function_type = (tensor<16x16xf32>) -> tensor<16x16xcomplex<f32>>, res_attrs = [{jax.result_info = "", mhlo.layout_mode = "default"}], sym_name = "main", sym_visibility = "public"}> ({
^bb0(%arg0: tensor<16x16xf32>):
%0 = "mhlo.convert"(%arg0) : (tensor<16x16xf32>) -> tensor<16x16xcomplex<f32>>
%1 = "mhlo.fft"(%0) {fft_length = dense<16> : tensor<2xi64>, fft_type = #mhlo<fft_type FFT>} : (tensor<16x16xcomplex<f32>>) -> tensor<16x16xcomplex<f32>>
"func.return"(%1) : (tensor<16x16xcomplex<f32>>) -> ()
}) : () -> ()
I'd be happy running more tests should you need them, I'm new to this, so not sure which just yet.
Many thanks!!
Hello!
I run into, what seem to be compiler issue. The shader source given to Metal is: https://shader-playground.timjones.io/1bcf3ffbb313878ccd594ddbb27b746e
This shader is generated by spirv-cross, from GLSL source, so for readability here is original source: https://github.com/Try/OpenGothic/blob/master/shader/hiz/hiz_mip.comp
(shader variant uses SSBO counter, not atomic-image)
Here is relevant path of application log:
2024-04-21 16:27:13.621218+0200 Gothic2Notr[23992:2003969] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED
2024-04-21 16:27:13.656559+0200 Gothic2Notr[23992:2003969] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED
2024-04-21 16:27:13.701323+0200 Gothic2Notr[23992:2003969] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED
2024-04-21 16:27:13.701477+0200 Gothic2Notr[23992:2003969] MTLCompiler: Compilation failed with XPC_ERROR_CONNECTION_INTERRUPTED on 3 try
2024-04-21 16:27:13.701817+0200 Gothic2Notr[23992:2003969] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED
iOS version: 15.8.2
MTL::CompileOptions::languageVersion: 2.4 (also tested other version - same result)
Offended part of shader:
void store(int mip, ivec2 uv, float z) {
// NOTE: replacing this function to NOP, avoid the crash
// NOTE2: this switch-case is crude emulation of bindless storage-image
switch(mip) {
case 1:
imageStore(mip1, uv, vec4(z));
break;
case 2:
imageStore(mip2, uv, vec4(z));
break;
case 3:
imageStore(mip3, uv, vec4(z));
break;
case 4:
imageStore(mip4, uv, vec4(z));
break;
case 5:
imageStore(mip5, uv, vec4(z));
break;
case 6:
imageStore(mip6, uv, vec4(z));
break;
case 7:
imageStore(mip7, uv, vec4(z));
break;
case 8:
imageStore(mip8, uv, vec4(z));
break;
}
}
Some extra info:
The shader is simplified single-pass mip-map generator.
The same shader is know to work on mac M1 laptop without any issues
Please have a look and looking forward for driver-fix. Thanks!
(Copied from https://github.com/google/jax/issues/20835)
I am attempting to use JAX on Metal (on a M1 Pro chip) to model discrete (count) data. I've installed the latest version jax-metal 0.0.6 using pip.
The installation seems to have worked overall as I can perform basic Jax array operations on GPU. However, when I try to compute the (log-)PMFs/PDFs of random variables which are defined in terms of the (log-)Gamma function I get errors like the one below which seems to indicate that the lax.lgamma function is not supported under the hood on M1 metal.
This is essential functionality for a wide class of probabilistic machine learning models. Note that following functions (among others) are broken as a result:
jax.scipy.stats.binom.logpmf
jax.scipy.stats.nbinom.logpmf
jax.scipy.stats.poisson.logpmf
jax.scipy.stats.dirichlet.logpdf
jax.scipy.stats.beta.logpdf
jax.scipy.stats.gamma.logpdf
...
>>> jax.scipy.stats.binom.logpmf(1, n=2, p=0.5)
jax.errors.SimplifiedTraceback: For simplicity, JAX has removed its internal frames from the traceback of the following exception. Set JAX_TRACEBACK_FILTERING=off to include these.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/ljb80/.virtualenvs/jax-metal/lib/python3.10/site-packages/jax/_src/scipy/stats/binom.py", line 31, in logpmf
gammaln(n + 1),
File "/Users/ljb80/.virtualenvs/jax-metal/lib/python3.10/site-packages/jax/_src/scipy/special.py", line 44, in gammaln
return lax.lgamma(x)
File "/Users/ljb80/.virtualenvs/jax-metal/lib/python3.10/site-packages/jax/_src/lax/special.py", line 46, in lgamma
return lgamma_p.bind(x)
File "/Users/ljb80/.virtualenvs/jax-metal/lib/python3.10/site-packages/jax/_src/core.py", line 422, in bind
return self.bind_with_trace(find_top_trace(args), args, params)
File "/Users/ljb80/.virtualenvs/jax-metal/lib/python3.10/site-packages/jax/_src/core.py", line 425, in bind_with_trace
out = trace.process_primitive(self, map(trace.full_raise, args), params)
File "/Users/ljb80/.virtualenvs/jax-metal/lib/python3.10/site-packages/jax/_src/core.py", line 913, in process_primitive
return primitive.impl(*tracers, **params)
File "/Users/ljb80/.virtualenvs/jax-metal/lib/python3.10/site-packages/jax/_src/dispatch.py", line 87, in apply_primitive
outs = fun(*args)
jaxlib.xla_extension.XlaRuntimeError: UNKNOWN: <stdin>:1:0: error: failed to legalize operation 'chlo.lgamma'
<stdin>:1:0: note: see current operation: %0 = "chlo.lgamma"(%arg0) : (tensor<f32>) -> tensor<f32>
System info (python version, jaxlib version, accelerator, etc.)
jax: 0.4.26
jaxlib: 0.4.23
numpy: 1.26.4
python: 3.10.6 | packaged by conda-forge | (main, Aug 22 2022, 20:38:29) [Clang 13.0.1 ]
jax.devices (1 total, 1 local): [METAL(id=0)]
process_count: 1
platform: uname_result(system='Darwin', node='PHS027794', release='23.4.0', version='Darwin Kernel Version 23.4.0: Fri Mar 15 00:10:42 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_T6000', machine='arm64')
[UE] Assertion failed: Binding.index < ML_MaxTextures [File:./Runtime/Apple/MetalRHI/Private/MetalPipeline.cpp] [Line: 563]
Metal texture index exceeded!
How can I solve it?
Hi there, I have some existing metal rendering / shader views that I would like to use to present stereoscopic content on the Vision Pro. Is there a metal shader function / variable that lets me know which eye we're currently rendering to inside my shader? Something like Unity's unity_StereoEyeIndex? I know RealityKit has GeometrySwitchCameraIndex, so I want something similar (but outside of a RealityKit context).
Many thanks,
Rich
What is the most efficient way to use a MTLTexture (created procedurally at run-time) as a RealityKit TextureResource? I update the MTLTexture per-frame using regular Metal rendering, so it’s not something I can do offline. Is there a way to wrap it without doing a copy?
A specific example would be great.
Thank you!
I'm implementing a bitonic sort in Metal with a Swift app. This requires 100's kernel dispatch calls for each of the swap stages which touch the whole array, the work required by the GPU is small. I haven't been able to get this to run fast enough in Swift and it seems its due to a high overhead for each dispatchThread command. I rewrote the test program in Objective C with a super-simple kernel function and its runs 25x faster from Objective C!
Kernel function
kernel void fill(device uint8_t *array [[buffer(0)]],
const device uint32_t &N [[buffer(1)]],
const device uint8_t &value [[buffer(2)]],
uint i [[thread_position_in_grid]])
{
if (i < N) {
array[i] = value;
}
}
The Swift code is:
func fill(pso:MTLComputePipelineState, buffer:MTLBuffer, N: Int, passes: Int) {
guard let commandBuffer = commandQueue.makeCommandBuffer() else { return }
let gridSize = MTLSizeMake(N, 1, 1)
var threadGroupSize = pso.maxTotalThreadsPerThreadgroup
if (threadGroupSize > N) {
threadGroupSize = N;
}
let threadgroupSize = MTLSizeMake(threadGroupSize, 1, 1);
for pass in 0..<passes {
guard let computeEncoder = commandBuffer.makeComputeCommandEncoder() else { return }
var value:UInt8 = UInt8(pass);
var NN:UInt32 = UInt32(N);
computeEncoder.setComputePipelineState(pso)
computeEncoder.setBuffer(buffer, offset: 0, index: 0)
computeEncoder.setBytes(&NN, length: MemoryLayout<UInt32>.size, index: 1)
computeEncoder.setBytes(&value, length: MemoryLayout<UInt8>.size, index: 2)
computeEncoder.dispatchThreadgroups(gridSize, threadsPerThreadgroup: threadgroupSize)
computeEncoder.endEncoding()
}
commandBuffer.commit()
commandBuffer.waitUntilCompleted()
}
let device = MTLCreateSystemDefaultDevice()!
let library = device.makeDefaultLibrary()!
let commandQueue = device.makeCommandQueue()!
let funcFill = library.makeFunction(name: "fill")!
let pso = try? device.makeComputePipelineState(function: funcFill)
var N = 16384
let passes = 100
let buffer = device.makeBuffer(length:N, options: [.storageModePrivate])!
for _ in 1...10 {
let startTime = DispatchTime.now()
fill(pso:pso!, buffer:buffer, N:N, passes:passes)
let endTime = DispatchTime.now()
let elapsedTime = endTime.uptimeNanoseconds - startTime.uptimeNanoseconds
print("Elapsed time:", Float(elapsedTime)/1_000_000, "ms");
}
and the Objective C code (which should be almost identical) is
void fill(id<MTLCommandQueue> commandQueue,
id<MTLComputePipelineState> funcPSO,
id<MTLBuffer> A,
uint32_t N,
int passes) {
id<MTLCommandBuffer> commandBuffer = [commandQueue commandBuffer];
MTLSize gridSize = MTLSizeMake(N, 1, 1);
NSUInteger threadGroupSize = funcPSO.maxTotalThreadsPerThreadgroup;
if (threadGroupSize > N) {
threadGroupSize = N;
}
MTLSize threadgroupSize = MTLSizeMake(threadGroupSize, 1, 1);
for(uint8_t pass=0; pass<passes; pass++)
{
id<MTLComputeCommandEncoder> computeEncoder = [commandBuffer computeCommandEncoder];
[computeEncoder setComputePipelineState:funcPSO];
[computeEncoder setBuffer:A offset:0 atIndex:0];
[computeEncoder setBytes:&N length:sizeof(uint32_t) atIndex:1];
[computeEncoder setBytes:&pass length:sizeof(uint8_t) atIndex:2];
[computeEncoder dispatchThreads:gridSize threadsPerThreadgroup:threadgroupSize];
[computeEncoder endEncoding];
}
[commandBuffer commit];
[commandBuffer waitUntilCompleted];
}
int main() {
NSError *error;
id<MTLDevice> device = MTLCreateSystemDefaultDevice();
id<MTLLibrary> library = [device newDefaultLibrary];
id<MTLCommandQueue> commandQueue = [device newCommandQueue];
id<MTLFunction> funcFill = [library newFunctionWithName:@"fill"];
id<MTLComputePipelineState> pso = [device newComputePipelineStateWithFunction:funcFill error:&error];
// Prepare data
int N = 16384;
int passes = 100;
id<MTLBuffer> bufferA = [device newBufferWithLength:N options:MTLResourceStorageModePrivate];
for(int it=1; it<=10; it++)
{
CFTimeInterval startTime = CFAbsoluteTimeGetCurrent();
fill(commandQueue, pso, bufferA, N, passes);
CFTimeInterval duration = CFAbsoluteTimeGetCurrent() - startTime;
NSLog(@"Elapsed time: %.1f ms", 1000*duration);
}
}
The Swift output is:
Elapsed time: 89.35556 ms
Elapsed time: 63.243744 ms
Elapsed time: 62.39568 ms
Elapsed time: 62.183224 ms
Elapsed time: 63.741913 ms
Elapsed time: 63.59463 ms
Elapsed time: 62.378654 ms
Elapsed time: 61.746098 ms
Elapsed time: 61.530384 ms
Elapsed time: 60.88774 ms
The objective C output is
2024-04-18 19:27:45.704 compute_test[3489:92754] Elapsed time: 3.6 ms
2024-04-18 19:27:45.706 compute_test[3489:92754] Elapsed time: 2.6 ms
2024-04-18 19:27:45.709 compute_test[3489:92754] Elapsed time: 2.6 ms
2024-04-18 19:27:45.712 compute_test[3489:92754] Elapsed time: 2.6 ms
2024-04-18 19:27:45.714 compute_test[3489:92754] Elapsed time: 2.7 ms
2024-04-18 19:27:45.717 compute_test[3489:92754] Elapsed time: 2.8 ms
2024-04-18 19:27:45.720 compute_test[3489:92754] Elapsed time: 2.8 ms
2024-04-18 19:27:45.723 compute_test[3489:92754] Elapsed time: 2.7 ms
2024-04-18 19:27:45.726 compute_test[3489:92754] Elapsed time: 2.5 ms
2024-04-18 19:27:45.728 compute_test[3489:92754] Elapsed time: 2.5 ms
I compile the Swift code for Release, optimised for speed.
I can't believe there should be a difference here, so what could be different, and what might I be doing wrong?
thanks
Adrian
The Drawing fully immersive content using Metal guide describes how to use Metal for visionOS immersive experiences, but seemingly requires swift to bring up the required CompositorLayer.
@main
struct MyApp: App {
var body: some Scene {
ImmersiveSpace(id: "MyContent") {
CompositorLayer { layerRenderer in
let renderThread = Thread {
let engine = myEngineCreate(layerRenderer)
myEngineRenderLoop(engine)
}
renderThread.name = "Render Thread"
renderThread.start()
}
}
}
The ImmersiveSpace scene can presumably be replaced with a call to
[UIApplication.sharedApplication activateSceneSessionForRequest:[UISceneSessionActivationRequest requestWithRole:UISceneSessionRoleImmersiveSpaceApplication] errorHandler:nil]
But is there a replacement for CompositorLayer? Or some other way to produce a cp_layer_renderer?
Perhaps it would be possible to write a small swift helper for this, but given the swift interface for CompositorLayer how would that be tied to an existing UIScene as created above?
@available(visionOS 1.0, *)
public struct CompositorLayer : SwiftUI.ImmersiveSpaceContent {
public init(configuration: any _CompositorServices_SwiftUI.CompositorLayerConfiguration = .default, renderer: @escaping (CompositorServices.LayerRenderer) -> Swift.Void)
public var body: Swift.Never {
get
}
public typealias Body = Swift.Never
}
as the title said, I want to sample the current pixel in the layer and if the color is the same as one in the parameter, I need to find a way to return that position back to SwiftUI.
Do you think that's possible?
in Shader.Argument I didn't find a way to directly convert to a pointer.
Is it possible to set the header include paths when compiling a metal library from source at runtime?
I'm dynamically generating kernel source that I'm compiling at runtime. I want to include some functions I have defined in a header. If I was doing online computation, I can specify include paths using -I /path/to/include using the command.
xcrun metal -c -I /path/to/include example.metal -o example.air
However when I'm online, it doesn't appear possible to define a header search path.
MTLCompileOptions *options = [MTLCompileOptions new];
options.fastMathEnabled = NO;
library = [device newLibraryWithSource:kernel_source
options:options
error:&error];
Is there a way to specify a header search path using newLibraryWithSource?
Where do I start with this error? I am using the Metal Debugger and have.a bunch of stuck command buffers. how do I look at the command buffers to see the errors? My suspicion is that the cause is some sort of memory leak. Not having access to the source for Metal leaves me stuck. The following message shows up in the logging pane of the execution.
Execution of the command buffer was aborted due to an error during execution. Ignored (for causing prior/excessive GPU errors) (00000004:kIOGPUCommandBufferCallbackErrorSubmissionsIgnored)
Type: Error | Timestamp: 2024-04-11 14:16:13.464336-05:00 | Library: Metal | Subsystem: Metal | Category: Default | TID: 0x2a0b8c
I just need some guidance
I'm brand new to Metal. I've googled, but can't get the right answer to come up. (Thanks, unhelpful ChatGPT generated answers polluting everything, but I digress...)
Ultimately, I'm trying to figure out how to use Metal to render 3D DICOM data on iOS specifically. If you're not familiar with DICOM, let's just say I've got a whole stack of CT image slices. Or to get really simple, I've got a cube of voxel values with differing values at each voxel coordinate.
Where do I even start in Metal to render something like this?
(I was trying to get the VTK toolkit compiled for iOS, which uses OpenGL, but that appears to be a dead end. And besides, Metal is supposed to be so much better.)
Thanks for any tips/leads/suggestions/general pointers.
Hi there -
Where would a dev go these days to get an initial understanding of SceneKit?
The WWDC videos linked in various places seem to be gone?!
For example, the SceneKit page at developer.apple.com lists features a session videos link that comes up without any result, https://developer.apple.com/scenekit/
Any advice..?
Cheers,
Jan
I have a fbo generated by last frame, which contain a color texture and a depth texture. Then i want blit that fbo to current MTKView, but failed. Thanks a lot for any suggestions.
Here is the code:
last frame:
id <MTLTexture> src_color;
id <MTLTexture> src_depth;
drawTo(src_color, src_depth);
current frame:
first, init a depth texture if null:
id<MTLTexture> depth_texture_;
if(depth_texture_) {
depth_texture_ = [device_ newTextureWithDescriptor:desc];
}
second,
create desc for encoder:
current_desc = metal_view_.currentRenderPassDescriptor;
current_desc.depthAttachment.texture = depth_texture_;
current_desc.depthAttachment.loadAction = MTLLoadActionClear;
current_desc.depthAttachment.clearDepth = 1.0;
current_desc.stencilAttachment.texture = depth_texture_;
current_desc.stencilAttachment.loadAction = MTLLoadActionClear;
current_desc.stencilAttachment.clearStencil = 0;
current_desc.colorAttachments[0].loadAction = MTLLoadActionClear;
current_desc.colorAttachments[0].clearColor = MTLClearColorMake(1.0, 1, 1, 1.0);
third,
blit to current MTKView:
auto dst_color = current_desc.colorAttachments[0].texture;
auto dst_depth = current_desc.depthAttachment.texture;
id<MTLCommandBuffer> commandBuffer = [command_queue_ commandBuffer];
commandBuffer.label = @"blit";
id <MTLBlitCommandEncoder> blitEncoder = [commandBuffer blitCommandEncoder];
[blitEncoder copyFromTexture:src_color
sourceSlice:0
sourceLevel:0
sourceOrigin:MTLOriginMake(srcRect.x, srcRect.y, 0)
sourceSize:MTLSizeMake(srcRect.width, srcRect.height, 1)
toTexture:dst_color
destinationSlice:0
destinationLevel:0
destinationOrigin:MTLOriginMake(dstRect.x, dstRect.y, 0)];
[blitEncoder copyFromTexture:src_depth
sourceSlice:0
sourceLevel:0
sourceOrigin:MTLOriginMake(srcRect.x, srcRect.y, 0)
sourceSize:MTLSizeMake(srcRect.width, srcRect.height, 1)
toTexture:dst_depth
destinationSlice:0
destinationLevel:0
destinationOrigin:MTLOriginMake(dstRect.x, dstRect.y, 0)];
[blitEncoder endEncoding];
[commandBuffer commit];
[commandBuffer waitUntilCompleted];