-
使用 Metal 构建实时神经网络渲染管线
探索如何使用 Metal 4 将机器学习整合到实时渲染管线中。我们将介绍利用 MetalFX 神经网络降噪实现生产级效果的实用模式和最佳做法，并分享来自 Maxon Redshift Live 的真实经验。了解如何使用 ML 命令编码器，在图形工作流程中训练和部署神经网络色调映射器。最后，深入探索新的张量 API，直接在着色器中构建并评估小型专用神经网络。
章节
- 0:00 - Introduction
- 2:16 - MetalFX Denoising
- 9:57 - Deploy custom ML networks with Metal 4
- 13:40 - Inline neural networks with tensorOps
- 20:55 - Next steps
资源
- Training a neural network to render irradiance in real time
- Metal sample code library
- Download the Metal Performance Primitives (MPP) Programming Guide
- Understanding the Metal 4 core API
相关视频
WWDC25
-
搜索此视频…
-
-
8:46 - Compute camera-only motion vectors
#include <metal_stdlib> using namespace metal; // Compute camera-only motion vectors float4 clipCurrent = viewProjCurrent * float4(worldPos, 1.0); float2 ndcCurrent = clipCurrent.xy / clipCurrent.w; float4 clipPrevious = viewProjPrevious * float4(worldPos, 1.0); float2 ndcPrevious = clipPrevious.xy / clipPrevious.w; float2 motion = ndcPrevious - ndcCurrent; // Get subpixel offset for current and previous frames float2 jitterCurrent = getJitter(frameIndex); float2 jitterPrevious = getJitter(frameIndexPrevious); motion -= jitterPrevious - jitterCurrent;
-
-
- 0:00 - Introduction
An overview of how machine learning is transforming real-time rendering pipelines on Apple platforms, and a preview of three levels of ML integration: MetalFX Denoising, deploying custom networks with Metal 4, and building tiny networks inline in shaders with tensorOps.
- 2:16 - MetalFX Denoising
How to integrate MetalFX Denoising into a path tracer running at one sample per pixel. Covers auxiliary inputs (albedo, depth, motion vectors), best practices for clean inputs, transparency overlays, the denoiser strength mask, and primary surface replacement for mirrors and glass — illustrated with Redshift Live from Maxon.
- 9:57 - Deploy custom ML networks with Metal 4
How to train a neural tone mapper offline (e.g., HDRNet), export it to Metal Performance Shaders Graph, and execute it inside a Metal 4 command buffer alongside your existing rendering passes to replace complex post-processing pipelines with a single network.
- 13:40 - Inline neural networks with tensorOps
How to build and run small multilayer perceptrons directly inside Metal shaders using the TensorOps API and cooperative tensors. Demonstrates an online-trained sky visibility probe that adapts to dynamic scenes each frame — enabling ML inference and training that runs alongside your existing compute and render work.
- 20:55 - Next steps
A recap of the three levels of ML integration in rendering pipelines, and guidance on where to start: download Xcode, explore Metal 4 sample code, and adopt MetalFX denoising for real-time applications first.