A specification for augmenting or postprocessing SceneKit's rendering of a scene using additional drawing passes with custom Metal or OpenGL shaders.
SDKs
- iOS 8.0+
- macOS 10.10+
- Mac Catalyst 13.0+
- tvOS 9.0+
- watchOS 2.0+
Framework
- Scene
Kit
Declaration
class SCNTechnique : NSObject
Overview
Examples of multipass rendering techniques include:
Postprocessing rendered pixels. To create effects such as color grading and displacement mapping, define techniques that use as input the color buffer rendered by SceneKit and process that buffer with a fragment shader.
Deferred shading. To create effects such as motion blur and screen-space ambient occlusion (SSAO), define techniques that capture information about the scene into an intermediary buffer during the main rendering pass and then perform additional drawing passes using that buffer to create the final output image. The figure below illustrates the rendering process for an SSAO technique.
Example of a multipass technique
To create an SCNTechnique object, you supply a technique definition that specifies the input and output image buffers, shader programs, shader input variables, and rendering options for each drawing pass in the technique. Defining a technique does not require Metal or OpenGL client code, but you should be familiar with the terminology and conventions of GPU rendering.
To use a technique, assign it to the technique property of a view (or other SceneKit renderer object) or a camera.
Defining a Technique
SceneKit treats rendering techniques—along with shaders, 3D models, and 2D art assets—as resources rather than as part of your application code. Because the effects you create with techniques are highly visual, this approach allows you to separate design efforts from development efforts and quickly iterate on creating the visual content of your app or game.
Create a technique object using the init(dictionary:) method, providing a dictionary that defines the technique as a series of drawing passes, each with its own shader program, inputs and outputs, and rendering options. Typically, you obtain this dictionary from a property list file included in your app’s bundle resources. Table 1 and the additional tables and sections that follow specify the format of this dictionary’s contents.
Technique definition dictionary format
Key | Value |
|---|---|
| A dictionary of drawing pass definitions. Each key is a unique string you provide to identify the pass, and the corresponding value is a dictionary defining that drawing pass. See Table 2. |
| An array of strings, each of which names a drawing pass from the |
| A dictionary defining the bindings for GLSL attributes and uniform variables used in the technique’s shader programs. Each key matches the name of a symbol used in the shader source code, and the corresponding value is a dictionary describing how that symbol should be used. See Table 4. |
| A dictionary defining intermediary rendering targets. Each key is a unique string you provide to identify the target (used in the inputs and outputs dictionaries of a drawing pass definition), and the corresponding value is a dictionary defining the rendering target. See Table 5. |
Drawing pass definition dictionary format
Key | Type | Value |
|---|---|---|
| String | What to render for the drawing pass:
|
| String | The GLSL shader program to use for the drawing pass. Shader source code files in the app’s bundle resources directory must have the same base name and the extensions |
| String | The names of the Metal vertex and fragment shader functions to use for the drawing pass. These functions must exist in the app’s default Metal library. You must specify both fragment and vertex shaders, and you must specify either a GLSL shader program, a pair of Metal functions, or both. If both are specified, SceneKit uses whichever shader is appropriate for the current renderer. |
| Dictionary | Definitions of input variables for the drawing pass. Each key is the name of a uniform variable used in the GLSL shader source code referenced by the program key. The corresponding value may be any of: A symbol name from the technique’s A render target name (see Render Targets, Inputs and Outputs) A dictionary describing a texture sampler (see Table 6) |
| Dictionary | Definitions of output image buffers for the drawing pass. Each key is one of the strings |
| String | The |
| Dictionary | Options for the color output; one or more of these keys and values:
|
| Dictionary | Options for the depth output; one or more of these keys and values:
|
| Dictionary | Options for the stencil output; one or more of these keys and values:
|
| String | The faces of scene geometry to be rendered:
|
| Dictionary | Options for color blending; one or more of these keys and values:
See Blending. |
| String | A custom viewport rectangle in view coordinates, formatted as four numbers (origin x, origin y, width, and height). By default, a drawing pass covers the entire bounds of the view. |
| String | The name of a node in the scene to use as the viewer’s position for rendering; equivalent to the |
| Number (unsigned integer) | The number of samples per pixel for enabling multisampled rendering for the drawing pass. Defaults to |
| Number (8-bit unsigned integer) | A bitfield used for including nodes in the drawing pass if the |
| Number (8-bit unsigned integer) | A bitfield used for excluding nodes from the drawing pass if the |
Stencil behavior definition dictionary format
Key | Type | Value |
|---|---|---|
| String | The stencil operation to use if the depth test fails: |
| String | The stencil operation to use if the depth test succeeds and the stencil test fails: |
| String | The stencil operation to use if the depth and stencil tests succeed: |
| String | The stencil test function: |
| Number (8-bit unsigned integer) | A bitmask for selecting the bit plane of the stencil buffer to be tested. |
| Number (8-bit unsigned integer) | A bitmask for selecting the bit plane of the stencil buffer to write to. |
| Number (8-bit unsigned integer) | The reference value for stencil tests. Defaults to |
Shader symbol definition dictionary format
Key | Type | Value |
|---|---|---|
| String | Use this option to bind shader symbols to data supplied by SceneKit: The values The values The value If you set a |
| String | The GLSL type of the input variable: Use this option when providing custom values to shader programs (see Handling Parameters for a Technique’s Shader Programs). |
| String | If the symbol’s type is |
Category Masks
When SceneKit performs a rendering pass whose draw option is DRAW or DRAW, you can use category masks to filter the set of nodes drawn during that pass. For each node in the scene (or for DRAW, in the node subtree), SceneKit compares the node’s category property and the include and exclude options in the pass definition using bitwise AND operations. If the node’s category mask and the include mask overlap (that is, the bitwise AND results in a nonzero value) and the node’s category mask and the exclude mask do not overlap, SceneKit includes the node in the drawing pass. Otherwise the node is not rendered in the pass.
Render Targets, Inputs and Outputs
A drawing pass renders pixel data into one or more target image buffer (or framebuffer). In SceneKit’s main drawing pass, the color render target is the screen (or rather, a view or layer for screen display), and a depth render target temporarily stores the information needed to ensure that rendered surfaces appear in the correct depth order.
A pass in a custom rendering technique may postprocess the pixel data in SceneKit’s render target, generate its own pixel data for display, or render to an intermediate target to be used as input in a later pass. You specify render targets using the following identifiers in the inputs and outputs dictionaries of a pass definition:
Use the
COLORandDEPTHtargets as inputs to identify the color and depth buffers rendered to in SceneKit’s main drawing pass.Use the
COLORtarget as an output to identify the image buffer displayed as the end result of a technique.To create an intermediate target, define your own identifier as a key in the
targetsdictionary of a technique definition. For the corresponding value, provide a dictionary defining the render target using the keys and values in Table 5. Intermediate targets may be color, depth, or combined depth and stencil buffers. After you define a target, you can use its identifier in theinputsandoutputsdictionaries of a pass definition.To use an image as an input texture for a pass, define a symbol of type
sampler2Din the technique’ssymbolsdictionary.
To specify a render target or image sampler in the inputs dictionary of a pass definition, you can provide either an identifier string or a dictionary with the format described in Table 6. The options for samplers correspond to similar properties for SceneKit material textures. For more details on each, see SCNMaterial.
Render target definition dictionary format
Key | Type | Value |
|---|---|---|
| String | The type of render target: |
| String | The render target’s pixel format; one of the following: Color: Depth/stencil: |
| String | The size of the render target image, in points, specified in a string of format “WxH”. For example, the string “320x480” specifies an image buffer 320 points wide by 480 points tall. |
| Number (floating-point) | The scale factor of the render target. Defaults to |
Sampler input dictionary format
Key | Type | Value |
|---|---|---|
| String |
|
| String |
|
| String |
|
| String |
|
Blending
The blend key of a pass definition defines color blending options. Blending is disabled by default for faster rendering performance. Including a dictionary for this key enables blending (unless the dictionary’s enable key specifies false). Color blending combines a source color (the color output by the drawing pass’s shader program) with a destination color (the existing contents of the output buffer) using specified modes and operations.
Available blend modes (for the
color,Src color,Dst alpha, andSrc alphakeys):Dst zero,one,src,Color one,Minus Src Color src,Alpha one,Minus Src Alpha dst,Color one,Minus Dst Color dst,Alpha one,Minus Dst Alpha constant,Color one,Minus Constant Color constant,Alpha one, andMinus Constant Alpha alpha.Saturate Available blend operations (for the
colorandOp alphakeys):Op add,subtract,reverse,Subtract min,max.
These values correspond to blending options defined by the OpenGL specification. For further details, consult the OpenGL API Registry or OpenGL ES API Registry.
Example Technique Definition
Listing 1 shows an example definition dictionary for a technique that uses displacement mapping with a noise texture to postprocess a rendered scene. For ease of reading, the dictionary is formatted in JSON syntax. (To load an NSDictionary object from text in this format, use the JSONSerialization class.) Listing 2 and Listing 3 show the GLSL source code for the technique’s vertex and fragment shaders.
JSON definition dictionary for a technique
{
"passes" : {
"displacement" : {
"outputs" : {
"color" : "COLOR"
},
"inputs" : {
"colorSampler" : "COLOR",
"noiseSampler" : "noiseSymbol",
"a_position" : "a_position-symbol"
},
"program" : "displacement",
"draw" : "DRAW_QUAD"
}
},
"sequence" : [
"displacement"
],
"symbols" : {
"a_position-symbol" : {
"semantic" : "vertex"
},
"noiseSymbol" : {
"image" : "noise.png",
"type" : "sampler2D"
}
}
}
GLSL Vertex shader source code for displacement mapping technique
attribute vec4 a_position;
varying vec2 uv;
void main() {
gl_Position = a_position;
uv = (a_position.xy + 1.0) * 0.5;
}
GLSL Fragment shader source code for displacement mapping technique
uniform sampler2D colorSampler;
uniform sampler2D noiseSampler;
varying vec2 uv;
void main() {
vec2 displacement = texture2D(noiseSampler, uv).rg - vec2(0.5, 0.5);
gl_FragColor = texture2D(colorSampler, uv + displacement * vec2(0.1,0.1));
}