Convolutional Neural Network Kernels

Build neural networks with layers.

Overview

  • Think carefully about the edge mode requested for pooling layers. The default value is MPSImageEdgeModeZero, but there are times when a MPSImageEdgeModeClamp value may be better.

  • To avoid reading off the edge of an image for filters that have a filter area (convolution, pooling), set MPSCNNKernel.offset = (MPSOffset){ .x = kernelWidth/2, .y = kernelHeight/2, .z = 0} and reduce the size of the output image by {kernelWidth-1, kernelHeight-1, 0}. The filter area stretches up and to the left of the kernel offset by {kernelWidth/2, kernelHeight/2}.

  • Always remember the following distinction:

    • The MPSCNNConvolution class takes weights in the order weight[outputChannels][kernelHeight][kernelWidth][inputChannels/groups].

    • The MPSCNNFullyConnected class takes weights in the order weight[outputChannels][sourceWidth][sourceHeight][inputChannels].

  • Initialize MPSCNNKernel objects once and reuse them.

  • You can use MPSCNNNeuron objects and similar to perform pre-processing of images, such as scaling and resizing.

  • Specify a neuron filter with an MPSCNNConvolutionDescriptor object to combine the convolution and neuron operations.

  • Use MPSTemporaryImage objects for intermediate images that live for a short period of time (one MTLCommandBuffer object).

    MPSTemporaryImage objects can reduce the amount of memory used by the CNN by several folds, and similarly reduce the amount of CPU time spent allocating storage and latency between the time a command buffer is committed and when it is actually executed on the GPU.

    You cannot read or write to a MPSTemporaryImage object using the CPU. Generally, MPSTemporaryImage objects should be created as needed and thrown away promptly. Persistent objects should not retain them.

    Please be sure to understand the purpose of the readCount property.

  • Because the Metal Performance Shaders framework encodes its work in place in your command buffer, you always have the option to insert your own code in between MPSCNNKernel encodings as a Metal function for tasks not covered by the framework. You do not need to use the Metal Performance Shaders framework for everything.

Topics

Arithmetic Layers

MPSCNNAdd

An addition operator.

MPSCNNAddGradient

A gradient addition operator.

MPSCNNSubtract

A subtraction operator.

MPSCNNSubtractGradient

A gradient subtraction operator.

MPSCNNMultiply

A multiply operator.

MPSCNNMultiplyGradient

A gradient multiply operator.

MPSCNNDivide

A division operator.

MPSCNNArithmetic

The base class for arithmetic operators.

MPSCNNArithmeticGradient

The base class for gradient arithmetic operators.

MPSCNNArithmeticGradientState

An object that stores the clamp mask used by gradient arithmetic operators.

Convolution Layers

MPSCNNBinaryConvolution

A convolution kernel with binary weights and an input image using binary approximations.

MPSCNNConvolution

A convolution kernel that convolves the input image with a set of filters, with each producing one feature map in the output image.

MPSCNNDepthWiseConvolutionDescriptor

A description of a convolution object that does depthwise convolution.

MPSCNNSubPixelConvolutionDescriptor

A description of a convolution object that does subpixel upsampling and reshaping.

MPSCNNConvolutionTranspose

A transposed convolution kernel.

MPSCNNConvolutionGradient

A gradient convolution kernel.

MPSCNNConvolutionGradientState

An object that exposes a gradient convolution kernel's gradient with respect to weights and biases.

MPSImageSizeEncodingState

A protocol for objects that contain information about an image size elsewhere in the graph.

MPSCNNConvolutionWeightsAndBiasesState

A class that stores weights and biases.

Pooling Layers

MPSCNNPoolingAverage

An average pooling filter.

MPSCNNPoolingAverageGradient

A gradient average pooling filter.

MPSCNNPoolingL2Norm

An L2-norm pooling filter.

MPSCNNPoolingMax

A max pooling filter.

MPSCNNDilatedPoolingMax

A dilated max pooling filter.

MPSCNNPooling

A pooling kernel.

MPSCNNPoolingGradient

A gradient pooling kernel.

MPSCNNDilatedPoolingMaxGradient

A gradient dilated max pooling filter.

MPSCNNPoolingL2NormGradient

A gradient L2-norm pooling filter.

MPSCNNPoolingMaxGradient

A gradient max pooling filter.

Fully Connected Layers

MPSCNNBinaryFullyConnected

A fully connected convolution layer with binary weights and optionally binarized input image.

MPSCNNFullyConnected

A fully connected convolution layer, also known as an inner product layer.

MPSCNNFullyConnectedGradient

A gradient fully connected convolution layer.

Neuron Layers

MPSCNNNeuronAbsolute

An absolute neuron filter.

MPSCNNNeuronELU

A parametric ELU neuron filter.

MPSCNNNeuronHardSigmoid

A hard sigmoid neuron filter.

MPSCNNNeuronLinear

A linear neuron filter.

MPSCNNNeuronPReLU

A parametric ReLU (Rectified Linear Unit) neuron filter.

MPSCNNNeuronReLUN

A ReLUN neuron filter.

MPSCNNNeuronReLU

A ReLU (Rectified Linear Unit) neuron filter.

MPSCNNNeuronSigmoid

A sigmoid neuron filter.

MPSCNNNeuronSoftPlus

A parametric softplus neuron filter.

MPSCNNNeuronSoftSign

A softsign neuron filter.

MPSCNNNeuronTanH

A hyperbolic tangent neuron filter.

MPSCNNNeuron

A filter that applies a neuron activation function.

MPSCNNNeuronExponential

An exponential neuron filter.

MPSCNNNeuronGradient

A gradient neuron filter.

MPSCNNNeuronLogarithm

A logarithm neuron filter.

MPSCNNNeuronPower

A power neuron filter.

MPSNNNeuronDescriptor

An object that specifies properties used by a neuron kernel.

Softmax Layers

MPSCNNSoftMax

A neural transfer function that is useful for classification tasks.

MPSCNNLogSoftMax

A neural transfer function that is useful for constructing a loss function to be minimized when training neural networks.

MPSCNNLogSoftMaxGradient

A gradient logarithmic softmax filter.

MPSCNNSoftMaxGradient

A gradient softmax filter.

Normalization Layers

MPSCNNCrossChannelNormalization

A normalization kernel applied across feature channels.

MPSCNNCrossChannelNormalizationGradient

A gradient normalization kernel applied across feature channels.

MPSCNNLocalContrastNormalization

A local-contrast normalization kernel.

MPSCNNLocalContrastNormalizationGradient

A gradient local-contrast normalization kernel.

MPSCNNSpatialNormalization

A spatial normalization kernel.

MPSCNNSpatialNormalizationGradient

A gradient spatial normalization kernel.

MPSCNNBatchNormalization

A batch normalization kernel.

MPSCNNBatchNormalizationGradient

A gradient batch normalization kernel.

MPSCNNBatchNormalizationState

An object that stores data required to execute batch normalization.

MPSCNNNormalizationMeanAndVarianceState

An object that stores mean and variance terms used to execute batch normalization.

MPSCNNBatchNormalizationStatistics

An object that stores statistics required to execute batch normalization.

MPSCNNBatchNormalizationStatisticsGradient

An object that stores the gradient of the loss function with respect to the batch statistics and batch normalization weights.

MPSCNNInstanceNormalization

An instance normalization kernel.

MPSCNNInstanceNormalizationGradient

A gradient instance normalization kernel.

MPSCNNInstanceNormalizationGradientState

An object that stores information required to execute a gradient pass for instance normalization.

MPSCNNNormalizationGammaAndBetaState

An object that stores gamma and beta terms used to apply a scale and bias in instance- or batch-normalization operations.

Upsampling Layers

MPSCNNUpsampling

A filter that resamples an existing MPS image.

MPSCNNUpsamplingBilinear

A bilinear spatial upsampling filter.

MPSCNNUpsamplingNearest

A nearest spatial upsampling filter.

MPSCNNUpsamplingBilinearGradient

A gradient bilinear spatial upsampling filter.

MPSCNNUpsamplingGradient

A gradient filter that upsamples an existing Metal Performance Shaders image.

MPSCNNUpsamplingNearestGradient

A gradient upsampling filter that samples the pixel nearest to the source when upsampling to the destination pixel.

Dropout Layers

MPSCNNDropout

A dropout filter.

MPSCNNDropoutGradient

A gradient dropout filter.

MPSCNNDropoutGradientState

A class that stores the mask used by dropout and gradient dropout filters.

Loss Layers

MPSCNNLoss

A kernel that computes the loss and loss gradient between specified predictions and labels.

MPSCNNLossDataDescriptor

An object that specifies properties used by a loss data descriptor.

MPSCNNLossDescriptor

An object that specifies properties used by a loss kernel.

MPSCNNLossLabels

A class that stores the per-element weight buffer used by loss and gradient loss kernels.

MPSCNNYOLOLoss

A kernel that computes the YOLO loss and loss gradient between specified predictions and labels.

MPSCNNYOLOLossDescriptor

An object that specifies properties used by a YOLO loss kernel.

Reduction Layers

MPSNNReduceRowMax

A reduction filter that returns the maximum value for each row in an image.

MPSNNReduceRowMin

A reduction filter that returns the minimum value for each row in an image.

MPSNNReduceRowSum

A reduction filter that returns the sum of all values for each row in an image.

MPSNNReduceRowMean

A reduction filter that returns the mean value for each row in an image.

MPSNNReduceColumnMax

A reduction filter that returns the maximum value for each column in an image.

MPSNNReduceColumnMin

A reduction filter that returns the minimum value for each column in an image.

MPSNNReduceColumnSum

A reduction filter that returns the sum of all values for each column in an image.

MPSNNReduceColumnMean

A reduction filter that returns the mean value for each column in an image.

MPSNNReduceFeatureChannelsMax

A reduction filter that returns the maximum value for each feature channel in an image.

MPSNNReduceFeatureChannelsMin

A reduction filter that returns the minimum value for each feature channel in an image.

MPSNNReduceFeatureChannelsSum

A reduction filter that returns the sum of all values for each feature channel in an image.

MPSNNReduceFeatureChannelsMean

A reduction filter that returns the mean value for each feature channel in an image.

MPSNNReduceFeatureChannelsArgumentMax

A reduction filter that returns the index of the location of the maximum value for each feature channel in an image.

MPSNNReduceFeatureChannelsArgumentMin

A reduction filter that returns the index of the location of the minimum value for each feature channel in an image.

MPSNNReduceFeatureChannelsAndWeightsSum

A reduction filter that returns the weighted sum of all values for each feature channel in an image.

MPSNNReduceFeatureChannelsAndWeightsMean

A reduction filter that returns the weighted sum for each feature channel in an image.

MPSNNReduceUnary

The base class for unary reduction filters.

MPSNNReduceBinary

The base class for binary reduction filters.

Reshape Layer

MPSNNReshape

The base class for reshape operations.

Slice Layer

MPSNNSlice

A kernel that extracts a slice from an image.

Optimization Layers

MPSNNOptimizerAdam

An optimization layer that performs an Adam pdate.

MPSNNOptimizerRMSProp

An optimization layer that performs a root mean square propagation update.

MPSNNOptimizerStochasticGradientDescent

An optimization layer that performs a gradient descent with an optional momentum update.

MPSNNOptimizer

The base class for optimization layers.

MPSNNOptimizerDescriptor

An object that specifies properties used by an optimizer kernel.

Layer Base Classes

MPSCNNKernel

Base class for neural network layers.

MPSCNNBinaryKernel

A convolution neural network kernel.

MPSCNNGradientKernel

The base class for gradient layers.

Predefined Padding Policies

MPSNNDefaultPadding

A class that provides predefined padding policies for common tasks.

See Also

Neural Networks

Training a Neural Network with Metal Performance Shaders

Use an MPS neural network graph to train a simple neural network digit classifier.

MPSImage

A texture that may have more than four channels for use in convolutional neural networks.

MPSTemporaryImage

A texture for use in convolutional neural networks that stores transient data to be used and discarded promptly.

Objects that Simplify the Creation of Neural Networks

Simplify the creation of neural networks using networks of filter, image, and state nodes.

Recurrent Neural Networks

Create recurrent neural networks.