Class

MPSRNNSingleGateDescriptor

A description of a simple recurrent block or layer.

Declaration

class MPSRNNSingleGateDescriptor : MPSRNNDescriptor

Overview

The recurrent neural network (RNN) layer initialized with a MPSRNNSingleGateDescriptor transforms the input data (image or matrix) and previous output with a set of filters. Each produces one feature map in the new output data.

You may provide the RNN unit with a single input or a sequence of inputs.

Description of Operation

  1. Let x_j be the input data (at time index t of sequence, j index containing quadruplet: batch index, x,y and feature index (x = y = 0 for matrices)).

  2. Let h0_j be the recurrent input (previous output) data from previous time step (at time index t-1 of sequence).

  3. Let h1_i be the output data produced at this time step.

  4. Let W_ij, U_ij be the weights for input and recurrent input data, respectively.

  5. Let b_i be a bias term.

  6. Let gi(x) be a neuron activation function.

The new output image h1_i data is computed as follows:

h1_i = gi( W_ij * x_j + U_ij * h0_j  + b_i )

The * stands for convolution (see MPSRNNImageInferenceLayer) or matrix-vector/matrix multiplication (see MPSRNNMatrixInferenceLayer).

Summation is over index j (except for the batch index), but there's no summation over repeated index i (the output index).

Note that for validity, all intermediate images must be of same size, and the U matrix must be square (that is, outputFeatureChannels == inputFeatureChannels). Also, the bias terms are scalars with regard to spatial dimensions.

Relationships

Inherits From

Conforms To

See Also

Recurrent Neural Networks

class MPSRNNImageInferenceLayer

A recurrent neural network layer for inference on Metal Performance Shaders images.

class MPSRNNMatrixInferenceLayer

A recurrent neural network layer for inference on Metal Performance Shaders matrices.

class MPSGRUDescriptor

A description of a gated recurrent unit block or layer.

class MPSLSTMDescriptor

A description of a long short-term memory block or layer.

enum MPSRNNSequenceDirection

Directions that a sequence of inputs can be processed by a recurrent neural network layer.

class MPSRNNMatrixTrainingLayer

A layer for training recurrent neural networks on Metal Performance Shaders matrices.

class MPSRNNMatrixTrainingState

A class that holds data from a forward pass to be used in a backward pass.