A fully connected convolution layer, also known as an inner product layer.
SDKs
- iOS 10.0+
- macOS 10.13+
- Mac Catalyst 13.0+
- tvOS 10.0+
Framework
- Metal Performance Shaders
Declaration
@interface MPSCNNFullyConnected : MPSCNNConvolution
Overview
A fully connected layer in a Convolutional Neural Network (CNN) is one where every input channel is connected to every output channel. The kernel width is equal to the width of the source image, and the kernel height is equal to the height of the source image. The width and height of the output is 1 x 1
.
A fully connected layer takes an MPSImage
object with dimensions source
, convolves it with Weights[No][source
,
and produces a 1 x 1 x No
output.
Thus, the following conditions must be true:
kernel
Width == source .width kernel
Height == source .height clip
Rect .size .width == 1 clip
Rect .size .height == 1
You can think of a fully connected layer as a matrix multiplication where the image is flattened into a vector of length source
, and the weights are arranged in a matrix of dimension No x (source
to produce an output vector of length No
.
The value of the strideInPixelsX, stride
, and groups
properties must be 1
. The offset
property is not applicable and it is ignored. Because the clip rectangle is clamped to the destination image bounds, if the destination is 1 x 1
, you do not need to set the clip
property.
Note
You can implement a fully connected convolution layer using an MPSCNNConvolution
object by setting the following property values:
offset = (kernel
clip
clip
stride
However, using an MPSCNNFully
object directly is better for performance as it lets the Metal Performance Shaders framework choose the most performant implementation method, which may not be possible when you use a general convolution. For example, the framework may internally use matrix multiplication or special reduction kernels for a specific Metal feature set.