<!--
{
  "availability" : [
    "iOS: 14.0.0 - 17.4.0",
    "iPadOS: 14.0.0 - 17.4.0",
    "macCatalyst: 14.0.0 - 17.4.0",
    "macOS: 11.0.0 - 14.3.0",
    "tvOS: 14.0.0 - 17.4.0"
  ],
  "documentType" : "symbol",
  "framework" : "MLCompute",
  "identifier" : "/documentation/MLCompute/MLCMultiheadAttentionLayer",
  "metadataVersion" : "0.1.0",
  "role" : "Class",
  "symbol" : {
    "kind" : "Class",
    "modules" : [
      "ML Compute"
    ],
    "preciseIdentifier" : "c:objc(cs)MLCMultiheadAttentionLayer"
  },
  "title" : "MLCMultiheadAttentionLayer"
}
-->

# MLCMultiheadAttentionLayer

A multihead, scaled dot-product attention layer that attends to one or more entries in the input key-value pairs.

```
class MLCMultiheadAttentionLayer
```

## Overview

The dimensions of projections are as follows:

- Query: `(1, headCount, keyDimension/headCount, modelDimension)`
- Key: `(1, headCount, keyDimension/headCount, modelDimension)`
- Value: `(1, headCount, valueDimension/headCount, modelDimension)`
- Output: `(1, 1, modelDimension, valueDimension)`

``

## Topics

### Creating Multi-Head Attention Layers

[`init(descriptor:weights:biases:attentionBiases:)`](/documentation/MLCompute/MLCMultiheadAttentionLayer/init(descriptor:weights:biases:attentionBiases:))

Creates a multi-head attention layer with the descriptor, weights, and biases you specify.

[`MLCMultiheadAttentionDescriptor`](/documentation/MLCompute/MLCMultiheadAttentionDescriptor)

A configuration object you use to create a multi-head attention layer.

### Inspecting Multi-Head Attention Layers

[`descriptor`](/documentation/MLCompute/MLCMultiheadAttentionLayer/descriptor)

The configuration object you use to create the multi-head attention layer.

[`weights`](/documentation/MLCompute/MLCMultiheadAttentionLayer/weights)

The array of weights you use for query, key, value, and output projections.

[`biases`](/documentation/MLCompute/MLCMultiheadAttentionLayer/biases)

The array of biases you use for query, key, value, and output projections.

[`attentionBiases`](/documentation/MLCompute/MLCMultiheadAttentionLayer/attentionBiases)

The array of attention biases you use for key and value.

[`weightsParameters`](/documentation/MLCompute/MLCMultiheadAttentionLayer/weightsParameters)

The array of weights tensor parameters you use for optimizer updates.

[`biasesParameters`](/documentation/MLCompute/MLCMultiheadAttentionLayer/biasesParameters)

The array of biases tensor parameters you use for optimizer updates.


---

Copyright &copy; 2026 Apple Inc. All rights reserved. | [Terms of Use](https://www.apple.com/legal/internet-services/terms/site.html) | [Privacy Policy](https://www.apple.com/privacy/privacy-policy)