<!--
{
  "availability" : [
    "Xcode: 27.0.0 -",
    "iOS: 27.0.0 -",
    "iPadOS: 27.0.0 -",
    "macCatalyst: 27.0.0 -",
    "macOS: 27.0.0 -",
    "visionOS: 27.0.0 -",
    "watchOS: 27.0.0 -"
  ],
  "documentType" : "symbol",
  "framework" : "Evaluations",
  "identifier" : "/documentation/Evaluations/EvaluatorProtocol",
  "metadataVersion" : "0.1.0",
  "role" : "Protocol",
  "symbol" : {
    "kind" : "Protocol",
    "modules" : [
      "Evaluations"
    ],
    "preciseIdentifier" : "s:11Evaluations17EvaluatorProtocolP"
  },
  "title" : "EvaluatorProtocol"
}
-->

# EvaluatorProtocol

A type that evaluates subjects and produces metrics.

```
protocol EvaluatorProtocol<Input, Subject> : Sendable
```

## Overview

Conform to `EvaluatorProtocol` to create custom evaluators that measure
the system’s output against expected criteria. Each evaluator returns an
array of [`Metric`](/documentation/Evaluations/Metric) values: one for each DataFrame column.

The protocol takes `Input` (the sample type) as a type parameter. `Subject` is
an associated type constrained to [`EvaluationSubject`](/documentation/Evaluations/EvaluationSubject), ensuring the
subject’s value type matches the sample’s expected value type.

Conforming types must be `Sendable`.

```swift
struct MyEvaluator<Input: SampleProtocol>: EvaluatorProtocol
where Input.ExpectedValue: Sendable & Codable {
    let metric = Metric("Quality")

    func metrics(
        subject: ModelSubject<Input.ExpectedValue>,
        input: Input
    ) async throws -> [Metric] {
        return [metric.scoring(1.0)]
    }
}
```

## Topics

### Associated Types

[`associatedtype Input : SampleProtocol`](/documentation/Evaluations/EvaluatorProtocol/Input)

The input sample type.

[`associatedtype Subject : EvaluationSubject`](/documentation/Evaluations/EvaluatorProtocol/Subject)

The type of subject the system under test produces.

### Instance Methods

[`func metrics(subject: Self.Subject, input: Self.Input) async throws -> [Metric]`](/documentation/Evaluations/EvaluatorProtocol/metrics(subject:input:))

Computes metrics for the given subject, given the input sample.

## Relationships

### Inherits From

[`SendableMetatype`](/documentation/Swift/SendableMetatype)

[`Sendable`](/documentation/Swift/Sendable)

### Conforming Types

[`Evaluator`](/documentation/Evaluations/Evaluator)

[`ModelJudgeEvaluator`](/documentation/Evaluations/ModelJudgeEvaluator)

[`ToolCallEvaluator`](/documentation/Evaluations/ToolCallEvaluator)

---

Copyright &copy; 2026 Apple Inc. All rights reserved. | [Terms of Use](https://www.apple.com/legal/internet-services/terms/site.html) | [Privacy Policy](https://www.apple.com/privacy/privacy-policy)