Interface EvaluateOptions

interface EvaluateOptions {
    data: DataT;
    client?: Client;
    description?: string;
    evaluators?: EvaluatorT[];
    experimentPrefix?: string;
    includeAttachments?: boolean;
    maxConcurrency?: number;
    metadata?: KVMap;
    numRepetitions?: number;
    summaryEvaluators?: SummaryEvaluatorT[];
}

Hierarchy

  • BaseEvaluateOptions
    • EvaluateOptions

Properties

data client? description? evaluators? experimentPrefix? includeAttachments? maxConcurrency? metadata? numRepetitions? summaryEvaluators?

Properties

data

data: DataT

The dataset to evaluate on. Can be a dataset name, a list of examples, or a generator of examples.

Optionalclient

client?: Client

The LangSmith client to use.

Default

undefined

Optionaldescription

description?: string

A free-form description of the experiment.

Optionalevaluators

evaluators?: EvaluatorT[]

A list of evaluators to run on each example.

Default

undefined

OptionalexperimentPrefix

experimentPrefix?: string

A prefix to provide for your experiment name.

Default

undefined

OptionalincludeAttachments

includeAttachments?: boolean

Whether to use attachments for the experiment.

Default

false

OptionalmaxConcurrency

maxConcurrency?: number

The maximum number of concurrent evaluations to run.

Default

undefined

Optionalmetadata

metadata?: KVMap

Metadata to attach to the experiment.

Default

undefined

OptionalnumRepetitions

numRepetitions?: number

The number of repetitions to perform. Each example will be run this many times.

Default

1

OptionalsummaryEvaluators

summaryEvaluators?: SummaryEvaluatorT[]

A list of summary evaluators to run on the entire dataset.

Default

undefined

Settings

Member Visibility

On This Page

Properties