Function getLangchainStringEvaluator

Parameters

• type: "criteria" | "labeled_criteria"

  Type of string evaluator, one of "criteria" or "labeled_criteria
• options: EmbeddingDistanceEvalChainInput & {
      agentTools?: StructuredToolInterface[];
      chainOptions?: Partial<Omit<LLMEvalChainInput, "llm">>;
      criteria?: CriteriaLike;
      llm?: BaseLanguageModelInterface;
  } & {
      formatEvaluatorInputs?: (
          run: Run,
          example: Example,
      ) => { prediction: string; input?: string; reference?: string };
  }

  Options for loading the evaluator

  • OptionalagentTools?: StructuredToolInterface[]

    A list of tools available to the agent, for TrajectoryEvalChain.

  • OptionalchainOptions?: Partial<Omit<LLMEvalChainInput, "llm">>

  • Optionalcriteria?: CriteriaLike

    The criteria to use for the evaluator.

  • Optionalllm?: BaseLanguageModelInterface

  • OptionalformatEvaluatorInputs?: (
        run: Run,
        example: Example,
    ) => { prediction: string; input?: string; reference?: string }