RunEvaluator

class langsmith.evaluation.evaluator.RunEvaluator

Evaluator interface class.

Methods

__init__()
aevaluate_run(run[, example])	Evaluate an example asynchronously.
evaluate_run(run[, example])	Evaluate an example.

__init__()

async aevaluate_run(run: Run, example: Example | None = None) → EvaluationResult | EvaluationResults

Evaluate an example asynchronously.

Parameters:

• run (Run) –

• example (Example | None) –

Return type:

EvaluationResult | EvaluationResults

abstract evaluate_run(run: Run, example: Example | None = None) → EvaluationResult | EvaluationResults

Evaluate an example.

Parameters:

• run (Run) –

• example (Example | None) –

Return type:

EvaluationResult | EvaluationResults