Evaluation

In development — not production-ready

Evaluation is still under development. The features on this page may change, and they are not yet ready for production use.

Systematically score your agents with LLM judges over curated datasets.

Datasets

Build datasets of input/expectation records — including from real traces (add a trace as a record in one click). Organize them into folders and mark favorites. Import, export, and edit records inline or as JSON.

Judges

Define LLM judges by type — safety, correctness, tool-call efficiency, tool-call correctness — each backed by a built-in prompt (editable on the Settings page). Choose a provider (SAP AI Core, Anthropic, Gemini) and model.

Evaluation runs

Attach judges to a dataset and start an evaluation. Runs are asynchronous — the dashboard polls progress every couple of seconds — and produce per-record, per-judge results with pass rates and rationales. Filter and sort runs by dataset, status, date, and tokens.

For a fast check without saving anything, quick-eval a single trace directly from its detail view.