POST
cURL

Store scored turns

Send score_turns: true to score each turn. Add persist_turns: true to save those turns’ fact readings as scoring runs. Both options default to false. Send these options as top-level request fields. Grading attempts use the turn request’s store_attempt_run flag instead of persist_turns.

Continue from a stored baseline

Add baseline_run_id when restoring a recorded conversation:
The baseline must belong to your organization, the recording, its scenario, and the selected turn boundary. The response includes baseline.run_id, baseline.overall_value on the 0 to 100 scale, and baseline.reward on the 0 to 1 scale. Each scored turn of the continuation uses the scenario’s current compiled scoring tree and keeps the baseline’s settled fact readings. The API refuses to score a turn when the current fact pins (each fact’s id and content hash) or the grading protocol digest differ from the baseline’s. It also refuses when the current scoring tree differs from the baseline’s. Use the baseline’s judge on subsequent turns. Baseline attempts reject reward_config overrides.

Weight the criteria for one task

Add a weights table beside baseline_run_id when a task values some criteria more than others. The table maps a criterion id to a positive number. Every criterion you do not name keeps weight 1.
The table changes how the scoring tree combines readings. It never changes the readings themselves, and it never asks the judge to read a fact again. The API applies the table in two places so both sides of a grading attempt use the same valuation:
  • The response’s baseline.overall_value and baseline.reward are composed from the stored baseline readings under the table.
  • Each scored turn’s reward, raw_overall_value and raw_reward are composed from that turn’s readings under the table. The per-fact observations at the full score access tier stay exactly as the judge read them.
Each criterion id must belong to the scenario’s compiled scoring set; the ids are the criterion_ids the evaluate configuration endpoint reports for the scenario. The API answers 400 and names every id it cannot place. A criterion the tree cannot weight, such as a safety gate, is also refused by name. Weights must be above zero; omit a criterion to keep it at weight 1. The table is pinned in the episode and bound into the grading identity hash, so two episodes that differ only in their weights never share a grading receipt. A grading attempt stored with store_attempt_run records the table beside its overall. weights requires baseline_run_id.

Authorizations

Authorization
string
header
required

API key obtained from Labs Portal

Body

application/json
collection_slug
string
required

The collection slug containing the scenario

scenario_slug
string

The scenario slug to start an episode for. Required unless from_capture_session is given; when both are given they must name the same scenario.

from_capture_session
object

Start from a recorded conversation at a chosen client message. The person's state at that point is rebuilt from the recording (what it disclosed, its findings) and the recording itself is never changed. The scenario comes from the recording, so scenario_slug may be omitted.

baseline_run_id
string<uuid>

Stored baseline run for a scored continuation from a recording.

Pattern: ^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-8][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$
weights
object

Per-task criterion weights: criterion id to a positive weight. Every criterion not named keeps weight 1. Requires baseline_run_id. Both the stored baseline and each scored turn are composed with this table over the stored per-fact readings, so changing it never re-judges a fact. Every id must belong to the compiled scoring set, or the request is refused.

persist_turns
boolean

Store fact readings for scored turns. Defaults to false. Grading attempts use store_attempt_run on the turn request instead.

score_turns
boolean

Whether to score the conversation on each turn. Defaults to false. When false, per-turn reward is 0 and scores are omitted. The /evaluate endpoint can be used for final scoring.

config
object

Optional configuration overrides for the episode

Response

Success

episode_id
string
required

Unique identifier for the created episode

initial_observation
object
required

The starting observation for the episode

tools
object[]
required

Tools available for this scenario

baseline
object

Validated stored baseline for a restored grading attempt.

checkpoint_sha256
string

Only for an episode started from a recorded conversation: the SHA-256 of the restored checkpoint (seeded history, rebuilt persona state, scenario, collection, source capture session and boundary message). Two episodes restored from the same boundary share it.