POST
/
api
/
v1
/
batch
/
evaluate
cURL
curl --request POST \
  --url https://labs.tacitintelligence.co/api/v1/batch/evaluate \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "items": [
    {
      "id": "<string>",
      "collection_slug": "<string>",
      "scenario_slug": "<string>",
      "messages": [
        {
          "content": "<string>",
          "tool_calls": [
            {
              "name": "<string>",
              "input": {}
            }
          ]
        }
      ],
      "normalization_key": "<string>",
      "normalization_windows": [
        "<string>"
      ]
    }
  ]
}
'
{
  "results": [
    {
      "id": "<string>",
      "status": "<string>",
      "reward": 0.5,
      "scores": {},
      "score_breakdown": {},
      "normalized": {}
    }
  ],
  "scoring_version": "<string>",
  "total_time_ms": 123
}

Authorizations

​
Authorization
string
header
required

API key obtained from Labs Portal

Body

application/json
​
items
object[]
required

Array of evaluation items (1-100 items per batch)

Required array length: 1 - 100 elements

Response

Success

​
results
object[]
required

Results for each item in the batch

Each result is either a success (with reward and scores) or an error (with error message). Check the status field to distinguish - error items have no reward and must not be used for training.

​
scoring_version
string
required

Version of the scoring algorithm used

​
total_time_ms
number
required

Total time to process the batch in milliseconds