POST
/
api
/
v1
/
compare
cURL
curl --request POST \
  --url https://labs.tacitintelligence.co/api/v1/compare \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "collection_slug": "<string>",
  "scenario_slug": "<string>",
  "response_a": {
    "messages": [
      {
        "content": "<string>",
        "tool_calls": [
          {
            "name": "<string>",
            "input": {}
          }
        ]
      }
    ]
  },
  "response_b": {
    "messages": [
      {
        "content": "<string>",
        "tool_calls": [
          {
            "name": "<string>",
            "input": {}
          }
        ]
      }
    ]
  }
}
'
{
  "reward_a": 0.5,
  "reward_b": 0.5,
  "margin": 123,
  "scoring_version": "<string>"
}

Authorizations

​
Authorization
string
header
required

API key obtained from Labs Portal

Body

application/json
​
collection_slug
string
required

The collection slug containing the scenario

​
scenario_slug
string
required

The scenario slug to compare responses for

​
response_a
object
required

First response to compare

​
response_b
object
required

Second response to compare

Response

Success

​
winner
enum<string>
required

Which response won: a, b, or tie

Available options:
a,
b,
tie
​
reward_a
number
required

Reward score for response_a from 0 to 1

Required range: 0 <= x <= 1
​
reward_b
number
required

Reward score for response_b from 0 to 1

Required range: 0 <= x <= 1
​
margin
number
required

Absolute difference between reward_a and reward_b

​
scoring_version
string
required

Version of the scoring algorithm used