Loads the suite, scores both variants OUTSIDE the write transaction (two-phase — judge I/O never holds the store), optionally runs the order-swapped pairwise comparison, then persists both runs atomically.
POST
/v1/m/evals/ab
const url = 'https://example.com/v1/m/evals/ab';const options = { method: 'POST', headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'}, body: '{"a":{"label":"example","outputs":{"additionalProperty":"example"}},"b":{"label":"example","outputs":{"additionalProperty":"example"}},"pairwise":true,"subject_kind":"example","subject_ref":"example","suite_ref":"example"}'};
try { const response = await fetch(url, options); const data = await response.json(); console.log(data);} catch (error) { console.error(error);}curl --request POST \ --url https://example.com/v1/m/evals/ab \ --header 'Authorization: Bearer <token>' \ --header 'Content-Type: application/json' \ --data '{ "a": { "label": "example", "outputs": { "additionalProperty": "example" } }, "b": { "label": "example", "outputs": { "additionalProperty": "example" } }, "pairwise": true, "subject_kind": "example", "subject_ref": "example", "suite_ref": "example" }'Loads the suite, scores both variants OUTSIDE the write transaction (two-phase — judge I/O never holds the store), optionally runs the order-swapped pairwise comparison, then persists both runs atomically.
Authorizations
Section titled “Authorizations ”Parameters
Section titled “ Parameters ”Header Parameters
Section titled “Header Parameters ” X-Olivares-Tenant
string format: uuid
Target tenant id; required when the principal can act in more than one tenant.
Request Body required
Section titled “Request Body required ” Media type application/json
object
a
required
object
label
string
outputs
required
object
key
additional properties
string
b
required
object
label
string
outputs
required
object
key
additional properties
string
pairwise
boolean
subject_kind
string
subject_ref
string
suite_ref
required
string
Example generated
{ "a": { "label": "example", "outputs": { "additionalProperty": "example" } }, "b": { "label": "example", "outputs": { "additionalProperty": "example" } }, "pairwise": true, "subject_kind": "example", "subject_ref": "example", "suite_ref": "example"}Responses
Section titled “ Responses ”OK
Media type application/json
object
Example generated
{}Bad request
Media type application/json
object
Example generated
{}Unauthenticated
Media type application/json
object
Example generated
{}Forbidden
Media type application/json
object
Example generated
{}Not found
Media type application/json
object
Example generated
{}Conflict / setup required
Media type application/json
object
Example generated
{}Rate limited
Media type application/json
object
Example generated
{}