Loads the suite, scores both variants OUTSIDE the write transaction (two-phase — judge I/O never holds the store), optionally runs the order-swapped pairwise comparison, then persists both runs atomically.

POST

/v1/m/evals/ab

curl --request POST \
  --url https://example.com/v1/m/evals/ab \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{ "a": { "baseline_ref": "example", "comparison": { "mode": "same_candidate", "version": 1 }, "label": "example", "model_ref": "example", "outputs": { "additionalProperty": "example" } }, "b": { "baseline_ref": "example", "comparison": { "mode": "same_candidate", "version": 1 }, "label": "example", "model_ref": "example", "outputs": { "additionalProperty": "example" } }, "pairwise": true, "subject_kind": "example", "subject_ref": "example", "suite_ref": "example" }'

• this engine

Loads the suite, scores both variants OUTSIDE the write transaction (two-phase — judge I/O never holds the store), optionally runs the order-swapped pairwise comparison, then persists both runs atomically.

Authorizations

• bearerAuth
• browserSession

Parameters

Header Parameters

X-Olivares-Tenant

string format: uuid

Target tenant id; required when the principal can act in more than one tenant.

Request Body ^required

Media type application/json

object

a

required

object

baseline_ref

string

comparison

object

mode

required

Candidate_change requires an explicit baseline_ref and declared candidate identities.

string

Allowed values: same_candidate candidate_change

version

required

integer

Allowed values: 1

label

string

model_ref

string

outputs

required

object

key

additional properties

string

b

required

object

baseline_ref

string

comparison

object

mode

required

Candidate_change requires an explicit baseline_ref and declared candidate identities.

string

Allowed values: same_candidate candidate_change

version

required

integer

Allowed values: 1

label

string

model_ref

string

outputs

required

object

key

additional properties

string

pairwise

boolean

subject_kind

string

subject_ref

string

suite_ref

required

string

>= 1 characters

Responses

200

OK

Media type application/json

object

Example ^generated

{}

400

Bad request

Media type application/json

object

Example ^generated

{}

401

Unauthenticated

Media type application/json

object

Example ^generated

{}

403

Forbidden

Media type application/json

object

Example ^generated

{}

404

Not found

Media type application/json

object

Example ^generated

{}

409

Conflict / setup required

Media type application/json

object

Example ^generated

{}

429

Rate limited

Media type application/json

object

Example ^generated

{}