> ## Documentation Index
> Fetch the complete documentation index at: https://judged.systems/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Experiments

> Store a dataset and judge it with a published pack. The app screen is on the roadmap.

A dataset is a stored set of tickets. A simulation judges that dataset with a pack's published version. The API is available. The Experiments item in the app is disabled and labeled on the roadmap.

<Note>
  Create datasets and start simulations over the API. The screen at `/experiments` does not open.
</Note>

## Datasets

`POST /api/datasets` with `{ "name": "May refunds" }` requires `packs:write` and returns `{ id, name }`.

`GET /api/datasets` requires `packs:read` and returns `{ items: [{ id, name, createdAt, ticketCount }] }`.

Add tickets with `POST /api/datasets/{datasetId}/tickets`. Tickets are redacted before they are stored. A dataset holds at most 250 tickets. Going past the cap is `400` with `dataset is capped at 250 tickets`.

<Tabs>
  <Tab title="JSON">
    ```bash theme={null}
    curl https://judged.systems/api/datasets/DATASET_ID/tickets \
      -X POST \
      -H "x-api-key: $JUDGED_API_KEY" \
      -H "content-type: application/json" \
      -d '{
        "tickets": [
          {
            "subject": "Charged twice for May",
            "customerMessage": "I was billed twice and want one refunded."
          }
        ]
      }'
    ```
  </Tab>

  <Tab title="CSV">
    The body is `text/csv`. The header row must include `subject` and `customerMessage`. `externalId` is optional.

    ```bash theme={null}
    curl https://judged.systems/api/datasets/DATASET_ID/tickets \
      -X POST \
      -H "x-api-key: $JUDGED_API_KEY" \
      -H "content-type: text/csv" \
      --data-binary $'subject,customerMessage,externalId\nCharged twice,I was billed twice,48213\n'
    ```

    A header without those columns is `400` with `csv header must include subject,customerMessage`.
  </Tab>
</Tabs>

`201` returns `{ "inserted": 1, "total": 1 }`.

## Simulations

`POST /api/simulations` requires `simulations:run`:

```json theme={null}
{ "datasetId": "DATASET_ID", "packId": "PACK_ID" }
```

`202` returns `{ "id": "..." }` before the run finishes. The pack must be published, or the response is `409`.

`GET /api/simulations/{id}` returns the run:

| Field | Meaning |
| - | - |
| `status` | `queued`, `running`, `complete`, or `failed` |
| `error` | Set when `status` is `failed`, otherwise null |
| `total` / `done` | Tickets in the dataset, and how many have a stored evaluation |
| `reviewRate` | Share of finished rows whose status is `review` |
| `avgLatencyMs` | Mean latency of finished rows |
| `totalTokens` | Input plus output tokens recorded on those rows |
| `evaluations` | `{ id, status, ticketHash }` for each stored row |

`GET /api/simulations` lists runs with dataset and pack names. Each evaluation is also a normal evaluation with source `simulation`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.