> ## Documentation Index
> Fetch the complete documentation index at: https://abstract-pbc.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Robot episodes

> Call get_action, execute the chunk, repeat.

Robots use [`POST /v1/get_action`](https://abstract-pbc.mintlify.site/robots/get-action), not [chat completions](https://abstract-pbc.mintlify.site/llm/chat-completions.md). An episode is a loop:

1. Send one frame per camera, the task, and the state fields the selected model requires.
2. Check the returned action space and execute its rows at `dt` seconds per row, capturing frames as you go.
3. Send those frames and the latest state with the same `session_id` and task.
4. Call `POST /v1/end_session` when done.

After the first call, send as many frames per camera as the last answer's `next_frames_per_camera`, oldest first.
An idle session expires after 60 seconds. `410 session_expired` means start a new episode with one frame per camera.

## Example

This example uses MolmoAct2's YAM end-effector contract. Replace the camera, state and execution
functions with your drivers. The example only reads files; it moves nothing.

```python episode.py theme={"theme":"css-variables"}
import base64
import os
import uuid

import requests

API = "https://api.abstract.inc/v1"
HEADERS = {"Authorization": f"Bearer {os.environ['ABSTRACT_API_KEY']}"}
MODEL = "allenai/molmoact2"
CAMERAS = {"overhead": "top", "wrist_l": "left", "wrist_r": "right"}
POSE = [0.3, 0.0, 0.2, -0.5, 0.0, -0.866, 0.0, 1.0, 0.0]


def capture(count):
    return {role: [base64.b64encode(open(f"{camera}.jpg", "rb").read()).decode()
                   for _ in range(count)] for role, camera in CAMERAS.items()}


def read_state():
    return {f"{side}_{field}": values for side in ("left", "right")
            for field, values in (("eef_xyz", POSE[:3]), ("eef_rot6", POSE[3:]), ("grip", [1.0]))}


def execute(actions, dt):
    pass  # Your controller checks and solves the poses, then plays one row every dt seconds.


session_id, frames = str(uuid.uuid4()), capture(1)
try:
    for step in range(3):
        response = requests.post(f"{API}/get_action", headers=HEADERS, json={
            "model": MODEL, "session_id": session_id,
            "observation_id": f"{session_id}.{step}", "expected_step": step,
            "observation": {"prompt": "fold the towel", "state": read_state(), "video": frames},
        }, timeout=300)
        if not response.ok:
            raise SystemExit(f"HTTP {response.status_code}: {response.text}")
        chunk = response.json()
        actions = [left + right for left, right in zip(
            chunk["actions"]["left_eef_chunk"], chunk["actions"]["right_eef_chunk"], strict=True)]
        execute(actions, chunk["dt"])
        frames = capture(chunk["next_frames_per_camera"])  # In production, capture during execution.
finally:
    requests.post(f"{API}/end_session", headers=HEADERS,
                  json={"model": MODEL, "session_id": session_id}, timeout=30)
```

## Retrying an observation

Choose a `session_id` before the first call. Send a unique `observation_id` and the next
`expected_step` (starting at zero) with each observation. Send only one call at a time.

If an answer is lost, retry the identical body under the same id. A replay does not run or bill another chunk.
While it runs, `409 observation_in_progress` carries `Retry-After`: wait, then retry the same id and body.
`observation_lost` means that id will never run again: do not execute a missing chunk, and send the next observation under a new id.
`observation_outcome_unknown`, from a legacy gateway, means retry only that id. Never send a new observation until its outcome is known.
`Idempotency-Key` can replace `observation_id`; if both are sent they must match.
The example stops on errors rather than retrying. See [Errors](https://abstract-pbc.mintlify.site/errors.md) for retry advice.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.