Robot actions

Turn one observation into an action chunk

One metered action chunk, using the selected model’s observation and action contract. YAM models take grasp-site poses and grippers and refuse joints. Xiaomi’s ARX X5 checkpoint additionally needs measured joints and uses RoboDojo/world-frame poses. FLUX’s DROID checkpoint takes seven Franka joints and one gripper closed fraction, with its own camera names, and returns native joint targets. Both native checkpoints are API-only. verne/v0.5 takes the YAM models’ fields, four frames per camera after the first request, and is API-only too. runway/praxis-1 takes the same fields and 21 frames per camera after the first request, one at the end of each row just run; it is API-only. Errors keep their HTTP status and answer a status object. FLUX applies mandatory prohibited-use prompt filters before billing/inference and a numeric-action/metadata allowlist before transmission. Input matches return 403 flux_content_blocked; refused backend replies return 502 flux_output_blocked without action rows and are not billed. Successful FLUX replies include AI provenance and the filter version. These filters do not assess scene hazards or replace physical controller limits. The entire chunk is refused if any joint target exceeds the published Panda/FER position bounds; targets and finite grippers are never clamped. This does not guarantee collision safety or velocity/acceleration limits. There is no request-level opt-out. See Robot action API.

POST

get_action

curl --request POST \
  --url https://api.abstract.inc/v1/get_action \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "model": "allenai/molmoact2",
  "session_id": null,
  "observation": {
    "prompt": "put the red cup in the bin",
    "state": {
      "left_eef_xyz": [
        0.3,
        0,
        0.2
      ],
      "left_eef_rot6": [
        -0.5,
        0,
        -0.866,
        0,
        1,
        0
      ],
      "left_grip": [
        1
      ],
      "right_eef_xyz": [
        0.3,
        0,
        0.2
      ],
      "right_eef_rot6": [
        -0.5,
        0,
        -0.866,
        0,
        1,
        0
      ],
      "right_grip": [
        0.5
      ]
    },
    "video": {
      "overhead": [
        "<top JPEG>"
      ],
      "wrist_l": [
        "<left JPEG>"
      ],
      "wrist_r": [
        "<right JPEG>"
      ]
    }
  }
}
'

{
  "status": {
    "code": "ok",
    "reason": null,
    "retryable": false,
    "retry_after_s": null
  },
  "session_id": "7c9e6679-7425-40de-944b-e07fc1f90ae7",
  "step": 0,
  "action_space": "ee_6d_position",
  "actions": {
    "left_eef_chunk": [
      [
        0.301,
        0,
        0.199,
        -0.5,
        0,
        -0.866,
        0,
        1,
        0,
        1
      ]
    ],
    "right_eef_chunk": [
      [
        0.3,
        0,
        0.2,
        -0.5,
        0,
        -0.866,
        0,
        1,
        0,
        0.5
      ]
    ]
  },
  "dt": 0.03333333333333333,
  "next_frames_per_camera": 1,
  "timing": {
    "decode": 4.1,
    "queue": 0,
    "prefill": 0,
    "denoise": 212.5,
    "encode": 0.3
  },
  "usage": {
    "prompt_tokens": 650,
    "prompt_tokens_details": {
      "cached_tokens": 0
    },
    "completion_tokens": 30,
    "total_tokens": 680
  },
  "id": "gen-3f9a1c2b7d4e"
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

Authorizations

Authorization

string

header

required

Your Abstract API key. Get it with abstract login, pass it as $(abstract token).

Headers

Abstract-Version

enum<string>

The API version your client was written for. 2026-10-27 is the only version. Without this header the request is answered under the newest version; a version the API does not answer is refused with 400 unsupported_api_version before anything runs.

Available options:

2026-10-27

Idempotency-Key

string

Replaces observation_id. If both are sent they must match.

Body

application/json

YAM end-effector
Xiaomi ARX X5
FLUX DROID Franka

model

enum<string>

required

A served robot model id. Each model selects its own state, camera and output contract. Check GET /v1/environments/models for current availability.

Available options:

allenai/molmoact2,

microsoft/rho,

perceptron/isaac-0.5,

physical-intelligence/pi0.5,

verne/v0.5,

runway/praxis-1

observation

object

required

The selected model's observation; state and camera fields must match its branch.

Show child attributes

action_horizon

integer

Rows to return, up to the selected model's chunk: FLUX 32, Xiaomi 30, and each YAM model's documented count. Default all. The model still computes and bills the whole chunk.

Required range: 1 <= x <= 30

session_id

string | null

The episode to continue. null or absent starts a new episode; the answer carries its id.

Maximum string length: 128

reset

boolean

default:false

true starts a new episode under a new id, even when session_id is set.

expected_step

integer

Expected next session step. A mismatch returns 409 without inference.

Required range: x >= 0

scaling

object

Fixes the chunk's noise seed. MolmoAct2 and π0.5 support it; other models return invalid_scaling.

Show child attributes

observation_id

string

Unique logical observation id. Reuse only for an identical retry.

Pattern: ^[A-Za-z0-9._:-]{1,128}$

handoff

boolean

default:false

true is refused today with handoff_unavailable.

Response

End-effector chunks for YAM/Xiaomi, or a native joint chunk for FLUX. Check action_space before interpreting rows.

End-effector answer
FLUX DROID answer

action_space

enum<string>

required

Selects the action row representation. YAM and Xiaomi return end-effector chunks; FLUX returns a native Franka joint chunk.

Available options:

ee_6d_position,

joint_position

Allowed value: "ee_6d_position"

actions

object

required

Option 1
Option 2

Show child attributes

status

object

required

Show child attributes

session_id

string

required

step

integer

required

The chunk's zero-based index in the episode.

number

required

Seconds between consecutive rows.

next_frames_per_camera

integer

required

usage

object

required

Show child attributes

string

required

The id of this generation.

timing

object

Server-side durations in milliseconds.

scaling

object

The seeds drawn and the chosen candidate, for seeded requests.

cache

object

Present when the model reports its denoiser step cache.

context

object

Present when the model reports its attention context.

provenance

object

Present on successful FLUX replies. Identifies AI-generated actions and the applied mandatory content-filter version; it is not a safety certification.

Show child attributes

curl --request POST \
  --url https://api.abstract.inc/v1/get_action \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "model": "allenai/molmoact2",
  "session_id": null,
  "observation": {
    "prompt": "put the red cup in the bin",
    "state": {
      "left_eef_xyz": [
        0.3,
        0,
        0.2
      ],
      "left_eef_rot6": [
        -0.5,
        0,
        -0.866,
        0,
        1,
        0
      ],
      "left_grip": [
        1
      ],
      "right_eef_xyz": [
        0.3,
        0,
        0.2
      ],
      "right_eef_rot6": [
        -0.5,
        0,
        -0.866,
        0,
        1,
        0
      ],
      "right_grip": [
        0.5
      ]
    },
    "video": {
      "overhead": [
        "<top JPEG>"
      ],
      "wrist_l": [
        "<left JPEG>"
      ],
      "wrist_r": [
        "<right JPEG>"
      ]
    }
  }
}
'

{
  "status": {
    "code": "ok",
    "reason": null,
    "retryable": false,
    "retry_after_s": null
  },
  "session_id": "7c9e6679-7425-40de-944b-e07fc1f90ae7",
  "step": 0,
  "action_space": "ee_6d_position",
  "actions": {
    "left_eef_chunk": [
      [
        0.301,
        0,
        0.199,
        -0.5,
        0,
        -0.866,
        0,
        1,
        0,
        1
      ]
    ],
    "right_eef_chunk": [
      [
        0.3,
        0,
        0.2,
        -0.5,
        0,
        -0.866,
        0,
        1,
        0,
        0.5
      ]
    ]
  },
  "dt": 0.03333333333333333,
  "next_frames_per_camera": 1,
  "timing": {
    "decode": 4.1,
    "queue": 0,
    "prefill": 0,
    "denoise": 212.5,
    "encode": 0.3
  },
  "usage": {
    "prompt_tokens": 650,
    "prompt_tokens_details": {
      "cached_tokens": 0
    },
    "completion_tokens": 30,
    "total_tokens": 680
  },
  "id": "gen-3f9a1c2b7d4e"
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

{
  "status": {
    "code": "model_overloaded",
    "reason": "The runtime admission is full.",
    "retryable": true,
    "retry_after_s": 1
  }
}

Overview

Chat completions

Robot actions

Catalog

Turn one observation into an action chunk

Authorizations

Headers

Body

Response