> ## Documentation Index > Fetch the complete documentation index at: https://abstract-pbc.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Turn one observation into an action chunk > One metered action chunk, using the selected model's observation and action contract. YAM models take grasp-site poses and grippers and refuse joints. Xiaomi's ARX X5 checkpoint additionally needs measured joints and uses RoboDojo/world-frame poses. FLUX's DROID checkpoint takes seven Franka joints and one gripper closed fraction, with its own camera names, and returns native joint targets. Both native checkpoints are API-only. `verne/v0.5` takes the YAM models' fields, four frames per camera after the first request, and is API-only too. `runway/praxis-1` takes the same fields and 21 frames per camera after the first request, one at the end of each row just run; it is API-only. Errors keep their HTTP status and answer a `status` object. FLUX applies mandatory prohibited-use prompt filters before billing/inference and a numeric-action/metadata allowlist before transmission. Input matches return 403 `flux_content_blocked`; refused backend replies return 502 `flux_output_blocked` without action rows and are not billed. Successful FLUX replies include AI provenance and the filter version. These filters do not assess scene hazards or replace physical controller limits. The entire chunk is refused if any joint target exceeds the published Panda/FER position bounds; targets and finite grippers are never clamped. This does not guarantee collision safety or velocity/acceleration limits. There is no request-level opt-out. See [Robot action API](/robots/get-action). ## OpenAPI ````yaml /api-reference/openapi.yaml post /v1/get_action openapi: 3.1.0 info: title: Abstract API version: 1.0.0 summary: Robot action models and LLMs behind one API key. description: > `POST /v1/chat/completions` serves the LLM models. `GET /v1/chat/completions/{completion_id}` reads an LLM completion your account created. `POST /v1/get_action` and `POST /v1/end_session` serve the robot action models in Dream Labs' request shape. `GET /v1/models` lists the LLM model ids. contact: name: Abstract url: https://abstract.inc servers: - url: https://api.abstract.inc security: - bearerAuth: [] tags: - name: Chat completions - name: Robot actions description: End-effector and model-specific native joint action contracts. - name: Catalog - name: CLI credentials paths: /v1/get_action: post: tags: - Robot actions summary: Turn one observation into an action chunk description: > One metered action chunk, using the selected model's observation and action contract. YAM models take grasp-site poses and grippers and refuse joints. Xiaomi's ARX X5 checkpoint additionally needs measured joints and uses RoboDojo/world-frame poses. FLUX's DROID checkpoint takes seven Franka joints and one gripper closed fraction, with its own camera names, and returns native joint targets. Both native checkpoints are API-only. `verne/v0.5` takes the YAM models' fields, four frames per camera after the first request, and is API-only too. `runway/praxis-1` takes the same fields and 21 frames per camera after the first request, one at the end of each row just run; it is API-only. Errors keep their HTTP status and answer a `status` object. FLUX applies mandatory prohibited-use prompt filters before billing/inference and a numeric-action/metadata allowlist before transmission. Input matches return 403 `flux_content_blocked`; refused backend replies return 502 `flux_output_blocked` without action rows and are not billed. Successful FLUX replies include AI provenance and the filter version. These filters do not assess scene hazards or replace physical controller limits. The entire chunk is refused if any joint target exceeds the published Panda/FER position bounds; targets and finite grippers are never clamped. This does not guarantee collision safety or velocity/acceleration limits. There is no request-level opt-out. See [Robot action API](/robots/get-action). operationId: getAction parameters: - $ref: '#/components/parameters/AbstractVersion' - name: Idempotency-Key in: header required: false description: Replaces `observation_id`. If both are sent they must match. schema: type: string requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/GetActionRequest' examples: molmoact2: summary: allenai/molmoact2, one frame per camera value: model: allenai/molmoact2 session_id: null observation: prompt: put the red cup in the bin state: left_eef_xyz: - 0.3 - 0 - 0.2 left_eef_rot6: - -0.5 - 0 - -0.866 - 0 - 1 - 0 left_grip: - 1 right_eef_xyz: - 0.3 - 0 - 0.2 right_eef_rot6: - -0.5 - 0 - -0.866 - 0 - 1 - 0 right_grip: - 0.5 video: overhead: - wrist_l: - wrist_r: - verneFirst: summary: verne/v0.5, first request of an episode value: model: verne/v0.5 session_id: 7c9e6679-7425-40de-944b-e07fc1f90ae7 observation_id: 7c9e6679-7425-40de-944b-e07fc1f90ae7.0 expected_step: 0 observation: prompt: fold the towel state: left_eef_xyz: - 0.3 - 0 - 0.2 left_eef_rot6: - -0.5 - 0 - -0.866 - 0 - 1 - 0 left_grip: - 1 right_eef_xyz: - 0.3 - 0 - 0.2 right_eef_rot6: - -0.5 - 0 - -0.866 - 0 - 1 - 0 right_grip: - 0.5 video: overhead: - wrist_l: - wrist_r: - verneNext: summary: verne/v0.5, a later request (4 frames per camera) value: model: verne/v0.5 session_id: 7c9e6679-7425-40de-944b-e07fc1f90ae7 observation_id: 7c9e6679-7425-40de-944b-e07fc1f90ae7.1 expected_step: 1 observation: prompt: fold the towel state: left_eef_xyz: - 0.3 - 0 - 0.2 left_eef_rot6: - -0.5 - 0 - -0.866 - 0 - 1 - 0 left_grip: - 1 right_eef_xyz: - 0.3 - 0 - 0.2 right_eef_rot6: - -0.5 - 0 - -0.866 - 0 - 1 - 0 right_grip: - 0.5 video: overhead: - - - - wrist_l: - - - - wrist_r: - - - - rho: summary: microsoft/rho, poses in the checkpoint's frame value: model: microsoft/rho session_id: null observation: prompt: put the red cup in the bin state: left_eef_xyz: - 0.3 - 0.2 - 0.3 left_eef_rot6: - 1 - 0 - 0 - 0 - 1 - 0 left_grip: - 0 right_eef_xyz: - 0.3 - -0.2 - 0.3 right_eef_rot6: - 1 - 0 - 0 - 0 - 1 - 0 right_grip: - 0 video: overhead: - wrist_l: - wrist_r: - xiaomi: summary: Xiaomi native ARX X5, measured joints and world-frame poses value: model: xiaomi/xiaomi-robotics-1 session_id: null observation: prompt: Inspect the object on the table. state: left_eef_xyz: - 0.3 - 0.2 - 0.3 left_eef_rot6: - 1 - 0 - 0 - 0 - 1 - 0 left_grip: - 1 right_eef_xyz: - 0.3 - -0.2 - 0.3 right_eef_rot6: - 1 - 0 - 0 - 0 - 1 - 0 right_grip: - 1 left_joint_position: - 0 - 0 - 0 - 0 - 0 - 0 right_joint_position: - 0 - 0 - 0 - 0 - 0 - 0 video: overhead: - wrist_l: - wrist_r: - flux: summary: FLUX native DROID Franka, one 640x360 frame per camera value: model: black-forest-labs/flux-3-action session_id: null action_horizon: 32 observation: prompt: put the cube in the bowl state: joint_position: - 0 - -0.628 - 0 - -2.513 - 0 - 1.885 - 0 gripper_position: - 0 video: exterior_image_0_left: - exterior_image_1_left: - wrist_image_left: - responses: '200': description: >- End-effector chunks for YAM/Xiaomi, or a native joint chunk for FLUX. Check action_space before interpreting rows. headers: Request-Id: $ref: '#/components/headers/RequestId' Abstract-Version: $ref: '#/components/headers/AbstractVersion' content: application/json: schema: $ref: '#/components/schemas/GetActionAnswer' example: status: code: ok reason: null retryable: false retry_after_s: null session_id: 7c9e6679-7425-40de-944b-e07fc1f90ae7 step: 0 action_space: ee_6d_position actions: left_eef_chunk: - - 0.301 - 0 - 0.199 - -0.5 - 0 - -0.866 - 0 - 1 - 0 - 1 right_eef_chunk: - - 0.3 - 0 - 0.2 - -0.5 - 0 - -0.866 - 0 - 1 - 0 - 0.5 dt: 0.03333333333333333 next_frames_per_camera: 1 timing: decode: 4.1 queue: 0 prefill: 0 denoise: 212.5 encode: 0.3 usage: prompt_tokens: 650 prompt_tokens_details: cached_tokens: 0 completion_tokens: 30 total_tokens: 680 id: gen-3f9a1c2b7d4e '400': $ref: '#/components/responses/ActionRefusal' '401': $ref: '#/components/responses/ActionRefusal' '402': $ref: '#/components/responses/ActionRefusal' '403': $ref: '#/components/responses/ActionRefusal' description: >- FLUX prompt matches the prohibited-use filter; flux_content_blocked, before billing or inference. '408': $ref: '#/components/responses/ActionRefusal' '409': $ref: '#/components/responses/ActionRefusal' '410': $ref: '#/components/responses/ActionRefusal' '413': $ref: '#/components/responses/ActionRefusal' '429': $ref: '#/components/responses/ActionRefusal' '500': $ref: '#/components/responses/ActionRefusal' '502': $ref: '#/components/responses/ActionRefusal' '503': $ref: '#/components/responses/ActionRefusal' components: parameters: AbstractVersion: name: Abstract-Version in: header required: false description: >- The API version your client was written for. `2026-10-27` is the only version. Without this header the request is answered under the newest version; a version the API does not answer is refused with `400 unsupported_api_version` before anything runs. schema: type: string enum: - '2026-10-27' schemas: GetActionRequest: type: object required: - model - observation properties: model: type: string enum: - allenai/molmoact2 - microsoft/rho - perceptron/isaac-0.5 - physical-intelligence/pi0.5 - verne/v0.5 - runway/praxis-1 - xiaomi/xiaomi-robotics-1 - black-forest-labs/flux-3-action description: >- A served robot model id. Each model selects its own state, camera and output contract. Check GET /v1/environments/models for current availability. session_id: type: - string - 'null' maxLength: 128 description: >- The episode to continue. `null` or absent starts a new episode; the answer carries its id. reset: type: boolean default: false description: >- `true` starts a new episode under a new id, even when `session_id` is set. observation: description: >- The selected model's observation; state and camera fields must match its branch. action_horizon: type: integer minimum: 1 maximum: 32 description: >- Rows to return, up to the selected model's chunk: FLUX 32, Xiaomi 30, and each YAM model's documented count. Default all. The model still computes and bills the whole chunk. expected_step: type: integer minimum: 0 description: >- Expected next session step. A mismatch returns 409 without inference. scaling: type: object additionalProperties: false required: - seed properties: seed: type: integer minimum: 0 maximum: 2147483647 description: >- Fixes the chunk's noise seed. MolmoAct2 and π0.5 support it; other models return `invalid_scaling`. observation_id: type: string pattern: ^[A-Za-z0-9._:-]{1,128}$ description: Unique logical observation id. Reuse only for an identical retry. handoff: type: boolean default: false description: '`true` is refused today with `handoff_unavailable`.' oneOf: - title: YAM end-effector properties: model: enum: - allenai/molmoact2 - microsoft/rho - perceptron/isaac-0.5 - physical-intelligence/pi0.5 - verne/v0.5 - runway/praxis-1 observation: $ref: '#/components/schemas/YamActionObservation' action_horizon: maximum: 30 - title: Xiaomi ARX X5 properties: model: const: xiaomi/xiaomi-robotics-1 observation: $ref: '#/components/schemas/XiaomiActionObservation' action_horizon: maximum: 30 - title: FLUX DROID Franka properties: model: const: black-forest-labs/flux-3-action observation: $ref: '#/components/schemas/FluxActionObservation' action_horizon: maximum: 32 GetActionAnswer: type: object required: - status - session_id - step - actions - dt - next_frames_per_camera - usage - id - action_space properties: status: $ref: '#/components/schemas/ActionStatus' session_id: type: string step: type: integer description: The chunk's zero-based index in the episode. actions: oneOf: - $ref: '#/components/schemas/EndEffectorActions' - $ref: '#/components/schemas/FrankaJointActions' dt: type: number description: Seconds between consecutive rows. next_frames_per_camera: type: integer timing: type: object description: Server-side durations in milliseconds. scaling: type: object description: The seeds drawn and the chosen candidate, for seeded requests. cache: type: object description: Present when the model reports its denoiser step cache. context: type: object description: Present when the model reports its attention context. usage: $ref: '#/components/schemas/ChunkUsage' id: type: string description: The id of this generation. action_space: type: string enum: - ee_6d_position - joint_position description: >- Selects the action row representation. YAM and Xiaomi return end-effector chunks; FLUX returns a native Franka joint chunk. provenance: $ref: '#/components/schemas/FluxActionProvenance' oneOf: - title: End-effector answer properties: action_space: const: ee_6d_position actions: $ref: '#/components/schemas/EndEffectorActions' - title: FLUX DROID answer required: - provenance properties: action_space: const: joint_position actions: $ref: '#/components/schemas/FrankaJointActions' dt: const: 0.06666666666666667 next_frames_per_camera: const: 1 YamActionObservation: type: object required: - prompt - state - video properties: prompt: $ref: '#/components/schemas/ActionPrompt' state: $ref: '#/components/schemas/YamActionState' video: $ref: '#/components/schemas/YamActionVideo' XiaomiActionObservation: type: object required: - prompt - state - video properties: prompt: $ref: '#/components/schemas/ActionPrompt' state: $ref: '#/components/schemas/XiaomiActionState' video: $ref: '#/components/schemas/XiaomiActionVideo' FluxActionObservation: type: object required: - prompt - state - video properties: prompt: $ref: '#/components/schemas/ActionPrompt' state: $ref: '#/components/schemas/FluxActionState' video: $ref: '#/components/schemas/FluxActionVideo' ActionStatus: type: object required: - code - reason - retryable - retry_after_s properties: code: type: string description: '`ok`, or an error code from the Errors page.' reason: type: - string - 'null' retryable: type: boolean description: The API's retry advice. Terminal refusals are not retryable. retry_after_s: type: - number - 'null' description: The `Retry-After` header, when there is one. param: type: string description: The offending request field, when known. name: type: string description: Uppercase status name. detail: type: string description: Human-readable refusal detail. trace_id: type: string pattern: ^[a-f0-9]{32}$ description: >- Logical reply trace. Retained on a replay and mirrored in X-Trace-Id. The request-id header is unique to each HTTP attempt. request_id: type: string pattern: ^req_[0-9a-f]{32}$ description: On a refusal, this answer's `Request-Id` header. EndEffectorActions: type: object required: - left_eef_chunk - right_eef_chunk properties: left_eef_chunk: type: array description: One row per step, ten numbers each — xyz, rotation6d, gripper. items: type: array items: type: number minItems: 10 maxItems: 10 minItems: 1 maxItems: 30 right_eef_chunk: type: array items: type: array items: type: number minItems: 10 maxItems: 10 minItems: 1 maxItems: 30 additionalProperties: false FrankaJointActions: type: object additionalProperties: false required: - joint_chunk properties: joint_chunk: type: array minItems: 1 maxItems: 32 description: >- Absolute native Franka targets: seven radians, then gripper closed fraction. Finite gripper predictions pass through even outside [0,1]; no output clamp. items: type: array items: type: number minItems: 8 maxItems: 8 ChunkUsage: type: object required: - prompt_tokens - completion_tokens - total_tokens properties: prompt_tokens: type: integer description: >- Prompt tokens, cached ones included. For a robot model, the model's prompt tokens for the chunk. prompt_tokens_details: type: object properties: cached_tokens: type: integer description: >- The part of `prompt_tokens` served from cache. For a robot model, reused from earlier chunks of the episode. completion_tokens: type: integer description: >- Output tokens. For a robot model, the action rows the model computed, before any `action_horizon` cut. total_tokens: type: integer FluxActionProvenance: type: object additionalProperties: false required: - ai_generated - model - filter_version description: >- Present on successful FLUX replies. Identifies AI-generated actions and the applied mandatory content-filter version; it is not a safety certification. properties: ai_generated: type: boolean const: true model: type: string const: black-forest-labs/flux-3-action filter_version: type: string example: flux-action-v1 ActionPrompt: type: string minLength: 1 maxLength: 512 description: The task instruction; whitespace alone is refused. YamActionState: type: object description: >- The six pose/gripper fields for YAM models. Rho uses its checkpoint's frame and gripper units; other YAM models use each arm's base frame and 0 closed/1 open grippers. Joint fields are refused. required: - left_eef_xyz - left_eef_rot6 - left_grip - right_eef_xyz - right_eef_rot6 - right_grip additionalProperties: false properties: left_eef_xyz: type: array items: type: number minItems: 3 maxItems: 3 description: Metres. left_eef_rot6: type: array items: type: number minItems: 6 maxItems: 6 description: The first two columns of the rotation matrix. left_grip: type: array items: type: number minItems: 1 maxItems: 1 description: 0 closed to 1 open. right_eef_xyz: type: array items: type: number minItems: 3 maxItems: 3 right_eef_rot6: type: array items: type: number minItems: 6 maxItems: 6 right_grip: type: array items: type: number minItems: 1 maxItems: 1 YamActionVideo: type: object description: >- Base64 JPEG frames per camera, oldest first, the same count for every camera: at most 9, or 21 for `runway/praxis-1`. A `data:image/jpeg;base64,` prefix is optional. `overhead`, `wrist_l` and `wrist_r` are `top`, `left` and `right` (Rho `scene`, `left_wrist`, `right_wrist`). required: - overhead - wrist_l - wrist_r additionalProperties: false properties: overhead: type: array items: type: string minItems: 1 maxItems: 21 wrist_l: type: array items: type: string minItems: 1 maxItems: 21 wrist_r: type: array items: type: string minItems: 1 maxItems: 21 XiaomiActionState: type: object description: >- ARX X5 measured joints and observed grasp-site poses from one observation. Poses use the RoboDojo/world frame; grippers are normalized 0 closed/1 open. No YAM kinematics are applied. required: - left_eef_xyz - left_eef_rot6 - left_grip - right_eef_xyz - right_eef_rot6 - right_grip - left_joint_position - right_joint_position additionalProperties: false properties: left_eef_xyz: type: array items: type: number minItems: 3 maxItems: 3 description: Metres. left_eef_rot6: type: array items: type: number minItems: 6 maxItems: 6 description: The first two columns of the rotation matrix. left_grip: type: array items: type: number minimum: 0 maximum: 1 minItems: 1 maxItems: 1 description: 0 closed to 1 open. right_eef_xyz: type: array items: type: number minItems: 3 maxItems: 3 right_eef_rot6: type: array items: type: number minItems: 6 maxItems: 6 right_grip: type: array items: type: number minimum: 0 maximum: 1 minItems: 1 maxItems: 1 left_joint_position: type: array items: type: number minItems: 6 maxItems: 6 description: Six measured ARX X5 joint angles in radians. right_joint_position: type: array items: type: number minItems: 6 maxItems: 6 description: Six measured ARX X5 joint angles in radians. XiaomiActionVideo: type: object description: >- One current base64 JPEG per camera, mapped in order to ego/left_wrist/right_wrist. Optional data:image/jpeg;base64, prefix. Each image is at most 256 KiB before base64. required: - overhead - wrist_l - wrist_r additionalProperties: false properties: overhead: type: array items: type: string minItems: 1 maxItems: 1 wrist_l: type: array items: type: string minItems: 1 maxItems: 1 wrist_r: type: array items: type: string minItems: 1 maxItems: 1 FluxActionState: type: object additionalProperties: false required: - joint_position - gripper_position description: >- Native DROID Franka state. No end-effector fields; both readings come from the same observation as the camera frames. properties: joint_position: type: array items: type: number minItems: 7 maxItems: 7 description: >- Seven measured Franka joint angles in radians, in native joint order. gripper_position: type: array items: type: number minimum: 0 maximum: 1 minItems: 1 maxItems: 1 description: 'Gripper closed fraction: 0 open, 1 closed.' FluxActionVideo: type: object additionalProperties: false required: - exterior_image_0_left - exterior_image_1_left - wrist_image_left description: >- One current 640x360 base64 JPEG from each native DROID camera. Optional data:image/jpeg;base64, prefix. Each image is at most 256 KiB before base64. properties: exterior_image_0_left: type: array items: type: string minItems: 1 maxItems: 1 exterior_image_1_left: type: array items: type: string minItems: 1 maxItems: 1 wrist_image_left: type: array items: type: string minItems: 1 maxItems: 1 headers: RequestId: description: >- `req_` and 32 hex characters, new for each HTTP attempt, a retry included. An error repeats it as `error.request_id`. Quote it when you report a problem. schema: type: string pattern: ^req_[0-9a-f]{32}$ AbstractVersion: description: >- The API version this answer was made under: the `Abstract-Version` the request sent, or the newest version when it sent none. schema: type: string responses: ActionRefusal: description: >- A robot action refusal. It keeps the HTTP status and answers a `status` object; `code` is an error code from the Errors page. content: application/json: schema: type: object required: - status properties: status: $ref: '#/components/schemas/ActionStatus' example: status: code: model_overloaded reason: The runtime admission is full. retryable: true retry_after_s: 1 securitySchemes: bearerAuth: type: http scheme: bearer description: >- Your Abstract API key. Get it with `abstract login`, pass it as `$(abstract token)`. ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.