Get action

POST https://api.abstract.inc/v1/get_action calls a robot model. Choose an available model from GET /v1/environments/models and use its own state, camera and action contract. The following request calls MolmoAct2’s bimanual YAM interface: a task, both grasp-site poses, and JPEGs from three cameras.

Request

{
	"model": "allenai/molmoact2",
	"session_id": null,
	"observation": {
		"prompt": "put the red cup in the bin",
		"state": {
			"left_eef_xyz": [0.3, 0.0, 0.2],
			"left_eef_rot6": [-0.5, 0.0, -0.866, 0.0, 1.0, 0.0],
			"left_grip": [1.0],
			"right_eef_xyz": [0.3, 0.0, 0.2],
			"right_eef_rot6": [-0.5, 0.0, -0.866, 0.0, 1.0, 0.0],
			"right_grip": [1.0]
		},
		"video": {
			"overhead": ["<base64 JPEG>"],
			"wrist_l": ["<base64 JPEG>"],
			"wrist_r": ["<base64 JPEG>"]
		}
	}
}

For this YAM interface, eef_xyz is in metres, eef_rot6 is the rotation matrix’s first two columns, and grip is one value, 0 closed to 1 open. Do not send joint angles to these models; see each model’s coordinate frame.
Each camera sends one JPEG on the first call, then next_frames_per_camera JPEGs, oldest first: at most 9 (21 for runway/praxis-1), and the same count for every camera. overhead, wrist_l, wrist_r map to the model’s three cameras in order.
Each JPEG is at most 256 KiB before base64 (larger is refused with 413 image_too_large); the whole body is at most 8 MiB.
prompt is at most 512 characters. Keep the same instruction for the whole episode; a changed one returns instruction_changed.
session_id: null starts an episode. Send the returned id to continue it. reset: true starts a new one.

Xiaomi ARX X5

For xiaomi/xiaomi-robotics-1, keep the six pose and gripper fields in the example and add left_joint_position and right_joint_position, each six measured joint angles in radians. Every field must come from the same observation. Poses and returned targets use the RoboDojo/world frame; grippers are normalized 0 closed, 1 open. Do not send YAM base-frame poses or placeholder joint readings. Send exactly one JPEG in each of overhead, wrist_l and wrist_r on every call. The answer has action_space: "ee_6d_position", up to 30 rows in each arm’s end-effector chunk, and dt: 0.04. See the full checkpoint contract.

FLUX DROID

black-forest-labs/flux-3-action takes seven measured Franka joint angles in radians and one gripper closed fraction (0 open, 1 closed). Send exactly one 640×360 JPEG for each of DROID’s two exterior cameras and its wrist camera. These are the only state and camera fields for this model; do not send the YAM end-effector fields.

{
	"model": "black-forest-labs/flux-3-action",
	"session_id": null,
	"observation": {
		"prompt": "put the cube in the bowl",
		"state": {
			"joint_position": [0.0, -0.628, 0.0, -2.513, 0.0, 1.885, 0.0],
			"gripper_position": [0.0]
		},
		"video": {
			"exterior_image_0_left": ["<base64 640x360 JPEG>"],
			"exterior_image_1_left": ["<base64 640x360 JPEG>"],
			"wrist_image_left": ["<base64 640x360 JPEG>"]
		}
	}
}

The numbers illustrate the format; use measured state from the same observation as your images. The answer has action_space: "joint_position", dt: 0.06666666666666667 and actions.joint_chunk: 32 rows of eight values, seven absolute joint targets in radians then one gripper closed fraction. next_frames_per_camera is 1. A smaller action_horizon returns fewer rows while billing all 32 computed rows. Finite gripper predictions can fall outside [0, 1]; they are not clamped. Validate targets in your controller before execution.

FLUX content filters

FLUX instructions pass through maintained prohibited-use prompt filters before billing or inference. A matching instruction returns HTTP 403 flux_content_blocked; no action chunk is generated or billed. Before sending an answer, the API permits only the declared numeric joint actions and supported metadata. Generated text, images and video are not returned. If any of the seven joint targets in any row falls outside the published Franka Panda/FER position bounds, the whole chunk is refused. Targets and finite gripper predictions are never clamped. A backend reply that fails this output filter returns HTTP 502 flux_output_blocked, with no action rows, and is not billed. There is no request option to disable these filters. Successful FLUX answers include provenance:

{
	"ai_generated": true,
	"model": "black-forest-labs/flux-3-action",
	"filter_version": "flux-action-v1"
}

These are automated content filters, not human review or a guarantee that a task is lawful or safe. Explicit prompt rules can miss paraphrases, other languages and intent hidden in the scene; the output checks do not understand scene hazards or determine copyright status. Position bounds alone do not establish collision safety, velocity or acceleration limits. Your controller must check targets and enforce physical limits before moving the robot.

Response

The JSON answer contains:

Field	Meaning
`status.code`	`ok` on success
`status.trace_id`	32 hex characters, also in `X-Trace-Id`; a replayed answer keeps its trace
`session_id`, `step`	Episode id and zero-based chunk index
`action_space`	`ee_6d_position` for YAM and Xiaomi; `joint_position` for FLUX
`actions.left_eef_chunk`, `actions.right_eef_chunk`	For the end-effector interface, rows of ten numbers: xyz, rotation6d, gripper
`actions.joint_chunk`	For FLUX, rows of eight numbers: seven native joint radians, then gripper
`dt`	Seconds between action rows
`next_frames_per_camera`	Frames each camera must send next
`usage`	The chunk’s billing units
`provenance`	FLUX only: AI-generated indicator, model id and applied filter version
`id`, `timing`	Generation id and server timings in milliseconds

For end-effector actions, your robot checks the poses and solves its own inverse kinematics. Always check the selected model’s action space before interpreting its rows. Errors keep their HTTP status and carry status.code, status.reason, retry advice, and an error object, with session_id: "", actions: null and timing: null. See Errors. A malformed chunk from the engine is refused with 502 engine_protocol_error and is not billed. Every answer has a request-id header, new for each attempt, which a refusal repeats as status.request_id. Quote it or the trace when you report a problem.

Options

observation_id and expected_step make retries safe; see Robot episodes. action_horizon limits returned rows, not the whole chunk’s cost. scaling: {"seed": 42} fixes the noise seed on MolmoAct2 and π0.5; its report comes back as scaling. handoff: true is not available.

End session

POST https://api.abstract.inc/v1/end_session with model and session_id releases the episode. The answer’s ended is true if it was open, false if already gone. This call is free. While a chunk of the episode is in flight it returns 409 session_busy. An episode also ends after 60 seconds without a call.

Get started

Robot models

LLM models

Reference