ActionCodec2

• Paper edition: first- and second-order tokenizers used in the paper; 5 mm translation precision and 4,096 tokens per profile.
• Extended edition: retrained after additional pretraining-data cleaning and filtering, with more precision and vocabulary options.
• Sparse version: standard and high-dynamic first-order tokenizers, each with EEF and joint profiles and binary (2bin) or 21-level (21bin) grippers.

All 34 variants use one shared runtime. Precision labels describe translation quantization; rotation and joint limits are in each profile's configuration.

Names encode edition, order (o1/o2), translation precision, vocabulary budget, gripper levels, and EEF speed per translation axis: 1p2ms = 1.2 m/s, 2ms = 2 m/s. Previous names remain compatibility aliases in variants.json; token IDs are unchanged.

Tokenizer usage

pip install numpy scipy torch "transformers>=5.2,<6" huggingface-hub pyyaml

import numpy as np
from transformers import AutoProcessor

codec = AutoProcessor.from_pretrained(
    "ZibinDong/ActionCodec2", variant="paper-o1-5mm-4k-2bin-1p2ms",
    action_space="single_eef_delta", trust_remote_code=True,
)
actions = np.zeros((1, 16, 7), dtype=np.float32)
tokens = codec.encode(actions, fps=15)
restored = codec.decode(tokens, fps=15)

Use variant="sparse-o1-5mm-4k-2bin-1p2ms" or "sparse-o1-5mm-4k-21bin-2ms" for Sparse; Sparse decoding requires horizon=actions.shape[-2] in source frames. Each profile has its own token namespace; the budget is per profile, not the total routed vocabulary.

The EEF layout is [x, y, z, rx, ry, rz, gripper], in metres and radians. Delta rotations use R_next = R_previous @ Exp(rotvec). Grippers are absolute open fractions (0 closed, 1 open). Inputs are (T,D) or (B,T,D) and are resampled to 15 Hz. Reconstruction is lossy. Absolute layouts also require current_state; second-order variants require previous_first_order in canonical 15 Hz increments in both encode and decode. Zero boundaries are valid only when the preceding motion is known to be zero. Use final_first_order to carry reconstructed second-order boundaries between chunks.

codec.print_action_spaces() lists the layouts. Dual arms are interleaved by time; BPE tokens never merge across arms. The historical single_joint6_abs_continuous_gripper name aliases single_joint6_abs_no_gripper: six joint coordinates, no separate gripper slot.

Train tokenizers

codec.fit(episodes, fps=15)
codec.save_pretrained("./my-tokenizer")

Cleaning is off by default, including VLA training and inference. Only tokenizer fitting accepts apply_preprocessing=True, with a recorded or explicitly supplied preprocessing configuration. It acts on continuous canonical trajectories before quantization. Split recording discontinuities first; second-order cleaning needs bootstrap_context=True. Saved cleaning settings do not enable filtering during encode or decode.

Available variants

Variant	Edition	Order	Translation precision	Budget / profile	Profiles
paper-o1-5mm-4k-2bin-1p2ms	Paper	1	5mm	4096	joint, eef
paper-o2-5mm-4k-2bin-1p2ms	Paper	2	5mm	4096	joint, eef
extended-o1-1mm-2k-2bin-1p2ms	Extended	1	1mm	2048	eef
extended-o1-1mm-4k-2bin-1p2ms	Extended	1	1mm	4096	joint, eef
extended-o1-1mm-8k-2bin-1p2ms	Extended	1	1mm	8192	joint, eef
extended-o1-1mm-16k-2bin-1p2ms	Extended	1	1mm	16384	joint, eef
extended-o1-5mm-1k-2bin-1p2ms	Extended	1	5mm	1024	joint, eef
extended-o1-5mm-2k-2bin-1p2ms	Extended	1	5mm	2048	joint, eef
extended-o1-5mm-4k-2bin-1p2ms	Extended	1	5mm	4096	joint, eef
extended-o1-5mm-8k-2bin-1p2ms	Extended	1	5mm	8192	joint, eef
extended-o1-5mm-16k-2bin-1p2ms	Extended	1	5mm	16384	joint, eef
extended-o1-10mm-1k-2bin-1p2ms	Extended	1	10mm	1024	joint, eef
extended-o1-10mm-2k-2bin-1p2ms	Extended	1	10mm	2048	joint, eef
extended-o1-10mm-4k-2bin-1p2ms	Extended	1	10mm	4096	joint, eef
extended-o1-10mm-8k-2bin-1p2ms	Extended	1	10mm	8192	joint, eef
extended-o1-10mm-16k-2bin-1p2ms	Extended	1	10mm	16384	joint, eef
extended-o2-1mm-2k-2bin-1p2ms	Extended	2	1mm	2048	eef
extended-o2-1mm-4k-2bin-1p2ms	Extended	2	1mm	4096	joint, eef
extended-o2-1mm-8k-2bin-1p2ms	Extended	2	1mm	8192	joint, eef
extended-o2-1mm-16k-2bin-1p2ms	Extended	2	1mm	16384	joint, eef
extended-o2-5mm-1k-2bin-1p2ms	Extended	2	5mm	1024	joint, eef
extended-o2-5mm-2k-2bin-1p2ms	Extended	2	5mm	2048	joint, eef
extended-o2-5mm-4k-2bin-1p2ms	Extended	2	5mm	4096	joint, eef
extended-o2-5mm-8k-2bin-1p2ms	Extended	2	5mm	8192	joint, eef
extended-o2-5mm-16k-2bin-1p2ms	Extended	2	5mm	16384	joint, eef
extended-o2-10mm-1k-2bin-1p2ms	Extended	2	10mm	1024	joint, eef
extended-o2-10mm-2k-2bin-1p2ms	Extended	2	10mm	2048	joint, eef
extended-o2-10mm-4k-2bin-1p2ms	Extended	2	10mm	4096	joint, eef
extended-o2-10mm-8k-2bin-1p2ms	Extended	2	10mm	8192	joint, eef
extended-o2-10mm-16k-2bin-1p2ms	Extended	2	10mm	16384	joint, eef
sparse-o1-5mm-4k-2bin-1p2ms	Sparse	1	5mm	4096	eef, joint
sparse-o1-5mm-4k-21bin-1p2ms	Sparse	1	5mm	4096	eef, joint
sparse-o1-5mm-4k-2bin-2ms	Sparse	1	5mm	4096	eef, joint
sparse-o1-5mm-4k-21bin-2ms	Sparse	1	5mm	4096	eef, joint

Pin revision for reproducibility. variants.json records asset hashes; a selected save_pretrained() directory contains its assets and the shared portable runtime. The optional C++17 kernels can be built with pip install ./downloaded-tokenizer.

VLA policies

Installation, paper-policy evaluation and VLA training: source repository. The existing six AC2 policies retain their single-arm tokenizer behavior. The released Sparse profiles have a 4,096-token budget; the current G0.5 Sparse adapter additionally needs an EOS position and therefore requires a vocabulary of at most 4,095 tokens when training a new VLA.

License

Tokenizer code and assets: Apache-2.0. G0.5 policy code and weights use their separate community license.