A coalition of leading institutions integrates large vision-language models, reinforcement learning, and model predictive control to create unified robotic systems. They blend pre-trained AI models with traditional pipelines, enabling explainable, safety-aware autonomous driving, dexterous bimanual manipulation, and adaptive human-robot interaction for practical deployment.

Key points

• Vision-language models integrated with MPC and RL deliver explainable, safety-aware autonomous driving with fewer infractions.
• SYMDEX exploits equivariant neural networks to leverage bilateral symmetry, boosting sample efficiency in ambidextrous bimanual tasks.
• CLAM’s continuous latent actions from unlabeled video demonstrations yield 2–3× higher manipulation success on real robot arms.

Why it matters: By merging AI’s flexible reasoning with proven control techniques, this approach unlocks deployable robots that are both intelligent and safe in real-world settings.