News: We release a benchmark to evaluate the ability of MLLM on the robot use, please see: RobotWorld😙

A curated list of research on general-purpose AI agents that perceive, program, and operate robots through tools, interfaces, and reusable skills.

Inspired by Phillip Isola's Robot-Use Agents. The emphasis is on how general-purpose intelligence can be connected to different robots and how improvements can spread through models, software interfaces, and reusable capabilities.

Contents

• General-Purpose Robot-Use Agents
• Reasoning-Acting and Dual-System Architectures
• Self-Evolving Robot Agents
• Embodied Agent Operating Systems and Runtimes
• Programming and Spatial Action Interfaces
• Planning, Skill Orchestration and Memory
  • Task Planning, Skill Orchestration and Memory
  • Safe Planning, Verification and Failure Recovery
  • Multi-Robot Coordination
• Language-Native Actions and Cross-Embodiment Transfer
• Infrastructure and Benchmarks
• Perspectives and Reports

General-Purpose Robot-Use Agents

General-purpose models operating robots through reusable harnesses, tools, and visual interfaces.

• SuperNav: An Agentic Navigation System for Any Task in Any Scene — A pretrained multimodal agent uses navigation skills, tools, progress tracking, and a visual-point interface to delegate motion to geometric or learned executors; includes simulated evaluations and quadruped deployment.

• SpatialHarness: Test-Time Spatial Scaffolding for Fine Robotic Manipulation — Adds synchronized virtual views and structured spatial observations to a frozen multimodal robot policy at test time; evaluated on four physical manipulation tasks with 15 trials per method and task.

• OpenRUA: Robot-Use Agents Are Zero-Shot Visuomotor Policies — Off-the-shelf coding agents write perception and control programs through native ROS 2 terminal interfaces, with no task-specific policy training; evaluated in simulation with time paused during agent deliberation.

• RoboICL: Embodied In-Context Learning with GPT-6 Astra — A frozen general-purpose VLM issues Cartesian action chunks through a validated robot interface, using demonstrations, execution receipts, and observation history for closed-loop correction; simulation and real-robot evaluation.

• ASENA: Self-evolving Agents for Embodied Navigation — A general-purpose coding agent navigates through sensor and execution tools while accumulating persistent notes and programs; evaluates repeated-task improvement in simulation and supervised humanoid demonstrations.

• MotorMind: Scaffolding General Vision Language Models for Zero-Shot Robot Manipulation — A frozen general-purpose VLM controls robot manipulation through mid-level actions, asynchronous monitoring, outcome verification, and background memory updates; evaluated in simulation and on an xArm6.

• Robot Manipulation with GPT-6-Astra: Body Knowledge, Experience Reuse, Emergent Skills, and Sim2Real Transfer — Body descriptions and recorded experience support reusable control code and sim-to-real transfer; physical evaluation comprises 12 trials using operator-confirmed button contact.

• Know Your Body: A Harness for Direct and Self-Improving Robot Control with VLMs — KnowBody; body-grounded robot control and iterative improvement through execution feedback, evaluated on a small real-robot task suite.

• Generalizing Manipulation Skills with a Local Coding Agent — A local open-weight VLM writes and executes perception and control code for real-world UR3e manipulation, with documented skills and in-session reuse.

• AR-WAM: A Visual-Conditioned Agent-Ready World Action Model for Robotic Manipulation — Agent-ready manipulation through visual grounding, operation tokens, and detect / execute / query interfaces.

• Structured World-State Reasoning for Agentic Robotic Search — WORLDS maintains a persistent world-state graph and lets agents request, verify, and revise observations before selecting a target.

• Transferring the Intelligence of VLMs to Robotic Control — RoboDawn

• RoboFind: Multi-Agent Personalized Object Search for People Who Are Blind or Have Low Vision

• Navi-Agent: Unlocalized Monocular Navigation Agent — Previously reported; coordinate-free spatial memory for closed-loop navigation, progress verification and recovery.

• In-Context Robot Learning with VLM Agents — GPT-Policy; in-context adaptation without parameter updates.

• WetRobo: A Reproducible Robot Kit for Coding Agents in Biological Laboratories — Laboratory robotics.

• HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness — Navigation.

• EMERGE-Policy: A Robot Mind Emerges Beyond a Single Policy

• Agent as Policy for Robotic Manipulation

• Show-Harness: Just a VLM Agent Can Play Robots

• VIA: Visual Interface Agent for Robot Control

• CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation

• Guava: Distilling Frontier VLMs into a Compact Agent through a Robotic Manipulation Harness

• ETA: A New Agentic Paradigm for Embodied Tasks

• Maestro: Orchestrating Robotics Modules with Vision-Language Models for Zero-Shot Generalist Robots

Reasoning-Acting and Dual-System Architectures

Architectures that organize reasoning and robot execution through intermediate plans, coupled reasoning and action modules, or adaptive think/act scheduling. Includes learned-policy building blocks and agent-level systems. For author-described System 1 (e.g., Jev-like system) /System 2 models (VLM-like models), fixed-rate and asynchronous coupling are distinguished from adaptive reasoning. Related task-time memory and recovery methods remain under Planning, Skill Orchestration and Memory.

• Bridging Frontier Reasoning and Robot Execution: From Autonomous Demonstration Generation to Dense Language Supervision — Connects frontier-model guidance to a trained local policy through autonomous demonstration collection, corrective examples, and dense language supervision; evaluated in simulation and on physical manipulation tasks.
• Code Owns the Simulation, Jev Owns the Evaluation — Studies Jev as an evaluator of outcomes supplied by code, combining lookahead and executable skills; includes simulated Stacker control and instruction variants, with action-abstraction differences limiting causal attribution.
• DSWAM: A Dual-System World Action Foundation Model for Fine-Grained Robot Manipulation — System 1 WAM execution with an optional System 2 subtask planner; video co-training without future-video generation at inference.
• Fast-ThinkAct: Efficient Vision-Language-Action Reasoning via Verbalizable Latent Planning — Preference-guided distillation compresses reasoning into verbalizable latents for action policies; evaluated in simulation and embodied reasoning benchmarks.
• ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planning — Action-aligned reinforcement learning connects a vision-language reasoner to a diffusion policy through visual plan latents; simulation evaluation.
• Fast-in-Slow: A Dual-System Foundation Model Unifying Fast Manipulation within Slow Reasoning — Partially shared System 1/System 2 parameters with asynchronous observations and action generation; simulation and real-robot evaluation.
• Agentic Robot: A Brain-Inspired Framework for Vision-Language-Action Models in Embodied Agents — Planner–executor–verifier coordination with subgoal verification and recovery; evaluated on LIBERO.
• OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoning — A unified VLA predicts reasoning or action mode tokens to think at critical moments and otherwise execute action chunks.
• GR00T N1: An Open Foundation Model for Generalist Humanoid Robots — An Eagle-2 vision-language System 2 conditions a flow-matching System 1 action module, jointly trained across heterogeneous data.
• Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Models — A high-level VLM guides a low-level VLA through language; planning refreshes periodically and on user feedback.
• Robotic Control via Embodied Chain-of-Thought Reasoning — ECoT; a VLA reasons about plans, sub-tasks, motion and visually grounded state before predicting robot actions.

Self-Evolving Robot Agents

Systems that turn experience into reusable knowledge, skill programs, improved policies, or validated capability upgrades. Includes human-guided methods and learned-policy precursors where noted.

Memory and Knowledge Evolution

• RoboHarn-Evo: Evolving Hierarchical Physical Knowledge for Self-Improving Robotic Manipulation — A dual-loop manipulation harness evolves persistent task and action knowledge from physical feedback while keeping the base model fixed, supporting retrieval, correction, and reuse across episodes.

• Recursive Harness Distillation across Agents for Robot Manipulation — A strong agent distills and refines intervention experience into a reusable playbook for lighter agents operating frozen robot policies; evaluated in simulation and real-world manipulation.

• NavHarness: Towards Lifelong Embodied Navigation — A training-free navigation harness that carries maps, task records, recovery handovers, and consolidated knowledge across reasoning sessions; evaluated in simulated navigation environments.

• MessyMem: Learning-from-Doing Memory for Mobile Manipulation

• Robo-Cortex: A Self-Evolving Embodied Agent via Dual-Grain Cognitive Memory and Autonomous Knowledge Induction

• Distilling and Retrieving Generalizable Knowledge for Robot Manipulation via Language Corrections

Skill and Program Evolution

• RoboRSI: Stable, Efficient, and Reusable Robot Self-Evolution in Complex Real-World Environments — Human-guided skill-tree refinement attributes failures, revises scoped capabilities, and validates reusable skills; includes 104 evolving real-robot development rounds and simulation benchmarks.

• Embodied Turing Machines: Stateful Code for Robot Recursive Self-Improvement — Coding agents develop a shared library of stateful robot programs from execution feedback; the resulting code-only policies run without VLM or VLA calls at test time, evaluated on simulated bimanual tasks.

• RoboBridge: A Self-Evolving Embodied Agent Framework for Sim-to-Real Transfer — Transfers revisable procedural skills from simulation to physical tasks and validates further skill updates from execution feedback, exposing a frozen VLA as a tool with optional inference-time guidance.

• Skill2Real: Agentic Skill Learning for Zero-Shot Sim-to-Real Robot Manipulation — Learns hierarchical executable skills through a shared robot API using simulation diagnosis and validation gates, then transfers frozen skill memories to a physical UR5e without target-task policy fine-tuning.

• InterEvolve: Test-Time Evolution of Reward Programs for Humanoid Loco-Manipulation — An LLM evolves staged reward programs using simulation feedback and a verified skill library, while numerical search tunes constants for a fixed pretrained humanoid controller; includes physical G1 deployment.

• Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents — Improves a persistent skill library and system prompt through simulation practice, privileged failure diagnosis, and cross-task validation, then freezes the calibrated system for physical deployment.

• RoboRSI: Stable, Efficient, and Reusable Robot Self-Evolution in Complex Real-World Environments — Research blog and open-source harness; multiple agents refine a task–skill tree from execution feedback, consolidate workflows into code, and gate skill changes with regression checks.

• SimEX: Simulation-Integrated Robotics AutoResearch — A coding agent develops reusable robot toolboxes in simulation and uses limited real-world trials to repair both tools and simulators; demonstrated on three bimanual task families.

• Explore, Execute, Evolve: A Skill Acquisition and Reuse Loop for Embodied Agents — RoboSkill; an exploration–execution–evolution loop accumulates reusable textual and executable skills from robot experience, with visual and tactile feedback and cross-agent skill reuse.

• Self-Evolving Coding Agents: From Digital Programs to Physical-World Intelligence — HexaAnything; executable world state and control workflows support tool-based robot execution, tool refinement, and trace-based model training; includes simulation and physical-robot studies.

• RoboFoundry: System-as-Policy Evolution for Self-Learning Embodied Agents — Evolves context management, persistent memory, and hierarchical skills through validated execution-driven system edits while keeping the foundation model frozen; includes physical robot evaluation.

• HarnessPAI: An Evolving Harness for Physical AI — Coding agents refine executable robot harnesses between rollouts using execution feedback, while keeping the underlying action backends frozen.

• RACaP: Agentic Reasoning, Acting, and Coding as Policies for Evolvable Robot Learning — Combines reasoning, acting, and code evolution to build reusable robot APIs and harness memory; evaluated in simulation.

• AdaHVLA: Adaptive Harnesses for Long-Horizon Vision-Language-Action Execution — Refines code-based coordination between reasoning agents and frozen VLAs through rollout evidence and persistent revision memory; simulation benchmarks and a real-world quadruped deployment case.

• Learning and Transferring Closed-Loop Robot Software — Coding-agent optimization and reuse of closed-loop robot programs in simulation.

• Self-Evolving Embodied Agents via Skill-Harness Evolution — SHAPER; frozen-model skill and harness optimization.

• ASPIRE: Agentic /Skills Discovery for Robotics

• Lifelong Robot Library Learning: Bootstrapping Composable and Generalizable Skills for Embodied Control with Language Models

• Eureka: Human-Level Reward Design via Coding Large Language Models

• DrEureka: Language Model Guided Sim-To-Real Transfer

Autonomous Data Collection and Policy Improvement

• Co-Evolving Robot Orchestrators and Policies through Deployment — Co-evolves a VLM orchestrator and robot policy using curated deployment experience, skill-level candidate verification, and rollback; evaluates frozen policy and memory on held-out simulated and physical tasks.
• Find Something You Can't Do: Agentic Real-World Reinforcement Learning for Self-Improving VLA Models — A vision-language agent selects feasible practice tasks and evaluates outcomes in a persistent workspace, driving residual reinforcement learning over a frozen VLA with reduced manual resets.
• MobiAgent: Dual-Loop Recursive Policy Self-Improvement for Long-Horizon Mobile Manipulation — A VLM planner and visual critic compose reusable skill policies, while an offline outer loop segments, verifies, and recycles deployment trajectories to fine-tune the skill library.
• SkillWeaver: Agentic Exploration over Neural Interaction Skills for Scalable Robot Data Generation — A VLM agent composes reusable closed-loop neural interaction skills through verifier-guided exploration, reflection, and memory to generate simulated robot demonstrations for downstream policy learning.
• KnowDemo: Knowledge-Guided Robot Demonstration Generation from Human Videos — VLM-based task-knowledge extraction for generating diverse, validated robot demonstrations.
• MAGMA-GEN: Validated Recovery Supervision from Ambiguous Failures via Counterfactual Re-Execution — Simulation-validated recovery data for fine-tuning tool-using robot language policies.
• RoboClaw: An Agentic Framework for Scalable Long-Horizon Robotic Tasks
• Zero2Skill: Bootstrapping Robot Skills through Autonomous Data Collection, Training, and Deployment
• ENPIRE: Agentic Robot Policy Self-Improvement in the Real World — Coding-agent-driven real-world policy and algorithm improvement.
• HARBOR: A Harness Framework for Agentic Robot Reinforcement Learning
• AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents
• Autonomous Improvement of Instruction Following Skills via Foundation Models
• RoboCat: A Self-Improving Generalist Agent for Robotic Manipulation
• RISE: Self-Improving Robot Policy with Compositional World Model
• AllDayNav: Lifelong Navigation via Real-World Reinforcement Learning

Capability Upgrades and Regression Control

• Scale and Selection: What Makes Automatic Harness Evolution Work for Visual-Interface Robot Agents — Studies rollout scale and matched champion–challenger selection for agent-driven evolution of visual robot harnesses, measuring generalization to held-out simulated tasks.
• DynaHarness: A Dynamic Physical Harness for Self-Evolving Robot Agents — A semantic planner and deterministic execution governor share contracts for refusal, substitution, and recovery, with evidence-based failure attribution and regression-gated capability updates around a frozen policy.
• Learning Without Losing Identity: Capability Evolution for Embodied Agents
• Governed Capability Evolution: Lifecycle-Time Compatibility Checking and Rollback for AI-Component-Based Systems, with Embodied Agents as Case Study

Related system-level memory and learning mechanisms: PhyAgentOS, ABot-Claw, and WCM. Execution-time recovery methods are listed under Safe Planning, Verification and Failure Recovery.

Embodied Agent Operating Systems and Runtimes

Persistent embodied-agent systems that organize robot capabilities, state, resources, execution checks, and feedback across tasks or robots.

• NavGPT-3: Harnessing Context in a Hierarchical Navigation Runtime — Connects language-model planning, a trained navigation VLA, and monitoring through a hierarchical runtime with separate contexts and motion permissions; evaluated in navigation benchmarks and physical interruption studies.
• RegenHarness: A Robot Agent Harness with Evidence-Gated Recursive Self-Improvement — Evidence-gated skill execution, versioned task state, bounded recovery, and a protocol for regression-checked harness configuration updates; demonstrated through quadruped inspection cases.
• NIMO Controller: a self-driving laboratory orchestrator based on the Model Context Protocol — MCP-based tool discovery and experimental workflow orchestration for AI agents and human users in self-driving laboratories.
• Retriever: Composing the Perception-Reasoning-Action Loop for Long-Horizon Manipulation — Asynchronous runtime and deterministic replay.
• Harness Robotic OS: A Unified Embodied-Agent Runtime for Closed-Loop Quadruped Inspection
• PhyAgentOS: A Self-Evolving Operating System for Embodied Agents with Decoupled Cognitive Planning and Physical Execution
• RoboOS: A Hierarchical Embodied Framework for Cross-Embodiment and Multi-Agent Collaboration
• ROSClaw: An OpenClaw ROS 2 Framework for Agentic Robot Control and Interaction
• ABot-Claw: A Foundation for Persistent, Cooperative, and Self-Evolving Robotic Agents
• AEROS: A Single-Agent Operating Architecture with Embodied Capability Modules
• HoloAgent-0: A Unified Embodied Agent Framework with 3D Spatial Memory
• EmbodiedSkills: A Unified Framework for Orchestrating, Training, and Deploying VLA Agents
• WCM: World-Cognition Model for Generalizable Human-Robot Interaction
• EMOS: Embodiment-aware Heterogeneous Multi-robot Operating System with LLM Agents

Programming and Spatial Action Interfaces

• Fewer Tokens, Better Action: GPT-6 Astra Robot Agents with 14% Higher Success Rate but 65% Fewer Tokens — An interactive Python interface composes robot primitives and VLA policies with conditional checks, local retries, and selective feedback; evaluated against tool calling on 700 paired simulated task instances.

• InCoRo: In-Context Learning for Robotics Control with Feedback Loops — An off-the-shelf LLM updates robot commands from scene-state feedback and in-context examples; demonstrated on physical SCARA and DELTA robots in static and dynamic environments.

• Make Code as Policy Great Again: Frontier Agents Write, Call, and Evolve Robot Tools — URAI; programming agents develop reusable robot tools with execution feedback and human guidance, while execution agents call validated tools for simulated and real bimanual tasks.

• KPI: A Promptable Kernel for Physical Interaction on Humanoids — A vision-language agent specifies reference trajectories and interaction contracts for a contact-control kernel above an unchanged whole-body tracker; demonstrated on instruction-driven humanoid interaction tasks.

• Encore: Few-Shot Agentic Discovery of Manipulation Strategies — A coding agent reads structured demonstration evidence, develops policy programs through a fixed perception-and-action API, and freezes them for held-out evaluation; includes two bimanual real-robot tasks.

• CognitiveReality: Robot-Agnostic Semantic Gaussian Mapping with an LLM Agent for Immersive Collaborative VR Teleoperation — An LLM routes language and spatial references to typed tools and robot missions over a shared semantic map, with operator confirmation and physical quadruped evaluation.

• RAPID: Robot Agentic Programming from Demonstrations — Converts visual demonstrations into robot programs through task specifications, reusable primitives, and interactive verification.

• Coding Agents for Generalized Task and Motion Planning Problems — Coding agents synthesize reusable planning programs for simulated environments; held-out evaluation executes fixed programs without test-time LLM calls.

• ManiSkillFormer: Demonstration-Free Compositional Manipulation via Geometric Contracts and Agentic Skill Graph

• AntiGrounding: Executable Robot Trajectories as Visual Prompts for VLM-Guided Manipulation — VLM selection of executable trajectories through a visual interface and an initialized digital twin.

• Auto-HSI: Personalized human control of a robot swarm on demand by using LLMs for online automatic code generation — Human-in-the-loop interface.

• KINO: A Keyframe Interface for VLM Planning and Whole-Body Control in Humanoid Loco-Manipulation — Learned low-level controller.

• AnchorVLN: Geometry-Anchored Vision-Language Grounding Reasoning for Open-Vocabulary Navigation

• GTA-2: A Multi-VLM Framework for Synthesizing Robot Manipulation Skills via Grounded Task Axes

• Evolve Vision-Language-Action Model into an Agent with On-the-fly Tool-use

• Code as Policies: Language Model Programs for Embodied Control

• ProgPrompt: Generating Situated Robot Task Plans using Large Language Models

• ChatGPT for Robotics: Design Principles and Model Abilities

• VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models

• ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation

• PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs

• MOKA: Open-World Robotic Manipulation through Mark-Based Visual Prompting

• SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulation

• LangNav: Language as a Perceptual Representation for Navigation

• LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action

• Language to Rewards for Robotic Skill Synthesis

• Trust the PRoC3S: Solving Long-Horizon Robotics Problems with LLMs and Constraint Satisfaction

Planning, Skill Orchestration and Memory

Agents that select and coordinate robot capabilities, track state, verify outcomes, and recover from failures.

Task Planning, Skill Orchestration and Memory

Task decomposition, reusable skill orchestration, and task-time memory. Includes learned hierarchical planners and action models where applicable.

• AgenticNav: Zero-Shot Vision-and-Language Navigation as a Tool-Calling Harness — Exposes pixel-target actions, on-demand depth queries, and selective visual memory recall as tools for a VLM navigation agent; evaluated in VLN-CE and on two physical robot platforms.

• NavHarness: Adaptive Goals for Agentic Vision-Language Navigation — Four agents coordinate goal generation, verification, memory, and visuomotor execution, compressing navigation history at verified goal boundaries; evaluated on 24 real-robot route trials.

• RoboHarness: Memory-Driven Orchestration of Heterogeneous Robot Policies for Long-Horizon Planning — An agentic orchestration layer exposes heterogeneous robot policies as reusable skills, using execution evidence and memory to route tasks and bridge policy handoffs without joint retraining; simulation and real-robot evaluation.

• Simple Agentic Memory for Generalist Robot Policies — SimpleARM; a training-free external memory layer maintains task-relevant typed state, retrieves history-dependent information, and re-grounds recalled entities for frozen robot policies; evaluated on RoboMME.

• NavJev: Efficient Vision-Language Navigation via Action-Centric Visual Compression and Discriminative Action-Semantic Memory — Action-centric visual compression and discriminative action-semantic memory support typed Jev decisions for vision-language navigation, evaluated in R2R-CE and two real indoor environments. Cross-cutting reasoning/action coupling and System 1/System 2 designs are listed under Reasoning-Acting and Dual-System Architectures.

• Actively Resolving Contextual Uncertainty for Underspecified Tasks in Natural Language — CLUE; an LLM agent resolves ambiguous goals through map queries, inspection, navigation, and manipulation skills, with closed-loop evaluation on a physical Spot robot.

• NavProbe: Evidence-Grounded Reasoning with Active Memory Retrieval for Zero-Shot Navigation — A hierarchical VLM navigation agent that retrieves visual and geometric evidence to revise subgoals and select parameterized navigation skills.

• World Action Planner: Generalizable Decision-Making with Action-Conditioned World Models — VLM action plans are refined through action-conditioned world-model imagination, optimization and search; simulation evaluation.

• MistyPilot: Enabling Social-Robot Control through Multi-Agent LLM Skill Orchestration — Natural-language skill orchestration, sensor-event binding, and dialogue-state management on a physical social robot.

• Bridging Thought and Action: Taming Long-Horizon Instability in Open-Source LLM Agents with a MetaTool-Enhanced ROS Framework

• 2AM: Grounding Agent-Side Memory as Guidance for Steerable Action Models in Long-Horizon Manipulation

• Memory as Plans: World-Action Modeling with Memory-Grounded Planning

• Towards Long-horizon Embodied Agents with Tool-Aligned Vision-Language-Action Models

• Do As I Can, Not As I Say: Grounding Language in Robotic Affordances

• SayCanPay: Heuristic Planning with Large Language Models using Learnable Domain Knowledge — Offline action-sequence search; simulation evaluation.

• Inner Monologue: Embodied Reasoning through Planning with Language Models

• SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planning

• RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Robotics

• Towards the Harness of Embodied Agents

• Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agents

• Being-0: A Humanoid Robotic Agent with Vision-Language Models and Modular Skills

• MOSAIC: Modular Foundation Models for Assistive and Interactive Cooking

• Creative Robot Tool Use with Large Language Models — RoboTool; executable plans over parameterized skills.

Safe Planning, Verification and Failure Recovery

Methods that assess risks, verify execution, and trigger corrective planning or recovery. Failure-recovery benchmarks are listed under Infrastructure and Benchmarks.

• Recova: Agent-Guided Failure Recovery for Autonomous Robotic Manipulation — An agent develops recovery programs in a digital twin and monitors, restores, verifies, and resumes robot tasks; real-world experience and human demonstrations improve separate task and recovery policies.

• FRAMES: Failure Recovery And Monitoring of Embodied Skills for Humanoid Loco-Manipulation — A planner, VLM monitor, recovery agent, and memory module form a failure-aware supervisory loop for humanoid skills.

• CommitFlow: Semantic Commitment Verification and Local Correction for Long-Horizon Robot Manipulation VLA Execution — Frozen-policy execution harness with semantic commitment monitoring, local correction and stage-level verification.

• When Should a Failing Robot Ask? Initiating Corrective Human-Robot Dialogue from Audited Sensor Evidence — Evidence-aware choice between autonomous action, additional sensing and human assistance after failure.

• Coding Agents with an Obstacle-Aware Harness for Safe Robot Manipulation — SafeHarness; obstacle-aware route verification, replanning, and contact execution, evaluated in simulation.

• GAVEL: Graph World Models for Verified and Efficient Long-Horizon LLM Task Planning

• Safe Task Planning with Long-Term Graph Memory for Embodied Agents

• VLA-Corrector: Stage-Aware Observable State Understanding for Prompt-Based Closed-Loop Recovery of Vision-Language-Action Policies

• CoPAL: Corrective Planning of Robot Actions with Large Language Models

• REFLECT: Summarizing Robot Experiences for Failure Explanation and Correction

• DoReMi: Grounding Language Model by Detecting and Recovering from Plan-Execution Misalignment

• AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation

• Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners — KnowNo; calibrated uncertainty and human clarification.

Multi-Robot Coordination

Task allocation, communication, and organizational structures for teams of robots or embodied agents. ORCH is evaluated in simulation; system-level runtimes are listed under Embodied Agent Operating Systems and Runtimes.

• Managing Context and Communication in Distributed Agentic UAV Swarms — Combines local language-model agents, structured note memory, and deterministic interest-aware gossip for distributed UAV coordination; evaluated with ten UAVs in simulated search and rescue.
• DuoMind: Enabling Distributed Multi-Robot Coordination with Semantic Communication — Distributed VLM orchestrators exchange semantic messages and guide fine-tuned VLA controllers through language, evaluated on two-agent RoboPoly and RoboTwin manipulation tasks in simulation.
• Systematic Multi-Agent Vision-and-Language Navigation: Formulation, Benchmark, and Method — MAVLN and TRISS; constrained multi-agent navigation with an LLM subtask scheduler, shared topological memory, and conflict-aware execution, evaluated in simulated scenes.
• AeroWeaver: An Embodied-Agent Harness for Weaving Aerial Skills into Distributed, Adaptive Swarm Execution — Simulation.
• ORCH: Organizational Principles Enable Collective Intelligence in Embodied AI
• RoCo: Dialectic Multi-Robot Collaboration with Large Language Models
• SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models

Language-Native Actions and Cross-Embodiment Transfer

Related learned-policy methods that preserve language interfaces or reduce adaptation to new embodiments; these generally involve robotics training.

• Actions as Language: Fine-Tuning VLMs into VLAs Without Catastrophic Forgetting
• LAP: Language-Action Pre-Training Enables Zero-shot Cross-Embodiment Transfer
• VLA-0: Building State-of-the-Art VLAs with Zero Modification
• LLARVA: Vision-Action Instruction Tuning Enhances Robot Learning
• LLaRA: Supercharging Robot Learning Data for Vision-Language Policy
• CLAP: Direct VLM-to-VLA Adaptation via Language-Action Grounding
• RT-H: Action Hierarchies Using Language
• RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
• In-Context World Modeling for Robotic Control

Infrastructure and Benchmarks

• RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments — Benchmarks multimodal agents through robot interfaces across 84 simulated tasks and diverse embodiments, with explicit interaction budgets and executable success checks; reports outcomes and execution-trace limitations.

Jev+Robot

• robo-jev: A 10 Hz Typed-Decision Layer for Physical Robots — A typed decision layer with deterministic execution; its R13 simulation audit passed a registered comparison after delegating gripper, path, and force choices to rules, while semantic and stop failures remain.

• EmbodiedJev: MuJoCo Robot Decision Workbench — Browser-based MuJoCo and Franka Panda workbench supporting Jev, Claude, OpenAI-compatible APIs, and local MiniCPM models with visible observe–decide–execute–feedback loops.

• RoboJEV: Two-Stage JEV Control of a Franka Panda in MuJoCo — Two-stage typed decisions for task intent followed by Cartesian motion and gripper commands, evaluated with independent physical success checks.

• Jev Robot Control — Reproducible xArm7 MuJoCo comparison of Jev, GPT-6 Astra, and GPT-4.1 mini with archived trajectories, offline verification, and replay.

• robojev: Natural-Language Control of a WidowX AI Arm with Jev Typed Judgments — A real-hardware prototype using typed Jev decisions to sequence reusable manipulation primitives; reported demonstrations are heavily fitted to one paper cup and one table.

Robot integration, deployment, latency, runtime reliability, and evaluation of model-plus-interface systems. Includes benchmarks for memory, safety, and recovery, as well as surveys of robot policy verification.

• RoboQuest: Generalist Physical Agents that Search, Inspect and Test — Evaluates information-seeking mobile-manipulation agents on ten simulated search, inspection, and interactive-testing tasks, separating missing evidence and premature commitment from execution failures.
• Are Frontier VLM Agents Ready to Be Robot Generalists? An Empirical Study with the Embodied Agent Arena — Evaluates seven VLM agents across 1,000 cases spanning geometry, spatial reasoning, affordance, planning, and manipulation, separating local competence from complete task success under source-specific interfaces.
• DrivingBench: Can Vision-Language Models Drive a Toyota Corolla? — A physical-vehicle benchmark exposing three MCP tools to vision-language models, evaluated at parking-lot speeds with a supervising safety driver and bounded retry attempts.
• LIBERO-Agent: Evaluating General-Purpose Agents for Direct Embodied Manipulation — An agent-native manipulation benchmark with evaluator-owned state and success checks; the public source release provides the 30-task primary suite, while the broader catalog is reserved for a later release.
• RLE-Bench: A Qualifying Exam for Coding Agents as Robot Learning Engineers — Evaluates coding agents across interactive robot control, policy development, perception and estimation, and mechanical design under explicit resource budgets and independent artifact evaluation.
• EmbodiedSWE: Coding Agents for Long Horizon Dexterous Robotics — A simulation benchmark for coding agents and a pipeline that diversifies verified solutions into demonstrations for robot policy learning.
• From Rollout to Reset: A Graph-Based Harness for Autonomous Long-Horizon Manipulation Evaluation — HALTER; scene-graph-based evaluation, reset planning, and reset verification on a physical robot.
• VABench: Measuring Embodied Spatial Intelligence through Visual Demonstrations, Active Perception, and Metric Control — Simulation benchmark for general-purpose MLLMs using active camera control, Cartesian action commands, and execution feedback.
• Towards Embodied Air-Ground Cooperative Object Search: Benchmark, Dataset and Agentic Method — AGOS; simulated aerial-ground collaboration.
• FluxVLA Engine: A One-Stop VLA Engineering Platform for Embodied Intelligence — Engineering platform.
• No Free Checker: A Survey of Verifiers for Robot Policies — Survey.
• EvoNav-Bench: Benchmarking Lifelong Navigation in Evolving Environments
• ReactHuman: A Physics-Grounded Benchmark for Human-Like Reactive Decision-Making in Embodied Multimodal LLMs
• MEMOBench: A Process Level Memory Benchmark for Robotic Manipulation
• LIBERO-RECOVER: Beyond Task Success Towards Failure Recovery in Robotic Manipulation Models
• Enabling Novel Mission Operations and Interactions with ROSA: The Robot Operating System Agent
• RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation — ROS-based deployment and code-generation benchmark.
• SPINE: Bridging the Cyber-Physical Gap with Agentic AI
• Harness Engineering for Physical AI: Robot Middleware Is the Harness Layer
• Reducing Latency in LLM-Based Natural Language Commands Processing for Robot Navigation
• EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents
• Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making
• PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Tasks

Perspectives and Reports

These are essays, research blogs, and evaluations; they are listed separately from papers.

• What Stops Recursive Self-Improvement in Robotics? Lessons from 123 Rounds of Agentic Skill Discovery — Technical report; a simulation case study of agent-driven skill discovery, exposing perception, evaluation, and memory bottlenecks despite 123 improvement rounds without success on the primary target task.
• Robot-Use Agents — Phillip Isola, 2026. Perspective.
• Claude plays robotics — Anthropic, 2026. Research report.
• Introducing Waddle: agents that control robots — Waddle Labs, 2026. Research blog / demo.
• GPT-6 Astra on robotic manipulation — Robocurve, 2026. Independent evaluation.
• Introducing Auto Engineering for Robotics - General Robotics, 2026. Blog / demo.