What if you could adapt a robot’s behavior just by talking to it? Our framework IROSA enables open-vocabulary robot skill adaptation through natural language using a tool-based architecture that maintains a protective abstraction layer between the LLM and robot hardware.
The LLM selects and parameterizes validated tool functions that modify the underlying motion model, enabling speed adjustments, trajectory corrections, and obstacle avoidance — all without any fine-tuning, zero-shot using only tool descriptions.
Validated on a 7-DoF torque-controlled DLR SARA robot performing an industrial bearing ring insertion task.