We presented our work on grounded embodied question-answering at the Generative Modeling meets HRI (GenAI-HRI) workshop at Robotics: Science and Systems (RSS) 2024 in Delft, Netherlands.
Our approach uses RAG (Retrieval-Augmented Generation) methods to ground foundation model (LLM) responses in actual robot state summaries from existing robot modules, enabling truthful question-answering about the robot’s current situation and capabilities.