LLMs help robots understand vague instructions and focus on key details
To help robots do chores in places like homes and factories, a new approach from MIT uses one language model to clarify users' instructions, then another to ignore irrelevant info. MIT CSAIL's "Masked IRL" algorithm helps a robot understand ambiguous instructions so it does chores safely. An LLM first elaborates on users' prompts based on demonstration data, then another narrows down which details an algorithm should incorporate into a motion plan.
Key Takeaways
- Alex Shipps | MIT CSAIL Publication Date : June 26, 2026 Press Inquiries Press Contact : Rachel Gordon Email: rachelg@csail.
edu MIT Computer Science and Artificial Intelligence Laboratory : "Masked IRL" helps a robot understand ambiguous instructions so it does chores safely.
- You'll prefer that the robot doesn't get too close to you and the laptop so that it doesn't interrupt your meeting.
To enable this behavior, the robot should be trained with data that clearly demonstrates the full task.
- Their "Masked Inverse Reinforcement Learning" (Masked IRL) approach uses a large language model (LLM) to elaborate on ambiguous prompts based on the data collected from a user's demo.
Another LLM then narrows down which details an algorithm should incorporate into a motion plan, so that a robot can safely complete chores in homes, offices, and factories.
- For example, a machine grabbing you a snack from the kitchen may not know to avoid bumping into your laptop.
Likewise, a factory robot placing items into different boxes must carefully navigate around shelves.
- The model also elaborates on what might be unclear in a prompt, turning a request like "stay close" into "stay close to the surface of the table.
MIT CSAIL's "Masked IRL" algorithm helps a robot understand ambiguous instructions so it does chores safely. An LLM first elaborates on users' prompts based on demonstration data, then another narrows down which details an algorithm should incorporate into a motion plan. To help robots do chores in places like homes and factories, a new approach from MIT uses one language model to clarify users' instructions, then another to ignore irrelevant info.
Alex Shipps | MIT CSAIL Publication Date : June 26, 2026 Press Inquiries Press Contact : Rachel Gordon Email: rachelg@csail. edu MIT Computer Science and Artificial Intelligence Laboratory : "Masked IRL" helps a robot understand ambiguous instructions so it does chores safely. An LLM first elaborates on users' prompts based on demonstration data, then another narrows down which details an algorithm should incorporate into a motion plan.
Credits : Image: Gabriel Maragaño Previous image Next image Imagine working at a warehouse or office sometime in the near future, and you're asked to help a new trainee learn the basics of their job. To teach them, you might want to play a game of "show and tell" - that is, physically showing how to do something a few different ways, while also explaining what you're doing. Let's say you asked the robot to place some coffee on your desk without disturbing you during a Zoom call.
For more details please read the original article at MIT News AI.
Continue Learning
Comments
Sign in to join the conversation