LLMs help robots understand vague instructions and focus on key details
To help robots do chores in places like homes and factories, a new approach from MIT uses one language model to clarify users' instructions, then another to ignore irrelevant info. MIT CSAIL's "Masked IRL" algorithm helps a robot understand ambiguous instructions so it does chores safely. An LLM first elaborates on users' prompts based on demonstration data, then another narrows down which details an algorithm should incorporate into a motion plan.
Key Takeaways
- Alex Shipps | MIT CSAIL Publication Date : June 26, 2026 Press Inquiries Press Contact : Rachel Gordon Email: rachelg@csail.mit.edu MIT Computer Science and Artificial Intelligence Laboratory : "Masked IRL" helps a robot understand ambiguous instructions so it does chores safely.
Credits : Image: Gabriel Maragaño Previous image Next image Imagine working at a warehouse or office sometime in the near future, and you're asked to help a new trainee learn the basics of their job.
- To enable this behavior, the robot should be trained with data that clearly demonstrates the full task.
Computer scientists have attempted to explain manipulation tasks to robots by recording lots of physical demonstrations or writing extensive directions.
- Another LLM then narrows down which details an algorithm should incorporate into a motion plan, so that a robot can safely complete chores in homes, offices, and factories.
"Our approach could come in handy when a human interacts with a robot but doesn't want to spell out all the details of a task," says MIT PhD student and CSAIL researcher Minyoung Hwang, who is a lead author on a paper presenting the project.
- Likewise, a factory robot placing items into different boxes must carefully navigate around shelves.
To learn new tasks in these situations, Masked IRL uses the robot's sensors to capture information about its surroundings.
- The model also elaborates on what might be unclear in a prompt, turning a request like "stay close" into "stay close to the surface of the table."
MIT CSAIL's "Masked IRL" algorithm helps a robot understand ambiguous instructions so it does chores safely. An LLM first elaborates on users' prompts based on demonstration data, then another narrows down which details an algorithm should incorporate into a motion plan. To help robots do chores in places like homes and factories, a new approach from MIT uses one language model to clarify users' instructions, then another to ignore irrelevant info.
Alex Shipps | MIT CSAIL Publication Date : June 26, 2026 Press Inquiries Press Contact : Rachel Gordon Email: rachelg@csail.mit.edu MIT Computer Science and Artificial Intelligence Laboratory : "Masked IRL" helps a robot understand ambiguous instructions so it does chores safely. An LLM first elaborates on users' prompts based on demonstration data, then another narrows down which details an algorithm should incorporate into a motion plan. Credits : Image: Gabriel Maragaño Previous image Next image Imagine working at a warehouse or office sometime in the near future, and you're asked to help a new trainee learn the basics of their job.
To teach them, you might want to play a game of "show and tell" - that is, physically showing how to do something a few different ways, while also explaining what you're doing. Let's say you asked the robot to place some coffee on your desk without disturbing you during a Zoom call. You'll prefer that the robot doesn't get too close to you and the laptop so that it doesn't interrupt your meeting.
For more details please read the original article at MIT News AI.
Continue Learning
Comments
Comments appear only after moderation. Your email identifies your submission to the moderator and is never displayed here.
No approved comments yet.