There are countless reasons why home robots have had little success since Roomba. Price, practicality, form factor, and mapping all contribute to failure after failure. Even if some or all of them are resolved, the question remains: what happens when the system makes the inevitable mistakes?
This is also a source of friction at the industry level, but large companies have the resources to respond appropriately when problems arise. But you can't expect consumers to learn programming or hire someone to help them whenever a problem arises. Thankfully, this is an excellent example of the use of his LLM (Large-Scale Language Model) in the field of robotics, as demonstrated in a new study from MIT.
The research, to be presented at the International Conference on Learning Representations (ICLR) in May, aims to bring a bit of “common sense” into the process of correcting mistakes.
“It turns out that robots have excellent imitation abilities,” the school explains. “But unless engineers program the robot to adapt to any bumps or tremors, the robot won't necessarily know how to deal with these situations other than by starting the task from scratch.”
Traditionally, when a robot encounters a problem, it exhausts its pre-programmed options before requiring human intervention. This is a particularly big problem in unstructured environments like the home, where any change to the status quo can negatively impact the robot's ability to function.
The researchers behind the study found that while imitative learning (learning how to perform a task through observation) is popular in the world of home robots, it fails to account for the countless small environmental variations that can disrupt normal behavior. It is pointed out that there are many cases where a system is needed for this purpose. To start over from square one. New research partially addresses this issue by breaking up demonstrations into smaller subsets, rather than treating them as part of a continuous behavior.
This is where LLM comes in, eliminating the need for programmers to individually label and assign large numbers of subactions.
“LLM has a way of communicating in natural language how to perform each step of a task. The human's continuous demonstration is the embodiment of those steps in physical space,” said graduate student Tsun-Hsuan Wang. say. “And we wanted to connect the two so that the robot automatically knows what stage of the task it is in and can replan and recover on its own.”
The particular demonstration featured in this study involves training a robot to scoop marbles and pour them into an empty bowl. What for humans is a simple, repeatable task, for robots it's a combination of many smaller tasks. LLM can list and label these subtasks. In the demonstration, the researchers disrupted the activity in small ways, such as knocking the robot off course or dropping a marble from a spoon. The system responded by self-correcting small tasks rather than starting from scratch.
“With our method, when a robot makes a mistake, it doesn't need a human to program it or provide additional demonstrations of how to recover from the mistake,” Wang added.
This is a convincing method that will help you avoid losing your marbles completely.