In the basement of MIT & # 39; s Building 3, a robot considers its next movement carefully. It sticks softly to a tower of blocks, looking for the best block to unpack without overthrowing the tower, in a lonely, slow and yet surprisingly agile game by Jenga.
The robot, developed by MIT engineers, is equipped with a soft-pronged gripper, a force-sensing wrist cuff and an external camera, which uses it all to see and feel the tower and its individual blocks.
While the robot gently presses against a block, a computer takes visual and tactile feedback from its camera and cuff and compares these measurements with movements that the robot made earlier. It also takes into account the results of these movements – in particular whether a block, in a certain configuration and with a certain amount of force was pressed, was successfully extracted or not. In real time, the robot "learns" whether to continue pushing or going to a new block to prevent the tower from falling.
Details of the Jenga gaming robot are published in the magazine Science Robotics. Alberto Rodriguez, the Walter Henry Gale Career Development Assistant Professor in the Department of Mechanical Engineering at MIT, says the robot shows something that was difficult to achieve in previous systems: the ability to quickly learn the best way to do a task to perform, not only of visual evidence, as is often studied nowadays, but also of tactile, physical interactions.
"In contrast to more pure cognitive tasks or games like chess or Go, playing the Jenga game also requires mastery of physical skills, such as probing, pushing, pulling, placing and aligning pieces, requiring interactive perception and manipulation, where you have to hit the tower and learn how and when blocks need to be moved, "says Rodriguez. "This is very difficult to simulate, so the robot has to learn in the real world, through interaction with the real Jenga tower. The main challenge is to learn from a relatively small number of experiments by using common sense about objects and physics."
He says that the tactile learning system that the researchers have developed can be used in applications outside of Jenga, especially in tasks that require careful physical interaction, including separating recyclables from landfills and composing consumer products.
"In an assembly line for mobile phones, in almost every step the feeling of a click fit, or a threaded screw, comes from strength and touch rather than from vision," says Rodriguez. "Learning models for those actions is the most important property for this type of technology."
The main author of the paper is MIT graduate student Nima Fazeli. The team also includes Miquel Oller, Jiajun Wu, Zheng Wu and Joshua Tenenbaum, professor of brain and cognitive sciences at MIT.
Push and pull
In the game of Jenga – Swahili for "building" – 54 rectangular blocks are stacked in 18 layers of three blocks each, with the blocks in each layer perpendicular to the underlying blocks. The aim of the game is to carefully unpack a block and place it at the top of the tower, thus building a new level without overthrowing the entire structure.
To program a robot to play Jenga, traditional mechanics for learning a machine may have to capture everything that is possible between a block, the robot and the tower – an expensive computer task that requires data from thousands, so not tens of thousands of block extraction attempts.
Instead, Rodriguez and his colleagues were looking for a more data-efficient way for a robot to learn to play Jenga, inspired by human cognition and the way we might approach the game ourselves.
The team adapted an industry standard ABB IRB 120 robot arm, then set up a Jenga tower within the reach of the robot and started a training period in which the robot first chose a random block and a location on the block to which it was to be pushed. Then it exerted a small amount of force in an attempt to push the block out of the tower.
For each block attempt, a computer recorded the associated visual and forced measurements and labeled whether each attempt was a success.
Instead of executing tens of thousands of such attempts (which would involve reconstructing the tower almost as often), the robot trained at around 300, with attempts of similar measurements and outcomes grouped into clusters representing certain block behaviors. For example, a cluster of attempts may consist of attempts on a block that was difficult to move, versus a block that was easier to move, or that knocked the tower over when it was moved. For each data cluster the robot developed a simple model to predict the behavior of a block, given the current visual and tactile measurements.
Fazeli says that this clustering technique significantly increases the efficiency with which the robot can learn to play the game, and is inspired by the natural way people cluster the same behavior: "The robot builds clusters and then learns models for each of these clusters, instead of learning a model that absolutely captures everything that could happen. "
The researchers tested their approach against other advanced algorithms for machine learning, in a computer simulation of the game using the MuJoCo simulator. The lessons learned in the simulator informed the researchers about how the robot would learn in the real world.
"We offer these algorithms the same information that our system receives, to see how they learn to play Jenga at a similar level," says Oller. "Compared to our approach, these algorithms must explore more than one towers to learn the game."
Curious how their machine-learning approach relates to actual human players, the team conducted a few informal tests with different volunteers.
"We saw how many blocks a person could extract before the tower fell, and the difference was not that much," says Oller.
But there is another way to go if the researchers want to use their robot competitively against a human player. In addition to physical interactions, Jenga requires strategy such as unpacking just the right block that will make it difficult for an opponent to pull out the next block without overthrowing the tower.
For now, the team is less interested in developing a robot Jenga champion and more focused on applying the new skills of the robot to other application domains.
"There are many tasks that we do with our hands, where the feeling to do it the right way is delivered in the language of forces and tactile signals," says Rodriguez. "A similar approach to ours could come true for such tasks."
This research was partially supported by the National Science Foundation through the National Robotics Initiative.