The brain controlled Mars rover

It’s cool enough that people are making robots to crawl around Mars, but what if we could con­trol those robots with nothing but our minds?! Well, that’s pretty much exactly what North­eastern grad stu­dent Umut Orhan of assis­tant pro­fessor Deniz Erdogmuscog­ni­tive sys­tems lab did on Saturday.

Except instead of Mars, the robot was in Worcester. But still.

So…what in the heck is she talking about? you may wonder. Here’s the deal:

Erdogmus’ lab focuses on signal pro­cessing, machine learning, and their appli­ca­tions to con­tem­po­rary prob­lems in biology and bio­med­ical engi­neering. In one project, Erdogmus has teamed up with fellow assis­tant pro­fes­sors of elec­trical and com­puter engi­neering Gunar Schirner and Kaushik Chowd­hury as well as Worcester Poly­technic Institute’s robotics engi­neering pro­fessor Taskin Padir, to develop a Brain Com­puter Inter­face (BCI) for the pop­u­la­tion of func­tion­ally locked-​​in indi­vid­uals, who are unable to interact with the phys­ical world through move­ment and speech.

An alien life form hanging out behind a rock at the RASC-​​AL competition

A BCI is a pro­gram that turns elec­trical sig­nals from the brain (cap­tured through elec­trodes strate­gi­cally placed on the head using the cap you see in the photo) into com­puter com­mands. There are sev­eral ways of doing this, but one of the most pop­ular uses stim­u­la­tion of the visual cortex. A patient looks at a screen with a set of checker­boards each flashing at their own fre­quency and each capable of gen­er­ating a unique elec­trical signal in the brain. Padir will develop robots that can respond to these elec­trical sig­nals for the pur­poses of both com­mu­ni­ca­tion and self-​​feeding, two of the most impor­tant pri­or­i­ties for the locked-​​in community.

Another of Padir’s robotics projects is designing Oryx, a “plan­e­tary explo­ration mobility plat­form” (aka planet crawling robot or rover). For two years in a row now, Oryx has won the national Stu­dent Rover Design Com­pe­ti­tion held by NASA and the National Insti­tute of Aero­space. RASC-​​AL Robo-​​Ops, as the com­pe­ti­tion is called, asks teams of under­grad­uate and grad­uate stu­dents to design and build a rover and then con­trol it from a remote loca­tion to pick up rocks and “alien life forms.” Tra­di­tion­ally, Oryx is con­trolled using stan­dard key­board and mouse com­mands via wire­less computer.

But the research teams came up with an even better idea. Over the weekend, Erdogmus and Padir’s labs par­tic­i­pated in WPI’s Touch­To­morrow fes­tival of sci­ence, tech­nology and robotics by com­bining BCI with Oryx. Instead of con­trol­ling a robot that brings food to a locked-​​in patient’s mouth, they con­trolled the rover with the same tech­nology. This par­tic­ular project was spear­headed by grad stu­dent Hooman Nezamfar.

Orhan and one of Erdogmus’ post doc­toral researchers, Murat Akcakaya, holed up in a dimly lit room in the cog­ni­tive sys­tems lab on Sat­urday morning while the other half of the team enjoyed the sun­shine over in Worcester. With a streaming video from a camera mounted atop Oryx, the Boston duo could see what the robot saw. When they con­nected the BCI to Oryx over the 4G net­work, stim­u­la­tion of Orhan’s visual cortex trans­lated into move­ment com­mands for the rover.

Oryx in Worcester. The dudes in the back­ground are skyping with Orhan and Akcakaya, trying to remedy the net­work con­nec­tion failure.

There were a few glitches: the net­work was all sorts of clogged so there was a sig­nif­i­cant delay between Orhan’s visual com­mands and the robot’s move­ments, but oth­er­wise it was a pretty stellar thing to watch. Very slight move­ments in Orhan’s visual atten­tion (even his periph­eral vision) caused the robot 50 miles away to do its little dance for the kids and adults watching at the festival.

Orhan said that in addi­tion to ther­a­peutic appli­ca­tions, BCI could be useful for mil­i­tary pur­poses. “It could allow a pilot to con­trol a plane if his hands were indis­posed or inca­pac­i­tated,” he said.