News Center

Imagine a world where robots help those with special health needs continue living independently at home.

That’s what Assistant Professor of Computer Science Janyl Jumadinova and three of her research students are striving to do.

We’re not talking about robots like Rosie from “The Jetsons.” What Jumadinova and Allegheny students Almog Boanos ’17, Michael Camara ’17 and Victor Zheng ’17 are doing is creating a monitoring system consisting of multiple robots, wearable sensors and software that can provide personalized monitoring of a user’s well-being in his or her own home.

For example, if a person is at risk for falling or for having a stroke, the robots can be trained to follow this person and monitor certain parts of his or her condition, such as temperature, speed, location and blood pressure. If there is a sudden change in the data – signaling a life-threatening situation – the robots can send an emergency message to a caregiver’s computer or cell phone, or to a doctor’s office. They also can send a message to 911, if needed.

Caregivers or physicians outside the home also can have access to the health data, allowing for continuous monitoring.

“With the growing special-needs and aging baby-boomer population, paired with a deficit in caregivers, there is an increasing need for personalized care,” Jumadinova says. “I have always had an interest in developing life-enhancing technologies, so that’s where this idea originated.”

The system requires the user to wear a small sensor which monitors the person’s vital signs, the researchers explain.

“My job for this project was to make sure the information coming from the sensor was transmitting to a database, which then analyzes the different health conditions,” says Boanos, who is double majoring in neuroscience and computer science. “If anything changes rapidly, the robot can sense the change and create an event, like calling emergency personnel. The GPS device can even give the person’s location, so it can send an ambulance if needed.”

According to Jumadinova, the sensor can communicate wirelessly with the robot, which looks similar to a Roomba vacuum cleaner with a laptop on it. The base, called a Turtlebot, is on wheels so it can move at different speeds.

Also part of the unit are Kinect sensors, which are the same sensors used in the Xbox gaming system. These sensors allow the computer to “see” a picture of a human. They also allow the robot to detect the distance between itself and an object in front of it.

“The laptop is basically the brain of the robot, and the Kinect sensors are the camera,” says Zheng, who is majoring in computer science with minors in math and economics. “My job for this project was writing algorithms to establish a connection between the laptop and the robot. ”

The third student, Camara, worked behind-the-scenes this summer to develop what’s called a “text mining system.”

“The robots can collect data and analyze it to find long-term trends. The data is then saved in a database, which then can be processed by the text mining system,” Jumadinova says. “The idea is that the robots may not see long-term trends, but the text mining system can go through the long-term data and find any alarming trends, and then notify the robots, if needed, by sending them a message.”

Allegheny currently has four robots. When turning them on, Jumadinova says they must first travel around the room to create a map of it. The robot then will use the map to find the person it is tracking.

“Only one robot will follow a person at a time. And if it needs to charge itself, it can go directly to its docking station and call another robot to its location,” she says.

“The greatest challenge has been getting the robots to talk to each other,” Zheng adds. “But they now can communicate and tell each other to ‘come here’ if needed.”

The opportunity to do this type of hands-on research as an undergraduate is surprising, says Boanos. “We are given a lot of independence. But if you have any questions, the professors are always there. It’s really amazing to be working on something like this as a junior. I can see the power of computer science through this project.”

Applying what he has learned in his computer classes to this research is enjoyable. “To me, computer science is about using all the information you learn in class in a really creative way. And learning how to program gives you the ability to use your imagination to create whatever you want; you can create amazing things. I think this project is impressive – it will really affect people’s lives,” says Boanos.

What’s next for the project? Jumadinova says they will continue testing and refining the system. But she doesn’t plan to stop there.

“In addition to monitoring a person, we hope that our team of robots will be able to provide motivation for cognitive and physical exercises to the user by considering the history of the user’s daily tasks and coaching the person to fulfill appropriate tasks, such as taking medicine, exercising or being socially active,” she says. “I also hope to meet with those in the medical community to get a better understanding of various health conditions so we can tailor the robots to those conditions.

“So far, I haven’t seen any other systems out there using data from wearable sensors with robots in this continuous way,” she adds. “It’s exciting.”