GERM&N

Introduction to GERM&N Engineering

An Introduction to Our Project: Simon Says “Exercise with Me!”, a Low-Cost, Open-Source, Socially Assistive Robot

About Our Project

The current cost of physical therapy is exorbitant for simple, exercise-based treatments. Meanwhile, the existing, therapy-based socially assistive robots (SARs) are either unavailable to the general public or costly with limited options for customization.

We will bring the expensive, research-focused world of socially assistive robot therapy from the lab to the home through a cost-effective, open-source alternative — Simon.

Simon Says "Exercise with Me!"

Simon, assembled from the open-source, 3D-printed InMoov, will guide patients through simple physical therapy exercises, complete with intuitive gestures, facial tracking, and natural language responses.

Simon’s estimated price point, including the costs of base InMoov, 3D printing, and materials will be $1,000. The software suite for Simon’s functionality will be completely free of charge and open-source, bringing the focus to consumers.

Competitors in the Current SARS Market

The current landscape is limited due to a large focus on research, with a multitude of applications to social therapy.

Simon does not have direct competitors in the socially assistive robot industry. The prototype robots, like USC’s “Bandit,” are research-focused and unavailable to the general public. The prebuilt robots, like SoftBank’s “NAO,” cost several thousands of dollars while still lacking features included in Simon.

Facial Tracking

Socially Assistive Robots double as friendly companions. We wanted to implement humanlike features, such as facial tracking, to allow easier interaction with end users. Additionally, the team wanted Simon to be capable of adapting to individual users with facial recognition through the robot’s neck and head movement.

We developed facial recognition software to identify a user's face. Next, to maintain continuous eye contact, we created an algorithm tracking user movement by calculating where Simon should relocate to in real time.

We tested our robot’s facial tracking functionality with a member standing within view of the mounted camera, facing the robot directly. Our test subject moved randomly to various spots within the camera's range of vision. We were able to complete accuracy, resolution, and repeatability testing to create a more friendly, humanlike robot.

Facial Recognition Open Source Code with OpenCV and Raspberry Pi 4.

Voice Recognition

Simon knows a large library of words borrowed and learned from Google's Voice API. Simon is able to listen and even converse with users, who can ask Simon to assist them with their daily dose of exercise! This library is constantly evolving!
Check out Simon's voice recognition software here.

Joint Motor Control

We have implemented fine joint motor control in the shoulder, elbow, wrist, and finger joints via Arduino. We are also able to individually control the motors and debug existing mechanical issues. These arm movements can be accurately and fluidly repeated upon commands to mimic five distinct arm exercises — arm lifts, bicep curls, fist clenches, hand flips, and reverse raises. Check out our software here!

Remote Robotic Control

We had to pivot our project in response to the challenges of COVID-19. With GERM&N Engineering separated across hundreds of miles, each subteam (Face Team and Arm Team) took Simon’s respective components to work remotely. Under the mentorship of our instructors, we are transitioning to Internet of Things (IoT) and remotely sending commands from the head to the arm through an online database. This is achieved with Google Drive API, as it is readily available and easy to use! Here is our open source code if you would like to try it out at home too!

Market Strategy

FDA

Simon will not be going through the FDA. GERM&N Engineering’s overall design lies within Simon’s software component, which does not fall under the FDA’s regulation of medical devices.

Target Market

Simon’s target market is the online socially assistive robot community. These may be individuals who have loved ones living alone. SARs assist users through social, non-contact interactions.

Open-Source Code

Our software is hosted as an open-source repository on GitHub, widening the field and reaching those without academic or financial resources. This will also shift SARs from the lab to the home or clinic.

Projected Impact

There is currently no dominant company holding market share for socially assistive robot therapy, allowing Simon to affect significant change.

Upcoming Features

We are currently working on redesigning Simon’s head and 3D-printed parts to be friendlier via LED panels to depict real-time facial reactions. We are also refining Simon’s speech response. Check in with us in the future for updates!