Wentworth Students Develop Robotic Communication Tool with Tatum for DeafBlind Community

September 18, 2024

The T1 Fingerspelling Hand

Perhaps it is notthe power of manufactured machines that runs modern society; instead, it is the power of ideas that pushes nations forward. Without this latter power, key infrastructures such as communication, construction, transportation, and computation would have never come to life.

It takes a creative individual with an entrepreneurial mindset to bridge the gap between vision and real-world challenges, bringing impactful solutions to life. Such a professional knows the art of applying scientific theories to real-world scenarios using analytical and critical thinking skills.

One such person is Samantha Johnson, the founder of Tatum Robotics, a resident startup at MassRobotics, who, along with her team, has created the T1 Fingerspelling Hand, an independent robotic communication tool that helps the deafblind community communicate more effectively through tactile language.

Tatum Robotics began after Johnson met a DeafBlind person for the first time. This meeting opened her eyes to the community's reliance on expensive human interpreters for communication. The lack of autonomy and access to interpreting services, particularly worsened during the COVID-19 pandemic, motivated her to find a new solution.

"The COVID-19 pandemic forced social distancing, causing increased isolation and a lack of access to important news updates due to an intensified shortage of crucial interpreting services," said Johnson. That experience motivated her to learn more about living with deafblindness and ultimately sparked the idea for a new kind of communication tool to support this historically underserved population.

Samantha Johnson

Partnering with Wentworth

Engaged as a guest speaker with Accelerate, Wentworth's Innovation and Entrepreneurship Center, Johnson saw an opportunity for partnership and innovation and worked with Accelerate Executive Director Ashley Lucas to bring an empathic idea to life.

Johnson's T1 Fingerspelling Hand was on its way toward completion but was facing a critical challenge related to the control mechanism of the fingers. The fingers were constructed from a single flexible piece of thermoplastic polyurethane (TPU) which made it difficult to precisely determine their position in space. Therefore, Johnson and her team were looking for assistance in creating a solution that would include the following features:

1) Real-time position feedback

2) Sensor integration

3) Small and easy integration

From there, Wentworth professors Afsaneh Ghanavati and Marisha Rawlins pulled together an interdisciplinary team of engineering students, and corresponding advisors, to work directly with Tatum Robotics.

Wentworth provides students the opportunity to work on solving problems with real-world impact, especially during their senior design capstone experience. The partnership formed with Tatum Robotics was no exception.

Five students representing four different majors, Paul Kim (BSME, 24), Adrianna Arona (BBME, 24), Matthew Buzalsky (BSEE, 24), Derek Huang (BSCO, 24), and Anthony Pham (BSEE, 24), quickly got to work on conducting a comprehensive evaluation of three types of sensors-flex sensors, Hall effect sensors, and accelerometers-to assess performance and applicability in various sensing applications.

Methods included a series of controlled experiments designed to measure the sensitivity, accuracy, and response time of each sensor type under different conditions. Flex sensors were evaluated on the resistance at various angular displacements. Hall effect sensors were evaluated based on magnetic field detection capabilities. Accelerometers were assessed for precision in measuring acceleration and tilt.

Standardized testing protocols ensured consistent and reliable data collection. Materials used included a range of mechanical fixtures and electronic interfaces necessary to connect the sensors with data acquisition systems.

Paul Kim, Adrianna Arona, Derek Huang, Matthew Buzalsky, and Anthony Pham

Program Summary

In the fall of 2023, Lucas initiated an opportunity for a senior design student team to collaborate with Johnson to support the development of her tactile fingerspelling robotic hand. Soon a team of faculty and students was created to work on the Tatum Robotics project. The internal members were as follows:

Previous students and current alumni of the interdisciplinary senior design team

• Paul Kim - BSME, Team Lead (Class of 2024)

• Adrianna Arona - BBME (Class of 2024)

• Matthew Buzalsky - BSEE (Class of 2024)

• Derek Huang - BSCO (Class of 2024)

• Anthony Pham - BSEE (Class of 2024)

Primary Advisors

• Afsaneh Ghanavati (PhD, Associate Professor, ECE, SoE)

• Marisha Rawlins (PhD, Associate Professor, ECE, SoE)

 Senior Design Course Advisors

• Douglas Dow (PhD, Professor, ECE, SoE)

• Uri Feldman (PhD, Associate Professor, BMED, SoE)

• Bo Tao (PhD, Associate Professor, ME, SoE)

What was the challenge? Tatum Robotics was facing a critical challenge related to the control mechanism of the fingers. The fingers were constructed from a single flexible piece of thermoplastic polyurethane (TPU) which made it difficult to precisely determine their position in space. Therefore, they needed assistance in creating a solution that would include the following features: 1) Real-Time Position Feedback, 2) Sensor Integration, and 3) Small and easy integration.

The senior design team's approach toward the challenge: The senior design team (named DigitSense) worked on the Tatum Robotics project during the Spring and Summer 2024 semesters. They conducted a comprehensive evaluation of three types of sensors: flex sensors, Hall effect sensors, and accelerometers, to assess performance and applicability in various sensing applications. Methods included a series of controlled experiments designed to measure the sensitivity, accuracy, and response time of each sensor type under different conditions. Flex sensors were evaluated on the resistance at various angular displacements. Hall effect sensors were evaluated based on magnetic field detection capabilities. Accelerometers were assessed for precision in measuring acceleration and tilt. Standardized testing protocols ensured consistent and reliable data collection. Materials used included a range of mechanical fixtures and electronic interfaces necessary to connect the sensors with data acquisition systems.

The next steps for the collaboration include additional validation of the sensor in real-time, using the sensor along with the fingers' traditional use cycles to determine longevity and accuracy. Importantly, the finger will also need to be updated for permanent sensor integration to ensure the sensor can be placed consistently and does not move with the flexion of the finger.

"It has been a profound privilege to work with Ms. Johnson's crew as this partnership has brought an invaluable amount of learning to the Wentworth team," the Wentworth group said in a statement.