Humanizing speech in deaf-blind community

Sophomore Noopur Ranganathan's work empowers those with visual and auditory impairments to take communication into their own hands.

Voice recognition seems to be an ever-growing part of daily life, as more and more households rely on software like Siri or Alexa to check the weather, turn on the stereo, or look up a recipe for dinner. But what if this software could take on an even greater role? Noopur Ranganathan, a sophomore in biology, is harnessing the power of these digital assistants to improve the lives of deaf-blind individuals halfway around the world. Taking her work from Boston to Chennai, India, Ranganathan's work empowers those with visual and auditory impairments to take communication into their own hands.

Ranganathan first started working with the visually impaired community as a high school student. "I was involved in a research project that focused on macular degeneration," she says. "As a part of that research project, I had to go to a visual eye care center and speak with the patients. Ever since then, I've been involved with the visually impaired population," Ranganathan explains.

An active member of her speech and debate club, Ranganathan was also interested in how we communicate, how we speak to each other, and how we can do so more effectively. "One of my best friends had a speech impairment in high school, and I always noticed the difficulties that she went through," Ranganathan says of her fascination with speech. "And her dad said something that always stuck with me, that in the real world it doesn't so much matter what you say but how you say it."

When she came to MIT, she was looking for her passions for communication and accessibility to intersect, which led her to connect with the local Perkins School for the Blind and, later, the National Institute for Empowerment of Persons with Multiple Disabilities (NIEPMD), an Indian government agency providing services to persons with multiple disabilities. "In the fall semester of my freshman year, I contacted NIEPMD and just asked them if it would be OK if I visited the institute over IAP [Independent Activities Period], just to see how the institute runs and what services they provide," Ranganathan says. "At the time, I didn't know what my project would be. I was very open-minded about that."

And so, when she first traveled to Chennai in early 2018, she did so without a personal agenda. It was a trip, first and foremost, to build the context of this community in order to truly understand their needs. Traveling alone, without any external impetus, the experience was a leap of faith of sorts that resulted in the life-changing project Ranganathan would implement almost exactly one year later. "The institute [as suggested in its name] focused on people with multiple disabilities. So, there were some people with speech impairments as well as visual impairments, others with cognitive impairments," Ranganathan explains of NIEPMD's work and environment. What truly changed her experience, and ultimately propelled her project moving forward, was a meeting she had with a NIEPMD patient named Miranda, a man who was both deaf and blind.

"My interaction with him was very different from anything I had ever experienced," Ranganathan explains. "Usually, when communicating with someone, we always take sound into factor. But since he was both deaf and blind, I wasn't able to speak to him, nor write something on a piece of paper to him." To communicate, Miranda had a refreshable braille unit that was linked to a simple word document. All you had to do was type out a word or a sentence into the document, and the braille unit would translate what had been written into braille. While simple enough, and effective in exchanging messages, Ranganathan felt uneasy about the whole process.

"There was no sound, no eye contact," she says of their first meeting. "It seemed very … not human. It seemed totally robotic, and so that's when I thought about apps like Siri or Alexa. We can speak into a device and these apps can pick up on our voices and translate them into text." This is where Ranganathan's IAP 2019 fellowship was truly born; a lightning bolt had struck and she got the idea that she could somehow make communicating with Miranda and his peers, a task that at first glance seemed impossible, seamless. She could make communicating with the deaf-blind community a human act, instead of a robotic one.

"With this project in mind, I applied to the PKG (Priscilla King Gray) Fellowship through ESG [Experimental Study Group]. I got the letter of approval from the community partner [NIEPMD], and over sophomore IAP I was finally able to implement this project," Ranganathan says, barely able to contain her excitement. The project consisted of implementing a Siri-like software to the patient's refreshable braille unit, such that speech could be translated into text, and then the text could be translated into braille. Ranganathan envisioned this seamless chain reaction: speech to text, text to braille, which could then be read and understood by the patient - in this case, Miranda. There would be no need for a middleman to type out sentences, to transcribe speech, to record notes from Miranda's lectures (he is currently working towards his medical doctor degree in special education). With Ranganathan's software, anyone could speak to him and, with the aid of his refreshable braille unit, he could understand them. The results were outstanding.

"I interviewed his wife and she just burst into tears," Ranganathan explains. "Before this project, he was very dependent on her for writing emails, and doing his work, and … everything. And she has her own job as well, so she felt like she had a lot on her shoulders. But with this software, he is now empowered to be independent. He can craft his own emails and see Word documents and search the internet and check his own mail." And so as this project's coming to fruition has made a notable impact on Ranganathan's life - the satisfaction and excitement of seeing years of dedication and hard work come to life in real, tangible ways - it has made an even larger impact on the community she has worked so hard to support.

As this project continues to impress, not just within NIEPMD, but also within the Perkins Global Network, earning accolades from organizations in Germany and Japan among others, Ranganathan looks forward to implementing this software on a much larger scale and with even more benefits. The next steps are making the software more adaptable to translation into native languages, as well as adapting the program to be compatible with portable devices like cell phones. While Miranda is currently the only patient with access to this program, Ranganathan hopes to implement this software for others as well. "The first time I spoke with this person, it was shocking and surprising," she says. "I was not able to speak with him. I had never come across someone like that before. It was something that was really saddening at first, but now … he's empowered now." Ranganathan hopes to empower many more, not only by continuing to work with NIEPMD, but also in her more long-term pre-med track towards ocular medicine.