A high-tech vest built by a University of Toronto Mississauga researcher will help guide Karolina Pakėnaitė as she attempts to become the first deafblind person to scale Mount Everest.
Pakėnaitė plans to wear a signal-sending vest co-created by Leon Lu, an assistant professor at U of T Mississauga's Institute of Communication, Culture, Information and Technology.
The Haptic Harness will allow guides to communicate with Pakėnaitė during the climb via vibrations sent by remote controls.
"I felt that this was just something meaningful to do," says Lu, who works at the intersection of accessibility and human-computer interaction. "It wasn't about what we might get from it, but more how to make her expedition possible and supported in some way."
Lu has studied the use of haptics - or vibrations, similar to those from a phone or video game controller - for several years.
He previously looked at how the technology can be used to help blind and low-vision musicians communicate with their instructors during performances and practices.
"Imagine a conductor using their hands or their faces to instruct the musician, or even a teacher communicating to the student ... If you're blind, you miss this information," says Lu, who developed wearable haptic devices that send vibration signals between musicians and teachers in real time to help them communicate.
After meeting Pakėnaitė at an accessibility conference a few years ago, Lu realized that some of the challenges she would face on Everest could be helped by haptics.
"It was actually very similar to what we were addressing (with musicians). It was just in a different context," Lu says.
Pakėnaitė, who lives in the United Kingdom, has Usher syndrome - she was born deaf and her sight began deteriorating at 19. Her Everest climb is currently planned for spring 2027 and she has been preparing by training, running marathons and scaling other peaks.
While she has some usable vision, her deafness means she won't be able to hear her guides on her Everest climb or read their lips while ascending to the summit since their faces will be covered by masks. Her limited vision also poses major difficulties for navigating the challenging terrain.
Lu says the Haptic Harness should be able to help guides better support Pakėnaitė on her historic climb.
The vest has four motors - one on each shoulder and two above the waist that can indicate signals to move left, right, up and down. The duration and pattern of the vibrations can indicate more complex messages, which are determined by the guides, via a custom remote, and Pakėnaitė, who will wear the vest.
Lu has been working with Germany's HapticLabs, which designs haptics, and Rodolfo Cossovich, an assistant professor from NYU Shanghai, who has a background in electrical engineering and accessibility.
For the harness fabric, Lu partnered with Hayley Stolee-Smith, a Hamilton, Ont.-based fashion designer.
"We have folks who are all in different places, but we've all come together to make this happen," Lu says.
The team worked on the device for the past year and a half, with Lu describing the development as "an ongoing process of design and iteration."
That included taking into account issues surrounding battery life and how to design assistive technology for extreme conditions - a challenge that saw them lean heavily on the experience of Pakėnaitė's team.
"We asked them if we should be concerned about moisture getting into the circuits. They said 'No,' because there isn't any moisture up in the mountains," Lu says. "We would never have known that."
While Lu has been focused on creating a device specifically for Pakėnaitė to help her achieve her goal, he says it could potentially have broader use.
"We haven't given much thought in terms of how this might get applied in other places, but it could definitely help in any scenario where voice or visual communication is not [possible] - either because of the context or the ability of the person," he says, adding that his accessibility work stems from his desire to make a positive impact on people's lives.
"Karolina is breaking barriers of what society might think people with disabilities can or cannot do. It also means that someone else can look at this and say, 'This person can do this - why can't I?'"
