Researchers from Georgia Institute of Technology in Atlanta (GA, USA) have developed speech-to-text software for Google Glass, allowing hard-of hearing users to converse with others. The "Captioning on Glass" (CoG) software, which provides real-time captioning, requires an Android smartphone paired with Glass to function. A deaf or hard-of-hearing person wears Glass while a second person speaks into the smartphone. The speech is converted into text and displayed on the heads-up display instantly.
Using a smartphone with Glass has several benefits as compared to using Glass by itself, said Thad Starner, professor at the School of Interactive Computing, Georgia Tech’s College of Computing, and leader of the Contextual Computing Group working on the project. “Glass has its own microphone, but it’s designed for the wearer,” Prof. Starner noted.
Captioning Errors Corrected by Smartphone Software
With the phone-to-Glass system, the conversation partner can speak clearly using the microphone on the smartphone. “The mobile phone puts a microphone directly next to the speaker’s mouth, reducing background noise and helping to eliminate errors,” Prof. Starner said. This will also avoid the "ums" and "uhs" in the speech. In case captioning errors are sent to Glass, the smartphone software enables the speaker to edit the mistakes, sending the changes to the person wearing the device, the professor explained.
Prof. Starner's group developed the COG software when one of its own said he was having trouble hearing and thought Glass could help him. “This system allows wearers like me to focus on the speaker’s lips and facial gestures,“ said School of Interactive Computing Professor Jim Foley. “If hard-of-hearing people understand the speech, the conversation can continue immediately without waiting for the caption. However, if I miss a word, I can glance at the transcription, get the word or two I need and get back into the conversation.”
The Android transcription API allows the smartphone to convert the audio to text, according to Jay Zuerndorfer, the Georgia Tech Computer Science graduate student who developed the software.
Prof. Foley and the students are collaborating with the Association of Late Deafened Adults in Atlanta to improve the system.
Another Project: Translation on Glass
The same group at Georgia Tech’s College of Computing is also working on a second project called "Translation on Glass". The program uses the same smartphone-to-Glass Bluetooth connection process to capture words spoken into the smartphone, translate them to another language and send them to Glass. The only difference is that the person wearing Glass, after reading the translation, can reply. The response is translated back to the original language on the smartphone, the research team said. Currently, two-way translations are available for English, French, Spanish, Japanese, Korean and Russian.
"For both uses, the person wearing Glass has to hand their smartphone to someone else to begin a conversation,” Prof. Starner said. “It’s not ideal for strangers, but we designed the program to be used among friends, trusted acquaintances or while making purchases.” Starner's group is working to get Translation on Glass ready for release to the market.
Source: Newswise.com
Image Credit: Georgia Institute of Technology
Using a smartphone with Glass has several benefits as compared to using Glass by itself, said Thad Starner, professor at the School of Interactive Computing, Georgia Tech’s College of Computing, and leader of the Contextual Computing Group working on the project. “Glass has its own microphone, but it’s designed for the wearer,” Prof. Starner noted.
Captioning Errors Corrected by Smartphone Software
With the phone-to-Glass system, the conversation partner can speak clearly using the microphone on the smartphone. “The mobile phone puts a microphone directly next to the speaker’s mouth, reducing background noise and helping to eliminate errors,” Prof. Starner said. This will also avoid the "ums" and "uhs" in the speech. In case captioning errors are sent to Glass, the smartphone software enables the speaker to edit the mistakes, sending the changes to the person wearing the device, the professor explained.
Prof. Starner's group developed the COG software when one of its own said he was having trouble hearing and thought Glass could help him. “This system allows wearers like me to focus on the speaker’s lips and facial gestures,“ said School of Interactive Computing Professor Jim Foley. “If hard-of-hearing people understand the speech, the conversation can continue immediately without waiting for the caption. However, if I miss a word, I can glance at the transcription, get the word or two I need and get back into the conversation.”
The Android transcription API allows the smartphone to convert the audio to text, according to Jay Zuerndorfer, the Georgia Tech Computer Science graduate student who developed the software.
Prof. Foley and the students are collaborating with the Association of Late Deafened Adults in Atlanta to improve the system.
Another Project: Translation on Glass
The same group at Georgia Tech’s College of Computing is also working on a second project called "Translation on Glass". The program uses the same smartphone-to-Glass Bluetooth connection process to capture words spoken into the smartphone, translate them to another language and send them to Glass. The only difference is that the person wearing Glass, after reading the translation, can reply. The response is translated back to the original language on the smartphone, the research team said. Currently, two-way translations are available for English, French, Spanish, Japanese, Korean and Russian.
"For both uses, the person wearing Glass has to hand their smartphone to someone else to begin a conversation,” Prof. Starner said. “It’s not ideal for strangers, but we designed the program to be used among friends, trusted acquaintances or while making purchases.” Starner's group is working to get Translation on Glass ready for release to the market.
Source: Newswise.com
Image Credit: Georgia Institute of Technology
Latest Articles
Google Glass, smartphones, captioning, deaf, hard of hearing
Researchers from Georgia Institute of Technology in Atlanta (GA, USA) have developed speech-to-text software for Google Glass, allowing hard-of hearing use...
