July 2019 India Today
Google makes some cool apps. But one of the most useful could be its Live Transcribe app, which will help users with hearing disability communicate better with the world. Hearing is one of the key senses that enables us to connect with the world around us. It not only helps in understanding the message that is being conveyed to us but it also empowers us to respond to those messages effectively. But many-a-times, factors such as ageing, infectious diseases and ear infections, recreational activities or accidents, affect an individual's ability to hear.
The result is that individuals with such impairment are unable to communicate and hence connect with people around them. While there are devices such as cochlear implants and other assistive device that can be used for help such individuals, none of them are easy to use. Google is hoping to change that.
The tech giant earlier this year launched an app called the Live Transcribe app that enables people connect with the world around them by transcribing conversations in real-time. The app, which is available for Android devices, uses a host of tools, both hardware and software, to enable people to follow a communication that they otherwise would have been excluded from. "It brings out the captions in the real world and helps you understand what is being spoken around the world," Google product manager Sagar Savla said during his presentation at the Solve with AI Conference in Tokyo. At the conference, Savla, besides explaining the technology behind the app, also gave a live demo of how the Live Transcibe app.
The app is simple to use. It uses a phone's microphones to listen to the sounds around an individual, analyses the audio and it then transcribes that sound and shows the users the relevant text. In the heart of the app are three complex machine learning (ML) models that analyse the sound for words and context. While the first model - called the Acoustic Model - converts the sounds to Phonemes waveform, the second model - called the Pronunciation model - converts phonemes into words. The third and the final model - also called the Language model - then adds context of the words to make sense of the entire conversation.
"It understands context. So if you are talking about buying a new jersey in New York, it actually understands the capitalisation and makes the jersey smaller so that it knows that you are talking about the shirt and not the place," added Savla.
The interesting thing about this app is that is already available in 150 countries across the world and that it can transcribe up to 70 different languages including Hindi, Portuguese, Spanish and Italian in real time. In addition to that, the Live Transcribe app can also recognise 40 different sound effects like car sirens horns, clapping, music, applause, people shouting at you, sound of a speeding vehicle and a baby crying among other things.
The uses of this app are not limited to people with hearing impairment. The Live Transcribe app can also be used by students and journalists to take notes. Users can also copy and save transcripts of their conversation on their devices for three days