Have a friend or family member with speech and hearing impairment? Is communicating with such people your daily requirement or you like the idea of having a seamless communication with the especially abled?

Well, all you need to do is, download a software on your smartphone, tablet, laptop or PCs and talk away in the sense of fulfillment.

This has been made possible by a third-year engineering student at BNMIT who has developed an Artificial Intelligence (AI) powered software application for the welfare of the deaf and mute people. The software, christened DnD Mate, does not only translate sign language into text and speech, but also translates speech into sign language, all in real time and as quick as the person speaks. Currently, there are no applications/software that facilitates a two-way communication channel.

This easy-to-use innovative digital translator works with your device’s in-built cameras, reads hand and facial gestures by the deaf and mute user and translates them into text and speech. That is not all! The software will also translate your voice or text input into sign language.

‘The software is based on a Deep Learning model and can work both offline and online. While in the offline mode, the deaf and mute person can communicate with you on the same device in real time; in the online mode, you can converse sitting in far off places as well, just like you talk to anyone over a video call,’ says Bhargav DV, the third year Electronics and Communication Engineering student at BNMIT.

While Bhargav is currently working on the software, he plans to extend it to mobile phones and tablets as well.

‘Apart from converting it into an application, I would like to make it as affordable as possible. I want to apply my skills as an engineer to improve differently abled communities and also the wider world. I want to design and produce similar product for other categories of people in special needs,’ says the 21-year-old.