HELLO, we are interpretAR
interpretAR uses Microsoft Cognitive Services Speech SDK, Blender Animations, Unity Gaming Engine, OpenCV, and Android Studio to create an application that translates speech into American Sign Language (ASL) through an Augmented Reality (AR) approach.
interpretAR won ‘Most Innovative Design’ at McMaster’s ECE Expo 2019
Background Giving purpose to interpretAR
When an individual suffers from hearing impairment, there exists a physical barrier that inherently hampers the ability to communicate. Conversations become inconvenient – getting the point across is challenging and meaning can be lost in the process. To combat this, sign language was developed as a tool to help hearing impaired individuals communicate; and while this does help bridge the gap when both parties are proficient, it falls short in situations where one party lacks fluency.
Over 5% of the world suffers from hearing loss which totals to over 360 million people. InterpretAR is a mobile application set to improve the quality of life for individuals who rely on sign language as their main medium of communication. Though the application’s usability extends universally, its main purpose is twofold:
1. To solve everyday inconveniences for the hearing impaired
2. To familiarize individuals afflicted with acute hearing loss with sign language
About Harnessing the power of Blender, Unity, and Android
Blender Animations
Unity Gaming Engine
Android Studio
Features Creating a seamless user experience
Face Recognition
Through pairing ASL translations with face recognition, interpretAR takes a mixed reality approach. We make communication for the user accessible through lip reading and/or ASL translations – not limiting our users to just one mode of communication.
Multilanguage
InterpretAR overcomes speech barriers such as speaking style and background noise using the power of Microsoft Cognitive Speech Services. The speech-to-text service is optimal for conversational and dictation scenarios and allows interpretAR to bridge the gap in communication across the globe.
Sign Language
Currently, interpretAR is able to perform a plethora of English word translations. The full vocabulary for ASL translations are listed under the ‘Vocab’ icon in our application main page.
Current Vocabulary interpretAR Version 1.0
Try interpretAR on your Android Device Click the GitHub link below to download interpretAR today!
Team Meltdown Founders of interpretAR
Brandon Rufino
I am an electrical and biomedical engineering student who is passionate about advancing the health care field through applying engineering principles. I strive under pressure (how else are diamond made?) and am always looking for the next opportunity to develop my skills with goals to one day pursue an academic, R&D, or entrepreneurial career!
Milos Bijelic
I am looking for opportunities in software engineering and data analytics roles. I am proficient in Python, Ruby, Java and have exposure to many tools and libraries like jupyter notebooks, numpy, pandas, matplotlib etc. In my free time I like to play soccer, ski, scuba dive and anything where I’m active, solving puzzles or making things
Justin Chau
I am a 5th year Electrical and Biomedical Engineering who is always striving to strengthen my skills and learn new things. When I’m not studying, I enjoy being active playing basketball and tennis, and occasionally solve crosswords to challenge my vocabulary. In the near future, I aim to make an impact in the workplace and make a difference in people’s lives.
Arvin Angue
I am an engineering student planning on pursuing a law degree after graduation. I intend to apply the technical knowledge I have gained in my undergrad to become a unique candidate in the practice. While I’m not developing interpretAR, I can be found at the gym, listening to K-pop, reading books, or watching movies.
Acknowledgements: Electrical and Computer Engineering Department at McMaster University as well as Professor Debruin and Professor Jeremic for their continued support
Contact Us See a problem? Let us know!
Hey there – thanks for visiting! We are excited to hear that you would like to get in touch with us.
We created this application with hopes to bridge the gap in communication and make this world a better place. That said, we would love to answer any questions, comments, or concerns you may have.
Here are a couple of other options to contact us:
- Got a question? Ask it via email: brandonrufino at hotmail dot ca
- Contact us on Facebook
We hope you enjoyed interpretAR as much as we enjoyed creating it!