Affiliation: Google AI
Role: AI Research Scientist/Tech Lead
https://soundsofindia.withgoogle.com/
Sounds of India is a unique and fun interactive musical experience launching for India’s 74th Independence Day, inspired by Indian tradition and powered by machine learning. When users throughout India (and around the world) sing the Indian National Anthem into the microphone of their mobile devices, machine learning models transform their voices into a range of classical Indian musical instruments live in the browser.
We partnered with Prasat Bharati (India’s NPR) to launch this experience and scale it to millions of individuals across India. We also worked with masters of three classical Indian instruments: the Bansuri, Shennai and Sarangi to develop generative machine learning models for these instruments.
HOW DOES IT WORK?
What makes this experience unique is the use of TFX or TensorFlow Extended to build, monitor and deploy the 3 musical instrument models. We used tensorflow.js to execute inference completely on-device (no servers), meaning we could scale this to millions of singers without adding any additional GPU compute. Unlike the traditional DDSP models based on GRU’s employed in research, we developed a Conv-based architecture better suited for accelerated computations using WebGL and reduce memory footprint by over 20x to make this experience smooth even on the mid to low-tier devices.
PRESS COVERAGE
https://blog.tensorflow.org/2020/08/creating-sounds-of-india-with-tensorflow.html
https://blog.google/around-the-globe/google-asia/the-sound-of-indias-ai-potential/
https://blog.google/intl/en-in/company-news/technology/calling-everyone-across-country-to/
https://tech.hindustantimes.com/tech/news/google-to-mark-india-s-independence-day-with-the-voices-of-its-people-and-ai-71597053806951.html
https://bangaloremirror.indiatimes.com/others/sci-tech/google-introduces-ai-powered-sounds-of-india-app/amp_articleshow/77634485.cmshttps://www.ciotechoutlook.com/news/google-introduces-aipowered-sounds-of-india-app-nid-5742-cid-78.html