I build Generative AI models for music and creative audio applications.
I was a Senior AI Research Scientist at ByteDance/TikTok in the Foundation Model SEED R&D team for Doubao (China’s ChatGPT). I developed music and audio generation features powered by Diffusion/LLM. I was previously an AI Resident at Google Brain/DeepMind in the Magenta team where I specialized in the intersection of machine learning with audio signal processing. I code primarily in Python/C++ and have degrees in Physics and Music.
A selection of my publications and patents in music technology and Generative AI. I specialize in audio signal processing, neural audio synthesis and LLM’s for audio.
SEED-MUSIC is a unified framework leveraging both auto-regressive language modeling and diffusion for controlled music generation and audio editing workflows.
2024
Doubao is an AI Agent akin to China’s ChatGPT. A unique feature is its integrated music generation features.
2024
Haimian is an AI Music generation app. It enables creators to make expressive vocal music and instrumental music in a variety of genres.
2024
Ripple is a mobile app for music-making powered by AI. Hum a tune. Type some lyrics. Upload a voice memo. Ripple will instantly transform it into a song.
2023
Mawf is a DAW plugin for transforming inputs like singing and everyday sounds into musical instruments. It uses industry-first realtime audio ML that runs on the cpu.
2022
Tone Transfer is a website that lets you transform everyday sounds into musical instruments. It is powered by audio ML running directly on the web.
2021
A unique celebration of India’s Independance Day powered by Music ML and culture.
2020
DDSP is an open-source library fusing modern machine learning with classical signal processing.
2020