Zhaoheng Ni

prof_pic.jpg

I’m a senior research scientist at Meta Reality Labs working on generative models for audio and text. Previously, I was a maintainer of TorchAudio library, the official audio library of PyTorch. Before Meta, I was a PhD student advised by Michael I Mandel and an undergraduate student advised by Yan Xu.

My research interests are single-channel/multi-channel speech enhancement, generative models, and natual language prorcessing.

News

Nov 18, 2024 We are organizing the URGENT 2025 Challenge at Interspeech 2025! Join in the challenge if you are interested in speech enhancement!
Sep 11, 2024 Check out the demo of our MelodyFlow paper, that can do text-guided music editing and generation on 48kHz sample rate music!
Sep 04, 2024 Three papers have been accepted by IEEE SLT 2024!
Jun 20, 2024 We are organizing “Audio Imagination Workshop” at NeurIPS 2024! We cordially invite you to submit your paper or demo through this link!
May 29, 2024 We are organizing the URGENT challenge at NeurIPS 2024 Competition track!

Selected Publications

  1. TorchAudio: Building Blocks for Audio and Speech Processing
    Yao-Yuan Yang ,  Moto Hira ,  Zhaoheng Ni ,  Artyom Astafurov ,  Caroline Chen ,  Christian Puhrsch ,  David Pollack ,  Dmitriy Genzel ,  Donny Greenberg ,  Edward Z Yang ,  and  others
    In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2022
    arXiv HTML Code
  2. WPD++: An Improved Neural Beamformer for Simultaneous Speech Separation and Dereverberation
    Zhaoheng Ni ,  Yong Xu ,  Meng Yu ,  Bo Wu ,  Shixiong Zhang ,  Dong Yu ,  and  Michael I Mandel
    In 2021 IEEE Spoken Language Technology Workshop (SLT) , 2021
    arXiv HTML
  3. TorchAudio-Squim: Reference-less Speech Quality and Intelligibility measures in TorchAudio
    Anurag Kumar ,  Ke Tan ,  Zhaoheng Ni ,  Pranay Manocha ,  Xiaohui Zhang ,  Ethan Henderson ,  and  Buye Xu
    In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2023
    arXiv HTML Code
  4. Scaling Speech Technology to 1,000+ Languages
    Vineel Pratap ,  Andros Tjandra ,  Bowen Shi ,  Paden Tomasello ,  Arun Babu ,  Sayani Kundu ,  Ali Elkahky ,  Zhaoheng Ni ,  Apoorv Vyas ,  Maryam Fazel-Zarandi ,  and  others
    arXiv preprint arXiv:2305.13516, 2023
    arXiv HTML Code