HierSpeech: Bridging the Gap between Text and Speech by Hierarchical Variational Inference using Self-supervised Representations for Speech Synthesis
28. listopadu 2022
Řečníci
Sang-Hoon Lee
Řečník · 0 sledujících
Seung-Bin Kim
Řečník · 0 sledujících
Ji-Hyun Lee
Řečník · 0 sledujících
Eunwoo Song
Řečník · 0 sledujících
Min-Jae Hwang
Řečník · 0 sledujících
Seong-Whan Lee
Řečník · 0 sledujících
O prezentaci
This paper presents HierSpeech, a high-quality end-to-end text-to-speech (TTS) system based on a hierarchical conditional variational autoencoder (VAE) utilizing self-supervised speech representations. Recently, single-stage TTS systems, which directly generate raw speech waveform from text, have been getting interest thanks to their ability in generating high-quality audio within a fully end-to-end training pipeline. However, there is still a room for improvement in the conventional TTS systems. Since it is challenging to infer both the linguistic and acoustic attributes from the text directly, missing the details of attributes, specifically linguistic information, is inevitable, which results in mispronunciation and over-smoothing problem in their synthetic speech. To address the aforementioned problem, we leverage self-supervised speech representations as additional linguistic representations to bridge an information gap between text and speech. Then, the hierarchical conditional VAE is adopted to connect these representations and to learn each attribute hierarchically by improving the linguistic capability in latent representations. Compared with the state-of-the-art TTS system, HierSpeech achieves +0.303 comparative mean opinion score, and reduces the phoneme error rate of synthesized speech from 9.16This paper presents HierSpeech, a high-quality end-to-end text-to-speech (TTS) system based on a hierarchical conditional variational autoencoder (VAE) utilizing self-supervised speech representations. Recently, single-stage TTS systems, which directly generate raw speech waveform from text, have been getting interest thanks to their ability in generating high-quality audio within a fully end-to-end training pipeline. However, there is still a room for improvement in the conventional TTS systems…
Organizátor
NeurIPS 2022
Účet · 962 sledujících
Baví vás formát? Nechte SlidesLive zachytit svou akci!
Profesionální natáčení a streamování po celém světě.
Sdílení
Doporučená videa
Prezentace na podobné téma, kategorii nebo přednášejícího
Efficient Sampling on Riemannian Manifolds via Langevin MCMC
Zhlédnout později
Xiang Cheng, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Understanding the Evolution of Linear Regions in Deep Reinforcement Learning
Zhlédnout později
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Reproducibility in Optimization: Theoretical Framework and Limits
Zhlédnout později
Kwangjun Ahn, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Combinatorial Bandits with Linear Constraints: Beyond Knapsacks and Fairness
Zhlédnout později
Qingsong Liu, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Capturing Failures of Large Language Models via Human Cognitive Biases
Zhlédnout později
Erik Jones, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
User and Technical Perspectives of Controllable Code Generation
Zhlédnout později
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %