Awesome
DEX-TTS: Diffusion-based EXpressive Text-to-Speech with Style Modeling on Time Variability
This repository is the official implementation of DEX-TTS: Diffusion-based EXpressive Text-to-Speech with Style Modeling on Time Variability.
In this repository, we provide steps for running DEX-TTS and GeDEX-TTS.
🙏 We recommend you visit our demo site. 🙏
DEX-TTS is diffusion-based expressive TTS using reference speech. The overall architecture of DEX-TTS is as below:
<p align="center"> <img src="./images/DEXTTS.png" alt="DEX-TTS" width="70%" height="70%"/> </p>GeDEX-TTS is the general version of DEX-TTS, which does not use reference speech. The overall architecture of GeDEX-TTS is as below:
<p align="center"> <img src="./images/GeDEXTTS.png" alt="GeDEX-TTS" width="60%" height="60%"/> </p>Shortcuts
You can find codes, a demo site, and paper links below.
[👉 Demo] [📄 Paper] [💻 DEX-TTS Code] [💻 GeDEX-TTS Code]
ToDo
- Bigvgan vocoder for multi-speaker TTS
- Multi-gpu training codes
- LibriTTS & Simpe preprocess recipes
- Pre-trained weight for DEX-TTS
- Pre-trained weight for GeDEX-TTS
- Precondition VE & VP
- Evaluation
Citation
@article{park2024dex,
title={DEX-TTS: Diffusion-based EXpressive Text-to-Speech with Style Modeling on Time Variability},
author={Park, Hyun Joon and Kim, Jin Sob and Shin, Wooseok and Han, Sung Won},
journal={arXiv preprint arXiv:2406.19135},
year={2024}
}
License
This repository will be released under the MIT license.
Thanks to the open source codebases such as RetNet, FastSpeech2, Grad-TTS, DiT, MaskDiT, and EDM. This repository is built on them.