Awesome

🇺🇦 Speech Recognition & Synthesis for Ukrainian

Overview

This repository collects links to models, datasets, and tools for Ukrainian Speech-to-Text and Text-to-Speech projects.

Community

• Discord: https://discord.gg/yVAjkBgmt4
• Speech Recognition: https://t.me/speech_recognition_uk
• Speech Synthesis: https://t.me/speech_synthesis_uk

🎤 Speech-to-Text

📦 Implementations

• 600M params: https://huggingface.co/Yehor/w2v-bert-2.0-uk-v2 (demo: https://huggingface.co/spaces/Yehor/w2v-bert-2.0-uk-v2-demo)

• 1B params (with language model based on small portion of data): https://huggingface.co/Yehor/wav2vec2-xls-r-1b-uk-with-lm
• 1B params (with language model based on News texts): https://huggingface.co/Yehor/wav2vec2-xls-r-1b-uk-with-news-lm
• 1B params (with binary language model based on News texts): https://huggingface.co/Yehor/wav2vec2-xls-r-1b-uk-with-binary-news-lm
• 1B params (with language model: OSCAR): https://huggingface.co/arampacha/wav2vec2-xls-r-1b-uk
• 1B params (with language model: OSCAR): https://huggingface.co/arampacha/wav2vec2-xls-r-1b-uk-cv
• 300M params (with language model based on small portion of data): https://huggingface.co/Yehor/wav2vec2-xls-r-300m-uk-with-lm
• 300M params (but without language model): https://huggingface.co/robinhad/wav2vec2-xls-r-300m-uk
• 300M params (with language model based on small portion of data): https://huggingface.co/Yehor/wav2vec2-xls-r-300m-uk-with-small-lm
• 300M params (with language model based on small portion of data) and noised data: https://huggingface.co/Yehor/wav2vec2-xls-r-300m-uk-with-small-lm-noisy
• 300M params (with language model based on News texts): https://huggingface.co/Yehor/wav2vec2-xls-r-300m-uk-with-news-lm
• 300M params (with language model based on Wikipedia texts): https://huggingface.co/Yehor/wav2vec2-xls-r-300m-uk-with-wiki-lm
• 90M params (with language model based on small portion of data): https://huggingface.co/Yehor/wav2vec2-xls-r-base-uk-with-small-lm
• 90M params (with language model based on small portion of data): https://huggingface.co/Yehor/wav2vec2-xls-r-base-uk-with-cv-lm
• ONNX model (1B and 300M models): https://github.com/egorsmkv/ukrainian-onnx-model

You can check demos out here: https://github.com/egorsmkv/wav2vec2-uk-demo

• data2vec-large: https://huggingface.co/robinhad/data2vec-large-uk

• NVIDIA Streaming Citrinet 1024 (uk): https://huggingface.co/nvidia/stt_uk_citrinet_1024_gamma_0_25
• NVIDIA Streaming Citrinet 512 (uk): https://huggingface.co/neongeckocom/stt_uk_citrinet_512_gamma_0_25

• NVIDIA Streaming ContextNet 512 (uk): https://huggingface.co/theodotus/stt_uk_contextnet_512

  </p>

• FastConformer Hybrid Transducer-CTC Large P&C: https://huggingface.co/theodotus/stt_ua_fastconformer_hybrid_large_pc
  • Demo: https://huggingface.co/spaces/theodotus/asr-uk-punctuation-capitalization
  </p>

• Squeezeformer-CTC ML: https://huggingface.co/theodotus/stt_uk_squeezeformer_ctc_ml

  • Demo 1: https://huggingface.co/spaces/theodotus/streaming-asr-uk
  • Demo 2: https://huggingface.co/spaces/theodotus/buffered-asr-uk

• Squeezeformer-CTC SM: https://huggingface.co/theodotus/stt_uk_squeezeformer_ctc_sm

• Squeezeformer-CTC XS: https://huggingface.co/theodotus/stt_uk_squeezeformer_ctc_xs

  </p>

• https://huggingface.co/taras-sereda/uk-pods-conformer

  </p>

• VOSK v3 nano (with dynamic graph): https://drive.google.com/file/d/1Pwlxmtz7SPPm1DThBPM3u66nH6-Dsb1n/view?usp=sharing (73 mb)
• VOSK v3 small (with dynamic graph): https://drive.google.com/file/d/1Zkambkw2hfpLbMmpq2AR04-I7nhyjqtd/view?usp=sharing (133 mb)
• VOSK v3 (with dynamic graph): https://drive.google.com/file/d/12AdVn-EWFwEJXLzNvM0OB-utSNf7nJ4Q/view?usp=sharing (345 mb)
• VOSK v3: https://drive.google.com/file/d/17umTgQuvvWyUiCJXET1OZ3kWNfywPjW2/view?usp=sharing (343 mb)
• VOSK v2: https://drive.google.com/file/d/1MdlN3JWUe8bpCR9A0irEr-Icc1WiPgZs/view?usp=sharing (339 mb, demo code: https://github.com/egorsmkv/vosk-ukrainian-demo)
• VOSK v1: https://drive.google.com/file/d/1nzpXRd4Gtdi0YVxCFYzqtKKtw_tPZQfK/view?usp=sharing (87 mb, an old model with less trained data)

Note: VOSK models are licensed under Apache License 2.0.

• DeepSpeech using transfer learning from English model: https://github.com/robinhad/voice-recognition-ua
  • v0.5: https://github.com/robinhad/voice-recognition-ua/releases/tag/v0.5 (1230+ hours)
  • v0.4: https://github.com/robinhad/voice-recognition-ua/releases/tag/v0.4 (1230 hours)
  • v0.3: https://github.com/robinhad/voice-recognition-ua/releases/tag/v0.3 (751 hours)

• m-ctc-t-large: https://huggingface.co/speechbrain/m-ctc-t-large

• official whisper: https://github.com/openai/whisper
• whisper (small, fine-tuned for Ukrainian): https://github.com/egorsmkv/whisper-ukrainian
• whisper (large, fine-tuned for Ukrainian): https://huggingface.co/arampacha/whisper-large-uk-2
• https://huggingface.co/mitchelldehaven/whisper-medium-uk
• https://huggingface.co/mitchelldehaven/whisper-large-v2-uk

• Flashlight Conformer: https://github.com/egorsmkv/flashlight-ukrainian

📊 Benchmarks

This benchmark uses Common Voice 10 test split.

wav2vec2-bert

wav2vec2

Citrinet

lm-4gram-500k is used as the LM

ContextNet

FastConformer P&C

This model supports text punctuation and capitalization

Squeezeformer

lm-4gram-500k is used as the LM

Flashlight

lm-4gram-500k is used as the LM

data2vec

VOSK

m-ctc-t

whisper

Fine-tuned version for Ukrainian:

If you want to fine-tune a Whisper model on own data, then use this repository: https://github.com/egorsmkv/whisper-ukrainian

DeepSpeech

📖 Development

• How to train own model using Kaldi (in Russian): https://github.com/egorsmkv/speech-recognition-uk/blob/master/vosk-model-creation/INSTRUCTION.md
• How to train a KenLM model based on Ukrainian Wikipedia data: https://github.com/egorsmkv/ukwiki-kenlm
• Export a traced JIT version of wav2vec2 models: https://github.com/egorsmkv/wav2vec2-jit

📚 Datasets

Compiled dataset from different open sources + Companies + Community = 188.31GB / ~1200 hours 💪

• Storage Share powered by Nextcloud: https://nx16725.your-storageshare.de/s/cAbcBeXtdz7znDN (use Wget to download, downloading in a browser has speed limitations)
• Torrent file: https://academictorrents.com/details/fcf8bb60c59e9eb583df003d54ed61776650beb8 (188.31 GB)

Voice of America (398 hours)

• Storage Share powered by Nextcloud: https://nx16725.your-storageshare.de/s/f4NYHXdEw2ykZKa

FLEURS

• Ukrainian subset: https://huggingface.co/datasets/google/fleurs/viewer/uk_ua/train

YODAS2

• Ukrainian subsets:
  • https://huggingface.co/datasets/espnet/yodas2/tree/main/data/uk000
  • https://huggingface.co/datasets/espnet/yodas2/tree/main/data/uk100

Companies

• Mozilla Common Voice has the Ukrainian dataset: https://commonvoice.mozilla.org/uk/datasets
• M-AILABS Ukrainian Corpus Ukrainian: http://www.caito.de/data/Training/stt_tts/uk_UK.tgz
• Espreso TV subset: https://blog.gdeltproject.org/visual-explorer-quick-workflow-for-downloading-belarusian-russian-ukrainian-transcripts-translations/

Ukrainian podcasts

• https://huggingface.co/datasets/taras-sereda/uk-pods

Cleaned Common Voice 10 (test set)

• Repository: https://github.com/egorsmkv/cv10-uk-testset-clean

Noised Common Voice 10

• Transcriptions: https://www.dropbox.com/s/ohj3y2cq8f4207a/transcriptions.zip?dl=0
• Audio files: https://www.dropbox.com/s/v8crgclt9opbrv1/data.zip?dl=0

Community

• VoxForge Repository: http://www.repository.voxforge1.org/downloads/uk/Trunk/

Other

• ASR Corpus created using a Telegram bot for Ukrainian: https://github.com/egorsmkv/asr-tg-bot-corpus
• Speech Dataset with Ukrainian: https://www.caito.de/2019/01/the-m-ailabs-speech-dataset/

⭐ Related works

Language models

• Ukrainian LMs: https://huggingface.co/Yehor/kenlm-ukrainian

Inverse Text Normalization:

• WFST for Ukrainian Inverse Text Normalization: https://github.com/lociko/ukraine_itn_wfst

Text Enhancement

• Punctuation and capitalization model: https://huggingface.co/dchaplinsky/punctuation_uk_bert (demo: https://huggingface.co/spaces/Yehor/punctuation-uk)

📢 Text-to-Speech

Test sentence with stresses:

К+ам'ян+ець-Под+ільський - м+істо в Хмельн+ицькій +області Укра+їни, ц+ентр Кам'ян+ець-Под+ільської міськ+ої об'+єднаної територі+альної гром+ади +і Кам'ян+ець-Под+ільського рай+ону.

Without stresses:

Кам'янець-Подільський - місто в Хмельницькій області України, центр Кам'янець-Подільської міської об'єднаної територіальної громади і Кам'янець-Подільського району.

📦 Implementations

• P-Flow TTS

https://github.com/egorsmkv/speech-recognition-uk/assets/7875085/18cfc074-f8a1-4842-90b6-9503d0bb7250

• RAD-TTS, the voice "Lada"
• RAD-TTS with three voices, voices of Lada, Tetiana, and Mykyta

https://user-images.githubusercontent.com/7875085/206881140-bf8c09e7-5553-43d9-8807-065c36b2904b.mp4

• v1.0.0 using M-AILABS dataset: https://github.com/robinhad/ukrainian-tts/releases/tag/v1.0.0 (200,000 steps)

• v2.0.0 using Mykyta/Olena dataset: https://github.com/robinhad/ukrainian-tts/releases/tag/v2.0.0 (140,000 steps)

https://user-images.githubusercontent.com/5759207/167480982-275d8ca0-571f-4d21-b8d7-3776b3091956.mp4

• Coqui TTS model implemented in the Neon Coqui TTS Python Plugin. An interactive demo is available on huggingface. This model and others can be downloaded from huggingface and more information can be found at neon.ai

https://user-images.githubusercontent.com/96498856/170762023-d4b3f6d7-d756-4cb7-89de-dc50e9049b96.mp4

• NVIDIA FastPitch: https://huggingface.co/theodotus/tts_uk_fastpitch

• Balacoon TTS, voices of Lada, Tetiana and Mykyta. Blog post on model release.

https://github.com/clementruhm/speech-recognition-uk/assets/87281103/a13493ce-a5e5-4880-8b72-42b02feeee50

📚 Datasets

• Open Text-to-Speech voices for 🇺🇦 Ukrainian: https://huggingface.co/datasets/Yehor/opentts-uk
  • Voice "LADA", female
  • Voice "TETIANA", female
  • Voice "KATERYNA", female
  • Voice "MYKYTA", male
  • Voice "OLEKSA", male

⭐ Related works

Accentors

• https://github.com/NeonBohdan/ukrainian-accentor-transformer
• https://github.com/lang-uk/ukrainian-word-stress
• https://github.com/egorsmkv/ukrainian-accentor

Misc

• Tool to make high quality text to speech (TTS) corpus from audio + text books: https://github.com/patriotyk/narizaka
• A model to do Text Normalization: https://huggingface.co/skypro1111/mbart-large-50-verbalization