Awesome

Supported functions

Supported platforms

Supported programming languages

For Rust support, please see sherpa-rs

It also supports WebAssembly.

Introduction

This repository supports running the following functions locally

• Speech-to-text (i.e., ASR); both streaming and non-streaming are supported
• Text-to-speech (i.e., TTS)
• Speaker diarization
• Speaker identification
• Speaker verification
• Spoken language identification
• Audio tagging
• VAD (e.g., silero-vad)
• Keyword spotting

on the following platforms and operating systems:

• x86, x86_64, 32-bit ARM, 64-bit ARM (arm64, aarch64), RISC-V (riscv64)
• Linux, macOS, Windows, openKylin
• Android, WearOS
• iOS
• HarmonyOS
• NodeJS
• WebAssembly
• NVIDIA Jetson Orin NX (Support running on both CPU and GPU)
• NVIDIA Jetson Nano B01 (Support running on both CPU and GPU)
• Raspberry Pi
• RV1126
• LicheePi4A
• VisionFive 2
• 旭日X3派
• 爱芯派
• etc

with the following APIs

• C++, C, Python, Go, C#
• Java, Kotlin, JavaScript
• Swift, Rust
• Dart, Object Pascal

Links for Huggingface Spaces

We also have spaces built using WebAssembly. They are listed below:

Links for pre-built Android APKs

Links for pre-built Flutter APPs

Real-time speech recognition

Text-to-speech

Note: You need to build from source for iOS.

Links for pre-built Lazarus APPs

Generating subtitles

Links for pre-trained models

Some pre-trained ASR models (Streaming)

Please see

• https://k2-fsa.github.io/sherpa/onnx/pretrained_models/online-transducer/index.html
• https://k2-fsa.github.io/sherpa/onnx/pretrained_models/online-paraformer/index.html
• https://k2-fsa.github.io/sherpa/onnx/pretrained_models/online-ctc/index.html

for more models. The following table lists only SOME of them.

Some pre-trained ASR models (Non-Streaming)

Please see

• https://k2-fsa.github.io/sherpa/onnx/pretrained_models/offline-transducer/index.html
• https://k2-fsa.github.io/sherpa/onnx/pretrained_models/offline-paraformer/index.html
• https://k2-fsa.github.io/sherpa/onnx/pretrained_models/offline-ctc/index.html
• https://k2-fsa.github.io/sherpa/onnx/pretrained_models/telespeech/index.html
• https://k2-fsa.github.io/sherpa/onnx/pretrained_models/whisper/index.html

for more models. The following table lists only SOME of them.

Useful links

• Documentation: https://k2-fsa.github.io/sherpa/onnx/
• Bilibili 演示视频: https://search.bilibili.com/all?keyword=%E6%96%B0%E4%B8%80%E4%BB%A3Kaldi

How to reach us

Please see https://k2-fsa.github.io/sherpa/social-groups.html for 新一代 Kaldi 微信交流群 and QQ 交流群.

Projects using sherpa-onnx

Open-LLM-VTuber

Talk to any LLM with hands-free voice interaction, voice interruption, and Live2D taking face running locally across platforms

See also https://github.com/t41372/Open-LLM-VTuber/pull/50

voiceapi

It shows how to use the ASR and TTS Python APIs with FastAPI.

腾讯会议摸鱼工具 TMSpeech

Uses streaming ASR in C# with graphical user interface.

Video demo in Chinese: 【开源】Windows实时字幕软件（网课/开会必备）

lol互动助手

It uses the JavaScript API of sherpa-onnx along with Electron

Video demo in Chinese: 爆了！炫神教你开打字挂！真正影响胜率的英雄联盟工具！英雄联盟的最后一块拼图！和游戏中的每个人无障碍沟通！