Home

Awesome

ctcdecode

ctcdecode is an implementation of CTC (Connectionist Temporal Classification) beam search decoding for PyTorch. C++ code borrowed liberally from Paddle Paddles' DeepSpeech. It includes swappable scorer support enabling standard beam search, and KenLM-based decoding. If you are new to the concepts of CTC and Beam Search, please visit the Resources section where we link a few tutorials explaining why they are needed.

Installation

The library is largely self-contained and requires only PyTorch. Building the C++ library requires gcc or clang. KenLM language modeling support is also optionally included, and enabled by default.

The below installation also works for Google Colab.

# get the code
git clone --recursive https://github.com/parlance/ctcdecode.git
cd ctcdecode && pip install .

How to Use

from ctcdecode import CTCBeamDecoder

decoder = CTCBeamDecoder(
    labels,
    model_path=None,
    alpha=0,
    beta=0,
    cutoff_top_n=40,
    cutoff_prob=1.0,
    beam_width=100,
    num_processes=4,
    blank_id=0,
    log_probs_input=False
)
beam_results, beam_scores, timesteps, out_lens = decoder.decode(output)

Inputs to CTCBeamDecoder

Inputs to the decode method

Outputs from the decode method

4 things get returned from decode

  1. beam_results - Shape: BATCHSIZE x N_BEAMS X N_TIMESTEPS A batch containing the series of characters (these are ints, you still need to decode them back to your text) representing results from a given beam search. Note that the beams are almost always shorter than the total number of timesteps, and the additional data is non-sensical, so to see the top beam (as int labels) from the first item in the batch, you need to run beam_results[0][0][:out_len[0][0]].
  2. beam_scores - Shape: BATCHSIZE x N_BEAMS A batch with the approximate CTC score of each beam (look at the code here for more info). If this is true, you can get the model's confidence that the beam is correct with p=1/np.exp(beam_score).
  3. timesteps - Shape: BATCHSIZE x N_BEAMS The timestep at which the nth output character has peak probability. Can be used as alignment between the audio and the transcript.
  4. out_lens - Shape: BATCHSIZE x N_BEAMS. out_lens[i][j] is the length of the jth beam_result, of item i of your batch.

Online decoding

from ctcdecode import OnlineCTCBeamDecoder

decoder = OnlineCTCBeamDecoder(
    labels,
    model_path=None,
    alpha=0,
    beta=0,
    cutoff_top_n=40,
    cutoff_prob=1.0,
    beam_width=100,
    num_processes=4,
    blank_id=0,
    log_probs_input=False
)

state1 = ctcdecode.DecoderState(decoder)

probs_seq = torch.FloatTensor([probs_seq])
beam_results, beam_scores, timesteps, out_seq_len = decoder.decode(probs_seq[:, :2], [state1], [False])
beam_results, beam_scores, timesteps, out_seq_len = decoder.decode(probs_seq[:, 2:], [state1], [True])

The Online decoder is copying CTCBeamDecoder interface, but it requires states and is_eos_s sequences.

States are used to accumulate sequences of chunks, each corresponding to one data source. Is_eos_s tells the decoder whether the chunks have stopped being pushed to the corresponding state.

More examples

Get the top beam for the first item in your batch beam_results[0][0][:out_len[0][0]]

Get the top 50 beams for the first item in your batch

for i in range(50):
     print(beam_results[0][i][:out_len[0][i]])

Note, these will be a list of ints that need decoding. You likely already have a function to decode from int to text, but if not you can do something like. "".join[labels[n] for n in beam_results[0][0][:out_len[0][0]]] using the labels you passed in to CTCBeamDecoder

Resources