How to Recognize Russian Speech More Accurately Than Whisper Using GigaAM

Anyone who has tried to force Whisper onto real call center recordings or quick voice messages in Russian knows this pain. The model often hallucinates, drops endings, or starts translating Russian speech into broken English. For a long time it seemed like you just had to live with this, until an open project from the SberDevices team called GigaAM came along.

The salute-developers/GigaAM repository contains a family of acoustic models based on the Conformer architecture. The main focus here is on Russian and CIS languages, although recent releases have added support for over 70 languages. The models are distributed under the permissive MIT license, making them an excellent choice for commercial products without legal risks.

What this model family can do

The project covers several core audio processing tasks. Inside the repository you'll find solutions for different scenarios:

1. Classic ASR with CTC and RNN-T decoders. The third version (v3) models were trained on 700,000 hours of audio. They provide a noticeable quality boost in complex domains: noise, background music, disfluent speech, and tech support call recordings.
2. End-to-end transcription e2e. Versions v3_e2e_ctc and v3_e2e_rnnt automatically add punctuation and perform text normalization, converting numbers and abbreviations into readable form. In blind tests (Side-by-Side with LLM as judge) this combo beats Whisper-large-v3 with a score of 70:30.
3. Emotion recognition. Model GigaAM-Emo is trained to determine the emotional tone of a phrase and outperforms popular baselines on the Macro F1 metric by approximately 15%.
4. Word-level timestamps and long audio. You can get precise timestamps for each word or connect the pyannote segmentation module for processing hour-long recordings.
5. Multilinguality. Branch multilingual offers encoders with 220M and 600M parameters, pretrained on 2 million hours of data, with strong quality on Kazakh, Kyrgyz, and Uzbek languages.

Quick start and working examples

You'll need Python 3.10+ and ffmpeg installed in your system to get started.

Installing the base package:

git clone https://github.com/salute-developers/GigaAM.git
cd GigaAM
pip install -e .[torch]

The basic pipeline is extremely simple. Here's an example where we first transcribe an audio file, get word timestamps, and then check the speaker's emotional tone:

import gigaam

# Скачиваем тестовый файл через встроенную утилиту
audio_path = gigaam.utils.download_short_audio()

# 1. Распознавание речи с пунктуацией
asr_model = gigaam.load_model("v3_e2e_rnnt")
text = asr_model.transcribe(audio_path)
print("Текст:", text)

# 2. Получение таймстемпов для каждого слова
timed_result = asr_model.transcribe(audio_path, word_timestamps=True)
for word in timed_result.words:
    print(f"[{word.start:.2f} - {word.end:.2f}] {word.text}")

# 3. Определение эмоций
emo_model = gigaam.load_model("emo")
emotions = emo_model.get_probs(audio_path)
for emotion, prob in emotions.items():
    print(f"{emotion}: {prob:.3f}")

There's an important detail explicitly stated in the documentation: the base .transcribe method is designed for segments up to 25 seconds long. If you need to feed the model a long meeting recording or podcast, you need to install additional dependencies pip install -e ".[longform]" and configure a Hugging Face token to download the segmenter pyannote/segmentation-3.0:

import os
import gigaam

os.environ["HF_TOKEN"] = "ваш_токен_huggingface"
long_audio = gigaam.utils.download_long_audio()

model = gigaam.load_model("v3_e2e_rnnt")
segments = model.transcribe_longform(long_audio)

for seg in segments:
    print(f"[{gigaam.format_time(seg.start)} - {gigaam.format_time(seg.end)}]: {seg.text}")

How to deploy to production

The developers took care of production deployment. The models aren't tightly coupled to pure PyTorch:

• ONNX export. Any model can be converted in a couple of lines via the model.to_onnx() method. FP16 is supported for GPU inference via onnxruntime-gpu.
• Integration with Triton Inference Server and TensorRT. The triton_scripts folder contains ready-made recipes for building high-throughput microservices.
• Fine-tuning. The repository has ready-made scripts based on PyTorch Lightning, so if you have your own labeled dataset, CTC and RNN-T adapters can be adapted to your company's specific vocabulary or terminology.

Example of exporting to ONNX:

import gigaam
import torch

model = gigaam.load_model("v3_ctc")
model.to_onnx(dir_path="onnx_models", dtype=torch.float16)

Who will find this project useful right now

If you're building a voice bot, call transcription system, operator speech analytics, or subtitle generator for Russian-language content, there's little practical point in looking at Whisper anymore. GigaAM runs faster, weighs a modest 220–600 million parameters, requires less VRAM, and handles complex Russian phonetics significantly better.

The easiest way to get started is with the interactive Google Colab notebook, a link to which is right in the official repository's header.