Skip to content

simplify live recommended parameters #137

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
egenthon-cmd wants to merge 3 commits into
base: main
Choose a base branch
from
Open

simplify live recommended parameters #137

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
465c459
simplify live recommended parameters
egenthon-cmd Sep 3, 2026
f299c2d
fix: feedbacks wording and structured
egenthon-cmd Sep 9, 2026
532ad16
fix: reverse changes
egenthon-cmd Sep 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 14 additions & 3 deletions chapters/language/code-switching.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,24 @@
description: "Transcribe conversations that mix multiple languages in real time or pre-recorded"
---

Code switching handles conversations where speakers switch languages mid-utterance or across turns. Gladia can detect and transcribe the active language dynamically, and annotate results with the detected languages code.

Check warning on line 6 in chapters/language/code-switching.mdx

View check run for this annotation

Mintlify / Mintlify Validation (gladia-95) - vale-spellcheck

chapters/language/code-switching.mdx#L6 

Did you really mean 'Gladia'?

<Note>
For best accuracy and latency, provide a small set of expected languages.
Avoid listing dozens of similar languages.
</Note>

## How to use code switching ?
## When to enable it

- Speakers **switch languages** mid-conversation (e.g. bilingual meetings, multilingual customer support).
- You need the detected `language` returned **per utterance**.

## When NOT to enable it

- The audio is in a **single language** — code switching adds unnecessary processing and can introduce misdetections.

Check warning on line 20 in chapters/language/code-switching.mdx

View check run for this annotation

Mintlify / Mintlify Validation (gladia-95) - vale-spellcheck

chapters/language/code-switching.mdx#L20 

Did you really mean 'misdetections'?
- You've set **exactly one language** in `languages` — in that case `code_switching` is ignored anyway.

## How to use code switching

Enable code switching in the session configuration:

Expand All @@ -32,11 +42,12 @@
<Warning>
**Do not enable `code_switching` with an empty `languages` list.** When no
languages are specified, the language detector evaluates every utterance
against 100+ supported languages, leading to frequent misdetections -

Check warning on line 45 in chapters/language/code-switching.mdx

View check run for this annotation

Mintlify / Mintlify Validation (gladia-95) - vale-spellcheck

chapters/language/code-switching.mdx#L45 

Did you really mean 'misdetections'?
especially between similar-sounding languages.
especially between similar-sounding languages. If possible, provide a short list of
languages you **actually expect** in the audio.
</Warning>

## Recommendations

- Limit the number of languages to those you expect.
- Combine with [custom vocabulary](/chapters/pre-recorded-stt/features/custom-vocabulary) per language for domain terms.
- Combine with [custom vocabulary](/chapters/audio-intelligence/custom-vocabulary) per language for domain terms.
44 changes: 37 additions & 7 deletions chapters/language/language-detection.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: Automatic language detection
description: "How to let Gladia detect the spoken language automatically"

Check warning on line 3 in chapters/language/language-detection.mdx

View check run for this annotation

Mintlify / Mintlify Validation (gladia-95) - vale-spellcheck

chapters/language/language-detection.mdx#L3 

Did you really mean 'Gladia'?
---

Gladia can automatically detect the spoken language from audio, either within a candidate set you provide or across all supported languages.

Check warning on line 6 in chapters/language/language-detection.mdx

View check run for this annotation

Mintlify / Mintlify Validation (gladia-95) - vale-spellcheck

chapters/language/language-detection.mdx#L6 

Did you really mean 'Gladia'?

<Note>
Provide a small list of likely languages when possible. It improves accuracy and reduces latency.
Expand All @@ -11,7 +11,27 @@

## How to use language detection ?

You can either omit `language_config.languages` to search across all supported languages, or pass a constrained list:
- You **know** the language of the audio ahead of time.
- The audio is **monolingual** (single language throughout).
- You want the **fastest, most accurate** results.

```json
{
"language_config": {
"languages": ["en"],
"code_switching": false
}
}
```

## When to use auto-detection

- You process audio in **many different languages** and don't know which one beforehand.
- You want Gladia to pick the language automatically.

## How to use language detection

You can either omit `language_config.languages` (or pass an empty list) to search across all supported languages, or pass a constrained list:

<CodeGroup>

Expand Down Expand Up @@ -41,16 +61,26 @@
}
```

</CodeGroup>

The detected `language` is returned on each utterance and can vary over time if you also enable [code switching](/chapters/language/code-switching).

</CodeGroup>
<Warning>
When `code_switching` is `false` and no language is set, the language is
detected on the **first utterance** and reused for the rest of the session or
file. If the beginning of your audio contains silence, music, or a different
language than the main content, this can lead to incorrect detection for the
whole transcription.
</Warning>

The detected `language` is returned on each utterance and can vary over time if you also enable <a href="/chapters/language/code-switching">code switching</a>.
<Tip>
Even when using auto-detection, pass a **small list of likely languages** in
`languages` to constrain the search. This improves both accuracy and
processing time.
</Tip>

## Tips

- Use <a href="/chapters/language/supported-languages">supported languages</a> to pick valid ISO 639-1 codes.
- Use [supported languages](/chapters/language/supported-languages) to pick valid ISO 639-1 codes.
- Combine with `messages_config.receive_partial_transcripts` in real time to get immediate feedback.
- For mixed-language conversations, enable <a href="/chapters/language/code-switching">code switching</a>.


- For mixed-language conversations, enable [code switching](/chapters/language/code-switching).
119 changes: 44 additions & 75 deletions chapters/live-stt/recommended-parameters.mdx
View file
Open in desktop
Comment thread
egenthon-cmd marked this conversation as resolved.
Comment thread
egenthon-cmd marked this conversation as resolved.
Original file line number Diff line number Diff line change
@@ -1,46 +1,12 @@
---
title: Recommended Parameters by Use Case
sidebarTitle: Recommended Parameters
description: Best parameter configurations for realtime transcription depending on your application (Voice Agents, Meeting Recorders, Call Centers, Subtitles).
---

import LanguageDetection from "/snippets/recommended-params/language-detection.mdx";
import CodeSwitching from "/snippets/recommended-params/code-switching.mdx";
import CustomVocabulary from "/snippets/recommended-params/custom-vocabulary.mdx";
import DiarizationVsMultichannel from "/snippets/recommended-params/diarization-vs-multichannel.mdx";

The right parameter configuration can make a significant difference in transcription quality and latency for realtime use cases. This guide covers recommended starting points for common scenarios and highlights pitfalls that frequently trip up new integrations.

<Info>
These recommendations apply to the **[Realtime
API](/chapters/live-stt/quickstart)** and are passed during session
initialization. They are starting points — tune them to match your
specific needs.
</Info>

---

## Language Configuration

<LanguageDetection />

---

## Code Switching

<CodeSwitching />
Comment thread
egenthon-cmd marked this conversation as resolved.

---

## Custom Vocabulary

<CustomVocabulary />

description: Best parameter configurations for realtime transcription depending on your use case.

Check warning on line 4 in chapters/live-stt/recommended-parameters.mdx

View check run for this annotation

Mintlify / Mintlify Validation (gladia-95) - vale-spellcheck

chapters/live-stt/recommended-parameters.mdx#L4 

Did you really mean 'realtime'?
---

## Voice Agents

For callbots, customer-service assistants, or voice-driven chatbots the top priority is **low latency**. The agent must react quickly to user speech, even if sentence boundaries are not perfectly formed.
Optimize for **low latency** and fast turn-taking.

| Parameter | Recommended value | Why |
| --------------------------------------------- | ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
Expand All @@ -50,63 +16,66 @@
| `realtime_processing.custom_vocabulary` | `true` | Add product names and action keywords so the agent can react accurately. |

<Tip>
This setup is optimized for **fast turn-taking**. If utterances get cut off
mid-sentence, raise `endpointing` slightly.
If utterances get cut off mid-sentence, raise `endpointing` slightly.
</Tip>

---

## Meeting Recorders

For apps that record and transcribe meetings in real time — team stand-ups, board sessions, 1-on-1s — the goal is to produce a **structured, speaker-attributed live transcript** that can feed downstream features like summarization or live note-taking.
Optimize for a **structured, speaker-attributed live transcript**.

| Parameter | Recommended value | Why |
| --------------------------------------------- | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `endpointing` | `0.3` - `0.5` | Lets speakers finish their sentences before closing an utterance. See [Endpointing](/chapters/live-stt/features/endpointing). |
| `maximum_duration_without_endpointing` | `15` | Prevents very long utterances in case a speaker doesn't pause. |
| `messages_config.receive_partial_transcripts` | `true` | Feeds live captions to the UI while waiting for final results. See [Partial transcripts](/chapters/live-stt/features/partial-transcripts). |
| `language_config.languages` | Set explicitly | Meeting language is almost always known in advance — setting it avoids detection overhead. |
| `realtime_processing.custom_vocabulary` | `true` | Add company-specific terms, project names, and participant names for better accuracy. |
| Parameter | Recommended value | Why |
| --------------------------------------------- | ----------------- | ------------------------------------------- |
| `endpointing` | `0.3` - `0.5` | Lets speakers finish sentences. |
| `maximum_duration_without_endpointing` | `15` | Caps utterances when a speaker doesn't pause. |
| `messages_config.receive_partial_transcripts` | `true` | Feeds live captions to the UI. |
| `language_config.languages` | Set explicitly | Avoids detection overhead. |
| `realtime_processing.custom_vocabulary` | `true` | Company, project, and participant names. |

<DiarizationVsMultichannel />

---

## Call Centers

For live phone calls the priorities are **speaker identification** and **fast, accurate transcription** despite variable audio quality (telephony codecs, background noise, cross-talk).
Optimize for **speaker identification** and accuracy on telephony audio.

| Parameter | Recommended value | Why |
| --------------------------------------------- | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `endpointing` | `0.2` - `0.4` | Keeps turn-taking responsive without cutting off mid-sentence. See [Endpointing](/chapters/live-stt/features/endpointing). |
| `maximum_duration_without_endpointing` | `15` | Prevents very long utterances in monologue-style segments. |
| `language_config.languages` | Set explicitly (e.g. `["en"]`) | Call center audio typically has a known language. Setting it avoids detection errors on noisy recordings. |
| `realtime_processing.custom_vocabulary` | `true` | Add product names, plan names, and internal terminology. |
| Parameter | Recommended value | Why |
| --------------------------------------- | ------------------------------ | ---------------------------------------- |
| `endpointing` | `0.2` - `0.4` | Responsive without cutting mid-sentence. |
| `maximum_duration_without_endpointing` | `15` | Caps monologue-style segments. |
| `language_config.languages` | Set explicitly (e.g. `["en"]`) | Avoids detection errors on noisy audio. |
| `realtime_processing.custom_vocabulary` | `true` | Product, plan, and internal terms. |

<DiarizationVsMultichannel />

<Tip>
For calls with more than two participants (e.g. conference bridges), use
`diarization_config.min_speakers` / `max_speakers` instead of `number_of_speakers` to give the
model a flexible range.
For noisy telephony audio, set `pre_processing.speech_threshold` to `0.8`.
</Tip>

---

## Subtitles & Captioning

When providing live subtitles, the goal is to **sync text with the speaker in real time**. The right balance between speed and segment quality depends on whether captions are displayed live or post-produced.
Optimize for **syncing text with the speaker**.

| Parameter | Recommended value | Why |
| --------------------------------------------- | -------------------------------------- | ----------------------------------------------------------------------------------------------- |
| `endpointing` | `0.3` (live) / `0.8` (post-production) | Lower values keep captions close to the speaker; higher values produce cleaner subtitle blocks. |
| `maximum_duration_without_endpointing` | `5` | Prevents excessively long subtitle segments that are hard to read on screen. |
| `messages_config.receive_partial_transcripts` | `true` | Shows words as they are spoken, then refines them when the final result arrives. |
| `language_config.languages` | Set explicitly | Avoids detection lag when the broadcast language is known. |
| Parameter | Recommended value | Why |
| --------------------------------------------- | -------------------------------------- | ------------------------------------------------ |
| `endpointing` | `0.3` (live) / `0.8` (post-production) | Lower = closer to speech; higher = cleaner blocks. |
| `maximum_duration_without_endpointing` | `5` | Keeps subtitle segments readable. |
| `messages_config.receive_partial_transcripts` | `true` | Shows words as they are spoken. |
| `language_config.languages` | Set explicitly | Avoids detection lag. |

<Tip>
For post-production subtitles generated from a recording, consider using the
[Pre-recorded API](/chapters/pre-recorded-stt/quickstart) with the dedicated
[subtitles feature](/chapters/audio-intelligence/subtitles) instead — it
produces SRT/VTT files with fine-grained timing controls.
For post-production subtitles from a recording, use the
[Pre-recorded API](/chapters/pre-recorded-stt/quickstart) with
[subtitles](/chapters/audio-intelligence/subtitles) instead.
</Tip>

# To enhance accuracy for any use case

Configure domain-specific words and phrases with [Custom vocabulary](/chapters/audio-intelligence/custom-vocabulary) or
[Custom spelling](/chapters/audio-intelligence/custom-spelling) to capture context-specific terms (brand names, technical words, etc.).

### Language set up

Set `language_config.languages` when the language is known.
See [Language detection](/chapters/language/language-detection) for more details.

### Code Switching

Enable `language_config.code_switching` when speakers mix languages in the same audio.
See [Code switching](/chapters/language/code-switching) for more details.
Loading