diff --git a/chapters/language/code-switching.mdx b/chapters/language/code-switching.mdx index 9284d59..469b291 100644 --- a/chapters/language/code-switching.mdx +++ b/chapters/language/code-switching.mdx @@ -10,7 +10,17 @@ Code switching handles conversations where speakers switch languages mid-utteran Avoid listing dozens of similar languages. -## How to use code switching ? +## When to enable it + +- Speakers **switch languages** mid-conversation (e.g. bilingual meetings, multilingual customer support). +- You need the detected `language` returned **per utterance**. + +## When NOT to enable it + +- The audio is in a **single language** — code switching adds unnecessary processing and can introduce misdetections. +- You've set **exactly one language** in `languages` — in that case `code_switching` is ignored anyway. + +## How to use code switching Enable code switching in the session configuration: @@ -33,10 +43,11 @@ When enabled, transcript messages include the detected `language` per utterance **Do not enable `code_switching` with an empty `languages` list.** When no languages are specified, the language detector evaluates every utterance against 100+ supported languages, leading to frequent misdetections - - especially between similar-sounding languages. + especially between similar-sounding languages. If possible, provide a short list of + languages you **actually expect** in the audio. ## Recommendations - Limit the number of languages to those you expect. -- Combine with [custom vocabulary](/chapters/pre-recorded-stt/features/custom-vocabulary) per language for domain terms. +- Combine with [custom vocabulary](/chapters/audio-intelligence/custom-vocabulary) per language for domain terms. diff --git a/chapters/language/language-detection.mdx b/chapters/language/language-detection.mdx index e74fd75..a714e62 100644 --- a/chapters/language/language-detection.mdx +++ b/chapters/language/language-detection.mdx @@ -11,7 +11,27 @@ Gladia can automatically detect the spoken language from audio, either within a ## How to use language detection ? -You can either omit `language_config.languages` to search across all supported languages, or pass a constrained list: +- You **know** the language of the audio ahead of time. +- The audio is **monolingual** (single language throughout). +- You want the **fastest, most accurate** results. + +```json +{ + "language_config": { + "languages": ["en"], + "code_switching": false + } +} +``` + +## When to use auto-detection + +- You process audio in **many different languages** and don't know which one beforehand. +- You want Gladia to pick the language automatically. + +## How to use language detection + +You can either omit `language_config.languages` (or pass an empty list) to search across all supported languages, or pass a constrained list: @@ -41,16 +61,26 @@ You can either omit `language_config.languages` to search across all supported l } ``` + +The detected `language` is returned on each utterance and can vary over time if you also enable [code switching](/chapters/language/code-switching). - + + When `code_switching` is `false` and no language is set, the language is + detected on the **first utterance** and reused for the rest of the session or + file. If the beginning of your audio contains silence, music, or a different + language than the main content, this can lead to incorrect detection for the + whole transcription. + -The detected `language` is returned on each utterance and can vary over time if you also enable code switching. + + Even when using auto-detection, pass a **small list of likely languages** in + `languages` to constrain the search. This improves both accuracy and + processing time. + ## Tips -- Use supported languages to pick valid ISO 639-1 codes. +- Use [supported languages](/chapters/language/supported-languages) to pick valid ISO 639-1 codes. - Combine with `messages_config.receive_partial_transcripts` in real time to get immediate feedback. -- For mixed-language conversations, enable code switching. - - +- For mixed-language conversations, enable [code switching](/chapters/language/code-switching). diff --git a/chapters/live-stt/recommended-parameters.mdx b/chapters/live-stt/recommended-parameters.mdx index f030868..fcafb1a 100644 --- a/chapters/live-stt/recommended-parameters.mdx +++ b/chapters/live-stt/recommended-parameters.mdx @@ -1,46 +1,12 @@ --- title: Recommended Parameters by Use Case sidebarTitle: Recommended Parameters -description: Best parameter configurations for realtime transcription depending on your application (Voice Agents, Meeting Recorders, Call Centers, Subtitles). ---- - -import LanguageDetection from "/snippets/recommended-params/language-detection.mdx"; -import CodeSwitching from "/snippets/recommended-params/code-switching.mdx"; -import CustomVocabulary from "/snippets/recommended-params/custom-vocabulary.mdx"; -import DiarizationVsMultichannel from "/snippets/recommended-params/diarization-vs-multichannel.mdx"; - -The right parameter configuration can make a significant difference in transcription quality and latency for realtime use cases. This guide covers recommended starting points for common scenarios and highlights pitfalls that frequently trip up new integrations. - - - These recommendations apply to the **[Realtime - API](/chapters/live-stt/quickstart)** and are passed during session - initialization. They are starting points — tune them to match your - specific needs. - - ---- - -## Language Configuration - - - ---- - -## Code Switching - - - ---- - -## Custom Vocabulary - - - +description: Best parameter configurations for realtime transcription depending on your use case. --- ## Voice Agents -For callbots, customer-service assistants, or voice-driven chatbots the top priority is **low latency**. The agent must react quickly to user speech, even if sentence boundaries are not perfectly formed. +Optimize for **low latency** and fast turn-taking. | Parameter | Recommended value | Why | | --------------------------------------------- | ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -50,63 +16,66 @@ For callbots, customer-service assistants, or voice-driven chatbots the top prio | `realtime_processing.custom_vocabulary` | `true` | Add product names and action keywords so the agent can react accurately. | - This setup is optimized for **fast turn-taking**. If utterances get cut off - mid-sentence, raise `endpointing` slightly. + If utterances get cut off mid-sentence, raise `endpointing` slightly. ---- - ## Meeting Recorders -For apps that record and transcribe meetings in real time — team stand-ups, board sessions, 1-on-1s — the goal is to produce a **structured, speaker-attributed live transcript** that can feed downstream features like summarization or live note-taking. +Optimize for a **structured, speaker-attributed live transcript**. -| Parameter | Recommended value | Why | -| --------------------------------------------- | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `endpointing` | `0.3` - `0.5` | Lets speakers finish their sentences before closing an utterance. See [Endpointing](/chapters/live-stt/features/endpointing). | -| `maximum_duration_without_endpointing` | `15` | Prevents very long utterances in case a speaker doesn't pause. | -| `messages_config.receive_partial_transcripts` | `true` | Feeds live captions to the UI while waiting for final results. See [Partial transcripts](/chapters/live-stt/features/partial-transcripts). | -| `language_config.languages` | Set explicitly | Meeting language is almost always known in advance — setting it avoids detection overhead. | -| `realtime_processing.custom_vocabulary` | `true` | Add company-specific terms, project names, and participant names for better accuracy. | +| Parameter | Recommended value | Why | +| --------------------------------------------- | ----------------- | ------------------------------------------- | +| `endpointing` | `0.3` - `0.5` | Lets speakers finish sentences. | +| `maximum_duration_without_endpointing` | `15` | Caps utterances when a speaker doesn't pause. | +| `messages_config.receive_partial_transcripts` | `true` | Feeds live captions to the UI. | +| `language_config.languages` | Set explicitly | Avoids detection overhead. | +| `realtime_processing.custom_vocabulary` | `true` | Company, project, and participant names. | - - ---- ## Call Centers -For live phone calls the priorities are **speaker identification** and **fast, accurate transcription** despite variable audio quality (telephony codecs, background noise, cross-talk). +Optimize for **speaker identification** and accuracy on telephony audio. -| Parameter | Recommended value | Why | -| --------------------------------------------- | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `endpointing` | `0.2` - `0.4` | Keeps turn-taking responsive without cutting off mid-sentence. See [Endpointing](/chapters/live-stt/features/endpointing). | -| `maximum_duration_without_endpointing` | `15` | Prevents very long utterances in monologue-style segments. | -| `language_config.languages` | Set explicitly (e.g. `["en"]`) | Call center audio typically has a known language. Setting it avoids detection errors on noisy recordings. | -| `realtime_processing.custom_vocabulary` | `true` | Add product names, plan names, and internal terminology. | +| Parameter | Recommended value | Why | +| --------------------------------------- | ------------------------------ | ---------------------------------------- | +| `endpointing` | `0.2` - `0.4` | Responsive without cutting mid-sentence. | +| `maximum_duration_without_endpointing` | `15` | Caps monologue-style segments. | +| `language_config.languages` | Set explicitly (e.g. `["en"]`) | Avoids detection errors on noisy audio. | +| `realtime_processing.custom_vocabulary` | `true` | Product, plan, and internal terms. | - - For calls with more than two participants (e.g. conference bridges), use - `diarization_config.min_speakers` / `max_speakers` instead of `number_of_speakers` to give the - model a flexible range. + For noisy telephony audio, set `pre_processing.speech_threshold` to `0.8`. ---- - ## Subtitles & Captioning -When providing live subtitles, the goal is to **sync text with the speaker in real time**. The right balance between speed and segment quality depends on whether captions are displayed live or post-produced. +Optimize for **syncing text with the speaker**. -| Parameter | Recommended value | Why | -| --------------------------------------------- | -------------------------------------- | ----------------------------------------------------------------------------------------------- | -| `endpointing` | `0.3` (live) / `0.8` (post-production) | Lower values keep captions close to the speaker; higher values produce cleaner subtitle blocks. | -| `maximum_duration_without_endpointing` | `5` | Prevents excessively long subtitle segments that are hard to read on screen. | -| `messages_config.receive_partial_transcripts` | `true` | Shows words as they are spoken, then refines them when the final result arrives. | -| `language_config.languages` | Set explicitly | Avoids detection lag when the broadcast language is known. | +| Parameter | Recommended value | Why | +| --------------------------------------------- | -------------------------------------- | ------------------------------------------------ | +| `endpointing` | `0.3` (live) / `0.8` (post-production) | Lower = closer to speech; higher = cleaner blocks. | +| `maximum_duration_without_endpointing` | `5` | Keeps subtitle segments readable. | +| `messages_config.receive_partial_transcripts` | `true` | Shows words as they are spoken. | +| `language_config.languages` | Set explicitly | Avoids detection lag. | - For post-production subtitles generated from a recording, consider using the - [Pre-recorded API](/chapters/pre-recorded-stt/quickstart) with the dedicated - [subtitles feature](/chapters/audio-intelligence/subtitles) instead — it - produces SRT/VTT files with fine-grained timing controls. + For post-production subtitles from a recording, use the + [Pre-recorded API](/chapters/pre-recorded-stt/quickstart) with + [subtitles](/chapters/audio-intelligence/subtitles) instead. + +# To enhance accuracy for any use case + +Configure domain-specific words and phrases with [Custom vocabulary](/chapters/audio-intelligence/custom-vocabulary) or +[Custom spelling](/chapters/audio-intelligence/custom-spelling) to capture context-specific terms (brand names, technical words, etc.). + +### Language set up + +Set `language_config.languages` when the language is known. +See [Language detection](/chapters/language/language-detection) for more details. + +### Code Switching + +Enable `language_config.code_switching` when speakers mix languages in the same audio. +See [Code switching](/chapters/language/code-switching) for more details. diff --git a/chapters/pre-recorded-stt/recommended-parameters.mdx b/chapters/pre-recorded-stt/recommended-parameters.mdx index 3ce2002..41abf76 100644 --- a/chapters/pre-recorded-stt/recommended-parameters.mdx +++ b/chapters/pre-recorded-stt/recommended-parameters.mdx @@ -1,7 +1,7 @@ --- title: Recommended Parameters by Use Case sidebarTitle: Recommended Parameters -description: Best parameter configurations for pre-recorded transcription depending on your application (Meeting Recorders, Call Centers, Podcasts, Subtitles, Multilingual Content). +description: Best parameter configurations for pre-recorded transcription depending on your use case. --- import LanguageDetection from "/snippets/recommended-params/language-detection.mdx"; @@ -9,118 +9,88 @@ import CodeSwitching from "/snippets/recommended-params/code-switching.mdx"; import CustomVocabulary from "/snippets/recommended-params/custom-vocabulary.mdx"; import DiarizationVsMultichannel from "/snippets/recommended-params/diarization-vs-multichannel.mdx"; -The right parameter configuration can significantly impact transcription quality for pre-recorded audio. This guide covers recommended starting points for common scenarios and highlights pitfalls that frequently trip up new integrations. - - - These recommendations apply to the **[Pre-recorded - API](/chapters/pre-recorded-stt/quickstart)** and are passed in the `POST - /v2/pre-recorded` request body. They are starting points — tune them to match - your specific needs. - - ---- - -## Language Configuration - - - ---- - -## Code Switching - - - ---- - -## Custom Vocabulary - - - ---- - ## Meeting Recorders -For apps that record and process meetings — team stand-ups, board sessions, 1-on-1s — the goal is to produce **structured, actionable meeting notes** with clear speaker attribution. Meetings typically have a known set of participants and benefit heavily from post-processing features like summarization. +Optimize for **structured, speaker-attributed meeting notes**. -| Parameter | Recommended value | Why | -| -------------------------------------------------- | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `diarization` | `true` | Attributes speech to each participant. See [Speaker diarization](/chapters/audio-intelligence/speaker-diarization). | -| `diarization_config.min_speakers` / `max_speakers` | Set a range (e.g. `2`-`10`) | Meeting size varies — a range lets the model adapt without over- or under-splitting speakers. | -| `summarization` | `true` | Generates a summary for quick review. Use `bullet_points` type for action-item style output. See [Summarization](/chapters/audio-intelligence/summarization). | -| `named_entity_recognition` | `true` | Surfaces people, organizations, dates, and other key entities mentioned during the meeting. See [NER](/chapters/audio-intelligence/named-entity-recognition). | -| `sentences` | `true` | Produces well-segmented, readable output suitable for meeting minutes. See [Sentences](/chapters/audio-intelligence/sentences). | -| `language_config.languages` | Set explicitly | Meeting language is almost always known in advance — setting it avoids detection overhead. | -| `custom_vocabulary` | `true` | Add company-specific terms, project names, and participant names for better accuracy. | +| Parameter | Recommended value | Why | +| -------------------------------------------------- | --------------------------- | ---------------------------------------- | +| `diarization` | `true` | Attributes speech to each participant. | +| `diarization_config.min_speakers` / `max_speakers` | Set a range (e.g. `2`-`10`) | Adapts without over- or under-splitting. | +| `summarization` | `true` | Quick review; use `bullet_points` type. | +| `named_entity_recognition` | `true` | Surfaces people, orgs, and dates. | +| `sentences` | `true` | Readable output for meeting minutes. | +| `language_config.languages` | Set explicitly | Avoids detection overhead. | +| `custom_vocabulary` | `true` | Company, project, and participant names. | ---- - ## Call Centers -For recorded phone calls the priorities are **speaker identification** and **accurate transcription** despite variable audio quality (telephony codecs, background noise, cross-talk). +Optimize for **speaker identification** and accuracy on telephony audio. -| Parameter | Recommended value | Why | -| --------------------------------------- | ------------------------------ | --------------------------------------------------------------------------------------------------------------------------- | -| `language_config.languages` | Set explicitly (e.g. `["en"]`) | Call center audio typically has a known language. Setting it avoids detection errors on noisy recordings. | -| `diarization` | `true` | Separates agent and customer speech. See [Speaker diarization](/chapters/audio-intelligence/speaker-diarization). | -| `diarization_config.number_of_speakers` | `2` | Most calls have exactly two participants — giving this hint improves speaker assignment accuracy. | -| `custom_vocabulary` | `true` | Add product names, plan names, and internal terminology. | -| `summarization` | `true` | Automatically generates a summary for agent wrap-up notes. See [Summarization](/chapters/audio-intelligence/summarization). | +| Parameter | Recommended value | Why | +| --------------------------------------- | ------------------------------ | ---------------------------------------- | +| `diarization` | `true` | Separates agent and customer speech. | +| `diarization_config.number_of_speakers` | `2` | Most calls have exactly two speakers. | +| `language_config.languages` | Set explicitly (e.g. `["en"]`) | Avoids detection errors on noisy audio. | +| `custom_vocabulary` | `true` | Product, plan, and internal terms. | +| `summarization` | `true` | Wrap-up notes for agents. | - +## Podcasts & Interviews ---- +Optimize for **readability** and correct speaker attribution. -## Podcasts & Interviews +| Parameter | Recommended value | Why | +| -------------------------------------------------- | -------------------------- | ---------------------------------------- | +| `diarization` | `true` | Essential for multi-speaker content. | +| `diarization_config.min_speakers` / `max_speakers` | Set a range (e.g. `2`-`4`) | Flexible when speaker count varies. | +| `sentences` | `true` | Readable output for publishing. | +| `custom_vocabulary` | `true` | Guest names, show terms, brand names. | +| `language_config.languages` | Set explicitly | Avoids detection overhead. | -For long-form audio with multiple speakers the focus is on **readability** and **correct speaker attribution**. Transcripts are often repurposed as articles or show notes, so segment quality matters. +## Subtitles & Captioning -| Parameter | Recommended value | Why | -| -------------------------------------------------- | -------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | -| `diarization` | `true` | Essential for multi-speaker content. | -| `diarization_config.min_speakers` / `max_speakers` | Set a range (e.g. `2`-`4`) | Provides a flexible hint when the exact count varies across episodes. | -| `sentences` | `true` | Produces well-segmented, readable output suitable for publishing. See [Sentences](/chapters/audio-intelligence/sentences). | -| `custom_vocabulary` | `true` | Add recurring guest names, show-specific terms, and brand names. | -| `language_config.languages` | Set explicitly | Podcast language is almost always known in advance. | +Optimize for **readable on-screen captions**. ---- +| Parameter | Recommended value | Why | +| --------------------------------------------- | ------------------ | ------------------------------------------------ | +| `subtitles` | `true` | Enables SRT/VTT generation. | +| `subtitles_config.formats` | `["srt", "vtt"]` | Covers common players and platforms. | +| `subtitles_config.maximum_characters_per_row` | `42` | Standard broadcast readability limit. | +| `subtitles_config.maximum_rows_per_caption` | `2` | Keeps captions compact on screen. | +| `subtitles_config.style` | `"compliance"` | Stricter rules for broadcast/accessibility. | +| `translation` | `true` (if needed) | Generates subtitles per target language. | -## Subtitles & Captioning + + For live captions streamed in real time, use the + [Realtime API](/chapters/live-stt/quickstart) instead — see + [Live recommended parameters](/chapters/live-stt/recommended-parameters#subtitles--captioning). + -When generating subtitle files from pre-recorded content, tune the formatting parameters for the best viewing experience. Gladia produces SRT and VTT files directly — no post-processing needed. See [Subtitles](/chapters/audio-intelligence/subtitles) for the full parameter reference. +## Multilingual Content + +Optimize for **mixed-language audio**. -| Parameter | Recommended value | Why | -| --------------------------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------ | -| `subtitles` | `true` | Enables subtitle generation. | -| `subtitles_config.formats` | `["srt", "vtt"]` | Generate both formats to cover different players and platforms. | -| `subtitles_config.maximum_characters_per_row` | `42` | Standard broadcast limit for readability. | -| `subtitles_config.maximum_rows_per_caption` | `2` | Keeps captions compact on screen. | -| `subtitles_config.style` | `"compliance"` | Uses stricter formatting rules suited for broadcast or accessibility requirements. | -| `translation` | `true` (if needed) | When enabled, subtitles are automatically generated for each target language. See [Translation](/chapters/audio-intelligence/translation). | +| Parameter | Recommended value | Why | +| -------------------------------- | ------------------------------------------------------ | ---------------------------------------- | +| `language_config.languages` | List of expected languages (e.g. `["en", "fr", "de"]`) | Constrain to 3–5 languages for accuracy. | +| `language_config.code_switching` | `true` | Detects language shifts across utterances. | +| `custom_vocabulary` | `true` | Add terms per language with `language` tags. | - For live captions streamed in real time, use the [Realtime - API](/chapters/live-stt/quickstart) with partial transcripts instead — see the - [Live recommended - parameters](/chapters/live-stt/recommended-parameters#subtitles--captioning) - guide. + Do not enable `code_switching` with an empty `languages` list — the detector + would evaluate against 100+ languages and misdetect frequently. ---- +## Language Configuration -## Multilingual Content + -For content with mixed languages — conferences, multilingual media, interviews with speakers from different countries — combine language detection with code switching. +## Code Switching -| Parameter | Recommended value | Why | -| -------------------------------- | ------------------------------------------------------ | --------------------------------------------------------------------------------------------------- | -| `language_config.languages` | List of expected languages (e.g. `["en", "fr", "de"]`) | Constrain to 3-5 expected languages for best accuracy. | -| `language_config.code_switching` | `true` | Detects language shifts across utterances. See [Code switching](/chapters/language/code-switching). | -| `custom_vocabulary` | `true` | Add terms for each language with appropriate `language` tags on each entry. | + - - Do not enable `code_switching` with an empty `languages` list. The detector - would evaluate every utterance against 100+ languages, leading to frequent - misdetections — especially between similar-sounding languages. - +## Custom Vocabulary + + diff --git a/snippets/recommended-params/diarization-vs-multichannel.mdx b/snippets/recommended-params/diarization-vs-multichannel.mdx index 13f1a49..6a5907d 100644 --- a/snippets/recommended-params/diarization-vs-multichannel.mdx +++ b/snippets/recommended-params/diarization-vs-multichannel.mdx @@ -1,5 +1,8 @@ - **Diarization vs. multi-channel:** if each speaker is on a **separate audio channel** (e.g. a, use the `channel` field on each utterance to identify who is speaking — diarization is not needed. See [Multiple channels](/chapters/limits-and-specifications/multiple-channels) + **Diarization vs. multi-channel:** if each speaker is on a **separate audio + channel**, use the `channel` field on each utterance — diarization is not + needed. See [Multiple channels](/chapters/limits-and-specifications/multiple-channels). - If all speakers share a **single audio channel**, enable `diarization` to separate the speakers. See [Speaker diarization](/chapters/audio-intelligence/speaker-diarization). + If all speakers share a **single audio channel**, enable `diarization`. See + [Speaker diarization](/chapters/audio-intelligence/speaker-diarization).