Skip to content

Add multipart audio alignment endpoint - #391

Merged
0xShug0 merged 1 commit into
mainfrom
feature/audio-alignments-upload
Sep 2, 2026
Merged

Add multipart audio alignment endpoint#391
0xShug0 merged 1 commit into
mainfrom
feature/audio-alignments-upload

Conversation

@0xShug0

@0xShug0 0xShug0 commented Sep 1, 2026

Copy link
Copy Markdown
Owner

Summary

Adds POST /v1/audio/alignments for forced alignment with uploaded audio bytes. This is the upload-based counterpart to /v1/tasks/run for remote or Docker clients that cannot pass a server-local audio path.

Request shape:

curl http://localhost:8080/v1/audio/alignments \
  -F "file=@speech.wav" \
  -F "model=qwen3-align" \
  -F "text=The task has completed successfully."

The endpoint requires file, model, and text; language and busy_timeout_ms are optional. It only accepts WAV uploads for now and requires the selected model to be configured with task=align and mode=offline.

Fixes #350.

Example Response

{
  "text": "Some call me nature. Others call me Mother Nature.",
  "language": "English",
  "words": [
    {
      "word": "Some",
      "start": 0.266667,
      "end": 0.64,
      "start_sample": 6400,
      "end_sample": 15360,
      "confidence": 0
    }
  ],
  "timing": {
    "wall_ms": 123.0,
    "audio_duration_ms": 456.0,
    "rtf": 0.27
  }
}

Validation

Built the debug server and validated multipart behavior with assets/resources/sample.wav.

Case Response
Qwen3 forced aligner upload 200, words=25, first word includes start, end, start_sample, and end_sample
MMS forced aligner upload 200, words=24, first word includes start, end, start_sample, and end_sample
non-multipart request 400, audio alignment requests must use multipart/form-data
missing file 400, multipart alignment request requires a non-empty 'file' field
empty file 400, multipart alignment request requires a non-empty 'file' field
missing model 400, multipart alignment request requires a 'model' field
missing text 400, multipart alignment request requires a non-empty 'text' field
non-WAV upload 400, only WAV audio uploads are currently supported for alignment
bad busy_timeout_ms 400, multipart busy_timeout_ms field must be an integer
negative busy_timeout_ms 400, busy_timeout_ms must be >= 0 (0 means no client-side bound)
unknown model 400, unknown model id: missing-align-model
wrong task model 400, audio alignment requires a model configured with task=align

@Geequlim

Geequlim commented Sep 2, 2026

Copy link
Copy Markdown

Tested and accepted.

I ran the Docker image built from this PR and verified the new multipart endpoint with the project fixture assets/resources/a.wav and its known transcript.

The request was sent as multipart/form-data to POST /v1/audio/alignments with file, model=qwen3-align, language=en, and text. The server returned HTTP 200 with all 16 expected words, monotonic timestamps, and consistent sample offsets. The first request loaded the model successfully; a warm request completed in about 23 ms server-side (RTF 0.0039).

The implementation matches the intended scope: dedicated upload route, required-field and WAV validation, in-memory audio decoding, task/mode checks, reuse of the existing alignment runtime, timing/sample-offset response fields, and updated documentation. All GitHub CI checks are passing.

One non-blocking observation: the returned confidence value is 0 for each word, matching the current response example. This does not affect the timestamp acceptance for this feature.

No blocking issues found. Accepted from my side — thank you!

@AnatolyRugalev

Copy link
Copy Markdown

This is a very much needed feature, thank you

@0xShug0
0xShug0 merged commit f94a810 into main Sep 2, 2026
9 checks passed
@0xShug0
0xShug0 deleted the feature/audio-alignments-upload branch September 2, 2026 14:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: upload audio for forced alignment

3 participants