Skip to content

apertus: add 1.5 Omni model and media support - #15

Merged
pd95 merged 21 commits into
feature/apertus-mlxfrom
feature/apertus-1.5-mlx-full
Aug 23, 2026
Merged

pd95 merged 21 commits into
feature/apertus-mlxfrom
feature/apertus-1.5-mlx-full

Conversation

@pd95

@pd95 pd95 commented Jul 30, 2026

Copy link
Copy Markdown
Owner

Motivation

Apertus 1.5 Omni models should import text and media tensors and support native
ordered image/audio conversations through MLX.

Dependency

Base: feature/gemma4-mlx-multi-media (#17).

Changes owned by this PR

  • Add Apertus 1.5 runtime, import, parser, renderer, and thinking controls.
  • Import media tokenizers and implement native ordered image/audio processing.
  • Support 8B/70B configuration, MP3, memory limits, and capability gating.
  • Retain staged dependencies, request state, and media history across runner
    sweeps.

v0.32.11 architecture

The implementation uses official lazy base.MediaModel behavior, model-owned
caches, media-aware prefill, and runner-owned evaluation. A small staged-media
contract preserves graph-cut dependencies; official Nemotron routing remains
unchanged and no obsolete eager-evaluation path is restored.

Validation

The exact nineteen-commit layer passes focused Go tests, integration compile,
locked Vitest, mandatory Linux lint/build, and Darwin MLX build. A fresh pinned
8B NVFP4 import passes the unchanged five-case release-media gate, including
image, audio, ordered history, MP3, metadata, semantic transcription, corrected
PNG count, unload, and empty /api/ps.

Full-stack evidence

The completed ten-layer v0.32.11 integration and release add-on passed their
owner-specific Linux, Darwin, import, inference, tool, thinking, media, and
unload gates. The signed/notarized preview DMG reports the exact packaged
version and passes its fresh six-source packaged-runtime matrix 6/6.

Limitations and out of scope

Apertus 1.1 Mini and affine checkpoints belong to #18.

AI assistance

Codex materially assisted with architecture adaptation, lifecycle validation,
tests, and evidence review. The layer received independent review before
acceptance.

@pd95
pd95 force-pushed the feature/apertus-1.5-mlx-full branch from b7cd1c7 to 4168916 Compare July 31, 2026 15:14
@pd95
pd95 changed the base branch from feature/gemma4-mlx-image-audio to main August 2, 2026 07:56
@pd95
pd95 changed the base branch from main to feature/gemma4-mlx-multi-media August 8, 2026 12:26
@pd95
pd95 force-pushed the feature/gemma4-mlx-multi-media branch from 10d6c82 to 311732f Compare August 8, 2026 17:37
@pd95
pd95 force-pushed the feature/apertus-1.5-mlx-full branch from 4168916 to 04abb65 Compare August 8, 2026 17:37
@pd95 pd95 changed the title mlx: add full Apertus 1.5 multimodal support mlx: add complete Apertus 1.5 support Aug 8, 2026
@pd95
pd95 force-pushed the feature/gemma4-mlx-multi-media branch from 311732f to 55c2562 Compare August 8, 2026 18:45
@pd95
pd95 force-pushed the feature/apertus-1.5-mlx-full branch from 04abb65 to e52ae8f Compare August 8, 2026 18:45
@pd95
pd95 force-pushed the feature/gemma4-mlx-multi-media branch from 55c2562 to daffcf7 Compare August 8, 2026 19:16
@pd95
pd95 force-pushed the feature/apertus-1.5-mlx-full branch from e52ae8f to e890c97 Compare August 8, 2026 19:16
@pd95 pd95 changed the title mlx: add complete Apertus 1.5 support apertus: add 1.5 Omni model and media support Aug 8, 2026
@pd95
pd95 force-pushed the feature/apertus-1.5-mlx-full branch from e890c97 to 64f815f Compare August 12, 2026 15:26
@pd95
pd95 force-pushed the feature/gemma4-mlx-multi-media branch 2 times, most recently from 21c69e6 to 84b7a05 Compare August 14, 2026 21:08
@pd95
pd95 force-pushed the feature/apertus-1.5-mlx-full branch from 64f815f to 3d45abb Compare August 14, 2026 21:08
pd95 and others added 11 commits August 22, 2026 19:27
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
pd95 and others added 10 commits August 22, 2026 19:48
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
@pd95
pd95 force-pushed the feature/apertus-1.5-mlx-full branch from 3d45abb to 441b1aa Compare August 23, 2026 07:28
Base automatically changed from feature/gemma4-mlx-multi-media to feature/apertus-mlx August 23, 2026 19:32
@pd95
pd95 merged commit 441b1aa into feature/apertus-mlx Aug 23, 2026
@pd95
pd95 deleted the feature/apertus-1.5-mlx-full branch August 23, 2026 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant