AMEVA-Runtime

Next-Gen Unified On-Device Hardware Orchestration & 6-Modality AI Acceleration Runtime (with BitNet 1.58-bit Vulkan Compute) for Mobile & Edge

---

Architecture & Overview

AMEVA-Runtime is a hardware abstraction layer (HAL) and compute orchestration engine engineered specifically for mobile ARM64 devices (Android Termux, Linux Edge). It continuously inspects underlying silicon topology (/dev/kgsl-3d0, /dev/mali0) to route tensor execution across Qualcomm Adreno, ARM Mali, and ARM Cortex CPU-NEON backends.

6-Modality Acceleration Matrix

Modality	Engine Integration	Status (v2.7.1)	Hardware Acceleration Mechanism
1. LLM (Text)	Llama.cpp & Termux-BitNet (1.58-bit i2_s)	Production (v2.7.1)	Vulkan 25/25 layer VRAM offload (ngl=999), BitNet full pipeline (Adreno 17.56 t/s, Mali 3.47 t/s) & strict AmbiguousModelMatchError
2. STT (Speech)	Whisper.cpp (Large-v3-Turbo)	Production (v2.7.1 / STT v1.2.7)	Vulkan GPU acceleration with greedy decoding default (-bs 1), 6-SoC fleet validated (Adreno 3.73x, Mali 2.26x)
3. TTS (Audio)	MeloTTS / Piper / Kokoro / Supertonic	Production (v2.7.1 / TTS v1.5.0)	Bionic Direct Vulkan (/system/lib64/libvulkan.so) & HiFi-GAN 32MB buffer temporal tiling (Adreno RTF 0.88x/0.264x, Mali RTF 0.18x~0.24x)
4. Vision (VLM)	CLIP / MobileVLM / LLaVA	In Development	GGML Vulkan vision encoder tensor bindings
5. Diffusion (Image)	Stable Diffusion v1.5 / FLUX.1	In Development	On-device Vulkan UNet & DiT tensor offload
6. Train (Training)	On-Device LoRA / QLoRA	In Development	Mobile Vulkan gradient descent backpropagation

---

Empirical Physical Device Benchmarks

Tested on physical devices running Android 16 under Termux ARM64:

1. LLM Generation (Qwen2.5-0.5B-Instruct Q4_K_M)

Target Device	Hardware Architecture	Active Backend	Layers in VRAM	Generation Speed	Prompt Processing	Speedup
Galaxy S25	Snapdragon 8 Elite / Adreno 830	Vulkan 1.3	25/25 (100%)	35.80 t/s (27.9 ms/t)	4.53 t/s	35.8x (vs CPU)
Galaxy A35	Exynos 1380 / ARM Mali-G68 MP5	Vulkan 1.3	25/25 (100%)	4.44 t/s (225 ms/t)	6.12 t/s	+26.9% (vs NEON)
Galaxy A35	Cortex-A78 CPU-NEON (3 Threads)	CPU-NEON	0/25	3.55 t/s (281 ms/t)	8.05 t/s	Baseline

2. Speech-to-Text (Whisper Large-v3-Turbo Q5_0, 548MB)

Target Device	Hardware Architecture	Backend Mode	Latency (1-min audio)	GPU Load	CPU Load	Speedup
Galaxy S25	Snapdragon 8 Elite / Adreno 830	Vulkan GPU	4.40 s (0.07x RTF)	Adreno Turbo	~12%	18.5x (vs CPU)
Galaxy S22	Snapdragon 8 Gen 1 / Adreno 730	Vulkan GPU	5.13 s (Encoder)	Adreno Native	~14%	3.73x (vs CPU)
Galaxy A35	Exynos 1380 / Mali-G68 MP5	Vulkan GPU	360.60 s (6m 00s)	949 MHz (100%)	20~30%	2.26x (56% time saved)
Galaxy A35	Cortex-A78 x4 Cores	CPU-NEON	816.48 s (13m 36s)	0%	291%	Baseline

3. Text-to-Speech (AMEVA Bionic Native Vulkan Fleet Benchmarks)

Target Device	Hardware Architecture	Neural Engine / Model	Backend Mode	Latency	RTF	Forensics (RMS/Peak)	Status
Galaxy S22	Snapdragon 8 Gen 1 / Adreno 730	MeloTTS Universal Bilingual	Bionic Vulkan GPU	2,750 ms	0.88x	0.0814 / 0.6974	Real-time Synthesized
Galaxy S21	Exynos 2100 / ARM Mali-G78	Piper VITS (On-chip Tiled)	Bionic Vulkan GPU	560 ms	0.18x	0.0921 / 0.7412	5.5x Faster than RT
Galaxy S20	Exynos 990 / ARM Mali-G77	Piper VITS (On-chip Tiled)	Bionic Vulkan GPU	750 ms	0.24x	0.0890 / 0.7105	4.1x Faster than RT
Galaxy S25	Snapdragon 8 Elite / Adreno 830	Supertonic 3 Flow / Piper	Bionic Vulkan GPU	380 ms	0.12x	0.1042 / 0.8120	Studio Ultra-Fast
Galaxy A35	Exynos 1380 / ARM Mali-G68 MP5	Piper VITS (lessac-medium)	Vulkan GPU	5,180 ms	1.146x	0.0782 / 0.6540	Validated

---

4. BitNet 1.58-bit LLM (Microsoft BitNet b1.58 2B-4T i2_s)

Target Device	Hardware Architecture	Active Backend	Generation Speed	Prompt Latency	Speedup	Status
Galaxy S25	Snapdragon 8 Elite / Adreno 830	AMEVA Vulkan GPU	17.558 t/s	205.9 ms	12.58x	Verified (Ground Truth)
Galaxy S25	Snapdragon 8 Elite / Oryon CPU	Native CPU (4 Threads)	1.396 t/s	2,041.0 ms	1.00x	Baseline
Galaxy A35	Exynos 1380 / Mali-G68 MP5	AMEVA Vulkan GPU	3.471 t/s	1,552.8 ms	5.94x	Verified (Ground Truth)
Galaxy A35	Exynos 1380 / Cortex-A78 CPU	Native CPU (4 Threads)	0.584 t/s	8,775.0 ms	1.00x	Baseline

Root-Cause Driver Solutions

1. ARM Mali-G68 Valhall Integer Truncation: Enforced medium tile matmul kernel dispatch (loadstride_b = 4 > 0), permanently eliminating shader zero-stride infinite loops on subgroup-16 hardware.
2. Qualcomm Adreno 830 JIT Register Bug: Bounded vector column specialization (mul_mat_vec_max_cols = 2), preventing compiler crash VK_ERROR_UNKNOWN (-13).
3. Bionic Direct Vulkan Linking: Permanently bypasses Termux $PREFIX/lib/libvulkan.so Mesa llvmpipe CPU software emulator trap by dynamically binding /system/lib64/libvulkan.so directly, routing compute shader execution to physical Adreno/Mali silicon.
4. Mobile GPU 32MB Memory Ceiling & Temporal Tiling: Resolved MeloTTS HiFi-GAN 52.4MB buffer overflow (VK_ERROR_OUT_OF_DEVICE_MEMORY / kernel TDR kill) via temporal chunk slicing ($T_{\text{chunk}} \le 819$) and Piper VITS on-chip SRAM tiling.
5. Zero-Silent-Fallback & Strict Model Resolution: Guaranteed fail-fast architecture without silent CPU degradation. Explicitly raises AmbiguousModelMatchError on multi-model ambiguity and rejects silent guessing.

---

Installation

# Python SDK
pip install ameva-runtime

# Node.js / TypeScript
npm install @ameva/runtime

---

Quickstart

Python SDK

import ameva_runtime as ameva
from ameva_runtime import vulkan

# 1. Inspect on-device silicon topology
profile = ameva.detect_hardware()
print(f"SoC: {profile.soc_name} | GPU: {profile.gpu_vendor}")

# 2. Run hardware self-test
doc = vulkan.Doctor()
report = doc.run_self_test()
print(f"GPU: {report.device_name} (Passed: {report.passed_stages}/{report.total_stages})")

Node.js / TypeScript

import { Doctor, createContext } from '@ameva/runtime';

const doc = new Doctor();
const report = await doc.runSelfTest();
console.log(`Vulkan GPU: ${report.deviceName}`);

---

Official Documentation & Benchmarks

• Official Architecture & API Reference
• Ecosystem Metrics & Registry Stats
• AMEVA Open-Source Foundation Portal

---

License

Licensed under the Apache-2.0 License. Copyright (c) 2026 Eunho Kim (@uno-km).