fusion-embedding

简体中文

fusion-embedding is a small Rust client for Qwen3-VL embedding deployments with fixed wire profiles for vLLM, llama.cpp, DashScope, and SGLang. A Client is constructed with one backend, endpoint, deployment identity, and expected vector dimension.

Backends and features

Feature	Backend	Constructor
vllm	vLLM OpenAI-compatible embeddings	Client::new(VllmConfig)
llama-cpp	llama.cpp embeddings	Client::new(LlamaCppConfig)
dashscope	DashScope native multimodal embeddings	Client::new(DashScopeConfig)
sglang	SGLang v0.5.11+ standard embeddings	Client::new(SglangConfig)

The configuration type selects the backend.

vLLM, llama.cpp, and DashScope are enabled by default; SGLang is opt-in. Select only the backend you need:

[dependencies]
fusion-embedding = { version = "0.1", default-features = false, features = ["sglang"] }

Scope and limits

embed_texts(&[S]) returns BatchEmbeddingResponse; embed_multimodal(&[MultimodalPart]) fuses all parts into one EmbeddingResponse. Parts are Text, ImageDataUrl, or VideoSource, with lowercase constructors. Images are canonical PNG or JPEG base64 data URLs.

Backend	Images per multimodal vector	Limits
vLLM	One or more, in order	Configure a sufficient multimodal image limit; the model must support multiple images.
llama.cpp	One or more, in order	Model and mmproj must support multiple images.
DashScope	Up to 5	1–20 parts.
SGLang standard endpoint	At most 1	No multiple images in one item; at most one video; image → video → text order.

Use embed_texts_with_options or embed_multimodal_with_options to pass EmbedOptions by value.

SGLang fusion parts must be in canonical image → video → text order. Each kind is optional but may appear at most once; non-canonical or repeated parts are rejected.

For Qwen text-only deployments, vLLM and llama.cpp apply a fixed Qwen instruction/chat profile to every embed_texts item before sending the batch. DashScope text requests remain native.

Install

[dependencies]
fusion-embedding = "0.1"
url = "2"

Add secrecy = "0.10" when constructing a vLLM or SGLang client with an API key, or any DashScope client.

Minimal calls

Construct a vLLM client and embed text:

use std::num::NonZeroUsize;

use fusion_embedding::{Client, VllmConfig};
use url::Url;

# async fn example() -> Result<(), Box<dyn std::error::Error>> {
let client = Client::new(VllmConfig::new(
    Url::parse("http://127.0.0.1:8000/v1/models")?,
    "qwen3-vl-embedding",
    "my-deployment-revision",
    NonZeroUsize::new(2048).unwrap(),
)?)?;
let response = client
    .embed_texts(&["a short sentence"])
    .await?;
println!("{}", response.embeddings()[0].len());
# Ok(())
# }

For image-text fusion, create the data URL outside the library and preserve the desired part order:

# use fusion_embedding::{Client, MultimodalPart};
# async fn example(client: Client, image_data_url: String) -> Result<(), fusion_embedding::Error> {
let response = client
    .embed_multimodal(
        &[
            MultimodalPart::image_data_url(image_data_url),
            MultimodalPart::text("a red square"),
        ],
    )
    .await?;
# let _ = response;
# Ok(())
# }

The value passed to ImageDataUrl has the form data:image/png;base64,<canonical-base64> or data:image/jpeg;base64,<canonical-base64>. Encode image bytes with your chosen base64 encoder; do not pass a path or URL.

Backend configuration

Backend	new arguments	Endpoint behavior	Dimensions and authentication
vLLM	base_endpoint, model, pipeline_id, expected_dimensions	Accepts / or /v1/models, then uses /v1/embeddings	Native dimensions by default; .with_server_dimensions(...) requests one. API key is optional via .with_api_key(SecretString).
llama.cpp	endpoint, model_identity, pipeline_id, expected_dimensions, media_marker	Accepts / or /v1/models, then uses /v1/embeddings	Native dimensions only. model_identity identifies the embedding space and is not sent to llama.cpp.
DashScope	full_native_post_url, model, pipeline_id, native_dimensions, api_key	Requires the complete native POST URL	API key is required. .with_server_dimensions(...) accepts only 256, 512, 768, 1024, 1536, 2048, or 2560.
SGLang	endpoint, model, pipeline_id, native_dimensions	Requires SGLang v0.5.11+; accepts / or /v1/models, then uses /v1/embeddings	Native dimensions only; API key is optional. Structured inputs are Jinja-templated by SGLang, so the client sends no manual Qwen text wrapper.

All configurations support .with_timeouts(connect_timeout, request_timeout). Each also has .allow_remote_http(true), which is only for an intentional non-loopback cleartext deployment.

Responses, errors, and safety

BatchEmbeddingResponse exposes embeddings()/into_embeddings(); EmbeddingResponse exposes embedding()/into_embedding(). Both expose metadata via space(), usage(), and diagnostics().

Error distinguishes invalid configuration or input, transport failures, provider API errors, invalid responses, and dimension mismatches. Successful responses are checked for complete unique indices, expected dimensions, finite non-zero vectors, then L2-normalized by the library.

HTTPS is accepted by default. Cleartext HTTP is allowed only for loopback endpoints, or after explicit .allow_remote_http(true) opt-in for a remote endpoint. Redirects are disabled. API keys and content are redacted in relevant Debug output.

E2E tests

The live tests are ignored and read process environment variables only; the library itself does not load .env. Use env.example as a names-only template, export the applicable values, and do not commit secrets. They also run a three-image COCO image-to-description retrieval sanity check with a fixed cosine threshold and a Top-1 assertion; sample attribution is in samples/coco-mini/README.md.

Backend	Required variables	Optional variables
vLLM	E2E_IMAGE_PATH, VLLM_MODELS_URL (or VLLM_BASE_URL), VLLM_CONFIGURED_MODEL, VLLM_PIPELINE_ID, VLLM_DIMENSIONS	VLLM_API_KEY, VLLM_SEND_DIMENSIONS, VLLM_ALLOW_REMOTE_HTTP
llama.cpp	E2E_IMAGE_PATH, LLAMA_CPP_MODELS_URL, LLAMA_CPP_CONFIGURED_MODEL, LLAMA_CPP_PIPELINE_ID, LLAMA_CPP_DIMENSIONS	LLAMA_CPP_MEDIA_MARKER, LLAMA_CPP_ALLOW_REMOTE_HTTP
DashScope official	E2E_IMAGE_PATH, DASHSCOPE_API_KEY, DASHSCOPE_BASE_URL, DASHSCOPE_CONFIGURED_MODEL, DASHSCOPE_PIPELINE_ID, DASHSCOPE_NATIVE_DIMENSIONS	DASHSCOPE_DIMENSIONS
DashScope enterprise	E2E_IMAGE_PATH, DASHSCOPE_EMBEDDING_API_KEY, DASHSCOPE_EMBEDDING_BASE_URL, DASHSCOPE_ENTERPRISE_CONFIGURED_MODEL, DASHSCOPE_ENTERPRISE_PIPELINE_ID, DASHSCOPE_ENTERPRISE_NATIVE_DIMENSIONS	DASHSCOPE_ENTERPRISE_DIMENSIONS, DASHSCOPE_ENTERPRISE_ALLOW_REMOTE_HTTP
SGLang	E2E_IMAGE_PATH, SGLANG_MODELS_URL (or SGLANG_BASE_URL), SGLANG_MODEL, SGLANG_PIPELINE_ID, SGLANG_DIMENSIONS, SGLANG_VIDEO_SOURCE	SGLANG_API_KEY, SGLANG_ALLOW_REMOTE_HTTP

If LLAMA_CPP_MEDIA_MARKER is absent, the E2E test probes the configured llama.cpp endpoint's /props route. Run one provider test, or all configured ignored tests:

cargo test --test e2e vllm_e2e -- --ignored
cargo test --no-default-features --features sglang --test e2e sglang_e2e -- --ignored
cargo test --test e2e -- --ignored

The SGLang client has local HTTP contract coverage only; no live SGLang E2E result has been recorded.

Development and release checks

CI runs these checks on pushes to main and pull requests. For a later release, bump and merge the Cargo.toml version first, then push the matching vX.Y.Z tag. Configure crates.io Trusted Publishing for owner agentsyaml, repository fusion-embedding, workflow release.yml, and environment release. Version 0.1.0 was published manually; do not create a release tag for it.

cargo fmt --check
cargo test
cargo package