Priority Level
Medium
Task Summary
Add a source-tree developer tool that deploys and manages local models through one vLLM process lifecycle. The tool should let contributors run Anonymizer against pinned generation and detection models on a GPU host or inside the supplied container without introducing a second deployment domain.
Technical Details & Implementation Plan
The implementation should:
- Compile pinned TOML profiles into normalized plans with
task, model, vllm, and local sections.
- Provide
compile, launch, probe, status, and stop commands.
- Support stock Hugging Face generation models and NVIDIA GLiNER or GLiNER2 through vLLM Factory.
- Preserve process identity with launch receipts, PID start markers, and process-group ownership checks.
- Probe task-specific capabilities rather than treating a listening port as readiness.
- Bound termination and cleanup so failed or stale processes cannot hang the tool indefinitely.
- Run the same compiler and local-process lifecycle on a GPU host and inside the supplied container.
- Document the deployment workflow and provide representative pinned profiles.
Acceptance criteria:
- Compilation is pure and produces deterministic, digest-protected plans.
- Plan digests are documented as accidental corruption detection, not authentication or semantic proof.
- Generation and GLiNER profiles share one lifecycle implementation.
- Capability probes validate chat completions or GLiNER dynamic labels, offsets, and scores as appropriate.
- Status and stop operations reject stale or reused process identities.
- Cleanup is bounded and reports whether it completed.
- Tests cover lifecycle invariants and behavior rather than profile inventory tables.
- Documentation covers both direct GPU-host deployment and the supplied container.
- A full Anonymizer dataset run succeeds with locally hosted GLiNER and a generation model, and both services clean up successfully.
Out of scope:
- Public package APIs for deployment.
- Remote compute orchestration or attachment to existing endpoints.
- Docker-specific compiler branches.
- Native Transformers GLiNER serving.
- Model-cache discovery.
- Cryptographic authentication of plans.
Implementation is tracked in PR #212.
Dependencies
vLLM Factory integration is pinned by the repository's local-model dependency group.
Priority Level
Medium
Task Summary
Add a source-tree developer tool that deploys and manages local models through one vLLM process lifecycle. The tool should let contributors run Anonymizer against pinned generation and detection models on a GPU host or inside the supplied container without introducing a second deployment domain.
Technical Details & Implementation Plan
The implementation should:
task,model,vllm, andlocalsections.compile,launch,probe,status, andstopcommands.Acceptance criteria:
Out of scope:
Implementation is tracked in PR #212.
Dependencies
vLLM Factory integration is pinned by the repository's local-model dependency group.