OCV is a training-free, post-decoding framework for reducing object hallucination in vision-language model captions. It treats each generated object mention as a commitment, ranks commitment risk within a matched evaluation pool, and applies only adapter-validated local edits under semantic rescue and a per-caption edit budget.
This repository releases a dependency-light offline OCV core, a synthetic end-to-end example, and audited aggregate results. It is not a complete model or benchmark reproduction package.
- Method overview: paper-consistent evidence, pool-relative selection, and bounded editing.
- Results overview: headline evidence, supporting rows, timing scope, and caveats.
- Reproduction guide: runnable offline path and the adapter contract for a full integration.
- Reproducibility status: what is released, partial, external, or not applicable.
- Limitations: interpretation boundaries and explicit non-claims.
- Public result table: five audited compact rows with provenance and caveats.
On COCO/CHAIR-500, InstructBLIP-7B CHAIRs decreases from 0.2360 under the
first-logit decoding baseline to 0.1340 with FLB + OCV, a 43.2% relative
reduction. Lower is better.
This is an aggregate public result, not a SOTA or official benchmark
reproduction claim. Raw generations and datasets are not redistributed. The
corresponding Regular + OCV row selected zero surviving edits and remains
0.2620, so the headline is specific to the FLB source-caption setting.
Evidence: compact result row, verbatim released matrix, and result provenance.
source caption + adapter-extracted U/V/D/A terms
-> normalize terms within one matched pool
-> fuse pool-relative commitment risk
-> select K = ceil(rho * N_pool) highest-risk commitments
-> semantic rescue
-> edit-safety veto
-> accept at most B local edits per caption
-> edited caption + decision trace
There are three high-level evidence families: generation uncertainty U,
visual support V, and contextual anchoring A. The visual family contributes
two fusion terms, level V and dynamics D, so fusion uses U/V/D/A. Larger
term values mean weaker support after risk orientation.
OCV runs after a complete caption is generated and does not change decoder logic or model parameters. A full integration still needs source-model probabilities and visual internal signals through an external adapter. It is decoder-decoupled verification, not API-only black-box text processing.
Python 3.10 or newer is sufficient; the offline core has no mandatory model dependencies.
python -m pip install -e . --no-deps
python scripts/run_offline_ocv.py examples/synthetic_pool.jsonThe invented example deterministically selects two of three commitments,
rescues street, removes the adapter-approved span containing bicycle, and
returns A man stands on a street. It performs no network, model, dataset, or
evaluator access.
| Component | Public status |
|---|---|
| Pool-scoped min-max normalization | Runnable in src/ocv/risk.py |
| Availability-masked fixed-weight fusion | Runnable in src/ocv/risk.py |
Top-rho selection with stable ties |
Runnable in src/ocv/pipeline.py |
| Semantic-rescue decision input | Runnable with adapter-provided reasons |
| Bounds, overlap, budget, and right-to-left edits | Runnable in the offline core |
| Model probability/attention/hidden-state extraction | External adapter |
| Commitment parser and linguistically safe span construction | External adapter |
| Benchmark datasets and evaluators | External resources |
The runnable package therefore verifies the model-independent decision logic on pre-extracted records. It does not claim full end-to-end reproduction of the reported benchmark rows.
Lower is better for every metric below. Values and displayed reductions are migrated without recomputation from the pinned public source record.
| Model | Benchmark | Metric | FLB | FLB + OCV | Delta vs FLB | Boundary |
|---|---|---|---|---|---|---|
| InstructBLIP-7B | COCO/CHAIR-500 | CHAIRs | 0.2360 | 0.1340 | 0.1020 (43.2%) | Strongest supported headline; Regular + OCV made no surviving edits. |
| LLaVA-1.5-7B | COCO/CHAIR-500 | CHAIRs | 0.1560 | 0.1100 | 0.0460 (29.5%) | Aggregate caption-side row. |
| LLaVA-1.5-7B | AMBER-1004 | Hal | 31.6 | 25.0 | 6.6 (20.9%) | Cover decreases from 50.1 to 47.8. |
| InstructBLIP-7B | AMBER-1004 | Hal | 52.5 | 48.4 | 4.1 (7.8%) | Cover decreases from 54.5 to 52.5. |
| LLaVA-1.5-7B | OpenCHAIR | OpenCHAIR | 0.162093 | 0.158447 | 0.003646 (2.2%) | Small gain; recorded local evaluator setup. |
The full 48-row aggregate matrix, including smaller, mixed, and
partial-coverage settings, is preserved in
results/released_caption_results.csv.
The released InstructBLIP microbenchmark records 1.7763 seconds for 20 images
(0.088816 seconds per image) for the OCV verification forward. Generation is
excluded and signal-extraction forward time is included. Other timing rows use
different artifact or hardware scopes, so this repository does not combine
them into a newly estimated end-to-end total.
- Regular: original caption generation without a decoding-stage intervention.
- FLB: first-logit baseline, a decoding-stage source generation method used in the experimental record.
- Object commitment: OCV's semantic interpretation that an object mention asserts image presence or visual relevance.
- Surface phrase: the literal caption span associated with a commitment and used for alignment and editing.
- Pool-relative high risk: high rank among commitments from the same source VLM, source generation method, and benchmark split; it is not a calibrated hallucination probability or evaluator label.
- CHAIRs / Hal / OpenCHAIR: hallucination-oriented metrics for which lower values are better in the released tables.
src/ocv/: offline risk, selection, and editing coreexamples/: invented pre-extracted pool used by the demoscripts/: offline runner and public consistency/hygiene checksresults/: audited aggregate summaries onlydocs/: method, reproduction boundary, setup, and limitationstests/: unit, CLI, provenance, and repository-hygiene tests
python scripts/check_repo_hygiene.py
python scripts/check_result_consistency.py
python -m unittest discover -s tests -v
python -m compileall -q src scripts tests
python scripts/run_offline_ocv.py examples/synthetic_pool.jsonNo model weights, datasets, raw generations, images, checkpoints, private manuscripts, or large evaluator logs are included. The repository makes no claim of SOTA performance, paper acceptance or submission, statistical significance, or official benchmark reproduction. See the public release scope and limitations.
The released code and documentation are available under the Apache License 2.0.