Skip to content

#POC-4 — Reliability scoring #36

Description

@nsunku

Summary

A deterministic step that blends the three signals into the 1–100 number, assigns a confidence band, and applies hard overrides for decisive red flags. No model call — pure and fully testable.

What to build:

  • score(document_signal, authenticity_signal, consistency_signal) -> ReliabilityResult
  • Weighted blend, starting weights (tunable, recorded into the result): document plausibility 0.45, image authenticity 0.30, document–narrative consistency 0.25
  • Map blended 0–1 → 1–100
  • Override: high-confidence AI-generated OR a hard document-vs-narrative contradiction → cap at the escalate band (≤30) regardless of the blend
  • Band assignment: high 80–100, medium 55–79, low 31–54, escalate 1–30
  • Build a plain-language justification string from the driving signals — no thresholds or prompts leaked, no “approved”/“rejected” language

Acceptance criteria

  • Each band boundary covered by a unit test
  • A decisive fraud signal forces the escalate band regardless of the blend
  • Weights echoed into the result
  • Identical input always yields an identical score (determinism test)
  • Result includes a human-readable justification; no approve/reject language anywhere

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions