diff --git a/README.md b/README.md index 64f777d..901b5ec 100644 --- a/README.md +++ b/README.md @@ -1,176 +1,64 @@ # TargetIntel-IO -[![Tests](https://github.com/rsolerortuno/TargetIntel-IO/actions/workflows/tests.yml/badge.svg)](https://github.com/rsolerortuno/TargetIntel-IO/actions/workflows/tests.yml) [![Latest release](https://img.shields.io/github/v/release/rsolerortuno/TargetIntel-IO)](https://github.com/rsolerortuno/TargetIntel-IO/releases/latest) [![Python](https://img.shields.io/badge/Python-3.11-3776AB.svg)](https://www.python.org/) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE) +**Transparent, deterministic therapeutic-intent target triage for anti-PD-1-resistant melanoma.** -**Explainable, therapeutic-intent-aware target intelligence for anti-PD-1-resistant melanoma.** +TargetIntel-IO helps research teams distinguish possible therapeutic targets, biomarkers, resistance mechanisms, immune-context markers, tumor-intrinsic drivers, and poor direct targets while preserving the rules and evidence boundaries behind each result. -TargetIntel-IO is a reproducible scientific software project for classifying, prioritizing, and explaining candidate therapeutic targets and biomarkers. It combines a deterministic translational-biology baseline with an emerging, auditable evidence layer for literature, functional genomics, single-cell, spatial, and clinical-response data. - -> **Not simply “What is the best target?” but “Best candidate for which therapeutic intent, supported by which evidence, and with which limitations?”** +> Research-use software only. It generates hypotheses and portfolio-triage outputs, not medical advice, treatment recommendations, validated therapeutic targets, qualified biomarkers, causal proof, or clinical-response predictions. ## Project status | Layer | Status | Purpose | |---|---|---| -| **v0.1.3 deterministic baseline** | Available | Transparent target classification, therapeutic-intent ranking, benchmark evaluation, hypothesis cards, reports, and sensitivity analysis | -| **v0.2.0 Common Evidence Layer** | Complete | Typed contracts, validation, immutable provenance, storage, retrieval, and post-ranking report decoration | -| **v0.3.0 grounded-evidence infrastructure** | Complete | Provider-neutral execution, audited extraction, mandatory review, reviewed snapshots, grounded synthesis, and safe Markdown export | -| **v0.4.0 target feasibility** | Complete | Offline deterministic feasibility retrieval, normalized profiles, coverage reporting, modality composition, and post-ranking presentation | - -v0.2.0 is infrastructure and report decoration, not clinical validation or a production LLM extractor. v0.3.0 remains research infrastructure and does not alter deterministic scores, rankings, or role classification. - -v0.4.0 adds a separate, descriptive feasibility layer after deterministic prioritization. It can retrieve an explicit directed target universe independently of association rank and report its coverage, while retaining unresolved, no-record, and failed outcomes. Source-linked profiles retain clinical precedence, modality-specific tractability, doability, safety-data state, missingness, and contradictions. Feasibility does not change scores, roles, ranks, ordering, or selection; missing safety data does not mean safety. -Run the fully offline deterministic demonstration: - -```bash -python examples/feasibility/run_v040_mock_demo.py --output-dir /tmp/targetintel-v040-demo -``` +| Deterministic baseline | Available | Role classification and therapeutic-intent ranking | +| Common Evidence Layer | Complete | Typed source-linked evidence and read-only report decoration | +| Grounded evidence infrastructure | Complete | Audited extraction, review and grounded synthesis | +| Target feasibility | Complete | Offline modality-specific feasibility and coverage | +| Functional dependency | Research preview | DepMap/CRISPR evidence, closure, portable reporting and human review | -See the [feasibility example](examples/feasibility/README.md) and [v0.4.0 release notes](docs/releases/v0.4.0.md). +The project has progressed through v0.1.3 deterministic therapeutic-intent baseline, v0.2.0 Common Evidence Layer, v0.3.0 grounded evidence and human-review infrastructure, v0.4.0 target feasibility, and v0.5.0 DepMap/CRISPR functional-dependency architecture. v0.5.0 implementation and reporting architecture are complete; real DepMap Public 26Q1 repository snapshot publication is pending Issue 512. -### v0.3.0 evidence-to-synthesis boundary - -The original deterministic pipeline still performs target classification and therapeutic-intent scoring. The separate v0.3.0 path moves source-linked observations through provider-neutral extraction, audit, mandatory human review, explicit persistence, immutable reviewed snapshots, and cited target-level synthesis. It creates no score, ranking, role, or treatment recommendation. Human approval permits controlled software promotion only; it is not scientific or clinical validation. Obsidian is an optional rendered destination, never a scientific source of truth. - -Run the fully offline synthetic demonstration: - -```bash -python examples/llm/run_v030_mock_demo.py --output-dir /tmp/targetintel-v030-demo -``` +## Biological problem and framing -See the [demo guide](examples/llm/README.md) and [v0.3.0 release notes](docs/releases/v0.3.0.md). - -### Version roadmap and current release - -v0.5.0 DepMap/CRISPR functional dependency is complete as a reproducible research-preview workflow: 56 benchmark targets, 331 discovery targets, 18,531 background genes, 100% benchmark and holdout coverage, a preserved 300-target baseline, disabled automatic activation, and zero differing scientific artifacts across two independent runs. - -Roadmap: v0.2.0 Common Evidence Layer; v0.3.0 Grounded Literature Copilot and provider-agnostic LLM integration; v0.4.0 Target feasibility and expanded Open Targets integration; v0.6.0 Single-cell and spatial context; v0.7.0 Clinical-response research model; v0.8.0 De novo target discovery and knowledge graph; v1.0.0 Multitumor target-intelligence platform. See the [v0.5.0 release notes](docs/releases/v0.5.0.md) and [repository-safe evidence](docs/releases/evidence/v0.5.0/). - -## Why this project exists - -A biologically relevant gene is not automatically a good drug target. The same gene may instead be: - -- a direct therapeutic target; -- an anti-PD-1 combination target; -- a resistance biomarker; -- a mechanistic resistance marker; -- a tumor-intrinsic driver; -- an immune-context signal; -- or a poor direct therapeutic candidate. - -TargetIntel-IO makes these distinctions explicit and preserves the reasoning behind each classification and ranking rather than returning one opaque score. +Anti-PD-1-resistant melanoma is biologically heterogeneous. A relevant gene is not automatically a direct drug target: it may instead support biomarker research, describe resistance context, mark an immune compartment, or be unsuitable for direct intervention. TargetIntel-IO therefore produces separate therapeutic-intent rankings rather than one undifferentiated “best target” list. ## Architecture ```mermaid flowchart TD - OT[Open Targets] --> FT[Deterministic feature table] - CFG[Curated resistance and modality rules] --> FT - FT --> RC[Stable biological-role classifier] - RC --> R1[Antibody / IO ranking] - RC --> R2[Resistance-biomarker ranking] - RC --> R3[Small-molecule ranking] - R1 --> REP[Cards, HTML reports, figures] - R2 --> REP - R3 --> REP - - LIT[Scientific literature] -->|v0.2 complete| EI[Normalized EvidenceItems] - DEP[DepMap / CRISPR] -->|v0.5 complete| FD[Dependency profiles and bounded overlay] --> REP - SC[Single-cell / spatial] -. roadmap .-> EI - CLIN[Clinical cohorts] -. roadmap .-> EI - EI --> VAL[Deterministic validation] - VAL --> DB[(Immutable DuckDB store)] - DB --> EVR[Evidence-aware reports] - DB -. future .-> LLM[Grounded LLM reasoner and critic] - LLM --> EVR - REP --> EVR + OT[Open Targets + curated rules] --> FT[Deterministic feature table] + FT --> RC[Stable role classification] + RC --> TR[Therapeutic-intent rankings] + TR --> OUT[Cards / HTML / figures] + + EV[Reviewed evidence] --> RD[Read-only report decoration] + FE[Target feasibility] --> RD + DP[Validated local DepMap release] --> PS[Portable aggregate snapshot] + PS --> EC[Immutable DependencyReportEvidence] + EC --> RH[Optional Markdown / HTML rendering] + RH --> RD + TR --> RD ``` -### Evidence before interpretation - -The LLM is not intended to be the source of truth. Future model-generated interpretations must be derived only from stored, source-linked evidence. TargetIntel-IO separates: - -1. retrieved or computed observations; -2. system-generated interpretations; -3. target-level recommendations. - -Recommendations must remain traceable to the exact observations, quotations, datasets, cohorts, experiments, and transformations that support them. - -## What the deterministic workflow produces - -For every candidate, the workflow generates: - -- a stable biological and translational role; -- a therapeutic direction; -- matched anti-PD-1 resistance programs; -- modality-fit assessments; -- evidence supporting and arguing against prioritization; -- confidence and uncertainty annotations; -- separate rankings for three therapeutic intents; -- structured Markdown hypothesis cards; -- browsable HTML reports; -- summary figures and rank-shift analyses. - -| Mode | Prioritizes | -|---|---| -| **Antibody / IO combination** | Surface-accessible checkpoints, myeloid targets, suppressive immune axes, and combination rationale | -| **Resistance biomarker** | Antigen-presentation loss, IFNγ resistance, immune exclusion, and patient-stratification potential | -| **Small molecule** | Tumor-intrinsic drivers, kinases, oncogenic pathways, and small-molecule tractability | - -## Evidence-layer example - -```python -EvidenceItem( - evidence_id="ev_b2m_example", - target_symbol="B2M", - disease_name="melanoma", - disease_id="MONDO:0005105", - treatment_name="anti-PD-1", - evidence_type="clinical_cohort", - evidence_direction="supports_biomarker", - observation="Source-grounded observation stored separately from interpretation.", - interpretation=None, - source="Europe PMC", - source_id="PMID:...", - quoted_span="Exact supporting source text.", - patient_cohort_id="cohort_identifier", - species="human", - model_system="patient_tumor_biopsy", - extraction_method="llm", - validation_status="citation_verified", -) -``` +The deterministic baseline is authoritative. Optional layers decorate reports after ranking; they do not modify feature construction, scores, ranks, roles, target selection, or candidate activation. -The evidence layer rejects records that claim verification without the required quotation, support, provenance, identifiers, and validation history. Sharing `source` and `source_id` does not automatically make two observations revisions of one another; revision links are explicit and caller-driven. +## Deterministic baseline -## Human-supervised multi-LLM development +The productive v0.1.3 baseline retrieves and annotates melanoma-associated targets, constructs a deterministic feature table, assigns a stable role, and ranks candidates for antibody/IO combination, resistance biomarker, and small-molecule intents. The original productive baseline remains 300 genes. Baseline scores and ranks remain unchanged by all optional reporting layers. -Recent v0.2.0 work uses a human-supervised multi-agent development workflow: +The internally curated 56-target benchmark measures implementation consistency and rule behavior. It is not an independent biological validation dataset and is not evidence of clinical performance. -```mermaid -flowchart LR - H[Human scientific objective] --> S[Gemini-assisted specification] - S --> I[Codex-assisted implementation] - I --> T[Automated tests and regression gates] - T --> R[Independent Claude review] - R --> A[Adversarial audit] - A --> H2[Human approval and merge] -``` +## Optional evidence, feasibility, and functional dependency -Shared agent instructions require the system to: +The v0.2.0 Common Evidence Layer stores typed, source-linked observations separately from interpretation and provides read-only report decoration. v0.3.0 provides audited extraction, mandatory review, reviewed snapshots, and grounded synthesis infrastructure; v0.2.0 is infrastructure and report decoration, not clinical validation or a production LLM extractor. v0.4.0 feasibility reports modality-specific coverage, missingness, contradictions, tractability, clinical precedence, doability, and safety-data state without changing deterministic prioritization. -- never invent biological evidence, numerical values, references, or API data; -- never present association as proof of causality; -- preserve observation separately from LLM interpretation; -- prevent future LLM components from silently changing baseline rankings; -- protect secrets and identifiable patient-level information; -- report unresolved failures explicitly; -- require human approval before publication or merge. +## v0.5.0 DepMap/CRISPR functional dependency -See [`AGENTS.md`](AGENTS.md) and [`CLAUDE.md`](CLAUDE.md). +v0.5.0 is a research-preview architecture for optional, post-ranking functional-dependency reporting in melanoma anti-PD-1 resistance research. DepMap Public 26Q1 is the pinned functional-dependency release. The curated benchmark contains 56 targets, the discovery universe contains 331 unique identities, and the 18,531-gene DepMap universe is background only. That background is not a productive ranking: no new 18,531-gene productive ranking is generated. -> Agentic AI currently helps engineer and review the platform. A production -> scientific LLM agent remains a roadmap feature. +Functional-dependency evidence can report gene-effect evidence, dependency-probability evidence, context-versus-reference comparisons, selectivity, and bounded integration-rank observations. It is deliberately distinct from clinical anti-PD-1 response evidence. DepMap cell-line dependency is not clinical anti-PD-1 response evidence, and absence of tumor-cell dependency does not invalidate an immune target. + +Implemented components include deterministic local ingestion; immutable file and schema contracts; functional-dependency profiles; benchmark and coverage evaluation; a bounded integration overlay; release closure and reproducibility checks; portable aggregate snapshot export; immutable report evidence contracts; deterministic Markdown and HTML rendering; and regression and isolation gates. Production activation remains disabled, no approved authorization is emitted, and human review remains mandatory. ## Quick start @@ -194,81 +82,47 @@ python -m pip install --upgrade pip python -m pip install -e ".[dev]" ``` -## Run the deterministic workflow +## Main workflow + +The normal deterministic workflow does not require a local DepMap release. ```bash targetintel run targetintel run --validate targetintel run --refresh targetintel run --help +python -m pytest -q ``` -## Main outputs +## Outputs -```text -data/processed/ -└── targetintel_feature_table_v0_1.csv - -results/ -├── ranked_targets.csv -├── target_cards/ -├── html_reports/ -│ └── index.html -├── figures/ -├── benchmark/ -└── sensitivity/ -``` +The productive workflow writes a deterministic feature table, therapeutic-intent ranked targets, Markdown cards, HTML reports and figures. Optional reviewed evidence, feasibility, and functional-dependency sections decorate matching reports only. Portable aggregate DepMap snapshots are a separate research-preview artifact; real 26Q1 repository publication artifacts are pending Issue 512. -Versioned examples: +Versioned examples include [HTML reports](examples/html_reports/), [figures](examples/figures/), the [benchmark snapshot](examples/benchmark/README.md), and [sensitivity outputs](examples/sensitivity/README.md). -- [`examples/html_reports/`](examples/html_reports/) -- [`examples/figures/`](examples/figures/) -- [`examples/benchmark/`](examples/benchmark/README.md) -- [`examples/sensitivity/`](examples/sensitivity/README.md) +## Validation and reproducibility -## How the deterministic baseline works +TargetIntel-IO uses deterministic rule application and tie-breaking, versioned configuration and benchmark material, immutable evidence contracts, and offline regression tests. Run `targetintel run --validate` for the existing workflow validation and `python -m pytest -q` for the test suite. The 56-target benchmark remains an internal consistency check, not independent validation or clinical performance evidence. -1. **Public evidence retrieval:** melanoma-associated targets are retrieved from -the Open Targets GraphQL API and cached locally. -2. **Feature construction:** targets are annotated with disease association, -resistance-axis membership, modality fit, tractability, known drugs, safety, contradictions, completeness, and confidence. -3. **Stable role classification:** each candidate receives one role independent -of ranking mode. -4. **Therapeutic-intent scoring:** candidates are scored separately for -antibody/IO, biomarker, and small-molecule use. -5. **Human-readable outputs:** rankings are converted into cards, reports, -figures, benchmark summaries, and machine-readable validation outputs. + +## Validation and benchmark boundaries -```text -therapeutic target ≠ biomarker ≠ resistance mechanism ≠ contextual marker -``` +### Internal benchmark snapshot -## Internal benchmark snapshot +The current 56-target benchmark is an internal implementation benchmark: -TargetIntel-IO includes a curated 56-target benchmark for internal rule-based sanity validation. +- Open Targets retrieval coverage is **25/56 (44.6%)**. +- The covered benchmark produced **100.0% stable-role accuracy**. +- Strict primary-intent accuracy is **91.1%**. +- Acceptable-intent accuracy is **100.0%**. -| Metric | Result | -|---|---:| -| Benchmark targets evaluated | 56 / 56 | -| TargetIntel evaluation coverage | 100% | -| Open Targets top-300 retrieval coverage | 44.6% | -| Stable-role accuracy | 100.0% | -| Strict primary-intent accuracy | 91.1% | -| Acceptable-intent accuracy | 100.0% | -| Cross-intent specificity | 90.6% | -| Control not-prioritized rate | 100.0% | -| Mean top-10 recall | 58.1% | -| Mean top-20 recall | 79.5% | +These results measure implementation consistency, not independent biological accuracy. +Complete TargetIntel evaluation coverage does not mean that Open Targets independently recovered every target. +The benchmark is internally curated rather than derived from an independent, blinded, prospective, or clinical validation dataset. -Only **25/56 (44.6%)** benchmark targets appeared among the top 300 melanoma associations retrieved from Open Targets. TargetIntel evaluation coverage therefore does not mean that Open Targets independently recovered every target. +### Sensitivity snapshot -The benchmark produced **100.0% stable-role accuracy**, **91.1% strict primary-intent accuracy**, and **100.0%** acceptable-intent accuracy. Expected roles and acceptable alternatives were internally curated rather than derived from an independent benchmark. These results measure implementation consistency, not independent biological accuracy. - -No external patient-level responder/non-responder cohort was used for this internal benchmark. The complete results are available in the [versioned benchmark snapshot](examples/benchmark/README.md). - -## Weight sensitivity - -The local analysis evaluates **42 scenarios**, changing one scoring weight by `-20%` or `+20%` before renormalization. +The current local sensitivity snapshot evaluates **42 scenarios** in which individual scoring weights are perturbed and renormalized. Worst-case top-5 retention was: **antibody/IO 100%, biomarker 100%, small-molecule 80%**. @@ -276,84 +130,64 @@ Worst-case top-10 retention was: **antibody/IO 90%, biomarker 100%, small-molecu Worst-case top-20 retention was: **antibody/IO 100%, biomarker 95%, small-molecule 100%**. -The minimum observed Spearman correlation was **0.8762**. The maximum absolute change in strict primary-intent accuracy was **5.36 percentage points**; the maximum acceptable-intent change was **3.57 percentage points**; and the maximum cross-intent-specificity change was **5.66 percentage points**. +The minimum observed Spearman rank correlation was **0.8762**. -![Worst-case ranking stability](examples/sensitivity/sensitivity_overview.png) +The maximum absolute primary-intent accuracy change was **5.36 percentage points**. -This is a local stability analysis. It does not prove that the selected weights are biologically optimal or that the rankings are independent of weight choice. +The maximum absolute acceptable-intent accuracy change was **3.57 percentage points**. -## Reproducibility and software quality +The maximum absolute cross-intent-specificity change was **5.66 percentage points**. -The project includes: +These observations describe local robustness around the configured deterministic rules. They do not establish that the rankings are independent of weight selection or biologically validated. +### External-validation boundary -- a reusable Python package and command-line interface; -- compatible dependency ranges in `pyproject.toml`; -- a Conda environment definition; -- an exact Python 3.11 lockfile with package hashes; -- deterministic ranking and tie-breaking; -- versioned benchmark, sensitivity, and DepMap release-closure evidence; -- GitHub Actions continuous integration; -- offline unit and regression tests; -- immutable evidence storage and Parquet verification; -- scientific and AI-agent safety instructions. +No external patient-level responder/non-responder cohort is currently used to validate the target rankings. -Install the exact locked environment used by CI: +Public associations, internal benchmark agreement, sensitivity analysis, feasibility annotations, reviewed evidence, and DepMap functional-dependency observations do not establish causality, clinical utility, therapeutic efficacy, or response prediction. All generated hypotheses require independent experimental, translational, and clinical validation. -```bash -python -m pip install \ - --require-hashes \ - --requirement requirements-lock.txt - -python -m pip install \ - --no-deps \ - --no-build-isolation \ - --editable . -``` +### Grounded evidence and human review -Run tests: +The optional grounded-evidence workflow is documented in [`examples/llm/README.md`](examples/llm/README.md), with release boundaries in [`docs/releases/v0.3.0.md`](docs/releases/v0.3.0.md). -```bash -python -m pytest tests -q -``` +This layer requires mandatory human review and does not alter deterministic scores, rankings, or role classification. + +## Scientific limitations + +TargetIntel-IO does not make treatment recommendations, validate targets or biomarkers, establish causality, or predict patient response. Missing evidence is not negative evidence. DepMap cell-line profiles do not reproduce the complete tumor microenvironment, and broad dependency can reflect general essentiality. Optional DepMap infrastructure is implemented, while real 26Q1 snapshot publication remains pending Issue 512. Future research directions include single-cell/spatial integration, clinical-response research models, and knowledge-graph expansion. ## Repository map ```text -configs/ Disease context, resistance axes, benchmark, scoring -targetintel/ Reusable Python package and command-line workflow -targetintel/evidence/ Typed evidence contracts, validation, immutable storage -scripts/ Pipeline and snapshot-management commands -tests/ Unit, integration, and regression tests -examples/ Versioned reports, figures, benchmark, sensitivity -docs/ Architecture, roadmap, specifications, release evidence -data/ Local cached and processed data; not versioned -results/ Generated local outputs; not versioned +targetintel/ Reusable package and CLI +targetintel/functional_dependency/ Optional portable reporting architecture +configs/ Disease, resistance, benchmark and scoring rules +tests/ Unit, integration, regression and isolation tests +examples/ Versioned example outputs +docs/ Specifications, roadmap and release evidence +data/ and results/ Local caches and generated outputs; normally uncommitted ``` -## Scope and limitations - -TargetIntel-IO is a hypothesis-generation and target-triage framework. It does not provide clinical recommendations, validated therapeutic targets, qualified biomarkers, causal biological proof, a diagnostic system, patient-level treatment predictions, or medical advice. +## Roadmap -The current deterministic implementation focuses on anti-PD-1-resistant melanoma. v0.2.0 evidence reporting remains optional and read-only. v0.5.0 adds reproducible DepMap/CRISPR dependency profiling as an explanatory post-ranking layer: the production baseline remains preserved and candidate activation requires separate human review. Production LLM extraction, single-cell/spatial integration, patient-response modelling, and knowledge-graph inference remain future work. - -All generated hypotheses require independent experimental, translational, and clinical validation. - -## Data governance - -The current workflow uses public data and curated public-domain biological knowledge, principally Open Targets Platform and DepMap Public releases. No confidential, proprietary, company-internal, or identifiable patient data is included. Source matrices, generated databases, local caches, and complete run directories remain outside version control; only portable, sanitized release evidence and checksums are committed. +- v0.2.0 Common Evidence Layer +- v0.3.0 Grounded Literature Copilot and provider-agnostic LLM integration +- v0.4.0 Target feasibility and expanded Open Targets integration +- v0.5.0 DepMap/CRISPR functional dependency +- v0.6.0 Single-cell and spatial context +- v0.7.0 Clinical-response research model +- v0.8.0 De novo target discovery and knowledge graph +- v1.0.0 Multitumor target-intelligence platform ## Citation + ```text Soler Ortuño R. TargetIntel-IO: Explainable therapeutic-intent-aware target intelligence for anti-PD-1-resistant melanoma. ``` ## Author -**Rafael Soler Ortuño, PhD** - -Computational biologist working across immuno-oncology, biomarker discovery, patient stratification, multi-omics, single-cell and spatial transcriptomics, scientific software engineering, and AI-assisted drug discovery. -[LinkedIn](https://www.linkedin.com/in/rafael-soler-ortuno/) +Rafael Soler Ortuño, PhD ## License diff --git a/tests/test_depmap_readme_documentation.py b/tests/test_depmap_readme_documentation.py new file mode 100644 index 0000000..8246673 --- /dev/null +++ b/tests/test_depmap_readme_documentation.py @@ -0,0 +1,21 @@ +"""Focused scientific-boundary checks for the repository README.""" +from pathlib import Path +import re + + +def test_readme_describes_current_v050_boundary_without_publication_claims() -> None: + text = (Path(__file__).parents[1] / "README.md").read_text(encoding="utf-8") + required = ("v0.5.0", "DepMap Public 26Q1", "300 genes", "56-target", "331 unique identities", + "18,531-gene", "not a productive ranking", "Baseline scores and ranks remain unchanged", + "optional, post-ranking", "Production activation remains disabled", "human review remains mandatory", + "DepMap cell-line dependency is not clinical anti-PD-1 response evidence", + "absence of tumor-cell dependency does not invalidate an immune target", "targetintel run", + "targetintel run --validate", "python -m pytest -q", "```mermaid", "## Citation", "## Author", "## License") + for item in required: + assert item.lower() in text.lower() + assert "DepMap integration as future work" not in text + assert "real DepMap Public 26Q1 repository snapshot publication is pending Issue 512" in text + assert not re.search(r"(?:26Q1|snapshot).{0,50}(?:already )?(?:published|downloadable|released)", text, re.I) + assert "/home/" not in text and "/media/" not in text + for unsupported in ("treatment recommendations", "validated therapeutic targets", "qualified biomarkers", "clinical-response predictions"): + assert unsupported in text diff --git a/tests/test_depmap_reporting_isolation.py b/tests/test_depmap_reporting_isolation.py new file mode 100644 index 0000000..3aeb839 --- /dev/null +++ b/tests/test_depmap_reporting_isolation.py @@ -0,0 +1,132 @@ +"""Isolation locks: constructed dependency evidence is presentation-only.""" +from __future__ import annotations + +import ast +import builtins +import io +from pathlib import Path +import socket +import urllib.request + +import pandas as pd +import requests + +import targetintel.pipeline as pipeline +from targetintel.functional_dependency.presentation import render_dependency_html, render_dependency_markdown +from targetintel.html_reports import make_target_html_report, write_top_html_reports +from targetintel.hypothesis_cards import make_target_card, write_top_target_cards +from test_depmap_reporting_regression import _evidence, _row + + +def _forbidden(*_args, **_kwargs): + raise AssertionError("upstream, file, environment, or network access invoked") + + +def _guard_network(monkeypatch) -> None: + """Fail on each common transport surface, not only socket creation.""" + monkeypatch.setattr(socket, "socket", _forbidden) + monkeypatch.setattr(urllib.request, "urlopen", _forbidden) + monkeypatch.setattr(requests, "request", _forbidden) + monkeypatch.setattr(requests, "get", _forbidden) + monkeypatch.setattr(requests, "post", _forbidden) + monkeypatch.setattr(requests, "Session", _forbidden) + + +def test_pure_renderers_are_in_memory_and_do_not_import_execution_or_snapshot_layers(monkeypatch) -> None: + import os + root = Path(__file__).parents[1] / "targetintel" / "functional_dependency" + source_by_name = { + name: (root / name).read_text(encoding="utf-8") + for name in ("report_contract.py", "report_snapshot.py", "presentation.py") + } + monkeypatch.setattr(builtins, "open", _forbidden) + monkeypatch.setattr(io, "open", _forbidden) + monkeypatch.setattr(Path, "open", _forbidden) + monkeypatch.setattr(os, "getenv", _forbidden) + _guard_network(monkeypatch) + evidence = _evidence() + assert "Functional dependency" in render_dependency_markdown(evidence) + assert "functional-dependency" in render_dependency_html(evidence) + forbidden_imports = { + "report_snapshot", "hypothesis_cards", "html_reports", "pipeline", "ingestion", "profiles", + "benchmark", "integration", + } + for name, source in source_by_name.items(): + tree = ast.parse(source) + imports = { + alias.name.split(".")[-1] + for node in ast.walk(tree) + if isinstance(node, ast.ImportFrom) and node.module + for alias in node.names + } + assert not imports & forbidden_imports + assert "dependency_profiles.jsonl" not in source_by_name["presentation.py"] + assert "CRISPRGeneEffect" not in source_by_name["presentation.py"] + assert "CRISPRGeneDependency" not in source_by_name["presentation.py"] + + +def test_card_and_html_rendering_survive_forbidden_upstream_systems(monkeypatch) -> None: + import targetintel.feature_table as feature_table + import targetintel.intent_ranking as ranking + import targetintel.role_classifier as roles + import targetintel.scoring as scoring + import targetintel.functional_dependency.depmap_ingestion as ingestion + import targetintel.functional_dependency.depmap_profiles as profiles + for module, name in ((feature_table, "build_feature_table"), (ranking, "build_intent_rankings"), + (roles, "classify_gene"), (scoring, "score_all_profiles"), + (ingestion, "ingest_local_release"), (profiles, "build_dependency_profiles")): + monkeypatch.setattr(module, name, _forbidden) + _guard_network(monkeypatch) + assert "Functional dependency" in make_target_card(_row(), dependency_evidence=_evidence()) + assert "functional-dependency" in make_target_html_report(_row(), dependency_evidence=_evidence()) + + +def test_rendering_cannot_open_depmap_matrices_or_profile_artifacts(monkeypatch) -> None: + """Synthetic report evidence must make all matrix and JSONL I/O unnecessary.""" + import os + monkeypatch.setattr(builtins, "open", _forbidden) + monkeypatch.setattr(io, "open", _forbidden) + monkeypatch.setattr(Path, "open", _forbidden) + monkeypatch.setattr(os, "getenv", _forbidden) + _guard_network(monkeypatch) + + evidence = _evidence() + assert "Functional dependency" in render_dependency_markdown(evidence) + assert "functional-dependency" in render_dependency_html(evidence) + assert "Functional dependency" in make_target_card(_row(), dependency_evidence=evidence) + assert "functional-dependency" in make_target_html_report(_row(), dependency_evidence=evidence) + + +def test_writers_are_confined_to_explicit_tmp_destination_and_leave_inputs_unchanged(tmp_path: Path) -> None: + frame = pd.DataFrame([_row("BRAF", 1), _row("NRAS", 2)]) + before = frame.copy(deep=True) + mapping = {"BRAF": _evidence()} + cards = write_top_target_cards(frame, tmp_path / "cards", top_n_per_mode=2, dependency_evidence_by_symbol=mapping) + html = write_top_html_reports(frame, tmp_path / "html", top_n_per_mode=2, dependency_evidence_by_symbol=mapping) + assert set(tmp_path.rglob("*")) == set(cards) | set(html) | {tmp_path / "cards", tmp_path / "html"} + assert all(path.is_relative_to(tmp_path) for path in cards + html) + pd.testing.assert_frame_equal(frame, before) + assert mapping == {"BRAF": _evidence()} + + +def test_productive_pipeline_is_unwired_from_dependency_evidence(monkeypatch, tmp_path: Path) -> None: + frame = pd.DataFrame([_row("BRAF", 1)]) + calls: list[tuple[str, object]] = [] + monkeypatch.setattr(pipeline, "build_feature_table", lambda **_k: frame.copy()) + monkeypatch.setattr(pipeline, "save_feature_table", lambda _df, path: path) + monkeypatch.setattr(pipeline, "build_intent_rankings", lambda df: df.copy()) + monkeypatch.setattr(pipeline, "save_ranked_targets", lambda _df, path: path) + def writer(name): + def call(df, **kwargs): + calls.append((name, (df.copy(), kwargs))) + return [] + return call + monkeypatch.setattr(pipeline, "write_top_target_cards", writer("cards")) + monkeypatch.setattr(pipeline, "write_top_html_reports", writer("html")) + monkeypatch.setattr(pipeline, "generate_summary_figures", lambda *_a, **_k: []) + outputs = pipeline.run_core_pipeline(project_root=tmp_path, top_n_per_mode=1) + assert outputs.target_cards_dir == tmp_path / "results" / "target_cards" + assert [name for name, _ in calls] == ["cards", "html"] + for _, (passed, kwargs) in calls: + pd.testing.assert_frame_equal(passed, frame) + assert "dependency_evidence_by_symbol" not in kwargs diff --git a/tests/test_depmap_reporting_regression.py b/tests/test_depmap_reporting_regression.py new file mode 100644 index 0000000..bf47485 --- /dev/null +++ b/tests/test_depmap_reporting_regression.py @@ -0,0 +1,164 @@ +"""Cross-module compatibility locks for optional DepMap report decoration.""" +from __future__ import annotations + +from pathlib import Path + +import pandas as pd + +from targetintel.evidence.reporting import EvidenceReportDecorator +from targetintel.functional_dependency.report_contract import DependencyReportEvidence +from targetintel.html_reports import make_target_html_report, write_top_html_reports +from targetintel.hypothesis_cards import make_target_card, write_top_target_cards +from test_feasibility_presentation import _annotation, _observation +from tests.test_evidence_models import evidence_item + + +def _evidence(symbol: str = "BRAF", **changes: object) -> DependencyReportEvidence: + values: dict[str, object] = { + "format_version": "v1", "release_identifier": "DepMap_Public_26Q1", + "release_manifest_id": "manifest", "configuration_id": "configuration", + "scientific_closure_identity": "closure", "context_identity": "melanoma_anti_pd1:v1", + "gene_symbol": symbol, "canonical_gene_identity": f"{symbol}:1", "profile_available": True, + "coverage_status": "sufficient_complete_coverage", "model_count": 10, + "context_model_count": 4, "reference_model_count": 6, + "available_context_observations": 4, "available_reference_observations": 6, + "coverage_fraction": 1.0, "missing_value_state": "resolved", "unavailable_reason": None, + "gene_effect": {"median": 0.0}, "dependency_probability": {"median": None}, + "context_reference_comparison": {"median_delta": -0.2}, "selectivity": {"value": 50.0}, + "dependency_interpretation_state": "valid", "baseline_rank": 7, + "dependency_aware_candidate_rank": 5, "rank_delta": -2, + "integration_state": "human_review_required", "baseline_preserved": True, + "production_activation_enabled": False, "approved_authorization_emitted": False, + "candidate_activation_readiness": "blocked", "human_review_required": True, + "limitations": ("A local limitation.",), + "provenance": {"source_artifact_names": ["a.json", "b.tsv"]}, + } + values.update(changes) + return DependencyReportEvidence.create(**values) + + +def _row(symbol: str = "BRAF", rank: int = 1) -> pd.Series: + return pd.Series({"target_symbol": symbol, "target_name": f"{symbol} name", "opentargets_score": 0.7, + "opentargets_rank": rank, "role_classification": "tumor-intrinsic driver", + "antibody_io_rank": rank, "biomarker_rank": rank, "small_molecule_rank": rank}) + + +def _ranked() -> pd.DataFrame: + return pd.DataFrame([_row("BRAF", 1), _row("NRAS", 2)]) + + +def _stored_evidence_card(): + item = evidence_item(evidence_id="stored", validation_status="citation_verified", + evidence_family="efam-v1:stored", evidence_family_basis="publication_id", + independence_eligible=True, independence_ineligibility_reason=None, + publication_id="PMID:1", experiment_id=None, patient_cohort_id=None) + return EvidenceReportDecorator().make_card("BRAF", [item]) + + +def _bytes(paths: list[Path]) -> dict[str, bytes]: + return {path.name: path.read_bytes() for path in paths} + + +def test_legacy_cards_and_html_are_byte_identical_without_dependency_evidence() -> None: + row = _row() + card = make_target_card(row) + html = make_target_html_report(row) + assert card == make_target_card(row, dependency_evidence=None) + assert html == make_target_html_report(row, dependency_evidence=None) + for rendered in (card, html): + assert "Functional dependency" not in rendered + for identifier in ("DepMap", "manifest", "configuration", "closure"): + assert identifier not in rendered + + annotation = (_annotation("antibody", (_observation("tractability", "antibody"),)),) + evidence_card = _stored_evidence_card() + assert make_target_card(row, feasibility_annotations=annotation, feasibility_target_identifier_type="gene_symbol") == make_target_card(row, feasibility_annotations=annotation, feasibility_target_identifier_type="gene_symbol", dependency_evidence=None) + assert make_target_html_report(row, evidence_card=evidence_card) == make_target_html_report(row, evidence_card=evidence_card, dependency_evidence=None) + assert make_target_card(row, evidence_card=evidence_card, feasibility_annotations=annotation, feasibility_target_identifier_type="gene_symbol") == make_target_card(row, evidence_card=evidence_card, feasibility_annotations=annotation, feasibility_target_identifier_type="gene_symbol", dependency_evidence=None) + assert make_target_html_report(row, evidence_card=evidence_card, feasibility_annotations=annotation, feasibility_target_identifier_type="gene_symbol") == make_target_html_report(row, evidence_card=evidence_card, feasibility_annotations=annotation, feasibility_target_identifier_type="gene_symbol", dependency_evidence=None) + + +def test_batch_writers_preserve_legacy_bytes_order_rows_and_mapping_inputs(tmp_path: Path) -> None: + ranked = _ranked() + before = ranked.copy(deep=True) + mapping = {"BRAF": _evidence(), "unused": _evidence("unused")} + mapping_before = {key: value.canonical_json() for key, value in mapping.items()} + legacy_cards = write_top_target_cards(ranked, tmp_path / "legacy_cards", top_n_per_mode=2) + none_cards = write_top_target_cards(ranked, tmp_path / "none_cards", top_n_per_mode=2, dependency_evidence_by_symbol=None) + empty_cards = write_top_target_cards(ranked, tmp_path / "empty_cards", top_n_per_mode=2, dependency_evidence_by_symbol={}) + legacy_html = write_top_html_reports(ranked, tmp_path / "legacy_html", top_n_per_mode=2) + none_html = write_top_html_reports(ranked, tmp_path / "none_html", top_n_per_mode=2, dependency_evidence_by_symbol=None) + empty_html = write_top_html_reports(ranked, tmp_path / "empty_html", top_n_per_mode=2, dependency_evidence_by_symbol={}) + assert [p.name for p in legacy_cards] == [p.name for p in none_cards] == [p.name for p in empty_cards] + assert _bytes(legacy_cards) == _bytes(none_cards) == _bytes(empty_cards) + assert [p.name for p in legacy_html] == [p.name for p in none_html] == [p.name for p in empty_html] + assert _bytes(legacy_html) == _bytes(none_html) == _bytes(empty_html) + + decorated_cards = write_top_target_cards(ranked, tmp_path / "decorated_cards", top_n_per_mode=2, dependency_evidence_by_symbol=mapping) + decorated_html = write_top_html_reports(ranked, tmp_path / "decorated_html", top_n_per_mode=2, dependency_evidence_by_symbol=mapping) + assert "Functional dependency" in (tmp_path / "decorated_cards" / "BRAF.md").read_text() + assert (tmp_path / "decorated_cards" / "NRAS.md").read_bytes() == (tmp_path / "legacy_cards" / "NRAS.md").read_bytes() + assert "Functional dependency" in (tmp_path / "decorated_html" / "BRAF.html").read_text() + assert (tmp_path / "decorated_html" / "NRAS.html").read_bytes() == (tmp_path / "legacy_html" / "NRAS.html").read_bytes() + assert (tmp_path / "decorated_html" / "index.html").read_bytes() == (tmp_path / "legacy_html" / "index.html").read_bytes() + assert [p.name for p in decorated_cards] == [p.name for p in legacy_cards] + assert [p.name for p in decorated_html] == [p.name for p in legacy_html] + pd.testing.assert_frame_equal(ranked, before) + assert {key: value.canonical_json() for key, value in mapping.items()} == mapping_before + assert list(ranked["target_symbol"]) == ["BRAF", "NRAS"] + assert set(ranked.columns) == set(before.columns) # rendering introduces no score field + for column in ("opentargets_score", "opentargets_rank", "antibody_io_rank", "biomarker_rank", "small_molecule_rank", "role_classification"): + assert ranked[column].equals(before[column]) + + +def test_all_decorations_have_deterministic_current_composition_order() -> None: + row = _row() + annotation = (_annotation("antibody", (_observation("tractability", "antibody"),)),) + evidence_card = _stored_evidence_card() + kwargs = {"evidence_card": evidence_card, "feasibility_annotations": annotation, + "feasibility_target_identifier_type": "gene_symbol", "dependency_evidence": _evidence()} + for rendered in (make_target_card(row, **kwargs), make_target_html_report(row, **kwargs)): + headings = ("Stored evidence observations", "Target feasibility", "Functional dependency") + # The feasibility HTML intentionally embeds its Markdown detail in a + # ``pre`` element, so count its semantic section rather than its title + # text; each optional decoration itself is emitted once. + if rendered.startswith(""): + assert rendered.count('

Target feasibility') == 1 + assert rendered.count('
') == 1 + assert rendered.count('

Stored evidence observations

') == 1 + else: + assert all(rendered.count(heading) == 1 for heading in headings) + assert rendered.index(headings[0]) < rendered.index(headings[1]) < rendered.index(headings[2]) + dependency_section = rendered[rendered.index("Functional dependency"):] + for claim in ("clinical recommendation", "target validation", "biomarker validation", "response prediction"): + assert claim not in dependency_section.lower() + + +def test_available_and_unavailable_evidence_remain_calibrated_and_deterministic() -> None: + row = _row() + available = _evidence() + unavailable = _evidence(profile_available=False, coverage_status="not_available", canonical_gene_identity=None, + model_count=None, context_model_count=None, reference_model_count=None, + available_context_observations=None, available_reference_observations=None, + coverage_fraction=None, missing_value_state=None, unavailable_reason="unresolved", + gene_effect=None, dependency_probability=None, context_reference_comparison=None, + selectivity=None, dependency_interpretation_state=None) + rendered = make_target_card(row, dependency_evidence=available) + absent = make_target_card(row, dependency_evidence=unavailable) + assert "**Gene-effect summary:** {\"median\":0.0}" in rendered + assert "not reported" in rendered and "0.0" in rendered + assert "No dependency conclusion is drawn." in absent + assert "not available" in absent and "Gene-effect summary" not in absent + assert "non-dependent" not in absent.lower() + assert make_target_card(row, dependency_evidence=available) == rendered + assert make_target_html_report(row, dependency_evidence=available) == make_target_html_report(row, dependency_evidence=available) + escaped = _evidence(gene_effect={"text": "&\""}) + escaped_html = make_target_html_report(row, dependency_evidence=escaped) + assert "&lt;tag&gt;" in escaped_html and "" not in escaped_html + reordered = {"other": _evidence("other"), "BRAF": available} + assert make_target_card(row, dependency_evidence=reordered["BRAF"]) == rendered + for forbidden in ("timestamp", "hostname", "/tmp/", "/home/", "mount"): + assert forbidden not in rendered.lower() + assert rendered.index("- A local limitation.") < rendered.index("- Absence of tumor-cell dependency") + for marker in ("Baseline preserved:** yes", "Production activation enabled:** disabled", "Approved authorization emitted:** not emitted", "Human review required:** required"): + assert marker in rendered