Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions research-gap-replication-planner/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Research Gap Replication Planner

Self-contained contribution for `SCIBASE-AI/SCIBASE.AI#16`, focused on the boundary between the AI Research Assistant Suite's peer-review, reproducibility, and research-gap capabilities.

This slice is intentionally not another broad assistant demo. It builds a deterministic planner that turns corpus limitations, prior reproducibility attempts, manuscript evidence coverage, and lab capabilities into reviewer-ready replication priorities.

## What It Does

- Generates auto peer-review flags for claim/evidence/citation alignment, statistical risk, and domain-specific omissions.
- Scores project reproducibility readiness from raw data, clean pipelines, environment locks, tests, reported outputs, and previous run attempts.
- Finds under-replicated research intersections with unresolved findings, negative-result signals, and lab-fit evidence.
- Produces a prioritized replication plan with required artifacts, first actions, and a stable audit digest.
- Uses only local synthetic data and Node.js standard library APIs.

## Requirement Map

| Issue #16 requirement | Implementation evidence |
| --- | --- |
| Auto peer review reports | `reviewManuscript` emits structured review findings with severity, category, claim id, and domain-specific template checks. |
| Claims vs. evidence alignment | Claims without evidence, citations, or registered support are flagged before release. |
| Statistical or methodological red flags | Small-sample p-value edge cases, missing controls, and missing domain evidence are detected. |
| Reproducibility checker | `scoreReproducibility` checks required artifacts, prior attempts, environment integrity, and output consistency. |
| Reproducibility confidence score | Each project receives a numeric confidence score plus pass/hold status and remediation actions. |
| Links to previous reproducibility attempts | Prior attempt ids and outcomes are included in each report and used in scoring. |
| Research gap finder | `rankResearchOpportunities` ranks under-replicated topic intersections using unresolved findings, negative signals, replication counts, and corpus activity. |
| User opportunity feed | Lab capabilities and researcher interests shape the replication-priority output. |

See `docs/requirement-map.md` for the detailed checklist.

## Demo

```bash
cd research-gap-replication-planner
npm run demo
```

Expected summary:

```text
Planner status: hold
Top opportunity: alzheimer + crispr + single-cell
Top replication decision: prioritize
Top peer-review flag: Claim has no supporting evidence artifact.
Weak reproducibility project: circadian-metabolomics-2026
Audit hash: de1b1fa44644dc24...
```

Visual demo assets:

- `docs/demo.svg`
- `docs/demo.mp4`

## Verification

```bash
cd research-gap-replication-planner
npm run check
```

Local verification covers:

- opportunity ranking
- peer-review flag generation
- reproducibility scoring
- deterministic audit hash generation
- CLI demo output

## AI-Assisted Disclosure

This contribution was produced with AI assistance and reviewed before submission. It uses no credentials, no live user data, and no external services.
Binary file added research-gap-replication-planner/docs/demo.mp4
Binary file not shown.
16 changes: 16 additions & 0 deletions research-gap-replication-planner/docs/demo.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
13 changes: 13 additions & 0 deletions research-gap-replication-planner/docs/requirement-map.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Requirement Map

| Requirement | Evidence in this module | Verification |
| --- | --- | --- |
| Natural-language review suggestions | `src/planner.js` creates reviewer action text for unsupported claims, missing citations, small samples, and domain-specific omissions. | `test/planner.test.js` checks the highest severity review finding. |
| Clarity and coherence checks | Review packets flag overbroad claims that lack an evidence artifact or citation trail. | `npm test` validates unsupported-claim detection. |
| Statistical or methodological red flags | Small sample sizes near significance thresholds and missing domain controls are flagged. | `npm test` checks statistical risk categories. |
| Claims vs. evidence alignment | Every claim is checked for evidence ids, citation ids, protocol linkage, and method support. | `npm test` checks claim alignment output. |
| Reproducibility confidence | Projects are scored from raw data, clean pipeline, environment lock, tests, reported output, and prior attempts. | `npm test` verifies low-confidence and high-confidence cases. |
| Previous reproducibility attempts | Attempt records are attached to project reports and affect the score. | `npm test` verifies failed attempts reduce confidence. |
| Research gap finder | Corpus findings are grouped by topic intersection and scored for unresolved status, replication count, corpus activity, lab fit, and feasibility. | `npm test` verifies the top ranked topic intersection. |
| Personalized opportunity feed | Lab capabilities and researcher interests are used to shape priority and first actions. | `npm run demo` prints the ranked plan and audit hash. |
| Reviewer-ready evidence | Output includes required artifacts, first actions, gap rationale, and audit digest. | `npm run check` runs tests plus demo. |
12 changes: 12 additions & 0 deletions research-gap-replication-planner/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"name": "research-gap-replication-planner",
"version": "0.1.0",
"description": "Deterministic research gap and replication-priority planner for SCIBASE issue #16.",
"type": "module",
"scripts": {
"demo": "node scripts/demo.js",
"test": "node test/planner.test.js",
"check": "npm test && npm run demo"
},
"license": "MIT"
}
109 changes: 109 additions & 0 deletions research-gap-replication-planner/sample/corpus.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
{
"lab": {
"name": "Neurogenomics Replication Lab",
"interests": ["Alzheimer", "CRISPR", "single-cell", "metabolomics"],
"capabilities": ["single-cell", "RNA-seq", "CRISPR perturbation", "metabolomics"],
"methods": ["notebook rerun", "containerized workflow"]
},
"manuscripts": [
{
"id": "ms-neuro-042",
"title": "Perturbation response in single-cell Alzheimer models",
"domain": "molecular-biology",
"claims": [
{
"id": "c1",
"text": "CRISPR perturbation improves cell viability across all donor lines.",
"sampleSize": 18,
"pValue": 0.051,
"evidenceIds": [],
"citationIds": ["doi:10.1000/example-a"]
},
{
"id": "c2",
"text": "A subtype-specific response appears in microglia-like cells.",
"sampleSize": 64,
"pValue": 0.02,
"evidenceIds": ["notebook:microglia-response"],
"citationIds": []
}
]
}
],
"projects": [
{
"id": "circadian-metabolomics-2026",
"title": "Circadian metabolomics benchmark",
"dependencyIntegrity": "stale",
"outputConsistency": "drifted",
"artifacts": {
"rawData": true,
"cleanPipeline": true,
"environmentLock": false,
"testSet": false,
"reportedResults": true
}
},
{
"id": "microglia-rnaseq-2026",
"title": "Microglia RNA-seq replication packet",
"dependencyIntegrity": "locked",
"outputConsistency": "consistent",
"artifacts": {
"rawData": true,
"cleanPipeline": true,
"environmentLock": true,
"testSet": true,
"reportedResults": true
}
}
],
"reproducibilityAttempts": [
{
"id": "run-031",
"projectId": "circadian-metabolomics-2026",
"topicId": "finding-crispr-ad",
"outcome": "failed",
"note": "Pipeline required an unavailable R package version."
},
{
"id": "run-044",
"projectId": "microglia-rnaseq-2026",
"topicId": "finding-crispr-ad",
"outcome": "passed",
"note": "Notebook reran with matching summary metrics."
}
],
"corpusFindings": [
{
"id": "finding-crispr-ad",
"title": "CRISPR perturbation in Alzheimer's single-cell models",
"topics": ["CRISPR", "Alzheimer", "single-cell"],
"methods": ["RNA-seq", "CRISPR perturbation"],
"citations": 126,
"activeProjects": 4,
"replicationCount": 1,
"limitation": "Unresolved donor variability and low replication across single-cell CRISPR Alzheimer studies."
},
{
"id": "finding-sleep-metabolomics",
"title": "Sleep disruption metabolomics in aging cohorts",
"topics": ["sleep", "metabolomics", "aging"],
"methods": ["metabolomics"],
"citations": 38,
"activeProjects": 2,
"replicationCount": 3,
"limitation": "Negative results were rarely reported and methods varied across cohorts."
},
{
"id": "finding-protein-folding",
"title": "Protein folding benchmark coverage",
"topics": ["protein folding", "benchmark"],
"methods": ["simulation"],
"citations": 14,
"activeProjects": 1,
"replicationCount": 5,
"limitation": "Benchmark appears stable with several successful replications."
}
]
}
21 changes: 21 additions & 0 deletions research-gap-replication-planner/scripts/demo.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import fs from "node:fs"
import path from "node:path"
import { fileURLToPath } from "node:url"
import { createResearchAssistantPlan } from "../src/planner.js"

const __dirname = path.dirname(fileURLToPath(import.meta.url))
const samplePath = path.join(__dirname, "..", "sample", "corpus.json")
const sample = JSON.parse(fs.readFileSync(samplePath, "utf8"))
const plan = createResearchAssistantPlan(sample)

const topOpportunity = plan.opportunityFeed[0]
const topPlan = plan.replicationPlans[0]
const topFinding = plan.peerReviewReports[0].findings[0]
const weakProject = plan.reproducibilityReports.find((report) => report.status !== "ready")

console.log(`Planner status: ${plan.status}`)
console.log(`Top opportunity: ${topOpportunity.topic}`)
console.log(`Top replication decision: ${topPlan.decision}`)
console.log(`Top peer-review flag: ${topFinding.message}`)
console.log(`Weak reproducibility project: ${weakProject.projectId}`)
console.log(`Audit hash: ${plan.auditHash.slice(0, 16)}...`)
Loading