ErenAri · Sep 18, 2026 · Sep 18, 2026 · Sep 18, 2026 · Sep 18, 2026 · Sep 18, 2026
@@ -11,7 +11,11 @@ on:
       - "scripts/research/verify-profile-lock-v1.sh"
       - "scripts/research/verify-study-matrix-v1.sh"
       - "scripts/research/test-profile-lock-v1.sh"
+      - "scripts/research/analyze-study-v1.py"
+      - "scripts/research/test-analysis-v1.sh"
       - "research/corpus/v1/**"
+      - "research/data/v1/**"
+      - "research/analysis/v1/**"
 
 permissions:
   contents: read
@@ -48,7 +52,9 @@ jobs:
           bash -n scripts/research/test-normalize-study-v1.sh
           bash -n scripts/research/verify-study-matrix-v1.sh
           bash -n scripts/research/test-profile-lock-v1.sh
+          bash -n scripts/research/test-analysis-v1.sh
           python3 -m py_compile scripts/research/normalize-study-v1.py
+          python3 -m py_compile scripts/research/analyze-study-v1.py
           jq -e . research/corpus/v1/study-plan.json >/dev/null
           jq -e . research/corpus/v1/profile-identities.json >/dev/null
           jq -e . research/corpus/v1/materialized-identities.json >/dev/null
@@ -65,6 +71,9 @@ jobs:
       - name: Normalization regressions
         run: bash scripts/research/test-normalize-study-v1.sh
 
+      - name: Research dataset and analysis reproducibility
+        run: bash scripts/research/test-analysis-v1.sh
+
   execute:
     name: Execute frozen 10-kernel pilot
     if: github.event_name == 'workflow_dispatch'

@@ -8,6 +8,12 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) once a
 ## [Unreleased]
 
 ### Added
+- Published the canonical research pilot v1 repository dataset from successful
+  workflow run `35393833464`: 70/70 normalized execution records, exact
+  environment and execution provenance, raw-report and processed-data SHA-256
+  locks, lossless per-case execution shards, and deterministic descriptive
+  RQ1–RQ4 tables guarded by a reproduction test. The full Actions artifact is
+  content-addressed in the dataset manifest for later durable archival.
 - Added the manual-only research pilot v1 execution protocol: seven frozen
   validation cases across ten logical kernel profiles, immutable profile and
   manifest locks, exact-environment derivation from observed kernel plus image

@@ -4,12 +4,37 @@ This directory defines the research-facing protocol for studying eBPF
 compatibility with BPFCompat. It is intentionally separate from product and CI
 documentation.
 
-**Status:** protocol and reproducibility scaffold. It is not a peer-reviewed
-publication and does not claim that the proposed study has been completed.
-
-Existing BPFCompat case studies motivated the questions below. Those existing
-observations must not be represented as prospectively preregistered evidence.
-The study dataset should be frozen and versioned before confirmatory analysis.
+**Status:** the prospective protocol is frozen and pilot corpus v1 has been
+collected and published as a versioned repository dataset. This is not a
+peer-reviewed publication, and the selected pilot must not be presented as a
+representative study of Linux deployments.
+
+Existing BPFCompat case studies motivated the questions below. Those earlier
+observations remain exploratory rather than being relabeled as prospectively
+collected evidence. Pilot v1 is tied to one canonical workflow run and immutable
+checksums before descriptive RQ1–RQ4 analysis.
+
+## Collected pilot v1
+
+The canonical pilot collection is GitHub Actions run `35393833464` at commit
+`5de550e5cc8c659872a90fc262eb0250c229bf23`. It produced all **70/70**
+planned execution records: 50 compatible, 13 incompatible, and 7 inconclusive.
+The collection is complete but not fully evaluable because the frozen Oracle
+Linux logical profile requested kernel family 5.15 while the vendor image booted
+6.12 UEK; those seven executions retain their exact observed environment and are
+inconclusive for the requested profile.
+
+Repository snapshot and integrity metadata:
+
+- [`data/v1/`](data/v1/) — immutable processed dataset shards, provenance,
+  raw-report checksums, and dataset manifest;
+- [`analysis/v1/`](analysis/v1/) — deterministic descriptive RQ1–RQ4 tables
+  and machine-readable analysis summary;
+- `scripts/research/analyze-study-v1.py` — analysis generator;
+- `scripts/research/test-analysis-v1.sh` — checksum and reproduction gate.
+
+The full workflow artifact remains staging evidence for later DOI archival; its
+artifact identity is recorded in the dataset manifest.
 
 ## Study objective
 

@@ -26,10 +26,12 @@
 ## Phase 3 — Reproducible study
 
 - [x] Implement a versioned study runner and normalizer for captured BPFCompat reports.
-- [ ] Generate every paper table and figure from normalized data.
+- [x] Generate pilot v1 descriptive RQ1–RQ4 tables from normalized data.
+- [ ] Generate every final paper table and figure from normalized data.
 - [ ] Add repeat-run checks for a representative sample.
 - [x] Document the v1 execution boundary, KVM/VM prerequisites, exact-environment identity, and collection completeness rules.
-- [ ] Publish raw/processed dataset checksums.
+- [x] Publish raw/processed dataset checksums and bind the repository snapshot to
+      the canonical successful workflow artifact.
 
 ## Phase 4 — Archival
 

@@ -0,0 +1,13 @@
+# Pilot v1 Analysis
+
+scripts/research/analyze-study-v1.py deterministically regenerates the
+descriptive RQ1–RQ4 tables under generated from research/data/v1.
+
+This is not a claim of preregistration. Corpus case roles and denominator
+policy were frozen before collection. The pilot is selected and x86_64-only,
+so rates are descriptive within this corpus.
+
+RQ3 reports paired verdict agreement only because the Cilium-derived
+contracts differ (load+attach versus load-only). RQ4 can show cross-vendor
+variation but cannot estimate patch-level longitudinal change from one exact
+environment per logical profile.
@@ -0,0 +1,29 @@
+{
+  "schema_version":"bpfcompat.research.analysis-plan.v1",
+  "corpus_version":"v1",
+  "status":"post-collection analysis implementation; denominator roles were frozen before collection",
+  "canonical_run_id":"35393833464",
+  "canonical_run_sha":"5de550e5cc8c659872a90fc262eb0250c229bf23",
+  "case_roles":{
+    "simple-pass-libbpf":"controlled_probe",
+    "ringbuf-modern-libbpf":"controlled_probe",
+    "perfbuf-fallback-libbpf":"controlled_probe",
+    "core-relocation-fail-libbpf":"calibration",
+    "cilium-tracepoint-libbpf":"primary_oss_derived",
+    "cilium-tracepoint-ebpf-go":"primary_oss_derived",
+    "falco-modern-bpf-scap-open":"primary_real_world"
+  },
+  "rq1":{
+    "case_id":"ringbuf-modern-libbpf",
+    "feature":"BPF_MAP_TYPE_RINGBUF",
+    "upstream_introduction_version":"5.8",
+    "upstream_commit":"457f44363a8894135c85b7a9afd2bd8196db24ab",
+    "source":"https://github.com/torvalds/linux/commit/457f44363a8894135c85b7a9afd2bd8196db24ab"
+  },
+  "rq3":{
+    "left_case":"cilium-tracepoint-libbpf",
+    "right_case":"cilium-tracepoint-ebpf-go",
+    "contracts_comparable":false,
+    "reason":"libbpf is load+attach while cilium/ebpf is load-only; report agreement but do not estimate a pure loader causal effect"
+  }
+}
@@ -0,0 +1,33 @@
+# Pilot v1 generated results
+
+This file is generated from the canonical v1 dataset; it is descriptive pilot analysis, not a population estimate.
+
+## Collection
+
+- 70/70 execution records collected.
+- Overall: 50 compatible, 13 incompatible, 7 inconclusive.
+- Non-calibration: 50 compatible, 4 incompatible, 6 inconclusive.
+
+## RQ1 — kernel-version predictiveness
+
+For the controlled BPF_MAP_TYPE_RINGBUF probe, a simple Linux 5.8 threshold agreed with 8/9 conclusive observations (88.889%). The exception was almalinux-8-4.18, whose observed 4.18 vendor kernel supported the probe.
+
+## RQ2 — failure taxonomy
+
+Outside calibration there were 4 incompatible observations among 54 evaluable executions: COMMAND_VALIDATION_FAILURE = 2, UNSUPPORTED_MAP_TYPE = 2.
+
+## RQ3 — loader-path observation
+
+The paired Cilium-derived paths had 0 verdict disagreements across 9 conclusive environments. The contracts differ (load+attach vs load-only), so this is not a pure loader-effect estimate.
+
+## RQ4 — vendor/environment variation
+
+The dataset contains 10 exact environments across 10 logical profiles and 2 cross-vendor version inversions. Patch-level longitudinal change is not evaluable in v1.
+
+## Limitations
+
+- Selected stratified x86_64 pilot; not representative of global Linux deployments.
+- Controlled probes and calibration are not mixed into real-world prevalence claims.
+- Cilium paths use load+attach versus load-only, so agreement is not a pure loader-effect estimate.
+- Oracle requested 5.15 but booted 6.12 UEK; seven executions are inconclusive for the requested profile.
+- One exact environment per logical profile prevents patch-level longitudinal inference.
@@ -0,0 +1,134 @@
+{
+  "canonical_run_id": "35393833464",
+  "canonical_run_sha": "5de550e5cc8c659872a90fc262eb0250c229bf23",
+  "collection_complete": true,
+  "corpus_version": "v1",
+  "fully_evaluable": false,
+  "limitations": [
+    "Selected stratified x86_64 pilot; not representative of global Linux deployments.",
+    "Controlled probes and calibration are not mixed into real-world prevalence claims.",
+    "Cilium paths use load+attach versus load-only, so agreement is not a pure loader-effect estimate.",
+    "Oracle requested 5.15 but booted 6.12 UEK; seven executions are inconclusive for the requested profile.",
+    "One exact environment per logical profile prevents patch-level longitudinal inference."
+  ],
+  "non_calibration": {
+    "attempts": 60,
+    "compatible": 50,
+    "evaluable": 54,
+    "incompatible": 4,
+    "inconclusive": 6
+  },
+  "overall": {
+    "attempts": 70,
+    "compatible": 50,
+    "evaluable": 63,
+    "incompatible": 13,
+    "inconclusive": 7
+  },
+  "rq1_version_predictiveness": {
+    "agreement": 8,
+    "agreement_pct": 88.889,
+    "below_threshold_compatible_profiles": [
+      "almalinux-8-4.18"
+    ],
+    "disagreement": 1,
+    "evaluable": 9,
+    "false_negative": 1,
+    "false_positive": 0,
+    "feature": "BPF_MAP_TYPE_RINGBUF",
+    "sensitivity_pct": 85.714,
+    "specificity_pct": 100.0,
+    "true_negative": 2,
+    "true_positive": 6,
+    "upstream_commit": "457f44363a8894135c85b7a9afd2bd8196db24ab",
+    "upstream_introduction_version": "5.8"
+  },
+  "rq2_failure_taxonomy": {
+    "all": {
+      "attempts": 70,
+      "compatible": 50,
+      "evaluable": 63,
+      "incompatible": 13,
+      "inconclusive": 7,
+      "taxonomy": {
+        "COMMAND_VALIDATION_FAILURE": 2,
+        "CORE_RELOCATION_FAILURE": 8,
+        "MISSING_BTF": 1,
+        "UNSUPPORTED_MAP_TYPE": 2
+      }
+    },
+    "calibration": {
+      "attempts": 10,
+      "compatible": 0,
+      "evaluable": 9,
+      "incompatible": 9,
+      "inconclusive": 1,
+      "taxonomy": {
+        "CORE_RELOCATION_FAILURE": 8,
+        "MISSING_BTF": 1
+      }
+    },
+    "controlled_probe": {
+      "attempts": 30,
+      "compatible": 25,
+      "evaluable": 27,
+      "incompatible": 2,
+      "inconclusive": 3,
+      "taxonomy": {
+        "UNSUPPORTED_MAP_TYPE": 2
+      }
+    },
+    "non_calibration": {
+      "attempts": 60,
+      "compatible": 50,
+      "evaluable": 54,
+      "incompatible": 4,
+      "inconclusive": 6,
+      "taxonomy": {
+        "COMMAND_VALIDATION_FAILURE": 2,
+        "UNSUPPORTED_MAP_TYPE": 2
+      }
+    },
+    "primary_oss_derived": {
+      "attempts": 20,
+      "compatible": 18,
+      "evaluable": 18,
+      "incompatible": 0,
+      "inconclusive": 2,
+      "taxonomy": {}
+    },
+    "primary_real_world": {
+      "attempts": 10,
+      "compatible": 7,
+      "evaluable": 9,
+      "incompatible": 2,
+      "inconclusive": 1,
+      "taxonomy": {
+        "COMMAND_VALIDATION_FAILURE": 2
+      }
+    }
+  },
+  "rq3_loader_pair_observations": [
+    {
+      "conclusive_pairs": 9,
+      "contracts_comparable": false,
+      "interpretation_constraint": "libbpf is load+attach while cilium/ebpf is load-only; report agreement but do not estimate a pure loader causal effect",
+      "verdict_disagreement_pct": 0.0,
+      "verdict_disagreements": 0,
+      "verdict_matrix": {
+        "compatible__compatible": 9,
+        "inconclusive__inconclusive": 1
+      }
+    }
+  ],
+  "rq4_vendor_and_environment_variation": {
+    "cross_vendor_version_inversions": 2,
+    "exact_environments": 10,
+    "logical_profiles": 10,
+    "patch_level_longitudinal_evaluable": false,
+    "ringbuf_below_upstream_threshold_compatible_profiles": [
+      "almalinux-8-4.18"
+    ]
+  },
+  "schema_version": "bpfcompat.research.analysis-summary.v1"
+}
@@ -0,0 +1,8 @@
+case_id,analysis_role,attempts,compatible,incompatible,inconclusive,evaluable,incompatible_rate_evaluable_pct
+cilium-tracepoint-ebpf-go,primary_oss_derived,10,9,0,1,9,0.0
+cilium-tracepoint-libbpf,primary_oss_derived,10,9,0,1,9,0.0
+core-relocation-fail-libbpf,calibration,10,0,9,1,9,100.0
+falco-modern-bpf-scap-open,primary_real_world,10,7,2,1,9,22.222
+perfbuf-fallback-libbpf,controlled_probe,10,9,0,1,9,0.0
+ringbuf-modern-libbpf,controlled_probe,10,7,2,1,9,22.222
+simple-pass-libbpf,controlled_probe,10,9,0,1,9,0.0
@@ -0,0 +1,11 @@
+scope,classification_code,count,evaluable_denominator,rate_evaluable_pct
+all,COMMAND_VALIDATION_FAILURE,2,63,3.175
+all,CORE_RELOCATION_FAILURE,8,63,12.698
+all,MISSING_BTF,1,63,1.587
+all,UNSUPPORTED_MAP_TYPE,2,63,3.175
+non_calibration,COMMAND_VALIDATION_FAILURE,2,54,3.704
+non_calibration,UNSUPPORTED_MAP_TYPE,2,54,3.704
+primary_real_world,COMMAND_VALIDATION_FAILURE,2,9,22.222
+controlled_probe,UNSUPPORTED_MAP_TYPE,2,27,7.407
+calibration,CORE_RELOCATION_FAILURE,8,9,88.889
+calibration,MISSING_BTF,1,9,11.111
@@ -0,0 +1,11 @@
+logical_profile_id,observed_kernel_release,observed_series,version_prediction,observed_verdict,conclusive,agreement,classification_code
+amazon-linux-2-4.14,4.14.26-54.32.amzn2.x86_64,4.14,incompatible,incompatible,true,true,UNSUPPORTED_MAP_TYPE
+almalinux-8-4.18,4.18.0-553.158.1.el8_10.x86_64,4.18,incompatible,compatible,true,false,
+ubuntu-20.04-5.4,5.4.0-216-generic,5.4,incompatible,incompatible,true,true,UNSUPPORTED_MAP_TYPE
+ubuntu-20.10-5.8,5.8.0-63-generic,5.8,compatible,compatible,true,true,
+amazon-linux-2-5.10,5.10.260-259.1054.amzn2.x86_64,5.10,compatible,compatible,true,true,
+ubuntu-22.04-5.15,5.15.0-186-generic,5.15,compatible,compatible,true,true,
+debian-12-6.1,6.1.0-53-cloud-amd64,6.1,compatible,compatible,true,true,
+opensuse-leap-15.6-6.4,6.4.0-150600.23.100-default,6.4,compatible,compatible,true,true,
+ubuntu-24.04-6.8,6.8.0-139-generic,6.8,compatible,compatible,true,true,
+oracle-linux-9-uek7-5.15,6.12.0-107.59.3.3.el9uek.x86_64,6.12,compatible,inconclusive,false,,
@@ -0,0 +1,11 @@
+logical_profile_id,environment_id,same_exact_environment,left_verdict,right_verdict,both_conclusive,verdict_disagreement,contracts_comparable
+almalinux-8-4.18,sha256:6f4a1bd0e9c63fe5f7f2902a5f693a4bb366c607742ad4df2f2efe94c5db8144,true,compatible,compatible,true,false,false
+amazon-linux-2-4.14,sha256:6204b062fb4b85925457e0d585434758e8b292369b9333433428904df98ba117,true,compatible,compatible,true,false,false
+amazon-linux-2-5.10,sha256:50777ced54463ac814bd22b3f8d3d61607d49a14522780c74ab60e181a5744c2,true,compatible,compatible,true,false,false
+debian-12-6.1,sha256:581a88d78253d5045522c603c154ac4abae0dddf77062b254c3aeacb0a106192,true,compatible,compatible,true,false,false
+opensuse-leap-15.6-6.4,sha256:d5a6da80eb4d68ae55f7654cd39211733ed88c72fbb4cb18f62cf1c9b28e5add,true,compatible,compatible,true,false,false
+oracle-linux-9-uek7-5.15,sha256:9a193ce171dd4778266a5393e89ef3044ca9fb27b1f861dcee71b408dd1e53d6,true,inconclusive,inconclusive,false,false,false
+ubuntu-20.04-5.4,sha256:1433c43e14af8a0fec2f9c03b9459fc4c68ec586456b4d94852f59d9bd169f0f,true,compatible,compatible,true,false,false
+ubuntu-20.10-5.8,sha256:3fc06d4ba98cb42d9281fb46fb718acab139f6321018761ce4dbafa24db30549,true,compatible,compatible,true,false,false
+ubuntu-22.04-5.15,sha256:1fa71e1a3f9deb69c9f7b2553527aa471ff51942e695872dc764a1f5a16f9721,true,compatible,compatible,true,false,false
+ubuntu-24.04-6.8,sha256:0f9927cfa4b18965b4014d92f1f78845ee222ecc3908f3fa41fa87d08958df66,true,compatible,compatible,true,false,false
@@ -0,0 +1,11 @@
+logical_profile_id,distribution,distribution_release,requested_kernel_family,observed_kernel_release,kernel_family_match,exact_environment_id,image_identity
+almalinux-8-4.18,almalinux,8,4.18,4.18.0-553.158.1.el8_10.x86_64,true,sha256:6f4a1bd0e9c63fe5f7f2902a5f693a4bb366c607742ad4df2f2efe94c5db8144,sha256:66a440b458e0a0f774868f1b5d74433080c6718e40f38b59d05ff9ab269ae240
+amazon-linux-2-4.14,amazon-linux,2,4.14,4.14.26-54.32.amzn2.x86_64,true,sha256:6204b062fb4b85925457e0d585434758e8b292369b9333433428904df98ba117,sha256:e8b2457e9bc899ee292491f38f6fdf715a8fff0323d96ca3954ad07783a6ba63
+amazon-linux-2-5.10,amazon-linux,2,5.10,5.10.260-259.1054.amzn2.x86_64,true,sha256:50777ced54463ac814bd22b3f8d3d61607d49a14522780c74ab60e181a5744c2,sha256:b2b090278880b4eaaafb41baaaf0efb7ebaa7cc457d462674cd3b60eab2970c3
+debian-12-6.1,debian,12,6.1,6.1.0-53-cloud-amd64,true,sha256:581a88d78253d5045522c603c154ac4abae0dddf77062b254c3aeacb0a106192,sha256:3033874999eef45d1e3009e6d68dba2e8e411cdddfc94c95387b24db4cf0d999
+opensuse-leap-15.6-6.4,opensuse,15.6,6.4,6.4.0-150600.23.100-default,true,sha256:d5a6da80eb4d68ae55f7654cd39211733ed88c72fbb4cb18f62cf1c9b28e5add,sha256:0a5720416d423f98aacaa793a57d56ec045e3dd25cd88713952660ad00da53bd
+oracle-linux-9-uek7-5.15,oracle,9,5.15,6.12.0-107.59.3.3.el9uek.x86_64,false,sha256:9a193ce171dd4778266a5393e89ef3044ca9fb27b1f861dcee71b408dd1e53d6,sha256:88c75cf913a66227e9ce74b0087ecac4cce1883f3e5649082e982d0d00310f1c
+ubuntu-20.04-5.4,ubuntu,20.04,5.4,5.4.0-216-generic,true,sha256:1433c43e14af8a0fec2f9c03b9459fc4c68ec586456b4d94852f59d9bd169f0f,sha256:18f2977d77dfea1b74aee14533bd21c34f789139e949c57023b7364894b7e5e9
+ubuntu-20.10-5.8,ubuntu,20.10,5.8,5.8.0-63-generic,true,sha256:3fc06d4ba98cb42d9281fb46fb718acab139f6321018761ce4dbafa24db30549,sha256:9d4933b1118b7d3e9a45cba0e87ff0b4cea2da3538327c15f0750976398326f2
+ubuntu-22.04-5.15,ubuntu,22.04,5.15,5.15.0-186-generic,true,sha256:1fa71e1a3f9deb69c9f7b2553527aa471ff51942e695872dc764a1f5a16f9721,sha256:757908b2fd6d5b1431bb45070fc1f56cbf017d4025568d292ece37d9cc75e812
+ubuntu-24.04-6.8,ubuntu,24.04,6.8,6.8.0-139-generic,true,sha256:0f9927cfa4b18965b4014d92f1f78845ee222ecc3908f3fa41fa87d08958df66,sha256:612b2c0cc1bc413a6cb8c38fd611794caf0f2b436c50013d8b3794db12ad7354
@@ -0,0 +1,3 @@
+case_id,analysis_role,older_profile,older_kernel,newer_profile,newer_kernel
+ringbuf-modern-libbpf,controlled_probe,almalinux-8-4.18,4.18.0-553.158.1.el8_10.x86_64,ubuntu-20.04-5.4,5.4.0-216-generic
+falco-modern-bpf-scap-open,primary_real_world,almalinux-8-4.18,4.18.0-553.158.1.el8_10.x86_64,ubuntu-20.04-5.4,5.4.0-216-generic