[L5] Test infrastructure: new streamhouse-e2e crate, shell E2E suite, and CI pipeline overhaul

[gbram1]

Part of an AI-generated stack — level 5 of 5.
Depends on: Level 4. Merges into: stack/pr-3/L04-byoc-s3-client-pool-new-streamhouse-stor.

Goal

Ship the new Rust E2E test harness, the reorganized shell test phases, and the updated CI workflow that runs them.

Risk

low — Test and CI changes only; no production code paths are affected beyond fixes to the compaction mock and the Kafka fetch LSO filter.

Rationale

Depends on L4 because streamhouse-e2e/src/lib.rs constructs AppState with the byoc_s3: None field added in L4, and the e2e tests reference OidcJwksAuth (L3) via AppState. The compaction mock in compaction.rs replaces unimplemented!() panics with sensible defaults — this is a test enablement fix that belongs here. The Kafka fetch LSO filter (fetch.rs) is a behavior improvement surfaced by the new E2E tests and is included here since splitting it would require hunk-level surgery on a file whose dominant change is enabling E2E coverage. The CI overhaul adds the new e2e job with Postgres + MinIO service containers and the benchmark regression gate.

Files (46)

• crates/streamhouse-e2e/Cargo.toml
• crates/streamhouse-e2e/src/lib.rs
• crates/streamhouse-e2e/tests/full_write_path.rs
• crates/streamhouse-e2e/tests/kafka_protocol.rs
• crates/streamhouse-kafka/src/handlers/fetch.rs
• crates/streamhouse-server/src/compaction.rs
• tests/build-e2e.sh
• tests/docker-compose.test.yml
• tests/e2e-tests.sh
• tests/lib/common.sh
• tests/run-all.sh
• tests/phases/legacy/00-full-path.sh
• tests/phases/legacy/01-smoke.sh
• tests/phases/legacy/02-rest-api.sh
• tests/phases/legacy/02-topics.sh
• tests/phases/legacy/03-grpc-api.sh
• tests/phases/legacy/03-produce-consume.sh
• tests/phases/legacy/04-schema-registry.sh
• tests/phases/legacy/04-schemas.sh
• tests/phases/legacy/05-lifecycle.sh
• tests/phases/legacy/05-sql.sh
• tests/phases/legacy/06-consumer-groups.sh
• tests/phases/legacy/06-negative.sh
• tests/phases/legacy/07-auth.sh
• tests/phases/legacy/07-concurrent-load.sh
• tests/phases/legacy/08-benchmarks.sh
• tests/phases/legacy/08-multi-tenancy.sh
• tests/phases/legacy/09-cli.sh
• tests/phases/legacy/09-connectors.sh
• tests/phases/legacy/10-pipelines.sh
• tests/phases/legacy/10-sql-verification.sh
• tests/phases/legacy/11-kafka-protocol.sh
• tests/phases/legacy/11-metrics.sh
• tests/phases/legacy/12-cross-protocol.sh
• tests/phases/legacy/13-multi-tenancy.sh
• tests/phases/legacy/13-negative.sh
• tests/phases/legacy/14-concurrent-load.sh
• tests/phases/legacy/14-connectors.sh
• tests/phases/legacy/15-cli.sh
• tests/phases/legacy/15-dashboard-metrics.sh
• tests/phases/legacy/16-advanced-sql.sh
• tests/phases/legacy/16-benchmarks.sh
• tests/phases/legacy/17-resource-storage.sh
• tests/phases/01-smoke.sh
• .github/workflows/ci.yml
• .github/workflows/docker-publish.yml