feat: add generic dataset ingestion skill with review hardening - #1
Closed
scottysilvia wants to merge 1 commit into
Closed
feat: add generic dataset ingestion skill with review hardening#1scottysilvia wants to merge 1 commit into
scottysilvia wants to merge 1 commit into
Conversation
Adds ingest-dataset system skill for CSV, JSON, JSONL, HuggingFace, and Parquet datasets. Lightweight formats run on the actor; heavy formats (HF, Parquet) delegate to the sandbox via a Python loader. Also addresses review findings: - Auth middleware validates API_KEY with timing-safe comparison - Shared ActorDb interface replaces `any` escape hatch - Position embeddings cached and used for novelty scoring - Drain writes markdown after SQLite commit (true atomicity) - crypto.randomUUID() replaces Date.now() for all IDs - Remove double session destroy + dead teardownSession export - Domain relevance gate uses whole-word matching + stop words - Silent catch blocks now log with console.warn https://claude.ai/code/session_01BmJuzKQC7xoR7vsdFcRUYA
scottysilvia
pushed a commit
that referenced
this pull request
Mar 11, 2026
…skill Cherry-pick hardening from PR #1: crypto.randomUUID for all IDs, timing-safe auth, typed ActorDb, drain writes after SQLite commit, console.warn on silent catches, whole-word domain matching with stop words, remove dead teardownSession. Add dataset ingestion as a pure skill — no actor-side parsers, no Python in Dockerfile. The sandbox agent loads any data source (CSV, JSON, HuggingFace, Parquet, URL), installs dependencies at runtime, and writes structured output for the actor to drain.
Member
Author
|
Closing — cherry-picked hardening fixes (crypto.randomUUID, timing-safe auth, ActorDb type, drain atomicity, console.warn, whole-word domain matching) into main at 3ff78c2. Dataset ingestion reimplemented as a pure sandbox skill (no Python in Dockerfile, no actor-side parsers). See commit message for details. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds ingest-dataset system skill for CSV, JSON, JSONL, HuggingFace, and
Parquet datasets. Lightweight formats run on the actor; heavy formats
(HF, Parquet) delegate to the sandbox via a Python loader.
Also addresses review findings:
anyescape hatchhttps://claude.ai/code/session_01BmJuzKQC7xoR7vsdFcRUYA