Repository navigation
feat(skills): vendor the ponytail skill pack across all three host trees - #36
Merged
Merged
Conversation
Vendors six MIT-licensed skills verbatim from dietrichgebert/ponytail (pinned at commit e3ba2aa, v4.10.0) into skills/, hermes/skills/ and codex/skills/, so the whole team gets the token-saving rule set from one install. The rule text is byte-for-byte upstream in every tree; only the YAML frontmatter and an appended host-notes section were adapted. Descriptions are deliberately not rewritten with the usual "(Hermes port of ...)" suffix, because the description is the trigger and changing it would change when the skill fires — the one thing the A/B measured. Attribution: - upstream MIT licence retained at skills/ponytail/LICENSE - upstream repo + commit pinned in every vendored skill's frontmatter - credited in README and CHANGELOG, explicitly as third-party (these skills are not Tapway-authored and do not carry the Karpathy framing) Rollout: - register all six in plugin.json, hermes/install.sh and codex/install.sh - bump version 2.3.0 -> 2.4.0 (plugin.json, marketplace.json) - sync counts: 30 canonical / 31 Hermes / 30 + $tapway umbrella (Codex) across README.md, hermes/README.md, codex/README.md, install comments and tests/check_docs_consistency.py - correct pre-existing stale marketplace counts (19 skills / 4 agents -> 30 skills / 5 agents) Tests (RED -> GREEN): - tests/test_ponytail_pack.py (109 checks): every canonical rule line survives in each host port, upstream provenance, declared counts agree with the filesystem, Hermes guard-trigger scrub, and that ponytail's description still carries its coding-task triggers - proven to discriminate: removing a vendored file fails 5 checks, making a declared count stale fails 1 - test_hermes_install.py: EXPECTED and blocked grow by six, so the community-install guard scrub is enforced, not merely documented - full suite green (10 test files, 811 checks) + docs consistency Measured before adopting (A/B, 5 Tapway repos, 30 headless sessions, n=3, deepseek-v4.1-flash): -40.3% input tokens, -50.4% output, -53.7% total, -38.4% API calls, -50.4% cost, -38.4% source LOC; correctness 100% baseline vs 93% ponytail, where the single miss was a malformed generation with zero edits rather than a quality regression. Caveat: n=3 with up to 3.6x run-to-run variance. Plan: docs/plans/2026-09-24-ponytail-skill-pack-adoption.md
Contributor
|
✅ Workflow audit passed — conventional commits ✓, plan ✓, no secrets ✓. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Vendors the ponytail skill pack — six MIT-licensed skills from
dietrichgebert/ponytail, pinned at commite3ba2aa(v4.10.0) — into all three host trees, and wires them into every declaration point.ponytailforces the laziest solution that actually works: YAGNI first, standard library beforecustom code, native platform features before dependencies, one line before fifty. It is
auto-triggering on coding tasks, so once loaded it shapes how code gets written for every
coding task, not only on request.
Why
We A/B'd it on five real repos before adopting. Same tickets, same model, same repo SHAs,
per-cell clones; the arms differ only by preloading the skill (30 headless sessions, n=3 per cell,
deepseek-v4.1-flash):Every repo used less; no repo regressed. Correctness was 100% (15/15) baseline vs
93% (14/15) ponytail. The one ponytail miss was a malformed generation — a tool-call fragment
leaked as final text with zero edits — not a quality regression. Importantly, neither arm ever
dropped a validation guard or broke an existing suite, which is the risk you'd expect a "write
less code" rule to introduce.
Honest caveats: n=3 per cell with up to 3.6× run-to-run variance, so read the means as
±10pp. One repo (
city-nexus) overlaps between arms and its per-task number is not reliable; theaggregate across all five is what holds. Five Python repos, one ticket each — this is not a
production-readiness claim.
How
Rule text is byte-for-byte upstream in all three trees. Only two things were adapted:
license: MIT,upstream,upstream-commitadded everywhere; Hermes/Codexhost keys (
version,author,platforms,metadata.hermes.*) on the ports.Deliberate deviation from convention: descriptions are not given the usual
(Hermes port of ...)suffix. The description is the trigger, and rewriting it would changewhen the skill fires — the one thing the A/B was measuring.
externalSkillsprecedentThis repo's existing pattern for third-party skills is to document them (
externalSkillsinplugin.json, e.g.code-refactor) rather than vendor them. This PR knowingly vendorsinstead, because that gives the team a one-command install and puts the rule text in-tree where it
can be reviewed and diffed against upstream. If you'd rather it be documentation-only, say so and
I'll rework it — the vendored copies are self-contained, so the revert is clean.
Attribution is explicit so nothing is misrepresented:
skills/ponytail/LICENSEretains the upstreamMIT licence, every skill pins the upstream repo + commit, and README/CHANGELOG state plainly that
these six are third-party and do not carry the Tapway/Karpathy framing the other 24 do.
Testing
New
tests/test_ponytail_pack.py— 109 checks, written RED first, and proven todiscriminate:
It asserts that every canonical rule line survives in each host port (so the skill that ships
is the skill that was measured), upstream provenance, that declared counts agree with the
filesystem, the Hermes guard-trigger scrub, and that
ponytail's description still carries itscoding-task triggers (a pack that never fires saves nothing).
tests/test_hermes_install.py—EXPECTEDandblockedgrow by six, so thecommunity-install guard scrub is enforced, not just documented
tests/check_docs_consistency.py— hardcoded expectations updatedRollout
$tapwayumbrella Codexplugin.json,marketplace.json)marketplace.jsonhad pre-existing stale counts (19 skills / 4 agents) that didn'tmatch the filesystem — corrected to 30 skills / 5 agents
Notes
master(the repo's default branch, where recent feature PRsdocs: add direct conversational workflow to team guide #23–feat(codex): port Tapway Superpowers to OpenAI Codex (24 skills + hooks + installer) #31+ landed, and what
workflow-audit.ymldiffs via${BASE_BRANCH:-master}). Notemainno longer exists on the remote — it was renamed to
master.docs/plans/2026-09-24-ponytail-skill-pack-adoption.mddocs/checklists/2026-09-24-ponytail-skill-pack-checklist.md