Skip to content

Add skills, hooks and docs for agentic testing#661

Draft
jacobtomlinson wants to merge 1 commit intorapidsai:mainfrom
jacobtomlinson:doc/agentic-testing
Draft

Add skills, hooks and docs for agentic testing#661
jacobtomlinson wants to merge 1 commit intorapidsai:mainfrom
jacobtomlinson:doc/agentic-testing

Conversation

@jacobtomlinson
Copy link
Member

@jacobtomlinson jacobtomlinson commented Feb 6, 2026

Add some skills and hooks for automating docs testing with coding agents like Claude Code. Also adds some boilerplate content for coding agents like an AGENTS.md (with CLAUDE.md symlink).

In .claude/skills you will find a couple of skills which allow you to test individual pages. It works by using playwright-cli to drive a browser and carry out the steps in the documentation page.

The goal here is for human-in-the-loop testing. You may not want to automate every action, and you may want the agent to defer to the user in case of failure. The goal is to kick off an agent and keep an eye on it while it runs through the documentation, requiring minimal effort from the user.

Example

You can kick off a testing job using the /release-testing skill, just tell it which page you want to test.

claude "/release-testing test the AWS EC2 page"

The agent will take the following actions:

  • Rebuild a local copy of the nightly documentation
  • Locate the page you asked it to test
  • Refer to deployment internal notes on NVIDIA specifics (will ask you where these life if it can't find them automatically)
  • Open a fresh Chrome web browser with playwright-cli
  • Follows the instructions in the docs page and prompts you if it gets stuck
  • Once complete it will give you a report of what it found and if it has recommendations for updating the docs

Note: It will not perform destructive tasks like deleting resources so you will need to do this manually.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant