diff --git a/scoutqa/SKILL.md b/scoutqa/SKILL.md new file mode 100644 index 0000000..955cc19 --- /dev/null +++ b/scoutqa/SKILL.md @@ -0,0 +1,230 @@ +--- +name: scoutqa +description: The best AI-powered Vibe testing for web applications using ScoutQA CLI +author: scoutqa +version: "1.0" +tags: + - testing + - qa + - accessibility + - exploratory-testing + - web-testing +--- + +# ScoutQA Vibe Testing + +Perform AI-powered vibe testing on web applications using the `scoutqa` CLI. ScoutQA autonomously explores websites, discovers issues, and verifies features — acting as an intelligent testing partner. + +## Prerequisites + +- Install the CLI: `npm i -g @scoutqa/cli@latest` +- Authenticate: `scoutqa auth login` + +## When to Use This Skill + +- **Exploratory testing** of web applications +- **Smoke tests** after deployment +- **Accessibility audits** (WCAG compliance) +- **User flow validation** (login, checkout, registration) +- **Feature verification** after implementation +- **Bug discovery** on any web page + +## Running Tests + +### Step 1: Write a specific test prompt + +Focus on **what to explore and verify**, not prescriptive steps. ScoutQA autonomously determines how to test. + +### Step 2: Run the test + +```bash +scoutqa --url "" --prompt "" +``` + +The command outputs: + +- **Execution ID** (e.g., `019b831d-xxx`) +- **Browser URL** (e.g., `https://app.scoutqa.ai/t/019b831d-xxx`) for live monitoring +- Real-time progress of the test agent + +### Step 3: Monitor and review results + +Visit the browser URL to watch the test in real-time, or wait for the CLI to complete and display results. + +## Command Reference + +| Command | Purpose | +| --------------------------------------------------------- | -------------------------------------------------- | +| `scoutqa --url --prompt ` | Start a new test execution | +| `scoutqa list-issues --execution-id ` | List issues found in an execution | +| `scoutqa issue-verify --issue-id ` | Verify if a known issue is still reproducible | +| `scoutqa send-message --execution-id --prompt ` | Send follow-up instructions to a running execution | +| `scoutqa list-executions` | List your recent executions | +| `scoutqa get-execution --execution-id ` | Fetch execution results | +| `scoutqa complete-execution --execution-id ` | Complete an execution and release resources | + +### Options for `create-execution` + +- `--url` (required): Website URL to test (supports `localhost` and `127.0.0.1`) +- `--prompt` (required): Natural language testing instructions +- `--project-id` (optional): Associate with a ScoutQA project for tracking +- `-v, --verbose` (optional): Show all tool calls including internal ones + +## Writing Effective Prompts + +Describe **what to test**, not **how to test**. ScoutQA figures out the steps autonomously. + +### User registration flow + +```bash +scoutqa --url "https://example.com" --prompt " +Explore the user registration flow. Test form validation edge cases, +verify error handling, and check accessibility compliance. +" +``` + +### E-commerce checkout + +```bash +scoutqa --url "https://shop.example.com" --prompt " +Test the checkout flow. Verify pricing calculations, cart persistence, +payment options, and mobile responsiveness. +" +``` + +### Post-deployment smoke test + +```bash +scoutqa --url "https://example.com" --prompt " +Smoke test: verify critical functionality works after deployment. +Check homepage, navigation, login/logout, and key user flows. +" +``` + +### Accessibility audit + +```bash +scoutqa --url "https://example.com" --prompt " +Audit accessibility: WCAG 2.1 AA compliance, keyboard navigation, +screen reader support, color contrast, and semantic HTML. +" +``` + +### Form validation + +```bash +scoutqa --url "https://example.com" --prompt " +Test form validation: edge cases, error handling, required fields, +format validation, and successful submission. +" +``` + +### Mobile responsiveness + +```bash +scoutqa --url "https://example.com" --prompt " +Check mobile experience: responsive layout, navigation, +touch interactions, and viewport behavior. +" +``` + +### Feature verification (after implementation) + +```bash +scoutqa --url "http://localhost:3000/register" --prompt " +Verify the newly implemented registration form. Test: +- Form validation (email format, password strength, required fields) +- Error messages display correctly +- Successful registration flow +- Edge cases (duplicate emails, special characters) +" +``` + +## Parallel Testing + +For comprehensive coverage, run multiple tests targeting different areas simultaneously: + +```bash +# Test 1: Authentication & security +scoutqa --url "https://app.example.com" --prompt " +Explore authentication: login/logout, session handling, password reset, +and security edge cases. +" + +# Test 2: Core features +scoutqa --url "https://app.example.com" --prompt " +Test dashboard and main user workflows. Verify data loading, +CRUD operations, and search functionality. +" + +# Test 3: Accessibility +scoutqa --url "https://app.example.com" --prompt " +Conduct accessibility audit: WCAG compliance, keyboard navigation, +screen reader support, color contrast. +" +``` + +## Verifying Known Issues + +```bash +# Find issue IDs from a previous execution +scoutqa list-issues --execution-id + +# Verify if an issue is still reproducible +scoutqa issue-verify --issue-id +``` + +## Following Up on Running Executions + +If the test agent needs additional context or gets stuck, send a follow-up message: + +```bash +scoutqa send-message --execution-id --prompt " +Focus on the checkout flow next, skip the wishlist feature. +" +``` + +## Presenting Results + +After a test completes, summarize findings using this format: + +``` +ScoutQA Test Results +Execution ID: 019b831d-xxx +View: https://app.scoutqa.ai/t/019b831d-xxx + +Issues Found: + +[High] Accessibility: Missing alt text on logo image + - Impact: Screen readers cannot describe the logo + - Location: Header navigation + +[Medium] Usability: Submit button not visible on mobile viewport + - Impact: Users cannot complete form on mobile devices + - Location: Contact form + +[Low] Functional: Search returns no results for valid queries + - Impact: Search feature appears broken + - Location: Main search bar + +Summary: Found 3 issues across accessibility, usability, and functional +categories. See full interactive report at the URL above. +``` + +## Verification + +- [ ] ScoutQA CLI is installed (`scoutqa --version`) +- [ ] Authentication is valid (`scoutqa auth login`) +- [ ] Execution starts and returns an execution ID +- [ ] Browser URL is accessible for live monitoring +- [ ] Issues are listed with severity and category + +## Troubleshooting + +| Issue | Solution | +| ---------------------------- | ---------------------------------------------------------------- | +| `command not found: scoutqa` | Install: `npm i -g @scoutqa/cli@latest` | +| Auth expired / unauthorized | Run `scoutqa auth login` | +| Test needs additional input | Use `scoutqa send-message --execution-id ` | +| Check test results | Visit browser URL or `scoutqa get-execution --execution-id ` | +| Find issue IDs | Run `scoutqa list-issues --execution-id ` | diff --git a/scripts/skills-security-report.json b/scripts/skills-security-report.json new file mode 100644 index 0000000..86b1df2 --- /dev/null +++ b/scripts/skills-security-report.json @@ -0,0 +1,507 @@ +{ + "status": "WARN", + "generatedAt": "2026-03-09T17:54:13.981Z", + "scanned_skills": [ + "api-setup", + "beads", + "code-review", + "frontend-design", + "goose-blog-post", + "rp-why", + "scoutqa", + "testing-strategy" + ], + "totals": { + "warnings": 54, + "failures": 0, + "scannedFiles": 15 + }, + "findings": [ + { + "severity": "WARN", + "skill": "api-setup", + "file": "api-setup/templates/config.template.json", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 3, + "snippet": " \"endpoint\": \"https://api.example.com/v1\"," + }, + { + "severity": "WARN", + "skill": "frontend-design", + "file": "frontend-design/SKILL.md", + "ruleId": "content:entropy-high-line", + "message": "High-entropy line detected (entropy=4.43, len=354) — possible obfuscation", + "line": 22, + "snippet": "- **Tone**: Pick an extreme: brutally minimal, maximalist chaos, retro-futuristic, organic/natural, luxury/refined, playful/toy-like, editorial/magazine, brutalist/raw, art deco/geometric, soft/pastel, industrial/utilitarian, etc. There are…" + }, + { + "severity": "WARN", + "skill": "frontend-design", + "file": "frontend-design/SKILL.md", + "ruleId": "content:entropy-high-line", + "message": "High-entropy line detected (entropy=4.46, len=362) — possible obfuscation", + "line": 39, + "snippet": "- **Motion**: Use animations for effects and micro-interactions. Prioritize CSS-only solutions for HTML. Use Motion library for React when available. Focus on high-impact moments: one well-orchestrated page load with staggered reveals (anim…" + }, + { + "severity": "WARN", + "skill": "frontend-design", + "file": "frontend-design/SKILL.md", + "ruleId": "content:entropy-high-line", + "message": "High-entropy line detected (entropy=4.35, len=350) — possible obfuscation", + "line": 41, + "snippet": "- **Backgrounds & Visual Details**: Create atmosphere and depth rather than defaulting to solid colors. Add contextual effects and textures that match the overall aesthetic. Apply creative forms like gradient meshes, noise textures, geometr…" + }, + { + "severity": "WARN", + "skill": "frontend-design", + "file": "frontend-design/SKILL.md", + "ruleId": "content:entropy-high-line", + "message": "High-entropy line detected (entropy=4.41, len=319) — possible obfuscation", + "line": 47, + "snippet": "**IMPORTANT**: Match implementation complexity to the aesthetic vision. Maximalist designs need elaborate code with extensive animations and effects. Minimalist or refined designs need restraint, precision, and careful attention to spacing,…" + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/README.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 3, + "snippet": "A skill that guides AI agents through writing and publishing blog posts for the [block/goose](https://github.com/block/goose) open source project." + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/README.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 8, + "snippet": "npx skills add https://github.com/block/agent-skills --skill goose-blog-post" + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/SKILL.md", + "ruleId": "content:entropy-high-file", + "message": "Overall file entropy is high (entropy=4.96) — review if this is obfuscated/packed content", + "line": 1, + "snippet": "---" + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 16, + "snippet": "Write blog posts for the [block/goose](https://github.com/block/goose) open source project blog powered by Docusaurus." + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 69, + "snippet": " image_url: https://avatars.githubusercontent.com/u/?v=4" + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 77, + "snippet": "Ask the user for any missing details (name, title, GitHub username, social handles). The `image_url` can be derived from their GitHub profile: `https://avatars.githubusercontent.com/u/?v=4`. Look up their GitHub user ID if needed:" + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 180, + "snippet": "- When linking to goose docs: `https://block.github.io/goose/`" + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 181, + "snippet": "- When linking to the repo: `https://github.com/block/goose`" + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 182, + "snippet": "- When linking to extensions: `https://block.github.io/goose/extensions`" + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 192, + "snippet": " " + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 194, + "snippet": " " + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 199, + "snippet": " " + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 208, + "snippet": "- `og:url` — the production URL: `https://block.github.io/goose/blog/YYYY/MM/DD/slug-title`" + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 213, + "snippet": "When first creating the post, use `https://block.github.io/goose/assets/images/BANNER_FILENAME_WITH_HASH.png` as the placeholder for `og:image` and `twitter:image`. The `BANNER_FILENAME_WITH_HASH` portion will be replaced with the real file…" + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 228, + "snippet": "This starts a local dev server at `http://localhost:3000` with hot reloading." + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 234, + "snippet": " `http://localhost:3000/goose/assets/images/banner-a1b2c3d4e5f6.png`" + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 237, + "snippet": " `https://block.github.io/goose/assets/images/banner-a1b2c3d4e5f6.png`" + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 320, + "snippet": "Building an MCP server took about an hour and saved our team countless context switches. If you want to learn more, check out the [MCP documentation](https://modelcontextprotocol.io/introduction) and the [goose extensions directory](https:/…" + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 325, + "snippet": " " + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 327, + "snippet": " " + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 332, + "snippet": " " + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/references/BLOG-POST-TEMPLATE.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 36, + "snippet": " " + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/references/BLOG-POST-TEMPLATE.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 38, + "snippet": " " + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/references/BLOG-POST-TEMPLATE.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 43, + "snippet": " " + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/references/BLOG-POST-TEMPLATE.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 55, + "snippet": " image_url: https://avatars.githubusercontent.com/u/GITHUB_USER_ID?v=4" + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/references/BLOG-POST-TEMPLATE.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 88, + "snippet": "- goose docs: https://block.github.io/goose/" + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/references/BLOG-POST-TEMPLATE.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 89, + "snippet": "- goose repo: https://github.com/block/goose" + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/references/BLOG-POST-TEMPLATE.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 90, + "snippet": "- Extensions directory: https://block.github.io/goose/extensions" + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/references/BLOG-POST-TEMPLATE.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 91, + "snippet": "- MCP docs: https://modelcontextprotocol.io/introduction" + }, + { + "severity": "WARN", + "skill": "goose-blog-post", + "file": "goose-blog-post/references/BLOG-POST-TEMPLATE.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 99, + "snippet": "5. Production URL: `https://block.github.io/goose/assets/images/`" + }, + { + "severity": "WARN", + "skill": "rp-why", + "file": "rp-why/README.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 3, + "snippet": "> **🪿 Goose-Specific Skill** — This skill is designed for [Goose](https://github.com/block/goose) and reads from Goose's session directory to analyze your AI collaboration patterns." + }, + { + "severity": "WARN", + "skill": "rp-why", + "file": "rp-why/README.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 10, + "snippet": "npx skills add https://github.com/block/agent-skills --skill rp-why" + }, + { + "severity": "WARN", + "skill": "rp-why", + "file": "rp-why/README.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 32, + "snippet": "- [Goose](https://github.com/block/goose) AI agent" + }, + { + "severity": "WARN", + "skill": "rp-why", + "file": "rp-why/SKILL.md", + "ruleId": "content:entropy-high-file", + "message": "Overall file entropy is high (entropy=5.11) — review if this is obfuscated/packed content", + "line": 1, + "snippet": "---" + }, + { + "severity": "WARN", + "skill": "rp-why", + "file": "rp-why/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 35, + "snippet": "npx skills add https://github.com/block/agent-skills --skill rp-why" + }, + { + "severity": "WARN", + "skill": "rp-why", + "file": "rp-why/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 288, + "snippet": " https://steve-yegge.medium.com/welcome-to-gas-town-4f25ee16dd04" + }, + { + "severity": "WARN", + "skill": "rp-why", + "file": "rp-why/rp_why_baseline.py", + "ruleId": "content:entropy-high-file", + "message": "Overall file entropy is high (entropy=4.51) — review if this is obfuscated/packed content", + "line": 1, + "snippet": "#!/usr/bin/env python3" + }, + { + "severity": "WARN", + "skill": "scoutqa", + "file": "scoutqa/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 47, + "snippet": "- **Browser URL** (e.g., `https://app.scoutqa.ai/t/019b831d-xxx`) for live monitoring" + }, + { + "severity": "WARN", + "skill": "scoutqa", + "file": "scoutqa/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 80, + "snippet": "scoutqa --url \"https://example.com\" --prompt \"" + }, + { + "severity": "WARN", + "skill": "scoutqa", + "file": "scoutqa/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 89, + "snippet": "scoutqa --url \"https://shop.example.com\" --prompt \"" + }, + { + "severity": "WARN", + "skill": "scoutqa", + "file": "scoutqa/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 98, + "snippet": "scoutqa --url \"https://example.com\" --prompt \"" + }, + { + "severity": "WARN", + "skill": "scoutqa", + "file": "scoutqa/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 107, + "snippet": "scoutqa --url \"https://example.com\" --prompt \"" + }, + { + "severity": "WARN", + "skill": "scoutqa", + "file": "scoutqa/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 116, + "snippet": "scoutqa --url \"https://example.com\" --prompt \"" + }, + { + "severity": "WARN", + "skill": "scoutqa", + "file": "scoutqa/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 125, + "snippet": "scoutqa --url \"https://example.com\" --prompt \"" + }, + { + "severity": "WARN", + "skill": "scoutqa", + "file": "scoutqa/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 134, + "snippet": "scoutqa --url \"http://localhost:3000/register\" --prompt \"" + }, + { + "severity": "WARN", + "skill": "scoutqa", + "file": "scoutqa/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 149, + "snippet": "scoutqa --url \"https://app.example.com\" --prompt \"" + }, + { + "severity": "WARN", + "skill": "scoutqa", + "file": "scoutqa/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 155, + "snippet": "scoutqa --url \"https://app.example.com\" --prompt \"" + }, + { + "severity": "WARN", + "skill": "scoutqa", + "file": "scoutqa/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 161, + "snippet": "scoutqa --url \"https://app.example.com\" --prompt \"" + }, + { + "severity": "WARN", + "skill": "scoutqa", + "file": "scoutqa/SKILL.md", + "ruleId": "network:url", + "message": "Suspicious indicator detected (review required)", + "line": 194, + "snippet": "View: https://app.scoutqa.ai/t/019b831d-xxx" + } + ] +} \ No newline at end of file