Skip to content

Feature/add tool filtering(r39 r40)#3

Merged
codeliftsleep2 merged 4 commits into
mainfrom
feature/add-tool-filtering(R39-R40)
Jun 15, 2026
Merged

Feature/add tool filtering(r39 r40)#3
codeliftsleep2 merged 4 commits into
mainfrom
feature/add-tool-filtering(R39-R40)

Conversation

@codeliftsleep2

Copy link
Copy Markdown
Owner

Description

Closes #(issue-number)


Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement
  • Test addition or improvement
  • CI/tooling change

PR Checklist

Code Quality

  • cargo check passes without errors
  • cargo clippy --all-targets -- -D warnings produces zero warnings
  • cargo test passes — all existing tests (currently 121) still pass
  • New tests added for new functionality, with at minimum:
    • Happy path test(s)
    • Error/invalid-input test(s)
    • Edge case test(s) (empty input, Unicode, boundaries)
  • cargo audit shows no known security vulnerabilities
  • No new #![allow(...)] annotations without a // SAFETY: or // Phase N: comment
  • No new .unwrap() calls without a // SAFETY: comment explaining why it cannot fail
  • No let _ = ... dead-code suppression — unused variables are removed
  • No unsafe blocks added (entire codebase is safe Rust)

Code Style

  • Follows the Single Responsibility Principle — each module/file owns one concern
  • Uses HashMap over BTreeMap unless sorted iteration is explicitly required
  • Error paths return Result, not panics
  • Functions are documented with doc comments (///) explaining their purpose, parameters, and return value
  • Inline comments explain non-obvious logic, especially around tree-sitter queries and opcode handling

Documentation

  • If adding a new language:
    • Cargo.toml updated with pinned tree-sitter grammar
    • Tree-sitter queries added to src/queries.rs
    • src/compression/language.rs updated with extension and heuristic
    • docs/DEVELOPER_DOCUMENTATION.md referenced (no changes needed — the guide covers the process)
    • Tests added for language detection
  • If adding a new MCP tool:
    • Handler function added in src/mcp/tools.rs
    • Tool definition added in get_tool_definitions()
    • Dispatch arm added in dispatch_tools_call()
    • Tests added in src/tests/mcp/tools.rs
  • If adding a new opcode:
    • Entry added to src/compression/opcodes.rs
    • Opcode table assertion count updated
    • Decompression marker added (if applicable)
    • README opcode reference updated
  • If adding a dependency:
    • Dependency is MIT or Apache-2.0 licensed (no GPL/AGPL)
    • cargo audit re-run after dependency change
  • If changing the public API or tool behavior:
    • README updated with new examples
    • docs/CHANGELOG.md updated

Performance (if applicable)

  • No unnecessary allocations in hot paths (format!write!/writeln! in loops)
  • Cache integration considered for repeat calls (content-hash registry, baseline snapshots, raw-token counts)
  • Large files (>1 MB) and large workspaces (>1,000 files) tested for reasonable performance

Migration / Backward Compatibility (if applicable)

  • No breaking changes to existing tool signatures (parameter names and types unchanged)
  • If breaking changes are unavoidable, they are documented in docs/CHANGELOG.md with migration instructions

Additional Context


Checklist for Reviewers

  • Changes are scoped and focused — consider splitting into multiple PRs if too large
  • Test coverage is adequate for the change
  • Edge cases are handled (empty input, Unicode, boundary conditions)
  • No regressions introduced in existing functionality

maerman added 4 commits June 15, 2026 15:39
… Scrubbing + FAANG audit

### Multi-Platform Proxy Support
- Add `PlatformAdapter` trait with Anthropic, OpenAI, and Generic implementations
- Auto-detect platform from model name in request body
- Support `PLATFORM` env var for manual override
- Support intercepting `/v1/messages` (Anthropic), `/v1/chat/completions` (OpenAI), `/chat` (Generic)
- Add platform-aware secret scrubbing and ANSI stripping (was Anthropic-only)
- Fix critical bugs: hardcoded `block["type"] == "tool_result"` in strip_ansi and scrub_secrets

### Tool Output Filtering (R-38)
- Implement 7-step filter pipeline: replace → match_output → strip/keep_lines → group_by → head/tail → max_lines → on_empty
- Add TOML-based filter loading with 7 built-in filters (cargo, npm, git-diff, pytest, tsc, dotnet, ng)
- Add most-specific-match-wins filter selection with priority tiebreaker
- Add per-program filter savings tracking with dashboard summary
- Add community filter loading from `.clean-ctx/filters/`
- Add pluggable `Pipeline` abstraction for OCP-compliant transform chaining

### Secret Scrubbing (R-37)
- Implement platform-agnostic secret scrubbing with 15+ secret type detection
- Add `ScrubFailClosed` semantics and `might_contain_secret()` pre-filter
- Support AWS keys, GitHub tokens, JWTs, PEM keys, API keys, and more

### FAANG Audit & Code Quality
- Add 243 total tests (112 lib + 112 bin + 18 audit regression + 1 integration)
- Add 18 FAANG regression tests covering all audit findings
- Fix 11 clippy warnings (sort_by → sort_by_key, filter_map → filter, derive Default, etc.)
- Fix field_reassign_with_default in ToolFilterConfig
- Add is_empty() to Pipeline for len_without_is_empty compliance

### Documentation
- Rewrite PROXY.md with multi-platform support, IDE integration guides (Cline, Cursor, Continue, Aider, Copilot)
- Update CHANGELOG.md with all v0.1.7 changes
- Add tool output filtering, secret scrubbing, community filter docs
Add filters for eslint, ruff, biome, go, make, pip, docker, docker-logs,
gh, curl, mvn, mypy, pyright, shellcheck, golangci-lint, kubectl, apt,
brew, pre-commit, ssh, systemctl, hadolint, yamllint, node-build,
dotnet-build, and dotnet-test.

Each filter is a TOML file with inline conformance tests covering:
- match_command regex for tool detection
- strip_lines_matching for noise removal
- match_output for collapse-on-success patterns
- on_empty fallback messages
- filter_stderr for tools that write diagnostics to stderr (biome, curl)

Also fixes:
- docker.toml: wrong test expectation ("make: ok" → "docker: ok"),
  added missing on_empty fallback
- brew.toml: consolidated duplicate strip_lines_matching patterns
  into single regex
- scrub_patterns.rs: fix test token format to avoid GitHub secret
  scanner false positive (xoxb-FAKE_TOKEN → xoxb-FAKETOKEN)

All 243 tests pass. Clippy: 0 errors.
@codeliftsleep2 codeliftsleep2 merged commit e6d6687 into main Jun 15, 2026
1 check passed
@codeliftsleep2 codeliftsleep2 deleted the feature/add-tool-filtering(R39-R40) branch June 15, 2026 23:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants