Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
"author": {
"name": "sonesuke"
},
"source": "./plugin"
"source": "./claude-plugin"
}
]
}
94 changes: 94 additions & 0 deletions .claude/skills/analyze-skill-timeline/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
---
name: analyze-skill-timeline
description: Analyze a skill-bench JSONL log file and output a structured timeline table showing tool calls, arguments, and durations. Use this skill whenever the user wants to review, inspect, or understand what happened during a skill-bench test run — including phrases like "ログを確認", "timelineを見て", "テストの内容を確認", "what happened in this test", or when they provide a path to a .log file from the logs/ directory. Also use when the user asks about execution time breakdown or MCP tool call patterns.
---

# Analyze Skill Timeline

Analyze a skill-bench log file and produce a structured timeline table. This helps quickly understand what a test did, how long each step took, and where time was spent.

## Input

The user provides a log file path as ARGUMENTS. The file is JSONL format produced by `skill-bench run --log`.

## Process

### 1. Get the overview with `skill-bench timeline`

Run `skill-bench timeline <log-file>` via Bash. This is the backbone of the analysis — it provides timestamps, event types, tool call summaries, and total duration.

### 2. Extract metadata

Read line 1 of the JSONL (the `type: "system"` init line). Extract:

- `model` — which model was used
- `cwd` — contains the test name
- `mcp_servers[].name` — connected MCP servers
- `skills` — loaded skills (filter out built-ins like `update-config`, `debug`, etc.)

### 3. Extract tool call details

Use jq to extract tool calls from the JSONL. jq parses JSON properly and can extract specific fields even from very long lines (unlike Grep which truncates them).

Two jq passes:

```bash
# Tool calls: timestamp, id, name, key input fields
cat <log-file> | jq -c 'select(.type == "assistant") | .timestamp as $ts | .message.content[]? | select(.type == "tool_use") | {ts: $ts, id: .id, name: .name, input: .input}'

# Tool results: timestamp, tool_use_id
cat <log-file> | jq -c 'select(.type == "user") | .timestamp as $ts | .message.content[]? | select(.type == "tool_result") | {ts: $ts, id: .tool_use_id}'
```

From the jq output, extract:

| Category | Pattern | What to extract |
| -------- | -------------------------------- | --------------------------------------------------------------------------------------- |
| MCP tool | name contains `mcp__` | Short name (last segment after `__`), key args: `query`, `assignee`, `country`, `limit` |
| Skill | name is `"Skill"` | `input.skill` value |
| File I/O | Read, Write, Glob, Grep | `input.file_path` or `input.pattern` |
| Other | Bash, TodoWrite, AskUserQuestion | Name only |

### 4. Calculate durations

For each tool call, match its `id` to a `tool_result`'s `tool_use_id`. Duration = result timestamp - call timestamp.

Detect simultaneous calls: if multiple tool calls share the same timestamp (within 0.01s tolerance), mark the 2nd and subsequent as "simultaneous" instead of showing a duration.

**Reasoning time**: For each gap between a tool_result and the next tool_use, calculate `next_tool_use.ts - last_tool_result.ts`. This is pure Claude reasoning time (no tool execution). If the gap is > 1s, insert a row in the timeline.

### 5. Output

Produce a markdown timeline combining `skill-bench timeline` overview with enriched details.

```
### Timeline: `<test-name>`

**Duration**: X.XXs | **Model**: `model-name` | **Skills**: `skill1, skill2`

| Time | Action | Duration |
|------|--------|----------|
| **0-1.5s** | Init | 1.5s |
| **6.3s** | `search_patents` #1: assignee=[Salesforce, HubSpot] query="chatbot" | **11.1s** |
| **27.1s** | `search_patents` #2: query=`"chatbot" "sentiment"` | — |
| **27.1s** | `search_patents` #3: query=`"chatbot" "CRM"` | simultaneous |
| **38.0s** | 🧠 Reasoning | 13.6s |
| **59.0s** | `search_patents` #5: query=`"chatbot" "sentiment analysis"` | 3.5s |
| **132.3s** | Write: targeting.md | 0.1s |

### Summary

- MCP calls: `search_patents` ×7 (19.3s), `check_assignee` ×2 (17.4s)
- Claude reasoning: 112.5s / 178.8s (63%)
```

### Formatting rules

- **Time column**: Use `**Xs**` for individual events. Group rapid sequential events if useful.
- **Bold durations** for operations > 5s — these are the bottlenecks worth investigating.
- **MCP tool names**: Use backticks with a `#N` counter per tool type (e.g., `` `search_patents` #1 ``).
- **Parameters**: Show key args concisely. Truncate `assignee` arrays to first 2 items + `...`. Truncate file paths to the last 2 segments.
- **Simultaneous calls**: If N > 1 calls share the same timestamp, mark 2nd+ as "simultaneous". Note that the MCP server processes requests sequentially (rmcp JSON-RPC is one-at-a-time), so even "simultaneous" calls complete one after another. The duration column for the first call in the group reflects this.
- **Summary**: Show both MCP time and Claude reasoning time with their percentages of total duration.
- **Reasoning rows**: Use 🧠 icon. Show for gaps > 1s between tool_result and next tool_use. Calculate from the last tool_result in a group (even for simultaneous calls, use the final result).
- Keep the table focused on MCP calls and file operations. Skip noise like TodoWrite unless the user seems interested.
15 changes: 15 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,5 +23,20 @@ jobs:
with:
node-version: 20

- name: Setup Rust
uses: dtolnay/rust-toolchain@stable

- name: Rust cache
uses: swatinem/rust-cache@v2

- name: Check formatting
run: cargo fmt --all -- --check

- name: Check formatting (Prettier)
run: npx --yes prettier@3.8.1 --log-level=debug --check .

- name: Run Clippy
run: cargo clippy -- -D warnings

- name: Run tests
run: cargo test
90 changes: 88 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ This repository (`patent-kit`) is a **Claude Plugin Marketplace** containing adv
## Architecture

- `.claude-plugin/marketplace.json`: The entry point defining the marketplace metadata.
- `plugin/`: The root directory of the `patent-kit` plugin containing `.claude-plugin/plugin.json`.
- `plugin/skills/`: Contains all analysis skills in flat directories. Each has a `SKILL.md` conforming to Claude's Official Skill Guidelines.
- `claude-plugin/plugin.json`: The plugin manifest declaring the `patent-kit` MCP server.
- `claude-plugin/skills/`: Contains all analysis skills in flat directories. Each has a `SKILL.md` conforming to Claude's Official Skill Guidelines.

## Mandatory AI Agent Rules

Expand All @@ -17,6 +17,92 @@ This repository (`patent-kit`) is a **Claude Plugin Marketplace** containing adv
3. **Commit Messages**: Always use Conventional Commits in English.
4. **Skill Instructions**: Do not instruct the execution of bash CLI commands like `google-patent-cli` in `SKILL.md`. Always instruct the use of the loaded MCP tools (`search_patents`, `fetch_patent`, `search_papers`, `fetch_paper`).

## Rust Binary (`patent-kit`)

The project includes a Rust MCP server and CLI installed via `cargo install --path .`.

### Build & Install

```bash
cargo install --path . # Build release and install to ~/.cargo/bin
cargo build --release # Build only (binary at target/release/patent-kit)
```

### CLI Commands

All commands support a `--verbose` flag for debugging (outputs search URLs and API status to stderr).

```bash
patent-kit mcp # Start MCP server over stdio
patent-kit check-assignee "Apple" --verbose
patent-kit search-patents "query" --assignee "Apple" --limit 5 --verbose
patent-kit import-csv <file>
patent-kit index-patents
patent-kit get-unscreened --limit 5
patent-kit screen-patent <id> --judgment relevant --reason "..."
patent-kit get-unevaluated --limit 5
patent-kit record-claims <id> <json>
patent-kit get-claims <id>
patent-kit record-elements <json>
patent-kit get-elements <id>
patent-kit get-unanalyzed --limit 5
patent-kit record-similarities <json>
patent-kit get-product-features
patent-kit record-product-feature --name "..." --description "..."
patent-kit get-unresearched --limit 5
patent-kit record-prior-arts <json>
patent-kit get-patent-detail <id>
patent-kit progress
```

### MCP Server

Defined in `claude-plugin/.mcp.json`. The server uses newline-delimited JSON-RPC over stdio (rmcp 0.16 transport). Tools are registered in `src/mcp/mod.rs`.

### Key Source Files

- `src/main.rs` — Entry point
- `src/cli/mod.rs` — CLI command definitions and dispatch
- `src/mcp/mod.rs` — MCP server: tool registration, handler, formatters
- `src/core/db.rs` — SQLite database operations
- `src/core/config.rs` — Configuration loading
- `src/core/models.rs` — Request/response types for MCP tools

### Dependencies (git)

- `google-patent-cli` — Google Patents search via headless Chromium (`~/.cargo/git/checkouts/google-patent-cli-*/`)
- `arxiv-cli` — arXiv paper search via headless Chromium (`~/.cargo/git/checkouts/arxiv-cli-*/`)

### Debugging Notes

- Google Patents may return generic/unfiltered results (same patents regardless of query) when the environment IP is rate-limited. Check `--verbose` output — if `total_results` is identical across different queries, this is likely the cause.
- The MCP server spawns Chromium on startup. Orphan Chromium processes are killed on shutdown.

## Testing

### Rust Unit Tests

```bash
cargo test # Run unit tests
mise run test # Same as above
mise run clippy # Lint with clippy
```

### Skill-Bench (E2E Tests)

```bash
mise run skill-bench # Run all E2E tests (auto-installs patent-kit, uses --plugin-dir)
skill-bench run tests/concept-interviewing/triggering.toml --plugin-dir ./claude-plugin --threads 4 --log ./logs
skill-bench run tests --plugin-dir ./claude-plugin --filter "triggering" --threads 4 --log ./logs
skill-bench list # List discovered tests (from `cases/` dir)
```

Key points:

- `--plugin-dir ./claude-plugin` is required for MCP server and skill loading
- Test cases are in `tests/<skill>/<test>.toml`
- Session logs are written to `./logs/` when `--log` is provided

## Development & Formatting

- Format all files (`.md`, `.json`) using Prettier: `npx prettier --write .` (or via `mise run fmt`).
Expand Down
23 changes: 23 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
[package]
name = "patent-kit"
version = "0.1.0"
edition = "2024"
rust-version = "1.94"
description = "Patent investigation MCP server with CLI interface"

[dependencies]
tokio = { version = "1", features = ["full"] }
clap = { version = "4", features = ["derive"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
anyhow = "1"
rusqlite = { version = "0.31", features = ["bundled"] }
csv = "1.3"
rmcp = { version = "0.16", features = ["server", "macros", "transport-io"] }
schemars = "1.2"
async-trait = "0.1"
thiserror = "2"
google-patent-cli = { git = "https://github.com/sonesuke/google-patent-cli", branch = "main" }
arxiv-cli = { git = "https://github.com/sonesuke/arxiv-cli", branch = "main" }
directories = "6"
toml = "0.8"
8 changes: 8 additions & 0 deletions claude-plugin/.mcp.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"mcpServers": {
"patent-kit": {
"command": "patent-kit",
"args": ["mcp"]
}
}
}
115 changes: 115 additions & 0 deletions claude-plugin/skills/claim-analyzing/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
---
name: claim-analyzing
description: |
Analyzes screened patents by decomposing claims into elements and comparing against product features.

Triggered when:
- The user asks to:
* "evaluate the patent"
* "analyze claims"
* "perform claim analysis"
* "analyze claim elements"
* "analyze claim similarities"
* "compare product features against patent elements"
- The user mentions:
* "claim analysis" with "patent" or "elements"
* "similarity" with "elements" or "claims"
- `patents.db` exists with screened and indexed patents
---

# Claim Analysis

## Purpose

Analyze screened patents by decomposing claims into elements, comparing product features against patent elements, and recording similarity results.

## Prerequisites

- `patents.db` must exist with screened and indexed patents (from screening skill)
- `features` table must exist with product features populated

## Constitution

> [!IMPORTANT]
> When instructed to call an MCP tool, call it directly using the tool name. **NEVER** use Bash to invoke MCP tools — the MCP server is already connected and tools are available directly. Do NOT construct JSON-RPC messages or use `echo | patent-kit mcp`.

### Core Principles

**Element-by-Element Analysis (The Golden Rule)**:

- Every claim analysis MUST test the target invention against the reference patent element by element
- Break down inventions into Elements A, B, C
- Find references disclosing A AND B AND C for anticipation (Novelty)
- Do not rely on "general similarity"

**Descriptive Technical Language**:

- Avoid legal assertions ("invalid", "valid", "Does not satisfy")
- Use descriptive technical language for analysis notes

**Mechanical Claims Recording**:

- Claims are already stored in the database by `index_patents` — read them via `get_claims`
- Do NOT re-generate or summarize claim text

## Skill Orchestration

### Execute Claim Analysis

**Do NOT delegate to subagents (Agent tool)** — call MCP tools directly from this session. Do NOT use Bash or Skill to invoke MCP tools.

**Process**:

1. **Get Next Patent**:
- Call the `get_unanalyzed` MCP tool directly (no parameters):
- If it says "All patents have been analyzed" → Analysis is complete
- Otherwise → Returns 1 patent with `needs: "elements"` or `needs: "similarities"`

2. **If needs: "elements"**:

a. Call `get_claims` with `decomposed: false` to get claims that have NOT been decomposed yet

b. For EACH claim:
1. Read the claim text
2. Decompose into constituent elements based on the means/steps described in the claim text
3. Call `record_elements`:
- `elements`: [{ patent_id, claim_number, element_label, element_description }, ...]

**CRITICAL Rules for Element Decomposition**:
- Decompose ALL claims including dependent claims — do NOT skip dependent claims
- Do NOT reference `specification.md` during decomposition — decompose based on claim text alone
- Cut elements by the number of means/steps in the claim — do NOT force a specific number of elements

c. **Go back to step 1** (get next patent — may return the same patent with needs: "similarities")

3. **If needs: "similarities"**:

a. Call `get_product_features` to retrieve product features

b. Call `get_elements` with `analyzed: false` to get elements that have NOT been analyzed yet

c. For EACH element:
1. Check if a matching product feature exists
2. If feature NOT found: present to the user using `AskUserQuestion` (max 4 questions per call, group by unique functionality)
3. If positive: Call `record_product_feature` with `presence='present'`
4. If negative: Call `record_product_feature` with `presence='absent'`

d. Determine similarity level: `Significant`, `Moderate`, or `Limited`

e. Call `record_similarities`:
- `similarities`: [{ patent_id, claim_number, element_label, similarity_level, analysis_notes }]

f. Use `Skill: legal-checking` with request "Check the following analysis notes for legal compliance: <analysis_notes>"
- Revise if violations found

g. **Go back to step 1** (get next patent)

## State Management

### Initial State

- Patents marked as `relevant` without corresponding elements/similarities entries exist

### Final State

- No patents marked as `relevant` without corresponding elements/similarities entries (all analyzed)
Loading
Loading