Skip to content

feat: integrate Perfetto SQL stdlib modules (P0+P1) - #12

Closed
mufans wants to merge 98 commits into
mainfrom
feat/stdlib-modules
Closed

feat: integrate Perfetto SQL stdlib modules (P0+P1)#12
mufans wants to merge 98 commits into
mainfrom
feat/stdlib-modules

Conversation

@mufans

@mufans mufans commented May 12, 2026

Copy link
Copy Markdown
Owner

Summary

集成13个Perfetto SQL Standard Library模块,大幅增强SmartInspector的trace分析能力。

P0 模块(核心能力)

  • 锁竞争分析 (android.monitor_contention) — Java synchronized锁竞争检测与阻塞链分析
  • Binder事务分析 (android.binder + binder_breakdown) — IPC延迟分解,client/server端追踪
  • 启动分析增强 (android.startup.*) — TTID/TTFD指标、启动瓶颈自动分解
  • GC分析 (android.garbage_collection) — GC pause检测、回收量统计
  • ANR分析 (android.anrs) — ANR事件检测与期间主线程slice分析
  • Slice CPU增强 (slices.cpu_time + slices.time_in_state) — Slice级精确CPU时间和线程状态分布

P1 模块(场景增强)

  • 逐帧指标 (android.frames.per_frame_metrics) — per-frame overrun/cpu_time/ui_time/jank分级
  • 输入延迟分解 (android.input) — dispatch→delivery→ACK全链路延迟
  • 调度延迟 (sched.latency) — Runnable→Running等待时间统计
  • OOM+RSS/Swap (android.memory.process + lmk) — OOM分数追踪、LMK kill事件
  • 精确CPU利用率 (linux.cpu.utilization) — 进程/线程级频率加权CPU利用率
  • 堆分析增强 (android.memory.heap_graph.*) — 堆统计、类聚合、支配树
  • SF帧匹配 (android.surfaceflinger) — App与SurfaceFlinger帧时间线关联

Stats

  • 13 commits, +1882 lines
  • 14 files changed (9 new collectors, 3 enhanced existing, 1 updated init)
  • All modules use INCLUDE PERFETTO MODULE with stdlib tables/views
  • Follow existing mixin pattern with try/except error handling

Test Plan

  • Verify each new collector with real Perfetto traces
  • Run existing test suite
  • Test with traces containing ANR/GC/Binder events

mufans and others added 30 commits April 6, 2026 13:21
refactor: merge post-refactoring fixes and improvements into master
refactor: merge P0 fixes and architecture improvements
refactor: P0 quality fixes, collector enhancements, and tooling improvements
…accuracy

- Extract actual method names from anonymous inner class names (e.g. Outer$1)
  and stack traces, storing the enclosing method as context_method
- Tag XML layout inflate calls as [XML布局] with call count and total time
- Reorder report sections to put attribution data first, preventing truncation
  of core input when total content exceeds token budget
- Add completeness check rule to report prompt to avoid missing entries
- Switch CLI entry point to graph module and add hatchling build config

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
refactor: 工具增强与匿名内部类归因支持
Covers plugin system, URL API, postMessage, trace_processor HTTP mode,
proposed local Web Server bridge architecture, and integration plan
with existing LangGraph pipeline.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Document the perfetto-plugin/build.sh usage, prerequisites, build steps,
and the --skip-clone flag for incremental builds.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…fetto UI

Add SI Bridge plugin for Perfetto UI enabling interactive frame-level
analysis: users select a time range in Perfetto UI and get AI-powered
analysis with source attribution via WebSocket bridge.

New components:
- perfetto-plugin/: TypeScript plugin + build.sh for self-hosted Perfetto UI
- frame_analyzer agent: queries slice data, runs attribution, LLM analysis
- bridge_server: serves Perfetto UI + WebSocket bridge to CLI agent
- /open, /close, /frame CLI commands
- HDC command-line research doc for HarmonyOS support

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
feat: Perfetto UI 交互帧分析 + 工具链增强 + 文档更新
Replace report example with actual 2026-04-15 analysis output, update
quick start examples, improve table alignment, clean up Todo checkbox
syntax, and remove unreferenced screenshot.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
docs: update README and add Perfetto UI interactive analysis docs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…te analysis

Add a fast path in attributor that skips LLM for straightforward java
type searches using direct Glob→Grep→Read. This reduces latency and
cost for cases without anonymous inner classes or unknown methods.

Add thread state analysis to perfetto collector that shows Running/S/D
distribution per SI$ slow slice, helping distinguish slow code from
blocked threads.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
# Conflicts:
#	src/smartinspector/agents/attributor.py
…r class handling

- Fix SI$block# slices (dur≈0) being squeezed out by LIMIT 50 SQL query:
  split into two queries prioritizing SI$ slices over system slices
- Fix anonymous inner class method resolution: always set context_method
  from _extract_method_from_anonymous, only override method_name with stack
- Add lightweight LLM analysis (_analyze_snippets) for fast-path results
  to replace raw source code with analysis text in source_snippet
- Fix _build_frame_hints: sort SI$ slices by effective dur DESC, aggregate
  repeated calls with cumulative duration and call count
- Fix dur extraction from SI$block# tag suffix (#NNms) in frame_analyzer
- Filter out negligible dur slices (< 0.01ms) from attribution
- Show "(匿名内部类, 定义在 method 内)" label for inner class results
- Add debug logging for Step 3 LLM input/output

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The previous query required thread_state entries to be entirely within
the slice window (ts >= start AND ts + dur <= end), which missed almost
all entries since thread_state records commonly straddle slice boundaries
(e.g. a long Running state starting before the slice). Changed to
overlap-based calculation: SUM(MIN(end, slice_end) - MAX(start, slice_start))
with proper handling for dur < 0 (still-running entries).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two bugs in collect_thread_state state normalization:

1. State distribution used overwrite instead of accumulation: when
   multiple raw states mapped to the same normalized name (e.g. R and R+
   both → Running), the second would overwrite the first instead of
   summing. This caused incorrect percentages when both R and R+ appeared.

2. S+ state was not mapped to Sleeping: S+ (interruptible sleep,
   preemptible) is common in Android apps but was left as-is instead of
   being normalized to "Sleeping".

Added comprehensive tests for normalization, overlap calculation, and
filter conditions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…s, and perfetto auto-degradation

P1-4: Add dependency reference search in Attributor Agent — after finding
target files, extract import statements for project-internal classes and
R.layout XML references, read related files, and include as dependency_context
for richer AI diagnosis.

P1-5: Add package_list cold-start fallback — new _resolve_target_process()
method tries process table first, falls back to package_list table for UID
lookup when process table is empty during cold start. Annotates view_slices
with target process info and includes resolved process in metadata.

P1-6: Add SELinux cat-pipe bypass — when stdin-pipe perfetto config fails,
fallback to pushing config file and using cat pipe (cat config | perfetto -c -).

P1-7: Add perfetto collection auto-degradation — when config mode fails
entirely, degrade to simpler cmdline mode with just atrace categories.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add design document for thread state blocking analysis and redesign.
Added project rules, overview, structure, conventions, logging rules, architecture, commands, configuration, known issues, and build notes to CLAUDE.md.
…ead_state for blocking details

Replace sched-based thread state inference with __intrinsic_thread_state
table queries that expose blocked_function, io_wait, and waker_utid fields.
This enables actionable blocking analysis instead of generic Running/Sleeping
labels. Falls back to legacy thread_state table when __intrinsic_thread_state
is unavailable.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
openclaw-workspace and others added 25 commits May 5, 2026 17:26
Both --src and --source-dir are accepted (--src is the primary name).
Updated CLAUDE.md docs to use --src.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DeepSeek thinking mode adds `reasoning_content` to AIMessage responses.
On subsequent API calls, if the message history contains these but gets
trimmed, the API rejects with 'reasoning_content must be passed back'.

Fix: strip reasoning_content from every LLM response before appending
to message history, and also from trimmed messages during window
trimming.

Also relax fast-path: allow anonymous inner classes ($ in class_name)
to use deterministic search. extract_class() already strips the $ suffix
for glob search, and context_method handles method-level grep.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When getResourceEntryName() fails on the app Context (e.g. for
android.R layouts like simple_list_item_1), try Resources.getSystem()
as fallback before falling back to hex ID. This produces readable
tag names like 'SI$inflate#simple_list_item_1#RecyclerView' instead
of 'SI$inflate#0x109010a#null'.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
For SI$block slices with pure anonymous classes (e.g. MainActivity$1),
extract_method returns 'unknown' and _extract_method_from_anonymous
finds no enclosing method. This blocks fast path and causes LLM path
to produce vague 'unknown' attribution.

Fix: when method_name is 'unknown' for an anonymous class block,
walk the stack trace to find the caller method from the same class
(e.g. 'loadAndDisplayItems' from 'MainActivity.loadAndDisplayItems').
Set it as context_method and use 'run' as method_name, enabling
fast-path deterministic search to locate the source code.

Stack trace used:
  DataRepository.buildPayload → DataRepository.loadItemsJson →
  MainActivity.loadAndDisplayItems → MainActivity$1.run

Now correctly attributes to MainActivity.loadAndDisplayItems.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The previous commit's edit replaced extract_class with
_extract_caller_from_stack, leaving extract_class's body as dead code
after a return statement. Restore the def line.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…attribution, and report section ordering

- Fix metadata query: column 'key' -> 'name' (Perfetto metadata table schema)
- Fix heap_graph_field error: add pre-check for heap_graph_object table existence
- Fix thread_state 0 entries: _resolve_main_utid now tries 3 strategies
  (name='main', name=package_name, lowest tid in target process)
- Remove structured output probe in attributor to prevent DeepSeek thinking mode
  activation and reasoning_content errors
- Add CPU hotspot extraction to attribution pipeline for background thread attribution
- Reorder report sections to match CLAUDE.md priority (thread state before frame timeline)
- Add [CPU热点] tag in report formatter

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
feat: metric QA, startup analysis, attribution improvements, and collector bug fixes
- Create si_tag.py with SITag dataclass and parse_si_tag() function
  that does single-pass parsing of all SI$ tag types (block, RV,
  inflate, view, handler, db, net, img, touch, default)
- Rewrite extract_class(), extract_method(), extract_fqn() as thin
  wrappers around parse_si_tag(), eliminating ~280 lines of duplicated
  parsing logic from attribution.py
- Move _split_fqn_method, _extract_method_from_anonymous, system class
  constants to si_tag.py as single source of truth
- Add SITag.is_system and SITag.is_system_method properties for
  declarative system class checking
- Rewrite is_system_class(), is_system_method(), classify_search_type()
  to use parse_si_tag() internally
- Add 90 new unit tests:
  - test_si_tag.py: 63 tests covering all SI$ tag types, edge cases,
    backward-compatible wrappers, and integration
  - test_deterministic.py: 27 tests for severity classification,
    call-chain distribution, RV hotspots, CPU hotspots, thread state,
    IO slices, memory analysis, frame budget detection, compute_hints

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…own)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…startup.*)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… slices.time_in_state)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…or tree (android.memory.heap_graph.*)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ger)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@mufans
mufans force-pushed the feat/stdlib-modules branch from 658b5f0 to 5fbf427 Compare May 12, 2026 02:13
@mufans mufans closed this May 12, 2026
@mufans
mufans deleted the feat/stdlib-modules branch May 12, 2026 02:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant