Skip to content

snake-eyes: classification signal extraction and classify_signals method #5

Description

@jflowers

How to use this issue with /opsx-propose

/opsx-propose classify-signals

Use this entire issue body as the change description. Do not ask clarifying questions. Every decision is already made below. If something is still unspecified, apply these defaults in order:

  1. Emit raw signals only. Never compute contractual/incidental/ambiguous labels. Never apply Gaze's 5-signal scoring formula, tier boost, or contradiction penalty.
  2. Lift gaze-py classify/signals/*.py extractors; do not lift classify/engine.py.
  3. Match classify_signals protocol v1.1.0 field names exactly.

OpenSpec change name: classify-signals

Depends on: #4 (analysis-methods) merged (analyze_path / detector + FunctionRecord + discovery).

Independent of: test-mapping. Do not wait on that issue; do not implement test mapping here.


Context

Gaze's Go core owns classification scoring (CC-001–CC-006). snake-eyes implements the optional classify_signals method so Gaze can run that formula on Python functions.

gaze-py already extracts five mechanical signals but then scores them locally. Lifting the engine would reintroduce drift. Lift extractors only.

Add runtime dependency: astroid>=3.0,<5 (needed for caller counts). If astroid fails to infer a module, caller_count is 0 and caller_signal may return None — not an RPC error.


Protocol

Request:

{"root_path": "/abs/path", "patterns": ["./..."]}

Result:

{
  "signals": [
    {
      "function": "divide",
      "package": "math_utils",
      "side_effect_type": "ErrorReturn",
      "source": "docstring",
      "weight": 15,
      "reasoning": "docstring mentions ZeroDivisionError"
    }
  ]
}

Field names are function and package (not name). weight is int. reasoning may be omitted if empty; prefer always including a short string.

Allowed source strings (exact):

source extractor
interface ABC / typing.Protocol base
visibility public/private naming, __all__
caller_count inbound call count
naming_convention function name vs effect type
docstring docstring keywords vs effect type

Do not invent other source strings. Do not send source: "type_annotation" unless you also implement a sixth extractor — do not. Stay at five.

Flip capabilities.classify_signals to true. Leave test_mapping and streaming as they were.


What to build

1. Lift extractors

From mpeter/gaze-py src/gaze_py/classify/signals/ into src/snake_eyes/signals/:

  • interface.py
  • visibility.py
  • caller.py
  • naming.py
  • docstring.py

Preserve copyright headers.

Adapt imports to snake_eyes.analysis.effects.SideEffectType. If an extractor switches on effect types, add cases for the 10 new types using the closest existing branch (e.g. GeneratorYield treated like ReturnValue for naming/docstring; MonkeyPatch like ReflectionMutation; DescriptorEffect / ResourceManagement like other P2 mutations). Do not leave KeyError on new types — return None (no signal) if no keyword/prefix applies.

Keep the gaze-py weights as-is. They are inputs Gaze expects to be stable. Do not retune weights to pass a local classifier test (there is no local classifier).

2. Caller count

src/snake_eyes/analysis/inference.py

def count_callers(root_path: str, module: str, func_name: str) -> int: ...

Use astroid to parse the project (or a Manager). Count Call nodes whose inferred callee matches func_name in module. If astroid raises or returns Uninferable: return 0.

Do not require a running environment with the project installed; operate on files on disk.

3. Adapter (new, do not lift engine.py)

src/snake_eyes/signals/adapter.py

def extract_signals(root_path: str, patterns: list[str]) -> list[dict]: ...

Algorithm:

  1. functions = analyze_path(root_path, patterns)
  2. For each function, for each effect:
    • Run all 5 extractors with the data you have (bases, name, docstring from AST, caller count, effect type).
    • Each extractor returns None or a structure with weight + reason.
    • For every non-None result, append a signal dict with that effect's type as side_effect_type.
  3. Functions with zero effects produce zero signals.
  4. Do not dedupe across extractors; multiple sources for the same effect are expected.

Do not sum weights. Do not clamp. Do not assign a label.

4. Server

Register classify_signals. Missing root: -32602.


Tests required

Use small AST fixtures; mock count_callers where needed.

  • ABC subclass method → source=interface, weight=30
  • Protocol subclass → interface signal fires
  • Plain class → no interface signal
  • def public_fn vs def _private → visibility weights differ (assert both fire or the private one is weaker per lifted implementation; do not change weights)
  • caller_signal(0) / (5) / (20) — if the lifted function uses buckets, assert those exact weights
  • naming: get_foo + ReturnValue positive; a name that gaze-py treats as negative still negative
  • docstring containing "returns" + ReturnValue → docstring signal
  • adapter: a fixture function with a raise and a docstring mentioning the exception → at least one signal with side_effect_type=ErrorReturn or ErrorSignal
  • JSON-RPC e2e: initialize has classify_signals: true; method returns a signals array
  • Negative test: grep the production code under src/snake_eyes/signals/ for contractual, incidental, ambiguous as assigned labels — must not appear as computed outputs. (The words may appear in comments forbidding them.)

Coverage strategy (Constitution IV)

Layer Target
each signals/*.py extractor 90%+
adapter.py 95%+
inference.py 85%+ (astroid failure path must be tested with a monkeypatch)
Project gate 85%

Out of scope

  • gaze-py classify/engine.py
  • Test pairing
  • Changing Gaze formula constants
  • Emitting classification on analyze results

Done when

  • classify_signals RPC works
  • capability flag true
  • no local classification labels
  • copyright retained on lifted signal files

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions