Duplicate-process guard misses symlinked and relative-path launches (normpath vs resolve mismatch in match_agent_run)

[huangzesen]

Summary

match_agent_run compares os.path.normpath(tail) against a target working directory, but every caller passes a fully symlink-resolved path (Path.resolve()). normpath is purely lexical and never resolves symlinks, so an agent launched via a symlinked path (e.g. macOS /tmp -> /private/tmp) or a relative path is invisible to the duplicate-process guard, the doctor's process check, and the refresh watcher's stale-duplicate cleanup. Adding an os.path.realpath fallback comparison to all three synced copies of the matcher closes the gap.

Narrative

LingTai defends against two agent processes running in the same working directory with layered checks. The innermost layer is a flock on .agent.lock, which prevents actual data corruption. But a duplicate Python process that loses the flock race can still linger in ps and confuse users and tooling, so there is an outer, human-friendly layer: before booting, _check_duplicate_process in src/lingtai/cli.py scans ps -eo pid=,command= and refuses to start if another lingtai ... run <dir> process for the same directory is already alive. The same matching logic is used by the doctor skill's process-consistency check and by the refresh watcher that lifecycle.py spawns to relaunch an agent after a refresh.

All three sites share one matcher, match_agent_run, whose canonical copy lives in src/lingtai_kernel/process_match.py:

def match_agent_run(cmdline: str, working_dir: str) -> str | None:
    target = os.path.normpath(working_dir)
    ...
                tail = cmdline[idx + len(token):].strip()
                if tail and os.path.normpath(tail) == target:
                    return label

(src/lingtai_kernel/process_match.py:19,29)

The matcher was deliberately made conservative — the docstring explains the anchoring rules that prevent grep lingtai run /a/foo from matching — and path comparison was done with normpath because that is enough to reconcile trailing slashes and .. segments between the ps text and the target. What the design missed is that the two strings being compared come from different worlds. The tail is whatever argv text the original launcher typed, frozen into the process table. The target is what the current checker computes — and every caller resolves it first:

• src/lingtai/cli.py:181 — abs_dir = str(working_dir.resolve()) inside _check_duplicate_process, and the run entry point already does working_dir = args.working_dir.resolve() (src/lingtai/cli.py:414).
• src/lingtai/intrinsic_skills/lingtai-doctor/scripts/doctor.py:759 — agent_dir = agent_dir.expanduser().resolve(), later passed as agent_str to match_agent_run at line 431.
• src/lingtai_kernel/base_agent/lifecycle.py — the refresh-watcher script embeds wd from the agent's (resolved) working directory and a third stdlib-only copy of match_agent_run (lines 1003–1017), used by _is_same_agent_run to decide whether a blocking PID is safe to SIGTERM.

Path.resolve() resolves symlinks; os.path.normpath does not. So the guard silently fails exactly when the launch path and the resolved path differ. This is not exotic: on macOS /tmp is a symlink to /private/tmp, and home directories, NFS mounts, and dotfile managers routinely introduce symlinks. Concretely, a user starts an agent with lingtai-agent run /tmp/agents/alice. ps shows the tail /tmp/agents/alice. Later (perhaps after a wedged teardown) they run the same command again from a shell where the path got canonicalized, or a tool invokes it as /private/tmp/agents/alice. The new process computes target = /private/tmp/agents/alice, compares it with normpath("/tmp/agents/alice") = "/tmp/agents/alice", gets no match, and boots straight into the flock — losing the friendly "another lingtai agent is already running, PID ..." message the guard exists to produce. Relative launches (lingtai-agent run ./alice) fail the same way, since the ps tail normalizes to alice, which never equals an absolute target.

The downstream effects are worse than a missing error message. The doctor's collect_process (doctor.py:431) reports "no lingtai process found — a fresh heartbeat would make this inconsistent" even though the agent is running and healthy, sending the operator down a false trail. And the refresh watcher's _cleanup_stale_duplicate (lifecycle.py, embedded script) uses the same matcher as a safety check before terminating a stale duplicate — _is_same_agent_run returns False for a symlink-launched duplicate, the cleanup records skipped_not_same_agent, and the watcher refuses to kill a genuinely stale process it is otherwise certain about, so the refresh fails permanently and pages the human.

Better looks like: the matcher keeps its cheap lexical comparison as the fast path, and falls back to os.path.realpath on both sides when the lexical comparison misses. realpath resolves symlinks the same way Path.resolve() does, so the two worlds finally agree. The fallback should only fire for absolute tails — resolving a relative ps tail against the checker's cwd would compare against the wrong base directory and could invent false matches; relative launches remain a documented residual limitation.

Current behavior

• src/lingtai_kernel/process_match.py — match_agent_run compares os.path.normpath(tail) == os.path.normpath(working_dir) (lines 19 and 29). No symlink resolution.
• src/lingtai/cli.py — _check_duplicate_process (line 171) passes str(working_dir.resolve()) (line 181) as the target. A duplicate launched via a symlinked or relative path is not detected; the launch proceeds to the flock with no friendly diagnostic.
• src/lingtai/intrinsic_skills/lingtai-doctor/scripts/doctor.py — a synced stdlib-only copy of the matcher (lines 47–71, kept in sync per its docstring) is called from collect_process (line 431) with a resolved agent_dir (line 759). Symlink-launched agents produce a spurious WARN: no lingtai process found.
• src/lingtai_kernel/base_agent/lifecycle.py — the refresh-watcher script string embeds a third copy (lines 1003–1017); _is_same_agent_run (line 1018) uses it to gate _cleanup_stale_duplicate, which then refuses (skipped_not_same_agent) to terminate a stale duplicate whose ps cmdline shows an unresolved path.
• tests/test_process_match.py — MATCH_CASES (lines 18–36) covers anchoring, prefix-sibling, and trailing-slash cases, and is replayed against all three copies, but contains no symlink or relative-path cases.

Detailed implementation plan

1. Canonical matcher — in src/lingtai_kernel/process_match.py, extend match_agent_run:

   target = os.path.normpath(working_dir)
   target_real = os.path.realpath(working_dir)
   ...
   tail = cmdline[idx + len(token):].strip()
   if tail:
       if os.path.normpath(tail) == target:
           return label
       if os.path.isabs(tail) and os.path.realpath(tail) == target_real:
           return label

   Notes: compute target_real once, outside the token loop. Call os.path.realpath(tail) only after the lexical comparison misses and only for absolute tails, so the syscall cost is paid solely for candidate tails that already passed token/anchoring checks (rare in a full ps listing). realpath on a nonexistent path degrades to lexical normalization on POSIX, so purely textual test fixtures keep working. Update the docstring: document the realpath fallback and add relative-path launches to the "residual limitation" paragraph.

2. Doctor copy — apply the identical edit to match_agent_run in src/lingtai/intrinsic_skills/lingtai-doctor/scripts/doctor.py (lines 47–71). This file is copied into agent skill bundles and cannot import kernel modules, so it must stay a literal sync (its docstring already says so).

3. Refresh-watcher copy — apply the identical edit to the embedded match_agent_run source string in src/lingtai_kernel/base_agent/lifecycle.py (lines 1003–1017). Keep it stdlib-only (os is already imported by the watcher script). The existing extraction-based test test_refresh_watcher_copy_matches_canonical_matrix will exercise the new code path automatically.

4. Tests — in tests/test_process_match.py:

   • Add relative-tail rows to the static MATCH_CASES matrix asserting the conservative behavior, e.g. ("lingtai-agent run agent", "/a/agent", None) — relative tails must not realpath-match against the checker's cwd.
   • Add a new symlink-based parametrized test (see Testing plan) that builds real symlinks under tmp_path and replays them against all three matcher copies, mirroring how the matrix tests already fan out (_load_doctor_module, _watcher_match_agent_run).

5. No schema, data-structure, or back-compat changes. The change strictly widens the match set to paths that denote the same directory; every string that matched before still matches. No migration needed.

Risks and alternatives

• False positives via symlink equivalence — the realpath fallback matches a ps tail that is a different string for the same directory. That is precisely the definition of a duplicate, so this is a correctness gain, not a risk. A crafted non-LingTai cmdline shaped like a launch could newly match through a symlink, but the same residual limitation already exists for the lexical path and is documented in the docstring.
• Relative tails resolved against the wrong cwd — guarded by os.path.isabs(tail). Without the guard, realpath("alice") in the checker would resolve against the checker's cwd and could match (or mis-match) unrelated directories. Relative launches stay undetected, which is the status quo, now documented.
• Per-line syscall cost in the ps scan — avoided by keeping normpath as the fast path and hoisting target_real out of the loop; realpath(tail) runs only for tails that already passed token anchoring.
• Copy drift — three synced copies is the existing design (the doctor script must be import-free; the watcher is a self-contained python -c script). The shared MATCH_CASES replay tests are the drift guard; the new symlink test extends that guard rather than introducing a new sync mechanism.
• Alternative: resolve at the callers instead — callers cannot resolve the ps tail before calling the matcher because tail extraction happens inside it; pushing extraction out would be a larger refactor across three copies for no behavioral gain.
• Alternative: compare os.stat().st_ino/st_dev — robust against bind mounts too, but requires both paths to exist at check time, fails messily for dead paths, and is harder to keep stdlib-simple in the embedded copies. realpath matches the callers' existing Path.resolve() semantics exactly, which is the actual invariant being restored.

Testing plan

All in tests/test_process_match.py:

• test_match_agent_run_symlinked_launch_path — parametrized over the three matcher implementations (canonical import, _load_doctor_module().match_agent_run, _watcher_match_agent_run(tmp_path)). Create real = tmp_path / "real" / "agent" and link = tmp_path / "link" with link.symlink_to(tmp_path / "real"); assert match(f"/usr/local/bin/lingtai-agent run {link}/agent", str(real.resolve())) == "console" and the symmetric case (resolved tail, symlinked target) also matches.
• test_match_agent_run_symlink_to_different_dir_no_match — symlink pointing at a sibling directory must still return None, proving the fallback does not loosen the prefix-sibling protection.
• New MATCH_CASES rows: ("lingtai-agent run agent", "/a/agent", None) and ("python -m lingtai run ./agent", "/a/agent", None) — relative tails never match an absolute target (conservative cwd behavior), replayed against all three copies by the existing matrix tests.
• test_cli_duplicate_process_detects_symlinked_launch — mirror test_cli_duplicate_process_detects_console_script: build the symlink under tmp_path, fake ps output containing the symlinked path, call _check_duplicate_process with the real directory, assert SystemExit.
• test_doctor_collect_process_detects_symlinked_launch — mirror test_doctor_collect_process_detects_console_script with a symlinked path in the faked ps stdout; assert the OK/lingtai process found finding.

Generated by a Claude Fable 5 deep review of the lingtai-kernel codebase.