Skip to content

feat(agent): add filesystem flag and log helpers#292

Open
rice-riley wants to merge 2 commits into
mainfrom
agent-go-flags-215
Open

feat(agent): add filesystem flag and log helpers#292
rice-riley wants to merge 2 commits into
mainfrom
agent-go-flags-215

Conversation

@rice-riley

@rice-riley rice-riley commented Jun 30, 2026

Copy link
Copy Markdown
Member

feat(agent): add filesystem flag and log helpers

Summary

This PR adds the Go agent's host-path, completion-flag, log-path, and log-retention helpers in a new internal/flags package.

It is the filesystem/flags half of #215. Package history is intentionally handled in a separate PR so the persistence and corruption-recovery behavior can be reviewed independently.

Relates to #215

What changed

  • Added a configurable flags.Layout for resolving state, flag, history, copied-step, and log directories beneath the mounted host root.
  • Added deterministic UTC log filenames and a matching glob helper, with timestamps passed explicitly by callers.
  • Added flags.Store for:
    • deriving per-step completion-flag paths;
    • writing step and control flags with contained-path validation;
    • evaluating flags into structured run/skip decisions;
    • returning errors instead of printing from filesystem helpers.
  • Added deterministic log cleanup that retains the newest configured number of regular files (5 by default).
  • Added path validation to prevent package names, versions, and step paths from escaping their intended host directories.
  • Added focused Ginkgo coverage for path construction, fingerprint stability, input changes, flag policy, control flags, invalid paths, and log retention.

Go-native design choices

This package introduces two new conventions because the Go agent did not yet have an established filesystem seam:

  • Layout receives resolved roots from process configuration instead of reading environment variables inside helpers. This keeps path construction deterministic and easy to test.
  • Store owns flag I/O and returns a Decision with a typed Reason. The eventual controller can decide how to log each outcome without coupling this low-level package to Python's print strings.

Times are explicit inputs rather than hidden time.Now() calls, filesystem failures retain operation and path context, and flag writes use private file permissions.

Compatibility note

The completion marker is intentionally Go-native rather than byte-compatible with the Python agent. It hashes a canonical JSON representation of the step path, arguments, and accepted return codes with SHA-256, producing a path-safe hexadecimal filename.

As a result, completion flags created by the Python agent are not recognized by this implementation. The first Go-agent execution after cutover will treat those steps as incomplete and run them again. This is deliberate for the rewrite and avoids carrying Python list-repr semantics into the Go API.

The host directory roots remain /etc/skyhook and /var/log/skyhook by default, while allowing the caller to supply different resolved roots.

Testing

  • make unit-tests — 142 specs passed, including 30 flags specs
  • make lint — 0 issues
  • make vet

Documentation

No user documentation changes are included because these helpers are not wired into the agent executable yet. The controller/entrypoint integration will land separately and will document any user-visible behavior at that point.

Checklist

  • I am familiar with the Contributing Guidelines.
  • My commits are signed off (git commit -s) per the DCO.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@rice-riley rice-riley self-assigned this Jun 30, 2026
@rice-riley rice-riley requested a review from a team June 30, 2026 19:28
@github-actions github-actions Bot added component/operator Skyhook operator (controller-manager) component/agent Skyhook agent (package executor) component/ci CI workflows, GitHub Actions, and repo tooling labels Jun 30, 2026
@coderabbitai

coderabbitai Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

A new agent/go/internal/flags package is added, implementing path layout resolution (Layout), per-step log file naming and retention (CleanupOldLogs), and a completion-flag Store that fingerprints steps, persists flag files, and determines whether a step should run via a Decide policy function based on flag existence, always-run overrides, stage type, and idempotence settings. A corresponding Ginkgo/Gomega test suite covers these behaviors.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: new filesystem helpers for flags and logs in the agent.
Description check ✅ Passed The description is detailed and directly matches the PR's filesystem flag and log helper changes.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch agent-go-flags-215

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@agent/go/internal/flags/flags.go`:
- Around line 66-79: Wrap the bare errors returned from Path(), Mark(), and
Check() so callers can tell which step failed; instead of returning helper
errors directly from validatePackagePath, describeStep, fingerprint, and the
store checks, return contextual errors with fmt.Errorf("...: %w", err). Update
the affected logic in Path, Mark, and Check to preserve the original error while
adding operation-specific context for validation, fingerprinting, and store
lookup/write failures.
- Around line 100-113: The current containment check in Store.Write and the flag
validation in Store.Check are only lexical and can be bypassed by symlinked
directories or flag files under s.dir. Update both paths to reject symlink
traversal by resolving the final path safely and verifying each existing
component/file with symlink-aware checks before mkdir/write or stat/accept. Make
sure Write does not follow a symlinked parent or target when creating the flag,
and Check does not trust os.Stat alone to mark a symlink target as a valid
completion flag.

In `@agent/go/internal/flags/layout.go`:
- Around line 48-55: The Layout root handling in NewLayout and the directory
helpers like StateDir and LogDir currently accept caller-supplied roots that can
resolve outside RootMount, violating the mounted-host-root contract. Add
validation/sanitization in NewLayout (and/or a shared helper used by StateDir
and LogDir) to reject absolute or path-traversal roots such as ../ or ../../
that would escape RootMount, and ensure the returned paths are always confined
beneath RootMount.
- Around line 99-104: LogFilePattern() is currently turning the literal stepPath
into a filepath.Glob pattern, which breaks valid names containing glob
metacharacters and can match unrelated files. Update LogFilePattern() and the
related cleanup logic in the same area so retention uses a literal prefix/suffix
check derived from packageLogDir()/LogFilePath() instead of globbing the raw
step path. Keep the pattern generation safe for names with [], * and ? by
matching on the exact log filename shape rather than embedding the full stepPath
in a glob.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 1627a6cc-8033-416a-a0a0-311fe854d9cc

📥 Commits

Reviewing files that changed from the base of the PR and between 2e18083 and 4be0fc0.

📒 Files selected for processing (4)
  • agent/go/internal/flags/flags.go
  • agent/go/internal/flags/flags_test.go
  • agent/go/internal/flags/layout.go
  • agent/go/internal/flags/logs.go

Comment thread agent/go/internal/flags/flags.go Outdated
Comment thread agent/go/internal/flags/flags.go Outdated
Comment thread agent/go/internal/flags/layout.go Outdated
Comment thread agent/go/internal/flags/layout.go Outdated
@rice-riley rice-riley force-pushed the agent-go-flags-215 branch from 4be0fc0 to b4af406 Compare June 30, 2026 19:44

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@agent/go/internal/flags/flags.go`:
- Around line 323-329: The writeFlagFile flow deletes the existing flag before
the replacement is safely persisted, so a failed create/write/close can lose a
valid flag. Update writeFlagFile to write the new contents to a same-directory
temporary file first, ensure the write and close succeed, and only then
atomically swap it into place when replace is true; keep the existing flag
untouched until that final step succeeds.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 5853f46f-9940-43ec-83a9-bad4af5171cd

📥 Commits

Reviewing files that changed from the base of the PR and between 4be0fc0 and b4af406.

📒 Files selected for processing (4)
  • agent/go/internal/flags/flags.go
  • agent/go/internal/flags/flags_test.go
  • agent/go/internal/flags/layout.go
  • agent/go/internal/flags/logs.go

Comment thread agent/go/internal/flags/flags.go Outdated
@rice-riley rice-riley force-pushed the agent-go-flags-215 branch from b4af406 to 004772c Compare June 30, 2026 19:56

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@agent/go/internal/flags/logs.go`:
- Around line 49-79: The log retention logic in logs.go is parsing the UTC
timestamp from each filename but the cleanup order in the candidate sorting
still uses ModTime(), which can keep the wrong files when mtimes differ or
match. Update the retention flow around the candidate slice and sort in the log
cleanup code to order files by the parsed timestamp extracted from the filename,
using path only as a tie-breaker if needed, and keep the existing regular-file
and parse validation intact.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 79de81e6-5b3f-41ae-b577-6ecc902b0fc3

📥 Commits

Reviewing files that changed from the base of the PR and between b4af406 and 004772c.

📒 Files selected for processing (6)
  • agent/go/internal/flags/flags.go
  • agent/go/internal/flags/flags_test.go
  • agent/go/internal/flags/layout.go
  • agent/go/internal/flags/layout_test.go
  • agent/go/internal/flags/logs.go
  • agent/go/internal/flags/logs_test.go

Comment thread agent/go/internal/flags/logs.go Outdated
@rice-riley rice-riley force-pushed the agent-go-flags-215 branch from 004772c to 9879b37 Compare June 30, 2026 20:09

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@agent/go/internal/flags/layout.go`:
- Around line 50-64: Wrap the helper failures in NewLayout, LogFilePath,
LogFilePattern, PrepareLogFile, and packageLogDir with operation-specific
context instead of returning the raw error. Update the error paths around
normalizeLayoutRoot and any other helper calls in those functions to use
fmt.Errorf with %w so callers can tell which step failed, while preserving the
original error for unwrapping. Keep the function names and their existing
behavior intact, only change the returned errors to include descriptive context.

In `@agent/go/internal/flags/logs_test.go`:
- Around line 47-48: The test is misleading because it uses os.Chtimes even
though CleanupOldLogs decides retention from the timestamp embedded in the
filename, not file mtime. Remove the Chtimes setup from the logs test and make
the fixture data reflect the filename-based timestamp behavior instead. Use the
CleanupOldLogs test case and the file creation loop in logs_test.go to locate
the affected setup, and keep the assertions focused on filename ordering rather
than modification times.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 6db033f8-0b36-4c36-907b-148334c5e32d

📥 Commits

Reviewing files that changed from the base of the PR and between 004772c and 9879b37.

📒 Files selected for processing (6)
  • agent/go/internal/flags/flags.go
  • agent/go/internal/flags/flags_test.go
  • agent/go/internal/flags/layout.go
  • agent/go/internal/flags/layout_test.go
  • agent/go/internal/flags/logs.go
  • agent/go/internal/flags/logs_test.go

Comment thread agent/go/internal/flags/layout.go
Comment thread agent/go/internal/flags/logs_test.go Outdated
@rice-riley rice-riley linked an issue Jun 30, 2026 that may be closed by this pull request
1 task
@rice-riley rice-riley force-pushed the agent-go-flags-215 branch from 9879b37 to 17c80ad Compare June 30, 2026 20:18

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@agent/go/internal/flags/flags.go`:
- Around line 193-247: The completion fingerprint in
describeStep/metadataFromRegularStep currently omits execution environment
fields, so changes to Env and OnHost can incorrectly reuse an old flag. Extend
stepMetadata to carry those fields, populate them in metadataFromRegularStep
from step.RegularStep, and include them in fingerprint’s JSON payload so the
hash changes whenever execution environment changes.

In `@agent/go/internal/flags/layout.go`:
- Around line 170-173: Update validatePathComponent in layout.go to explicitly
reject "." as an invalid single path component in addition to the existing
empty/local/base checks. Add a direct guard in validatePathComponent(name,
value) so package name/version inputs cannot be "." before they reach
filepath.Join and collapse into the parent namespace.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 5442dde7-96bc-4083-a693-5d3586d78438

📥 Commits

Reviewing files that changed from the base of the PR and between 9879b37 and 17c80ad.

📒 Files selected for processing (6)
  • agent/go/internal/flags/flags.go
  • agent/go/internal/flags/flags_test.go
  • agent/go/internal/flags/layout.go
  • agent/go/internal/flags/layout_test.go
  • agent/go/internal/flags/logs.go
  • agent/go/internal/flags/logs_test.go

Comment thread agent/go/internal/flags/flags.go
Comment thread agent/go/internal/flags/layout.go
Signed-off-by: Riley Rice <riceriley59@gmail.com>
@rice-riley rice-riley force-pushed the agent-go-flags-215 branch from 17c80ad to 425b625 Compare June 30, 2026 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/agent Skyhook agent (package executor) component/ci CI workflows, GitHub Actions, and repo tooling component/operator Skyhook operator (controller-manager)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEA]: Filesystem helpers (paths, flags, log paths) + history file

1 participant