Addressing PR comments - #10
Conversation
📝 WalkthroughWalkthroughThis pull request establishes comprehensive governance and architectural documentation for AuraLock. It adds three community standards documents (Code of Conduct, Contributing Guide, Security Policy), updates the README with new documentation pointers, and introduces 20 detailed system design documents covering architecture, requirements, components, workflows, security, observability, and technical roadmap. Changes
Estimated Code Review Effort🎯 2 (Simple) | ⏱️ ~18 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c9a903e502
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| pip install -e "[ui,dev]" | ||
| pip install -e "[benchmark,dev]" |
There was a problem hiding this comment.
The optional extras install commands are malformed for pip -e: "[ui,dev]" and "[benchmark,dev]" are not valid editable requirements because editable mode expects a local path or VCS URL. This breaks contributor setup for anyone following this section; e.g., python -m pip install -e "[ui,dev]" --dry-run fails with “not a valid editable requirement.” These should include the current project path (.") before extras.
Useful? React with 👍 / 👎.
Claude
AI
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 9
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
README.md (2)
97-97:⚠️ Potential issue | 🟡 MinorFix the git clone URL.
The trailing period will cause the git clone command to fail. Same issue as in CONTRIBUTING.md line 20.
🔧 Proposed fix
-git clone https://github.com/VoDaiLocz/Lock-ART. +git clone https://github.com/VoDaiLocz/Lock-ART.git🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@README.md` at line 97, Remove the trailing period from the git clone URL so the command succeeds: update the string "git clone https://github.com/VoDaiLocz/Lock-ART." to "git clone https://github.com/VoDaiLocz/Lock-ART" (same fix as applied for the CONTRIBUTING.md clone line).
218-230:⚠️ Potential issue | 🟡 MinorRemove duplicate documentation links.
The "Research Roadmap" and "System Design Hub" links appear twice: once under "Notes and Documentation" (lines 221-222) and again under "Project Standards" (lines 229-230). This duplication may confuse readers.
Consider organizing these sections more clearly:
- Keep research/design docs under "Notes and Documentation"
- Keep governance docs (CONTRIBUTING, CODE_OF_CONDUCT, SECURITY) under "Project Standards"
📝 Suggested organization
## Notes and Documentation - [Product Audit](docs/PRODUCT_AUDIT.md) - [Implementation Plan](docs/IMPLEMENTATION_PLAN.md) - [Research Roadmap](docs/RESEARCH_ROADMAP.md) - [System Design Hub](docs/system-design/README.md) ## Project Standards - [Contributing Guide](CONTRIBUTING.md) - [Code of Conduct](CODE_OF_CONDUCT.md) - [Security Policy](SECURITY.md) -- [Research Roadmap](docs/RESEARCH_ROADMAP.md) -- [System Design Hub](docs/system-design/README.md) - [Colab Benchmark Notebook](notebooks/AuraLock_LoRA_Benchmark_Colab.ipynb)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@README.md` around lines 218 - 230, Remove the duplicate documentation links by keeping "Research Roadmap" and "System Design Hub" only under the "Notes and Documentation" section and removing those same links from the "Project Standards" list; ensure "Project Standards" retains only governance docs like "Contributing Guide", "Code of Conduct", and "Security Policy" so the README entries for "Research Roadmap" and "System Design Hub" appear once and in the intended section.
🧹 Nitpick comments (2)
docs/system-design/13_ERROR_HANDLING.md (1)
3-8: Clarify that the custom exception classes are proposals, not current implementation.The document proposes custom exception classes (
InputValidationError,ProfileConfigError,PipelineExecutionError,ReportSerializationError), but the codebase exclusively raises standardValueErrorexceptions across all modules (protection.py,style.py,profiles.py,pipeline.py,metrics.py,image.py).Add a note stating these are architectural proposals for future refactoring, or create a tracking issue to guide implementation.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/system-design/13_ERROR_HANDLING.md` around lines 3 - 8, Update the Error Classes section to state these names (`InputValidationError`, `ProfileConfigError`, `PipelineExecutionError`, `ReportSerializationError`) are proposed/architectural suggestions and not yet implemented in the codebase (current modules raise `ValueError` in protection.py, style.py, profiles.py, pipeline.py, metrics.py, image.py); either add a clear note in the document marking them as proposals with a suggested rollout plan, or create and link a tracking issue/ticket for implementing these custom exceptions so reviewers know this is future work.docs/system-design/01_ARCHITECTURE_CONTEXT.md (1)
5-7: Add measurable quality criteria in the problem statement.“Giữ chất lượng đủ dùng cho công bố” is clear directionally, but still ambiguous for implementation/review. Add concrete target metrics (or link to the NFR/metrics doc section) so this scope can be validated consistently.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/system-design/01_ARCHITECTURE_CONTEXT.md` around lines 5 - 7, Trong đoạn mô tả vấn đề, thay câu mơ hồ "Giữ chất lượng đủ dùng cho công bố" bằng các tiêu chí định lượng hoặc liên kết tới mục NFR/metrics; ví dụ bổ sung mục tiêu như PSNR ≥ X, SSIM ≥ Y, human evaluation ≥ Z%/A-B điểm, hoặc một liên kết tới tài liệu "NFR/metrics" để kiểm chứng; sửa cụm văn bản trong tài liệu (phần chứa "Giữ chất lượng đủ dùng cho công bố") để nêu rõ các chỉ số mục tiêu hoặc đường dẫn tới tài liệu đo lường.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@CONTRIBUTING.md`:
- Line 20: The git clone command string in CONTRIBUTING.md currently includes a
trailing period ("git clone https://github.com/VoDaiLocz/Lock-ART.") which
breaks cloning; remove the trailing period so the URL is valid (e.g., update the
line containing the git clone example to use "git clone
https://github.com/VoDaiLocz/Lock-ART" without the final dot).
- Line 35: The pip install command is missing the opening quote for the extras
spec; change the line containing pip install -e "[ui,dev]" to include the dot
before the extras and proper quoting so it reads pip install -e ".[ui,dev]"
(update the exact line with the current pip install -e "[ui,dev]" text to the
corrected pip install -e ".[ui,dev]").
In `@docs/system-design/02_ARCHITECTURE_PRINCIPLES.md`:
- Line 7: Update the documented module list to match the actual src/auralock/
directories by replacing "attack/core/service/ui" with the correct plural and
complete names: "attacks/benchmarks/core/services/ui" so the doc reflects the
real module folders (attacks, benchmarks, core, services, ui).
In `@docs/system-design/07_DATA_MODEL.md`:
- Around line 10-17: The docs' "Report Schema (đề xuất)" is ambiguous versus
actual output; update either the markdown or code: if the schema is proposed,
change the heading or add a short note stating it's a future proposal and not
current; otherwise update the documentation to match the actual output of
BenchmarkSummary.to_report_dict() (which currently returns input_path,
image_count, entries, profile_summaries) and ensure fields like run_id,
timestamp, protection_score, transforms_robustness, and environment are removed
or added to the implementation accordingly so the doc and the code
(BenchmarkSummary.to_report_dict) are consistent.
- Around line 3-8: The docs for Core Entities are out of sync with the
implementation: update the markdown to match the actual dataclasses by replacing
the listed fields for ProtectionConfig with its real fields (e.g., use num_steps
instead of steps and remove seed), replace the ProtectionResult bullet to
enumerate the actual attributes found on the ProtectionResult dataclass
(original_tensor, protected_tensor, protected_image, quality_report,
protection_report, etc.) and remove references to a non-existent BenchmarkResult
while documenting the real types BenchmarkEntry and BenchmarkSummary (and their
structure) or add a clear note that the doc describes a proposed design rather
than current implementation; reference the ProtectionConfig, ProtectionResult,
BenchmarkEntry and BenchmarkSummary symbols when making the edits so the doc
mirrors the code.
In `@docs/system-design/08_PROFILE_STRATEGY.md`:
- Around line 14-16: The profile governance rules in 08_PROFILE_STRATEGY.md are
currently only reviewer guidance and lack enforcement; either (A) implement
enforcement by adding CI checks and PR template checklist items that validate
"benchmark on avatar dataset", "no single-metric regressions", and "document
failure cases" (update the repository's PR template and CI workflow to run those
checks and surface failures), or (B) change 08_PROFILE_STRATEGY.md to explicitly
label these three bullets as "recommendations" rather than requirements and add
a cross-reference to CONTRIBUTING.md -> "Research and benchmark integrity" for
where measurable evidence and reproducibility are enforced; update the document
title/intro to reflect the chosen enforcement level and mention the exact
checklist items to make expectations unambiguous.
In `@docs/system-design/17_CI_CD_WORKFLOW.md`:
- Around line 5-7: The docs state three required CI checks but the GitHub
Actions workflow ci.yml only runs lint + unit tests; either add the missing
automated checks or update the docs in 17_CI_CD_WORKFLOW.md. To fix, add a
docs-validation job in ci.yml that triggers on changes to CLI/API paths (e.g.,
patterns like cli/** and api/**) and runs a markdown/docs linter or a diff-based
check, and add a profile-benchmark check that scans PR file changes for
profile-related files (e.g., profiles/**) and uses a small action or a script to
detect missing benchmark evidence and post an automated comment (via
actions/github-script or a dedicated comment action) failing the check or
notifying authors; alternatively, update 17_CI_CD_WORKFLOW.md to remove or
reword the two checks if you choose not to implement them.
- Around line 11-13: The doc lists three required release steps ("Tag semantic
version.", "Xuất changelog tóm tắt thay đổi + giới hạn.", "Đính kèm báo cáo
benchmark chuẩn.") but the actual CI only publishes containers after a release
exists; either implement automation or update the doc: add GitHub Actions
workflows (or steps in existing release workflow) to 1) create and push a
semantic version tag (use git tag + push or actions/create-release with
tag_name), 2) generate a changelog summary automatically (use git-cliff or
github-changelog-generator or generate from PR/release notes and attach as
release body), and 3) run benchmark job and upload the artifact to the GitHub
Release (actions/upload-artifact + actions/upload-release-asset); alternatively,
if these are manual processes, update 17_CI_CD_WORKFLOW.md to mark those three
items as manual steps and document the exact manual commands and where to attach
benchmark reports.
In `@docs/system-design/18_RISKS_AND_MITIGATIONS.md`:
- Line 13: Create and commit a reproducible dependency lock for the benchmark
environment: run Poetry (poetry lock) or pip-tools (pip-compile) against
pyproject.toml to generate a poetry.lock or requirements.txt.lock that pins all
transitive versions for the listed benchmark packages (diffusers, accelerate,
transformers, peft, safetensors), add that lock file to the repo, and update the
docs line referencing "Mitigation: lock file cho benchmark quan trọng." to
indicate the presence of the generated lock file and how to regenerate it (e.g.,
"poetry lock" or "pip-compile").
---
Outside diff comments:
In `@README.md`:
- Line 97: Remove the trailing period from the git clone URL so the command
succeeds: update the string "git clone https://github.com/VoDaiLocz/Lock-ART."
to "git clone https://github.com/VoDaiLocz/Lock-ART" (same fix as applied for
the CONTRIBUTING.md clone line).
- Around line 218-230: Remove the duplicate documentation links by keeping
"Research Roadmap" and "System Design Hub" only under the "Notes and
Documentation" section and removing those same links from the "Project
Standards" list; ensure "Project Standards" retains only governance docs like
"Contributing Guide", "Code of Conduct", and "Security Policy" so the README
entries for "Research Roadmap" and "System Design Hub" appear once and in the
intended section.
---
Nitpick comments:
In `@docs/system-design/01_ARCHITECTURE_CONTEXT.md`:
- Around line 5-7: Trong đoạn mô tả vấn đề, thay câu mơ hồ "Giữ chất lượng đủ
dùng cho công bố" bằng các tiêu chí định lượng hoặc liên kết tới mục
NFR/metrics; ví dụ bổ sung mục tiêu như PSNR ≥ X, SSIM ≥ Y, human evaluation ≥
Z%/A-B điểm, hoặc một liên kết tới tài liệu "NFR/metrics" để kiểm chứng; sửa cụm
văn bản trong tài liệu (phần chứa "Giữ chất lượng đủ dùng cho công bố") để nêu
rõ các chỉ số mục tiêu hoặc đường dẫn tới tài liệu đo lường.
In `@docs/system-design/13_ERROR_HANDLING.md`:
- Around line 3-8: Update the Error Classes section to state these names
(`InputValidationError`, `ProfileConfigError`, `PipelineExecutionError`,
`ReportSerializationError`) are proposed/architectural suggestions and not yet
implemented in the codebase (current modules raise `ValueError` in
protection.py, style.py, profiles.py, pipeline.py, metrics.py, image.py); either
add a clear note in the document marking them as proposals with a suggested
rollout plan, or create and link a tracking issue/ticket for implementing these
custom exceptions so reviewers know this is future work.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: a558f6c5-b729-4f4a-b2cc-40c625f50032
📒 Files selected for processing (26)
CODE_OF_CONDUCT.mdCONTRIBUTING.mdREADME.mdSECURITY.mddocs/system-design/00_OVERVIEW.mddocs/system-design/01_ARCHITECTURE_CONTEXT.mddocs/system-design/02_ARCHITECTURE_PRINCIPLES.mddocs/system-design/03_REQUIREMENTS_FUNCTIONAL.mddocs/system-design/04_REQUIREMENTS_NON_FUNCTIONAL.mddocs/system-design/05_COMPONENTS.mddocs/system-design/06_RUNTIME_VIEWS.mddocs/system-design/07_DATA_MODEL.mddocs/system-design/08_PROFILE_STRATEGY.mddocs/system-design/09_EXPERIMENT_WORKFLOWS.mddocs/system-design/10_BENCHMARK_DESIGN.mddocs/system-design/11_SECURITY_MODEL.mddocs/system-design/12_OBSERVABILITY.mddocs/system-design/13_ERROR_HANDLING.mddocs/system-design/14_PERFORMANCE_SCALING.mddocs/system-design/15_DEPLOYMENT_AND_ENV.mddocs/system-design/16_TESTING_STRATEGY.mddocs/system-design/17_CI_CD_WORKFLOW.mddocs/system-design/18_RISKS_AND_MITIGATIONS.mddocs/system-design/19_ROADMAP_TECHNICAL.mddocs/system-design/20_GLOSSARY.mddocs/system-design/README.md
| ## 2) Development setup | ||
|
|
||
| ```bash | ||
| git clone https://github.com/VoDaiLocz/Lock-ART. |
There was a problem hiding this comment.
Fix the git clone URL.
The trailing period will cause the git clone command to fail.
🔧 Proposed fix
-git clone https://github.com/VoDaiLocz/Lock-ART.
+git clone https://github.com/VoDaiLocz/Lock-ART.git📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| git clone https://github.com/VoDaiLocz/Lock-ART. | |
| git clone https://github.com/VoDaiLocz/Lock-ART.git |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@CONTRIBUTING.md` at line 20, The git clone command string in CONTRIBUTING.md
currently includes a trailing period ("git clone
https://github.com/VoDaiLocz/Lock-ART.") which breaks cloning; remove the
trailing period so the URL is valid (e.g., update the line containing the git
clone example to use "git clone https://github.com/VoDaiLocz/Lock-ART" without
the final dot).
| Optional extras: | ||
|
|
||
| ```bash | ||
| pip install -e "[ui,dev]" |
There was a problem hiding this comment.
Fix the pip install command syntax.
The opening quote is missing from the extras specification.
🔧 Proposed fix
-pip install -e "[ui,dev]"
+pip install -e ".[ui,dev]"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| pip install -e "[ui,dev]" | |
| pip install -e ".[ui,dev]" |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@CONTRIBUTING.md` at line 35, The pip install command is missing the opening
quote for the extras spec; change the line containing pip install -e "[ui,dev]"
to include the dot before the extras and proper quoting so it reads pip install
-e ".[ui,dev]" (update the exact line with the current pip install -e "[ui,dev]"
text to the corrected pip install -e ".[ui,dev]").
| 2. **Transparent trade-offs**: mọi profile phải nêu rõ đổi chác quality/protection. | ||
| 3. **CPU-first baseline**: workflow cơ bản chạy được không cần GPU. | ||
| 4. **Benchmark-driven changes**: thay đổi logic phải đi kèm bằng chứng đo lường. | ||
| 5. **Small composable modules**: tách attack/core/service/ui rõ ràng. |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify src/auralock directory structure
echo "=== Actual src/auralock structure ==="
tree -L 1 -d src/auralock/ || ls -d src/auralock/*/Repository: VoDaiLocz/Lock-ART.
Length of output: 185
Update module names to match actual directory structure.
Line 7 documents "tách attack/core/service/ui rõ ràng", but the actual src/auralock/ structure contains attacks/, benchmarks/, core/, services/, and ui/ (plural names and includes benchmarks/ not mentioned in the documentation).
Update to: "tách attacks/benchmarks/core/services/ui rõ ràng"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@docs/system-design/02_ARCHITECTURE_PRINCIPLES.md` at line 7, Update the
documented module list to match the actual src/auralock/ directories by
replacing "attack/core/service/ui" with the correct plural and complete names:
"attacks/benchmarks/core/services/ui" so the doc reflects the real module
folders (attacks, benchmarks, core, services, ui).
| ## Core Entities | ||
|
|
||
| - `ImageInput`: đường dẫn, mode, kích thước gốc. | ||
| - `ProtectionConfig`: profile + override epsilon/steps/seed. | ||
| - `ProtectionResult`: output path, metrics, warnings. | ||
| - `BenchmarkResult`: nhiều `ProtectionResult` + ranking. |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify actual dataclass definitions for core entities
echo "=== ProtectionConfig definition ==="
ast-grep --pattern 'class ProtectionConfig:
$$$'
echo -e "\n=== ProtectionResult definition ==="
ast-grep --pattern 'class ProtectionResult:
$$$'
echo -e "\n=== Benchmark-related classes ==="
rg -A 5 'class Benchmark' --type=pyRepository: VoDaiLocz/Lock-ART.
Length of output: 5177
Update Core Entities documentation to match actual implementation.
The documented entities have significant discrepancies with the actual code:
-
Line 6 - ProtectionConfig: Documented as having
stepsandseed, but the actualProtectionConfigdataclass (src/auralock/core/profiles.py:9-20) usesnum_stepsand has noseedfield. -
Line 7 - ProtectionResult: Documented as "output path, metrics, warnings" but the actual
ProtectionResultdataclass (src/auralock/services/protection.py:50-97) does not haveoutput_pathorwarningsfields. It includesoriginal_tensor,protected_tensor,protected_image,quality_report,protection_report, and other fields not mentioned in the documentation. -
Line 8 - BenchmarkResult: This class does not exist in the codebase. Instead, the code defines
BenchmarkEntryandBenchmarkSummary(src/auralock/services/protection.py) with a different structure.
Update the documentation to reflect the actual data structures or clarify whether this describes a proposed design rather than current implementation.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@docs/system-design/07_DATA_MODEL.md` around lines 3 - 8, The docs for Core
Entities are out of sync with the implementation: update the markdown to match
the actual dataclasses by replacing the listed fields for ProtectionConfig with
its real fields (e.g., use num_steps instead of steps and remove seed), replace
the ProtectionResult bullet to enumerate the actual attributes found on the
ProtectionResult dataclass (original_tensor, protected_tensor, protected_image,
quality_report, protection_report, etc.) and remove references to a non-existent
BenchmarkResult while documenting the real types BenchmarkEntry and
BenchmarkSummary (and their structure) or add a clear note that the doc
describes a proposed design rather than current implementation; reference the
ProtectionConfig, ProtectionResult, BenchmarkEntry and BenchmarkSummary symbols
when making the edits so the doc mirrors the code.
| ## Report Schema (đề xuất) | ||
|
|
||
| - `run_id`, `timestamp`, `profile`. | ||
| - `input_meta` (size, channels, dtype). | ||
| - `metrics` (`psnr`, `ssim`, `protection_score`). | ||
| - `transforms_robustness`. | ||
| - `environment` (python, torch, device). | ||
| - `notes` và `limitations`. |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check the actual to_report_dict implementation
echo "=== BenchmarkSummary report structure ==="
ast-grep --pattern 'def to_report_dict($$$):
$$$'
echo -e "\n=== Report serialization calls ==="
rg 'to_report_dict' --type=py -C 3Repository: VoDaiLocz/Lock-ART.
Length of output: 9930
Clarify whether Report Schema section describes current or proposed implementation.
The documented schema fields (run_id, timestamp, protection_score, transforms_robustness, environment) do not appear in the current BenchmarkSummary.to_report_dict() implementation (src/auralock/services/protection.py), which returns input_path, image_count, entries, and profile_summaries.
The section heading indicates "(đề xuất)" [proposed], but this distinction is not explained in the document. Either clarify that this is a proposed future schema, or update the documentation to reflect the current structure.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@docs/system-design/07_DATA_MODEL.md` around lines 10 - 17, The docs' "Report
Schema (đề xuất)" is ambiguous versus actual output; update either the markdown
or code: if the schema is proposed, change the heading or add a short note
stating it's a future proposal and not current; otherwise update the
documentation to match the actual output of BenchmarkSummary.to_report_dict()
(which currently returns input_path, image_count, entries, profile_summaries)
and ensure fields like run_id, timestamp, protection_score,
transforms_robustness, and environment are removed or added to the
implementation accordingly so the doc and the code
(BenchmarkSummary.to_report_dict) are consistent.
| - Cần benchmark tối thiểu trên tập ảnh đại diện. | ||
| - Không merge nếu chỉ cải thiện một metric mà phá mạnh metric còn lại. | ||
| - Bắt buộc ghi rõ failure cases. |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check for governance enforcement in CI, PR templates, or review guidelines
echo "=== Checking CI workflows for profile benchmark requirements ==="
rg -n "profile|benchmark" .github/workflows/ || echo "No profile governance checks in CI"
echo -e "\n=== Checking for PR templates with profile checklist ==="
fd -e md . .github/ --exec rg -l "profile|benchmark" || echo "No PR template found"
echo -e "\n=== Checking CONTRIBUTING.md for profile governance ==="
rg -n "profile|governance|benchmark" CONTRIBUTING.md || echo "No governance guidance in CONTRIBUTING"Repository: VoDaiLocz/Lock-ART.
Length of output: 743
Clarify enforcement level for profile governance rules.
The document specifies three governance requirements for new profiles. However, verification shows:
- No CI checks enforce these rules
- No PR templates include profile governance checklists
- Guidelines exist in CONTRIBUTING.md (section "Research and benchmark integrity"):
- Line 15: Profiles must be "accompanied by measurable evidence"
- Line 95: Checklist requires "Any benchmark claims include reproducible context"
The requirements are documented as reviewer guidelines, not automated enforcement. Either enforce these through CI/templates or explicitly label them as "recommendations" in the document.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@docs/system-design/08_PROFILE_STRATEGY.md` around lines 14 - 16, The profile
governance rules in 08_PROFILE_STRATEGY.md are currently only reviewer guidance
and lack enforcement; either (A) implement enforcement by adding CI checks and
PR template checklist items that validate "benchmark on avatar dataset", "no
single-metric regressions", and "document failure cases" (update the
repository's PR template and CI workflow to run those checks and surface
failures), or (B) change 08_PROFILE_STRATEGY.md to explicitly label these three
bullets as "recommendations" rather than requirements and add a cross-reference
to CONTRIBUTING.md -> "Research and benchmark integrity" for where measurable
evidence and reproducibility are enforced; update the document title/intro to
reflect the chosen enforcement level and mention the exact checklist items to
make expectations unambiguous.
| 1. Lint + unit test bắt buộc. | ||
| 2. Kiểm tra thay đổi docs nếu touch CLI/API. | ||
| 3. Comment tự động nếu thiếu benchmark evidence cho thay đổi profile. |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify CI workflow steps match documentation claims
echo "=== Checking CI workflow for documentation checks ==="
rg -n "docs|documentation|CLI|API" .github/workflows/ci.yml || echo "No doc checks found"
echo -e "\n=== Checking CI workflow for benchmark comment automation ==="
rg -n "benchmark|comment|profile" .github/workflows/ci.yml || echo "No benchmark automation found"
echo -e "\n=== Current CI workflow steps ==="
rg -n "name:|run:" .github/workflows/ci.yml | head -20Repository: VoDaiLocz/Lock-ART.
Length of output: 779
🏁 Script executed:
# First, verify the documentation file content at lines 5-7
echo "=== Documentation file content (lines 1-15) ==="
head -15 docs/system-design/17_CI_CD_WORKFLOW.md
echo -e "\n=== Full CI workflow file ==="
cat .github/workflows/ci.yml
echo -e "\n=== List all workflow files ==="
ls -la .github/workflows/Repository: VoDaiLocz/Lock-ART.
Length of output: 1557
Implement missing PR validation checks or update documentation.
The documented PR flow specifies three required checks (lines 5-7), but .github/workflows/ci.yml only implements linting and unit tests. The following are not automated:
- Line 6: Documentation validation when CLI/API changes are touched
- Line 7: Automatic comment when benchmark evidence is missing for profile changes
Either implement these checks or update the documentation to match actual CI capabilities.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@docs/system-design/17_CI_CD_WORKFLOW.md` around lines 5 - 7, The docs state
three required CI checks but the GitHub Actions workflow ci.yml only runs lint +
unit tests; either add the missing automated checks or update the docs in
17_CI_CD_WORKFLOW.md. To fix, add a docs-validation job in ci.yml that triggers
on changes to CLI/API paths (e.g., patterns like cli/** and api/**) and runs a
markdown/docs linter or a diff-based check, and add a profile-benchmark check
that scans PR file changes for profile-related files (e.g., profiles/**) and
uses a small action or a script to detect missing benchmark evidence and post an
automated comment (via actions/github-script or a dedicated comment action)
failing the check or notifying authors; alternatively, update
17_CI_CD_WORKFLOW.md to remove or reword the two checks if you choose not to
implement them.
| - Tag semantic version. | ||
| - Xuất changelog tóm tắt thay đổi + giới hạn. | ||
| - Đính kèm báo cáo benchmark chuẩn. |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify release workflow steps match documentation claims
echo "=== Checking release workflow for semantic versioning ==="
rg -n "semantic|version|tag" .github/workflows/publish-package.yml || echo "No semantic versioning found"
echo -e "\n=== Checking release workflow for changelog generation ==="
rg -n "changelog|CHANGELOG" .github/workflows/publish-package.yml || echo "No changelog generation found"
echo -e "\n=== Checking release workflow for benchmark reports ==="
rg -n "benchmark|report" .github/workflows/publish-package.yml || echo "No benchmark reports found"
echo -e "\n=== Current release workflow steps ==="
rg -n "name:|run:" .github/workflows/publish-package.yml | head -30Repository: VoDaiLocz/Lock-ART.
Length of output: 726
🏁 Script executed:
#!/bin/bash
# View the documentation file to understand the full context
echo "=== Documentation excerpt (lines 5-20) ==="
cat -n docs/system-design/17_CI_CD_WORKFLOW.md | sed -n '5,20p'
echo -e "\n=== Full publish-package.yml workflow ==="
cat -n .github/workflows/publish-package.ymlRepository: VoDaiLocz/Lock-ART.
Length of output: 2416
🏁 Script executed:
#!/bin/bash
# Check what other workflows exist
echo "=== All workflow files ==="
ls -la .github/workflows/
echo -e "\n=== Search for release-related workflows ==="
fd -e yml -e yaml .github/workflows/ | while read f; do
echo "File: $f"
rg -l "release|changelog|benchmark" "$f" && echo " -> Has release/changelog/benchmark keywords"
doneRepository: VoDaiLocz/Lock-ART.
Length of output: 766
🏁 Script executed:
#!/bin/bash
# View ci.yml to see if it handles release tasks
echo "=== ci.yml workflow ==="
cat -n .github/workflows/ci.yml
echo -e "\n=== Search for release/changelog/benchmark across all workflows ==="
rg "release|changelog|benchmark" .github/workflows/Repository: VoDaiLocz/Lock-ART.
Length of output: 1311
Release Flow requirements are not implemented in actual release workflows.
The documentation specifies three required release steps (lines 11-13), but the existing workflows only handle container publishing after a release is already created. Missing implementations:
- Semantic version tagging — No workflow automates tag creation
- Changelog generation — No step generates changelog summaries
- Benchmark report attachment — No step attaches benchmark reports
Either implement these release automation steps or update the documentation to reflect the actual release workflow. If these steps are manual, document them separately.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@docs/system-design/17_CI_CD_WORKFLOW.md` around lines 11 - 13, The doc lists
three required release steps ("Tag semantic version.", "Xuất changelog tóm tắt
thay đổi + giới hạn.", "Đính kèm báo cáo benchmark chuẩn.") but the actual CI
only publishes containers after a release exists; either implement automation or
update the doc: add GitHub Actions workflows (or steps in existing release
workflow) to 1) create and push a semantic version tag (use git tag + push or
actions/create-release with tag_name), 2) generate a changelog summary
automatically (use git-cliff or github-changelog-generator or generate from
PR/release notes and attach as release body), and 3) run benchmark job and
upload the artifact to the GitHub Release (actions/upload-artifact +
actions/upload-release-asset); alternatively, if these are manual processes,
update 17_CI_CD_WORKFLOW.md to mark those three items as manual steps and
document the exact manual commands and where to attach benchmark reports.
|
|
||
| ## R3: Dependency breakage | ||
|
|
||
| - Mitigation: lock file cho benchmark quan trọng. |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check for dependency lock files
echo "=== Checking for lock files in repository ==="
fd -t f "lock|requirements.*txt" . --max-depth 2
echo -e "\n=== Checking pyproject.toml for version pinning ==="
rg -n "==" pyproject.toml || echo "No pinned versions in pyproject.toml"
echo -e "\n=== Checking for benchmark-specific requirements ==="
fd "benchmark" . --type f -e txt -e lock -e tomlRepository: VoDaiLocz/Lock-ART.
Length of output: 300
🏁 Script executed:
#!/bin/bash
echo "=== Looking for ALL lock files with multiple patterns ==="
fd -t f '\.(lock|pipfile)$' . --max-depth 3
echo "---"
find . -name "*.lock" -type f 2>/dev/null | head -20
echo "---"
find . -name "poetry.lock" -o -name "Pipfile.lock" -o -name "requirements.lock" 2>/dev/null
echo -e "\n=== Checking if pyproject.toml exists ==="
find . -name "pyproject.toml" -type f
echo -e "\n=== Checking for any dependency management files ==="
fd -t f "(pyproject|setup|requirements|Pipfile)" . --max-depth 2Repository: VoDaiLocz/Lock-ART.
Length of output: 259
🏁 Script executed:
cat -n pyproject.toml | head -100Repository: VoDaiLocz/Lock-ART.
Length of output: 3077
Lock files for critical benchmarks are missing. The repository has no lock files (poetry.lock, Pipfile.lock, etc.) and the benchmark dependencies in pyproject.toml (diffusers, accelerate, transformers, peft, safetensors) have no version constraints. This prevents reproducible benchmark runs. Generate a lock file using Poetry, pip-tools, or similar to pin all transitive dependencies for the benchmark environment.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@docs/system-design/18_RISKS_AND_MITIGATIONS.md` at line 13, Create and commit
a reproducible dependency lock for the benchmark environment: run Poetry (poetry
lock) or pip-tools (pip-compile) against pyproject.toml to generate a
poetry.lock or requirements.txt.lock that pins all transitive versions for the
listed benchmark packages (diffusers, accelerate, transformers, peft,
safetensors), add that lock file to the repo, and update the docs line
referencing "Mitigation: lock file cho benchmark quan trọng." to indicate the
presence of the generated lock file and how to regenerate it (e.g., "poetry
lock" or "pip-compile").
1 participant
No description provided.