From b521f48362db357dac876184bc827e047d791b2e Mon Sep 17 00:00:00 2001 From: Thomas Almenningen Date: Mon, 26 Jan 2026 10:12:38 +0100 Subject: [PATCH] docs: add AI agent guidelines documentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add AGENTS.md with comprehensive guidelines for AI agents working on this repository, including: - Development workflow and branch naming conventions - Pre-commit hooks and testing setup - Commit message and PR requirements - AWS CodeArtifact configuration 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- AGENTS.md | 259 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 259 insertions(+) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..5806439 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,259 @@ +# AI Agent Guidelines for per-transformers + +This document provides guidelines for AI agents working on the `per-transformers` repository. It covers the development workflow, conventions, and important commands specific to this project. + +## Repository Overview + +- **Purpose**: [To be filled in based on repository purpose] +- **Language**: Python 3.12.3 +- **Package Manager**: `uv` +- **Testing Framework**: pytest with PySpark +- **Linting**: Ruff (via pre-commit hooks) +- **CI/CD**: Jenkins +- **Container Runtime**: Databricks Runtime 16.4 (for tests) + +## Development Workflow + +### 1. Before Starting Work + +**Always ask for the Linear ticket ID** before implementing any feature, fix, or chore. This ticket ID is used for: +- Branch naming +- Commit messages +- PR titles and descriptions + +### 2. Create a Feature Branch + +Follow this naming convention based on the type of work: + +```bash +# For new features +git checkout -b feat/-short-description + +# For bug fixes +git checkout -b fix/-short-description + +# For maintenance/chores +git checkout -b chore/-short-description +``` + +**Example**: `feat/PER-123-add-album-metadata-support` + +### 3. Development Steps + +1. Make your changes following the codebase patterns +2. Run pre-commit hooks (see section below) +3. Run tests locally (see section below) +4. Commit with proper message format (see section below) +5. Push your branch +6. Create a Pull Request (see PR requirements below) + +## Pre-commit Hooks + +This repository uses Ruff for linting and formatting via pre-commit hooks. + +### Running Pre-commit Hooks + +```bash +# Run on all files +pre-commit run --all-files + +# Pre-commit will automatically run on staged files when you commit +git commit -m "your message" +``` + +### What the Hooks Do + +- **ruff**: Lints Python code and auto-fixes issues +- **ruff-format**: Formats Python code + +Configuration is in `.pre-commit-config.yaml` using Ruff v0.9.6. + +## Running Tests + +### Prerequisites + +1. **Python 3.12.3**: Install with `uv python install 3.12.3` +2. **uv**: Package manager +3. **AWS Credentials**: Configured for `tidal-backoffice-production--personalization` profile +4. **Docker**: For running tests in Databricks Runtime container + +### Setup + +```bash +# Install Python 3.12.3 +uv python install 3.12.3 + +# Create virtual environment +uv venv + +# Activate virtual environment +source .venv/bin/activate + +# Install dependencies +cd utils && sh install_deps.sh +``` + +### Environment Variables + +When running tests locally (outside Docker), set: + +```bash +export PYTHONPATH=src +export PYSPARK_PYTHON=python3 +export PYSPARK_DRIVER_PYTHON=python3 +``` + +### Running Tests + +```bash +# Run tests locally (after setting environment variables) +python -m pytest + +# Run specific test file +python -m pytest test/path/to/test_file.py + +# Run specific test function +python -m pytest test/path/to/test_file.py::test_specific_function +``` + +**Note**: `make test` runs tests in a Docker container using Databricks Runtime 16.4 image, which mirrors the production environment. + +## Commit Message Convention + +Follow the Conventional Commits format: + +``` +(): + +[optional body] + +[optional footer] +``` + +### Types + +- `feat`: New feature +- `fix`: Bug fix +- `chore`: Maintenance, refactoring, dependencies +- `docs`: Documentation changes +- `test`: Adding or updating tests +- `perf`: Performance improvements +- `refactor`: Code refactoring without functionality change + +### Scopes (Examples) + +- `writer`: Changes to writer classes +- `embedding`: Changes to embedding data classes +- `opensearch`: Changes to OpenSearch client +- `tests`: Test-related changes +- `deps`: Dependency changes +- `ci`: CI/CD changes + +### Examples + +```bash +# Feature commit +git commit -m "feat(embedding): PER-123 add album metadata support" + +# Bug fix +git commit -m "fix(writer): PER-456 handle null values in track embeddings" + +# Chore +git commit -m "chore(deps): PER-789 update pyspark to 3.5.2" + +# Multiple line commit with body +git commit -m "refactor(opensearch): PER-234 improve connection pooling + +Refactored the OpenSearch client to use connection pooling +for better performance under high load." +``` + +## Pull Request Requirements + +### PR Title + +Must match the commit message convention: + +``` +(): +``` + +### PR Description + +Include the following sections: + +```markdown +## Summary +Brief description of what this PR does and why. + +## Testing +- [ ] Unit tests added/updated +- [ ] Tested locally with sample data +- [ ] Pre-commit hooks pass +- [ ] All tests pass + +Describe specific testing performed. + +## Risk/Rollout +- Risk level: [Low/Medium/High] +- Rollout plan: [Gradual/Immediate] +- Rollback plan: [Description] + +## Linear Ticket +[Link to Linear ticket: PER-XXX] +``` + +### PR Size Guidelines + +- **Preferred**: < 400 lines changed +- **Maximum**: < 800 lines changed +- For larger changes, consider breaking into multiple PRs + +## Safety Guidelines + +### No Secrets in Code + +- Never commit AWS credentials, API keys, or secrets +- Use AWS profiles and environment variables +- Reference: AWS profile `tidal-backoffice-production--personalization` + +### Dependency Management + +- Justify any new dependencies in PR description +- Use version pinning in `pyproject.toml` +- Update lock file: `uv lock` + +## Useful Commands + +### Build and Clean + +```bash +# Build the package (creates dist/jobs.zip and dist/main.py) +make build +``` + +### Dependencies + +```bash +# Install dependencies from lock file +cd utils && sh install_deps.sh +``` + +## Repository-Specific Details + +### AWS CodeArtifact + +Dependencies are installed via AWS CodeArtifact. The URL is generated dynamically: + +```bash +./utils/generate_codeartifact_url.sh tidal-backoffice-production--personalization +``` + +## Key Files Reference + +- `pyproject.toml`: Python dependencies and Ruff configuration +- `Makefile`: Build, test, and deployment commands +- `.pre-commit-config.yaml`: Pre-commit hook configuration +- `src/main.py`: Entry point (copied to dist during build) +- `utils/install_deps.sh`: Dependency installation script +- `utils/generate_codeartifact_url.sh`: AWS CodeArtifact URL generator