Skip to content

Update CLAUDE.md with detailed info about pulp-service #955

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
decko wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Update CLAUDE.md with detailed info about pulp-service #955

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
76818a0
Update CLAUDE.md with detailed info about pulp-service
decko Feb 25, 2026
b664aee
docs: address review feedback on CLAUDE.md
decko Mar 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
160 changes: 140 additions & 20 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ pulp-service is a **Django REST Framework plugin for Pulpcore** that extends the

The plugin is registered via the `pulpcore.plugin` entry point in `pulp_service/setup.py`.

Upstream: https://pulpproject.org/ | Source: https://github.com/pulp/pulp-service

## Repository Layout

- `pulp_service/` — The Python package (all source code lives here)
Expand All @@ -19,6 +21,121 @@ The plugin is registered via the `pulpcore.plugin` entry point in `pulp_service/
- `docs/ARCHITECTURE.md` — Comprehensive architecture reference (read this for deep context)
- `CHANGES/` — Towncrier changelog fragments

## Quick Reference Index

### Core Plugin Code

| Component | File | Description |
|-----------|------|-------------|
| **Settings** | `pulp_service/app/settings.py` | Django settings overrides |
| **Models** | `pulp_service/app/models.py` | DomainOrg, FeatureContentGuard, VulnerabilityReport |
| **ViewSets** | `pulp_service/app/viewsets.py` | REST API endpoints |
| **Serializers** | `pulp_service/app/serializers.py` | DRF serializers |
| **Authentication** | `pulp_service/app/authentication.py` | X.509 cert, SAML, registry auth backends |
| **Authorization** | `pulp_service/app/authorization.py` | Domain-based RBAC permissions |
| **Middleware** | `pulp_service/app/middleware.py` | Profiler, edge host, SAML, OTEL metrics |
| **Storage** | `pulp_service/app/storage.py` | OCIStorage (OCI/ORAS backend) |
| **Signals** | `pulp_service/app/signals.py` | User creation, domain creation hooks |
| **Tasks** | `pulp_service/app/tasks/` | Package scanning, domain metrics |
| **Content** | `pulp_service/app/content.py` | aiohttp middleware for pulp-content |
| **Admin** | `pulp_service/app/admin.py` | Django admin (Users, Groups, Domains, Tasks) |

### Deployment & Config

| Component | File | Description |
|-----------|------|-------------|
| **ClowdApp** | `deploy/clowdapp.yaml` | Production deployment (5 deployments + jobs) |
| **Dockerfile** | `Dockerfile` | Container image (UBI9, Python 3.11 venv) |
| **WSGI Middleware** | `images/assets/log_middleware.py` | User extraction from X-RH-IDENTITY |
| **Gunicorn Config** | `images/assets/gunicorn_config.py` | Gunicorn hooks and middleware |
| **Dependencies** | `pulp_service/requirements.txt` | Pinned plugin versions |
| **Patches** | `images/assets/patches/` | Upstream plugin patches applied at build time |

## Architecture

**Three-service model:**
1. **pulp-api** — Gunicorn WSGI serving Django REST API (port 24817 local, 8000 prod)
2. **pulp-content** — Gunicorn + aiohttp async content delivery (port 24816 local, 8000 prod)
3. **pulp-worker** — Redis-based workers for background tasks (`WORKER_TYPE=redis`, added via patch 0033; uses Redis for resource locking instead of PostgreSQL)

**Request flow:**
```
Client -> Load Balancer (sets X-Forwarded-For, X-RH-IDENTITY)
-> Gunicorn
-> WSGI: UserExtractionMiddleware (extracts user/org_id from X-RH-IDENTITY)
-> Django Middleware Stack
-> DRF ViewSets
-> Response
```

**Key patterns:**
- **Authentication**: `X-RH-IDENTITY` header (base64-encoded JSON) → custom auth classes in `app/authentication.py`
- **Multi-tenancy**: `DomainOrg` model maps org_id → Pulp domain; domain-based routing for content APIs
- **Context variables**: `ContextVar` instances in `app/middleware.py` carry request-scoped data (org_id, user_id, request_path) across layers
- **Storage backends**: Based on `django-storages`; S3 via `storages.backends.s3boto3.S3Boto3Storage` (upstream), `OCIStorage` (OCI/ORAS) in `app/storage.py`
- **Tasks**: Background work in `app/tasks/` (package scanning, domain metrics, RDS testing)

**Upstream plugins this extends**: pulpcore, pulp-python, pulp-container, pulp-rpm, pulp-gem, pulp-npm, pulp-maven, pulp-hugging-face.

## Middleware Stack

**WSGI** (`images/assets/log_middleware.py`):
1. **UserExtractionMiddleware** — Decodes base64 X-RH-IDENTITY, sets `REMOTE_USER` and `ORG_ID`

**Django** (`pulp_service/app/middleware.py`):
1. **ProfilerMiddleware** — cProfile on `X-Profile-Request` header
2. **RhEdgeHostMiddleware** — Maps `X-RH-EDGE-HOST` to `X-FORWARDED-HOST`
3. **RHSamlAuthHeaderMiddleware** — Auth for `/pulp-mgmt/` paths
4. **RequestPathMiddleware** — Stores path in ContextVar for signals
5. **ActiveConnectionsMetricMiddleware** — OTEL concurrent connection tracking

**aiohttp** (`pulp_service/app/content.py`):
1. **add_rh_org_id_resp_header** — Adds `X-RH-ORG-ID` response header

## Authentication Classes

1. **RHServiceAccountCertAuthentication** — X.509 cert via X-RH-IDENTITY
2. **RHTermsBasedRegistryAuthentication** — Registry auth from standard RH identity (`identity.user`)
3. **TurnpikeTermsBasedRegistryAuthentication** — Registry auth from Turnpike identity (`identity.registry`)
4. **RHSamlAuthentication** — SAML via X-RH-IDENTITY
5. **SessionAuthentication** — Django sessions (upstream)
6. **BasicAuthentication** — HTTP Basic (upstream)

X-RH-IDENTITY header: base64-encoded JSON. Standard format:
`{"identity": {"org_id": "123456", "user": {"username": "..."}}}`.
Turnpike registry format:
`{"identity": {"type": "Registry", "auth_type": "registry-auth", "registry": {"org_id": "...", "username": "..."}}}`.

## Context Variables

- `org_id_var` — Organization ID (set by authorization)
- `user_id_var` — User ID (set by authorization)
- `request_path_var` — Request path (set by RequestPathMiddleware)
- `repository_name_var` — Current repository name
- `x_quay_auth_var` — Quay authentication token
- `x_task_diagnostics_var` — Task diagnostics flag

## Key Environment Variables

- `PULP_AUTHENTICATION_JSON_HEADER=HTTP_X_RH_IDENTITY`
- `PULP_OTEL_ENABLED=true` / `OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:10000/`
- `PULP_CLAMAV_HOST` / `PULP_CLAMAV_PORT=10000`
- `PULP_FEATURE_SERVICE_API_URL` / `PULP_FEATURE_SERVICE_API_CERT_PATH`
- `PULP_UVLOOP_ENABLED=false`
- `SENTRY_DSN` (optional)

Full env var reference: `docs/ARCHITECTURE.md`

## Common Development Tasks

- **New API endpoint**: viewset in `viewsets.py`, serializer in `serializers.py`, register URL
- **Custom auth**: class in `authentication.py`, register in Django settings
- **New middleware**: class in `middleware.py`, register in `PULP_MIDDLEWARE`
- **Background task**: create in `tasks/`, call from viewset or signal
- **Modify logging**: update `--access-logformat` in `images/assets/pulp-api` AND `deploy/clowdapp.yaml`
- **Add metrics**: use `init_otel_meter()` from `pulpcore.metrics`
- **New model**: add to `models.py`, run `pulpcore-manager makemigrations`

## Build and Development Commands

```bash
Expand All @@ -36,34 +153,37 @@ pytest pulp_service/pulp_service/tests/functional/test_authentication.py

# Run a single test
pytest pulp_service/pulp_service/tests/functional/test_authentication.py::TestClass::test_method
```

Test dependencies: `pytest`, `pytest-django` (see `unittest_requirements.txt` / `functest_requirements.txt`).

## Architecture
# Build container image
docker build -t pulp-service .

**Three-service model:**
1. **pulp-api** — Gunicorn WSGI serving Django REST API (port 24817 local, 8000 prod)
2. **pulp-content** — Gunicorn + aiohttp async content delivery (port 24816 local, 8000 prod)
3. **pulp-worker** — Celery workers for background tasks (Redis broker)
# Ephemeral deployment
bonfire namespace reserve --duration 8h
bonfire deploy-env -n <namespace> --template-file deploy/clowdapp.yaml
```

**Request flow:**
WSGI middleware (`images/assets/log_middleware.py`) → Django middleware stack (`app/middleware.py`) → DRF ViewSets (`app/viewsets.py`)
Test dependencies: `pytest`, `pytest-django` (see `unittest_requirements.txt` / `functest_requirements.txt`).

**Key patterns:**
- **Authentication**: `X-RH-IDENTITY` header (base64-encoded JSON) → custom auth classes in `app/authentication.py`
- **Multi-tenancy**: `DomainOrg` model maps org_id → Pulp domain; domain-based routing for content APIs
- **Context variables**: `ContextVar` instances in `app/middleware.py` carry request-scoped data (org_id, user_id, request_path) across layers
- **Storage backends**: `AIPCCStorageBackend` (S3) and `OCIStorageBackend` (OCI/ORAS) in `app/storage.py`
- **Tasks**: Background work in `app/tasks/` (package scanning, domain metrics, RDS testing)
## Code Style

**Upstream plugins this extends**: pulpcore, pulp-python, pulp-container, pulp-rpm, pulp-gem, pulp-npm, pulp-maven, pulp-hugging-face.
- **Formatter**: Black, line-length 100
- **Testing**: pytest + pytest-django
- **Plugin pattern**: models, serializers, viewsets, tasks (same as all Pulp plugins)
- Entry point: `pulp_service = pulp_service:default_app_config`

## Changelog Process

Uses **towncrier**. For any non-trivial change, create a file in `CHANGES/` named `{issue_number}.{category}` where category is one of: `feature`, `bugfix`, `doc`, `removal`, `deprecation`, `misc`.

## Code Style
## Task Completion Checklist

1. `black --line-length 100` on changed files
2. `pytest pulp_service/pulp_service/tests/functional/` on relevant tests
3. Towncrier entry in `CHANGES/` for non-trivial changes
4. `pulpcore-manager makemigrations` if models changed
5. Check `images/assets/patches/` if modifying upstream plugin behavior
6. Sync changes between `images/assets/pulp-*` scripts and `deploy/clowdapp.yaml`

## Full Reference

- **Black** formatter, line length 100, targeting py36/py37
- Excludes: migrations, docs, build directories
For deployment specs, ClowdApp breakdown, resource limits, and complete env var documentation see `docs/ARCHITECTURE.md`.