Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/docker-publish-master.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ on:

env:
REGISTRY: ghcr.io
IMAGE_PREFIX: ghcr.io/bluedotiya/web-crawler
IMAGE_PREFIX: ghcr.io/nabi-allenby/web-crawler

jobs:
frontend-ci:
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
[![Release & Publish](https://github.com/bluedotiya/web-crawler/actions/workflows/docker-publish-master.yml/badge.svg)](https://github.com/bluedotiya/web-crawler/actions/workflows/docker-publish-master.yml)[![PR Title Check](https://github.com/bluedotiya/web-crawler/actions/workflows/pr-title.yml/badge.svg)](https://github.com/bluedotiya/web-crawler/actions/workflows/pr-title.yml)
[![Release & Publish](https://github.com/nabi-allenby/web-crawler/actions/workflows/docker-publish-master.yml/badge.svg)](https://github.com/nabi-allenby/web-crawler/actions/workflows/docker-publish-master.yml)[![PR Title Check](https://github.com/nabi-allenby/web-crawler/actions/workflows/pr-title.yml/badge.svg)](https://github.com/nabi-allenby/web-crawler/actions/workflows/pr-title.yml)

# Web Crawler

Expand Down Expand Up @@ -47,7 +47,7 @@ graph TD
### Install

```bash
helm install web-crawler oci://ghcr.io/bluedotiya/web-crawler/charts/web-crawler \
helm install web-crawler oci://ghcr.io/nabi-allenby/web-crawler/charts/web-crawler \
--version 1.0.0 -n web-crawler --create-namespace
```

Expand Down Expand Up @@ -128,7 +128,7 @@ See the [Development Guide](docs/development.md) for detailed setup instructions

## Security

- Report security vulnerabilities via [GitHub Security Advisories](https://github.com/bluedotiya/web-crawler/security/advisories)
- Report security vulnerabilities via [GitHub Security Advisories](https://github.com/nabi-allenby/web-crawler/security/advisories)
- All services run as non-root users in containers
- Neo4j credentials are stored in Kubernetes secrets

Expand Down
22 changes: 11 additions & 11 deletions docs/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ All configuration is managed through the parent chart's `values.yaml`.
The Helm chart is published as an OCI artifact to GitHub Container Registry. No `helm repo add` is needed.

```bash
helm install web-crawler oci://ghcr.io/bluedotiya/web-crawler/charts/web-crawler \
helm install web-crawler oci://ghcr.io/nabi-allenby/web-crawler/charts/web-crawler \
--version 1.0.0 -n web-crawler --create-namespace
```

Expand Down Expand Up @@ -86,7 +86,7 @@ Or use the web UI at `http://<FRONTEND_URL>/new`.
Customize via `--set` flags or a values override file:

```bash
helm install web-crawler oci://ghcr.io/bluedotiya/web-crawler/charts/web-crawler \
helm install web-crawler oci://ghcr.io/nabi-allenby/web-crawler/charts/web-crawler \
--version 1.0.0 -n web-crawler --create-namespace \
--set feeder.replicaCount=16 \
--set neo4j.neo4j.password=SecurePassword123
Expand Down Expand Up @@ -124,7 +124,7 @@ helm install web-crawler ./web-crawler -n web-crawler --create-namespace \
|-------|---------|-------------|
| `feeder.enabled` | `true` | Deploy feeders |
| `feeder.replicaCount` | `8` | Number of worker replicas |
| `feeder.image.repository` | `ghcr.io/bluedotiya/web-crawler/feeder` | Docker image |
| `feeder.image.repository` | `ghcr.io/nabi-allenby/web-crawler/feeder` | Docker image |
| `feeder.image.tag` | `latest` | Image tag |
| `feeder.image.pullPolicy` | `IfNotPresent` | Pull policy |
| `feeder.resources.requests.memory` | `256Mi` | Memory request |
Expand All @@ -140,7 +140,7 @@ helm install web-crawler ./web-crawler -n web-crawler --create-namespace \
|-------|---------|-------------|
| `manager.enabled` | `true` | Deploy manager |
| `manager.replicaCount` | `1` | Number of replicas |
| `manager.image.repository` | `ghcr.io/bluedotiya/web-crawler/manager` | Docker image |
| `manager.image.repository` | `ghcr.io/nabi-allenby/web-crawler/manager` | Docker image |
| `manager.image.tag` | `latest` | Image tag |
| `manager.service.type` | `ClusterIP` | Service type |
| `manager.service.port` | `80` | Service port |
Expand All @@ -158,7 +158,7 @@ helm install web-crawler ./web-crawler -n web-crawler --create-namespace \
|-------|---------|-------------|
| `frontend.enabled` | `true` | Deploy frontend |
| `frontend.replicaCount` | `1` | Number of replicas |
| `frontend.image.repository` | `ghcr.io/bluedotiya/web-crawler/frontend` | Docker image |
| `frontend.image.repository` | `ghcr.io/nabi-allenby/web-crawler/frontend` | Docker image |
| `frontend.image.tag` | `latest` | Image tag |
| `frontend.service.type` | `NodePort` | Service type |
| `frontend.service.port` | `80` | Service port |
Expand All @@ -175,13 +175,13 @@ Each service has its own Dockerfile:

```bash
# Manager (Rust, multi-stage build)
docker build -t ghcr.io/bluedotiya/web-crawler/manager:latest -f manager/Dockerfile .
docker build -t ghcr.io/nabi-allenby/web-crawler/manager:latest -f manager/Dockerfile .

# Feeder (Rust, multi-stage build)
docker build -t ghcr.io/bluedotiya/web-crawler/feeder:latest -f feeder/Dockerfile .
docker build -t ghcr.io/nabi-allenby/web-crawler/feeder:latest -f feeder/Dockerfile .

# Frontend (Node.js build + nginx)
docker build -t ghcr.io/bluedotiya/web-crawler/frontend:latest -f frontend/Dockerfile frontend
docker build -t ghcr.io/nabi-allenby/web-crawler/frontend:latest -f frontend/Dockerfile frontend
```

## Service Architecture in Kubernetes
Expand Down Expand Up @@ -227,9 +227,9 @@ minikube start
eval $(minikube docker-env)

# Build images directly in minikube
docker build -t ghcr.io/bluedotiya/web-crawler/manager:latest -f manager/Dockerfile .
docker build -t ghcr.io/bluedotiya/web-crawler/feeder:latest -f feeder/Dockerfile .
docker build -t ghcr.io/bluedotiya/web-crawler/frontend:latest -f frontend/Dockerfile frontend
docker build -t ghcr.io/nabi-allenby/web-crawler/manager:latest -f manager/Dockerfile .
docker build -t ghcr.io/nabi-allenby/web-crawler/feeder:latest -f feeder/Dockerfile .
docker build -t ghcr.io/nabi-allenby/web-crawler/frontend:latest -f frontend/Dockerfile frontend

# Install with local images
helm dependency build web-crawler
Expand Down
18 changes: 9 additions & 9 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,9 +103,9 @@ The Vite dev server proxies `/api/*` requests to the manager at `http://localhos
```bash
# Build images in minikube's Docker daemon, deploy with Helm, verify pods
eval $(minikube docker-env)
docker build -t ghcr.io/bluedotiya/web-crawler/manager:latest -f manager/Dockerfile .
docker build -t ghcr.io/bluedotiya/web-crawler/feeder:latest -f feeder/Dockerfile .
docker build -t ghcr.io/bluedotiya/web-crawler/frontend:latest -f frontend/Dockerfile .
docker build -t ghcr.io/nabi-allenby/web-crawler/manager:latest -f manager/Dockerfile .
docker build -t ghcr.io/nabi-allenby/web-crawler/feeder:latest -f feeder/Dockerfile .
docker build -t ghcr.io/nabi-allenby/web-crawler/frontend:latest -f frontend/Dockerfile .
helm upgrade --install web-crawler ./web-crawler -n web-crawler --create-namespace
kubectl rollout status deployment manager feeder frontend -n web-crawler
```
Expand Down Expand Up @@ -168,9 +168,9 @@ PR titles **must** follow [conventional commit](https://www.conventionalcommits.
All images are published to GitHub Container Registry:

```
ghcr.io/bluedotiya/web-crawler/feeder:<version|latest>
ghcr.io/bluedotiya/web-crawler/manager:<version|latest>
ghcr.io/bluedotiya/web-crawler/frontend:<version|latest>
ghcr.io/nabi-allenby/web-crawler/feeder:<version|latest>
ghcr.io/nabi-allenby/web-crawler/manager:<version|latest>
ghcr.io/nabi-allenby/web-crawler/frontend:<version|latest>
```

## Helm Chart OCI Publishing
Expand All @@ -179,12 +179,12 @@ The Helm chart is published as an OCI artifact — no `helm repo add` needed:

```bash
# Install directly from GHCR
helm install web-crawler oci://ghcr.io/bluedotiya/web-crawler/charts/web-crawler \
helm install web-crawler oci://ghcr.io/nabi-allenby/web-crawler/charts/web-crawler \
--version 1.0.0 -n web-crawler --create-namespace

# Pull chart locally for inspection
helm pull oci://ghcr.io/bluedotiya/web-crawler/charts/web-crawler --version 1.0.0
helm pull oci://ghcr.io/nabi-allenby/web-crawler/charts/web-crawler --version 1.0.0

# Show chart metadata
helm show all oci://ghcr.io/bluedotiya/web-crawler/charts/web-crawler --version 1.0.0
helm show all oci://ghcr.io/nabi-allenby/web-crawler/charts/web-crawler --version 1.0.0
```
2 changes: 1 addition & 1 deletion web-crawler/charts/feeder/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
replicaCount: 8

image:
repository: ghcr.io/bluedotiya/web-crawler/feeder
repository: ghcr.io/nabi-allenby/web-crawler/feeder
tag: latest
pullPolicy: IfNotPresent

Expand Down
2 changes: 1 addition & 1 deletion web-crawler/charts/frontend/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
replicaCount: 1

image:
repository: ghcr.io/bluedotiya/web-crawler/frontend
repository: ghcr.io/nabi-allenby/web-crawler/frontend
tag: latest
pullPolicy: IfNotPresent

Expand Down
2 changes: 1 addition & 1 deletion web-crawler/charts/manager/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
replicaCount: 1

image:
repository: ghcr.io/bluedotiya/web-crawler/manager
repository: ghcr.io/nabi-allenby/web-crawler/manager
tag: latest
pullPolicy: IfNotPresent

Expand Down
2 changes: 1 addition & 1 deletion web-crawler/templates/NOTES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,6 @@ Namespace: {{ .Release.Namespace }}
- Configure proper StorageClass in values.yaml for persistent storage
- Update Neo4j password in values.yaml for production use

📖 Documentation: https://github.com/bluedotiya/web-crawler
📖 Documentation: https://github.com/nabi-allenby/web-crawler

Happy Crawling! 🕷️
6 changes: 3 additions & 3 deletions web-crawler/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ feeder:
replicaCount: 8

image:
repository: ghcr.io/bluedotiya/web-crawler/feeder
repository: ghcr.io/nabi-allenby/web-crawler/feeder
tag: latest
pullPolicy: IfNotPresent

Expand Down Expand Up @@ -85,7 +85,7 @@ manager:
replicaCount: 1

image:
repository: ghcr.io/bluedotiya/web-crawler/manager
repository: ghcr.io/nabi-allenby/web-crawler/manager
tag: latest
pullPolicy: IfNotPresent

Expand Down Expand Up @@ -135,7 +135,7 @@ frontend:
replicaCount: 1

image:
repository: ghcr.io/bluedotiya/web-crawler/frontend
repository: ghcr.io/nabi-allenby/web-crawler/frontend
tag: latest
pullPolicy: IfNotPresent

Expand Down