Feat/migrate to s3

.github/workflows/terraform-lock.yml

@@ -23,7 +23,7 @@ jobs:
       fail-fast: false
       matrix:
         root:
-          - state-backend
+          - terraform-state-bucket
           - cluster/local
           - cluster/scaleway
           - github-ci
@@ -39,6 +39,7 @@ jobs:
 
       - name: Re-lock providers for both platforms and assert no drift
         run: |
+          terraform -chdir=${{ matrix.root }} init -backend=false
           terraform -chdir=${{ matrix.root }} providers lock \
             -platform=darwin_arm64 -platform=linux_amd64
           git diff --exit-code -- ${{ matrix.root }}/.terraform.lock.hcl

CLAUDE.md

@@ -17,7 +17,7 @@ mise run dev             # Full local env: start minikube + terraform init + app
 mise run reset           # Destroy minikube cluster
 
 # Provider lock files
-mise run lock            # Re-generate all 3 roots' .terraform.lock.hcl for darwin_arm64 + linux_amd64
+mise run lock            # Re-generate all 4 roots' .terraform.lock.hcl for darwin_arm64 + linux_amd64
 
 # Linting
 actionlint .github/workflows/*.yml   # Lint GitHub Actions workflows (also runs as pre-push hook)
@@ -34,36 +34,45 @@ The `.terraform.lock.hcl` in each root must cover **both** `darwin_arm64` (local
 `mise run lock` is equivalent to:
 
 ```bash
-terraform -chdir=state-backend    providers lock -platform=darwin_arm64 -platform=linux_amd64
-terraform -chdir=cluster/local    providers lock -platform=darwin_arm64 -platform=linux_amd64
-terraform -chdir=cluster/scaleway providers lock -platform=darwin_arm64 -platform=linux_amd64
-terraform -chdir=github-ci        providers lock -platform=darwin_arm64 -platform=linux_amd64
+terraform -chdir=terraform-state-bucket providers lock -platform=darwin_arm64 -platform=linux_amd64
+terraform -chdir=cluster/local          providers lock -platform=darwin_arm64 -platform=linux_amd64
+terraform -chdir=cluster/scaleway       providers lock -platform=darwin_arm64 -platform=linux_amd64
+terraform -chdir=github-ci              providers lock -platform=darwin_arm64 -platform=linux_amd64
 ```
 
 Commit the updated lock files alongside the version change.
 
 ## Architecture
 
-Two cluster environments, each its own Terraform root module:
+Several Terraform root modules. One (./terraform-state-bucket) manage the shared terraform state bucket; any
+others cloud-based environments will store it's data on it.
 
 ```
+terraform-state-bucket/   # shared AWS S3 bucket holding every root's remote state
 cluster/
-  local/      # minikube — local dev and debugging
+  local/      # minikube — local dev and debugging. Using local backend (e.g. local files)
   scaleway/   # Scaleway Kapsule cluster + ArgoCD bootstrap (homelab; WIP, not yet wired into mise)
+github-ci/    # Scaleway IAM identity GitHub Actions authenticates with
 ```
 
+Otherwise, this repo, for now, is an agregate of terraform root modules without specific structure yet.
+
+### `./clusters/*`
+
+Terraform here is only a **one-time bootstrapper** — everything after ArgoCD is up lives in the `gitops` repo. The cluster internal state nor status will be reflected in the terraform state. 
+
 ### `cluster/local/`
 
+Warning : This environment expect you an accessible local kubernetes cluster access, likely configured within your ~/.kube/config. This is automatically handled via `mise run dev`
+
 Two-step, one-time bootstrap:
 1. Fetch secrets from **Infisical** (universal auth machine identity). Credentials come from `nico.auto.tfvars` (per-developer, not shared).
 2. Deploy **ArgoCD** via Helm with the admin bcrypt password hash from Infisical (pre-hashed to prevent Terraform drift).
-3. Deploy the **argocd-apps bootstrap** chart, pointing ArgoCD at `https://github.com/IntegratedDynamic/gitops.git`. ArgoCD then self-manages all further cluster state from that separate GitOps repo.
-
-Terraform here is only a **one-time bootstrapper** — everything after ArgoCD is up lives in the `gitops` repo.
+3. Deploy the **argocd-apps bootstrap** Application, pointing ArgoCD at `https://github.com/IntegratedDynamic/gitops.git`. ArgoCD then self-manages all further cluster state from that separate GitOps repo.
 
 ### `cluster/scaleway/`
 
-Same bootstrap pattern as `local/`, plus the Kapsule cluster + node pool (`DEV1-M`, min=0/max=3) in one consolidated module. Writes the kubeconfig to `~/.kube/scaleway-homelab.yaml`. Scaleway credentials are read from the `scw` CLI config (`~/.config/scw/config.yaml`), not from tfvars. Still early — intentionally undocumented in the commands above for now.
+Same bootstrap pattern as `local/`, but with the Kapsule cluster + node pool (`DEV1-M`, min=0/max=3) instead.
 
 ### `github-ci/`
 

README.md

@@ -1,5 +1,13 @@
 # Infrastructure
 
+This infrastructure contains the actual ScalePack infrastructure.
+
+All remote based environment states are stored under `./terraform-state-bucket` bucket. Check `version.tf` files from any root module in this repository for an example, stored under eu-west-3 region.
+
+## Forking
+
+Forking support process coming soon.
+
 ## Onboarding
 
 ### Installation des dépendances
@@ -17,3 +25,49 @@ Cette commande installe toutes les dépendances du projet définies dans `mise.t
 ```
 
 Cela configure les git hooks locaux pour automatiser les vérifications avant les commits/pushes.
+
+### Login with your credentials
+
+> Skip 2 and more if you only need local environment. 1 is kept because right now, Infisical SSO features are PayGated, but allow long lived credentials within your environment. All hail long lived credentials! (Don't forget to look into your own local secret encryption solution to avoid storing them in clear. In my case, macOS built-in's `security` binary is enough. See https://dev.to/alsaheem/how-to-store-secrets-in-the-mac-keychain-and-use-them-like-environment-variables-1aj7).
+
+1. Infisical
+
+    Why Infisical? It's a free SSO organization solution for External Infrastructure Secrets. Will ultimately rely on External Secret Operator to synchronize over things. Somehow.
+    1. Login to : https://app.infisical.com/login with SSO
+    2. Create API_KEYS for your local environment.
+    3. Create other long lived credentials you want for your infrastructure in this External Storage SaaS solution.
+
+2. AWS as terraform remote backend solution : 
+
+    Why AWS S3? It's extremely cheap, great DevX for mono storage management (fine grained IAM policies) and AWS includes free SSO setup for devops without friction. And Scaleway doesn't provide OIDC yet, which I do favor when it comes to CI runners, especially for such high-risk files.
+
+    Run `aws sso login`. This needs to be done daily and at bootstrap before running any terraform command locally, or state fetch will fail. 
+
+    #### AWS SSO first setup: 
+
+    1. Go to https://d-806761a6d4.awsapps.com/start/#/
+
+    2. Login with your credentials. Contact your manager or an admin if you didn't get one yet via email.
+
+    3. Follow instructions for an AccessKey, and follow SSO setup instructions.
+
+    Public documentation available here : https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html#sso-configure-profile-token-auto-sso
+
+
+2. Scaleway as cloud provider
+    > Scaleway is the current provider I want to use, because it offers :
+        > 1. A cheap way to operate Kubernetes clusters without too many issues. I mean, I want to play with Kubernetes, and this GitHub org is my homelab. And I can have as many clusters as I want; as long as there's no node within, I won't pay. It's a great FinOps safeguard which I do want to take advantage of. Deal with it.
+        > 2. As an EU citizen, I disapprove of the Cloud Act and any of those attacks against security, which in the end create a backdoor. I see those legalized actions as one. And yes, my supply chain includes storing extremely sensitive data in some of those companies (Infisical in free mode, Google, AWS...). At least I intend to use Scaleway, and have I told you about their DevX around Kubernetes managed service? It's within the EU, and not exposed to that, at least. That's my attempt toward sovereign data. With love <3
+
+    #### Scaleway CLI/terraform bootstrap
+    1. Log in with SSO with Scaleway : https://console.scaleway.com/organization
+
+    2. Go to https://console.scaleway.com/iam/users
+
+    3. Create an API KEY.
+
+    4. [Optional] : I recommend one dedicated API KEY per tool. But unlike AWS, there is no rich default option with the Scaleway CLI that works out of the box, which is one of my requirements (passwordless/secretless CLI access for humans via SSO + passwordless/secretless for machines via OIDC). But, free kube, okay? Just grab your API keys — one for AWS, another for Terraform in external runtime contexts (kube, GitHub Actions, ...). This at least distinguishes local usage from "safe/legit" remote usage such as organization trusted runtimes. In the end, the more fine-grained, the better, but I'm not at that level yet :D
+
+### Start playing
+
+[WIP] This section will be added soon, and will explain how to start/stop the cluster nodes (and whatever makes me pay for something), in order to stop paying the bill, such as whenever I should sleep, or should not work. But right now, it's manual, and I will not explain it. Homelab, remember?

cluster/scaleway/version.tf

@@ -1,22 +1,12 @@
 terraform {
   # Remote state in the org-wide bucket (state-backend/). Creds: see mise.toml.
   backend "s3" {
-    bucket = "id-terraform-state"
-    key    = "cluster/scaleway/terraform.tfstate"
-    region = "fr-par"
-
-    # Root-specific prefix so this root's workspaces don't mix with others'.
+    bucket = "id-terraform-state20260612164136440800000001"
+    region = "eu-west-3"
     workspace_key_prefix = "cluster/scaleway"
-
-    endpoints = { s3 = "https://s3.fr-par.scw.cloud" }
-
-    # Disable the backend's AWS-only preflight checks (IMDS, STS account-id,
-    # region allowlist): Scaleway speaks the S3 API but isn't AWS itself.
-    skip_credentials_validation = true
-    skip_region_validation      = true
-    skip_requesting_account_id  = true
-    skip_metadata_api_check     = true
-    skip_s3_checksum            = true
+    key    = "terraform.tfstate"
+    encrypt = true
+    use_lockfile = true
   }
 
   required_providers {

github-ci/version.tf

@@ -1,24 +1,15 @@
 terraform {
-  # Remote state in the org-wide bucket (state-backend/). Creds: see mise.toml.
   backend "s3" {
-    bucket = "id-terraform-state"
-    key    = "github-ci/terraform.tfstate"
-    region = "fr-par"
-
-    # Root-specific prefix so this root's workspaces don't mix with others'.
+    bucket = "id-terraform-state20260612164136440800000001"
+    region = "eu-west-3"
     workspace_key_prefix = "github-ci"
-
-    endpoints = { s3 = "https://s3.fr-par.scw.cloud" }
-
-    # Disable the backend's AWS-only preflight checks (IMDS, STS account-id,
-    # region allowlist): Scaleway speaks the S3 API but isn't AWS itself.
-    skip_credentials_validation = true
-    skip_region_validation      = true
-    skip_requesting_account_id  = true
-    skip_metadata_api_check     = true
-    skip_s3_checksum            = true
+    key    = "terraform.tfstate"
+    encrypt = true
+    use_lockfile = true
   }
 
+
+
   required_providers {
     scaleway = {
       source  = "scaleway/scaleway"

mise.toml

@@ -6,15 +6,6 @@ actionlint = "1.7"
 helm = "4.1.3"
 argocd = "3.3.6"
 
-# The s3 state backend (state-backend/) needs AWS-style creds — the Scaleway
-# provider reads the scw CLI config, the backend does not. Derive them here so
-# terraform just works; `|| true` keeps the local/minikube workflow usable
-# without scw configured (empty vars, which the local backend ignores).
-# https://registry.terraform.io/providers/scaleway/scaleway/2.75.0/docs/guides/backend_guide
-[env]
-AWS_ACCESS_KEY_ID = "{{ exec(command='scw config get access-key 2>/dev/null || true') }}"
-AWS_SECRET_ACCESS_KEY = "{{ exec(command='scw config get secret-key 2>/dev/null || true') }}"
-
 # ── Local (minikube) ────────────────────────────────────────────────────────
 
 [tasks.minikube]
@@ -117,8 +108,8 @@ run = "terraform apply"
 [tasks.lock]
 description = "Re-generate all .terraform.lock.hcl files for darwin_arm64 + linux_amd64"
 run = """
-terraform -chdir=state-backend    providers lock -platform=darwin_arm64 -platform=linux_amd64
-terraform -chdir=cluster/local    providers lock -platform=darwin_arm64 -platform=linux_amd64
-terraform -chdir=cluster/scaleway providers lock -platform=darwin_arm64 -platform=linux_amd64
-terraform -chdir=github-ci        providers lock -platform=darwin_arm64 -platform=linux_amd64
+terraform -chdir=terraform-state-bucket providers lock -platform=darwin_arm64 -platform=linux_amd64
+terraform -chdir=cluster/local          providers lock -platform=darwin_arm64 -platform=linux_amd64
+terraform -chdir=cluster/scaleway       providers lock -platform=darwin_arm64 -platform=linux_amd64
+terraform -chdir=github-ci              providers lock -platform=darwin_arm64 -platform=linux_amd64
 """