📖 docs: update documentation from PR #766

docs/development/architecture.md

@@ -37,7 +37,7 @@ The transform phase uses a multi-stage Kustomize pipeline:
 
 1. **Stage discovery** — Scans `transform/` for existing stage directories matching `<number>_<name>` pattern
 2. **Plugin execution** — For plugin-based stages (name ending in `Plugin`), loads and runs the matching plugin to generate JSONPatch operations
-3. **Resource writing** — Writes individual resource files to `<stage>/input/`
+3. **Resource writing** — Writes individual resource files to `<stage>/input/` or `<stage>/new/` (for plugin-generated new resources)
 4. **Patch writing** — Writes plugin-generated patches to `<stage>/patches/`
 5. **Kustomization generation** — Generates `kustomization.yaml` linking resources and patches
 
@@ -48,7 +48,7 @@ The transform phase uses a multi-stage Kustomize pipeline:
 
 ### Sequential Consistency
 
-In multi-stage pipelines, each stage runs on the fully materialized output of the previous stage (not raw patches). Each stage contains `input/`, `patches/`, and `output/` directly within the stage directory (e.g., `transform/<stage>/input/`, `transform/<stage>/output/`).
+In multi-stage pipelines, each stage runs on the fully materialized output of the previous stage (not raw patches). Each stage contains `input/`, `patches/`, `output/` and optionally `new/` directly within the stage directory (e.g., `transform/<stage>/input/`, `transform/<stage>/new/`).
 
 ### Key Components
 
@@ -88,6 +88,8 @@ Plugins are external binaries that:
 
 The built-in `KubernetesPlugin` (from `crane-lib`) removes server-managed fields like `metadata.uid`, `metadata.resourceVersion`, `metadata.creationTimestamp`, `metadata.managedFields`, and `status`.
 
+Plugins may optionally generate entirely new resources. These are stored in the `new/` directory within a stage, represented as a minimal skeleton resource and a JSON patch that defines the full object state.
+
 ## PVC Transfer
 
 **Package:** `cmd/transfer-pvc/`

docs/development/plugin-development.md

@@ -6,8 +6,8 @@ Crane uses a plugin system for transforming Kubernetes resources during migratio
 
 1. Crane discovers plugin binaries in the plugin directory (`~/.local/share/crane/plugins/` by default)
 2. During transform, each plugin receives a Kubernetes resource on stdin
-3. The plugin analyzes the resource and returns JSONPatch operations on stdout
-4. Crane writes the patches to the stage's `patches/` directory
+3. The plugin analyzes the resource and returns a PluginResponse object on stdout, which contains JSONPatch operations and optional new resources
+4. Crane writes the patches to the stage's `patches/` directory and new resources to the `new/` directory
 5. During apply, the embedded kustomize engine applies the patches to the resources
 
 ## Plugin Interface
@@ -40,18 +40,27 @@ A single Kubernetes resource in JSON format:
 
 ### Output (stdout)
 
-A JSON array of RFC 6902 JSONPatch operations:
+Plugins return a PluginResponse object containing an optional array of RFC 6902 JSONPatch operations in the `patches` field. Additionally, plugins can optionally return entirely new resources in the `newResources` field to be added to the transformation pipeline.
 
 ```json
-[
-  {"op": "remove", "path": "/metadata/uid"},
-  {"op": "remove", "path": "/metadata/resourceVersion"},
-  {"op": "remove", "path": "/status"},
-  {"op": "add", "path": "/metadata/labels/migrated", "value": "true"}
-]
+{
+  "version": "v1",
+  "isWhiteOut": false,
+  "patches": [
+    {"op": "remove", "path": "/metadata/uid"}
+  ],
+  "newResources": [
+    {
+      "apiVersion": "v1",
+      "kind": "ConfigMap",
+      "metadata": {"name": "generated-config", "namespace": "default"},
+      "data": {"key": "value"}
+    }
+  ]
+}
 ```
 
-Return an empty array `[]` if no transformations are needed.
+Return an empty array for `patches` if no transformations are needed. `newResources` is optional; if provided, each resource must contain a valid `kind`, `name`, and `apiVersion`.
 
 ## Writing a Plugin in Go
 
@@ -60,52 +69,32 @@ package main
 
 import (
     "encoding/json"
-    "fmt"
     "os"
 
     "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
+    "github.com/evanphx/json-patch"
+    cranelib "github.com/konveyor/crane-lib/transform"
 )
 
-type PatchOp struct {
-    Op    string      `json:"op"`
-    Path  string      `json:"path"`
-    Value interface{} `json:"value,omitempty"`
-}
-
 func main() {
     var resource unstructured.Unstructured
     if err := json.NewDecoder(os.Stdin).Decode(&resource); err != nil {
-        fmt.Fprintf(os.Stderr, "failed to decode resource: %v\n", err)
         os.Exit(1)
     }
 
-    var patches []PatchOp
-
-    // Example: add a migration label (ensure parent object exists first)
-    patches = append(patches, PatchOp{
-        Op:    "add",
-        Path:  "/metadata/labels",
-        Value: map[string]interface{}{},
-    })
-    patches = append(patches, PatchOp{
-        Op:    "add",
-        Path:  "/metadata/labels/migrated-by",
-        Value: "my-plugin",
-    })
-
-    // Example: remove a specific annotation
-    annotations := resource.GetAnnotations()
-    if annotations != nil {
-        if _, ok := annotations["source-cluster-only"]; ok {
-            patches = append(patches, PatchOp{
-                Op:   "remove",
-                Path: "/metadata/annotations/source-cluster-only",
-            })
-        }
+    // Build JSONPatch operations
+    patchData := []byte(`[{"op": "add", "path": "/metadata/labels/migrated", "value": "true"}]`)
+    patches, err := jsonpatch.DecodePatch(patchData)
+    if err != nil {
+        os.Exit(1)
     }
 
-    if err := json.NewEncoder(os.Stdout).Encode(patches); err != nil {
-        fmt.Fprintf(os.Stderr, "failed to encode patches: %v\n", err)
+    response := cranelib.PluginResponse{
+        Version: "v1",
+        Patches: patches,
+    }
+
+    if err := json.NewEncoder(os.Stdout).Encode(response); err != nil {
         os.Exit(1)
     }
 }
@@ -121,14 +110,11 @@ go build -o ~/.local/share/crane/plugins/MyCustomPlugin
 
 ```bash
 #!/bin/bash
-# Simple plugin that adds a label to all resources
-# Note: ensure /metadata/labels exists before adding child keys
-
+# Simple plugin that returns patches
 cat <<EOF
-[
-  {"op": "add", "path": "/metadata/labels", "value": {}},
-  {"op": "add", "path": "/metadata/labels/environment", "value": "production"}
-]
+{
+  "patches": [{"op": "add", "path": "/metadata/labels/environment", "value": "production"}]
+}
 EOF
 ```
 

docs/multistage-pipeline.md

@@ -19,6 +19,7 @@ transform/
 │   │   ├── deployment.yaml          # Grouped by resource type
 │   │   ├── service.yaml
 │   │   └── configmap.yaml
+│   ├── new/                          # Plugin-generated new resources
 │   ├── patches/
 │   │   ├── deployment-myapp-default.yaml
 │   │   └── service-myapp-default.yaml
@@ -36,11 +37,12 @@ transform/
 Resources are grouped by type (kind + API group) into multi-document YAML files:
 - Core resources: `deployment.yaml`, `service.yaml`, `pod.yaml`
 - Non-core resources: `route.route.openshift.io.yaml`, `imagestream.image.openshift.io.yaml`
+- New resources: Generated resources are written to the `new/` directory as minimal skeletons and reconstructed using patches.
 
 ### Kustomization File
 
 Each stage contains a `kustomization.yaml` that references:
-- **resources**: List of resource files from the `input/` directory
+- **resources**: List of resource files from the `input/` and `new/` directories
 - **patches**: Strategic merge patches or JSON patches with target selectors
 
 Example:
@@ -49,15 +51,14 @@ apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 resources:
 - input/deployment.yaml
-- input/service.yaml
+- new/Build.yaml
 patches:
-- path: patches/deployment-myapp-default.yaml
+- path: patches/Build-my-app-build.yaml
   target:
-    group: apps
-    version: v1
-    kind: Deployment
-    name: myapp
-    namespace: default
+    group: shipwright.io
+    version: v1beta1
+    kind: Build
+    name: my-app-build
 ```
 
 ### Resource Cleanup via Plugins
@@ -75,29 +76,10 @@ The `kubernetes` plugin (from `crane-lib`) automatically removes:
 
 **How it works**:
 
-1. **Transform phase**: Plugins analyze exported resources and generate JSONPatch operations
-2. **Patches written**: Operations saved as Kustomize patches in `patches/` directory
-3. **Apply phase**: Embedded kustomize applies patches to resources
-4. **Result**: Clean, declarative manifests ready for target cluster
-
-**Example patch** (auto-generated by kubernetes plugin):
-```yaml
-- op: remove
-  path: /metadata/resourceVersion
-- op: remove
-  path: /metadata/creationTimestamp
-- op: remove
-  path: /metadata/managedFields
-- op: remove
-  path: /status
-- op: remove
-  path: /metadata/uid
-```
-
-This approach ensures:
-- No conflicts with server-managed fields
-- Resources can be applied to any cluster
-- Idempotent operations (safe to re-apply)
+1. **Transform phase**: Plugins analyze exported resources and generate JSONPatch operations. Plugins may also generate entirely new resources.
+2. **Patches and resources written**: JSONPatch operations are saved in the `patches/` directory. Generated resource skeletons are saved in the `new/` directory.
+3. **Apply phase**: Embedded kustomize applies patches to resources.
+4. **Result**: Clean, declarative manifests ready for target cluster.
 
 ## CLI Usage