Skip to content

Explore testing options for feast-operator rock #50

Open

Description

@misohu

Context

We want to create a rock for the quay.io/feastdev/feast-operator:0.49.0. As part of this issue we want to explore how we can use this operator rock and we want to design some testing Jupyter notebook.

What needs to get done

  1. Explore how is the feast-operator image used upstream.

Definition of Done

  1. Documentation about the usage of the image
  2. Testing jupyter notebook with code on how to test the rock

Activity

  1. syncronize-issues-to-jira commented on Jul 31, 2025

    @syncronize-issues-to-jira

    Thank you for reporting your feedback to us!

    The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-7809.

    This message was autogenerated

  2. misohu commented on Aug 5, 2025

    @misohu
    MemberAuthor

    Investigation: quay.io/feastdev/feast-operator:0.49.0 Docker Image

    Docker Image

    The image in question is:

    quay.io/feastdev/feast-operator:0.49.0
    

    This image is built from the following Dockerfile in the Feast upstream repository:
    https://github.com/feast-dev/feast/blob/v0.49.0/infra/feast-operator/Dockerfile

    This Docker image contains the Feast Kubernetes Operator — a controller process that manages FeatureStore custom resources in Kubernetes and orchestrates the deployment of Feast services based on declarative specifications.

    How the Image Is Used (Feast Operator)

    The image is used as the core component of the Feast Operator, which runs as a Kubernetes controller and performs the following functions:

    • Watches for custom resources of type FeatureStore in the cluster.
    • Clones Feast project repositories or runs feast init to bootstrap feature stores.
    • Deploys associated services such as:
      • Feature server (online store)
      • Offline store
      • Registry
      • Feast UI (optional)
    • Handles configuration for persistence (PVCs, databases), authentication (RBAC/OIDC), and TLS.
    • Sets up materialization jobs via Kubernetes CronJobs.
    • Reconciles the state automatically when specs change.

    How the CRD Is Used

    The Feast Operator defines and manages a Kubernetes Custom Resource Definition (CRD) called FeatureStore.

    CRD source:
    https://github.com/feast-dev/feast/blob/v0.49.0/infra/feast-operator/bundle/manifests/feast.dev_featurestores.yaml

    This CRD allows users to declaratively define the configuration of an entire Feast feature store deployment in Kubernetes.

    A minimal example:

    apiVersion: feast.dev/v1alpha1
    kind: FeatureStore
    metadata:
      name: my-featurestore
    spec:
      feastProject: fraud_detection
      feastProjectDir:
        git:
          url: https://github.com/my-org/feast-repo.git
          ref: main
      services:
        onlineStore:
          store:
            type: redis
            secretRef:
              name: redis-secret
        offlineStore:
          store:
            type: bigquery
            secretRef:
              name: bq-secret

    Key CRD Sections

    Services
    Under spec.services, users can configure:

    • Feature server (online store)
    • Offline store
    • Registry
    • UI (optional)

    Each can be configured with its own persistence, environment, resource limits, TLS, etc.

    CronJob
    Under spec.cronJob, users can define periodic materialization jobs using Kubernetes CronJob syntax:

    cronJob:
      schedule: "0 * * * *"
      containerConfigs:
        commands: ["feast materialize-incremental ..."]

    Persistence
    Persistence is configurable for each component:

    • File-based with PVCs
    • Store-based with database integrations (Redis, BigQuery, Snowflake, etc.)

    Conclusion

    Although the feast-operator and its image (quay.io/feastdev/feast-operator:0.49.0) provide a Kubernetes-native, declarative, and automated way to manage Feast deployments, we cannot directly use this image in our current setup.

    Reasons:

    • Our implementation uses a custom feast_integrator and Feast UI, which do not conform to the upstream operator model.
    • The operator expects to own the deployment lifecycle and assumes responsibility for cloning repos, initializing projects, and managing services — which would conflict with our current architecture.

    Therefore, adopting the Feast Operator as-is would require significant changes to how we deploy and integrate Feast.

    Note: when designing our feast bundle this solution was not available.

  3. misohu commented on Aug 6, 2025

    @misohu
    MemberAuthor

    Testing the Feast Operator with a custom image

    This document outlines the steps to test a custom Feast Operator image.

    1. Clone the Feast repository

    git clone [https://github.com/feast-dev/feast](https://github.com/feast-dev/feast)
    cd feast
    git checkout v0.49.0

    2. Replace the Feast Operator image

    Navigate to the feast/infra/feast-operator/ directory and replace the image in the bundle/manifests/feast-operator.clusterserviceversion.yaml file.

    This script replaces the default image with your desired image

    cd infra/feast-operator/
    IMAGE_TAG="<YOUR_IMAGE_TAG>" # e.g., my-registry/my-feast-operator:latest
    sed -i "s|quay.io/feastdev/feast-operator:0.49.0|${IMAGE_TAG}|g" bundle/manifests/feast-operator.clusterserviceversion.yaml
    

    3. Install Go and Kubernetes

    Make sure you have Go version 1.23 or later installed.
    You can follow the official instructions to install Go:
    https://go.dev/doc/install

    Ensure you have a local Kubernetes cluster running.

    4. Deploy the Feast Operator

    From the feast/infra/feast-operator/ directory, run the following commands to install the CRDs and deploy the operator:

    make install
    make deploy
    

    This will deploy the Feast CRDs and the operator to your Kubernetes cluster.

    5. Verify the Operator Pod

    The Feast Operator should now be running in the feast-operator-system namespace. You can verify this with the following command:

    kubectl get pods -n feast-operator-system
    You should see a pod similar to this:
    NAME                                                READY   STATUS    RESTARTS   AGE
    feast-operator-controller-manager-94fc86679-h59mz   1/1     Running   0          166m
    

    6. Run a sample configuration

    You can now apply one of the sample configurations to create a Feast feature store. For example:

    kubectl apply -f config/samples/v1alpha1_featurestore_git_repopath.yaml
    This should create a new pod in your current namespace, such as:
    feast-sample-git-repopath-7ff67bb5b5-6fjk2
    

    You can verify the pod status with:

    kubectl get pods
    
  4. misohu commented on Aug 8, 2025

    @misohu
    MemberAuthor

    Updated Feast Operator Testing Instructions (Simplified Approach)

    After practical testing, the simplest and most effective way to test a custom Feast image is by deploying the upstream Feast operator without modifying the repo, and then manually updating the image using kubectl edit. This avoids the need to recompile or modify manifests manually.

    Steps

    1. Clone and checkout Feast repo (optional, for reference)

      git clone https://github.com/feast-dev/feast
      cd feast
      git checkout v0.49.0
      cd infra/feast-operator/
    2. Ensure prerequisites

      • Go 1.23 installed:
        wget https://go.dev/dl/go1.23.0.linux-amd64.tar.gz
        sudo rm -rf /usr/local/go && sudo tar -C /usr/local -xzf go1.23.0.linux-amd64.tar.gz
        export PATH=$PATH:/usr/local/go/bin
        go version
      • Local Kubernetes cluster (kind, minikube, etc.)
    3. Deploy the upstream Feast Operator

      make install
      make deploy
    4. Edit the deployment to use your custom image

      kubectl edit deploy feast-operator-controller-manager -n feast-operator-system

      In the opened editor, find and replace the line under spec.template.spec.containers.image:

      image: quay.io/feastdev/feast-operator:0.49.0

      with:

      image: <your-custom-image>
    5. Verify that the new image is in use

      kubectl get pods -n feast-operator-system-<id> -o wide
    6. Apply a sample config to trigger the operator

      kubectl apply -f config/samples/v1alpha1_featurestore_git_repopath.yaml

      You should now see a running pod like:

      kubectl get pods
      # feast-sample-git-repopath-xxxxx   1/1   Running   0   1m
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions