Skip to content

Allow nested buckets in default_artifact_root for mlflow-server #376

Description

@toaksoy

Context

Currently the charm accepts a bucket-only value for mlflow-server’s --default-artifact-root (effectively s3://). Many deployments (e.g., MinIO Gateway or shared S3 accounts) need a stable per-env/project prefix inside a single bucket (e.g., s3://my-data-ml-platform-minio-storage-pro/mlflow).

If modified directly in the mlflow-db, nested buckets work fine and runs are not affected:

MySQL  localhost:33060+ ssl  mlflow  SQL > UPDATE experiments SET artifact_location='s3://my-data-ml-platform-minio-storage-pro/'
WHERE experiment_id='0';
Query OK, 1 rows affected (0.0003 sec)

MySQL  localhost:33060+ ssl  mlflow  SQL > select * from experiments;
+---------------+----------------------------------+--------------------------------------------------------+-----------------+---------------+------------------+
| experiment_id | name                             | artifact_location                                      | lifecycle_stage | creation_time | last_update_time |
+---------------+----------------------------------+--------------------------------------------------------+-----------------+---------------+------------------+
|             0 | Default                          | s3://my-data-ml-platform-minio-storage-pro/mlflow/ | active          | 1758620739545 |    1758620739545 |
+---------------+----------------------------------+--------------------------------------------------------+-----------------+---------------+------------------+
3 rows in set (0.0005 sec)

But the charm doesn't allow this:

ubuntu:~$ juju config mlflow-server default_artifact_root
my-data-ml-platform-minio-storage-pro/mlflow

ubuntu:~$ juju status mlflow-server
Model     Controller     Cloud/Region         Version  SLA          Timestamp
kubeflow  aws-eu-west-1  k8s-cloud/eu-west-1  3.6.9    unsupported  07:20:00Z

...

App            Version  Status   Scale  Charm          Channel      Rev  Address       Exposed  Message
mlflow-server           blocked      1  mlflow-server  2.22/stable  997  172.20.72.90  no       Invalid value for config default_artifact_root 'my-data-ml-platform-minio-storage-pro/mlflow' - value must be a v...

Unit              Workload  Agent  Address       Ports  Message
mlflow-server/0*  blocked   idle   172.22.35.82         Invalid value for config default_artifact_root 'my-data-ml-platform-minio-storage-pro/mlflow' - value must be a v...

What needs to get done

  • Replace validation to accept <bucket>[/<prefix>...] (e.g., regex like ^[^/]+(/.*)?$), preserving backward compatibility with bucket-only values

Definition of Done

juju config mlflow-server default_artifact_root=my-data-ml-platform-minio-storage-pro/mlflow

is accepted (no validation error) and the unit reaches active without restart loops.

  • MLflow server starts with --default-artifact-root s3://my-data-ml-platform-minio-storage-pro/mlflow (observable with pebble or container processes)
  • A test run’s artifacts are written under the exact prefix (mlflow/…) in the specified bucket

No activity

Activity on this issue will appear here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions