feat(BA-3209): Add `artifact_storages` common table by jopemachine · Pull Request #7057 · lablup/backend.ai

jopemachine · 2025-12-02T09:29:05Z

Checklist: (if applicable)

Mention to the original issue

📚 Documentation preview 📚: https://sorna--7057.org.readthedocs.build/en/7057/

📚 Documentation preview 📚: https://sorna-ko--7057.org.readthedocs.build/ko/7057/

Copilot

Pull request overview

This PR introduces a new artifact_storages common table to centralize storage metadata management across object storage and VFS storage backends. The change refactors the storage architecture by extracting the name column from both object_storages and vfs_storages tables into a shared artifact_storages table.

Key changes:

Added new ArtifactStorageRow model to store common metadata (name, storage_id, type) for all storage backends
Migrated existing storage names to the new artifact_storages table via Alembic migration
Updated repository queries to use eager loading (selectinload) for the new meta relationship

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
`src/ai/backend/manager/models/artifact_storages.py`	Introduces new `ArtifactStorageRow` model with bidirectional relationships to object and VFS storage tables
`src/ai/backend/manager/models/alembic/versions/35dfab3b0662_add_artifact_storages_common_table.py`	Alembic migration that creates `artifact_storages` table and migrates existing data from both storage types
`src/ai/backend/manager/models/vfs_storage.py`	Removes `name` column, adds `meta` relationship, updates `to_dataclass()` to retrieve name from relationship
`src/ai/backend/manager/models/object_storage.py`	Removes `name` column, adds `meta` relationship, updates `to_dataclass()` to retrieve name from relationship
`src/ai/backend/manager/repositories/vfs_storage/db_source/db_source.py`	Adds eager loading of `meta` relationship in all query methods to prevent N+1 queries
`src/ai/backend/manager/repositories/object_storage/db_source/db_source.py`	Adds eager loading of `meta` relationship in all query methods to prevent N+1 queries
`src/ai/backend/manager/models/__init__.py`	Registers the new `artifact_storages` module in the models package
`changes/7057.feature.md`	Changelog entry describing the new feature

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/ai/backend/manager/models/vfs_storage/row.py

src/ai/backend/manager/models/object_storage/row.py

changes/7057.feature.md

...i/backend/manager/models/alembic/versions/35dfab3b0662_add_artifact_storages_common_table.py

src/ai/backend/manager/models/vfs_storage/row.py

src/ai/backend/common/data/storage/exceptions.py

seedspirit · 2026-02-20T06:07:42Z

...i/backend/manager/models/alembic/versions/35dfab3b0662_add_artifact_storages_common_table.py

+        sa.Column("storage_id", GUID(), nullable=False),
+        sa.Column("type", sa.String(), nullable=False),
+        sa.PrimaryKeyConstraint("id", name=op.f("pk_artifact_storages")),
+        sa.UniqueConstraint("name", name=op.f("uq_artifact_storages_name")),


Is it acceptable to apply a global unique constraint to the artifact storage name? Could there be cases where the storage type differs but the names are identical? (As this is a policy matter, please verify this for me.)

I think since artifact storage management is an admin-only feature, it doesn’t look awkward so far.
If, in the future, artifact storage management becomes available to regular users as well, it would make more sense to adjust the constraints at that point.

src/ai/backend/manager/models/artifact_storages.py

src/ai/backend/common/types.py

fregataa

I think that using SQLAlchemy Table inheritance would be great here.

src/ai/backend/manager/models/artifact_storages.py

src/ai/backend/manager/data/artifact_storages/types.py

src/ai/backend/manager/repositories/vfs_storage/db_source/db_source.py

- Add artifact_storages table and migration script - Move name column to artifact_storages - Add proper selectinload in db_source - Add ID type and update version - Append admin prefix - Fix rename types and error domain - Add news fragment

…experimetal)

Co-authored-by: octodog <mu001@lablup.com>

fregataa

Are there storage insert/update unit tests? It would be good to have them.
Plus, how do you think of Single Table Inheritance? Update and insert execution got complicated because of joined table inheritance, so I guess single table one makes it easy

fregataa · 2026-03-04T04:48:40Z

src/ai/backend/manager/repositories/vfs_storage/db_source/db_source.py

+        Uses plain UPDATE + SELECT instead of execute_updater's RETURNING+from_statement,
+        which is incompatible with JTI (RETURNING only includes child table columns).


is it not allowed to update VFSStorageRow?

jopemachine · 2026-03-04T05:07:59Z

Are there storage insert/update unit tests? It would be good to have them. Plus, how do you think of Single Table Inheritance? Update and insert execution got complicated because of joined table inheritance, so I guess single table one makes it easy

I believe STI would not good for extensibility because it will introduce many nullable columns.

github-actions bot assigned jopemachine Dec 2, 2025

github-actions bot added size:L 100~500 LoC comp:manager Related to Manager component require:db-migration Automatically set when alembic migrations are added or updated labels Dec 2, 2025

jopemachine changed the title ~~feat: Add artifact_storages common table~~ feat(BA-3209): Add artifact_storages common table (WIP) Dec 2, 2025

jopemachine changed the title ~~feat(BA-3209): Add artifact_storages common table (WIP)~~ feat(BA-3209): Add artifact_storages common table Dec 3, 2025

jopemachine requested review from HyeockJinKim and seedspirit December 3, 2025 05:15

jopemachine marked this pull request as ready for review December 3, 2025 05:15

Copilot AI review requested due to automatic review settings December 3, 2025 05:15

Copilot started reviewing on behalf of jopemachine December 3, 2025 05:16 View session

Copilot finished reviewing on behalf of jopemachine December 3, 2025 05:17

Copilot AI reviewed Dec 3, 2025

View reviewed changes

jopemachine added this to the 25.19 milestone Dec 16, 2025

jopemachine marked this pull request as draft December 16, 2025 07:33

HyeockJinKim force-pushed the main branch 2 times, most recently from 9552aac to 4af738e Compare December 31, 2025 15:41

jopemachine modified the milestones: 25.19, 26.1 Jan 5, 2026

jopemachine force-pushed the feat/BA-3209 branch from ecaaa5a to bc37c21 Compare January 5, 2026 09:02

jopemachine marked this pull request as ready for review January 5, 2026 09:32

seedspirit reviewed Jan 5, 2026

View reviewed changes

src/ai/backend/manager/models/vfs_storage/row.py Outdated Show resolved Hide resolved

jopemachine force-pushed the feat/BA-3209 branch from 55612c7 to ab42a38 Compare January 9, 2026 08:58

jopemachine marked this pull request as draft January 9, 2026 09:09

jopemachine removed this from the 26.1 milestone Jan 26, 2026

jopemachine added this to the 26.2 milestone Feb 4, 2026

jopemachine force-pushed the feat/BA-3209 branch from 6a86174 to 4bedf71 Compare February 4, 2026 04:41

github-actions bot added size:XL 500~ LoC area:docs Documentations and removed size:L 100~500 LoC labels Feb 4, 2026

jopemachine requested a review from seedspirit February 19, 2026 08:14

seedspirit reviewed Feb 20, 2026

View reviewed changes

jopemachine force-pushed the feat/BA-3209 branch from 05278f8 to 46220f3 Compare February 24, 2026 06:19

jopemachine requested review from fregataa and seedspirit February 24, 2026 06:19

fregataa reviewed Feb 25, 2026

View reviewed changes

src/ai/backend/manager/models/artifact_storages.py Outdated Show resolved Hide resolved

src/ai/backend/manager/models/artifact_storages.py Show resolved Hide resolved

fregataa requested changes Feb 25, 2026

View reviewed changes

src/ai/backend/manager/data/artifact_storages/types.py Outdated Show resolved Hide resolved

src/ai/backend/manager/repositories/vfs_storage/db_source/db_source.py Outdated Show resolved Hide resolved

github-actions bot added size:M 30~100 LoC comp:agent Related to Agent component comp:client Related to Client component comp:cli Related to CLI component and removed size:XL 500~ LoC labels Feb 26, 2026

jopemachine force-pushed the feat/BA-3209 branch from 12f09fb to 4200531 Compare February 26, 2026 04:03

github-actions bot added size:XL 500~ LoC and removed size:M 30~100 LoC labels Feb 26, 2026

jopemachine requested review from a team and fregataa March 4, 2026 03:44

jopemachine and others added 8 commits March 4, 2026 03:59

feat: Add artifact_storages table

7217e21

- Add artifact_storages table and migration script - Move name column to artifact_storages - Add proper selectinload in db_source - Add ID type and update version - Append admin prefix - Fix rename types and error domain - Add news fragment

fix: Reflect feedback

de9bec3

fix: alembic conflcit

984ec0f

fix: Apply SQLAlchemy Joined Table Inheritance to artifact_storages (…

ecdb3b2

…experimetal)

chore: update api schema dump

9c1d73b

Co-authored-by: octodog <mu001@lablup.com>

fix: Migration script

1e0a7ad

wip

032aab3

wip

5371c73

jopemachine force-pushed the feat/BA-3209 branch from 3e05451 to 5371c73 Compare March 4, 2026 04:00

test: Fix Broken tests

8be3451

fregataa reviewed Mar 4, 2026

View reviewed changes

test: Add

18253bc

		Uses plain UPDATE + SELECT instead of execute_updater's RETURNING+from_statement,
		which is incompatible with JTI (RETURNING only includes child table columns).

Conversation

jopemachine commented Dec 2, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

seedspirit Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

jopemachine Feb 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

fregataa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fregataa left a comment

Choose a reason for hiding this comment

Uh oh!

fregataa Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

jopemachine commented Mar 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jopemachine commented Dec 2, 2025 •

edited by github-actions bot

Loading

jopemachine Feb 20, 2026 •

edited

Loading