docs(BA-3994): Add BEP-1036 for Artifact Storage Usage Tracking and Quota Enforcement

[jopemachine]

resolves #8198 (BA-3994)

Checklist: (if applicable)

• Milestone metadata specifying the target backport version
• Mention to the original issue
• Installer updates including:
  • Fixtures for db schema changes
  • New mandatory config options
• Update of end-to-end CLI integration tests in ai.backend.test
• API server-client counterparts (e.g., manager API -> client SDK)
• Test case(s) to:
  • Demonstrate the difference of before/after
  • Demonstrate the flow of abstract/conceptual models with a concrete implementation
• Documentation
  • Contents in the docs directory
  • docstrings in public interfaces and type annotations

[Copilot]

Pull request overview

Adds BEP-1036 documentation describing artifact storage usage tracking and quota enforcement for both default (StorageNamespace) and VFolder-destination imports.

Changes:

• Add draft BEP document for unified quota pre-validation and proposed API surface.
• Add supporting design notes for StorageNamespace quota and VFolder quota behavior.
• Add a Towncrier news fragment announcing the documentation addition.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
proposals/BEP-1036/vfolder_storage.md	Documents proposed VFolder quota pre-check flow and error behavior.
proposals/BEP-1036/storage_namespace.md	Documents proposed StorageNamespace quota model, aggregation query, and API.
proposals/BEP-1036-artifact-storage-quota.md	Main BEP describing unified quota service and overall system behavior.
changes/8205.docs.md	Adds a changelog/news fragment entry for the new BEP.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

[Copilot]

In the _get_quota_scope_limit() example, quota_scope_id is typed as QuotaScopeID, but the snippet calls quota_scope_id.split(":", 1). In the current codebase QuotaScopeID is an attrs class (with scope_type/scope_id) and does not implement split(), so this example would not work as written. Update the snippet to use quota_scope_id.scope_type / quota_scope_id.scope_id (or convert to str(quota_scope_id) before parsing) to align with the actual type.

Suggested change

	scope_type, scope_uuid = quota_scope_id.split(":", 1)
	scope_type = quota_scope_id.scope_type
	scope_uuid = str(quota_scope_id.scope_id)

[fregataa]

Is the new feature planned for vfolder_storage only for now?
I am curious how other storages check quota in the import flow

[jopemachine]

Details about how quota limits are handled for artifact storage are documented in storage_namespace.md in the BEP-1036 directory. (For vfolder storage, see vfolder_storage.md.) @fregataa