Skip to content

Syncs artists csv with relational and object storage - #2

Open
Ernesto905 wants to merge 21 commits into
codefordayton:mainfrom
Ernesto905:sync-artists-digital-ocean
Open

Ernesto905 wants to merge 21 commits into
codefordayton:mainfrom
Ernesto905:sync-artists-digital-ocean

Conversation

@Ernesto905

Copy link
Copy Markdown

This PR takes a local artists CSV file and adds a sync_artists django admin command, which does the following:

(1) uploads the artist's profile picture and artwork as object storage in digital ocean spaces
(2) populates the Artist and Piece tables, for both local dev (sqlite) or prod (postgres on railway)

@DavidEBest

Copy link
Copy Markdown
Contributor

This looks really nice! I'll aim to get it tested and merged later this evening.

@DavidEBest

Copy link
Copy Markdown
Contributor

I think we could clean up the file upload section by using the Django model built in functionality.

config/settings/production.py wires STORAGES['default'] to storages.backends.s3boto3.S3Boto3Storage, driven by generic env vars: S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY, S3_BUCKET_NAME, S3_ENDPOINT_URL, S3_REGION, S3_CUSTOM_DOMAIN. That'll work for Digital Ocean Spaces.When that's active, any ImageField/FileField on any model uploads through it automatically without manual boto3.client() uploads

The pattern to use instead of the manual client:

import requests
from django.core.files.base import ContentFile

resp = requests.get(wix_url, stream=True)
resp.raise_for_status()
_, ext = os.path.splitext(wix_url)
artist.profile_image.save(f"{artist.slug}{ext}", ContentFile(resp.content), save=False)
artist.save()

Django's configured storage backend handles bucket, path (upload_to='artists/profiles/'), and URL generation. artist.profile_image.url comes out correct automatically, so the template changes that swap .url for the raw field value can be reverted. Same pattern would work for Piece if it's changed from a CharField(url) to a FileField.

Few updates made to image.py to make it executable
This commit replaces the original script. Previously, the script attempted to
scrape essentialartistsdayton.org for each artist. The new approach
makes use of the Artists.csv file, which includes the slug for sample
pieces.
Profile images and artwork now uses digital ocean object storage instead of Wix
@Ernesto905
Ernesto905 force-pushed the sync-artists-digital-ocean branch from de15af7 to 18cc8a1 Compare August 15, 2026 14:56
@Ernesto905

Copy link
Copy Markdown
Author

Cleaned up the file upload section with the pattern you recommended. I ended up moving the object storage config from production.py to base.py, since i'm guess we'll want that function in development as well. Lmk if i can change that back, though

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants