Skip to content

S3 reading support#198

Draft
Bisaloo wants to merge 8 commits into
HelenaLC:mainfrom
Bisaloo:s3-read
Draft

S3 reading support#198
Bisaloo wants to merge 8 commits into
HelenaLC:mainfrom
Bisaloo:s3-read

Conversation

@Bisaloo

@Bisaloo Bisaloo commented May 15, 2026

Copy link
Copy Markdown
Collaborator
  • Enable reading on S3 (excepted for the tables element, as this needs to be implemented separately in anndataR. Prerequisite: Consolidated Zarr metadata at the root of the anndata zarr store scverse/anndataR#461)
  • Warn when trying to read a folder which is not a spatialdata folder. Currently, we just return an empty spatialdata object, which is not helpful.
  • Improve performance on local spatialdata folders which already provide consolidated metadata.

Comment thread R/read.R
Comment thread R/read.R
Comment thread R/read.R Outdated
.readLayer <- \(l) {
j <- list.dirs(file.path(x, l), recursive=FALSE, full.names=TRUE)
names(j) <- basename(j)
message(" reading ", l, "...")

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I put this for now because S3 reading is quite slow and I wanted to see what was happening but I don't usually love super verbose functions.

Comment thread R/read.R
Comment on lines +55 to +56
# Until we have a complete store interface (https://github.com/Huber-group-EMBL/Rarr/pull/176),
# only local objects can be fully validated.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be handled by rome eventually anyways.

@Bisaloo Bisaloo force-pushed the s3-read branch 2 times, most recently from 2df1075 to 4d35c5b Compare May 15, 2026 15:15
Comment thread R/read.R
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant