From d4e411758354ac14d4556acefd20acfe70f9c4be Mon Sep 17 00:00:00 2001 From: TheMeinerLP Date: Mon, 13 Jul 2026 22:22:45 +0200 Subject: [PATCH 1/2] feat: add Cloudflare R2 support via account-id and optional list caching MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Ports the useful parts of Tresillo2017's R2 fork (compare/main...Tresillo2017:BlueMapS3Storage:main) cleanly: that fork's diff turned out to contain no actual Java changes despite its CHANGELOG_R2.md describing new R2Storage/R2Configuration classes, just three overlapping markdown files and an accidentally committed node_modules/. R2 is S3-compatible, so it already works through the existing themeinerlp:s3 storage type. Rather than duplicating that as a parallel R2-specific implementation, this adds: - account-id: optional Cloudflare account ID. When set (and endpoint-url is left empty), the endpoint is derived automatically as https://.r2.cloudflarestorage.com and path-style access is forced on, since R2 only supports path-style. - list-cache-ttl-seconds: optional TTL cache for mapIds() (map enumeration), since directory listings are a billed "Class A" operation on R2. Off by default (0); only affects map enumeration at init/reload, not tile reads/writes. (Fixes a pre-existing resource leak in mapIds() along the way — the underlying DirectoryStream was never closed.) Consolidates the fork's R2_GUIDE.md/R2_QUICK_START.md/CHANGELOG_R2.md into a single docs/cloudflare-r2.md, and documents both new options plus the existing checksum-validation option (added in #73 but never added to the README) in the main config table. --- README.md | 22 +++++ docs/cloudflare-r2.md | 84 +++++++++++++++++++ .../bluemap/s3/storage/S3Configuration.java | 4 + .../s3/storage/S3FileSystemFactory.java | 12 ++- .../bluemap/s3/storage/S3Storage.java | 39 +++++++-- .../s3/storage/S3StorageConfiguration.java | 28 +++++++ 6 files changed, 180 insertions(+), 9 deletions(-) create mode 100644 docs/cloudflare-r2.md diff --git a/README.md b/README.md index f54949f8..c967e09b 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,7 @@ A storage addon for [BlueMap](https://github.com/BlueMap-Minecraft/BlueMap) that BlueMapS3Storage is an addon for BlueMap that provides the ability to store map data in S3-compatible storage services, such as: - Amazon S3 +- Cloudflare R2 (see [docs/cloudflare-r2.md](docs/cloudflare-r2.md) for R2-specific setup) - MinIO - DigitalOcean Spaces - Backblaze B2 @@ -107,6 +108,9 @@ force-path-style: true | `endpoint-url` | Optional: The endpoint URL for S3-compatible services (leave empty for AWS S3) | `http://localhost:9000` | | `root-path` | Optional: The root path in the S3 bucket where BlueMap data will be stored | `.` | | `force-path-style` | Optional: Force path style access for S3 (needed for MinIO) | `false` | +| `checksum-validation` | Controls when the AWS SDK attaches/validates request and response checksums (`when_required` or `when_supported`). Since SDK 2.30 the default is `when_supported`, which many S3-compatible stores (Ceph RGW included) reject with a bare 400 Bad Request. Set to `when_supported` only if you're on real AWS S3 or a provider you've confirmed handles it. | `when_required` | +| `account-id` | Optional: Cloudflare account ID. If set and `endpoint-url` is left empty, the endpoint is derived automatically as `https://.r2.cloudflarestorage.com` with path-style access enabled. See [docs/cloudflare-r2.md](docs/cloudflare-r2.md). | (empty) | +| `list-cache-ttl-seconds` | Optional: How long to cache the list of available maps (`mapIds()`). Directory listings are a billed operation on some providers (e.g. R2); this list rarely changes at runtime. `0` disables caching. | `0` | ## Usage Examples @@ -148,6 +152,24 @@ root-path: "." force-path-style: true ``` +### Cloudflare R2 + +```hocon +## ## +## BlueMap ## +## Storage-Config ## +## ## + +storage-type: "themeinerlp:s3" +compression: gzip +bucket-name: "bluemap-storage" +account-id: "your-cloudflare-account-id" +access-key-id: "your-r2-access-key" +secret-access-key: "your-r2-secret-key" +``` + +See [docs/cloudflare-r2.md](docs/cloudflare-r2.md) for full setup steps, the listing-cache option, and troubleshooting. + ### DigitalOcean Spaces ```hocon diff --git a/docs/cloudflare-r2.md b/docs/cloudflare-r2.md new file mode 100644 index 00000000..a18eadaa --- /dev/null +++ b/docs/cloudflare-r2.md @@ -0,0 +1,84 @@ +# Cloudflare R2 + +R2 is S3-compatible, so it works through the same `themeinerlp:s3` storage type as any other +S3-compatible provider. There's no separate `themeinerlp:r2` storage type; two small +R2-specific conveniences are built into the regular S3 config instead. + +## Setup + +1. Create a bucket in the Cloudflare dashboard under **R2**. +2. Under **Manage R2 API Tokens**, create a token with **Object Read & Write** permissions and + copy the Access Key ID / Secret Access Key immediately, you won't see them again. +3. Copy your **Account ID** from the R2 overview page. +4. Create a storage config file: + + | Platform | Path | + |---|---| + | Spigot/Paper | `./plugins/BlueMap/storages/r2.conf` | + | Sponge/Forge/Fabric | `./config/bluemap/storages/r2.conf` | + | CLI | `./config/storages/r2.conf` | + + ```hocon + storage-type: "themeinerlp:s3" + compression: gzip + bucket-name: "your-bucket-name" + account-id: "your-account-id" + access-key-id: "your-r2-access-key" + secret-access-key: "your-r2-secret-key" + ``` + + Setting `account-id` derives the endpoint (`https://.r2.cloudflarestorage.com`) + and enables path-style access automatically, since R2 only supports path-style. Leave + `endpoint-url` and `force-path-style` unset when using `account-id` this way. + + If you'd rather set the endpoint yourself (e.g. for a jurisdictional/EU-restricted bucket), + skip `account-id` and use `endpoint-url`/`force-path-style` directly instead, same as any + other S3-compatible provider: + + ```hocon + storage-type: "themeinerlp:s3" + compression: gzip + bucket-name: "your-bucket-name" + access-key-id: "your-r2-access-key" + secret-access-key: "your-r2-secret-key" + region: "auto" + endpoint-url: "https://your-account-id.eu.r2.cloudflarestorage.com" + force-path-style: true + ``` + +5. Reference the storage in your main BlueMap config: + + ```hocon + storages: { + r2: "storages/r2.conf" + } + + maps: [ + { id: "world", storage: "r2" } + ] + ``` + +6. Restart or reload BlueMap and confirm objects show up in the bucket. + +## Cost note: directory listing cache + +Directory listings (used for `mapIds()`, i.e. once per storage init/reload to enumerate +configured maps, not per-tile) are a billed "Class A" operation on R2. Since that list rarely +changes at runtime, you can cache it for a while: + +```hocon +list-cache-ttl-seconds: 300 +``` + +`0` (default) disables caching. This only affects map enumeration, not tile reads/writes. + +## Troubleshooting + +**`SignatureDoesNotMatch` / 403.** Almost always a copy-paste issue: regenerate the R2 API +token, make sure the secret key isn't wrapped in extra quotes or has trailing whitespace, and +double check the account ID is the exact 32-character hex string from the R2 overview page +(not the bucket name or a zone ID). + +**`PutObject => 400` / tile saves failing silently.** Not R2-specific, see the +[`checksum-validation`](../README.md#configuration-options) option. R2 (like most non-AWS +S3-compatible stores) needs `checksum-validation: when_required` (the default). diff --git a/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3Configuration.java b/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3Configuration.java index de333803..efb7afed 100644 --- a/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3Configuration.java +++ b/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3Configuration.java @@ -35,4 +35,8 @@ public sealed interface S3Configuration permits S3StorageConfiguration { String getChecksumValidation(); + String getAccountId(); + + int getListCacheTtlSeconds(); + } diff --git a/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3FileSystemFactory.java b/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3FileSystemFactory.java index ce6112ab..d940a48d 100644 --- a/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3FileSystemFactory.java +++ b/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3FileSystemFactory.java @@ -39,7 +39,11 @@ public static S3Fs build(S3Configuration cfg) { throw new IllegalArgumentException("bucketName is required"); } - final boolean thirdParty = cfg.getEndpointUrl() != null && !cfg.getEndpointUrl().isBlank(); + final boolean noExplicitEndpoint = cfg.getEndpointUrl() == null || cfg.getEndpointUrl().isBlank(); + final boolean usingAccountId = noExplicitEndpoint && cfg.getAccountId() != null && !cfg.getAccountId().isBlank(); + final String effectiveEndpoint = + usingAccountId ? "https://" + cfg.getAccountId() + ".r2.cloudflarestorage.com" : cfg.getEndpointUrl(); + final boolean thirdParty = effectiveEndpoint != null && !effectiveEndpoint.isBlank(); try { final URI uri; System.setProperty(AWS_REGION_KEY, cfg.getRegion() != null ? cfg.getRegion() : DEFAULT_AWS_REGION); @@ -54,11 +58,13 @@ public static S3Fs build(S3Configuration cfg) { System.setProperty("aws.requestChecksumCalculation", cfg.getChecksumValidation()); System.setProperty("aws.responseChecksumValidation", cfg.getChecksumValidation()); if (thirdParty) { - var url = URI.create(cfg.getEndpointUrl()); + var url = URI.create(effectiveEndpoint); if (!url.toString().startsWith("https")) { System.setProperty("s3.spi.endpoint-protocol", "http"); } - if (cfg.forcePathStyle()) { + // R2 only supports path-style access, so force it when we derived the endpoint + // from account-id, regardless of what force-path-style is set to. + if (cfg.forcePathStyle() || usingAccountId) { System.setProperty("s3.spi.force-path-style", "true"); } PROVIDER = new S3XFileSystemProvider(); diff --git a/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3Storage.java b/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3Storage.java index 2cb84b94..e55df24e 100644 --- a/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3Storage.java +++ b/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3Storage.java @@ -17,6 +17,7 @@ */ package dev.themeinerlp.bluemap.s3.storage; +import com.github.benmanes.caffeine.cache.Cache; import com.github.benmanes.caffeine.cache.Caffeine; import com.github.benmanes.caffeine.cache.LoadingCache; import de.bluecolored.bluemap.core.storage.MapStorage; @@ -24,23 +25,37 @@ import de.bluecolored.bluemap.core.storage.compression.Compression; import de.bluecolored.bluemap.core.storage.file.FileMapStorage; import java.io.IOException; +import java.io.UncheckedIOException; import java.nio.file.FileSystem; import java.nio.file.Files; import java.nio.file.Path; +import java.time.Duration; +import java.util.List; import java.util.stream.Stream; public final class S3Storage implements Storage { + private static final String MAP_IDS_CACHE_KEY = "mapIds"; + private final S3Configuration configuration; private final Compression compression; private FileSystem s3FileSystem; private boolean closed = false; private final LoadingCache mapStorages; + // Directory listings are a billed "Class A" operation on R2 and similar providers, so + // mapIds() (which rarely changes at runtime) can optionally be cached for a while. Null + // when list-cache-ttl-seconds is 0, meaning caching stays off unless explicitly configured. + private final Cache> mapIdsCache; public S3Storage(S3Configuration configuration, Compression compression) { this.configuration = configuration; this.compression = compression; mapStorages = Caffeine.newBuilder().build(this::create); + int listCacheTtlSeconds = configuration.getListCacheTtlSeconds(); + mapIdsCache = + listCacheTtlSeconds > 0 + ? Caffeine.newBuilder().expireAfterWrite(Duration.ofSeconds(listCacheTtlSeconds)).build() + : null; } @Override @@ -83,12 +98,24 @@ public Stream mapIds() throws IOException { if (isClosed()) { throw new IOException("Storage is closed"); } - - // List all directories in the root path - return Files.list(getRooPath()) - .filter(Files::isDirectory) - .map(Path::getFileName) - .map(Path::toString); + + try { + List ids = + mapIdsCache != null + ? mapIdsCache.get(MAP_IDS_CACHE_KEY, key -> listMapIdsUncached()) + : listMapIdsUncached(); + return ids.stream(); + } catch (UncheckedIOException e) { + throw e.getCause(); + } + } + + private List listMapIdsUncached() { + try (Stream entries = Files.list(getRooPath())) { + return entries.filter(Files::isDirectory).map(Path::getFileName).map(Path::toString).toList(); + } catch (IOException e) { + throw new UncheckedIOException(e); + } } @Override diff --git a/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3StorageConfiguration.java b/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3StorageConfiguration.java index 843ffb4a..e9a59247 100644 --- a/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3StorageConfiguration.java +++ b/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3StorageConfiguration.java @@ -66,6 +66,24 @@ public final class S3StorageConfiguration extends StorageConfig implements S3Con """) private String checksumValidation = "when_required"; + @Comment(""" + Optional: Cloudflare account ID. If set and endpoint-url is left empty, the + endpoint is derived automatically as https://.r2.cloudflarestorage.com + and path-style access is enabled (required by R2). Leave empty for AWS S3 or any + other S3-compatible provider - use endpoint-url directly for those instead. + """) + @DebugDump(exclude = true) + private String accountId = ""; + + @Comment(""" + How long (in seconds) to cache the result of listing available maps (mapIds()). + 0 disables caching (default). Directory listings are a billed "Class A" operation + on R2 and similar providers, so a short cache here cuts down on repeated listing + calls; this list rarely changes at runtime (only when a map is added/removed), so + a stale cache for a few minutes is normally harmless. + """) + private int listCacheTtlSeconds = 0; + @Override public Storage createStorage() throws ConfigurationException { // Validate required configuration @@ -135,4 +153,14 @@ public boolean forcePathStyle() { public String getChecksumValidation() { return ChecksumValidationMode.parse(checksumValidation).propertyValue(); } + + @Override + public String getAccountId() { + return accountId; + } + + @Override + public int getListCacheTtlSeconds() { + return Math.max(0, listCacheTtlSeconds); + } } From df6d6412aff499783b5e61f462492eaa5d0b4e55 Mon Sep 17 00:00:00 2001 From: Phillipp Glanz <6745190+TheMeinerLP@users.noreply.github.com> Date: Mon, 13 Jul 2026 22:45:36 +0200 Subject: [PATCH 2/2] refactor: extract provider-specific config into ProviderProfile (Strategy) (#78) --- README.md | 1 + docs/cloudflare-r2.md | 4 ++ .../bluemap/s3/storage/GenericS3Profile.java | 43 +++++++++++ .../bluemap/s3/storage/ProviderProfile.java | 43 +++++++++++ .../bluemap/s3/storage/ProviderProfiles.java | 72 +++++++++++++++++++ .../bluemap/s3/storage/R2Profile.java | 49 +++++++++++++ .../bluemap/s3/storage/S3Configuration.java | 2 + .../s3/storage/S3FileSystemFactory.java | 16 ++--- .../s3/storage/S3StorageConfiguration.java | 13 ++++ 9 files changed, 233 insertions(+), 10 deletions(-) create mode 100644 src/main/java/dev/themeinerlp/bluemap/s3/storage/GenericS3Profile.java create mode 100644 src/main/java/dev/themeinerlp/bluemap/s3/storage/ProviderProfile.java create mode 100644 src/main/java/dev/themeinerlp/bluemap/s3/storage/ProviderProfiles.java create mode 100644 src/main/java/dev/themeinerlp/bluemap/s3/storage/R2Profile.java diff --git a/README.md b/README.md index c967e09b..be90aebf 100644 --- a/README.md +++ b/README.md @@ -109,6 +109,7 @@ force-path-style: true | `root-path` | Optional: The root path in the S3 bucket where BlueMap data will be stored | `.` | | `force-path-style` | Optional: Force path style access for S3 (needed for MinIO) | `false` | | `checksum-validation` | Controls when the AWS SDK attaches/validates request and response checksums (`when_required` or `when_supported`). Since SDK 2.30 the default is `when_supported`, which many S3-compatible stores (Ceph RGW included) reject with a bare 400 Bad Request. Set to `when_supported` only if you're on real AWS S3 or a provider you've confirmed handles it. | `when_required` | +| `provider` | Optional: explicitly picks the provider profile (`r2` or `generic`) instead of auto-detecting from which fields are set. Only needed if you want to be explicit; see [docs/cloudflare-r2.md](docs/cloudflare-r2.md). | (auto-detect) | | `account-id` | Optional: Cloudflare account ID. If set and `endpoint-url` is left empty, the endpoint is derived automatically as `https://.r2.cloudflarestorage.com` with path-style access enabled. See [docs/cloudflare-r2.md](docs/cloudflare-r2.md). | (empty) | | `list-cache-ttl-seconds` | Optional: How long to cache the list of available maps (`mapIds()`). Directory listings are a billed operation on some providers (e.g. R2); this list rarely changes at runtime. `0` disables caching. | `0` | diff --git a/docs/cloudflare-r2.md b/docs/cloudflare-r2.md index a18eadaa..9f724f73 100644 --- a/docs/cloudflare-r2.md +++ b/docs/cloudflare-r2.md @@ -31,6 +31,10 @@ R2-specific conveniences are built into the regular S3 config instead. and enables path-style access automatically, since R2 only supports path-style. Leave `endpoint-url` and `force-path-style` unset when using `account-id` this way. + This is auto-detected: setting `account-id` (with no `endpoint-url`) is enough on its own. + Add `provider: "r2"` explicitly only if you want to be unambiguous about it, e.g. in a + config generated by tooling rather than typed by hand. + If you'd rather set the endpoint yourself (e.g. for a jurisdictional/EU-restricted bucket), skip `account-id` and use `endpoint-url`/`force-path-style` directly instead, same as any other S3-compatible provider: diff --git a/src/main/java/dev/themeinerlp/bluemap/s3/storage/GenericS3Profile.java b/src/main/java/dev/themeinerlp/bluemap/s3/storage/GenericS3Profile.java new file mode 100644 index 00000000..ee690a2e --- /dev/null +++ b/src/main/java/dev/themeinerlp/bluemap/s3/storage/GenericS3Profile.java @@ -0,0 +1,43 @@ +/** + * A simple storage implementation for bluemap to save data into s3 storage solution. + * Copyright (C) 2025 TheMeinerLP and contributors + *

+ * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + *

+ * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + *

+ * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ +package dev.themeinerlp.bluemap.s3.storage; + +/** + * Fallback profile: passes the configured endpoint-url/force-path-style/region through + * unchanged. Always applies, and {@link ProviderProfiles} tries it last so any provider-specific + * profile gets a chance to take over first. + */ +final class GenericS3Profile implements ProviderProfile { + + static final String ID = "generic"; + + @Override + public String id() { + return ID; + } + + @Override + public boolean appliesTo(S3Configuration cfg) { + return true; + } + + @Override + public ResolvedEndpoint resolve(S3Configuration cfg) { + return new ResolvedEndpoint(cfg.getEndpointUrl(), cfg.forcePathStyle(), cfg.getRegion()); + } +} diff --git a/src/main/java/dev/themeinerlp/bluemap/s3/storage/ProviderProfile.java b/src/main/java/dev/themeinerlp/bluemap/s3/storage/ProviderProfile.java new file mode 100644 index 00000000..8ef5ae92 --- /dev/null +++ b/src/main/java/dev/themeinerlp/bluemap/s3/storage/ProviderProfile.java @@ -0,0 +1,43 @@ +/** + * A simple storage implementation for bluemap to save data into s3 storage solution. + * Copyright (C) 2025 TheMeinerLP and contributors + *

+ * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + *

+ * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + *

+ * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ +package dev.themeinerlp.bluemap.s3.storage; + +/** + * Resolves a provider's own shorthand config (e.g. Cloudflare R2's account-id) into the plain + * connection settings {@link S3FileSystemFactory} actually needs. Keeps provider-specific + * conventions out of the generic S3 engine: add a new profile here when a provider gets its + * own convenience config, register it in {@link ProviderProfiles}, and the factory itself + * never needs to change. + */ +interface ProviderProfile { + + /** Stable name users can set explicitly via the provider config option, e.g. "r2". */ + String id(); + + /** + * Whether this profile's own shorthand config is present, so it should be picked + * automatically when the user didn't set provider explicitly. Only consulted as a + * fallback - see {@link ProviderProfiles#resolve(S3Configuration)}. + */ + boolean appliesTo(S3Configuration cfg); + + /** Only called once this profile has been selected, either explicitly or via {@link #appliesTo}. */ + ResolvedEndpoint resolve(S3Configuration cfg); + + record ResolvedEndpoint(String endpointUrl, boolean forcePathStyle, String region) {} +} diff --git a/src/main/java/dev/themeinerlp/bluemap/s3/storage/ProviderProfiles.java b/src/main/java/dev/themeinerlp/bluemap/s3/storage/ProviderProfiles.java new file mode 100644 index 00000000..e9ccae45 --- /dev/null +++ b/src/main/java/dev/themeinerlp/bluemap/s3/storage/ProviderProfiles.java @@ -0,0 +1,72 @@ +/** + * A simple storage implementation for bluemap to save data into s3 storage solution. + * Copyright (C) 2025 TheMeinerLP and contributors + *

+ * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + *

+ * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + *

+ * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ +package dev.themeinerlp.bluemap.s3.storage; + +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; + +/** + * Selects a {@link ProviderProfile} and resolves it against the given config. + * + *

If {@code provider} is set explicitly, that profile is used directly, an unmistakable, + * explicit strategy choice, and an unknown name fails fast rather than silently falling back. + * If left unset, profiles are tried in order and the first one whose {@link + * ProviderProfile#appliesTo(S3Configuration)} matches wins, so existing zero-config setups + * (e.g. just setting account-id) keep working without needing to also learn about + * {@code provider}. {@link GenericS3Profile} always applies and is tried last. + */ +final class ProviderProfiles { + + private static final List PROFILES = List.of(new R2Profile(), new GenericS3Profile()); + + private static final Map BY_ID = new LinkedHashMap<>(); + + static { + for (ProviderProfile profile : PROFILES) { + BY_ID.put(profile.id(), profile); + } + } + + private ProviderProfiles() {} + + static ProviderProfile.ResolvedEndpoint resolve(S3Configuration cfg) { + return select(cfg).resolve(cfg); + } + + private static ProviderProfile select(S3Configuration cfg) { + String explicit = cfg.getProvider(); + if (explicit != null && !explicit.isBlank()) { + ProviderProfile profile = BY_ID.get(explicit.trim().toLowerCase()); + if (profile == null) { + throw new IllegalArgumentException( + "Unknown provider '" + + explicit + + "', expected one of: " + + String.join(", ", BY_ID.keySet())); + } + return profile; + } + + return PROFILES.stream() + .filter(profile -> profile.appliesTo(cfg)) + .findFirst() + .orElseThrow(() -> new IllegalStateException( + "No provider profile matched (GenericS3Profile should always match)")); + } +} diff --git a/src/main/java/dev/themeinerlp/bluemap/s3/storage/R2Profile.java b/src/main/java/dev/themeinerlp/bluemap/s3/storage/R2Profile.java new file mode 100644 index 00000000..962a44e5 --- /dev/null +++ b/src/main/java/dev/themeinerlp/bluemap/s3/storage/R2Profile.java @@ -0,0 +1,49 @@ +/** + * A simple storage implementation for bluemap to save data into s3 storage solution. + * Copyright (C) 2025 TheMeinerLP and contributors + *

+ * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + *

+ * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + *

+ * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ +package dev.themeinerlp.bluemap.s3.storage; + +/** + * Cloudflare R2: derives the endpoint from account-id so users don't have to look up/type the + * full https://<account-id>.r2.cloudflarestorage.com URL themselves. Applies only when + * account-id is set and no explicit endpoint-url was given, so setting endpoint-url always + * still wins (e.g. for R2's jurisdictional/EU-restricted endpoints). + * + *

R2 only supports path-style access, so this always forces it on regardless of + * force-path-style. + */ +final class R2Profile implements ProviderProfile { + + static final String ID = "r2"; + + @Override + public String id() { + return ID; + } + + @Override + public boolean appliesTo(S3Configuration cfg) { + boolean noExplicitEndpoint = cfg.getEndpointUrl() == null || cfg.getEndpointUrl().isBlank(); + return noExplicitEndpoint && cfg.getAccountId() != null && !cfg.getAccountId().isBlank(); + } + + @Override + public ResolvedEndpoint resolve(S3Configuration cfg) { + String endpoint = "https://" + cfg.getAccountId() + ".r2.cloudflarestorage.com"; + return new ResolvedEndpoint(endpoint, true, cfg.getRegion()); + } +} diff --git a/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3Configuration.java b/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3Configuration.java index efb7afed..1803c575 100644 --- a/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3Configuration.java +++ b/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3Configuration.java @@ -39,4 +39,6 @@ public sealed interface S3Configuration permits S3StorageConfiguration { int getListCacheTtlSeconds(); + String getProvider(); + } diff --git a/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3FileSystemFactory.java b/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3FileSystemFactory.java index d940a48d..9a4827cc 100644 --- a/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3FileSystemFactory.java +++ b/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3FileSystemFactory.java @@ -39,14 +39,12 @@ public static S3Fs build(S3Configuration cfg) { throw new IllegalArgumentException("bucketName is required"); } - final boolean noExplicitEndpoint = cfg.getEndpointUrl() == null || cfg.getEndpointUrl().isBlank(); - final boolean usingAccountId = noExplicitEndpoint && cfg.getAccountId() != null && !cfg.getAccountId().isBlank(); - final String effectiveEndpoint = - usingAccountId ? "https://" + cfg.getAccountId() + ".r2.cloudflarestorage.com" : cfg.getEndpointUrl(); - final boolean thirdParty = effectiveEndpoint != null && !effectiveEndpoint.isBlank(); + final ProviderProfile.ResolvedEndpoint resolved = ProviderProfiles.resolve(cfg); + final boolean thirdParty = resolved.endpointUrl() != null && !resolved.endpointUrl().isBlank(); try { final URI uri; - System.setProperty(AWS_REGION_KEY, cfg.getRegion() != null ? cfg.getRegion() : DEFAULT_AWS_REGION); + String region = resolved.region() != null && !resolved.region().isBlank() ? resolved.region() : DEFAULT_AWS_REGION; + System.setProperty(AWS_REGION_KEY, region); System.setProperty("aws.accessKeyId", cfg.getAccessKeyId()); System.setProperty("aws.secretAccessKey", cfg.getSecretAccessKey()); // AWS SDK for Java 2.30.0+ defaults to attaching a flexible checksum (e.g. a CRC32 @@ -58,13 +56,11 @@ public static S3Fs build(S3Configuration cfg) { System.setProperty("aws.requestChecksumCalculation", cfg.getChecksumValidation()); System.setProperty("aws.responseChecksumValidation", cfg.getChecksumValidation()); if (thirdParty) { - var url = URI.create(effectiveEndpoint); + var url = URI.create(resolved.endpointUrl()); if (!url.toString().startsWith("https")) { System.setProperty("s3.spi.endpoint-protocol", "http"); } - // R2 only supports path-style access, so force it when we derived the endpoint - // from account-id, regardless of what force-path-style is set to. - if (cfg.forcePathStyle() || usingAccountId) { + if (resolved.forcePathStyle()) { System.setProperty("s3.spi.force-path-style", "true"); } PROVIDER = new S3XFileSystemProvider(); diff --git a/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3StorageConfiguration.java b/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3StorageConfiguration.java index e9a59247..5f7b9b5e 100644 --- a/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3StorageConfiguration.java +++ b/src/main/java/dev/themeinerlp/bluemap/s3/storage/S3StorageConfiguration.java @@ -66,6 +66,14 @@ public final class S3StorageConfiguration extends StorageConfig implements S3Con """) private String checksumValidation = "when_required"; + @Comment(""" + Optional: explicitly picks which provider profile resolves this config, one of: + "r2", "generic". Leave empty to auto-detect: setting account-id below picks "r2" + automatically, otherwise "generic" (uses endpoint-url/force-path-style as-is). + Only needed if you want to be explicit rather than rely on auto-detection. + """) + private String provider = ""; + @Comment(""" Optional: Cloudflare account ID. If set and endpoint-url is left empty, the endpoint is derived automatically as https://.r2.cloudflarestorage.com @@ -163,4 +171,9 @@ public String getAccountId() { public int getListCacheTtlSeconds() { return Math.max(0, listCacheTtlSeconds); } + + @Override + public String getProvider() { + return provider; + } }