-
Notifications
You must be signed in to change notification settings - Fork 12
Description
I encountered unexpected behavior with Less3 while using a supported Multipart Upload APIs
Expected behavior
When uploading a file with the latest AWSSDK.S3 for .NET(4.0.18.8) using TransferUtility and TransferUtilityUploadRequest(Multipart Upload APIs under the hood), Less3 should store only the original file bytes.
For multipart uploads, Less3 should correctly process the request payload and persist the final object without any transport-level chunk metadata.
Actual behavior
The upload completes successfully, but the object stored by Less3 contains chunked streaming markers in the actual object body, for example:
14000;chunk-signature=ab8f...
Because of that:
the file stored by Less3 is corrupted
downloading the same object returns the same markers
the downloaded content does not match the original uploaded file
Relevant AWS documentation:
https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html
https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html
https://docs.aws.amazon.com/sdkfornet/v3/apidocs/items/S3/TTransferUtility.html
Steps to reproduce
Use the latest AWSSDK.S3 for .NET.
Create an AmazonS3Client targeting a Less3 endpoint.
Upload a file using TransferUtility.UploadAsync(new TransferUtilityUploadRequest { ... }).
Let the SDK perform its normal upload flow.
Inspect the stored object in Less3 internal storage, or download it back through S3 API.
Result
The object body contains chunk framing data instead of only the original file bytes.
Minimal client context
Less3 - latest docker image using default compose config
.NET 10
AWSSDK.S3 4.0.18.8
Our code path uses:
Amazon.S3.Transfer.TransferUtility
TransferUtilityUploadRequest
UploadAsync(...)
200mb file
The code was verified against a different S3 provider and functioned as expected.
private static IAmazonS3 CreateClient(S3StorageOptions options)
{
var config = new AmazonS3Config
{
ServiceURL = BuildServiceUrl(options),
ForcePathStyle = true,
UseHttp = !options.UseSSL,
AuthenticationRegion = options.Region,
Timeout = options.RequestTimeout,
};
return new AmazonS3Client(new BasicAWSCredentials(options.AccessKey, options.SecretKey), config);
}
private async Task<string> UploadWithTransferUtilityAsync(string objectKey, StagedUploadPayload stagedPayload, CancellationToken ct)
{
using var transferUtility = new TransferUtility(_s3Client);
var request = new TransferUtilityUploadRequest
{
BucketName = _options.BucketName,
Key = objectKey,
FilePath = stagedPayload.FilePath,
};
string versionId = string.Empty;
request.UploadCompletedEvent += (_, args) =>
{
versionId = args.Response?.VersionId ?? string.Empty;
};
await transferUtility.UploadAsync(request, ct);
return versionId;
}