Is your feature request related to a problem? Please describe.
The S3 backend currently uses deprecated AWS SDK v2 APIs (manager.NewUploader and manager.NewDownloader) for file upload and download operations. These APIs are marked as deprecated in favor of the new feature/s3/transfermanager package. This generates staticcheck warnings (SA1019) across the codebase:
backend/s3/file.go:336 - manager.NewUploader is deprecated
backend/s3/file.go:340 - uploader.Upload is deprecated
backend/s3/file.go:641 - manager.NewDownloader is deprecated
backend/s3/file.go:809 - manager.NewUploader is deprecated
backend/s3/file.go:817 - uploader.Upload is deprecated
backend/s3/file.go:847-848 - manager.Downloader type is deprecated
backend/s3/file.go:853-854 - manager.Uploader type is deprecated
While these APIs still function correctly, they may be removed in future AWS SDK releases, and the new transfermanager provides improved performance and better resource management.
Describe the solution you'd like
Migrate the S3 backend to use the new github.com/aws/aws-sdk-go-v2/feature/s3/manager transfermanager APIs:
- Replace
manager.NewUploader with the new transfermanager upload APIs
- Replace
manager.NewDownloader with the new transfermanager download APIs
- Update the
withUploadPartitionSize and withDownloadPartitionSize helper functions to work with the new API
- Ensure partition size configuration options continue to work as expected
- Verify that all upload/download scenarios (temp file writes, streaming writes, copy operations) work correctly with the new APIs
The migration should maintain backward compatibility with existing Options and configuration while eliminating the deprecation warnings.
Describe alternatives you've considered
- Keep using deprecated APIs with nolint directives (current approach) - This works but kicks the can down the road and may break when AWS SDK eventually removes these APIs
- Wait for AWS SDK to force the migration - Not recommended as it could break production code unexpectedly
- Implement a compatibility layer - Would add unnecessary complexity when direct migration is feasible
Additional context
- AWS SDK discussion on the deprecation: S3 Transfer Manager v2 Now Generally Available aws/aws-sdk-go-v2#3306
- The current implementation uses partition size configuration for performance tuning, which must be preserved in the migration
- All upload/download operations in
backend/s3/file.go are affected:
writeFileToS3() - temp file uploads
copyWithinS3() - S3-to-S3 copies
copyToLocalTempReader() - downloads to temp files
getS3Writer() - streaming uploads via pipe
- The migration should be thoroughly tested with integration tests to ensure no regression in functionality or performance
- Consider this a good opportunity to review and optimize the S3 transfer logic while updating the API usage
Is your feature request related to a problem? Please describe.
The S3 backend currently uses deprecated AWS SDK v2 APIs (
manager.NewUploaderandmanager.NewDownloader) for file upload and download operations. These APIs are marked as deprecated in favor of the newfeature/s3/transfermanagerpackage. This generates staticcheck warnings (SA1019) across the codebase:backend/s3/file.go:336-manager.NewUploaderis deprecatedbackend/s3/file.go:340-uploader.Uploadis deprecatedbackend/s3/file.go:641-manager.NewDownloaderis deprecatedbackend/s3/file.go:809-manager.NewUploaderis deprecatedbackend/s3/file.go:817-uploader.Uploadis deprecatedbackend/s3/file.go:847-848-manager.Downloadertype is deprecatedbackend/s3/file.go:853-854-manager.Uploadertype is deprecatedWhile these APIs still function correctly, they may be removed in future AWS SDK releases, and the new transfermanager provides improved performance and better resource management.
Describe the solution you'd like
Migrate the S3 backend to use the new
github.com/aws/aws-sdk-go-v2/feature/s3/managertransfermanager APIs:manager.NewUploaderwith the new transfermanager upload APIsmanager.NewDownloaderwith the new transfermanager download APIswithUploadPartitionSizeandwithDownloadPartitionSizehelper functions to work with the new APIThe migration should maintain backward compatibility with existing Options and configuration while eliminating the deprecation warnings.
Describe alternatives you've considered
Additional context
backend/s3/file.goare affected:writeFileToS3()- temp file uploadscopyWithinS3()- S3-to-S3 copiescopyToLocalTempReader()- downloads to temp filesgetS3Writer()- streaming uploads via pipe