I am using TigrisFS to perform backups of Tigris S3 buckets using BorgBackup (onto Tigris S3 buckets as well but that's not important here).
The last couple of days, I was hunting down the reason for the backup runs to be unexpectedly slow even if nothing changed in the source bucket. A source data set of about 7700 files amounting to 2.56 GB took over 8 minutes to check for changed modification time and size even though file listing was basically instantaneous.
I found that BorgBackup spent most of its time in sequentially retrieving xattr attributes for the files in the bucket and calling borg create with --noxattrs brought down the backup duration from 8 minutes to 16 seconds (!).
That's fine for me as I don't use extended attributes on the files in my buckets but I wanted to raise the question if an optimization like flagging all files without any extended attributes as such in the initial listing such that the extra calls can be avoided would make sense for you.
I am using TigrisFS to perform backups of Tigris S3 buckets using BorgBackup (onto Tigris S3 buckets as well but that's not important here).
The last couple of days, I was hunting down the reason for the backup runs to be unexpectedly slow even if nothing changed in the source bucket. A source data set of about 7700 files amounting to 2.56 GB took over 8 minutes to check for changed modification time and size even though file listing was basically instantaneous.
I found that BorgBackup spent most of its time in sequentially retrieving xattr attributes for the files in the bucket and calling
borg createwith--noxattrsbrought down the backup duration from 8 minutes to 16 seconds (!).That's fine for me as I don't use extended attributes on the files in my buckets but I wanted to raise the question if an optimization like flagging all files without any extended attributes as such in the initial listing such that the extra calls can be avoided would make sense for you.