Feature idea: Secure file saving/hosting

[Hekkta]

I currently use a local version of the mega database whilst keeping smaller personal databases on onedrive for my laptop and desktop.

This is slightly awkward since if I make changes to the megadatabase I need to then transfer from one PC to the next the updated version.

Also, using onedrive isn't safe with scid's database files, it synchronises as you go and can corrupt the files.

I have 3 propositions (with the help of chatgpt) that escelate in difficulty but get closer to synchronised "web" hosting of databases which is a massive advantage of chessbase.

Easiest

1. Optional delayed-commit mode. This is an option that would stop scid from making any changes to databases until scid is closed or the databse is closed or something the user triggers themselves (I'm not sure which is best). This would mean that hosting on something like ondrive is much safer as there are no incremental changes being made which can immediately corrupt a database.

Mid-range
2) Clever lock file handling with mounted folders. Example would be:
Machine A opens DB -> adds a lock to the database in the form of MyBase.lock.
Machine B tries to open/change DB -> refuses to open it or opens it read-only.
Machine A closes the DB -> removes the lock

This change would allow a self/cloud hosted mounted folder from a server to act reliably with multiple machines using the files.

Difficult
3) Full sync with WebDAV + lock files
A built-in, explicit database sync feature that only runs when the DB is closed, using WebDAV and a simple lock file protocol.
The user would host the DBs on a server which scid would ineract with through WebDAV and sync safely bearing in mind the fragility of the file types.

I hope this makes sense, it may not be the best way to implement this but I thought I'd get the ball rolling and put the idea out there. It may be too niche which is fair enough.

[whelanh]

I have also had issues in the past with OneDrive being prone to corrupt Scid database files. Before I think more about the ideas you've proposed, I have a strong recommendation for you and that is to set up Syncthing on all you machines. Basically you can create one (or two in my case) folders on each machine (e.g., C:\chessFiles), move (or copy) all of your database files into that directory and then setup Syncthing to keep them synced).

With Syncthing you are self-hosting (it does not go to a third party and it's free) and I've found it's flawless at keeping Scid database files up to date with each other with no corruption issues. The Synthing console opens up in your web browser and is easy to use. There is an option when you set up the files on each machine called File Versioning (see picture) which also will keep one (in my case) backup of every file in your folder in a .stversions sub-folder, so if anything does ever go wrong, you can restore from there. I keep my ~1.5 Gb databases updated this way and I am confident it could handle larger files as well. The windows setup is at: https://syncthing.net/downloads/

[Hekkta]

This seems like a perfect solution to my problems. I look forward to giving it a go and seeing if I get any more corruptions.

I wonder if just the easiest or moderate steps above could still be useful, but I agree there may be no need.

[Hekkta]

Syncthing seems to be working well, I've had no corruptions yet. There may be some use for putting in some safe guards against file corruption but if it's a very rare occurence it's not a big deal in my eyes. I'll keep testing for now.

[whelanh]

I'm very glad Syncthing has been working for you. I was pleasantly surprised when I learned about it. I hadn't realized there was a "non-cloud" solution like that. Do let me know if you run into any corruption issues. I could think about other solutions as well if needed. I've added a few other minor improvements to scidCommunity recently (the Tree View "Best Games" button honoring whether the "all games" checkbox is on or off, etc.). I hope you are continuing to enjoy it and don't hesitate to offer suggestions.