Monitoring of network shares

[abrain]

Hi,

for several months now I use your fs-file-lister node on a Raspberry Pi to check a network share for new PDF files. It gets triggered by an injection node in interval mode. The found files get moved from the mountpoint to the local file system.

Today I experienced the problem, that a file has been moved, but was 0 bytes in size. So I guess it has been moved while it still was being written.

The workaround for now is to skip events for files that are 0 bytes in size and to increase the time between two file listings. The safest way would probably be to cache events for several seconds to see if the file size changes, but I didn't have time to implement that yet.

Do you think this could be a feature of the existing or maybe even a new node specialized for network shares?

[TotallyInformation]

Ah, you seem to have found an edge case! In truth, the file lister isn't really designed with robust file moving in mind. At the least I should update the documentation with this limitation.

I think that you are always going to get issues like this on slow file systems. Have you tried using the watch node? It doesn't always work on network drives I think, but if it does, it would be a far more robust solution.

[abrain]

Yes, I tried the watch node, but it didn't work with the network share. That's why I chose the not-so-realtime approach to check every few seconds.

As far as I could see, the size remains 0 until the file is written completely. So my workaround seems to be ok for now.

I just wanted to let you know, in case you need some ideas to work on ;)

[TotallyInformation]

Thanks, at least I can update the documentation.