Download images is not working

[MathGaron]

There have been changes in download from camera feature (maybe im not up to date with development)
-Seems like when sftp connect to client and go fetch the files it complain about not finding it.
-In fact, capturing image is not downloading anything to the pi, is download feature updated with this?

[soravux]

You're right, i'm not done with ticket #9 , hence the download doesn't work. For now, all the images are on the SD card and we download them manually at the end of the day.

[MathGaron]

@soravux , I started working with rsync, and every py will have a service that will call rsync to the folder that the master gave, all we need is to set a way to make the pi able to log with the server.. do we use something like ssh-keygen, or simply let ssh with no password (wich is dangerous)?

[soravux]

@MathGaron I think you're right, using keys generated with ssh-keygen is the way to go.

Which way have you planned to do the sync? Did you want to perform the rsync on the client or the server?

[MathGaron]

For now, load is always called from gphoto, it checks how many pictures are on the cam, and if there is more than lets say 50, the controller will load them in the pi.. so from there maybe it would be better if the device would push them to the server? this way, the server wont pull the pictures from every device at the same time, and anyway the server dont really know when to pull them.

But if we do that, we automate the way the picture are transfered, its a good and a bad thing for the user maybe.. It depend of the way you guys wish to use it?

[MathGaron]

Ok I made some tests and I would like to have your point of views :

• the install script will generate a key to the server.. we need to enhance the script to ask the user about the server's user and its IP or hostname.

-The clients will sync with the server after every x pictures, and pictures will be erased from camera (it check for error before) and then from the device with rsync

-The server will subscribe its IP address and username in rosparam so every device have access to it (still not implemented)

• all these bring a lot of possible crashs.. I will need to check through each possible bugs :s and find a way to secure it and make it bullet proof, I hope it wont be too long..

[soravux]

I would advise that a good setup documentation would be useful in case the install script becomes obsolete.

Otherwise, (and aside from comment I said on #6), it's fine with me and they seems like awesome features!

What are the pieces that you suspect as crash-prone? From what I understand, there is the auto-load after a given number of images and there is the rsync section. Is there something else you're thinking of?

Aside from that, I would recommend to remove dead code (such as this section). Git will allow you to get your code back if you ever want to reference it. Just create a tag if you want a marker on when this particular code was stripped out (or added).

Just a little feedback on subprocess management: I just found out about Envoy which seems to simplifies a lot subprocess launch and stream management. That could help you in similar development if you ever need. (Using rsync seemed to have removed a lot of code also, which is nice! Is there still some code requiring paramiko? I would advice removing it as a dependency, if so.)

Sorry for all the enhancement propositions, I hope you don't feel like it's negative feedback! Thanks for all your great work!

[MathGaron]

Thanks for the advice, I planned to remove the old download code after making sure everything is right, but in fact its true that git keep track of it anyway!

I will give a look to Envoy thank you

Paramiko wont be used anymore, I will remove it from dependency etc

The things that need to be checked upon are the setup problems : example if there is no key generated and ros is launched, it wont be able to connect etc. I need to make the setup and the launch bulletproof witch is hard because we use env variables etc.

So the device need to be able to detect if it cant send data to server and advertise it if there is a problem.

I will try the installation process on a fresh pi to test everything.

[soravux]

I think we could decrease our fear of operational mishaps if we use suitably ROS' logging facilities. For instance, when there's an issue with a subprocess call or similar, we could be able to log it and show it to the user, that would allow us to understand, assess, debug and fix the problem way more easily. I'm also thinking about the rule of transparency and mainly the rule of repair from the Art of Unix Programming: if a program has to fail, it should fail as soon as possible and noisily.

I would also advice for the use of a large --timeout value to rsync (and all commands issued if possible). That way, if a problem occurs, it won't clog the system by launching a heavenload of instance of the program.

Don't forget to document your fresh installation! :)

[MathGaron]

Good news, this issue is soon terminated! All I need to do wigth now is make rsync call safe, i will put the timeouts etc, I checked Envoy, it seems interesting I will give you feedback about it!

So in the webgui, you can set after how many pictures you want to download the datas, you can set -1 for never download.. The only thing is if you want to download on demand, you neet to put 0 and take a picture @jflalonde