Skip to content

Data processing

Jump to bottom
Alessandro edited this page Mar 21, 2018 · 6 revisions
  1. Move headcam videos to Sherlock
  2. Extract frames from videos
  3. Rotate extracted frames
  4. Run MTCNN on rotated extracted frames
  5. Run OpenPose on original videos (pass in rotate=180 option)
  6. Convert OpenPose JSON output to CSV format
  7. Push to Git

Move data from source to Sherlock

To move data from the langcog server to Sherlock, use the following rsync command on the langcog server:

rsync -avP [source dir] [SUNet ID]@login.sherlock.stanford.edu:[destination dir] &

You can also run scp on Sherlock and pull the files from langcog server. Rsync is preferred for large amounts of data however, as it is able to deal with closed connections better (it only transfers the deltas between the two filesystems).

Another option is Globus, which may be used if the data is stored on your local machine.

All data should be transferred to the $SCRATCH or $PI_SCRATCH filesystems, as they have a capacity of 20TB and 30TB, respectively. However, as these filesystems are not backed up / purged every 6 months, they should eventually be moved to $PI_HOME if possible. More info on the available filesystems on Sherlock here.

Extract frames from videos

Run Face Detectors

The next step is to run the MTCNN and OpenPose detectors. MTCNN operates on images, not videos, so we need to extract frames first. OpenPose did not work for some reason on directories of images, but instead took in the raw videos, with a flag to rotate the extracted frames (which it did on its own). TODO: Try OpenPose again on extracted frames, as this will make the data processing consistent with MTCNN.

MTCNN

TODO: which script to run from where, and where the output is populated, any postprocessing involved, and where resulting data is stored in github for subsequent analysis.

OpenPose

Preprint available here

Clone this wiki locally