RoboFlow

a semi-automated TensorFlow image classification explorer
in a python command line app.

Purpose & Goal

RoboFlow was created to get a better sense for TensorFlow's image classifier by making it easier to gather 1000s of similar images by hashtag (such as "#robot" or "#robotart") to serve as re/training examples, and to enable easy testing of different TensorFlow hyperparameter settings for creating classifiers. specifically, tagged images are downloaded (your choice of Imgur API or from web.stagram.com) and then sorted into labeled sub-directories, according to a 'confidencemin' variable, which are periodically 'harvested' to retrain TensorFlow to create new classifiers.
Issues, Contributions, and Pull Requests welcomed!

downloads sources (imgur api or webstagram)

As of now, choose in the config file from two useful (and randomly non-responsive) sources:
Imgur API (much quicker, fewer tags, requires you get a free API key) or
webstagram (slower, more tags, no API key required)

'basetag' concept

Because you can use roboflow for many separate classifiers, you need to pick a term for the broad master classification or theme of your classifier (such as 'robots', or 'birds', or whatever) so that images, classifier models and more can be stored separately under that BASETAG directory. You will do this in either the guided or advanced usages (see below).

Bootstrapping

There is an initial bootstrap stage in which you must manually sort a minimum number of images to allow the first retraining to create the first classifier per basetag. This tool will help you download 1000s of images pretty easily. After that first manual sorting, subsequent cycles of downloading, classifying/auto-sorting, and harvesting sorted images into the training_photos/labeled_directories for another cycle of retraining is waaaaay more automated.

hint: you can skip this step with a 1.4 gb download of robot images (pre-sorted into 'built', 'drawn', 'not' dirs)
to place into training_photos/

https://drive.google.com/file/d/1zvTq5vKqME7sW9O8lEtgn-Wosoavc2gi/view?usp=sharing

Getting Started

These instructions will get you a copy of the roboflow project up and running on your local machine for development and testing purposes.

Prerequisites

software

• Python (2.7 - but i havent tried 3.x so that might work, too) (https://www.python.org/downloads/)
• TensorFlow - learn about it: https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/
  install shortcut: https://github.com/googlecodelabs/tensorflow-for-poets-2
  or also at https://www.tensorflow.org/install/
• Terminal/Command Line familiarity
• Optional: Imgur API Client-ID (https://apidocs.imgur.com/)
  ** added to your ENVIRONMENT variables as 'IMGURAPI_ID', see config file line 59
  ** http://osxdaily.com/2015/07/28/set-enviornment-variables-mac-os-x/
• Optional: Tensorboard (https://github.com/tensorflow/tensorboard)
• Optional: Twilio for txt msg notifications (https://www.twilio.com/sms)

permissions

• Internet Access
• Image downloading
• File and directory creation permissions
• Create ENVIRONMENT variables in ~./bash_profile (or equivalent)

Installing

You will need tensorflow, the image classification codelab, and then roboflow:
STEP 1 (to get tensorflow)

https://www.tensorflow.org/install/

STEP 2 (to get the image classification setup)

https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/

STEP 3 (to get the exploration/semi-automation tool)
Make sure to 'cd' into your TensorFlow directory at the same level as "tf_files" and "scripts" first.
Then clone the RoboFlow dirs/files with:

git clone https://github.com/mariochampion/roboflow

Your directories should look like this:

├── tensorflow
|  ├── roboflow
|  ├── scripts
|  ├── tf_files
|  |  ├── bottlenecks
|  |  ├── models
|  |  ├── testing_photos
|  |  |  ├── [autogenerated basetag dirs...]
|  |  ├── training_photos
|  |  |  ├── [autogenerated basetag dirs...]
|  |  ├── training_summaries
|  |  |  ├── [autogenerated basetag dirs...]

Et voila, you are ready to explore!

Guided usage:

python roboflow.py

As a shortcut, add an alias in your .bash_profile to launch it via alias 'roboflow':

alias roboflow="cd path/to/dir/roboflow/;python roboflow.py"

Advanced usage:

python roboflow.py [basetag] [imagequantity] [searchtag] [optional flowsteps]

for example:

python roboflow.py robots 200 robotart download

Help:(or read the help docs online)

python roboflow.py --help

and dont forget to explore the config file.

Built With

• Python 2.7 - The programming language
• TensorFlow - An open-source software library for Machine Intelligence

Contributing

I am very open to issues and pull requests. Looking for a place to start helping?
https://github.com/mariochampion/roboflow/issues

Authors

• Mario Champion - Initial work - mariochampion

Contributions, Issues, and Pull requests Welcome!

See also the list of contributors who participated in this project.

License

This project is licensed under the Apache License 2.0 - see the LICENSE.md file for details

Acknowledgments

• Google's TensorFlow-for-poets2
• Imgur API (altho originally from Webstagram )
• StackExchange
• Twilio sms/python tools