Skip to content

Merge new model and config structures, CRUD functions#3

Open
neal-logan wants to merge 5 commits into
mainfrom
dev
Open

Merge new model and config structures, CRUD functions#3
neal-logan wants to merge 5 commits into
mainfrom
dev

Conversation

@neal-logan

Copy link
Copy Markdown
Collaborator
  • Project Structure: Extracted hard-coded information about gestures to .json files in configs folder: gesture-index mappings, gesture-action mappings, and defaults for each. Moved paths for models, data, and configs to environmental variables for coordinate; see .env.example. Moved raw hand landmark data to .csv format for maximum flexibility in data folder.
  • Model Features: Added left/right handedness to feature set. When added to 21 landmarks * 3 dimensions, this brings us to a nice, round 64 dimension. Moved to the new models folder. Expanded training logic to use a balanced sample (by right/left hand and gesture), extend training by a few epochs per prediction class.
  • Model Classes: Added an "other_gesture" class and corresponding data intended to classify hands that aren't making any particular gesture, intended to map to no action. This non-gesture includes a larger dataset, and the model should always be trained with a larger sample of this, perhaps as large as all other samples combined (it's currently about 1/3 of the data sampled).
  • Landmark Normalization: Unified normalization of hand landmarks using mean-centered normalization, dimension by dimension, with scaling based on maximum deviation from the mean in that dimension.
  • CRUD: Added methods for CRUD (create/read/update/delete) functionality for gestures, contexts, and gesture-action mappings in crud.py in the src folder. Minimally tested but they all seem to work.
  • Dev/Testing Notebook: Added a notebook for development/testing. Currently includes sample code for running data capture for new gestures as well as for training a new model after gestures are created, deleted, or altered. Intended for use in development/testing of CRUD functions as well as supporting functions so the UI can be developed separately.
  • Streamlit UI for configs: Broke it; it's going to need to get expanded to handle the new range of CRUD functions.
  • Archived python files, configs, data, and models not currently in use

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants