Skip to content

[Worker] rewrite worker.sh in python for better maintainability #20

@mm4rks

Description

@mm4rks

Proposal

Rewrite the worker.sh script into python:

  • git handler + tests
  • docker handler + tests
  • systemd service change (invoke python worker.py with options, run as correct user that has access to python modules)
  • add user to docker group
  • install.sh for installing requirements.txt, registering service etc.
  • add worker queue (simple: read sample hashes from file) over samples in data/samples
  • filtering samples by type (pe, elf, macos, etc)
  • move and rename samples (sha256sum in data/samples)
  • add "incoming" folder that holds files for the above
  • add progress logging and stateful restarts (save samples to process, last processed, last run type, etc.)
  • save/load state in json file
  • refactor filter.py and use it as submodule in worker.py
  • add integration test, that tests the whole thing on some samples.

Approach

work on branch: python-worker

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions