Improve onboarding documentation

[buckyroberts]

Usually, I make detailed step-by-step installation instructions where you do not need to think or have variations/ambiguity of how to setup (please, see an example below). It would be nice to have such instructions in our project (probably in a dedicated INSTALL.rst/md not to overcrowd the README).

README.rst

Local development environment
=============================

This section describes how to setup development environment for Linux Mint 18.3.

Initial setup
+++++++++++++

Once initial setup is done only corresponding `Update`_ section should be performed
to get the latest version for development.

#. Install PyCharm to use as IDE
#. Install prerequisites::

    apt update
    apt install git

#. [if you have not configured it globally] Configure git::

    git config user.name 'Firstname Lastname'
    git config user.email 'youremail@youremail_domain.com'

#. Install Docker according to https://docs.docker.com/install/linux/docker-ce/ubuntu/
   (known working: Docker version 19.03.2, build 6a30dfc)

#. Add your user to docker group::

    sudo usermod -aG docker $USER
    exit

#. Install Docker Compose according to https://docs.docker.com/compose/install/
   (known working: docker-compose version 1.24.1, build 4667896b)

#. Install prerequisites (
   as prescribed at https://github.com/pyenv/pyenv/wiki/Common-build-problems )::

    apt update
    apt install make build-essential libssl-dev zlib1g-dev libbz2-dev \
                libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev \
                libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev python-openssl

#. Install and configure `pyenv` according to https://github.com/pyenv/pyenv#basic-github-checkout
#. Install Python 3.8.5::

    pyenv install 3.8.5

#. Install `pip`::

    # Ensure you are in `nda-server` repo root directory
    pip install pip==20.3.1

#. Install Poetry (known working version: 1.1.4)::

    pip install 'poetry>=1.1.4'

#. Configure Poetry to use $HOME/.virtualenvs::

    poetry config virtualenvs.path $HOME/.virtualenvs

#. Create virtualenv with Poetry::

    poetry shell

#. Install pip::

    pip install pip==20.3.1

#. Run dependencies with Docker Compose (in a separate terminal)::

    make dev-dependencies-up

#. Create local configuration file::

    mkdir local
    cat << EOF > ./local/settings.py
    DEBUG = True

    DATABASES['default']['NAME'] = 'postgres'
    DATABASES['default']['USER'] = 'postgres'
    DATABASES['default']['PASSWORD'] = 'postgres'
    DATABASES['default']['HOST'] = '127.0.0.1'

    CELERY_BROKER_URL = 'amqp://localhost:5672'

    SECRET_KEY = 'rydt2lf(cooe_w)b6tzw)7_8q+3g5o*^5(h7dac9@^k))zd2!j'

    MIDDLEWARE += ('nda_server.core.middleware.LoggingMiddleware',)
    LOGGING['root']['level'] = 'DEBUG'
    LOGGING['loggers']['nda_server']['level'] = 'DEBUG'
    LOGGING['handlers']['console']['level'] = 'DEBUG'
    EOF

    cp ./local/settings.py ./local/settings.unittests.py

#. Continue to `Update`_ section

#. Create superuser::

    make dev-create-superuser

Update
++++++

#. Activate shell::

    poetry shell

#. Update dependencies::

    poetry install

#. Run migrations::

    make dev-migrate

#. Install pre-commit::

    pre-commit install

Run
+++

#. Run dependencies with Docker Compose (in a dedicated terminal)::

    make dev-dependencies-up

#. Run Django server (in a dedicated terminal)::

    make dev-run-server

#. Run Celery (in a dedicated terminal)::

    make dev-run-celery

#. Run Celery Beat (in a dedicated terminal)::

    make dev-run-celery-beat

Development hints
+++++++++++++++++

#. Run unittests::

    make test

#. Lint the code base::

    make lint

Development, staging or production environment on Heroku
========================================================

The instruction is given for a development environment. For other environments replace
"nda-server-dev" with "nda-server-staging" or "nda-server-prod".

Initial setup
+++++++++++++

#. Create pipeline named "nda-server"
#. Create Heroku application::

    heroku apps:create --team=nda --addons=heroku-postgresql:hobby-basic,cloudamqp,papertrail,bucketeer,sentry --region=us --ssh-git nda-server-dev
    heroku pipelines:add --app nda-server-dev nda-server

#. Add Poetry buildpack::

    heroku buildpacks:clear --app nda-server-dev
    heroku buildpacks:add --app nda-server-dev https://github.com/moneymeets/python-poetry-buildpack.git
    heroku buildpacks:add --app nda-server-dev heroku/python

#. Add environment variables::

    heroku config:set --app nda-server-dev POETRY_VERSION=1.1.4
    heroku config:set --app nda-server-dev NDA_SERVER_ON_HEROKU=true
    heroku config:set --app nda-server-dev NDA_SERVER_SECRET_KEY=$(dd if=/dev/urandom bs=500 count=1 2> /dev/null | tr -dc 'a-zA-Z0-9' | head -c50)
    heroku config:set --app nda-server-dev NDA_SERVER_SENDGRID_API_KEY=<replace with SendGrid API key>
    heroku config:set --app nda-server-dev NDA_SERVER_OPEN_FDA_API_KEY=<replace with openFDA API key>
    heroku config:set --app nda-server-dev NDA_SERVER_SENDGRID_PASSWORD_RESET_EMAIL_TEMPLATE_ID=<replace with template id>
    heroku config:set --app nda-server-dev NDA_SERVER_SENDGRID_WELCOME_EMAIL_TEMPLATE_ID=<replace with template id>

#. Configure CORS::

    # For development environment
    heroku config:set --app nda-server-dev NDA_SERVER_CORS_ALLOW_ALL_ORIGINS=true

    # For production
    heroku config:set --app nda-server-prod NDA_SERVER_CORS_ALLOWED_ORIGINS='["https://app.nda.com", "https://nda-web-prod.herokuapp.com"]'

#. Add `HEROKU_SLUG_COMMIT` environment variable ::

    heroku labs:enable runtime-dyno-metadata --app nda-server-dev

#. (for development/staging environment: nda-server-dev) Navigate to "Deploy" tab:

    #. Open "nda" team: https://dashboard.heroku.com/teams/nda/apps
    #. Click "nda-server-dev" to open https://dashboard.heroku.com/apps/nda-server-dev
    #. Open "Deploy" tab

#. (for development/staging environment: nda-server-dev) Connect github repository for deployment:

    #. Choose "GitHub" "Deployment method" ( https://dashboard.heroku.com/apps/nda-server-dev/deploy/github )
    #. Connect https://github.com/nda/nda-server repository

#. (for development/staging environment: nda-server-dev) Enable automatic deploys:

    #. Set "Automatic deploys" -> "Choose a branch to deploy" = "master"
    #. Click "Automatic deploys" -> "Enable Automatic Deploys"

#. (for development/staging environment: nda-server-dev) Deploy "master" branch manually for the first time:

    #. "Manual deploy" -> choose "master" branch
    #. Click "Deploy Branch"

#. (for production: nda-server-prod) Promote nda-server-dev to nda-server-prod

#. Create superuser::

    heroku run --app nda-server-dev python -m nda_server.manage createsuperuser

#. Configure custom domain ( https://devcenter.heroku.com/articles/custom-domains ) and
   enable Automated Certificate Management
   ( https://devcenter.heroku.com/articles/automated-certificate-management )

#. Configure Django Sites framework to point to frontend domain name.

Update
++++++

#. (for production: nda-server-prod) Promote nda-server-dev to nda-server-prod

Makefile

.PHONY: dev-run-server
dev-run-server:
	poetry run python -m nda_server.manage runserver 127.0.0.1:8001

.PHONY: dev-run-celery
dev-run-celery:
	poetry run celery -A nda_server.project worker --loglevel=debug

.PHONY: dev-run-celery-beat
dev-run-celery-beat:
# TODO(dmu) MEDIUM: Should we have `--pidfile=`?
# TODO(dmu) MEDIUM: Do we need Celery beat at all?
	poetry run celery -A nda_server.project beat --loglevel=debug --pidfile=

.PHONY: dev-migrate
dev-migrate:
	poetry run python -m nda_server.manage migrate

.PHONY: dev-dbshell
dev-dbshell:
	poetry run python -m nda_server.manage dbshell

.PHONY: dev-create-superuser
dev-create-superuser:
	poetry run python -m nda_server.manage createsuperuser

.PHONY: dev-dependencies-up
dev-dependencies-up:
	docker-compose up

.PHONY: test
test:
	poetry run pytest -v

.PHONY: lint
lint:
	poetry run pre-commit run --all-files

[DucPhamTV]

This would be great. Following README instructions isn't enough to make it work. After installing dependencies, such as gcc, upgrade pip-tool, pip, eventually I can run the unit tests.
My machine isn't a fresh ubuntu-20.04 instance , probably new installed machine could need more dependencies.