AWS sigv4 update

[PrajaktaPurohit]

https://aws.amazon.com/blogs/aws/amazon-s3-update-sigv2-deprecation-period-extended-modified/

Please see: #1911

The authentication protocol for aws request signing has changed to v4. To support this work needs to be in 2 parts:

1. Erchef should henceforth generate appropriate v4 signed URL's.
2. Bookshelf should accept these URL's.

Tasks:
1) Update Erlang R20 DONE
2) Update Chef Server aws request signing to v4:

• There is currently some work done to get mini_s3 to support sigv4 and sigv2. But that adds significant complexity to support both the protocols.
• It would be a better path forward to use one of the modern and more mature aws signing libraries instead of mini_s3. <- Check if this is an option.

Some things to consider while choosing the library:

• Does it have a compatible license?
• Does it support newer algorithms like sts assumerole and instance profiles?
• Are we able to mock/modify the time of the request? [CLARIFICATION NEEDED]

some eg libraries

• erlcloud: https://github.com/erlcloud/erlcloud
  LICENSE: compliant https://github.com/erlcloud/erlcloud/blob/master/COPYRIGHT
  sts assumerole and instance profiles: supported
  Erlang versions: supported erlang version only until erlang 21. [ADDENDUM: Appears to compile and run under Erlang 22 in limited testing]
  Documentation: NONE
  Notes: It appears that erlcloud does not do any signing; you must roll that yourself. Appears to do signing, but not presigning.
  Issues and PRs regarding presigning:
  Add erlcloud_s3:make_presigned_v4_url function erlcloud/erlcloud#562
  Added functions to create pre-signed URLs for S3 erlcloud/erlcloud#586
  Add erlcloud_s3:make_presigned_v4_url function erlcloud/erlcloud#560
  Use V4 signature for signing S3 urls erlcloud/erlcloud#342
• erliam: https://github.com/AdRoll/erliam
  LICENSE: https://github.com/AdRoll/erliam/blob/master/LICENSE (BSD)
  supports sts
  supported erlang version 20 and 21
• aws-erlang: https://github.com/aws-beam/aws-erlang
  LICENSE: Apache 2.0
  appears to support erlang 22 as well?

Signing for aws is done in very limited places in erchef:

• src/oc_erchef/apps/chef_objects/src/chef_s3.erl
• src/oc_erchef/apps/chef_objects/src/chef_s3_ops.erl
• src/oc_erchef/apps/chef_objects/test/chef_s3_tests.erl

3. The assumption here is that all newly-created buckets will support v4 signing and that amazon will support both v2 and v4 signing with older buckets <- needs to be verified.
   [This seems to indicate that v2 will continue to be supported with older buckets; however, there can never be a guarantee as to what Amazon will decide to do in the future: "Any new buckets created after June 24, 2020 will not support SigV2 signed requests, although existing buckets will continue to support SigV2 while we work with customers to move off this older request signing method." https://aws.amazon.com/blogs/aws/amazon-s3-update-sigv2-deprecation-period-extended-modified/]
4. Bookshelf needs to be made v4 compliant. As a next task we can think of migration from bookshelf to minio. Automate also currently supports bookshelf.

NOTE:

1. The testing work from the mini_s3 is captured at: Add logging to capture context from working code mini_s3#29
2. mini_s3 sig_v4 work is captured at Ma/v4 signing mini_s3#28
3. Update chef server to use the work in mini_s3 in chef-server: https://github.com/chef/chef-server/pull/1869/files
4. Backporting some tests: WIP DO NOT MERGE - mini_s3_add_tests #1870

Related issues:
chef/mini_s3#23 (comment)

[PrajaktaPurohit]

also resolves #1013

[PrajaktaPurohit]

Closing in favor of: https://chefio.atlassian.net/jira/software/c/projects/INFS/boards/369?modal=detail&selectedIssue=INFS-34