Proposal: Add IaC (Terraform) based automated multi-region benchmark feature

[sunwoo383838]

Hello :)

During our research, we needed to run multi-region benchmarks and have integrated an Infrastructure as Code (IaC) based feature into this HotStuff framework to automate the process.

We believe this functionality could be a valuable addition to the project. If you are interested, we would be happy to clean up our code and prepare it for a contribution.

Here is a brief overview of the implementation:

It adds an inframanager built on Terraform.

This manager reads a CUE configuration file to define the infrastructure and benchmark parameters.

It currently automates the deployment and execution of benchmarks across multiple AWS regions.

We are also willing to extend this feature. If desired, we can abstract the inframanager to create a common interface. This would allow for runtime allocation and support for other major cloud providers like GCP, Azure, and OCI.
Please let us know if this is a feature you would be interested in merging.

[meling]

Yes, this sounds very interesting. Clearly, we would need to review it to make sure it fits with our existing custom deployment facilities in internal/orchestration based on the iago library. Alternatively, if it can replace the iago library, and still be able to deploy to our own bare-metal cluster, that would also help to reduce our maintenance burden. I should mention that one of iago's primary features is the ability to produce a single log based on the logged events at each replica. This is quite useful for debugging. Learning about Terraform has been on my agenda for a while, but I haven't gotten around to it yet. So I may also need some pointers to documentation for the purpose of reviewing such a contribution.

[sunwoo383838]

Thanks for the positive and detailed reply.
To clarify, I have already implemented the IaC-related inframanager to be compatible with this project, and I'm currently using it to run automated multi-region benchmarks.
My implementation does not replace iago; it's designed to integrate with it. The inframanager's role is to automate the cloud provisioning part, and then hand off the execution to iago.
Here is the specific workflow:

1. The inframanager (using Terraform) provisions the instances and adds their new public and private IPs into the experiment config.
2. It also generates a temporary SSH config file and passes this file to iago.

Additionally, I slightly modified the run code in internal/orchestration to correctly utilize these public and private IPs.
This setup is working well on my local machine.
The main task remaining is just to refactor the inframanager to be CSP-agnostic, as it's currently dependent on AWS.
I will send a draft PR as soon as this cleanup is complete.