[CI] Add new "experimental" CI runs with additional build flags

[jbonnell-amd]

Motivation

We want to add CI coverage for new features to ensure that they're working as expected.

Technical Details

• Updated .github/workflows/ci-controller.yml to include new ci-<OS>-experimental steps that add additional build options to their respective base ci-<OS>.yml files
  • These new steps are triggered the same as the previous ones, however, on workflow_dispatch calls it requires an additional input enable_ci_experimental to be enabled as well
• Updated ci-<OS>.yml workflow files to accept the new nightly-experimental S3 bucket directory option in workflow calls
• Additionally, the Install OpenSSL step in .github/workflows/ci-redhat.yml was updated to fix an issue where it could not find the openssl package when enabled

Test Plan

• Manually trigger runs on all OS combinations with the experimental flags and ensure builds and tests are passing

Test Result

• Some test failures currently but these are expected with the current state of these features
  • Can leave this PR open until they are resolved and fixed,

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

[tomk-amd]

I think the experimental builds are launched on every PR -- I'm not sure we want this while they are still unstable? example the profiler tests that currently fail...

[jbonnell-amd]

I think the experimental builds are launched on every PR -- I'm not sure we want this while they are still unstable? example the profiler tests that currently fail...

Oh good point, I've updated the logic on the if statements to only allow the experimental runs to be ran on scheduled (nightly) runs as well as any manual dispatches where we specify the input flags correctly. Thanks!