Writing CI/CD Tests

[kapkic]

Each change in the system can cause issues, we need to ensure proper tests are done for all aspects of CB.
CI/CD and tests are specifically required for the library. This should run separately for three OS'es.

[kapkic]

9/5 Update: Create a test user, add their login credentials into every test machine.
Also, find a way to resolve the email-pass credentials without leaking any sensitive information if we were to run on github's own runners.

[kapkic]

For tests that include publishing or populating the actual dataset or system, we should push them into a test server/db. We don't want to populate the run system with testing data.

This means two things:
1- Creating a local run server inside emitlab, where all 3 os'es should do the runs.
1.1- This has an issue, we don't want to publish this configuration into public, as it is not usable.

2- Creating a test functionality or run, which is essentially a script that will change the code/configuration so that the calls are routed to the test server.

Tagging @AbhinavGor for the test server, and @prat-man for detail configurations.

[kapkic]

@ertugrulcobann Until we resolve this issue, still write tests for publish, but don't focus on them as they may change.

[kapkic]

New info:

More tests for future:

• Creating a dataset, model and metric using the library
• Publish

[AbhinavGor]

If we are extending the scope of this issue, my suggestion is to close this one and re-open a new one referencing this original issue and the updated scope