diff --git a/README.md b/README.md index bed897cf..81204b37 100644 --- a/README.md +++ b/README.md @@ -34,7 +34,7 @@ This will install `trakka` to `~/.local/bin`. You can pass a custom directory li The CLI requires Python to run. If you would like to use conda to install Python, install the CLI, and save the necessary environment variables, you can first install either Miniforge (https://github.com/conda-forge/miniforge) or Miniconda (https://docs.conda.io/en/latest/miniconda.html). We recommend Miniforge for most users. -Note that as a part of installing the CLI, you will need to set the environment's `AT_URI` variable. +Note that as a part of installing the CLI, you will need to set the environment's `TRAKKA_URI` variable. ### Install into a conda environment (optional but recommended) @@ -44,9 +44,9 @@ variables set and the `at-login` alias, run: conda create -n trakka python=3.12 conda activate trakka python -m pip install trakka -conda env config vars set AT_URI=[VALUE] +conda env config vars set TRAKKA_URI=[VALUE] mkdir -p ${CONDA_PREFIX}/etc/conda/activate.d -echo "alias at-login=\"export AT_TOKEN=\\\$(trakka auth user)\"" > ${CONDA_PREFIX}/etc/conda/activate.d/trakka-alias.sh +echo "alias at-login=\"export TRAKKA_TOKEN=\\\$(trakka auth user)\"" > ${CONDA_PREFIX}/etc/conda/activate.d/trakka-alias.sh ``` Note that the last two lines are valid only for Linux/Mac and will not work on Windows. These lines create an alias `at-login` in the conda environment, which will log you in to the CLI. @@ -67,28 +67,28 @@ To install without conda, simply install with python -m pip install trakka ``` -You will need to set the environment variable `AT_URI`. +You will need to set the environment variable `TRAKKA_URI`. You can do this by running: > #### Mac / Linux >``` ->export AT_URI=[VALUE] +>export TRAKKA_URI=[VALUE] >``` >You may wish to add this to your `.bashrc` or `.zshrc` file. >#### Windows: Powershell >``` ->$Env:AT_URI = [VALUE] +>$Env:TRAKKA_URI = [VALUE] >``` -To use the CLI, you must log in by setting the `AT_TOKEN` environment variable using the +To use the CLI, you must log in by setting the `TRAKKA_TOKEN` environment variable using the `trakka auth user` command (see User Authentication, below). > #### Mac / Linux >You may wish to configure >a login command for convenience: >``` ->alias at-login="export AT_TOKEN=\$(trakka auth user)" +>alias at-login="export TRAKKA_TOKEN=\$(trakka auth user)" >``` >You may wish to add this to your `.bashrc` or `.zshrc` file. @@ -96,7 +96,7 @@ To use the CLI, you must log in by setting the `AT_TOKEN` environment variable u >You may wish to configure >a login command for convenience: >``` ->Function at-login { $Env:AT_TOKEN = trakka auth user } +>Function at-login { $Env:TRAKKA_TOKEN = trakka auth user } >``` >You may wish to add this to your `config.ps1` file. @@ -131,26 +131,26 @@ to the Trakka web interface, and will authenticate you via your institution's id >at-login >``` > ->Otherwise, you will need to set the `AT_TOKEN` environment variable. In a Mac or Linux environment you can run: +>Otherwise, you will need to set the `TRAKKA_TOKEN` environment variable. In a Mac or Linux environment you can run: >``` ->export AT_TOKEN=$(trakka auth user) +>export TRAKKA_TOKEN=$(trakka auth user) >``` >#### Windows: Powershell > >``` ->$Env:AT_TOKEN = trakka auth user +>$Env:TRAKKA_TOKEN = trakka auth user >``` >#### Windows: Cmd > ->Set the `AT_TOKEN` environment variable by first running +>Set the `TRAKKA_TOKEN` environment variable by first running >``` >trakka auth user >``` >to obtain a token string, and then running >``` ->set AT_TOKEN= +>set TRAKKA_TOKEN= >``` >:w > to set the environment variable. @@ -161,31 +161,31 @@ This authentication mode is intended for long-term automated processes. Most use To authenticate a process, you'll need to set the following environment variables: ```bash -AT_AUTH_PROCESS_ID -AT_AUTH_PROCESS_SECRET +TRAKKA_AUTH_PROCESS_ID +TRAKKA_AUTH_PROCESS_SECRET ``` -Values for `AT_AUTH_PROCESS_ID` and `AT_AUTH_PROCESS_SECRET` will be provided to you by the Trakka team. Note that the secret value is sensitive. +Values for `TRAKKA_AUTH_PROCESS_ID` and `TRAKKA_AUTH_PROCESS_SECRET` will be provided to you by the Trakka team. Note that the secret value is sensitive. Once these variables are set, run the following to authorise: >#### Mac/Linux >``` ->export AT_TOKEN=$(trakka auth process) +>export TRAKKA_TOKEN=$(trakka auth process) >``` >#### Windows: Powershell >``` ->$Env:AT_TOKEN = trakka auth process +>$Env:TRAKKA_TOKEN = trakka auth process >``` >#### Windows: Cmd ->Set the `AT_TOKEN` environment variable by first running +>Set the `TRAKKA_TOKEN` environment variable by first running >``` >trakka auth process >``` >to obtain a token string, and then running >``` ->set AT_TOKEN= +>set TRAKKA_TOKEN= >``` >to set the environment variable. @@ -214,17 +214,17 @@ to see the usage of the `metadata add` command to upload metadata files. | Name | Description | |-------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------| -| `AT_TOKEN` | Trakka auth token | -| `AT_URI` | URI for API endpoint | -| `AT_LOG_LEVEL` | Level of logging | -| `AT_LOG` | Set to `file` to redirecting logging to a temp file | -| `AT_CMD_SET` | Set to `trakka-admin` to display admin commands (these will not actually run successfully unless you have an appropriate role on the server) | -| `AT_TIMEZONE` | Set to change the default timezone used for datetime display and parsing. Default if unset is to use your local timezone. | -| `AT_SKIP_CERT_VERIFY` | Skips verification of the cert used by the Trakka backend | -| `AT_SKIP_VERSION_CHECK` | Skips checking of new CLI version | -| `AT_USE_HTTP2` | Uses HTTP2 (experimental) | - -All commands require `AT_URI` and `AT_TOKEN` to be set, except for `auth` commands. +| `TRAKKA_TOKEN` | Trakka auth token | +| `TRAKKA_URI` | URI for API endpoint | +| `TRAKKA_LOG_LEVEL` | Level of logging | +| `TRAKKA_LOG` | Set to `file` to redirecting logging to a temp file | +| `TRAKKA_CMD_SET` | Set to `trakka-admin` to display admin commands (these will not actually run successfully unless you have an appropriate role on the server) | +| `TRAKKA_TIMEZONE` | Set to change the default timezone used for datetime display and parsing. Default if unset is to use your local timezone. | +| `TRAKKA_SKIP_CERT_VERIFY` | Skips verification of the cert used by the Trakka backend | +| `TRAKKA_SKIP_VERSION_CHECK` | Skips checking of new CLI version | +| `TRAKKA_USE_HTTP2` | Uses HTTP2 (experimental) | + +All commands require `TRAKKA_URI` and `TRAKKA_TOKEN` to be set, except for `auth` commands. ## Project Structure diff --git a/test/end_to_end_tests/ete_cmd_bricks.py b/test/end_to_end_tests/ete_cmd_bricks.py index 3891ec06..75e5eea2 100644 --- a/test/end_to_end_tests/ete_cmd_bricks.py +++ b/test/end_to_end_tests/ete_cmd_bricks.py @@ -303,7 +303,7 @@ def _create_field_if_not_exists(cli: TrakkaTestCli, field_name): assert result.exit_code == 0, f'Failed to list fields as part of test setup: {result.output}' # parse json array to find field_name matching "columnName" in the json - fields = json.loads(result.output) + fields = json.loads(result.stdout) if field_name.casefold() not in [field['columnName'].casefold() for field in fields]: result = cli.invoke([ 'field', @@ -339,4 +339,4 @@ def _list_seq_by_group(cli: TrakkaTestCli, group: str): ]) assert result.exit_code == 0, f'Failed to list sequences by group {group} as part of test setup: {result.output}' - return json.loads(result.output) + return json.loads(result.stdout) diff --git a/test/end_to_end_tests/test_proforma_commands.py b/test/end_to_end_tests/test_proforma_commands.py index b67e5213..028f3af6 100644 --- a/test/end_to_end_tests/test_proforma_commands.py +++ b/test/end_to_end_tests/test_proforma_commands.py @@ -20,7 +20,7 @@ def _get_proforma_fields(self, proforma_name: str): '--format', 'json' ]) - return json.loads(result.output) + return json.loads(result.stdout) def test_add_version_no_op_when_spec_is_identical(self): # Arrange diff --git a/test/end_to_end_tests/test_role_remove_command.py b/test/end_to_end_tests/test_role_remove_command.py index 8a406aba..7c24fcb7 100644 --- a/test/end_to_end_tests/test_role_remove_command.py +++ b/test/end_to_end_tests/test_role_remove_command.py @@ -60,7 +60,7 @@ def _role_exists(cli: TrakkaTestCli, role_name: str) -> bool: # Parse JSON output to check if role exists import json try: - roles_data = json.loads(result.output) + roles_data = json.loads(result.stdout) if isinstance(roles_data, list): return any(role.get('name') == role_name for role in roles_data) elif isinstance(roles_data, dict) and 'data' in roles_data: diff --git a/test/end_to_end_tests/test_seq_list_command.py b/test/end_to_end_tests/test_seq_list_command.py index 4571f268..657e123c 100644 --- a/test/end_to_end_tests/test_seq_list_command.py +++ b/test/end_to_end_tests/test_seq_list_command.py @@ -89,7 +89,7 @@ def test_list__given_type_is_not_specified__expect_all_types_are_listed(self): # Assert assert result.exit_code == 0, f'The seq get command should succeed: {result.output}' - json_result = json.loads(result.output) + json_result = json.loads(result.stdout) # Count elements by type fastq_ill_pe_count = sum(1 for item in json_result if item.get("type") == "fastq-ill-pe") @@ -138,7 +138,7 @@ def test_list__given_type_is_specified__expect_only_sequence_of_that_type_listed # Assert assert result.exit_code == 0, f'The seq get command should succeed: {result.output}' - json_result = json.loads(result.output) + json_result = json.loads(result.stdout) # Count elements by type fastq_ill_pe_count = sum(1 for item in json_result if item.get("type") == "fastq-ill-pe") @@ -188,7 +188,7 @@ def test_list__given_group_name_is_specified__expect_only_sequences_in_that_grou # Assert assert result.exit_code == 0, f'The seq get command should succeed: {result.output}' - json_result = json.loads(result.output) + json_result = json.loads(result.stdout) # Count elements by type fastq_ill_pe_count = sum(1 for item in json_result if item.get("type") == "fastq-ill-pe") @@ -237,7 +237,7 @@ def test_list__given_seq_id_is_specified__expect_only_sequence_for_the_sample_wi # Assert assert result.exit_code == 0, f'The seq get command should succeed: {result.output}' - json_result = json.loads(result.output) + json_result = json.loads(result.stdout) # Count elements by type fastq_ill_pe_count = sum(1 for item in json_result if item.get("type") == "fastq-ill-pe") diff --git a/test/utils/trakka_test_cli.py b/test/utils/trakka_test_cli.py index 498ab9aa..1b6e764a 100644 --- a/test/utils/trakka_test_cli.py +++ b/test/utils/trakka_test_cli.py @@ -29,7 +29,7 @@ def _validate(self) -> None: :raises CliTestException: there is a problem with the provided config """ - at_uri = os.getenv(TrakkaCxt.get_env_var_name(CxtKey.URI), '') + at_uri = TrakkaCxt.get_env_var_value(CxtKey.URI, '') if PRODLIKE_DOMAIN in at_uri: raise CliTestException(f"Unable to run tests against {at_uri}") diff --git a/trakka/components/auth/opts.py b/trakka/components/auth/opts.py index 7738f5d7..d9c79529 100644 --- a/trakka/components/auth/opts.py +++ b/trakka/components/auth/opts.py @@ -1,5 +1,6 @@ import click from trakka.components.auth.enums import Auth +from trakka.utils.context import CxtKey, TrakkaCxt def opt_tenant_id(func): @@ -7,7 +8,7 @@ def opt_tenant_id(func): '--tenant-id', show_envvar=True, required=True, - envvar='AT_AUTH_TENANT_ID', + envvar=TrakkaCxt.get_env_var_names(CxtKey.AUTH_TENANT_ID), help='Tenant ID', default=Auth.TENANT_ID.value, )(func) @@ -18,7 +19,7 @@ def opt_client_id(func): '--client-id', show_envvar=True, required=True, - envvar='AT_AUTH_CLIENT_ID', + envvar=TrakkaCxt.get_env_var_names(CxtKey.AUTH_CLIENT_ID), help='Client ID', default=Auth.CLIENT_ID.value, )(func) @@ -29,7 +30,7 @@ def opt_backend_app_uri(func): '--app-uri', show_envvar=True, required=True, - envvar='AT_AUTH_APP_URI', + envvar=TrakkaCxt.get_env_var_names(CxtKey.AUTH_APP_URI), help='API URI', default=Auth.APP_SCOPE.value, )(func) @@ -41,7 +42,7 @@ def opt_process_auth_id(func): 'process_id', show_envvar=True, required=True, - envvar='AT_AUTH_PROCESS_ID', + envvar=TrakkaCxt.get_env_var_names(CxtKey.AUTH_PROCESS_ID), help="Process account ID" )(func) @@ -51,6 +52,6 @@ def opt_process_auth_secret(func): '--secret', show_envvar=True, required=True, - envvar='AT_AUTH_PROCESS_SECRET', + envvar=TrakkaCxt.get_env_var_names(CxtKey.AUTH_PROCESS_SECRET), help='Process account secret' )(func) diff --git a/trakka/main.py b/trakka/main.py index 6ce5e33e..7a800d4d 100644 --- a/trakka/main.py +++ b/trakka/main.py @@ -1,5 +1,4 @@ # pylint: disable=expression-not-assigned -import os import sys import uuid @@ -38,7 +37,7 @@ from trakka.utils.logger import setup_logger from trakka.utils.logger import LOG_LEVEL_INFO from trakka.utils.logger import LOG_LEVELS -from trakka.utils.cmd_filter import show_admin_cmds +from trakka.utils.cmd_filter import USER, show_admin_cmds from trakka.utils.version import check_version @@ -55,19 +54,19 @@ @click.option( TrakkaCxt.get_option_name(CxtKey.URI), show_envvar=True, - envvar=TrakkaCxt.get_env_var_name(CxtKey.URI), + envvar=TrakkaCxt.get_env_var_names(CxtKey.URI), required=True ) @click.option( TrakkaCxt.get_option_name(CxtKey.TOKEN), show_envvar=True, - envvar=TrakkaCxt.get_env_var_name(CxtKey.TOKEN), + envvar=TrakkaCxt.get_env_var_names(CxtKey.TOKEN), required=True ) @click.option( TrakkaCxt.get_option_name(CxtKey.LOG_LEVEL), show_envvar=True, - envvar=TrakkaCxt.get_env_var_name(CxtKey.LOG_LEVEL), + envvar=TrakkaCxt.get_env_var_names(CxtKey.LOG_LEVEL), default=LOG_LEVEL_INFO, type=click.Choice(LOG_LEVELS), show_default=True @@ -76,7 +75,7 @@ TrakkaCxt.get_option_name(CxtKey.TIMEZONE), '-tz', show_envvar=True, - envvar=TrakkaCxt.get_env_var_name(CxtKey.TIMEZONE), + envvar=TrakkaCxt.get_env_var_names(CxtKey.TIMEZONE), default=LOCAL_TIMEZONE, show_default=True, help='Timezone to use for any datetime output or parsing. ' @@ -86,7 +85,7 @@ @click.option( TrakkaCxt.get_option_name(CxtKey.SKIP_CERT_VERIFY), show_envvar=True, - envvar=TrakkaCxt.get_env_var_name(CxtKey.SKIP_CERT_VERIFY), + envvar=TrakkaCxt.get_env_var_names(CxtKey.SKIP_CERT_VERIFY), required=True, default=False, show_default=True, @@ -96,7 +95,7 @@ @click.option( TrakkaCxt.get_option_name(CxtKey.USE_HTTP2), show_envvar=True, - envvar=TrakkaCxt.get_env_var_name(CxtKey.USE_HTTP2), + envvar=TrakkaCxt.get_env_var_names(CxtKey.USE_HTTP2), required=True, default=False, show_default=True, @@ -106,13 +105,23 @@ @click.option( TrakkaCxt.get_option_name(CxtKey.SKIP_VERSION_CHECK), show_envvar=True, - envvar=TrakkaCxt.get_env_var_name(CxtKey.SKIP_VERSION_CHECK), + envvar=TrakkaCxt.get_env_var_names(CxtKey.SKIP_VERSION_CHECK), required=True, default=False, show_default=True, type=bool, help="Skip check for new CLI version" ) +@click.option( + TrakkaCxt.get_option_name(CxtKey.CMD_SET), + show_envvar=True, + envvar=TrakkaCxt.get_env_var_names(CxtKey.CMD_SET), + required=True, + default=USER, + show_default=True, + type=str, + help="Hide/show admin commands" +) @click.option( '--log', 'log_var', @@ -131,6 +140,7 @@ def cli( use_http2: bool, skip_version_check: bool, log_var: str, + cmd_set: str, ): ctx.context = { CxtKey.URI.value: uri, @@ -141,10 +151,12 @@ def cli( CxtKey.LOG_LEVEL.value: log_level, CxtKey.SESSION_ID.value: str(uuid.uuid4()), CxtKey.TIMEZONE.value: timezone, + CxtKey.CMD_SET.value: cmd_set, } setup_logger(log_level, log_var) if not skip_version_check: check_version(VERSION) + TrakkaCxt.check_deprecated_env_vars() def get_cli(): @@ -178,7 +190,8 @@ def main(): sys.exit(1) except Exception as ex: # pylint: disable=broad-except # Cannot use TrakkaCxt.value here because there is no click context - if is_debug(os.getenv(TrakkaCxt.get_env_var_name(CxtKey.LOG_LEVEL), '')): + + if is_debug(TrakkaCxt.get_env_var_value(CxtKey.LOG_LEVEL, '')): logger.exception(ex) else: logger.error(ex) diff --git a/trakka/utils/cmd_filter.py b/trakka/utils/cmd_filter.py index e862fcc8..0706dbcb 100644 --- a/trakka/utils/cmd_filter.py +++ b/trakka/utils/cmd_filter.py @@ -1,13 +1,37 @@ -import os +from loguru import logger +from trakka.utils.context import CxtKey, TrakkaCxt AUSTRAKKA_ADMIN = 'austrakka-admin' TRAKKA_ADMIN = 'admin' +USER = 'user' +_DEPRECATION_WARNING_PRINTED = False def show_admin_cmds(): - cmd_set = os.getenv('AT_CMD_SET') + cmd_set = TrakkaCxt.get_env_var_value(CxtKey.CMD_SET, '') + deprecation_warning(cmd_set) return cmd_set and cmd_set.lower() in [AUSTRAKKA_ADMIN, TRAKKA_ADMIN] def hide_admin_cmds(): return not show_admin_cmds() + + +def deprecation_warning(value: str): + # This global flag is required as this function will be called + # for every instance of a admin command + # pylint: disable=global-statement + global _DEPRECATION_WARNING_PRINTED + if _DEPRECATION_WARNING_PRINTED: + return + if value == AUSTRAKKA_ADMIN: + env_var_names = ", ".join(TrakkaCxt.get_env_var_names(CxtKey.CMD_SET)) + # This unfortunately occurs before we've set up our logger + # as it's part of the click initialisation, which + # means this line might look different; it's temporary, + # and as only internal users are using this flag we can + # remove it sooner. + logger.warning("Value " + AUSTRAKKA_ADMIN + " for env vars " + + env_var_names + " is deprecated and will be replaced with " + + TRAKKA_ADMIN + " in a future release.") + _DEPRECATION_WARNING_PRINTED = True diff --git a/trakka/utils/context.py b/trakka/utils/context.py index c0f8d606..eae1bc93 100644 --- a/trakka/utils/context.py +++ b/trakka/utils/context.py @@ -1,11 +1,15 @@ from enum import Enum +import os +from typing import List from click import get_current_context +from loguru import logger from trakka.utils.exceptions import TrakkaCliException -CLI_PREFIX = 'AT' +AT_CLI_PREFIX = 'AT' +TRAKKA_CLI_PREFIX = 'TRAKKA' class CxtKey(Enum): @@ -20,6 +24,12 @@ class CxtKey(Enum): LOG_LEVEL = 'log_level' SESSION_ID = 'session_id' TIMEZONE = 'timezone' + CMD_SET = 'cmd_set' + AUTH_PROCESS_SECRET = 'auth_process_secret' + AUTH_PROCESS_ID = 'auth_process_id' + AUTH_APP_URI = 'auth_app_uri' + AUTH_CLIENT_ID = 'auth_client_id' + AUTH_TENANT_ID = 'auth_tenant_id' class TrakkaCxt: @@ -52,9 +62,53 @@ def get_option_name(ctx_key: CxtKey): return f"--{ctx_key.value.replace('_', '-')}" @staticmethod - def get_env_var_name(ctx_key: CxtKey): + def _get_at_env_var_name(ctx_key: CxtKey): """ :param ctx_key: context key :return: The env var name for the context key """ - return f"{CLI_PREFIX}_{ctx_key.value.upper()}" + return f"{AT_CLI_PREFIX}_{ctx_key.value.upper()}" + + @staticmethod + def _get_trakka_env_var_name(ctx_key: CxtKey): + """ + :param ctx_key: context key + :return: The env var name for the context key + """ + return f"{TRAKKA_CLI_PREFIX}_{ctx_key.value.upper()}" + + @staticmethod + def get_env_var_names(ctx_key: CxtKey) -> List[str]: + """ + :param ctx_key: context key + :return: The env var name for the context key + """ + return [ + TrakkaCxt._get_trakka_env_var_name(ctx_key), + TrakkaCxt._get_at_env_var_name(ctx_key), + ] + + @staticmethod + def get_env_var_value(ctx_key: CxtKey, default) -> str: + """ + Use in situations where we don't have access to the click context. + :param ctx_key: context key + :return: The env var value + """ + trakka_name = TrakkaCxt._get_trakka_env_var_name(ctx_key) + at_name = TrakkaCxt._get_at_env_var_name(ctx_key) + if trakka_name in os.environ: + return os.getenv(trakka_name, default) + if at_name in os.environ: + return os.getenv(at_name, default) + return default + + @staticmethod + def check_deprecated_env_vars(): + for k in CxtKey: + trakka_name = TrakkaCxt._get_trakka_env_var_name(k) + at_name = TrakkaCxt._get_at_env_var_name(k) + if at_name in os.environ and trakka_name not in os.environ: + logger.warning("Environment variable " + at_name + + " is deprecated and will be replaced with " + + trakka_name + " in a future release.") diff --git a/trakka/utils/misc.py b/trakka/utils/misc.py index d0959d95..465ff973 100644 --- a/trakka/utils/misc.py +++ b/trakka/utils/misc.py @@ -10,11 +10,11 @@ HELP_OPTS = ['-h', '--help'] -MISSING_TOKEN_HELP = '''Error: Environment variable AT_TOKEN is not set. +MISSING_TOKEN_HELP = '''Error: Environment variable TRAKKA_TOKEN is not set. This value can be obtained by running the following command: trakka auth user''' -MISSING_URI_HELP = '''Error: Environment variable AT_URI is not set. +MISSING_URI_HELP = '''Error: Environment variable TRAKKA_URI is not set. Please contact a Trakka admin if you do not have this value.'''