From bda53412303117697402f0971dcaa4347f8656cd Mon Sep 17 00:00:00 2001 From: ckunki Date: Wed, 18 Feb 2026 14:45:38 +0100 Subject: [PATCH 1/7] #705: Described how the versions of poetry and python are retrieved --- doc/changes/unreleased.md | 4 ++ .../github_workflows/configuration.rst | 32 --------- .../github_workflows/create_and_update.rst | 67 +++++++++++++++++++ .../github_project_configuration.rst | 32 +++++++++ .../features/github_workflows/index.rst | 20 ++---- .../github_workflows/template_variables.rst | 14 ++++ 6 files changed, 123 insertions(+), 46 deletions(-) delete mode 100644 doc/user_guide/features/github_workflows/configuration.rst create mode 100644 doc/user_guide/features/github_workflows/create_and_update.rst create mode 100644 doc/user_guide/features/github_workflows/github_project_configuration.rst create mode 100644 doc/user_guide/features/github_workflows/template_variables.rst diff --git a/doc/changes/unreleased.md b/doc/changes/unreleased.md index 77106267c..62bf42637 100644 --- a/doc/changes/unreleased.md +++ b/doc/changes/unreleased.md @@ -6,6 +6,10 @@ * #691: Started customization of PTB workflows by defining the YML schema +## Documentation + +* #705: Described how the versions of poetry and python are retrieved + ## Refactoring * #664: Removed deprecation warning for projects to switch over to BaseConfig diff --git a/doc/user_guide/features/github_workflows/configuration.rst b/doc/user_guide/features/github_workflows/configuration.rst deleted file mode 100644 index aa654fcaa..000000000 --- a/doc/user_guide/features/github_workflows/configuration.rst +++ /dev/null @@ -1,32 +0,0 @@ -.. _github_workflows_configuration: - -Configuration -============= - -1. Configure the GitHub project -+++++++++++++++++++++++++++++++ - -* Make sure your GitHub project has access to a deployment token for PyPi with the following name: **PYPI_TOKEN**. It should be available to the repository either as an Organization-, Repository-, or Environment-secret. - -* If your CI workflow involves slow or expensive steps you can guard these to be executed only after manual approval. The CI workflow will automatically create a GitHub environment named :code:`manual-approval`. You only need to add reviewers in (:code:`Settings/Environments/manual-approval`) and move the steps to be guarded into the related section in job :code:`slow-checks` in file :code:`.github/workflows/merge-gate.yml`. - -2. Add all workflows to your project -++++++++++++++++++++++++++++++++++++ - -.. code-block:: shell - - tbx workflow install all - -.. warning:: - - #. If you already have various workflows, you may want to run the :code:`update` command instead of the :code:`install` command. - - #. Some workflows depend on other workflows. Please ensure you have all the required workflows if you do not install all of them. - -3. Update Branch Protection -++++++++++++++++++++++++++++ - -The best and most maintainable way to have solid branch protection (:code:`Settings/Branches/main`) is to require the workflow :code:`CI / Allow Merge` to pass successfully. - -.. note:: - Setting the required status checks to pass before merging is only possible after running a CI build at **least once** on the affected branch. diff --git a/doc/user_guide/features/github_workflows/create_and_update.rst b/doc/user_guide/features/github_workflows/create_and_update.rst new file mode 100644 index 000000000..ac991edb2 --- /dev/null +++ b/doc/user_guide/features/github_workflows/create_and_update.rst @@ -0,0 +1,67 @@ +.. _update_github_workflows: + +Creating and Updating the GitHub Workflows in Your Project +========================================================== + +PTB can initially generate the GitHub workflows in your project and also +update existing workflows. + +The workflows are based on Jinja templates with variables populated by the +PTB. The PTB reads the values from various places within your project, see +:ref:`template_variables`. + +Please note that the PTB only updates the values in the GitHub workflows when +*updating* the workflows. So, after updating the :ref:`list of Python versions +` in file ``noxconfig.py``, you need to :ref:`re-generate +` the GitHub workflows. + +Poetry Version +-------------- + +PTB has a default value for the Poetry version but you can override it in file +``noxconfig.py``, e.g. + +.. code-block:: shell + + PROJECT_CONFIG = Config( + dependency_manager=DependencyManager(name="poetry", version="2.3.0"), + ) + +.. _python_versions: + +Versions of Python and Exasol Docker DB +--------------------------------------- + +Many workflows are using a Build-matrix to iterate over multiple versions of +Python and/or the Exasol Docker DB. This is to make sure your code is valid, +free of bugs and working correctly for each combination of these items. + +PTB has a default for these versions, but you can override it in file +``noxconfig.py``, e.g. + +.. code-block:: shell + + PROJECT_CONFIG = Config( + python_versions=("3.10", "3.12), + exasol_versions=("7.1.30", "8.29.13", "2025.1.8"), + ) + +Some workflows are expected to not depend on a specific python version and +will use only the first of the Python version: +``PROJECT_CONFIG.python_versions[0]``. + +.. _update_workflows: + +Add all Workflows to Your Project +--------------------------------- + +.. code-block:: shell + + tbx workflow install all + +.. warning:: + #. If you already have various workflows, you may want to run the + :code:`update` command instead of the :code:`install` command. + + #. Some workflows depend on other workflows. Please ensure you have all + the required workflows if you do not install all of them. diff --git a/doc/user_guide/features/github_workflows/github_project_configuration.rst b/doc/user_guide/features/github_workflows/github_project_configuration.rst new file mode 100644 index 000000000..c6360fda0 --- /dev/null +++ b/doc/user_guide/features/github_workflows/github_project_configuration.rst @@ -0,0 +1,32 @@ +.. _github_project_configuration: + +GitHub Project Configuration +============================ + +Branch Protection +----------------- + +The best and most maintainable way to have solid branch protection +(:code:`Settings/Branches/main`) is to require the workflow :code:`CI / Allow +Merge` to pass successfully. + +.. note:: + Setting the required status checks to pass before merging is only possible + after running a CI build at **least once** on the affected branch. + +Manual Approval +--------------- + +If your CI workflow involves slow or expensive steps you can guard these to be +executed only after manual approval. The CI workflow will automatically create +a GitHub environment named :code:`manual-approval`. You only need to add +reviewers in (:code:`Settings/Environments/manual-approval`) and move the +steps to be guarded into the related section in job :code:`slow-checks` in +file :code:`.github/workflows/merge-gate.yml`. + +PYPI Token +---------- + +Make sure your GitHub project has access to a deployment token for PyPi with +the following name: **PYPI_TOKEN**. It should be available to the repository +either as an Organization-, Repository-, or Environment-secret. diff --git a/doc/user_guide/features/github_workflows/index.rst b/doc/user_guide/features/github_workflows/index.rst index b0f05b0ad..13a8b3d7c 100644 --- a/doc/user_guide/features/github_workflows/index.rst +++ b/doc/user_guide/features/github_workflows/index.rst @@ -1,13 +1,15 @@ .. _GitHub Workflows: -Enabling GitHub Workflows +GitHub Workflow Templates ========================= .. toctree:: - :maxdepth: 2 + :maxdepth: 1 :hidden: - configuration + github_project_configuration + create_and_update + template_variables The PTB ships with configurable GitHub workflow templates covering the most common CI/CD setup variants for Python projects. The templates are defined in: @@ -25,16 +27,6 @@ workflows from the templates. In most cases, we recommend using _all_ workflows without change to ensure consistent interdependencies. For more details, see :ref:`ci_actions`. -Underlying the CLI, the PTB uses Jinja to dynamically generate project-specific -workflows. The rendering process is supported by the ``github_template_dict`` found in -your ``noxconfig.py::PROJECT_CONFIG``. This dictionary is inherited by default from -``BaseConfig.py``, ensuring a standardized baseline that can be easily overridden, if -necessary. - -.. literalinclude:: ../../../../exasol/toolbox/config.py - :language: python - :start-at: github_template_dict - Workflows --------- @@ -112,7 +104,7 @@ Pull Request When any pull request is opened, synchronized, or reopened, then the ``ci.yml`` will be triggered. -When configured as described on :ref:`github_workflows_configuration`, the +When configured as described on :ref:`github_project_configuration`, the ``run-slow-tests`` job requires a developer to manually approve executing the slower workflows, like ``slow-checks.yml``. This allows developers to update their pull request more often and to only periodically run the more time-expensive tests. diff --git a/doc/user_guide/features/github_workflows/template_variables.rst b/doc/user_guide/features/github_workflows/template_variables.rst new file mode 100644 index 000000000..f8355e3fa --- /dev/null +++ b/doc/user_guide/features/github_workflows/template_variables.rst @@ -0,0 +1,14 @@ +.. _template_variables: + +Template Variables +================== + +Underlying the CLI, the PTB uses Jinja to dynamically generate project-specific +workflows. The rendering process is supported by the ``github_template_dict`` found in +your ``noxconfig.py::PROJECT_CONFIG``. This dictionary is inherited by default from +``BaseConfig.py``, ensuring a standardized baseline that can be easily overridden, if +necessary. + +.. literalinclude:: ../../../../exasol/toolbox/config.py + :language: python + :start-at: github_template_dict From a8b31d1e95c729a2e2a4e387af4718c347b51ef1 Mon Sep 17 00:00:00 2001 From: Christoph Kuhnke Date: Thu, 19 Feb 2026 09:59:53 +0100 Subject: [PATCH 2/7] Apply suggestions from code review Co-authored-by: Steffen Pankratz --- doc/user_guide/features/github_workflows/create_and_update.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/user_guide/features/github_workflows/create_and_update.rst b/doc/user_guide/features/github_workflows/create_and_update.rst index ac991edb2..27c3f6649 100644 --- a/doc/user_guide/features/github_workflows/create_and_update.rst +++ b/doc/user_guide/features/github_workflows/create_and_update.rst @@ -46,7 +46,7 @@ PTB has a default for these versions, but you can override it in file exasol_versions=("7.1.30", "8.29.13", "2025.1.8"), ) -Some workflows are expected to not depend on a specific python version and +Some workflows are expected to not depend on a specific Python version and will use only the first of the Python version: ``PROJECT_CONFIG.python_versions[0]``. From bfd7c9f7c934adf73c9dac7c187cf6a120d2a79d Mon Sep 17 00:00:00 2001 From: ckunki Date: Thu, 19 Feb 2026 10:18:45 +0100 Subject: [PATCH 3/7] Updated user guide based on review findings --- .../github_workflows/create_and_update.rst | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/doc/user_guide/features/github_workflows/create_and_update.rst b/doc/user_guide/features/github_workflows/create_and_update.rst index 27c3f6649..da7875bf4 100644 --- a/doc/user_guide/features/github_workflows/create_and_update.rst +++ b/doc/user_guide/features/github_workflows/create_and_update.rst @@ -46,9 +46,18 @@ PTB has a default for these versions, but you can override it in file exasol_versions=("7.1.30", "8.29.13", "2025.1.8"), ) -Some workflows are expected to not depend on a specific Python version and -will use only the first of the Python version: -``PROJECT_CONFIG.python_versions[0]``. +Some workflows are expected to not depend on a specific python version and +will use only the lowest Python version in the list specified above. + +If you want to use a different version, though, you can override the default +in your ``PROJECT_CONFIG``: + +.. code-block:: shell + + PROJECT_CONFIG = Config( + python_versions=("3.10", "3.12), + minimum_python_version="3.12", + ) .. _update_workflows: From f8937948dd36d4797eba77b019f9c96de08d7cff Mon Sep 17 00:00:00 2001 From: ckunki Date: Thu, 19 Feb 2026 10:35:03 +0100 Subject: [PATCH 4/7] Added notes on Sonar token --- .../github_project_configuration.rst | 27 +++++++++++++++---- 1 file changed, 22 insertions(+), 5 deletions(-) diff --git a/doc/user_guide/features/github_workflows/github_project_configuration.rst b/doc/user_guide/features/github_workflows/github_project_configuration.rst index c6360fda0..1f8026cee 100644 --- a/doc/user_guide/features/github_workflows/github_project_configuration.rst +++ b/doc/user_guide/features/github_workflows/github_project_configuration.rst @@ -24,9 +24,26 @@ reviewers in (:code:`Settings/Environments/manual-approval`) and move the steps to be guarded into the related section in job :code:`slow-checks` in file :code:`.github/workflows/merge-gate.yml`. -PYPI Token ----------- +Secrets +------- -Make sure your GitHub project has access to a deployment token for PyPi with -the following name: **PYPI_TOKEN**. It should be available to the repository -either as an Organization-, Repository-, or Environment-secret. +For accessing specific services in the Internet, your project often needs a +related *token* or other credentials. These credentials can be acquired by +registering on the service's website. + +In many cases your company or organization may manage the credentials +centrally and enable to use the in multiple projects. The credentials can be +managed as Secrets in GitHub and can be made accessible to GitHub projects and +used by their workflows. + +In summary, your GitHub project may have individual secrets and/or secrets +inherited from the enclosing GitHub *Organization*. As soon as a secret is +accessible to your project, your GitHub workflows can use it and probably will +map it to *environment variables* that are used by your CI/CD automation. + +* **PYPI_TOKEN**: This secret is required to publish your project on the + `Python Package Index `_ (PyPi). Most projects will use the + org-secret. +* **SONAR_TOKEN**: See :ref:`Sonar Configuration `. + +.. _pypi: https://pypi.org/ From 616c23e870f40e422040caae12a91e9fa99bd8b4 Mon Sep 17 00:00:00 2001 From: Christoph Kuhnke Date: Thu, 19 Feb 2026 11:25:33 +0100 Subject: [PATCH 5/7] Apply suggestions from code review Co-authored-by: Steffen Pankratz --- doc/user_guide/features/github_workflows/create_and_update.rst | 2 +- .../features/github_workflows/github_project_configuration.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/user_guide/features/github_workflows/create_and_update.rst b/doc/user_guide/features/github_workflows/create_and_update.rst index da7875bf4..9d0c8f3ab 100644 --- a/doc/user_guide/features/github_workflows/create_and_update.rst +++ b/doc/user_guide/features/github_workflows/create_and_update.rst @@ -46,7 +46,7 @@ PTB has a default for these versions, but you can override it in file exasol_versions=("7.1.30", "8.29.13", "2025.1.8"), ) -Some workflows are expected to not depend on a specific python version and +Some workflows are expected to not depend on a specific Python version and will use only the lowest Python version in the list specified above. If you want to use a different version, though, you can override the default diff --git a/doc/user_guide/features/github_workflows/github_project_configuration.rst b/doc/user_guide/features/github_workflows/github_project_configuration.rst index 1f8026cee..c962f2fb5 100644 --- a/doc/user_guide/features/github_workflows/github_project_configuration.rst +++ b/doc/user_guide/features/github_workflows/github_project_configuration.rst @@ -32,7 +32,7 @@ related *token* or other credentials. These credentials can be acquired by registering on the service's website. In many cases your company or organization may manage the credentials -centrally and enable to use the in multiple projects. The credentials can be +centrally and enable the use in multiple projects. The credentials can be managed as Secrets in GitHub and can be made accessible to GitHub projects and used by their workflows. From 6639ebc6dc180a6154db40776b2cd747511773b4 Mon Sep 17 00:00:00 2001 From: ckunki Date: Thu, 19 Feb 2026 11:27:28 +0100 Subject: [PATCH 6/7] Updated user guide on overriding minimum_python_version --- .../features/github_workflows/create_and_update.rst | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/doc/user_guide/features/github_workflows/create_and_update.rst b/doc/user_guide/features/github_workflows/create_and_update.rst index 9d0c8f3ab..9aa3b7710 100644 --- a/doc/user_guide/features/github_workflows/create_and_update.rst +++ b/doc/user_guide/features/github_workflows/create_and_update.rst @@ -49,16 +49,6 @@ PTB has a default for these versions, but you can override it in file Some workflows are expected to not depend on a specific Python version and will use only the lowest Python version in the list specified above. -If you want to use a different version, though, you can override the default -in your ``PROJECT_CONFIG``: - -.. code-block:: shell - - PROJECT_CONFIG = Config( - python_versions=("3.10", "3.12), - minimum_python_version="3.12", - ) - .. _update_workflows: Add all Workflows to Your Project From 24c8f35f8cfae0d9c4f7c7d7bf2e3186a29cbb6d Mon Sep 17 00:00:00 2001 From: ckunki Date: Thu, 19 Feb 2026 11:29:59 +0100 Subject: [PATCH 7/7] Removed obsolete file from doc --- .../features/github_workflows/troubleshooting.rst | 9 --------- 1 file changed, 9 deletions(-) delete mode 100644 doc/user_guide/features/github_workflows/troubleshooting.rst diff --git a/doc/user_guide/features/github_workflows/troubleshooting.rst b/doc/user_guide/features/github_workflows/troubleshooting.rst deleted file mode 100644 index 1913048d4..000000000 --- a/doc/user_guide/features/github_workflows/troubleshooting.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. _workflows_troubleshooting: - -Troubleshooting -=============== - -.. toctree:: - :maxdepth: 2 - - ../../troubleshooting/debug_github_workflows