diff --git a/.markdownlint.yml b/.markdownlint.yml new file mode 100644 index 00000000..d34e1b1e --- /dev/null +++ b/.markdownlint.yml @@ -0,0 +1,14 @@ +# For further information on the warnings visit +# https://github.com/markdownlint/markdownlint/blob/main/docs/RULES.md + +default: true + +# Allow some exceptions for inline HTML +MD033: + allowed_elements: ['picture', 'source', 'img'] + +# Disables need for first line to be a big heading +MD041: false + +# Disable line length warnings +MD013: false \ No newline at end of file diff --git a/.vscode/extensions.json b/.vscode/extensions.json index ad841eca..34365753 100644 --- a/.vscode/extensions.json +++ b/.vscode/extensions.json @@ -1,6 +1,7 @@ { "recommendations": [ "ms-python.python", - "ms-python.black-formatter" + "ms-python.black-formatter", + "DavidAnson.vscode-markdownlint" ] } diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 97caeec0..0cb6b708 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -20,10 +20,10 @@ besides a technical review: - **Python coding guidelines**: Make sure to follow the coding schema of the project. - **Ankaios SDK API alignment**: Keep in mind how the APIs of the other Ankaios SDKs would look like as all SDKs shall target similar API semantics. -If a version change is required in the PR, use the [update_version.sh](./tools/update_version.sh) script to update the version. +For information about development tools, workflows, and project setup, see the [Development Guide](DEVELOPMENT.md). ## Communication Please join our [developer mailing list](https://accounts.eclipse.org/mailing-list/ankaios-dev) for up to date information or use the Ankaios [discussion forum](https://github.com/eclipse-ankaios/ankaios/discussions). -If you are looking for the main project, you can find it [here](https://github.com/eclipse-ankaios/ankaios/tree/main). +If you are looking for the main project, you can find it [here](https://github.com/eclipse-ankaios/ankaios/tree/main). You can also join the conversion with the community in the [Ankaios Slack workspace](https://github.com/eclipse-ankaios/ankaios/wiki#slack). diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md new file mode 100644 index 00000000..b7580eaf --- /dev/null +++ b/DEVELOPMENT.md @@ -0,0 +1,38 @@ +# Development Guide + +This guide provides information about the development tools and workflows available for this project. + +## Quick Start + +### Project Setup + +Install the SDK in development mode with the dev dependencies: + +```shell +pip install -e ".[dev]" +``` + +## Commands and Scripts + +The project provides a unified check runner ([run_checks.py](./run_checks.py)) for common development workflows. For a complete list of the available checks and actions, run `python3 run_checks.py --help`. + +In addition, the [tools](tools/) folder contains helper scripts for specific tasks. Check the tools [readme](tools/README.md) for more info. + +## Updating Documentation + +When making changes to project documentation, keep the following in mind: + +**Documentation custom handling:** + +There are markdown files which are part of the published documentation. Because the links do not work directly, a [custom code](docs/source/conf.py) replaces them with the correct linkage. + +## Troubleshooting + +### Tests fail with strange errors regarding the protobuf objects + +If this is the case, the proto files do not match the local code. The fetching of the proto files is documented in the [setup.py](setup.py) script, in the `extract_the_proto_files` method. It might be the case that the proto files must be fetched again. + +## Additional Resources + +- [Contributing Guidelines](CONTRIBUTING.md) +- [Python Coding Guidelines](https://peps.python.org/pep-0008/) diff --git a/README.md b/README.md index cb941704..e04acccf 100644 --- a/README.md +++ b/README.md @@ -56,12 +56,12 @@ pip install -e ".[dev]" Please make sure the Python SDK is compatible with the version of Ankaios you are using. For information regarding versioning, please refer to this table: -| Ankaios | Python SDK | -| -------- | ------- | +| Ankaios | Python SDK | +| --------------- | ----------------------------------------------- | | 0.4.x and below | No Python SDK available. Please update Ankaios. | -| 0.5.x | 0.5.x | -| 0.6.x | 0.6.x | -| 0.7.x | 0.7.x | +| 0.5.x | 0.5.x | +| 0.6.x | 0.6.x | +| 0.7.x | 0.7.x | ## Usage @@ -134,6 +134,7 @@ with Ankaios() as ankaios: ``` For more details, please visit: + * [Ankaios documentation](https://eclipse-ankaios.github.io/ankaios/latest/) * [Python SDK documentation](https://eclipse-ankaios.github.io/ank-sdk-python/) diff --git a/docs/source/conf.py b/docs/source/conf.py index 3dc2ad11..ba5047a0 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -65,11 +65,14 @@ contrib[i] = line.replace( "./CODE_OF_CONDUCT.md", "./code_of_conduct.html" ) - break + if "./DEVELOPMENT.md" in line: + contrib[i] = line.replace( + "./DEVELOPMENT.md", + "https://github.com/eclipse-ankaios/ank-sdk-python/blob/main/DEVELOPMENT.md", + ) with open(contrib_out, "w") as f: f.writelines(contrib) - # -- Prepare the Code of Conduct file - foot note warning ----------------------------------------- coc_in = os.path.abspath(os.path.join(ROOT_DIR, "CODE_OF_CONDUCT.md")) coc_out = os.path.abspath(os.path.join(BUILD_DIR, "CODE_OF_CONDUCT.md")) diff --git a/examples/README.md b/examples/README.md index 1c4a70d4..87cdf7d2 100644 --- a/examples/README.md +++ b/examples/README.md @@ -9,7 +9,7 @@ In order to run the examples, Ankaios must be installed and available on the sys Alternatively you can compile from source and provide the path to the executables to the run scripts with the `ANK_BIN_DIR` environment variable. > **IMPORTANT** -> Make sure that no other Ankaios instance is running on the local machine. If there is an instance running, the example could change its configuration. The stop script would even stop all running instances and **delete all podman containers**. +> Make sure that no other Ankaios instance is running on the local machine. If there is an instance running, the example could change its configuration. The stop script would even stop all running instances and **delete all podman containers**. ## Building and running @@ -42,13 +42,13 @@ To stop the example just call ``` > **WARNING** -> The stop script terminates all running ankaios instances, both server and agents and **delete all podman containers** on the system. If you don't want this, you would need to clean up manually. +> The stop script terminates all running ankaios instances, both server and agents and **delete all podman containers** on the system. If you don't want this, you would need to clean up manually. ## Examples This folder currently contains three examples: -### `hello_ankaios` +### `hello_ankaios` This example executes a workload that runs the example script from the main [README](../README.md#usage) of this repo. @@ -60,13 +60,13 @@ To view the logs of an example like the `hello_ankaios` workload just use the co ./ank-logs.sh ``` -### `sleepy` +### `sleepy` -`sleepy` is an "empty" workload that just executes an endless sleep. +`sleepy` is an "empty" workload that just executes an endless sleep. This example has the Python SDK installed and allows to execute an interactive shell in the container in order to manually trigger Python commands to interact with Ankaios. -### `follow_state` +### `follow_state` `follow_state` executes indefinitely with a request to get the state being sent every 5 seconds. diff --git a/tools/README.md b/tools/README.md new file mode 100644 index 00000000..6a1014a1 --- /dev/null +++ b/tools/README.md @@ -0,0 +1,21 @@ +# Helper Scripts + +This directory contains utility scripts for specific tasks. All scripts support the `--help` flag for detailed usage information. + +## fetch_protos.sh + +This script is used to fetch the proto files from a specific branch. + +## generate_docs.sh + +Generates project documentation using Sphinx. Creates a temporary virtual environment, installs documentation dependencies, builds the HTML docs, and cleans up. + +Output is saved to `docs/build`. + +## update_version.sh + +Updates SDK, Ankaios, and API versions across the project. Supports updating versions individually or all at once. + +## install_ankaios.sh + +This script provides multiple ways to install ankaios, from an official release to the executables generated by a github action of a specific branch. Use the `--help` for more details. diff --git a/tools/fetch_protos.sh b/tools/fetch_protos.sh index e82aa5b8..110ae5a9 100755 --- a/tools/fetch_protos.sh +++ b/tools/fetch_protos.sh @@ -1,14 +1,29 @@ #!/bin/bash set -e -# Script to fetch the proto files from the Ankaios repository -# Usage: ./fetch_protos.sh [-v|--version ] +usage() { + echo "Usage: $0 [-v|--version ] [--help]" + echo "Fetch the proto files from the Ankaios repository." + echo "" + echo " The branch to fetch the proto files from." + echo " -v / --version The SDK version to use for the proto directory." + echo " If not provided, the version is read from setup.cfg." + echo " -h / --help Display this help message and exit." + echo "" + echo "Example:" + echo " $0 main" + echo " $0 main --version 0.2.0" + exit 0 +} # Parse arguments if [ $# -lt 1 ]; then echo "Error: Missing required argument " >&2 - echo "Usage: $0 [-v|--version ]" >&2 - exit 1 + usage +fi + +if [ "$1" = "--help" ] || [ "$1" = "-h" ]; then + usage fi BRANCH="$1" @@ -26,10 +41,12 @@ while [ $# -gt 0 ]; do SDK_VERSION="$2" shift 2 ;; + --help|-h) + usage + ;; *) echo "Error: Unknown argument: $1" >&2 - echo "Usage: $0 [-v|--version ]" >&2 - exit 1 + usage ;; esac done diff --git a/tools/generate_docs.sh b/tools/generate_docs.sh index 14e86696..caf5a7ef 100755 --- a/tools/generate_docs.sh +++ b/tools/generate_docs.sh @@ -1,9 +1,41 @@ #!/bin/bash + +# Copyright (c) 2025 Elektrobit Automotive GmbH +# +# This program and the accompanying materials are made available under the +# terms of the Apache License, Version 2.0 which is available at +# https://www.apache.org/licenses/LICENSE-2.0. +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# SPDX-License-Identifier: Apache-2.0 + set -e script_dir=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) base_dir="$script_dir/.." +usage() { + echo "Usage: $0 [--help]" + echo "" + echo "Generate project documentation using Sphinx." + echo "Creates a temporary virtual environment, installs documentation" + echo "dependencies, builds the HTML docs, and cleans up." + echo "" + echo "Output is saved to docs/build." + echo "" + echo " --help Display this help message and exit." + exit 0 +} + +if [ "$1" = "--help" ] || [ "$1" = "-h" ]; then + usage +fi + python3 -m venv docs_env source docs_env/bin/activate diff --git a/tools/install_ankaios.sh b/tools/install_ankaios.sh new file mode 100755 index 00000000..0b6d97c2 --- /dev/null +++ b/tools/install_ankaios.sh @@ -0,0 +1,296 @@ +#!/bin/bash + +# Copyright (c) 2026 Elektrobit Automotive GmbH +# +# This program and the accompanying materials are made available under the +# terms of the Apache License, Version 2.0 which is available at +# https://www.apache.org/licenses/LICENSE-2.0. +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +# SPDX-License-Identifier: Apache-2.0 + +set -e + +# Configuration +REPO="eclipse-ankaios/ankaios" +WORKFLOW_NAME="build.yml" +JOB_NAME="Build Linux {target} debian package" +ARTIFACT_NAME="ankaios-linux-{target}-bin" +INSTALL_LATEST_URL="https://github.com/eclipse-ankaios/ankaios/releases/latest/download/install.sh" +INSTALL_VERSION_URL="https://github.com/eclipse-ankaios/ankaios/releases/download//install.sh" +INSTALL_PATH="/usr/local/bin" +TEMP_DIR=$(mktemp -d) + +# Variables +branch_name="" +action_id="" +version="" +gh_token="" + +target="" +mode="" # branch, action, version + +usage() { + echo "Usage: $0 [--branch ] [--action ] [--version ] [--latest] [--token ] [--help]" + echo "Fetch the requested Ankaios binaries from Github." + echo "" + echo " -b / --branch Get binaries from the last successful action on this branch." + echo " -a / --action Get binaries from the matching action." + echo " -v / --version Get binaries from this specific release version (example: v0.1.0)." + echo " -l / --latest Get binaries from the latest release of Ankaios." + echo " -t / --token GitHub token for authentication." + echo " -h / --help Display this help message and exit." + echo "" + echo "Example:" + echo " $0 --branch main --token ghp_xxxxxxxxxxxxx" + exit 1 +} + +multiple_arguments_error() { + echo "Error: Only one of --branch, --action, --version, or --latest can be specified." >&2 + usage +} + +parse_arguments() { + while [ "$#" -gt 0 ]; do + case "$1" in + --branch|-b) + shift + branch_name="$1" + if [ -n "$mode" ]; then + multiple_arguments_error + fi + mode="branch" + ;; + --action|-a) + shift + action_id="$1" + if [ -n "$mode" ]; then + multiple_arguments_error + fi + mode="action" + ;; + --version|-v) + shift + version="$1" + if [ -n "$mode" ]; then + multiple_arguments_error + fi + mode="version" + ;; + --latest|-l) + version="latest" + if [ -n "$mode" ]; then + multiple_arguments_error + fi + mode="version" + ;; + --token|-t) + shift + gh_token="$1" + ;; + --help|-h) + usage + ;; + *) + echo "Unknown argument: $1" + usage + ;; + esac + shift + done +} + +if [ "$#" -eq 0 ]; then + usage +fi + +parse_arguments "$@" +if [ -z "$mode" ]; then + echo "Error: One of --branch, --action, --version, or --latest must be specified." >&2 + usage +fi + +# Function to get target platform +get_target() { + case "$(uname -m 2>/dev/null)" in + x86_64|amd64) echo amd64 ;; + aarch64|arm64) echo arm64 ;; + *) + echo "Error: target architecture not supported" >&2 + exit 1 + ;; + esac +} +target=$(get_target) + +# Install by version + +install_by_version() { + if [ "$version" = "latest" ]; then + curl -sfL "$INSTALL_LATEST_URL" | bash - + else + VERSION_URL=${INSTALL_VERSION_URL///$version} + curl -sfL "$VERSION_URL" | bash -s -- -v "$version" + fi +} +if [ "$mode" = "version" ]; then + install_by_version + exit 0 +fi + +# Install by branch + +branch_exists() { + if curl -s "https://api.github.com/repos/$REPO/branches/$1" | grep -q '"name":'; then + return 0 + else + return 1 + fi +} +get_action_id_for_branch() { + # Get workflow ID + WORKFLOW_ID=$(curl -s "https://api.github.com/repos/$REPO/actions/workflows/$WORKFLOW_NAME" | \ + python3 -c "import sys,json; print(json.load(sys.stdin).get('id',''))") + + if [ -z "$WORKFLOW_ID" ]; then + echo "Error: Workflow '$WORKFLOW_NAME' not found" >&2 + exit 1 + fi + + # Get recent runs and find one with successful job + local job_name="${JOB_NAME//\{target\}/$target}" + RUN_ID=$(curl -s "https://api.github.com/repos/$REPO/actions/workflows/$WORKFLOW_ID/runs?branch=$branch_name&per_page=10" | \ + python3 -c " +import sys, json +data = json.load(sys.stdin) +for run in data.get('workflow_runs', []): + run_id = run['id'] + jobs_url = f\"https://api.github.com/repos/$REPO/actions/runs/{run_id}/jobs\" + import urllib.request + jobs_data = json.loads(urllib.request.urlopen(jobs_url).read()) + for job in jobs_data.get('jobs', []): + if job['name'] == '$job_name' and job['conclusion'] == 'success': + print(run_id) + sys.exit(0) +") + + if [ -z "$RUN_ID" ]; then + echo "Error: No runs with successful '$job_name' job found on branch '$branch_name'" >&2 + exit 1 + fi + + echo "$RUN_ID" +} +if [ "$mode" = "branch" ]; then + if ! branch_exists "$branch_name"; then + echo "Error: Branch '$branch_name' does not exist in repository '$REPO'" >&2 + exit 1 + fi + + action_id=$(get_action_id_for_branch) + echo "Action ID found: $action_id" +fi + +# Cleanup function +cleanup() { + rm -rf "$TEMP_DIR" +} +trap cleanup EXIT + +# Install by action + +# Get artifact name +artifact_name="${ARTIFACT_NAME//\{target\}/$target}" + +# Fetch artifact list +ARTIFACT_JSON=$(curl -s "https://api.github.com/repos/$REPO/actions/runs/$action_id/artifacts") + +# Find artifact ID +ARTIFACT_ID=$(echo "$ARTIFACT_JSON" | python3 -c " +import sys, json +try: + data = json.load(sys.stdin) + for artifact in data.get('artifacts', []): + if artifact.get('name') == '$artifact_name': + print(artifact.get('id')) + sys.exit(0) + print('ERROR: Artifact not found', file=sys.stderr) + sys.exit(1) +except Exception as e: + print(f'ERROR: {e}', file=sys.stderr) + sys.exit(1) +" 2>&1) + +if echo "$ARTIFACT_ID" | grep -q "ERROR"; then + echo "Error: Could not find artifact '$artifact_name' in action $action_id" >&2 + exit 1 +fi + +echo "Downloading artifact..." + +provide_token() { + if [ -n "$GITHUB_TOKEN" ]; then + gh_token="$GITHUB_TOKEN" + else + echo -n "Enter GitHub token: " + read -sr gh_token + fi +} +check_token() { + if ! curl -s -H "Authorization: Bearer $gh_token" "https://api.github.com/user" | grep -q '"login":'; then + echo "Error: Invalid GitHub token." >&2 + exit 1 + fi +} + +if [ -z "$gh_token" ]; then + provide_token +fi +check_token + +# Download artifact - gh token required +HTTP_CODE=$(curl -L -w "%{http_code}" -o "$TEMP_DIR/artifact.zip" \ + -H "Authorization: Bearer $gh_token" \ + -H "Accept: application/vnd.github+json" \ + -H "X-GitHub-Api-Version: 2022-11-28" \ + "https://api.github.com/repos/$REPO/actions/artifacts/$ARTIFACT_ID/zip") + +if [ "$HTTP_CODE" != "200" ]; then + echo "Error: Failed to download artifact (HTTP $HTTP_CODE)" >&2 + exit 1 +fi + +# Extract artifact +cd "$TEMP_DIR" +unzip -q artifact.zip + +# Verify binaries +if [ ! -f "ank" ] || [ ! -f "ank-server" ] || [ ! -f "ank-agent" ]; then + echo "Error: Expected binaries not found in artifact" >&2 + exit 1 +fi + +# Install binaries +echo "Installing binaries to $INSTALL_PATH..." + +if [ -w "$INSTALL_PATH" ]; then + SUDO="" +else + SUDO="sudo" +fi + +$SUDO cp ank "$INSTALL_PATH/ank" +$SUDO cp ank-server "$INSTALL_PATH/ank-server" +$SUDO cp ank-agent "$INSTALL_PATH/ank-agent" + +$SUDO chmod +x "$INSTALL_PATH/ank" +$SUDO chmod +x "$INSTALL_PATH/ank-server" +$SUDO chmod +x "$INSTALL_PATH/ank-agent" + +echo "Installation complete!"