-
Notifications
You must be signed in to change notification settings - Fork 4
Add upstream integration and release workflows for toolkit updates #6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
54 commits
Select commit
Hold shift + click to select a range
ceccf7b
Add upstream integration workflow for toolkit updates
opajonk 96d204a
Add a test step for fastdev image
vzeissler e99817c
Refactor BitBake execution fix to use the dedicated script
opajonk fd73e46
Enhance upstream integration workflow
opajonk 67013cd
file does not yet have the refactoring
opajonk 1084fdf
Fix path to BitBake execution fixer script in upstream integration wo…
opajonk caa6855
remove build output as well
opajonk c476acb
keep github.com specific files / folders
opajonk dae13c2
temporarily disable image building
opajonk cb7cf6c
do not copy prebuilt and build
opajonk 3cc8e7c
package also prebuilt as asset
opajonk 76138dc
update paths for user manual and prebuilt assets in workflow
opajonk a717207
set working directory for draft release creation step
opajonk c4b5c2f
create PR, update container tags and populate containers
matthiasb85 72157d0
added PR permission
matthiasb85 bcaea31
fixed grep error
matthiasb85 42a74e5
fixed broken variable
matthiasb85 00b5470
fixed broken escape sequence
matthiasb85 7d925d3
fix container name and exclude build container due to broken name
matthiasb85 4b35d32
only update the branch when we have changes
matthiasb85 06323e6
added login in order to push the docker images
matthiasb85 f39ff07
also push buildcontainer
matthiasb85 dd9ba75
use "v" as prefix for container tags
opajonk 13930e2
add also individual assets per target, use tgz instead of ZIP
opajonk 6b3583f
replace zip with tgz for user manual HTML assets
opajonk 25a340a
change file extensions from .tgz to .tar.gz for user manual and prebu…
opajonk d3be42d
add release workflow to publish user guide asset to GitHub Pages
opajonk 22a2fda
update user guide asset filename to user_manual_html.tar.gz
opajonk 84df917
add GitHub Actions workflow for release activities and user guide ass…
opajonk d2f6faf
add GitHub Actions workflow for executing release activities and publ…
opajonk 91d0161
test
opajonk a8d18b6
add push trigger to Hello World Echo workflow
opajonk d419d3d
prevent workflow execution on push events; require manual dispatch in…
opajonk 53cfda9
remove Hello World Echo workflow file
opajonk 5491524
remove branch trigger from release workflow and clean up abort step
opajonk f0a4c2e
update archive path and environment variables in upstream integration…
opajonk 1482f39
update README to enhance getting started instructions and clarify usa…
opajonk ac02cc3
refactor release workflow to trigger on published releases only and r…
opajonk 6a0c269
add error handling for missing release in user guide asset verification
opajonk 35a41dd
update README for clarity and consistency in getting started instruct…
opajonk 7976991
re-renable test builds and streamline workflow
opajonk b60aa53
update workflow name for clarity on upstream integration process
opajonk 26bf505
refactor workflow steps for clarity in tagging and asset packaging
opajonk f817d74
refactor release workflow for clarity and consistency in asset verifi…
opajonk ed72017
refactor upstream integration workflow for improved error handling an…
opajonk 9fcf384
add scripts for downloading, integrating, and packaging release asset…
opajonk e58bce1
integrate all validation to the earliest possible location (fail fast)
opajonk 679fc63
enhance workflow documentation for release and upstream integration p…
opajonk f4232d4
Apply suggestions from code review
opajonk 16407ed
Update after review, add notes about env vars
opajonk 1a2b3e0
Refactor scripts to use command-line arguments and improve error hand…
opajonk 5194531
Enhance QEMU smoke test logging and update integration workflow to ca…
opajonk bef5534
Refactor workflows to use default shell settings for improved error h…
opajonk 3eeb9b2
push only in case of changes
opajonk File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,101 @@ | ||
| #!/usr/bin/env bash | ||
| # Copyright 2026 Elektrobit. All rights reserved. | ||
| # | ||
| # Download the upstream toolkit archive, extract it into a temporary directory | ||
| # and print the resulting paths as KEY=VALUE lines on stdout. | ||
| # | ||
| # Usage: download_archive.sh --url <URL> --token <TOKEN> | ||
| set -euo pipefail | ||
|
|
||
| url="" | ||
| token="" | ||
|
|
||
| while [[ $# -gt 0 ]]; do | ||
| case "$1" in | ||
| --url) url="$2"; shift 2 ;; | ||
| --token) token="$2"; shift 2 ;; | ||
| *) echo "Unknown argument: $1" >&2; exit 1 ;; | ||
| esac | ||
| done | ||
|
|
||
| if [[ -z "${url}" ]]; then | ||
| echo "ERROR: Required argument --url is not set." >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| if [[ -z "${token}" ]]; then | ||
| echo "ERROR: Required argument --token is not set." >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| tmp_dir="$(mktemp -d)" | ||
| archive_path="${tmp_dir}/eb_corbos_toolkit.tar.gz" | ||
| extract_root="${tmp_dir}/extracted" | ||
|
|
||
| mkdir -p "${extract_root}" | ||
|
|
||
| echo "Starting upstream delivery archive download (this may take a while for large files)..." >&2 | ||
| echo "Download target: ${archive_path}" >&2 | ||
|
|
||
| curl \ | ||
| --fail \ | ||
| --progress-bar \ | ||
| --show-error \ | ||
| --location \ | ||
| --retry 3 \ | ||
| --retry-delay 5 \ | ||
| --retry-all-errors \ | ||
| --header "Authorization: Bearer ${token}" \ | ||
| --output "${archive_path}" \ | ||
| --url "${url}" | ||
|
|
||
| archive_size_bytes="$(wc -c < "${archive_path}")" | ||
| archive_size_mib="$(awk "BEGIN { printf \"%.2f\", ${archive_size_bytes}/1024/1024 }")" | ||
| echo "Download complete: ${archive_size_bytes} bytes (${archive_size_mib} MiB)." >&2 | ||
|
|
||
| echo "Starting upstream delivery archive extraction into ${extract_root}..." >&2 | ||
|
|
||
| tar -xzf "${archive_path}" -C "${extract_root}" | ||
|
|
||
| echo "Extraction complete." >&2 | ||
|
|
||
| toolkit_root="${extract_root}/eb_corbos_toolkit" | ||
| workspace_root="${toolkit_root}/workspace" | ||
| devcontainer_archive="${toolkit_root}/containers/devcontainer-trixie-ebclfsa-amd64.docker-archive.zst" | ||
| buildcontainer_archive="${toolkit_root}/containers/buildcontainer-trixie-ebclfsa-amd64.docker-archive.zst" | ||
| run_script="${workspace_root}/scripts/run.sh" | ||
| bitbake_execution_fixer="${workspace_root}/.devcontainer/scripts/check_userns_restriction.sh" | ||
| manual_pdf="${toolkit_root}/doc/user_manual.pdf" | ||
| manual_html_dir="${toolkit_root}/doc/html" | ||
| prebuilt_dir="${workspace_root}/prebuilt" | ||
|
|
||
| echo "Validating extracted layout..." >&2 | ||
| for f in "${devcontainer_archive}" "${buildcontainer_archive}" "${run_script}" "${bitbake_execution_fixer}" "${manual_pdf}"; do | ||
| if [[ ! -f "${f}" ]]; then | ||
| echo "ERROR: Expected file not found in extracted upstream delivery archive: ${f}" >&2 | ||
| exit 1 | ||
| fi | ||
| done | ||
| for d in "${manual_html_dir}" "${prebuilt_dir}"; do | ||
| if [[ ! -d "${d}" ]]; then | ||
| echo "ERROR: Expected directory not found in extracted upstream delivery archive: ${d}" >&2 | ||
| exit 1 | ||
| fi | ||
| done | ||
| echo "Extracted layout looks good." >&2 | ||
|
|
||
| cat << EOF | ||
| TMP_WORK_DIR=${tmp_dir} | ||
| EXTRACT_ROOT=${extract_root} | ||
| ARCHIVE_PATH=${archive_path} | ||
| WORKSPACE_ROOT=${workspace_root} | ||
| DEVCONTAINER_ARCHIVE=${devcontainer_archive} | ||
| BUILDCONTAINER_ARCHIVE=${buildcontainer_archive} | ||
| DEVCONTAINER_IMAGE=ghcr.io/elektrobit/eb-corbos-toolkit-devcontainer-amd64 | ||
| BUILDCONTAINER_IMAGE=ghcr.io/elektrobit/eb-corbos-toolkit-buildcontainer-amd64 | ||
| RUN_SCRIPT=${run_script} | ||
| BITBAKE_EXECUTION_FIXER=${bitbake_execution_fixer} | ||
| MANUAL_PDF=${manual_pdf} | ||
| MANUAL_HTML_DIR=${manual_html_dir} | ||
| PREBUILT_DIR=${prebuilt_dir} | ||
| EOF |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| #!/usr/bin/env bash | ||
| # Copyright 2026 Elektrobit. All rights reserved. | ||
| # | ||
| # Replace the integration repository's working tree with the upstream workspace, | ||
| # preserving repository-owned metadata files. Must be run from the root of the | ||
| # integration repository checkout. | ||
| # | ||
| # Usage: integrate_workspace.sh --workspace-root <DIR> | ||
| set -euo pipefail | ||
|
|
||
| workspace_root="" | ||
|
|
||
| while [[ $# -gt 0 ]]; do | ||
| case "$1" in | ||
| --workspace-root) workspace_root="$2"; shift 2 ;; | ||
| *) echo "Unknown argument: $1" >&2; exit 1 ;; | ||
| esac | ||
| done | ||
|
|
||
| if [[ -z "${workspace_root}" ]]; then | ||
| echo "ERROR: Required argument --workspace-root is not set." >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| if [[ ! -d "${workspace_root}" ]]; then | ||
| echo "ERROR: Workspace root '${workspace_root}' does not exist." >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Remove all tracked and untracked content except the repository metadata and | ||
| # files that must remain in the integration repository. Add entries to this list | ||
| # when additional repository-owned content should be preserved. | ||
| preserve_paths=( | ||
| '.git' | ||
| '.github' | ||
| '.gitignore' | ||
| 'CODE_OF_CONDUCT.md' | ||
| 'README.md' | ||
| ) | ||
|
|
||
| while IFS= read -r -d '' path; do | ||
| name="${path#./}" | ||
| preserve=false | ||
| for preserve_path in "${preserve_paths[@]}"; do | ||
| if [[ "${name}" == "${preserve_path}" ]]; then | ||
| preserve=true | ||
| break | ||
| fi | ||
| done | ||
|
|
||
| if [[ "${preserve}" != true ]]; then | ||
| rm -rf -- "${path}" | ||
| fi | ||
| done < <(find . -mindepth 1 -maxdepth 1 -print0) | ||
|
|
||
| # Copy the upstream workspace on top, preserving flags, mtimes, ownership. | ||
| # Ignore the "prebuilt" and "build" sub-folders. | ||
| rsync -a --exclude='prebuilt' --exclude='build' "${workspace_root}/." . | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| #!/usr/bin/env bash | ||
| # Copyright 2026 Elektrobit. All rights reserved. | ||
| # | ||
| # Collect the release assets (toolkit archive, user manual, prebuilt per-target | ||
| # images, SDK sysroots and SSH keys) into a single directory and print its path | ||
| # as a KEY=VALUE line on stdout. | ||
| # | ||
| # Usage: package_release_assets.sh --tmp-work-dir <DIR> --archive-path <FILE> \ | ||
| # --manual-pdf <FILE> --manual-html-dir <DIR> --prebuilt-dir <DIR> | ||
| set -euo pipefail | ||
|
|
||
| tmp_work_dir="" | ||
| archive_path="" | ||
| manual_pdf="" | ||
| manual_html_dir="" | ||
| prebuilt_dir="" | ||
|
|
||
| while [[ $# -gt 0 ]]; do | ||
| case "$1" in | ||
| --tmp-work-dir) tmp_work_dir="$2"; shift 2 ;; | ||
| --archive-path) archive_path="$2"; shift 2 ;; | ||
| --manual-pdf) manual_pdf="$2"; shift 2 ;; | ||
| --manual-html-dir) manual_html_dir="$2"; shift 2 ;; | ||
| --prebuilt-dir) prebuilt_dir="$2"; shift 2 ;; | ||
| *) echo "Unknown argument: $1" >&2; exit 1 ;; | ||
| esac | ||
| done | ||
|
|
||
| for arg_name in tmp_work_dir archive_path manual_pdf manual_html_dir prebuilt_dir; do | ||
| if [[ -z "${!arg_name}" ]]; then | ||
| echo "ERROR: Required argument --${arg_name//_/-} is not set." >&2 | ||
| exit 1 | ||
| fi | ||
| done | ||
|
|
||
| assets_dir="${tmp_work_dir}/release-assets" | ||
| mkdir -p "${assets_dir}" | ||
|
|
||
| mv "${archive_path}" "${assets_dir}/eb_corbos_toolkit.tar.gz" | ||
| cp "${manual_pdf}" "${assets_dir}/user_manual.pdf" | ||
| tar -czf "${assets_dir}/user_manual_html.tar.gz" -C "${manual_html_dir}" . | ||
|
|
||
| # Per-target assets: one image archive (named after its .wic file), the SDK | ||
| # sysroot tarball and the SSH target key for every target directory. | ||
| for target_dir in "${prebuilt_dir}"/*/; do | ||
| [[ -d "${target_dir}" ]] || continue | ||
|
|
||
| image_dir="${target_dir}image" | ||
| if [[ -d "${image_dir}" ]]; then | ||
| wic_file="$(find "${image_dir}" -maxdepth 1 -name '*.wic' -print -quit)" | ||
| if [[ -z "${wic_file}" ]]; then | ||
| echo "ERROR: No .wic file found in ${image_dir}." >&2 | ||
| exit 1 | ||
| fi | ||
| image_base="$(basename "${wic_file}" .wic)" | ||
| tar -czf "${assets_dir}/${image_base}.tar.gz" -C "${image_dir}" . | ||
| fi | ||
|
|
||
| sdk_dir="${target_dir}sysroot" | ||
| if [[ -d "${sdk_dir}" ]]; then | ||
| find "${sdk_dir}" -maxdepth 1 -type f -exec cp {} "${assets_dir}/" \; | ||
| fi | ||
|
|
||
| key_dir="${target_dir}ssh_keys" | ||
| if [[ -d "${key_dir}" ]]; then | ||
| find "${key_dir}" -maxdepth 1 -type f -exec cp {} "${assets_dir}/" \; | ||
| fi | ||
| done | ||
|
opajonk marked this conversation as resolved.
|
||
|
|
||
| echo "RELEASE_ASSETS_DIR=${assets_dir}" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| #!/usr/bin/env bash | ||
| # Copyright 2026 Elektrobit. All rights reserved. | ||
| # | ||
| # Smoke test: boot the freshly built "fastdev" image in QEMU inside the | ||
| # devcontainer, wait for SSH availability, power it down cleanly and confirm the | ||
| # shutdown message appears in the QEMU log. | ||
| # | ||
| # Usage: qemu_smoke_test.sh --workspace-root <DIR> --run-script <PATH> | ||
| set -euo pipefail | ||
|
|
||
| workspace_root="" | ||
| run_script="" | ||
|
|
||
| while [[ $# -gt 0 ]]; do | ||
| case "$1" in | ||
| --workspace-root) workspace_root="$2"; shift 2 ;; | ||
| --run-script) run_script="$2"; shift 2 ;; | ||
| *) echo "Unknown argument: $1" >&2; exit 1 ;; | ||
| esac | ||
| done | ||
|
|
||
| if [[ -z "${workspace_root}" ]]; then | ||
| echo "ERROR: Required argument --workspace-root is not set." >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| if [[ -z "${run_script}" ]]; then | ||
| echo "ERROR: Required argument --run-script is not set." >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| cd "${workspace_root}" | ||
|
|
||
| # Run the dev container in detached mode and keep it running. | ||
| export EXTRA_DOCKER_OPTIONS="-d --log-driver local" | ||
| LOG_FILE=$(mktemp -t ebcl_run_devcontainer-XXXXXX.log 2>/dev/null) || | ||
| LOG_FILE="/tmp/ebcl_run_devcontainer.log" | ||
|
|
||
| echo "Starting devcontainer in the background" >&2 | ||
| DEV_CONTAINER_ID=$("${run_script}" -d -- bash -c "sleep infinity" 2>&1 | tee "${LOG_FILE}" | grep -oE '^[0-9a-f]{64}$') | ||
| if [[ -z "${DEV_CONTAINER_ID}" ]]; then | ||
| echo "ERROR: Failed to start devcontainer. Log output:" >&2 | ||
| cat "${LOG_FILE}" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "DEV_CONTAINER_ID=${DEV_CONTAINER_ID}" | ||
| sleep 10 | ||
|
|
||
| devcontainer_exec() { | ||
| docker exec "${DEV_CONTAINER_ID}" "$@" | ||
| } | ||
|
|
||
| echo "Running fastdev in qemu in background, logging to qemu.log" >&2 | ||
| devcontainer_exec bash -c "./scripts/qemu.sh -t fastdev < /dev/null > qemu.log 2>&1 &" >&2 | ||
| devcontainer_exec bash -c \ | ||
| "log=/workspace/qemu.log; source /workspace/scripts/includes/common/common.inc; wait_for_ssh fastdev-qemuarm64 70 1" >&2 | ||
|
|
||
| echo "Terminating fastdev" >&2 | ||
| devcontainer_exec ssh fastdev-qemuarm64 crinit-ctl poweroff >&2 || true | ||
|
|
||
| echo "Waiting for fastdev to power down" >&2 | ||
| max_tries=20 | ||
| msg="Power down" | ||
| log="${workspace_root}/qemu.log" | ||
|
|
||
| for i in $(seq 1 "${max_tries}"); do | ||
| if grep -q "${msg}" "${log}" 2>/dev/null; then | ||
| echo "Waiting for \"${msg}\" message in ${log} succeeded" >&2 | ||
| rm -f "${log}" | ||
| exit 0 | ||
| fi | ||
| echo "Waiting for \"${msg}\" message in ${log} ... (${i}/${max_tries})" >&2 | ||
| sleep 1 | ||
| done | ||
|
|
||
| echo "ERROR: Waiting for \"${msg}\" message in ${log} timed out!" >&2 | ||
| cat "${log}" >&2 | ||
| exit 1 |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.