Skip to content

add onnx2c_stable #101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
andife wants to merge 9 commits into
base: main
Choose a base branch
from
Open

add onnx2c_stable #101

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
e4736b6
update
andife Mar 21, 2026
f1e57bd
Fix ruff lint errors in onnx2c backend
andife Mar 21, 2026
5cd2f8e
Fix TVM runtime package compatibility (#102)
strimo378 Mar 21, 2026
fdb7033
Fix ruff format: collapse subprocess.run call to single line
andife Mar 21, 2026
0b538ac
Add path-filtered push/PR triggers to benchmark workflow
andife Mar 21, 2026
78622b8
Fix onnx2c build: clone with --recurse-submodules
andife Mar 21, 2026
790e117
Pin onnx2c to last known-good commit
andife Mar 21, 2026
1044a0b
Narrow path filters: exclude setup/config.json from triggers
andife Mar 21, 2026
6b09801
update
andife Mar 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
153 changes: 152 additions & 1 deletion .github/workflows/benchmark-backends.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,109 @@
schedule:
- cron: '0 0 * * *'
workflow_dispatch:
push:
branches: [main]
paths:
- 'runtimes/**'
- 'backends/**'
- 'test/**'
- 'setup/docker-setup.sh'
- 'setup/env.list'
- 'setup/requirements_report.txt'
pull_request:
paths:
- 'runtimes/**'
- 'backends/**'
- 'test/**'
- 'setup/docker-setup.sh'
- 'setup/env.list'
- 'setup/requirements_report.txt'

jobs:
changes:
runs-on: ubuntu-latest
if: github.event_name == 'push' || github.event_name == 'pull_request'
outputs:
onnxruntime: ${{ steps.filter.outputs.onnxruntime }}
onnx_tf: ${{ steps.filter.outputs.onnx_tf }}
jaxonnxruntime: ${{ steps.filter.outputs.jaxonnxruntime }}
emx_onnx_cgen: ${{ steps.filter.outputs.emx_onnx_cgen }}
tract: ${{ steps.filter.outputs.tract }}
onnx_reference: ${{ steps.filter.outputs.onnx_reference }}
opencv: ${{ steps.filter.outputs.opencv }}
tvm: ${{ steps.filter.outputs.tvm }}
onnx2c: ${{ steps.filter.outputs.onnx2c }}
steps:
- uses: actions/checkout@v6
- uses: dorny/paths-filter@v3
id: filter
with:
filters: |
onnxruntime:
- 'runtimes/ort/**'
- 'test/**'
- 'setup/docker-setup.sh'
- 'setup/env.list'
- 'setup/requirements_report.txt'
onnx_tf:
- 'runtimes/onnx-tf/**'
- 'test/**'
- 'setup/docker-setup.sh'
- 'setup/env.list'
- 'setup/requirements_report.txt'
jaxonnxruntime:
- 'runtimes/jaxonnxruntime/**'
- 'test/**'
- 'setup/docker-setup.sh'
- 'setup/env.list'
- 'setup/requirements_report.txt'
emx_onnx_cgen:
- 'runtimes/emx-onnx-cgen/**'
- 'test/**'
- 'setup/docker-setup.sh'
- 'setup/env.list'
- 'setup/requirements_report.txt'
tract:
- 'runtimes/tract/**'
- 'backends/tract/**'
- 'test/**'
- 'setup/docker-setup.sh'
- 'setup/env.list'
- 'setup/requirements_report.txt'
onnx_reference:
- 'runtimes/onnx-reference/**'
- 'backends/onnx_reference/**'
- 'test/**'
- 'setup/docker-setup.sh'
- 'setup/env.list'
- 'setup/requirements_report.txt'
opencv:
- 'runtimes/opencv/**'
- 'backends/opencv/**'
- 'test/**'
- 'setup/docker-setup.sh'
- 'setup/env.list'
- 'setup/requirements_report.txt'
tvm:
- 'runtimes/tvm/**'
- 'backends/tvm/**'
- 'test/**'
- 'setup/docker-setup.sh'
- 'setup/env.list'
- 'setup/requirements_report.txt'
onnx2c:
- 'runtimes/onnx2c/**'
- 'backends/onnx2c/**'
- 'test/**'
- 'setup/docker-setup.sh'
- 'setup/env.list'
- 'setup/requirements_report.txt'

onnxruntime_stable:

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}
runs-on: ubuntu-latest
timeout-minutes: 90
needs: changes
if: always() && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || needs.changes.outputs.onnxruntime == 'true')
permissions:
contents: read
steps:
Expand All @@ -31,6 +129,8 @@
onnx_tf_stable:
runs-on: ubuntu-latest
timeout-minutes: 90
needs: changes
if: always() && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || needs.changes.outputs.onnx_tf == 'true')
permissions:
contents: read
steps:
Expand All @@ -54,6 +154,8 @@
onnx_tf_dev:
runs-on: ubuntu-latest
timeout-minutes: 90
needs: changes
if: always() && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || needs.changes.outputs.onnx_tf == 'true')
permissions:
contents: read
steps:
Expand All @@ -77,6 +179,8 @@
jaxonnxruntime_stable:
runs-on: ubuntu-latest
timeout-minutes: 90
needs: changes
if: always() && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || needs.changes.outputs.jaxonnxruntime == 'true')
permissions:
contents: read
steps:
Expand All @@ -99,6 +203,8 @@
jaxonnxruntime_dev:
runs-on: ubuntu-latest
timeout-minutes: 90
needs: changes
if: always() && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || needs.changes.outputs.jaxonnxruntime == 'true')
permissions:
contents: read
steps:
Expand All @@ -125,6 +231,8 @@
onnxruntime_dev:
runs-on: ubuntu-latest
timeout-minutes: 90
needs: changes
if: always() && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || needs.changes.outputs.onnxruntime == 'true')
permissions:
contents: read
steps:
Expand All @@ -150,6 +258,8 @@
emx_onnx_cgen_stable:
runs-on: ubuntu-latest
timeout-minutes: 90
needs: changes
if: always() && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || needs.changes.outputs.emx_onnx_cgen == 'true')
permissions:
contents: read
steps:
Expand All @@ -175,6 +285,8 @@
tract_stable:
runs-on: ubuntu-latest
timeout-minutes: 90
needs: changes
if: always() && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || needs.changes.outputs.tract == 'true')
permissions:
contents: read
steps:
Expand All @@ -200,6 +312,8 @@
onnx_reference_stable:
runs-on: ubuntu-latest
timeout-minutes: 90
needs: changes
if: always() && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || needs.changes.outputs.onnx_reference == 'true')
permissions:
contents: read
steps:
Expand All @@ -225,6 +339,8 @@
opencv_stable:
runs-on: ubuntu-latest
timeout-minutes: 90
needs: changes
if: always() && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || needs.changes.outputs.opencv == 'true')
permissions:
contents: read
steps:
Expand All @@ -250,6 +366,8 @@
tvm_stable:
runs-on: ubuntu-latest
timeout-minutes: 90
needs: changes
if: always() && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || needs.changes.outputs.tvm == 'true')
permissions:
contents: read
steps:
Expand All @@ -272,11 +390,38 @@
name: results-tvm-stable
path: results/tvm/stable/

onnx2c_stable:
runs-on: ubuntu-latest
timeout-minutes: 90
needs: changes
if: always() && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || needs.changes.outputs.onnx2c == 'true')
permissions:
contents: read
steps:
- name: Checkout code
uses: actions/checkout@v6

- name: Build docker image
run: docker build -t scoreboard/onnx2c -f runtimes/onnx2c/stable/Dockerfile .

- name: Prepare results directory
run: mkdir -p ${{ github.workspace }}/results/onnx2c/stable && chmod 777 ${{ github.workspace }}/results/onnx2c/stable

- name: Run docker container
run: docker run --name onnx2c --env-file setup/env.list -v ${{ github.workspace }}/results/onnx2c/stable:/root/results scoreboard/onnx2c || true

- name: Upload results
if: ${{ github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' }}
uses: actions/upload-artifact@v7
with:
name: results-onnx2c-stable
path: results/onnx2c/stable/

collect_and_deploy:
runs-on: ubuntu-latest
permissions:
contents: write
needs: [onnxruntime_stable, onnxruntime_dev, onnx_tf_stable, onnx_tf_dev, jaxonnxruntime_stable, jaxonnxruntime_dev, emx_onnx_cgen_stable, tract_stable, onnx_reference_stable, opencv_stable, tvm_stable]
needs: [onnxruntime_stable, onnxruntime_dev, onnx_tf_stable, onnx_tf_dev, jaxonnxruntime_stable, jaxonnxruntime_dev, emx_onnx_cgen_stable, tract_stable, onnx_reference_stable, opencv_stable, tvm_stable, onnx2c_stable]
if: ${{ github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' }}
steps:
- name: Checkout code
Expand Down Expand Up @@ -360,6 +505,12 @@
name: results-tvm-stable
path: results/tvm/stable/

- name: Download onnx2c stable results
uses: actions/download-artifact@v8
with:
name: results-onnx2c-stable
path: results/onnx2c/stable/

- name: Deploy results
run: |
git add results
Expand Down
Loading
Loading