Skip to content

feat: build apisix-runtime with ngx_http_ffi_client #480

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
shreemaan-abhishek wants to merge 3 commits into from
Closed

feat: build apisix-runtime with ngx_http_ffi_client #480

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
bb13a2c
feat: build apisix-runtime with ngx_http_ffi_client
shreemaan-abhishek Aug 5, 2026
74ead74
fix: keep the ngx_http_ffi_client token out of the build trace
shreemaan-abhishek Aug 5, 2026
1f18875
ci: fail a release build that has no ngx_http_ffi_client token
shreemaan-abhishek Aug 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/package-apisix-runtime-deb-openresty-1.21.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ jobs:
sudo apt-get install -y make ruby ruby-dev rubygems build-essential

- name: Build apisix-runtime deb
env:
NGX_HTTP_FFI_CLIENT_TOKEN: ${{ secrets.NGX_HTTP_FFI_CLIENT_TOKEN }}
run: |
if [ "${{ matrix.platform.arch }}" == "arm64" ]; then
make package type=deb app=apisix-runtime runtime_version=${BUILD_APISIX_RUNTIME_VERSION} image_base=debian image_tag=bullseye-slim arch=linux/arm64/v8
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/package-apisix-runtime-deb-ubuntu20.04.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ jobs:
sudo apt-get install -y make ruby ruby-dev rubygems build-essential

- name: build apisix-runtime deb
env:
NGX_HTTP_FFI_CLIENT_TOKEN: ${{ secrets.NGX_HTTP_FFI_CLIENT_TOKEN }}
run: |
make package type=deb app=apisix-runtime runtime_version=${BUILD_APISIX_RUNTIME_VERSION} image_base=ubuntu image_tag=24.04

Expand Down
1 change: 1 addition & 0 deletions .github/workflows/package-apisix-runtime-rpm-el.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ jobs:
env:
# Stream inner docker build output so a failing build step surfaces in CI.
BUILDKIT_PROGRESS: plain
NGX_HTTP_FFI_CLIENT_TOKEN: ${{ secrets.NGX_HTTP_FFI_CLIENT_TOKEN }}
run: |
make package type=rpm app=apisix-runtime runtime_version=${BUILD_APISIX_RUNTIME_VERSION} \
image_base=rockylinux image_tag=${{ matrix.dist.image_tag }} arch=linux/amd64
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/package-apisix-runtime-rpm-ubi.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ jobs:
sudo apt-get install -y make ruby ruby-dev rubygems build-essential

- name: build apisix-runtime rpm
env:
NGX_HTTP_FFI_CLIENT_TOKEN: ${{ secrets.NGX_HTTP_FFI_CLIENT_TOKEN }}
run: |
make package type=rpm app=apisix-runtime runtime_version=${BUILD_APISIX_RUNTIME_VERSION} image_base=registry.access.redhat.com/ubi9/ubi image_tag=9.6

Expand Down
29 changes: 29 additions & 0 deletions .github/workflows/release-apisix-runtime.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,27 @@ jobs:
sudo apt-get update
sudo apt-get install -y make ruby ruby-dev rubygems build-essential

# A released runtime must carry ngx_http_ffi_client. The build itself is
# lenient so fork PRs, which get no secrets, still pass; here the secret
# has to be there, or the release would silently ship without the module.
- name: Require the ngx_http_ffi_client token
env:
NGX_HTTP_FFI_CLIENT_TOKEN: ${{ secrets.NGX_HTTP_FFI_CLIENT_TOKEN }}
run: |
if [ -z "${NGX_HTTP_FFI_CLIENT_TOKEN}" ]; then
echo "NGX_HTTP_FFI_CLIENT_TOKEN is not set; a release build must carry ngx_http_ffi_client" >&2
exit 1
fi

- name: Build apisix-runtime deb
env:
NGX_HTTP_FFI_CLIENT_TOKEN: ${{ secrets.NGX_HTTP_FFI_CLIENT_TOKEN }}
run: |
make package type=deb app=apisix-runtime runtime_version="${VERSION}" image_base=debian image_tag=bookworm-slim arch=${{ matrix.platform.build_arch }}

- name: Build apisix-runtime-debug deb
env:
NGX_HTTP_FFI_CLIENT_TOKEN: ${{ secrets.NGX_HTTP_FFI_CLIENT_TOKEN }}
run: |
make package type=deb app=apisix-runtime runtime_version="${VERSION}" image_base=debian image_tag=bookworm-slim arch=${{ matrix.platform.build_arch }} build_latest=latest artifact=apisix-runtime-debug

Expand Down Expand Up @@ -111,10 +127,23 @@ jobs:
sudo apt-get update
sudo apt-get install -y make ruby ruby-dev rubygems build-essential rpm

# A released runtime must carry ngx_http_ffi_client. The build itself is
# lenient so fork PRs, which get no secrets, still pass; here the secret
# has to be there, or the release would silently ship without the module.
- name: Require the ngx_http_ffi_client token
env:
NGX_HTTP_FFI_CLIENT_TOKEN: ${{ secrets.NGX_HTTP_FFI_CLIENT_TOKEN }}
run: |
if [ -z "${NGX_HTTP_FFI_CLIENT_TOKEN}" ]; then
echo "NGX_HTTP_FFI_CLIENT_TOKEN is not set; a release build must carry ngx_http_ffi_client" >&2
exit 1
fi

- name: Build apisix-runtime rpm (${{ matrix.dist.el }} ${{ matrix.platform.rpm_arch }})
env:
# Stream inner docker build output so a failing build step surfaces in CI.
BUILDKIT_PROGRESS: plain
NGX_HTTP_FFI_CLIENT_TOKEN: ${{ secrets.NGX_HTTP_FFI_CLIENT_TOKEN }}
run: |
make package type=rpm app=apisix-runtime runtime_version="${VERSION}" \
image_base=rockylinux image_tag=${{ matrix.dist.image_tag }} \
Expand Down
14 changes: 13 additions & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,16 @@ endif
# Set arch to linux/amd64 if it's not defined
arch ?= linux/amd64

# api7/ngx_http_ffi_client is private, so the runtime build reads a token from
# a BuildKit secret rather than a build arg, which would land in image history.
# Without the token the runtime is built without that module.
NGX_HTTP_FFI_CLIENT_TOKEN ?=
ifneq ($(NGX_HTTP_FFI_CLIENT_TOKEN),)
ffi_client_secret=--secret id=ngx_http_ffi_client_token,env=NGX_HTTP_FFI_CLIENT_TOKEN
else
ffi_client_secret=
endif

# Detect the CPU architecture
CPU_ARCH := $(shell uname -m)
# Map the architecture to Docker platform
Expand Down Expand Up @@ -94,14 +104,15 @@ endif
### $(4) is code path
ifneq ($(buildx), True)
define build_runtime
docker build -t apache/$(1)-$(3):$(runtime_version) \
DOCKER_BUILDKIT=1 docker build -t apache/$(1)-$(3):$(runtime_version) \
--build-arg checkout_v=$(checkout) \
--build-arg VERSION=$(version) \
--build-arg RUNTIME_VERSION=$(runtime_version) \
--build-arg IMAGE_BASE=$(image_base) \
--build-arg IMAGE_TAG=$(image_tag) \
--build-arg BUILD_LATEST=$(build_latest) \
--build-arg CODE_PATH=$(4) \
$(ffi_client_secret) \
--platform $(arch) \
-f ./dockerfiles/Dockerfile.$(2).$(3) .
endef
Expand All @@ -115,6 +126,7 @@ define build_runtime
--build-arg IMAGE_TAG=$(image_tag) \
--build-arg BUILD_LATEST=$(build_latest) \
--build-arg CODE_PATH=$(4) \
$(ffi_client_secret) \
--load \
--cache-from=$(cache_from) \
--cache-to=$(cache_to) \
Expand Down
18 changes: 18 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
| image_base | False | the environment for packaging, if type is `rpm` the default image_base is `centos`, if type is `deb` the default image_base is `ubuntu` | image_base=centos |
| image_tag | False | the environment for packaging, it's value can be `16.04\|18.04\|20.04\|6\|7\|8`, if type is `rpm` the default image_tag is `7`, if type is `deb` the default image_tag is `20.04` | image_tag=7 |
| buildx | False | if `True`, use buildx to build docker images, which may speed up GitHub Actions | buildx=True |
| NGX_HTTP_FFI_CLIENT_TOKEN | False | environment variable holding a token that can read `api7/ngx_http_ffi_client`. It is passed to the runtime build as a BuildKit secret and never as a build arg. Without it the runtime is built without that module | NGX_HTTP_FFI_CLIENT_TOKEN=ghp_xxx make package ... |

## Example

Expand Down Expand Up @@ -106,6 +107,23 @@ ls output/
apisix-runtime_1.0.0-0~ubuntu20.04_amd64.deb
```

### ngx_http_ffi_client

`ngx_http_ffi_client` is the C HTTP client the AI plugins use for outbound LLM
requests. `api7/ngx_http_ffi_client` is a private repository, so the runtime
build fetches it only when `NGX_HTTP_FFI_CLIENT_TOKEN` is set, and otherwise
prints a warning and builds the runtime without it. `ai-proxy` falls back to
`lua-resty-http` on a runtime that does not carry the module, so both runtimes
work; only the outbound CPU cost differs.

```sh
NGX_HTTP_FFI_CLIENT_TOKEN=<token> \
make package type=deb app=apisix-runtime version=1.0.0
```

The commit is pinned by `ngx_http_ffi_client_ver` in `build-apisix-runtime.sh`,
since the repository carries no tags yet.

## Details

- `Makefile` the entrance of the packager
Expand Down
58 changes: 58 additions & 0 deletions build-apisix-runtime.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,24 @@ fi
wasm_nginx_module_ver="0.7.0"
lua_var_nginx_module_ver="v0.5.3"
lua_resty_events_ver="0.2.0"
# api7/ngx_http_ffi_client is still a private repository and carries no tags,
# so it is pinned by commit and fetched with a token. A build without the token
# leaves the module out and is otherwise unchanged.
ngx_http_ffi_client_ver=${ngx_http_ffi_client_ver:-"f13fcfa4e923ad82844bf49d9d3b3d283371ef66"}
if [[ ! "$ngx_http_ffi_client_ver" =~ ^[A-Za-z0-9._/-]+$ ]]; then
echo "ERROR: invalid ngx_http_ffi_client_ver: $ngx_http_ffi_client_ver" >&2
exit 1
fi
Comment on lines +42 to +46

@coderabbitai coderabbitai Bot Aug 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Find every entry point that can supply ngx_http_ffi_client_ver.
rg -n -C 3 '\bngx_http_ffi_client_ver\b|build_apisix_runtime|build-apisix-runtime\.sh' \
  --glob '*.sh' --glob 'Dockerfile*' --glob 'Makefile' .

# Confirm whether any build path forwards this variable into the build-script environment.
rg -n -C 3 'NGX_HTTP_FFI_CLIENT_TOKEN|ngx_http_ffi_client_ver|--build-arg|--secret' \
  Makefile dockerfiles .github build-apisix-runtime.sh

Repository: api7/apisix-build-tools

Length of output: 21159

Dependency Pin Bypass (CWE-829): Inclusion of Functionality from Untrusted Control Sphere

Enforce the fixed ngx_http_ffi_client commit.

ngx_http_ffi_client_ver accepts refs such as main. When the token is set, git fetch retrieves that ref and compiles it into the runtime. Require the exact approved commit f13fcfa4e923ad82844bf49d9d3b3d283371ef66, verify the fetched commit, and keep README.md consistent.

📍 Affects 2 files
  • build-apisix-runtime.sh#L42-L46 (this comment)
  • build-apisix-runtime.sh#L147-L172
  • README.md#L124-L125
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@build-apisix-runtime.sh` around lines 42 - 46, Enforce the approved commit in
ngx_http_ffi_client_ver instead of allowing arbitrary refs: validate that it
equals f13fcfa4e923ad82844bf49d9d3b3d283371ef66, then verify the commit fetched
by the ngx_http_ffi_client build flow before compiling it. Update
build-apisix-runtime.sh at lines 42-46 and 147-172, and keep README.md lines
124-125 consistent with the fixed commit.

# the trace stays off around the token, and only the derived yes/no reaches it
set +x
NGX_HTTP_FFI_CLIENT_TOKEN=${NGX_HTTP_FFI_CLIENT_TOKEN:-}
if [ -n "$NGX_HTTP_FFI_CLIENT_TOKEN" ]; then
ngx_http_ffi_client_have_token="yes"
else
ngx_http_ffi_client_have_token="no"
fi
Comment on lines +49 to +54

@coderabbitai coderabbitai Bot Aug 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the token lifecycle without executing the build.
rg -n -C 8 \
  'NGX_HTTP_FFI_CLIENT_TOKEN|unset|env -u|git|Authorization|http\.extraHeader|https?://' \
  build-apisix-runtime.sh

Repository: api7/apisix-build-tools

Length of output: 5890

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the exact credentialed fetch and all subsequent child-process launches.
sed -n '1,12p;145,190p;213,245p' build-apisix-runtime.sh

# Confirm Bash keeps an imported variable exported after the assignment used by the script.
env NGX_HTTP_FFI_CLIENT_TOKEN='probe-token' bash -c '
  NGX_HTTP_FFI_CLIENT_TOKEN=${NGX_HTTP_FFI_CLIENT_TOKEN:-}
  printf "parent-exported=%s\n" "$(export -p | grep -c "NGX_HTTP_FFI_CLIENT_TOKEN")"
  printf "child-value=%s\n" "$(env | sed -n "s/^NGX_HTTP_FFI_CLIENT_TOKEN=//p")"
'

Repository: api7/apisix-build-tools

Length of output: 3714

Sensitive Data Exposure (CWE-526)

Reachability: External

Clear NGX_HTTP_FFI_CLIENT_TOKEN after the authenticated fetch. The variable remains exported and is inherited by later commands such as ./patch.sh and ./configure. Add unset NGX_HTTP_FFI_CLIENT_TOKEN immediately after the fetch block.

🧰 Tools
🪛 ast-grep (0.45.0)

[warning] 50-50: A credential-bearing variable (e.g. PASSWORD, PASSWD, SECRET, TOKEN, API_KEY) is assigned a hardcoded string literal. Secrets committed to a script are exposed in source control, process listings, and shell history, and cannot be rotated without a code change. Read the value from a secrets manager or an injected environment variable at runtime instead (e.g. PASSWORD="${DB_PASSWORD:?must be set}"), and never commit the literal.
Context: ngx_http_ffi_client_have_token="yes"
Note: [CWE-798] Use of Hard-coded Credentials.

(hardcoded-password-assignment-bash)

[warning] 52-52: A credential-bearing variable (e.g. PASSWORD, PASSWD, SECRET, TOKEN, API_KEY) is assigned a hardcoded string literal. Secrets committed to a script are exposed in source control, process listings, and shell history, and cannot be rotated without a code change. Read the value from a secrets manager or an injected environment variable at runtime instead (e.g. PASSWORD="${DB_PASSWORD:?must be set}"), and never commit the literal.
Context: ngx_http_ffi_client_have_token="no"
Note: [CWE-798] Use of Hard-coded Credentials.

(hardcoded-password-assignment-bash)

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@build-apisix-runtime.sh` around lines 49 - 54, Unset
NGX_HTTP_FFI_CLIENT_TOKEN immediately after the authenticated fetch block
completes, before subsequent commands such as patch.sh or configure run; leave
the existing token-detection logic unchanged.

set -x
ngx_http_ffi_client_dir="ngx_http_ffi_client-${ngx_http_ffi_client_ver}"


install_openssl_3(){
Expand Down Expand Up @@ -134,6 +152,33 @@ else
lua-var-nginx-module-${lua_var_nginx_module_ver}
fi

if [ "$repo" == ngx_http_ffi_client ]; then
cp -r "$prev_workdir" "./$ngx_http_ffi_client_dir"
elif [ "$ngx_http_ffi_client_have_token" == "yes" ]; then
# A private repository pinned by commit, so fetch rather than clone -b.
# The token stays off the trace and out of the repository's git config.
mkdir "$ngx_http_ffi_client_dir"
(
set +x
cd "$ngx_http_ffi_client_dir" || exit 1
git init -q
git -c "http.extraheader=Authorization: Basic $(printf 'x-access-token:%s' \
"$NGX_HTTP_FFI_CLIENT_TOKEN" | base64 | tr -d '\n')" \
fetch -q --depth=1 \
https://github.com/api7/ngx_http_ffi_client.git "$ngx_http_ffi_client_ver"
git checkout -q FETCH_HEAD
)
else
echo "WARNING: NGX_HTTP_FFI_CLIENT_TOKEN is not set, building apisix-runtime" \
"without ngx_http_ffi_client. ai-proxy falls back to lua-resty-http" \
"on such a runtime." >&2
fi

ngx_http_ffi_client_configure_arg=""
if [ -d "$ngx_http_ffi_client_dir" ]; then
ngx_http_ffi_client_configure_arg="--add-module=../$ngx_http_ffi_client_dir"
fi

cd ngx_multi_upstream_module-${ngx_multi_upstream_module_ver} || exit 1
./patch.sh ../openresty-${OPENRESTY_VERSION}
cd ..
Expand Down Expand Up @@ -165,6 +210,11 @@ else
fi


# ngx_http_ffi_client compiles against lua-nginx-module's public API, which it
# reaches through the bundled copy rather than a separate checkout.
ngx_lua_bundle_dir=$(find bundle -maxdepth 1 -type d -name 'ngx_lua-*' | head -n 1)
export NGX_HTTP_LUA_MODULE_DIR="$PWD/$ngx_lua_bundle_dir"

./configure --prefix="$OR_PREFIX" \
--with-cc-opt="-DAPISIX_RUNTIME_VER=$runtime_version $cc_opt" \
--with-ld-opt="-Wl,-rpath,$OR_PREFIX/wasmtime-c-api/lib $ld_opt" \
Expand All @@ -177,6 +227,7 @@ fi
--add-module=../wasm-nginx-module-${wasm_nginx_module_ver} \
--add-module=../lua-var-nginx-module-${lua_var_nginx_module_ver} \
--add-module=../lua-resty-events-${lua_resty_events_ver} \
$ngx_http_ffi_client_configure_arg \
--with-poll_module \
--with-pcre-jit \
--without-http_rds_json_module \
Expand Down Expand Up @@ -220,6 +271,13 @@ sudo install -d "$OR_PREFIX"/lualib/resty/events/compat/
sudo install -m 644 lualib/resty/events/compat/*.lua "$OR_PREFIX"/lualib/resty/events/compat/
cd ..

if [ -d "$ngx_http_ffi_client_dir" ]; then
# the C module needs its FFI bindings on the runtime's lua_package_path
sudo install -d "$OR_PREFIX"/lualib/resty/
sudo install -m 644 "$ngx_http_ffi_client_dir"/lib/resty/ngx_http_ffi_client.lua \
"$OR_PREFIX"/lualib/resty/
fi

cd "apisix-nginx-module-${apisix_nginx_module_ver}" || exit 1
sudo OPENRESTY_PREFIX="$OR_PREFIX" make install
cd ..
Expand Down
7 changes: 6 additions & 1 deletion dockerfiles/Dockerfile.apisix-runtime.deb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# syntax=docker/dockerfile:1
ARG IMAGE_BASE="debian"
ARG IMAGE_TAG="bullseye-slim"

Expand All @@ -20,7 +21,11 @@ ENV build_latest=${BUILD_LATEST:-}

COPY ${CODE_PATH} ./

RUN mv ./utils/build-common.sh ./utils/determine-dist.sh ./ \
# the secret is optional: without it the runtime is built without
# ngx_http_ffi_client
RUN --mount=type=secret,id=ngx_http_ffi_client_token \
export NGX_HTTP_FFI_CLIENT_TOKEN="$(cat /run/secrets/ngx_http_ffi_client_token 2>/dev/null || true)" \
&& mv ./utils/build-common.sh ./utils/determine-dist.sh ./ \
&& ./build-common.sh build_apisix_runtime_deb ${build_latest} \
# determine dist and write it into /tmp/dist file
&& ./determine-dist.sh
Expand Down
7 changes: 6 additions & 1 deletion dockerfiles/Dockerfile.apisix-runtime.rpm
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# syntax=docker/dockerfile:1
ARG IMAGE_BASE="registry.access.redhat.com/ubi9/ubi"
ARG IMAGE_TAG="9.6"

Expand All @@ -18,7 +19,11 @@ ENV runtime_version=${RUNTIME_VERSION}

COPY ${CODE_PATH} ./

RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y \
# the secret is optional: without it the runtime is built without
# ngx_http_ffi_client
RUN --mount=type=secret,id=ngx_http_ffi_client_token \
export NGX_HTTP_FFI_CLIENT_TOKEN="$(cat /run/secrets/ngx_http_ffi_client_token 2>/dev/null || true)" \
&& curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y \
&& source "$HOME/.cargo/env" \
&& rustup install 1.69 \
&& rustup default 1.69 \
Expand Down
Loading