failed to open 2048 locks in /libpod_lock: numerical result out of range when switching architectures (x86_64<->aarch64)

[rmsc]

This is essentially the same issue as osbuild/bootc-image-builder#927, which never got fixed:

First build with x86_64 goes fine:

$ sudo podman run --rm -it --platform=linux/x86_64 --privileged --pull=newer  --security-opt label=type:unconfined_t  -v ./output:/output  -v /var/lib/containers/storage:/var/lib/containers/storage ghcr.io/osbuild/image-builder:latest build qcow2 --arch x86_64 --bootc-default-fs btrfs --bootc-ref quay.io/fedora/fedora-bootc:44 --output-name fedora-bootc-44
INFO[0000] Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled
INFO[0000] Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled
[/] Image building step
[5 / 5] Pipeline qcow2 [-------------------------------------------------------------------------------------------------------------------->] 100.00%
[2 / 2] Stage org.osbuild.qemu [------------------------------------------------------------------------------------------------------------>] 100.00%
Message: Finished pipeline qcow2
Image build successful: bootc-fedora-44-qcow2-x86_64/fedora-bootc-44.qcow2

Switching to aarch64 then fails:

$ sudo podman run --rm -it --platform=linux/aarch64 --privileged --pull=newer  --security-opt label=type:unconfined_t  -v ./output:/output  -v /var/lib/containers/storage:/var/lib/containers/storage ghcr.io/osbuild/image-builder:latest build qcow2 --arch aarch64 --bootc-default-fs btrfs --bootc-ref quay.io/fedora/fedora-bootc:44 --output-name fedora-bootc-44
Trying to pull ghcr.io/osbuild/image-builder:latest...
Getting image source signatures
Copying blob 4d5ccdf141c6 skipped: already exists
Copying blob f0016b0c686f skipped: already exists
Copying blob 343e72d56f9f skipped: already exists
Copying blob 961622004378 skipped: already exists
Copying blob 7efe3526e4d8 skipped: already exists
Copying config 10e201bdf5 done   |
Writing manifest to image destination
error: running quay.io/fedora/fedora-bootc:44 container failed: exit status 125
stderr:
Error: failed to open 2048 locks in /libpod_lock: numerical result out of range

This seemed related to #2486 (comment) but is not: testing with image-builder:latest didn't fix the issue.

As reported originally, nothing I could do fixed the situation except rebooting.

[supakeen]

Needs some investigation from my side, hope to get to it soon.