Embedded `HEALTHCHECK` doesn't work with Podman

[openscript]

Describe the bug

Running the image on Docker inside a docker-compose.yml works well. The health is reported correctly. When I use the same image on podman with podman compose no health is reported.

I'm not sure, but maybe Podman parses the HEALTHCHECK wrong?

               {
                    "created": "2025-08-11T18:59:46.801634342Z",
                    "created_by": "HEALTHCHECK &{[\"CMD\" \"/watchtower\" \"--health-check\"] \"0s\" \"0s\" \"0s\" \"0s\" '\\x00'}",
                    "comment": "buildkit.dockerfile.v0",
                    "empty_layer": true
               },

I've found this issue regarding that: podman-container-tools/podman#18904

Unfortunately I wasn't able to come up with a workaround. I've tried to formulate my own HEALTHCHECK in my compose file as such:

    healthcheck:
      test: ["/watchtower", "--health-check"]
      start_period: 10s
      interval: 10s
      timeout: 5s
      retries: 3

This will cause Podman not to start any services from my compose file. I suspect that it tries to find a shell inside the container to run the command? Any other idea how I could check the health, without wrapping the image?

Steps to reproduce

1. Set up a podman with podman compose environment
2. Create a docker-compose.yml:

   name: "example"
   
   services:
     watchtower:
       image: docker.io/nickfedor/watchtower:1.11.7
       restart: unless-stopped
       volumes:
         - "$XDG_RUNTIME_DIR/podman/podman.sock:/var/run/docker.sock:ro"
       command:
         - "--debug"

3. Run podman compose up
4. Check for health report with podman ps

Expected behavior

Health is reported correctly.

Screenshots

No response

Environment

• Platform & Architecture: Linux 4.18.0-553.77.1.el8_10.x86_64 #1 SMP Fri Sep 19 03:36:32 EDT 2025 x86_64 x86_64 x86_64 GNU/Linux
• Docker Version: podman version 4.9.4-rhel

Your logs

Not applicable

Additional context

No response

[nicholas-fedor]

Thanks for reporting this issue.

I'll have to dig into this to determine a suitable solution. After doing some initial research, this appears to be a long-standing compatibility issue between Docker and Podman that may be related to how the Watchtower image is built.

If you manage to get it working or someone else wants to pick this up, then I'm open to suggestions / PR's.

[nicholas-fedor]

I moved this from the Bug Backlog to the Feature Backlog, as resolving this is more closely aligned with bringing improved Podman compatibility.

[openscript]

Thanks for your fast reply!

I think it's a Podman problem, that isn't really related to watchtower.

How would you implement your own health check in a docker-compose.yml file? Would it be viable to add a minimal shell to the image?

[teizz]

I've built my own container to test an issue with Podman (specifically with v4.3.1 which comes with Debian 12) where a container without health check still had the Health attribute in the Status field of the container. It even has a Status string, but it's empty.

According to the Docker documentation this is allowed to be one of starting, healthy, unhealthy, or none. The last one is for when no health check is configured, but my version of Podman somehow reports an empty string.

[
     {
          "State": {
               "Health": {
                    "Status": "",
                    "FailingStreak": 0,
                    "Log": null
               }
          }
    }
]

On topic for this particular issue though, when building the image with v5.7.1 on a buildhost, Podman indeed complains about the HEALTHCHECK argument in the Dockerfile when doing a plain podman image build -t <tag> . and says it wants the --format parameter.

Building it with podman image build --format docker -t <tag> . works fine, and running the image on the Debian 12 host with Podman v4.3.1 does not give any issues.

[teizz]

As an off-topic aside, I also had to patch out the Ulimits from the hostConfig since Watchtower fails to create a new container after removing the first one on this version of Podman. I think it's a known error which newer version of Podman don't have issues with:

"Error response from daemon: fill out specgen: ulimit option \"RLIMIT_NOFILE=1048576:1048576\" requires name=SOFT:HARD, failed to be parsed: invalid ulimit type: RLIMIT_NOFILE"

These are the two patches I did to get it working on my older version of Podman:

diff --git a/pkg/container/client.go b/pkg/container/client.go
index ead57d6b..5b3a54fc 100644
--- a/pkg/container/client.go
+++ b/pkg/container/client.go
@@ -863,6 +863,12 @@ func (c client) WaitForContainerHealthy(
                        }
 
                        status := inspect.State.Health.Status
+
+                       if status == "none" || len(status) == 0 {
+                               clog.Debug("No health check configured for container, proceeding")
+                               return nil
+                       }
+
                        clog.WithField("health_status", status).Debug("Checked container health status")
 
                        if status == "healthy" {

diff --git a/pkg/container/container_target.go b/pkg/container/container_target.go
index dc2baecf..9761cd6b 100644
--- a/pkg/container/container_target.go
+++ b/pkg/container/container_target.go
@@ -54,6 +54,7 @@ func StartTargetContainer(
        // Extract configuration from the source container.
        config := sourceContainer.GetCreateConfig()
        hostConfig := sourceContainer.GetCreateHostConfig()
+       hostConfig.Ulimits = nil
 
        // Set MemorySwappiness to nil for Podman compatibility if flag is enabled.
        if disableMemorySwappiness {