feat: Allow for external services and declarative port binding by scottpledger · Pull Request #69 · hermeticbuild/rules_itest

scottpledger · 2026-06-16T17:24:45Z

Allows services to be described in more detail than just as a local port, so a test suite can run against either locally-managed services or production-like instances (selected with Bazel select()).

Important changes/features

itest_port — declare a port as a first-class target. The binding info (value + host) is supplied by whichever service binds it, and a port can only be bound once.
itest_external_service — point at a fixed FQDN instead of a locally-spawned binary. It exposes the same provider as itest_service, so it's a drop-in replacement via select(). The manager never starts/stops it (optional health check only).
Hostname/domain support — itest_service gains domain (default 127.0.0.1) and a ports attribute.
New env vars ITEST_PORTS_MAP and ITEST_SERVICES_MAP (string-encoded JSON) describe every port/service with {origin, domain, port}, injected into the test and all child services.
Control API GET /v0/ports and GET /v0/services list this for all services at once.

I also made this fully backward-compatible: existing macros auto-create the needed ports/aliases; port(), ASSIGNED_PORTS, GET_ASSIGNED_PORT_BIN, /v0/port, and --//pkg:svc.port=N overrides all still work. The underlying rules are now exported for extension.

Example

load("@bazel_skylib//rules:common_settings.bzl", "bool_flag")
load(
    "@rules_itest//:itest.bzl",
    "itest_port",
    "itest_service",
    "itest_external_service",
    "service_test",
    "port_ref",
)
# Declare a port as a target.
itest_port(name = "db_port")
# A locally-managed service binds it.
itest_service(
    name = "db",
    exe = "//db:server",
    ports = {":db_port": "sql"},
    args = ["-port", port_ref(":db_port")],
    http_health_check_address = "http://127.0.0.1:" + port_ref(":db_port"),
)
# A production-like instance exposes the same port at a fixed FQDN.
itest_external_service(
    name = "db_external",
    domain = "db.staging.mycompany.com",
    ports = {":db_port": "sql"},
    port_numbers = {"sql": "5432"},
)
# A flag to choose which implementation to test against.
bool_flag(
    name = "use_external_db",
    build_setting_default = False,
)
config_setting(
    name = "external",
    flag_values = {":use_external_db": "True"},
)
# Swap between them with the flag:
#   bazel test //myapp:db_test                              # local service
#   bazel test --//myapp:use_external_db //myapp:db_test       # production-like instance
service_test(
    name = "db_test",
    test = ":_db_test",
    services = select({
        ":external": [":db_external"],
        "//conditions:default": [":db"],
    }),
)

The test can then read the connection info from either:

ITEST_PORTS_MAP (or /v0/ports) (keyed by port target label, plus any aliases):

{
  "@@//myapp:db_port": { "origin": "127.0.0.1:54321", "domain": "127.0.0.1", "port": "54321" }
}

Or, if run with --//myapp:use_external_db, the same keys instead resolve to the production-like instance:

{
  "@@//myapp:db_port": { "origin": "db.staging.mycompany.com:5432", "domain": "db.staging.mycompany.com", "port": "5432" }
}

ITEST_SERVICES_MAP (or /v0/services), e.g.:

{
  "@@//myapp:db": { "sql": { "origin": "127.0.0.1:54321", "domain": "127.0.0.1", "port": "54321" } }
}

Note: in the example above, :db_external and :db both bind :db_port — that's valid because select() resolves to only one of them at a time in the service_test rule, so the port is still only bound once.

Testing

Added tests/ports, which covers internal ports, an external service, the select() swap, the new maps/endpoints, and ensuring ports & aliases are only ever bound once. The full test suite passes (48/48), and the examples build cleanly.

Caveats

This code does not automatically cache-bust when using remote services. This is intentional at present, though we could probably add this in a future implementation.
This code does not allow for specifying subpaths or other similar service/port metadata. For example, if a local server serves an app from root, but a remote one serves it from /my/app or something. I don't think this would be too hard to add in a follow-up, though.

Solves: #70

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3d42ba6037

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-06-16T17:29:31Z

+		if service.Type == "external_service" {
+			return service.WaitUntilHealthy(ctx)


Honor health_check_timeout for external services

When an itest_external_service has an HTTP or command health check that keeps failing and health_check_timeout is set, this early return calls WaitUntilHealthy before the timeout-wrapping block below runs. The external-service health loop therefore polls until the outer Bazel/test timeout instead of failing after the configured service timeout, which makes unreachable external dependencies hang much longer than requested.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-06-16T17:29:31Z

        "so_reuseport_aware": ctx.attr.so_reuseport_aware,
        "deferred": ctx.attr.deferred,
+        "domain": ctx.attr.domain,
+        "port_bindings": _compute_port_bindings(ctx),


Allow SO_REUSEPORT with declarative ports

When a service uses only the new ports = {":p": "name"} binding, _compute_port_bindings(ctx) still creates an autoassigned socket, but the validation above only allows so_reuseport_aware with legacy autoassign_port or named_ports. As a result, users adopting first-class itest_port targets cannot enable the collision-avoidance mode for those autoassigned ports and get an analysis failure even though the runtime path supports SoReuseportAware for all bindings.

Useful? React with 👍 / 👎.

dzbarsky · 2026-06-16T17:33:56Z

Thanks for sending this! I've definitely considered a first class port target in the past and I'm excited to look at what you've cooked up here. It might take me a bit to get to it but wanted to ack the PR

scottpledger · 2026-06-16T17:54:07Z

Thanks for sending this! I've definitely considered a first class port target in the past and I'm excited to look at what you've cooked up here. It might take me a bit to get to it but wanted to ack the PR

No problem! This is something I've been thinking about for a while, given how we use this library at my company. We currently just create multiple test targets - one for local, test, preview, and prod. However, this approach doesn't give us the flexibility to mix and match service locations (eg, a local web server with test remote APIs).

scottpledger · 2026-06-29T15:57:16Z

Hey, @dzbarsky! Any chance you will be able to look at this sometime soon? I now have an actual use-case for it and I'd love to get feedback on this approach before I start adopting it.

dzbarsky · 2026-06-29T18:45:54Z

+	// legacy string->port map (for substitution / ASSIGNED_PORTS) and the rich maps.
+	register := func(serviceLabel, portName, domain, portStr, target string, aliases []string) {
+		info := svclib.BindingInfo{
+			Origin: domain + ":" + portStr,


net.JoinHostPort

alternately, why not construct it on the fly when needed instead of storing twice?

Done — Origin is built with net.JoinHostPort(hostname, portStr) now. (e0f453c)

origin is part of the documented public schema of ITEST_PORTS_MAP/ITEST_SERVICES_MAP ({origin, hostname, port}) and backs the $${…::origin} substitution token, so consumers get it without re-joining host+port (or re-implementing IPv6 bracketing). Since it has to be in the serialized output regardless, I kept it materialized on BindingInfo rather than reconstructing it at each use site — but happy to compute-on-serialize instead if you'd rather not carry the field.

dzbarsky · 2026-06-29T19:03:40Z

Hey, @dzbarsky! Any chance you will be able to look at this sometime soon? I now have an actual use-case for it and I'd love to get feedback on this approach before I start adopting it.

Apologies for the delay, I've left some feedbacks. At a high level, I think the general idea makes sense, hopefully we can streamline the API a bit, but I don't think it would require major rework from you to migrate from how it's setup now to whatever the final state ends up, I expect it would be fairly close. I've also tagged in @darkrift to help review this as he is using this ruleset more actively than I am these days :)

darkrift · 2026-07-01T17:05:05Z

I very much like the global idea, while I don't have a use case for this yet, it's something that makes alot of sense. I'm currently a bit busy but will try to get my eyes on this by the end of the week

darkrift · 2026-07-29T14:29:14Z

+    return _finalize_service(ctx, service, transitive_runfiles = _services_runfiles(ctx, "deps") + extra_exe_runfiles)
+
+_itest_external_service_attrs = {
+    "domain": attr.string(


This should be renamed to hostname which is more common than domain

Done — renamed domain → hostname on both itest_service and itest_external_service, and threaded it through the Go ServiceSpec/BindingInfo fields, the ITEST_PORTS_MAP/ITEST_SERVICES_MAP key, the port_hostname helper + $${…::hostname} token, and the tests/docs. Bonus: hostname matches net/url.URL.Hostname() semantics (host-or-IP minus the port), where domain was misleading. (e0f453c)

darkrift · 2026-07-29T14:30:34Z

+    "deferred": attr.bool(
+        doc = "If set, the external service will not be health-checked on boot up.",
+    ),
+    "http_health_check_address": attr.string(


It should have the same command line for health check if the service uses a different protocol to validate it's health

Agreed — for non-HTTP protocols a command health check is the right tool. itest_external_service already exposes health_check + health_check_args (mirroring itest_service), and I've now added an example that exercises it (see the cmd_health_db reply below). (e0f453c)

darkrift · 2026-07-29T14:32:11Z

+func (s *ServiceInstance) PollUntilHealthy(ctx context.Context) error {
+	sleepDuration, err := time.ParseDuration(s.HealthCheckInterval)
+	if err != nil {
+		sleepDuration = 200 * time.Millisecond


This should be a default value in the rule instead

Done — the health_check_interval default (200ms) lives in the rule and is validated in Starlark, so I dropped the hardcoded fallback in PollUntilHealthy. It now returns an error on a malformed value instead of silently defaulting. (e0f453c)

darkrift · 2026-07-29T14:32:54Z

 	Deferred                bool              `json:"deferred"`
+	// Domain is the host that this service's ports are reachable on. Internal
+	// services default to "127.0.0.1"; external services set it to their FQDN.
+	Domain string `json:"domain"`


rename to Hostname

Done — renamed the field to Hostname (json hostname) as part of the same domain→hostname rename. (e0f453c)

darkrift · 2026-07-29T14:34:12Z

+        bindings.append(_port_binding(label, "", [], str(ctx.attr.port[BuildSettingInfo].value)))
+
+    for port_flag, name in ctx.attr.named_ports.items():
+        bindings.append(_port_binding(label + "." + name, name, [], str(port_flag[BuildSettingInfo].value)))


use _to_relative_named_port instead of concatenating "."

Good call. _to_relative_named_port lives in //:itest.bzl and relies on native.package_relative_label, which is loading-phase only, so it can't run inside a rule implementation. I added an analysis-phase counterpart _named_port_target(label, name) in private/itest.bzl and use it there; both share the <label>.<name> convention (noted in a comment so they stay in sync). (e0f453c)

darkrift · 2026-07-29T14:35:23Z

+    name = "external_db",
+    domain = "db.test.invalid",
+    port_numbers = {"sql": "5432"},
+    ports = {":ext_db_port": "sql"},


This is an example of where a cmd health check would come useful to know if the database is available but doesn't have an http endpoint for health check

Added exactly this: a new cmd_health_db external service with no HTTP endpoint whose readiness is verified by a command health check — a small tcp_probe binary that dials the (locally-managed, to stay hermetic) DB port. It's covered by //ports:cmd_health_db_hygiene_test. (e0f453c)

darkrift · 2026-07-29T14:38:56Z

@scottpledger Sorry for the late review, I just did a pass, I agree with all of David's original review and added a few more to it.

jaqx0r · 2026-07-31T04:53:49Z

I very much like the global idea, while I don't have a use case for this yet, it's something that makes alot of sense. I'm currently a bit busy but will try to get my eyes on this by the end of the week

Glad to see the recent review -- we definitely have a need for this feature and I came to just check in to see if I could help push this along. Looking forward to seeing it landed!

scottpledger · 2026-08-14T14:37:30Z

Hey y'all, I'm so sorry for the delays — I've had several large issues hogging my resources. I'm hoping to get back to this next week.

…vices Co-authored-by: Cursor <cursoragent@cursor.com> # Conflicts: # cmd/svcinit/main.go # docs/itest.md

… checks - Rename `domain` -> `hostname` across the Starlark attrs, Go ServiceSpec/ BindingInfo fields, the ITEST_PORTS_MAP/ITEST_SERVICES_MAP key, the `port_domain`->`port_hostname` helper and `::domain`->`::hostname` token, tests, and docs. `hostname` matches `net/url.URL.Hostname()` semantics (host-or-IP minus the port), whereas `domain` was misleading. - Drop the hardcoded 200ms fallback in ServiceInstance.PollUntilHealthy and rely on the rule-provided (and validated) health_check_interval default. - Add a shared `_named_port_target` helper instead of ad-hoc `label + "." + name` concatenation when building named-port bindings. - Demonstrate command-based health checks for itest_external_service with a new tcp_probe example (a DB-style service with no HTTP endpoint), verified by a hygiene test. Co-authored-by: Cursor <cursoragent@cursor.com>

scottpledger · 2026-08-25T19:11:29Z

Sorry for the long delay on this — I've pushed an update (e0f453c) that brings the PR current and works through the latest review.

Upstream's port-reservation refactor (reserveReusablePort / reservedPorts / closeReservedPorts) conflicted with assignPorts. I reconciled it so we keep the rich PortsMap/ServicesMap model and adopt the reservation mechanism — assignPorts now returns (PortsMap, ServicesMap, map[string][]io.Closer, error) and the caller defer closeReservedPorts(...).

Let me know your thoughts. I should note that I used Claude for a fair amount of this since my Go knowledge is pretty sparse (the last time I did much Go code was in 2009 when it first came out 😬 ).

scottpledger · 2026-08-25T20:20:05Z

@dzbarsky @darkrift This is now ready for another pass whenever y'all have a cycle or two to spare. Sorry again for the delay!

…vices Co-authored-by: Cursor <cursoragent@cursor.com> # Conflicts: # go.sum # runner/runner.go

chatgpt-codex-connector Bot reviewed Jun 16, 2026

View reviewed changes

scottpledger force-pushed the feat/external-services branch from 3d42ba6 to 7337a01 Compare June 16, 2026 17:50

dzbarsky reviewed Jun 29, 2026

View reviewed changes

feat: Allow for external services and declarative port binding

c816346

scottpledger force-pushed the feat/external-services branch from 7337a01 to c816346 Compare July 8, 2026 16:43

yaroslavvoloshchuk-codaio mentioned this pull request Jul 19, 2026

Proposal: opt-in lazy shared services for independent test-suite members #74

Open

darkrift requested changes Jul 29, 2026

View reviewed changes

darkrift mentioned this pull request Aug 20, 2026

Add a hermetic local bb-portal stack using rules_itest buildbarn/bb-portal#308

Open

Scott Pledger and others added 2 commits August 25, 2026 12:58

Merge remote-tracking branch 'upstream/master' into feat/external-ser…

d1da773

…vices Co-authored-by: Cursor <cursoragent@cursor.com> # Conflicts: # cmd/svcinit/main.go # docs/itest.md

scottpledger requested a review from darkrift August 25, 2026 19:51

Merge remote-tracking branch 'upstream/master' into feat/external-ser…

2dd28bf

…vices Co-authored-by: Cursor <cursoragent@cursor.com> # Conflicts: # go.sum # runner/runner.go

darkrift approved these changes Sep 14, 2026

View reviewed changes

		if service.Type == "external_service" {
		return service.WaitUntilHealthy(ctx)

Conversation

scottpledger commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Important changes/features

Example

Testing

Caveats

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dzbarsky commented Jun 16, 2026

Uh oh!

scottpledger commented Jun 16, 2026

Uh oh!

scottpledger commented Jun 29, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dzbarsky commented Jun 29, 2026

Uh oh!

darkrift commented Jul 1, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

scottpledger Aug 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

darkrift commented Jul 29, 2026

Uh oh!

jaqx0r commented Jul 31, 2026

Uh oh!

scottpledger commented Aug 14, 2026

Uh oh!

scottpledger commented Aug 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

scottpledger commented Jun 16, 2026 •

edited

Loading

scottpledger Aug 25, 2026 •

edited

Loading

scottpledger commented Aug 25, 2026 •

edited

Loading