Skip to content

Security/version compatibility filter is not applied uniformly across discovery paths — Closes #293#304

Draft
conradbzura wants to merge 4 commits into
wool-labs:releasefrom
conradbzura:293-always-apply-compatibility-filters
Draft

Security/version compatibility filter is not applied uniformly across discovery paths — Closes #293#304
conradbzura wants to merge 4 commits into
wool-labs:releasefrom
conradbzura:293-always-apply-compatibility-filters

Conversation

@conradbzura

Copy link
Copy Markdown
Contributor

Summary

Every discovery source now passes one admission rule: the worker sentinel applies the combined security/version predicate to every worker-added and worker-updated event, regardless of how the proxy was constructed. Previously the predicate lived in the construction paths — the pool-URI arm baked it into its subscription filter and the static-workers arm filtered at construction — so user-supplied discovery subscribers and every WorkerPool subscription admitted workers unfiltered, deferring the failure to connect/handshake where it surfaced as transient-error and eviction noise.

This is a behavior change for callers relying on unfiltered custom discovery: an incompatible worker is never admitted, and quorum waits no longer count one. A proxy restored from a pickle re-derives the predicate from the restoring process's credential context and protocol version — the gate is deliberately not serialized, so admission always reflects the environment doing the admitting.

Closes #293

Proposed changes

Enforce the admission gate in the worker sentinel

Gate worker-added and worker-updated events on the combined security/version predicate before the lease check, so a rejected worker never consumes lease capacity and never wakes the quorum wait. Leave worker-dropped ungated — a drop is not an admission, and removing an unknown worker is a no-op. Reduce the pool-URI subscription filter to tag semantics only; the static-workers arm keeps its construction-time filter for the fail-fast quorum-versus-compatible-count validation, and the sentinel re-checks those events idempotently on restore. Document the behavior change on the discovery parameters of WorkerProxy and WorkerPool.

Adapt the worker test suite to the gate

Advertise the ambient protocol version from the mock worker factories and discovery fixtures so test doubles behave like real in-process workers, pin the local version explicitly in tests that feed the sentinel, and rewrite the pool-URI subscription-filter test to assert tag-only semantics. Remove test_property_workers_list_accurate: it asserted nothing and spent ten seconds of wall clock, and the fold-model property below supersedes it.

Pin the gate at every scope

Cover the gate at proxy scope (per-rule admission tests, quorum and lease interplay, serialization re-gating, Hypothesis properties over the version predicates and arbitrary event streams), at pool scope (durable-arm composition with a real proxy, tag-only subscription predicates, credentials forwarding), and end to end (real workers plus fabricated incompatible metadata published through the real local-discovery pipeline, and both security-posture crosses). The integration tests are standalone scenarios rather than pairwise dimensions: a mismatched security posture breaks the covering array's dispatch-success oracle, and the admitted-set oracle needs the active proxy, which the scenario builder does not surface.

Test cases

# Test Suite Given When Then Coverage Target
1 TestWorkerProxy A discovery stream with compatible, same-major-older, next-major, and prior-major workers The sentinel processes all events Only the compatible worker is admitted Version rule at admission
2 TestWorkerProxy A stream with secure and insecure workers and no credentials The sentinel processes all events Only the insecure worker is admitted Security rule, credentials absent
3 TestWorkerProxy A stream with secure and insecure workers and mTLS credentials The sentinel processes all events Only the secure worker is admitted Security rule, credentials present
4 TestWorkerProxy A stream with an unparseable-version worker and a compatible control The sentinel processes all events Only the control is admitted Unparseable worker version
5 TestWorkerProxy lease=1 and an incompatible worker preceding a compatible one The sentinel processes all events The compatible worker holds the slot Gate precedes the lease check
6 TestWorkerProxy An admitted worker and an incompatible update for the same uid The sentinel processes both events The update is ignored and no second connection is built Gated worker-updated
7 TestWorkerProxy An incompatible worker-added followed by its worker-dropped The sentinel processes both events The drop is a safe no-op and nothing is admitted Ungated worker-dropped
8 TestWorkerProxy A pool-URI proxy whose subscription surfaces a tag-matching incompatible worker and a compatible one The sentinel processes both events Only the compatible worker is admitted Gate on the pool-URI path
9 TestWorkerProxy A pool URI and extra tags The subscription filter is evaluated Tag intersection alone decides, including the extra-tag member Tag-only subscription filter
10 TestWorkerProxy quorum=1, a short timeout, and an all-incompatible stream The proxy is entered eagerly asyncio.TimeoutError propagates and nothing is admitted Gate never satisfies quorum
11 TestWorkerProxy A lazy proxy, an incompatible worker, and a gated compatible one Dispatch is awaited, then retried after the gate opens The first dispatch times out un-started and the retry dispatches Retryability after gate-caused timeout
12 TestWorkerProxy A proxy constructed inside a credential context and started outside it The sentinel processes secure and insecure workers Only the secure worker is admitted Credential capture at construction
13 TestWorkerProxy A local protocol version of "unknown" The sentinel processes valid and "unknown"-version workers Nothing is admitted Fail-closed local version
14 TestWorkerProxy quorum=2 and a stepped add/drop/add/add stream Entry runs while events release one at a time Entry completes only when two workers are live Quorum tracks the live count
15 TestWorkerProxy An un-started lazy proxy The workers property is read An empty list returns without error Pre-start workers access
16 TestWorkerProxy Three static workers, two compatible, quorum=2 The proxy is constructed and started Construction succeeds and both compatible workers are admitted Quorum equals the compatible count
17 TestWorkerProxy A static list of one insecure and one secure worker, no credentials, quorum=2 The proxy is constructed ValueError reports one of two compatible Security in the construction count
18 TestWorkerProxy An all-incompatible static list and quorum=None The proxy is constructed and started The pool starts empty without error Quorum-gated fail-fast
19 TestWorkerProxy A credentialed proxy over a mixed-security stream, pickled The payload loads outside any credential context and starts Only the insecure worker is admitted Credentials are not serialized
20 TestWorkerProxy An insecure proxy over a mixed-security stream, pickled The payload loads inside a credential context and starts after it exits Only the secure worker is admitted Gate binds at load time
21 TestWorkerProxy A proxy pickled under local 1.0.0 with 1.0.0 and 2.5.0 workers The payload loads under local 2.0.0 and starts Only the 2.5.0 worker is admitted Version gate rebuilt on restore
22 TestWorkerProxy A static-workers proxy pickled from an insecure context The payload loads inside a credential context and starts Nothing is admitted Carried events re-gated on restore
23 TestWorkerProxy A started static-workers proxy with its stream consumed, pickled The payload loads and starts in the same context Both workers are admitted again Carried stream replays on restore
24 Module level Arbitrary text biased with stringified versions parse_version is called None or a re-parseable Version returns, never an exception Parse tolerance
25 Module level Any generated PEP 440 version Compatibility is checked against itself True returns Reflexivity
26 Module level Two distinct same-major versions Compatibility is checked both ways Only the lower client passes Same-major ordering
27 Module level Two versions with differing majors Compatibility is checked both ways Both directions fail Major fencing
28 TestWorkerProxy A generated static list with mixed security and versions, and a generated quorum The proxy is constructed ValueError exactly when quorum exceeds the predicate-compatible count Constructor-gate consistency
29 TestWorkerProxy A generated add/update/drop stream over a small universe with a drawn local version and lease A non-lazy proxy consumes the stream The admitted set equals a fold-model oracle Gate and lease under arbitrary interleavings
30 TestWorkerPool A durable pool over a custom source yielding only a next-major worker The pool is entered eagerly asyncio.TimeoutError raises at entry Pool-level gate and quorum
31 TestWorkerPool A durable pool with lease=1 over an incompatible-then-compatible stream The pool is entered eagerly Only the compatible worker is admitted Pool wiring composes with the gate
32 TestWorkerPool A hybrid pool with a tag and a filter-capturing source The captured predicate is evaluated Tag intersection alone decides Tag-only spawn-arm subscriptions
33 TestWorkerPool A hybrid pool with no tags The captured predicate is evaluated Every worker matches Match-all no-tags branch
34 TestWorkerPool A durable pool constructed with credentials The pool is entered The proxy receives the same credentials instance Credentials forwarding
35 TestDiscoveryAdmissionGate A real worker and four fabricated incompatible records in one namespace A durable pool enters and dispatches five times Exactly the real worker is admitted throughout and every dispatch succeeds End-to-end chaff screening
36 TestSecurityPostureAdmission Real insecure and mTLS workers and an insecure pool The pool enters and dispatches Only the insecure worker is admitted Insecure pool cross
37 TestSecurityPostureAdmission The same workers and an mTLS pool The pool enters and dispatches Only the secure worker is admitted over mTLS Credentialed pool cross
38 TestHybridAdmissionGate A hybrid pool spawning one worker into a chaff-seeded namespace The pool enters and dispatches Only the spawned worker is admitted Hybrid path end to end
39 TestStaticWorkersAdmission A static list of one real worker plus the chaff records The proxy enters and dispatches Only the real worker is admitted and the dispatch returns Static path end to end

https://claude.ai/code/session_01Da7KQMtQgbgwqvPeL4Wfu3

The secure-match and same-major version floor were enforced only on
the pool-URI and static-workers construction paths; user-supplied
discovery subscribers and every WorkerPool subscription admitted
workers unfiltered, deferring failures to connect/handshake with
transient-error and eviction noise.

Move enforcement into the worker sentinel as a single admission rule:
worker-added and worker-updated events must satisfy the combined
security/version predicate regardless of construction path, checked
before the lease so a rejected worker never consumes capacity;
worker-dropped events stay ungated. Construction-path subscription
filters retain only their tag semantics, and the static-workers arm
keeps its fail-fast quorum-versus-compatible-count validation. The
predicate is deliberately not serialized: a restored proxy rebuilds
its gate from its own credential context and protocol version.

This is a behavior change for callers relying on unfiltered custom
discovery: incompatible workers are no longer admitted, and quorum
waits no longer count them.
Adapt existing pins to the gate: mock worker factories and discovery
fixtures advertise the ambient protocol version, sentinel-feeding
tests pin the local version explicitly, and the pool-URI subscription
filter test now asserts tag-only semantics with compatibility
deferred to admission.

Add gate coverage on the user-supplied discovery path (version,
security, unparseable-version, and lease interplay; ignored
incompatible updates; ungated drops), construction-path boundaries
for the static-workers quorum count, quorum-timeout and retry
behavior on all-incompatible streams, credential resolution captured
at construction, the fail-closed unparseable local version, restored
proxies re-gating carried events under the restoring context, and
Hypothesis properties for the version predicates plus a fold-model
oracle over arbitrary event streams.

Remove test_property_workers_list_accurate: it asserted nothing and
spent ten seconds of wall clock; the fold-model property supersedes
it.
Cover the durable arm with a real proxy: an all-incompatible custom
discovery source times out the eager quorum wait, and a mixed stream
admits only the compatible worker without the rejected one consuming
the single lease slot. Pin the tag-only subscription predicate on the
spawn arms, including the match-all no-tags branch, and the
credentials forwarding that aims the security half of the gate.
Standalone scenarios rather than pairwise dimensions: a mismatched
security posture would break the covering array's dispatch-success
oracle, and the admitted-set oracle needs the active proxy, which the
scenario builder does not surface. Publish fabricated incompatible
metadata through the real local discovery pipeline alongside real
workers and assert the admitted set is exactly the compatible workers
before and after dispatching, across the durable, hybrid, and
static-workers paths and both security-posture crosses.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Security/version compatibility filter is not applied uniformly across discovery paths

1 participant