Optional retry policy for transient connection failures (#158)

[bertysentry]

Closes #158.

What

A new opt-in, connection-scoped builder setting:

WinRMClient client = WinRMClient.builder("server01")
    .credentials("ACME\admin", password)
    .retries(2, Duration.ofSeconds(5))   // up to 2 retries, pausing 5 s before each
    .build();

Default stays no retry — current behavior unchanged.

Design decisions (from the issue's open points)

• What is retryable: the retry loop lives in WsmanClient.send() and wraps only the connect-and-authenticate phase — TCP connect, DNS resolution, the TLS handshake (forced in ensureConnected()), and the authentication handshake round trips. The transport now connects explicitly before authenticating (HttpTransport.connect()), so every "could not reach the endpoint" failure surfaces in that phase, where the operation's request provably never reached the server. Anything after the operation's own POST started is never retried, preserving at-most-once execution for non-idempotent commands. Credential rejections (WinRMAuthenticationException) and WSMan faults are never retried either.
• WQL gets no broader surface, even though queries are idempotent: re-issuing a WS-Enumeration Pull whose response was lost could skip or duplicate rows, because the server-side enumeration context advances.
• Timeout interaction: blocking operations run under their wall-clock deadline (Utils.execute cancels the worker with an interrupt), so a retry pause is cut short and the operation reports the documented WinRMTimeoutException. A deadline-bounded poll (pollChunk) skips a retry whose pause no longer fits its hard budget (HttpTransport.remainingPollBudgetMillis()).
• Scope: the policy applies to every round trip of every operation on the client, including transparent reconnections in the middle of an operation (e.g. after the server dropped an idle keep-alive connection). close()'s best-effort cleanup never retries.

Tests

WsmanRetryTest exercises the policy end to end against the in-process FakeWsmanServer (extended with dropNextConnections(n) and enqueueDrop()):

• a connection dropped during the handshake is retried and the operation succeeds, with the operation request sent exactly once;
• without a policy the same failure surfaces immediately;
• retries are exhausted against an unreachable endpoint (pauses proven by elapsed time);
• the policy plumbs through the fluent builder;
• a Command whose connection dropped after the request reached the server is not re-sent;
• a credential rejection is not retried;
• the wall-clock deadline cuts the retry loop short.

Plus builder validation tests, and documentation in timeouts-and-errors.md (including the winrm4j failureRetryPolicy equivalence for migrating users).

🤖 Generated with Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8a742d1ac1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Recheck closure after each retry delay

When another thread calls close() while an operation is sleeping here, close() sets closed and returns, but it does not interrupt this worker. Once the delay ends, continue re-enters send() without passing through request()'s closure check, so the worker can reconnect, authenticate, and send the original SOAP request—including a non-idempotent command—after the client has closed. Check closed again before every retry attempt so closing during a pause cannot revive the transport or execute delayed side effects.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Preserve retryable I/O failures from authentication fallback

With an ordered HTTPS authentication list such as Kerberos then NTLM, auth is a FallbackAuthScheme, whose authenticate() catches each candidate's exception and ultimately wraps the last failure in IllegalStateException. Consequently, if Kerberos is unavailable and the NTLM handshake then encounters a transient IOException, this catch never sees it and the configured retry policy is skipped. Preserve or inspect the retryable transport failure after fallback exhaustion so .retries(...) also works with ordered authentication schemes.

Useful? React with 👍 / 👎.

[bertysentry]

Both findings addressed in cbea78a:

• P1 (recheck closure after the retry pause): the retry now rechecks closed after Thread.sleep(...) and rethrows the original failure, so a close() that lands during a pause can never revive the hard-closed transport nor send the delayed (possibly non-idempotent) operation. Regression test: closeDuringARetryPauseAbortsInsteadOfRevivingTheConnection asserts that zero requests reach the server after close().
• P2 (fallback-wrapped I/O failures): retryability is now classified by walking the cause chain for an IOException (isConnectionFailure), so the IllegalStateException("All requested authentication schemes failed", cause) thrown by FallbackAuthScheme is retried when its cause was a transport error, while protocol violations, credential rejections, and interrupts still surface immediately. Regression test: retriesATransportFailureWrappedByAnOrderedAuthenticationFallback.

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: cbea78a027

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".