0.7.0: remove :adapter_error default retry, narrow AdapterCaller rescue#13
0.7.0: remove :adapter_error default retry, narrow AdapterCaller rescue#13
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces the 0.7.0 breaking changes to retry behavior and adapter error handling to reduce redundant retries and avoid masking programmer/configuration errors.
Changes:
- Remove
:adapter_errorfromRetryPolicy::DEFAULT_RETRY_ON(now defaults to[:validation_failed, :parse_error]). - Narrow
AdapterCallerrescue fromStandardErrorto::RubyLLM::Error, allowing non-provider errors to propagate. - Update specs, changelog, and version/lockfile for the 0.7.0 release and new opt-in behavior.
Reviewed changes
Copilot reviewed 10 out of 11 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| spec/ruby_llm/contract/step/runner_spec.rb | Updates adapter failure fixture to raise RubyLLM::Error. |
| spec/ruby_llm/contract/step/retry_policy_spec.rb | Updates default retryable statuses expectation to exclude :adapter_error. |
| spec/ruby_llm/contract/step/retry_integration_spec.rb | Makes :adapter_error retry behavior explicit via retry_on :adapter_error and updates fixtures. |
| spec/ruby_llm/contract/edge_cases_spec.rb | Updates adapter failure fixture to raise RubyLLM::Error. |
| spec/ruby_llm/contract/adversarial_round10_spec.rb | Inverts default retryability expectation for :adapter_error and adds explicit opt-in test. |
| spec/integration/step_end_to_end_spec.rb | Updates adapter failure fixture to raise RubyLLM::Error. |
| lib/ruby_llm/contract/version.rb | Bumps gem version to 0.7.0. |
| lib/ruby_llm/contract/step/retry_policy.rb | Changes retry defaults and documents the breaking rationale. |
| lib/ruby_llm/contract/step/adapter_caller.rb | Narrows rescue to ::RubyLLM::Error and preserves :adapter_error mapping for provider errors. |
| Gemfile.lock | Updates lockfile entry for ruby_llm-contract to 0.7.0. |
| CHANGELOG.md | Adds 0.7.0 release notes and migration guidance. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Breaking (0.7.0): was `rescue StandardError`. Now only catches provider | ||
| # errors from ruby_llm (RubyLLM::Error hierarchy). Programmer errors | ||
| # (NoMethodError, ArgumentError, etc.) propagate instead of being silently | ||
| # coerced into :adapter_error and retried — bugs should be fixed, not retried. |
There was a problem hiding this comment.
The change to only rescue ::RubyLLM::Error is a breaking behavioral change, but there doesn’t appear to be a spec that asserts a non-RubyLLM::Error raised by an adapter (e.g., ArgumentError/NoMethodError) now propagates instead of being coerced into :adapter_error. Adding an explicit regression spec for this new contract would help prevent accidental broadening of the rescue in the future.
… rescue Breaking changes 1. DEFAULT_RETRY_ON = [:validation_failed, :parse_error]. :adapter_error removed from the default. ruby_llm's Faraday middleware already retries transport errors (RateLimitError, ServerError, ServiceUnavailableError, OverloadedError, timeouts) with backoff, so the old default re-ran the same model on errors transport had already retried — retry x retry with no change in context. Opt in explicitly with retry_on :adapter_error, meaningful primarily paired with escalate "model_a", "model_b" where a different model/provider can bypass what transport could not. 2. AdapterCaller rescues ::RubyLLM::Error instead of StandardError. Programmer errors (NoMethodError, ArgumentError, config bugs) propagate rather than being silently converted into :adapter_error and retried. Provider errors still produce :adapter_error as before. Migration: add retry_on :validation_failed, :parse_error, :adapter_error to restore pre-0.7 behavior, or preferably combine with escalate for a real fallback chain. Full rationale: analysis in internal overlap study (H20) vs ruby_llm 1.14.1 Faraday retry scope. Codex-verified.
8d0af56 to
f9eae16
Compare
|
Codex review findings addressed in the amended commit. P1 (transport errors leaking) — fixed.
P2 (ArgumentError still coerced to :input_error) — acknowledged as known limitation. Full suite: 1339 examples, 0 failures, 8 pending. |
|
@copilot-pull-request-reviewer Review was on the pre-amend commit. The current HEAD (f9eae16) already has the regression spec you requested: |
ADR-0021 deliverable 2. Follow-up to Codex review finding P2 on PR #13 (v0.7.0), acknowledged there as known limitation. Before: Step::Base#run_once had a blanket `rescue ArgumentError` around the entire Runner chain. Intent was to catch DSL misconfiguration such as `prompt has not been set` and return it as :input_error. Side effect: any ArgumentError raised from adapter code during Runner#call — wrong arity, bad config arg, programmer bug — was silently coerced into :input_error and re-tried as if the user had supplied bad input. After: rescue is scoped to the Runner-construction phase only. DSL configuration errors still produce :input_error (regression test for `prompt has not been set` kept passing). ArgumentError raised during Runner#call propagates to the caller. Input-type validation failures continue to produce :input_error via InputValidator's own scoped rescue, unchanged. Two new regression specs under "transport error handling" describe: - adapter code raising ArgumentError now propagates (was: :input_error) - DSL misconfiguration (missing prompt) still returns :input_error Full suite: 1341 examples, 0 failures. Codex review on v0.7.0 explicitly flagged this case under P2; this closes that follow-up.
#15) ADR-0021 deliverable 2. Follow-up to Codex review finding P2 on PR #13 (v0.7.0), acknowledged there as known limitation. Before: Step::Base#run_once had a blanket `rescue ArgumentError` around the entire Runner chain. Intent was to catch DSL misconfiguration such as `prompt has not been set` and return it as :input_error. Side effect: any ArgumentError raised from adapter code during Runner#call — wrong arity, bad config arg, programmer bug — was silently coerced into :input_error and re-tried as if the user had supplied bad input. After: rescue is scoped to the Runner-construction phase only. DSL configuration errors still produce :input_error (regression test for `prompt has not been set` kept passing). ArgumentError raised during Runner#call propagates to the caller. Input-type validation failures continue to produce :input_error via InputValidator's own scoped rescue, unchanged. Two new regression specs under "transport error handling" describe: - adapter code raising ArgumentError now propagates (was: :input_error) - DSL misconfiguration (missing prompt) still returns :input_error Full suite: 1341 examples, 0 failures. Codex review on v0.7.0 explicitly flagged this case under P2; this closes that follow-up.
Summary
0.7.0 — breaking changes targeting redundancy between
ruby_llm-contractand upstreamruby_llm1.14.1.:adapter_errorremoved fromDEFAULT_RETRY_ON. New default:[:validation_failed, :parse_error].ruby_llm's Faraday middleware already retries transport errors (RateLimitError,ServerError,ServiceUnavailableError,OverloadedError, timeouts) with backoff (connection.rb:77), so the previous default re-ran the same model on errors the HTTP layer had already retried — retry × retry with no change in context. Opt in explicitly if needed, paired withescalatefor meaningful model fallback.AdapterCallernarrowsrescue StandardError→rescue ::RubyLLM::Error. Programmer errors (NoMethodError,ArgumentError, config bugs) now propagate instead of being silently coerced into:adapter_errorand retried. Provider errors still produce:adapter_erroras before.Why not a 0.6.x deprecation first?
Closed the previous PR #12 (which added a deprecation warning in 0.6.5) because this gem has no real external user base. A deprecation → grace period → breaking pattern is enterprise theater when nobody is between releases to see the warning. Ship the change, document migration clearly.
Migration
Restore pre-0.7 behavior (if you were relying on
:adapter_errorretry):Preferred pattern — pair with a model fallback chain:
Test plan
bundle exec rspec— 1336 examples, 0 failures, 8 pending (all pending are API-key-gated live LLM tests)bundle exec rubocop lib/ruby_llm/contract/step/retry_policy.rb lib/ruby_llm/contract/step/adapter_caller.rb— no offenses:adapter_errorretry tests now use explicitretry_on :adapter_erroropt-in; adapter failure fixtures raiseRubyLLM::Errorinstead ofStandardError; adversarial test inverted (:adapter_error is NOT retryable by default+ new test for explicit opt-in).0.6.4→0.7.0; Gemfile.lock updated.