Replace reqwest with a direct hyper transport - #195
Conversation
Record the migration plan for replacing the two bespoke reqwest-based GitHub clients (the GraphQL `GraphQLClient` and the feature-gated REST `RestClient`) with `octocrab` 0.54 as the transport layer. The plan keeps the existing `GraphQLClient` facade, `VkError` taxonomy, `backon` retry, transcript recording, and the `GITHUB_GRAPHQL_URL` / `GITHUB_API_URL` endpoint overrides intact, swapping only the transport internals via octocrab's raw `_post` path. Work is staged as four milestones gated by the existing test suite, beginning with a go/no-go spike, and includes authoring `docs/adr-001-adopt-octocrab.md`. Link the plan from `docs/contents.md`. No implementation has begun; the plan awaits approval.
Revise the octocrab adoption plan following review: octocrab's `graphql()` helper hides the raw responses that `vk`'s transcript, error-snippet, and retry machinery depend on, so routing GraphQL through it traded away the bespoke client's observability for little gain, while its typed REST surface remains a genuine win. The programme is now three pull requests: 1. Adopt octocrab for the REST resolve path only. 2. Replace `reqwest` inside the bespoke GraphQL client with a direct hyper transport, converging the binary on one HTTP stack. 3. Adopt `graphql_client` codegen (schema vendored from GitHub's published SDL) so queries and variables are checked at compile time, behind a conversion layer that keeps the exported domain types stable. Rename the planned ADR to `adr-001-github-api-client-modernisation.md` and update the plan's constraints, tolerances, risks, decision log, and interfaces to match. No implementation has begun.
Record a decision to keep vk-specific coupling (`VkError`, `vk::environment`) at the edges of the refitted GraphQL client so the executor (transport, retry, transcript, typed operations, pagination) can later be extracted into a shared crate for reuse by frankie, which will need the GraphQL-only review-thread surface. Extraction remains out of scope for this plan.
Author `docs/adr-001-github-api-client-modernisation.md`, the first ADR in the repository, recording the accepted three-part programme: octocrab for the REST resolve path, a direct hyper transport inside the bespoke GraphQL client (removing reqwest), and `graphql_client` codegen for compile-time-checked queries. Link the ADR from `docs/vk-design.md` and a new "Architecture decision records" section in `docs/contents.md`. Add the `octocrab ~0.54` dependency (tilde-pinned per the ADR's known-risks section; octocrab has shipped breaking changes in patch releases). Default features are disabled in favour of `default-client`, `rustls`, `rustls-ring`, and `timeout`; the `retry` feature is excluded so the REST reply path stays retry-free. `jwt-rust-crypto` is included because octocrab fails to compile without one of its JWT crypto features even when app auth is unused — recorded as a discovery in the ExecPlan. Mark the ExecPlan as IN PROGRESS with Stage A complete.
Rework `src/resolve/rest.rs` so `RestClient` wraps an
`octocrab::Octocrab` instance instead of a bespoke `reqwest::Client`,
deleting `github_client` and the hand-rolled header constants. The
`pub(crate)` surface (`RestClient::new`, `post_reply`) and observable
semantics are unchanged: the reply POSTs to the same
`repos/{owner}/{name}/pulls/{n}/comments/{id}/replies` route, 404
remains a warn-and-continue, and other non-2xx statuses remain fatal
with the route and status code in the error text.
Use octocrab's raw `_post` rather than the typed reply route: the typed
path reports GitHub errors without the request path or status code that
`tests/resolve.rs` asserts, while `_post` returns the raw response and
keeps status handling in `vk`.
Restore the `x-github-api-version: 2022-11-28` pin and
`Accept: application/vnd.github+json` via the builder's `add_header`,
adding a direct `http` dependency (already in the tree through hyper).
The REST path's `User-Agent` becomes `octocrab` (not asserted by any
test; recorded in the ExecPlan decision log). reqwest's total-request
timeout maps to octocrab's read and write timeouts, the closest
available analogue.
Update the ExecPlan progress and decision log accordingly.
Note in the resolve-threads section that the REST reply is served through octocrab with the `x-github-api-version: 2022-11-28` pin and raw-route status handling, referencing ADR 001.
Record gates, zero-finding CodeRabbit review, and draft PR #194 in the progress log.
Add `src/api/client/transport.rs`, a private pooled transport built on `hyper-util`'s legacy client with a `hyper-rustls` connector (webpki roots + ring, matching what reqwest's `rustls-tls` feature provided; `https_or_http` keeps loopback test servers working). The transport enforces the total-request timeout with a single `tokio::time::timeout` spanning send and body collection, mirroring reqwest's `.timeout()` scope, and reproduces the existing `VkError::RequestContext` context strings so retry classification and error text are unchanged. `GraphQLClient` now holds a `Transport` instead of a `reqwest::Client`; `execute_single_request` delegates to it and keeps transcript logging, status handling, and body-snippet error context as before. Header construction switches from `reqwest::header` to `http::header` (the same types; reqwest re-exported them). Remove `reqwest` from the manifest entirely — `cargo tree -i reqwest` now finds no package in the normal, dev, or all-features graphs — and promote `hyper`, `hyper-util`, `hyper-rustls`, and `http-body-util` to runtime dependencies, unifying the former dev-only entries. Drop the binary-internal `VkError::Request` variant (constructible only from reqwest errors) and its retry-classifier arm. System proxies and redirects, which reqwest honoured by default, are deliberately unsupported by the new transport; neither is used or tested on the GraphQL path, and both are documented in the module. All 274 tests pass unchanged; no test file was edited beyond a mandatory reqwest-to-http import swap in one inline test module.
Note in the design document's networking section that requests travel through the private hyper-based transport with rustls and webpki roots, that the timeout spans send plus body collection, and that system proxies and redirects are unsupported. Correct `docs/vk-end-to-end-testing-guide.md`, which described the test harness as a third-wheel man-in-the-middle proxy intercepting traffic. The implemented harness runs plain hyper loopback stub servers and points the binary at them through `GITHUB_GRAPHQL_URL` and `GITHUB_API_URL`; third-wheel contributes only re-exported hyper types. The MITM material is now framed as background, with a new subsection describing what the harness actually does.
Record gates, transcript-replay verification, zero-finding CodeRabbit review, and the stacked draft PR in the progress log.
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Gates Failed
Enforce advisory code health rules
(1 file with Excess Number of Function Arguments)
Our agent can fix these. Install it.
Gates Passed
5 Quality Gates Passed
Reason for failure
| Enforce advisory code health rules | Violations | Code Health Impact | |
|---|---|---|---|
| transport.rs | 1 advisory rule | 9.69 | Suppress |
Quality Gate Profile: Pay Down Tech Debt
Install CodeScene MCP: safeguard and uplift AI-generated code. Catch issues early with our IDE extension and CLI tool.
|
Oops, something went wrong! Please try again later. 🐰 💔 |
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response. Ensure that this is validated against the current version of the codegraph. If further refinement to address this finding would be deleterious, please supply a clear explanatory one to two paragraph markdown message I can paste into the CodeScene web ui's diagnostic suppression function so this diagnostic can be silenced. Comment on lines +86 to +151 pub(super) async fn post_json(
&self,
endpoint: &Endpoint,
headers: &HeaderMap,
payload: &Value,
timeout: Duration,
) -> Result<HttpResponse, VkError> {
// Reconstruct the same request-context strings the reqwest client
// produced: the operation name (or a query snippet fallback) and a
// redacted payload snippet. Derived from the payload so this transport
// stays self-contained behind its fixed signature.
let snip = payload_snippet(payload);
let query = payload.get("query").and_then(Value::as_str).unwrap_or("");
let operation = operation_name(query).map_or_else(|| snippet(query, 64), str::to_string);
let make_ctx = |status: Option<u16>| {
let base = format!("operation {operation}; {snip}");
// Disambiguate from `BodyExt::boxed`, which is also in scope.
BoxedStr::boxed(match status {
Some(s) => format!("{base}; status {s}"),
None => base,
})
};
let request = build_request(endpoint, headers, payload).map_err(|source| {
VkError::RequestContext {
context: make_ctx(None),
source,
}
})?;
let exchange = async {
let response =
self.client
.request(request)
.await
.map_err(|e| VkError::RequestContext {
context: make_ctx(None),
source: Box::new(e),
})?;
let status = response.status().as_u16();
let collected =
response
.into_body()
.collect()
.await
.map_err(|e| VkError::RequestContext {
context: make_ctx(Some(status)),
source: Box::new(e),
})?;
// Lossily decode, matching reqwest's UTF-8 text handling; decoding
// never fails, so only a collection error can surface above.
let body = String::from_utf8_lossy(&collected.to_bytes()).into_owned();
Ok::<HttpResponse, VkError>(HttpResponse { status, body })
};
match tokio::time::timeout(timeout, exchange).await {
Ok(result) => result,
Err(_elapsed) => Err(VkError::RequestContext {
context: make_ctx(None),
source: Box::new(std::io::Error::new(
std::io::ErrorKind::TimedOut,
format!("request timed out after {timeout:?}"),
)),
}),
}
}❌ New issue: Excess Number of Function Arguments |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
fd -a 'transport.rs' src/api/client || true
echo "---outline---"
ast-grep outline src/api/client/transport.rs --items all 2>/dev/null | head -100
echo "---full file---"
sed -n '1,220p' src/api/client/transport.rsLength of output: 8705 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "---mod.rs outline---"
ast-grep outline src/api/client/mod.rs --items all 2>/dev/null | head -80
echo "---callers of post_json---"
rg -n 'post_json' src -A5 -B5Length of output: 2663 🏁 Script executed: #!/bin/bash
set -euo pipefail
sed -n '100,145p' src/api/client/mod.rsLength of output: 1837
Bundle the four data arguments into a request struct so Introduce a /// Bundles the per-request inputs to [`Transport::post_json`].
pub(super) struct PostJsonRequest<'a> {
pub(super) endpoint: &'a Endpoint,
pub(super) headers: &'a HeaderMap,
pub(super) payload: &'a Value,
pub(super) timeout: Duration,
}Change the method signature and destructure at the top of the body: pub(super) async fn post_json(
&self,
request: PostJsonRequest<'_>,
) -> Result<HttpResponse, VkError> {
let PostJsonRequest { endpoint, headers, payload, timeout } = request;
// ... rest of the body is unchanged, referring to `endpoint`, `headers`,
// `payload`, and `timeout` as before.Update the call site in let resp = self
.transport
.post_json(transport::PostJsonRequest {
endpoint: &self.endpoint,
headers: &self.headers,
payload,
timeout: self.retry.request_timeout,
})
.await?;(Export This drops Prompt for an AI coding agentSuppression rationale (if you choose not to refactor)If you decide the struct-bundling refactor is not worth the churn for a single call site, paste this into CodeScene's suppression field:
That said, the struct-based fix above is low-risk and mechanical (one struct, one call site), so I'd recommend applying it rather than suppressing, unless you have a specific reason to keep the flat signature. ✏️ Learnings added
|
Summary
This branch delivers PR 2 of the three-phase GitHub API modernisation
governed by
ADR 001:
the bespoke GraphQL client now sends requests through a direct hyper
transport, and
reqwestleaves the dependency graph entirely. The binaryconverges on one HTTP stack (hyper/rustls), shared with octocrab.
Execplan: docs/execplans/adopt-octocrab.md
— PR 2 of its three phases lands here. Stacked on #194 (PR 1, octocrab
REST); the typed-GraphQL phase (PR 3) remains as authorized future work.
Observable behaviour is unchanged: transcript format, retry semantics and
attempt counts, error-message text, header set, and the
GITHUB_GRAPHQL_URLendpoint override all survive, evidenced by the fullsuite passing with no test edits (one inline test module swapped a
reqwest::headerimport forhttp::header— the same types reqwestre-exported).
Review walkthrough
src/api/client/transport.rs
— a pooled
hyper-utillegacy client with ahyper-rustlsconnector(webpki roots + ring, matching what reqwest's
rustls-tlsfeatureprovided;
https_or_httpkeeps loopback test servers working). Onetokio::time::timeoutspans send plus body collection, mirroringreqwest's
.timeout()scope; timeouts map toVkError::RequestContext, which the retry classifier already treatsas transient. System proxies and redirects are deliberately
unsupported and documented.
src/api/client/mod.rs
—
GraphQLClientholds aTransportinstead of areqwest::Client;transcript logging, status handling, and body-snippet error context
are untouched.
Cargo.toml
—
reqwestremoved;hyper,hyper-util,hyper-rustls, andhttp-body-utilpromoted to runtime dependencies, unifying formerdev-only entries.
docs/vk-end-to-end-testing-guide.md
— corrects a long-standing inaccuracy: the harness is plain hyper
loopback stubs driven by env-var endpoint overrides, not a
man-in-the-middle proxy.
Validation
make check-fmt/make lint/make test: pass (274 tests, 1 ignored)cargo test --test e2e -- --ignored e2e_pr_42: pass (transcript-format parity replayed from the recorded fixture)cargo tree -i reqwest(also-e normaland--all-features): package not foundmake markdownlint/make nixie: passcoderabbit review --agent: completed, zero findingsNotes
VkError::Requestvariant (constructible only fromreqwest errors) was removed with its retry-classifier arm;
VkErrorisnot exported from
src/lib.rs, so the library surface is unchanged.HTTP(S)_PROXYand followed redirects by default; thenew transport does neither. Both behaviours are unused and untested on
the GraphQL path and are documented in the transport module and the
ExecPlan decision log.