Skip to content

Replace reqwest with a direct hyper transport - #195

Draft
leynos wants to merge 10 commits into
adopt-octocrabfrom
hyper-graphql-transport
Draft

Replace reqwest with a direct hyper transport#195
leynos wants to merge 10 commits into
adopt-octocrabfrom
hyper-graphql-transport

Conversation

@leynos

@leynos leynos commented Jul 9, 2026

Copy link
Copy Markdown
Owner

Summary

This branch delivers PR 2 of the three-phase GitHub API modernisation
governed by
ADR 001:
the bespoke GraphQL client now sends requests through a direct hyper
transport, and reqwest leaves the dependency graph entirely. The binary
converges on one HTTP stack (hyper/rustls), shared with octocrab.

Execplan: docs/execplans/adopt-octocrab.md
— PR 2 of its three phases lands here. Stacked on #194 (PR 1, octocrab
REST); the typed-GraphQL phase (PR 3) remains as authorized future work.

Observable behaviour is unchanged: transcript format, retry semantics and
attempt counts, error-message text, header set, and the
GITHUB_GRAPHQL_URL endpoint override all survive, evidenced by the full
suite passing with no test edits (one inline test module swapped a
reqwest::header import for http::header — the same types reqwest
re-exported).

Review walkthrough

  • Start with
    src/api/client/transport.rs
    — a pooled hyper-util legacy client with a hyper-rustls connector
    (webpki roots + ring, matching what reqwest's rustls-tls feature
    provided; https_or_http keeps loopback test servers working). One
    tokio::time::timeout spans send plus body collection, mirroring
    reqwest's .timeout() scope; timeouts map to
    VkError::RequestContext, which the retry classifier already treats
    as transient. System proxies and redirects are deliberately
    unsupported and documented.
  • Then
    src/api/client/mod.rs
    GraphQLClient holds a Transport instead of a reqwest::Client;
    transcript logging, status handling, and body-snippet error context
    are untouched.
  • Check
    Cargo.toml
    reqwest removed; hyper, hyper-util, hyper-rustls, and
    http-body-util promoted to runtime dependencies, unifying former
    dev-only entries.
  • Finish with
    docs/vk-end-to-end-testing-guide.md
    — corrects a long-standing inaccuracy: the harness is plain hyper
    loopback stubs driven by env-var endpoint overrides, not a
    man-in-the-middle proxy.

Validation

  • make check-fmt / make lint / make test: pass (274 tests, 1 ignored)
  • cargo test --test e2e -- --ignored e2e_pr_42: pass (transcript-format parity replayed from the recorded fixture)
  • cargo tree -i reqwest (also -e normal and --all-features): package not found
  • make markdownlint / make nixie: pass
  • coderabbit review --agent: completed, zero findings

Notes

  • The binary-internal VkError::Request variant (constructible only from
    reqwest errors) was removed with its retry-classifier arm; VkError is
    not exported from src/lib.rs, so the library surface is unchanged.
  • reqwest honoured HTTP(S)_PROXY and followed redirects by default; the
    new transport does neither. Both behaviours are unused and untested on
    the GraphQL path and are documented in the transport module and the
    ExecPlan decision log.

leynos added 10 commits July 9, 2026 01:11
Record the migration plan for replacing the two bespoke reqwest-based
GitHub clients (the GraphQL `GraphQLClient` and the feature-gated REST
`RestClient`) with `octocrab` 0.54 as the transport layer.

The plan keeps the existing `GraphQLClient` facade, `VkError` taxonomy,
`backon` retry, transcript recording, and the `GITHUB_GRAPHQL_URL` /
`GITHUB_API_URL` endpoint overrides intact, swapping only the transport
internals via octocrab's raw `_post` path. Work is staged as four
milestones gated by the existing test suite, beginning with a go/no-go
spike, and includes authoring `docs/adr-001-adopt-octocrab.md`.

Link the plan from `docs/contents.md`. No implementation has begun;
the plan awaits approval.
Revise the octocrab adoption plan following review: octocrab's
`graphql()` helper hides the raw responses that `vk`'s transcript,
error-snippet, and retry machinery depend on, so routing GraphQL
through it traded away the bespoke client's observability for little
gain, while its typed REST surface remains a genuine win.

The programme is now three pull requests:

1. Adopt octocrab for the REST resolve path only.
2. Replace `reqwest` inside the bespoke GraphQL client with a direct
   hyper transport, converging the binary on one HTTP stack.
3. Adopt `graphql_client` codegen (schema vendored from GitHub's
   published SDL) so queries and variables are checked at compile
   time, behind a conversion layer that keeps the exported domain
   types stable.

Rename the planned ADR to `adr-001-github-api-client-modernisation.md`
and update the plan's constraints, tolerances, risks, decision log,
and interfaces to match. No implementation has begun.
Record a decision to keep vk-specific coupling (`VkError`,
`vk::environment`) at the edges of the refitted GraphQL client so the
executor (transport, retry, transcript, typed operations, pagination)
can later be extracted into a shared crate for reuse by frankie, which
will need the GraphQL-only review-thread surface. Extraction remains
out of scope for this plan.
Author `docs/adr-001-github-api-client-modernisation.md`, the first ADR
in the repository, recording the accepted three-part programme: octocrab
for the REST resolve path, a direct hyper transport inside the bespoke
GraphQL client (removing reqwest), and `graphql_client` codegen for
compile-time-checked queries. Link the ADR from `docs/vk-design.md` and
a new "Architecture decision records" section in `docs/contents.md`.

Add the `octocrab ~0.54` dependency (tilde-pinned per the ADR's
known-risks section; octocrab has shipped breaking changes in patch
releases). Default features are disabled in favour of `default-client`,
`rustls`, `rustls-ring`, and `timeout`; the `retry` feature is excluded
so the REST reply path stays retry-free. `jwt-rust-crypto` is included
because octocrab fails to compile without one of its JWT crypto
features even when app auth is unused — recorded as a discovery in the
ExecPlan.

Mark the ExecPlan as IN PROGRESS with Stage A complete.
Rework `src/resolve/rest.rs` so `RestClient` wraps an
`octocrab::Octocrab` instance instead of a bespoke `reqwest::Client`,
deleting `github_client` and the hand-rolled header constants. The
`pub(crate)` surface (`RestClient::new`, `post_reply`) and observable
semantics are unchanged: the reply POSTs to the same
`repos/{owner}/{name}/pulls/{n}/comments/{id}/replies` route, 404
remains a warn-and-continue, and other non-2xx statuses remain fatal
with the route and status code in the error text.

Use octocrab's raw `_post` rather than the typed reply route: the typed
path reports GitHub errors without the request path or status code that
`tests/resolve.rs` asserts, while `_post` returns the raw response and
keeps status handling in `vk`.

Restore the `x-github-api-version: 2022-11-28` pin and
`Accept: application/vnd.github+json` via the builder's `add_header`,
adding a direct `http` dependency (already in the tree through hyper).
The REST path's `User-Agent` becomes `octocrab` (not asserted by any
test; recorded in the ExecPlan decision log). reqwest's total-request
timeout maps to octocrab's read and write timeouts, the closest
available analogue.

Update the ExecPlan progress and decision log accordingly.
Note in the resolve-threads section that the REST reply is served
through octocrab with the `x-github-api-version: 2022-11-28` pin and
raw-route status handling, referencing ADR 001.
Record gates, zero-finding CodeRabbit review, and draft PR #194 in the
progress log.
Add `src/api/client/transport.rs`, a private pooled transport built on
`hyper-util`'s legacy client with a `hyper-rustls` connector (webpki
roots + ring, matching what reqwest's `rustls-tls` feature provided;
`https_or_http` keeps loopback test servers working). The transport
enforces the total-request timeout with a single `tokio::time::timeout`
spanning send and body collection, mirroring reqwest's `.timeout()`
scope, and reproduces the existing `VkError::RequestContext` context
strings so retry classification and error text are unchanged.

`GraphQLClient` now holds a `Transport` instead of a `reqwest::Client`;
`execute_single_request` delegates to it and keeps transcript logging,
status handling, and body-snippet error context as before. Header
construction switches from `reqwest::header` to `http::header` (the
same types; reqwest re-exported them).

Remove `reqwest` from the manifest entirely — `cargo tree -i reqwest`
now finds no package in the normal, dev, or all-features graphs — and
promote `hyper`, `hyper-util`, `hyper-rustls`, and `http-body-util` to
runtime dependencies, unifying the former dev-only entries. Drop the
binary-internal `VkError::Request` variant (constructible only from
reqwest errors) and its retry-classifier arm.

System proxies and redirects, which reqwest honoured by default, are
deliberately unsupported by the new transport; neither is used or
tested on the GraphQL path, and both are documented in the module.

All 274 tests pass unchanged; no test file was edited beyond a
mandatory reqwest-to-http import swap in one inline test module.
Note in the design document's networking section that requests travel
through the private hyper-based transport with rustls and webpki roots,
that the timeout spans send plus body collection, and that system
proxies and redirects are unsupported.

Correct `docs/vk-end-to-end-testing-guide.md`, which described the test
harness as a third-wheel man-in-the-middle proxy intercepting traffic.
The implemented harness runs plain hyper loopback stub servers and
points the binary at them through `GITHUB_GRAPHQL_URL` and
`GITHUB_API_URL`; third-wheel contributes only re-exported hyper types.
The MITM material is now framed as background, with a new subsection
describing what the harness actually does.
Record gates, transcript-replay verification, zero-finding CodeRabbit
review, and the stacked draft PR in the progress log.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @leynos, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: dd01bf2e-5e03-4bf6-b3be-43036205e2a2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch hyper-graphql-transport

Comment @coderabbitai help to get the list of available commands.

@codescene-access codescene-access Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gates Failed
Enforce advisory code health rules (1 file with Excess Number of Function Arguments)

Our agent can fix these. Install it.

Gates Passed
5 Quality Gates Passed

Reason for failure
Enforce advisory code health rules Violations Code Health Impact
transport.rs 1 advisory rule 9.69 Suppress

See analysis details in CodeScene

Quality Gate Profile: Pay Down Tech Debt
Install CodeScene MCP: safeguard and uplift AI-generated code. Catch issues early with our IDE extension and CLI tool.

Comment thread src/api/client/transport.rs
@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown

Oops, something went wrong! Please try again later. 🐰 💔

@leynos

leynos commented Jul 9, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response. Ensure that this is validated against the current version of the codegraph.

If further refinement to address this finding would be deleterious, please supply a clear explanatory one to two paragraph markdown message I can paste into the CodeScene web ui's diagnostic suppression function so this diagnostic can be silenced.

src/api/client/transport.rs

Comment on lines +86 to +151

    pub(super) async fn post_json(
        &self,
        endpoint: &Endpoint,
        headers: &HeaderMap,
        payload: &Value,
        timeout: Duration,
    ) -> Result<HttpResponse, VkError> {
        // Reconstruct the same request-context strings the reqwest client
        // produced: the operation name (or a query snippet fallback) and a
        // redacted payload snippet. Derived from the payload so this transport
        // stays self-contained behind its fixed signature.
        let snip = payload_snippet(payload);
        let query = payload.get("query").and_then(Value::as_str).unwrap_or("");
        let operation = operation_name(query).map_or_else(|| snippet(query, 64), str::to_string);
        let make_ctx = |status: Option<u16>| {
            let base = format!("operation {operation}; {snip}");
            // Disambiguate from `BodyExt::boxed`, which is also in scope.
            BoxedStr::boxed(match status {
                Some(s) => format!("{base}; status {s}"),
                None => base,
            })
        };

        let request = build_request(endpoint, headers, payload).map_err(|source| {
            VkError::RequestContext {
                context: make_ctx(None),
                source,
            }
        })?;

        let exchange = async {
            let response =
                self.client
                    .request(request)
                    .await
                    .map_err(|e| VkError::RequestContext {
                        context: make_ctx(None),
                        source: Box::new(e),
                    })?;
            let status = response.status().as_u16();
            let collected =
                response
                    .into_body()
                    .collect()
                    .await
                    .map_err(|e| VkError::RequestContext {
                        context: make_ctx(Some(status)),
                        source: Box::new(e),
                    })?;
            // Lossily decode, matching reqwest's UTF-8 text handling; decoding
            // never fails, so only a collection error can surface above.
            let body = String::from_utf8_lossy(&collected.to_bytes()).into_owned();
            Ok::<HttpResponse, VkError>(HttpResponse { status, body })
        };

        match tokio::time::timeout(timeout, exchange).await {
            Ok(result) => result,
            Err(_elapsed) => Err(VkError::RequestContext {
                context: make_ctx(None),
                source: Box::new(std::io::Error::new(
                    std::io::ErrorKind::TimedOut,
                    format!("request timed out after {timeout:?}"),
                )),
            }),
        }
    }

❌ New issue: Excess Number of Function Arguments
Transport.post_json has 5 arguments, max arguments = 4

@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail
fd -a 'transport.rs' src/api/client || true
echo "---outline---"
ast-grep outline src/api/client/transport.rs --items all 2>/dev/null | head -100
echo "---full file---"
sed -n '1,220p' src/api/client/transport.rs

Length of output: 8705


🏁 Script executed:

#!/bin/bash
set -euo pipefail
echo "---mod.rs outline---"
ast-grep outline src/api/client/mod.rs --items all 2>/dev/null | head -80
echo "---callers of post_json---"
rg -n 'post_json' src -A5 -B5

Length of output: 2663


🏁 Script executed:

#!/bin/bash
set -euo pipefail
sed -n '100,145p' src/api/client/mod.rs

Length of output: 1837


@leynos here's the fix for Transport::post_json in src/api/client/transport.rs.

Bundle the four data arguments into a request struct so post_json takes &self plus one parameter instead of &self plus four. The sole call site is GraphQLClient::execute_single_request in src/api/client/mod.rs (lines 130–138), which is a mechanical one-line change.

Introduce a PostJsonRequest<'a> struct alongside Transport in src/api/client/transport.rs:

/// Bundles the per-request inputs to [`Transport::post_json`].
pub(super) struct PostJsonRequest<'a> {
    pub(super) endpoint: &'a Endpoint,
    pub(super) headers: &'a HeaderMap,
    pub(super) payload: &'a Value,
    pub(super) timeout: Duration,
}

Change the method signature and destructure at the top of the body:

pub(super) async fn post_json(
    &self,
    request: PostJsonRequest<'_>,
) -> Result<HttpResponse, VkError> {
    let PostJsonRequest { endpoint, headers, payload, timeout } = request;
    // ... rest of the body is unchanged, referring to `endpoint`, `headers`,
    // `payload`, and `timeout` as before.

Update the call site in src/api/client/mod.rs:

let resp = self
    .transport
    .post_json(transport::PostJsonRequest {
        endpoint: &self.endpoint,
        headers: &self.headers,
        payload,
        timeout: self.retry.request_timeout,
    })
    .await?;

(Export PostJsonRequest from the transport module — it's already pub(super), so no visibility change needed beyond the struct itself; mod.rs already has mod transport; in scope.)

This drops post_json to two effective arguments (self, request), satisfying CodeScene's threshold, without touching the transcript, retry, or error-context logic that the PR explicitly preserves.


Prompt for an AI coding agent

Refactor `Transport::post_json` in `src/api/client/transport.rs` (vk repository) to
resolve a CodeScene "Excess Number of Function Arguments" finding (5 arguments,
max 4).

1. In `src/api/client/transport.rs`, add a new struct near the top of the file
   (after the `Transport` struct definition, before `impl Transport`):

   ```rust
   /// Bundles the per-request inputs to [`Transport::post_json`].
   pub(super) struct PostJsonRequest<'a> {
       pub(super) endpoint: &'a Endpoint,
       pub(super) headers: &'a HeaderMap,
       pub(super) payload: &'a Value,
       pub(super) timeout: Duration,
   }
   ```

2. Change the signature of `Transport::post_json` from:

   ```rust
   pub(super) async fn post_json(
       &self,
       endpoint: &Endpoint,
       headers: &HeaderMap,
       payload: &Value,
       timeout: Duration,
   ) -> Result<HttpResponse, VkError> {
   ```

   to:

   ```rust
   pub(super) async fn post_json(
       &self,
       request: PostJsonRequest<'_>,
   ) -> Result<HttpResponse, VkError> {
       let PostJsonRequest { endpoint, headers, payload, timeout } = request;
   ```

   Keep the rest of the method body unchanged — it already references
   `endpoint`, `headers`, `payload`, and `timeout` by name.

3. Update the doc comment above `post_json` to describe the single `request`
   parameter instead of the four separate ones (keep the existing `# Errors`
   section intact).

4. In `src/api/client/mod.rs`, update the sole call site inside
   `GraphQLClient::execute_single_request` from:

   ```rust
   let resp = self
       .transport
       .post_json(
           &self.endpoint,
           &self.headers,
           payload,
           self.retry.request_timeout,
       )
       .await?;
   ```

   to:

   ```rust
   let resp = self
       .transport
       .post_json(transport::PostJsonRequest {
           endpoint: &self.endpoint,
           headers: &self.headers,
           payload,
           timeout: self.retry.request_timeout,
       })
       .await?;
   ```

   Confirm `transport` is already imported as a module (it is, via
   `mod transport;` at the top of `mod.rs`) so `transport::PostJsonRequest` is
   directly reachable; adjust the path if the module is aliased differently.

5. Run `cargo build` and `cargo test` (or the repository's `make check-fmt`,
   `make lint`, `make test` targets) to confirm the change compiles and all
   274 existing tests still pass, with no behavioural change to transcript
   logging, retry semantics, or error-context strings.

Do not change any other logic inside `post_json`'s body, the timeout handling,
or the error-context construction (`make_ctx`); this is purely a
parameter-bundling refactor.

Suppression rationale (if you choose not to refactor)

If you decide the struct-bundling refactor is not worth the churn for a single call site, paste this into CodeScene's suppression field:

Transport::post_json takes five arguments (&self, endpoint, headers, payload, timeout) because it deliberately reconstructs, in one self-contained call, exactly the inputs reqwest previously received implicitly via a pre-configured reqwest::Client and request builder chain. It has exactly one call site (GraphQLClient::execute_single_request), all four data parameters are independently meaningful (a fixed GraphQL endpoint, per-request auth headers, the JSON payload, and a per-call timeout), and none of them naturally group into a pre-existing domain type without introducing a purpose-built wrapper struct solely to satisfy this metric. Splitting the method further would fragment request construction, timeout scoping, and error-context derivation (make_ctx) across multiple functions, working against the single-timeout-spans-everything design documented at the top of transport.rs and increasing, not decreasing, the surface for a reviewer to hold in mind at once.

That said, the struct-based fix above is low-risk and mechanical (one struct, one call site), so I'd recommend applying it rather than suppressing, unless you have a specific reason to keep the flat signature.


✏️ Learnings added
Learnt from: leynos
URL: https://github.com/leynos/vk/pull/195

Timestamp: 2026-07-09T15:35:00.212Z
Learning: In the vk repository (Rust), CodeScene's "Excess Number of Function Arguments" check counts `&self` as one of the arguments, so a method with `&self` plus 4 parameters (5 total) already exceeds the max-4-arguments threshold.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant