From 3a1e9ede2fb71580b440b0e70dcff07e50367ad1 Mon Sep 17 00:00:00 2001 From: aleekaz Date: Fri, 4 Sep 2026 16:31:04 +0300 Subject: [PATCH 1/2] Prepare the package page for the repository going public Six of these were held back while nodemaven-python was internal, where a link into it answers 404 to anyone not signed in. - pyproject: Source and Issues added to [project.urls]. PyPI bakes that table into each uploaded distribution and never re-reads it, so these reach the existing 0.1.2 page through a release and never through the visibility flip. The same is true of everything below that touches the README, which is the long description. - README: the quickstart now makes its first live request through ANY proxy, using Provider plus host= and port=. Nothing in the library changed - the capability shipped in 0.1.0 and the README buried it in the last section. Verified 2026-09-04 against the 0.1.2 wheel from PyPI in a clean venv with NODEMAVEN_* stripped from the environment: 7 of 7 cases pass, including credentials read from a file-defined gateway's own env prefix. - README: a Parameters table for the shipped gateway carrying the dates the values were measured on, and an Errors table. Names are validated, values mostly are not, and that is now said outright. - README: the CI badge, an absolute LICENSE link, and the benchmark harness the retry claim is measured on. - CHANGELOG.md, written from the tags and the PyPI upload records rather than from memory. It records that norotate never shipped in any release. - ci.yml: a ci-ok job that is green only when every matrix job is. A required check is matched by name, so the six matrix-derived names go stale the moment the Python list changes and block every PR until somebody edits the ruleset. It needs if: always() and an explicit test of the result, because a SKIPPED required check counts as passing. - The two public members that had no docstring got one: Provider.is_measured and Proxy.provider. Ordering constraint, measured 2026-09-04 while the repository was still internal: the CI badge renders "tests: repo or workflow not found" and both new GitHub URLs 404 anonymously although the paths exist when authenticated. So this lands with the visibility flip or after it, never before - otherwise the package page carries a broken badge and a dead link, which is what the flip is meant to fix. --- .github/workflows/ci.yml | 23 +++++ CHANGELOG.md | 87 ++++++++++++++++ README.md | 203 ++++++++++++++++++++++++++++++++----- pyproject.toml | 17 +++- src/nodemaven/providers.py | 6 ++ src/nodemaven/proxy.py | 7 ++ 6 files changed, 314 insertions(+), 29 deletions(-) create mode 100644 CHANGELOG.md diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 36f4943..7480765 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -44,3 +44,26 @@ jobs: - run: python -m pip install --upgrade pip - run: python -m pip install . pytest - run: python -m pytest -q + + # One name for the branch ruleset to require, instead of the six matrix jobs. + # + # A required status check is matched by name, and a name that stops being + # produced never reports - so a pull request waits on it forever. Requiring + # `test (3.9, ubuntu-latest)` directly means that dropping 3.9 from the matrix + # above silently blocks every pull request in the repository, and the ruleset + # is somewhere nobody edits when they edit this file. This job's name does not + # depend on the matrix, so the two cannot drift. + # + # `if: always()` with an explicit result check, and not a bare `needs:`. A + # skipped or cancelled dependency leaves a plain `needs:` job skipped rather + # than failed, and a skipped required check is treated as passing - which is + # the same hole in the other direction. + ci-ok: + if: always() + needs: test + runs-on: ubuntu-latest + steps: + - name: every matrix job succeeded + run: | + echo "test -> ${{ needs.test.result }}" + test "${{ needs.test.result }}" = "success" diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..aa96aa1 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,87 @@ +# Changelog + +Written against the tags and the PyPI upload records rather than from memory, on +2026-09-04. Release times are UTC as PyPI recorded them. + +A note on how entries here are worded, because it is the same rule the library +itself is built on: a change to what the gateway is believed to accept carries +the probe that established it and the date it was run. "The vendor's documentation +says so" is not one of those, and an entry that rests on it says so outright. + +## Unreleased + +- `Source` and `Issues` added to the package metadata, and the README gained a + CI badge, a link to its own LICENSE and a link to the benchmark harness the + retry table is measured on. All four were held back while the repository was + internal, where they answered 404 to anyone not signed in. +- The quickstart now shows a live first request through **any** proxy, not only + through NodeMaven's gateway. Nothing in the library changed - the capability + was there and the README buried it in the last section. +- A `Parameters` reference for the shipped gateway, and an `Errors` list. + +## 0.1.2 - 2026-08-26 09:23 + +- **`values`**, a per-parameter list of legal values, added to the provider + schema. A value outside the list is refused before anything is sent. It ships + empty for NodeMaven on purpose: what was known was which values the vendor's + proxy generator *emits*, which is a sample of one interface's output and not + the set the gateway accepts. A wrong entry here blocks a setting that would + have worked, which is worse than the gap it closes. + + The key exists before there is anything to put in it because four language + SDKs read this schema: filling it in later is an edit to a data file, adding + the key later is an edit to four parsers. + +- The gateway's known parameters and their reactions were re-recorded in the + shipped definition with the probes and dates behind them. + +**`norotate` never shipped, and the round trip is the reason this file exists.** +It was added to the known parameter list on 2026-08-21 because it appears in the +vendor's own proxy generator, and removed on 2026-08-26 after being probed: three +ways, each with a negative control carrying a name nobody could have implemented +and a positive control on `filter`. It takes a junk value with 200 exactly as the +unknown name does; with no session id it draws 6 distinct exits of 6, exactly as +both controls; with a fixed session id and `ttl=1m` sampled at 0, 78 and 157 +seconds it draws 3 distinct of 3, again exactly as both controls. The gateway +answers 200 and drops it. + +Both changes sat between the 0.1.1 and 0.1.2 tags, so no release ever accepted +the parameter. What was wrong for those five days was the reasoning, not a +published artifact: an unrecognised name and a name we have not heard of produce +identical evidence on this gateway, and the vendor's file was allowed to break +the tie instead of a probe. + +## 0.1.1 - 2026-08-20 20:58 + +No change to the library. The version exists because the README shipped an +example that did not run, and a README is the package page. + +- The separator rule was restated as a measurement rather than a precaution. On + 2026-08-20 a probe opened tunnels with `sid-order8e3bf9-4417` and with + `sid-order8e3bf9`, four rounds each, interleaved; both landed on **one** exit + address, while a third arm spelling `sid-order8e3bf94417` held a different one + throughout. The gateway cuts the value at the separator, so every order id + beginning `order` would quietly share one session and one exit. + +## 0.1.0 - 2026-08-20 19:42 + +First release. + +- `Proxy`: builds the gateway username, and refuses input the gateway would + mishandle. Credentials fall back to the environment. `url()`, `requests()`, + `httpx()` and `playwright()` for the clients; `replace()` and `session()` + return new objects, because on this gateway the session key is the whole + parameter set and changing one parameter is a different exit address rather + than an adjustment. +- `Provider`, `load()`, `load_file()`, `available()`: a gateway dialect is a + TOML file and never a code path, so a definition you write yourself goes + through the same builder and the same validation as the shipped one. +- `NodeMavenError` and the three exceptions under it. +- No socket, no session, no connection pool, and **no retry**. The last one is + the only design decision here taken against a measurement rather than a + preference: over 1464 attempts the chance the next one succeeds falls from 75% + with no prior failure to 5.8% after five and 0.5% after seven. + +This one was uploaded with an API token, before Trusted Publishing was set up. +That token was revoked and no token is stored anywhere now; every release since +is published by the `publish` workflow against an OIDC identity. diff --git a/README.md b/README.md index 5ad3e81..bdfbe81 100644 --- a/README.md +++ b/README.md @@ -1,36 +1,57 @@
- + `img src`, `height` and the badges all survive its sanitiser. --> NodeMaven # nodemaven **Builds the proxy username a gateway expects, and refuses the input it would silently drop.** - + [![pypi](https://img.shields.io/pypi/v/nodemaven?style=flat-square)](https://pypi.org/project/nodemaven/) [![python](https://img.shields.io/pypi/pyversions/nodemaven?style=flat-square)](https://pypi.org/project/nodemaven/) +[![ci](https://img.shields.io/github/actions/workflow/status/nodemaven/nodemaven-python/ci.yml?branch=main&style=flat-square&label=tests)](https://github.com/nodemaven/nodemaven-python/actions/workflows/ci.yml) + dead link readme_renderer showed in this file. The badge keeps pointing at + opensource.org because it is a claim about the licence rather than about this + repository; the file itself is linked from the License section at the bottom, + which it could not be while the repository was internal. --> [![license](https://img.shields.io/pypi/l/nodemaven?style=flat-square)](https://opensource.org/licenses/MIT) -[Quickstart](#quickstart) · [Sticky sessions](#sticky-sessions) · [Why the validation is the point](#why-the-validation-is-the-point) · [What it does not do](#what-this-library-does-not-do) · [Docs](https://docs.nodemaven.com?utm_source=github&utm_medium=sdk_python&utm_campaign=readme) +[Quickstart](#quickstart) · [Sticky sessions](#sticky-sessions) · [Why the validation is the point](#why-the-validation-is-the-point) · [What it does not do](#what-this-library-does-not-do) · [Other gateways](#other-gateways) · [Docs](https://docs.nodemaven.com?utm_source=github&utm_medium=sdk_python&utm_campaign=readme)
@@ -57,9 +78,6 @@ the account you sign in with. The other option there is IP whitelisting, which n credentials in the username at all; both are described in [authentication methods](https://docs.nodemaven.com/en/articles/9979031-authentication-methods). -Everything below except this first snippet builds strings offline and runs without an -account. - ```python import requests from nodemaven import Proxy @@ -71,6 +89,41 @@ r = requests.get("https://api.ipify.org", proxies=proxy.requests()) print(r.text) ``` + + +**No account? Any proxy you already have works.** A gateway is a data +description, not a code path, so one that ships no definition here goes through +the same builder and the same validation: + +```python +import requests +from nodemaven import Proxy, Provider + +# An empty known_params is not a stub. It says nobody has established what this +# gateway recognises, so every parameter is refused rather than sent to be +# silently dropped - see "Why the validation is the point" below. +mine = Provider(id="mine", label="My proxy", known_params=frozenset()) + +proxy = Proxy(provider=mine, login="your-login", password="your-password", + host="proxy.example.com", port=8000) + +r = requests.get("https://api.ipify.org", proxies=proxy.requests()) +print(r.text) +``` + +Describe the parameters it does take and it validates those too - see +[Other gateways](#other-gateways). Everything from here to the end of that +section builds strings offline and opens no socket at all. + Credentials can come from the environment instead, so nothing is in your source: ```python @@ -99,6 +152,56 @@ with sync_playwright() as p: context = browser.new_context(proxy=proxy.playwright()) ``` +## Parameters + + + +What the shipped NodeMaven definition accepts. Every name here was confirmed +against the gateway rather than transcribed: + +| parameter | what it selects | values measured to work | +|---|---|---| +| `country` | country code, or `any` | `us`, `de`, ... | +| `region` | area inside the country | a name | +| `city` | city inside the country | a name | +| `isp` | the exit's ISP | a name | +| `sid` | the sticky session - see below | any string with no `-` | +| `ttl` | how long that session is held | `10m`, `1m` | +| `filter` | IP quality | `low`, `medium`, `high` (2026-08-13) | +| `speed` | connection speed class | `fast`, `slow` (2026-08-12) | +| `ipv4` | force IPv4 | `True` / `False` | + +**Names are validated, values are not.** Passing a name that is not in this table +raises before anything is sent, because the gateway answers an unknown name with +200 and drops the setting. Values are passed through, because what is known is +which ones have been observed to work - and that is not the same as the set the +gateway accepts. Refusing on a guessed list would block a setting that would have +worked, which is the worse mistake of the two. + +Credentials come from `NODEMAVEN_LOGIN` and `NODEMAVEN_PASSWORD` when not passed +in, and the gateway address from `NODEMAVEN_HOST` and `NODEMAVEN_PORT`. + +## Errors + +All four inherit from `NodeMavenError`, so one `except` catches everything this +library raises. + +| exception | raised when | +|---|---| +| `ParamError` | a parameter name is unknown, a value is empty, or a value contains the gateway's separator | +| `CredentialsError` | no login, no password, or no gateway address, from arguments or environment | +| `ProviderError` | a gateway definition is missing, unreadable, or internally inconsistent | +| `NodeMavenError` | the base, never raised on its own | + +Nothing here is raised from a response, because nothing here sends one. Every +failure this library reports is a failure it found before a socket existed. + ## Sticky sessions One `Proxy` is one identity. Pin it to a sticky session: @@ -183,23 +286,62 @@ attempts per delivered page, against 1.7 in a healthy session. A library that shipped automatic retry as a default would be spending that on your behalf without telling you. + + +Those 1464 attempts, and the cells they came from, are in +[nodemaven/proxy-benchmark](https://github.com/nodemaven/proxy-benchmark) - the +harness that measured them, open source, so the table above can be re-run rather +than believed. + It also does not own an HTTP client, a connection pool or a browser. Those are yours, and they are better than anything a vendor SDK would bundle. ## Other gateways -Parameters are data, not hardcoded keywords. A gateway is one TOML file - its -prefix, separators, session parameter and the set of parameter names it actually -recognises - and a definition read from disk goes through the same builder and -the same validation as the one shipped here: +Parameters are data, not hardcoded keywords. A gateway is its prefix, +separators, session parameter and the set of parameter names it actually +recognises - and a definition written by you goes through the same builder and +the same validation as the one shipped here. Either build it in place, as in the +[quickstart](#quickstart), or keep it in a TOML file: + +```toml +# my-gateway.toml +label = "My proxy" +known_params = ["country", "session"] +session_param = "session" +host = "proxy.example.com" +port = 8000 +``` ```python from nodemaven import Proxy, load_file mine = load_file("my-gateway.toml") proxy = Proxy(provider=mine, login="u", password="p", country="us") +proxy.session("order4417") # u-country-us-session-order4417 ``` +`known_params` is the whole point of the file: name a parameter that is not in +it and the call raises instead of connecting. Leave the list empty and every +parameter is refused, which is the correct thing to say about a gateway whose +dialect nobody has established. + +Credentials fall back to the environment under the definition's id in upper case, +so this one reads `MY_GATEWAY_LOGIN` and `MY_GATEWAY_PASSWORD` and never +`NODEMAVEN_*`. One process can hold several gateways without their credentials +reaching each other. + +**The id comes from the filename, not from the variable you assign it to.** +`load_file("my-gateway.toml")` is `my-gateway` however it is named in your code, +and `-` becomes `_` in the variable names. Pass `provider_id=` to say it +outright. The error raised when a credential is missing prints the exact pair it +looked for, so this is one guess you never have to make. + Every definition carries a `status`. `measured` means traffic has gone through that gateway and the dialect was read off the wire. `documented` means it was transcribed from documentation and never exercised. Only `nodemaven` is shipped @@ -209,6 +351,17 @@ here, and it is `measured`. Python 3.9 or newer. No dependencies on 3.11 and newer; `tomli` on older ones. +## Changes + +[CHANGELOG.md](https://github.com/nodemaven/nodemaven-python/blob/main/CHANGELOG.md). +Entries carry the probe and the date behind any change to what the gateway is +believed to accept. + ## License -MIT. + + +[MIT](https://github.com/nodemaven/nodemaven-python/blob/main/LICENSE). diff --git a/pyproject.toml b/pyproject.toml index 94a5f5a..571d0a9 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -29,12 +29,21 @@ dependencies = ["tomli>=1.1.0; python_version < '3.11'"] [project.urls] # These render as the sidebar on the PyPI page, so every one of them has to answer -# 200 to a logged-out visitor. Checked 2026-08-25. -# `Source` is deliberately absent: nodemaven/nodemaven-python is internal and -# answers 404 anonymously, exactly as a repository that does not exist does. Add it -# in the same commit that makes the repository public, not before. +# 200 to a logged-out visitor. +# +# `Source` and `Issues` were held back while nodemaven/nodemaven-python was +# internal, because an internal repository answers 404 to an anonymous visitor +# exactly as a repository that does not exist does. They go in with the commit +# that makes it public and not before. +# +# **They will not appear on the package page that is up now.** PyPI bakes this +# table into each uploaded distribution and never re-reads the repository, so +# 0.1.2's sidebar is fixed whatever this file says. The links arrive with the +# next release; the visibility flip on its own does nothing to them. Homepage = "https://nodemaven.com" Documentation = "https://docs.nodemaven.com" +Source = "https://github.com/nodemaven/nodemaven-python" +Issues = "https://github.com/nodemaven/nodemaven-python/issues" [tool.setuptools.dynamic] version = { attr = "nodemaven.__version__" } diff --git a/src/nodemaven/providers.py b/src/nodemaven/providers.py index 0023247..f5f4356 100644 --- a/src/nodemaven/providers.py +++ b/src/nodemaven/providers.py @@ -87,6 +87,12 @@ def allowed(self, name: str) -> Optional[Tuple[str, ...]]: @property def is_measured(self) -> bool: + """Whether traffic has actually gone through this gateway. + + False covers both "transcribed from documentation" and any status + nobody has defined, which is the safe way round: a definition is + unverified until something says otherwise. + """ return self.status == "measured" diff --git a/src/nodemaven/proxy.py b/src/nodemaven/proxy.py index 050cbf8..bf5c7f6 100644 --- a/src/nodemaven/proxy.py +++ b/src/nodemaven/proxy.py @@ -103,6 +103,13 @@ def __init__( @property def provider(self) -> Provider: + """The gateway definition this identity was built against. + + Exposed because it answers the two questions a caller cannot otherwise + ask: ``provider.known_params`` is what this object will accept, and + ``provider.is_measured`` is whether anyone has confirmed the dialect + against the gateway or only read it out of documentation. + """ return self._provider @property From 49270e81da5a2fa34409d5e5a61c2bc79a506d05 Mon Sep 17 00:00:00 2001 From: aleekaz Date: Fri, 4 Sep 2026 16:39:39 +0300 Subject: [PATCH 2/2] Correct two counts the automated review caught Both are the same kind of mistake - a sentence that was true of an earlier draft and was not re-read after the draft grew. - The Errors table gave ParamError three triggers and there are more. A value outside a list the definition declares is refused too, and so are the two structural cases: session() on a definition with no session parameter, and a definition whose parameter names collide with the constructor's own. - "Names are validated, values are not" is true of the shipped gateway and false of the schema. The values key exists and is enforced; NodeMaven's definition leaves it empty on purpose, which is a fact about that definition rather than about the library. - The changelog said "all four" over a list of five, and the fifth was held back for a reason about a different repository: proxy-benchmark was internal until 2026-09-01, this one still is. --- CHANGELOG.md | 7 +++++-- README.md | 21 ++++++++++++++------- 2 files changed, 19 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index aa96aa1..6c75407 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,8 +12,11 @@ says so" is not one of those, and an entry that rests on it says so outright. - `Source` and `Issues` added to the package metadata, and the README gained a CI badge, a link to its own LICENSE and a link to the benchmark harness the - retry table is measured on. All four were held back while the repository was - internal, where they answered 404 to anyone not signed in. + retry table is measured on. That is five things and they were held back for + two versions of one reason: the first four point into this repository, which + is internal and answers 404 to anyone not signed in - the badge reads "repo or + workflow not found" for the same reason - and the fifth points at + `proxy-benchmark`, which was internal until 2026-09-01. - The quickstart now shows a live first request through **any** proxy, not only through NodeMaven's gateway. Nothing in the library changed - the capability was there and the README buried it in the last section. diff --git a/README.md b/README.md index bdfbe81..462c296 100644 --- a/README.md +++ b/README.md @@ -177,12 +177,15 @@ against the gateway rather than transcribed: | `speed` | connection speed class | `fast`, `slow` (2026-08-12) | | `ipv4` | force IPv4 | `True` / `False` | -**Names are validated, values are not.** Passing a name that is not in this table -raises before anything is sent, because the gateway answers an unknown name with -200 and drops the setting. Values are passed through, because what is known is -which ones have been observed to work - and that is not the same as the set the -gateway accepts. Refusing on a guessed list would block a setting that would have -worked, which is the worse mistake of the two. +**Names are validated. Values, on this gateway, are not.** Passing a name that is +not in this table raises before anything is sent, because the gateway answers an +unknown name with 200 and drops the setting. Values are passed through, because +what is known is which ones have been observed to work - and that is not the same +as the set the gateway accepts. Refusing on a guessed list would block a setting +that would have worked, which is the worse mistake of the two. The schema does +carry a per-parameter list of legal values and refuses anything outside it; the +shipped definition leaves that list empty for every parameter, deliberately, and +a definition you write yourself gets the check as soon as you fill it in. Credentials come from `NODEMAVEN_LOGIN` and `NODEMAVEN_PASSWORD` when not passed in, and the gateway address from `NODEMAVEN_HOST` and `NODEMAVEN_PORT`. @@ -194,11 +197,15 @@ library raises. | exception | raised when | |---|---| -| `ParamError` | a parameter name is unknown, a value is empty, or a value contains the gateway's separator | +| `ParamError` | a parameter name is unknown, a value is empty, a value contains the gateway's separator, or a value is outside a list the definition declares | | `CredentialsError` | no login, no password, or no gateway address, from arguments or environment | | `ProviderError` | a gateway definition is missing, unreadable, or internally inconsistent | | `NodeMavenError` | the base, never raised on its own | +`ParamError` also covers the two structural cases: `session()` on a definition +that declares no session parameter, and a definition whose parameter names +collide with `login`, `password`, `host`, `port` or `provider`. + Nothing here is raised from a response, because nothing here sends one. Every failure this library reports is a failure it found before a socket existed.