Bump ruff from 0.15.22 to 0.16.0 in the python-dev-tools group

docs/memory/log/2026-08-07-ruff-defaults.md

@@ -0,0 +1,26 @@
+---
+title: Adopt ruff's default rule set via ruff.toml
+summary: ruff 0.16 expanded its implicit defaults from 59 rules to 413; ruff.toml now records the rule set explicitly
+created: 2026-08-07
+author: Eric Case
+tags: [lint, ruff, ci, conventions]
+---
+
+# Adopt ruff's default rule set via ruff.toml (2026-08-07)
+
+The repo had no `[tool.ruff]` section anywhere, so lint policy was whatever ruff's implicit defaults happened to be. Ruff [0.16.0](https://astral.sh/blog/ruff-v0.16.0) expanded those defaults from 59 rules (`E4`, `E7`, `E9`, `F`) to 413, and the Dependabot bump surfaced 72 violations.
+
+The decision is to **adopt the new defaults** rather than pin back to the old set. `ruff.toml` now exists to make the rule set an explicit, reviewable choice.
+
+Deviations recorded in `ruff.toml`:
+
+- **`BLE001` ignored repo-wide.** Twelve sites in `download.py`, `content_changed.py`, `tlds.py`, and `parse/root_db_html.py` catch `Exception` at an I/O or parse boundary, log it, and degrade to a defined fallback (`"error"`, treat-as-changed, omit optional field). That is the pipeline's deliberate resilience convention: one malformed source file must not abort a run. Narrowing all twelve would mean enumerating exception types for `json.load`, `.decode("idna")`, and httpx, where guessing wrong turns a logged degradation into a crash. A thirteenth site was flagged by both `BLE001` and `S110` and is fixed below rather than ignored.
+
+Nothing else is ignored. The remaining 59 violations were fixed, notably:
+
+- **`S110` in `build/tlds.py`** was the one genuine outlier: a silent, unlogged `except Exception: pass` around the IDN punycode decode, three lines from twelve siblings that all log. Narrowed to `except UnicodeError` (covers both `UnicodeDecodeError` from bad punycode and `UnicodeEncodeError` from non-ASCII input) and given a `logger.warning`.
+- **`EXE001` in `src/cli.py`**: the `#!/usr/bin/env python3` shebang was vestigial. Every caller uses `python -m src.cli` (Makefile, `bin/build`), so the shebang was removed rather than the file made executable.
+- **`PLW1510`** (7 sites): tests that deliberately assert on `returncode` now pass `check=False` explicitly.
+- **`TRY002`** (6 sites in `tests/utilities/test_download.py`): mock guards for unexpected URLs became `AssertionError`; simulated transport failures became `RuntimeError`. Both still exercise the same `except Exception` production paths.
+
+`target-version` is not set in `ruff.toml`. Ruff still infers 3.13 from `requires-python` in `pyproject.toml` even when config resolves from a standalone `ruff.toml` (verified with `ruff check --show-settings`). This matters because `UP017` (`datetime.UTC`) is 3.11+ only; a mis-inferred target would silently drop 15 of the findings.

docs/memory/memory-index.md

@@ -9,6 +9,7 @@ Agents: consult before suggesting layout, naming, dependencies, vendors, or conv
 - [Architecture](architecture.md) - current implementation: stack, layout, conventions
 
 ## Log (newest first)
+- [2026-08-07 Adopt ruff default rules](log/2026-08-07-ruff-defaults.md) - ruff 0.16 grew its implicit defaults from 59 rules to 413 and broke CI; `ruff.toml` now records the rule set explicitly. Only `BLE001` is ignored (boundary handlers that log and degrade on purpose); the other 59 violations were fixed.
 - [2026-07-16 iptoasn AS drift (teleinfo + verisign)](log/2026-07-16-teleinfo-asn-rename.md) - one refresh window renamed CAICT's AS (teleinfo -> "CAICT-AS-AP...") and migrated Verisign's .com/.net nameservers off "VERISIGN-AS" onto VRSN-AC28/VRSN-AC50-340, stranding both asn seeds; fix is data (reseed to live string, retire departed one to aliases). Reproduce against CI's pinned update-iptoasn artifact via gh run download, NOT iptoasn.com live-latest (which drifts ahead of CI)
 - [2026-06-07 Nightly rebuild picks up manual curation](log/2026-06-07-nightly-manual-regen.md) - update-data.yaml always rebuilds (--all on IANA source change, else --preserve-asn) so data/manual/ edits propagate to data/generated/ nightly without ASN churn; was gated on data/source/ only. Keep local `iptoasn` fresh or local passes while CI (daily artifact) fails
 - [2026-06-07 Verisign per-instance ASN block](log/2026-06-07-verisign-asn-block.md) - AS36616-36632 are all Verisign, one ASN per authoritative-server instance ([letter]GTLD for gtld-servers.net letters a-m plus X/Y/Z, AROOT for the roots it runs); each is a separate opaque as_org string, folded into the one VeriSign record as it appears (HGTLD/AS36623 added; other siblings not yet in data)

pyproject.toml

@@ -22,7 +22,7 @@ dev = [
     "pyright>=1.1.411",
     "pytest>=9.1.1",
     "pytest-cov>=7.1.0",
-    "ruff>=0.15.22",
+    "ruff>=0.16.0",
 ]
 
 [tool.pytest.ini_options]

ruff.toml

@@ -0,0 +1,6 @@
+[lint]
+ignore = [
+    # I/O and parse boundaries deliberately catch Exception, log it, and degrade
+    # to a defined fallback so one bad source file cannot abort a pipeline run.
+    "BLE001",
+]

src/analyze/__init__.py

@@ -5,8 +5,8 @@
 from .tlds_txt import analyze_tlds_txt, get_tlds_analysis
 
 __all__ = [
+    "analyze_rdap_json",
+    "analyze_root_db_html",
     "analyze_tlds_txt",
     "get_tlds_analysis",
-    "analyze_root_db_html",
-    "analyze_rdap_json",
 ]

src/analyze/root_db_html.py

@@ -51,23 +51,23 @@ def analyze_root_db_html(filepath: Path) -> int:
         delegated_idn_by_type[entry["type"]] += 1
 
     # Count unique managers for delegated TLDs
-    unique_managers = set(entry["manager"] for entry in delegated_entries)
+    unique_managers = {entry["manager"] for entry in delegated_entries}
     total_unique_managers = len(unique_managers)
 
     # Count unique gTLD managers (generic types)
-    gtld_managers = set(
+    gtld_managers = {
         entry["manager"]
         for entry in delegated_entries
         if entry["type"] in generic_types
-    )
+    }
     total_unique_gtld_managers = len(gtld_managers)
 
     # Count unique ccTLD managers (country-code)
-    cctld_managers = set(
+    cctld_managers = {
         entry["manager"]
         for entry in delegated_entries
         if entry["type"] == "country-code"
-    )
+    }
     total_unique_cctld_managers = len(cctld_managers)
 
     # Report results

src/build/agreements.py

@@ -6,7 +6,7 @@
 """
 
 import logging
-from datetime import datetime, timezone
+from datetime import UTC, datetime
 from pathlib import Path
 
 from ..config import REGISTRY_AGREEMENT_TYPE_MAPPING
@@ -64,7 +64,7 @@ def build_agreements_json(tlds: list[dict], output_path: Path) -> tuple[bool, st
 
     output = {
         "description": _DESCRIPTION,
-        "publication": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
+        "publication": datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%SZ"),
         "sources": _SOURCES,
         "agreements": agreements,
     }

src/build/cultures.py

@@ -5,7 +5,7 @@
 """
 
 import logging
-from datetime import datetime, timezone
+from datetime import UTC, datetime
 from pathlib import Path
 
 from ..utilities.content_changed import write_json_if_changed
@@ -53,7 +53,7 @@ def build_cultures_json(
 
     output = {
         "description": _DESCRIPTION,
-        "publication": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
+        "publication": datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%SZ"),
         "sources": _SOURCES,
         "cultures": cultures,
     }

src/build/organizations.py

@@ -5,7 +5,7 @@
 """
 
 import logging
-from datetime import datetime, timezone
+from datetime import UTC, datetime
 from pathlib import Path
 
 from ..parse.organizations import OrgRecord, OrgResolver
@@ -64,7 +64,7 @@ def build_organizations_json(
 
     output = {
         "description": _DESCRIPTION,
-        "publication": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
+        "publication": datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%SZ"),
         "sources": _SOURCES,
         "orgs": out_orgs,
     }

src/build/places.py

@@ -10,7 +10,7 @@
 """
 
 import logging
-from datetime import datetime, timezone
+from datetime import UTC, datetime
 from pathlib import Path
 
 import pycountry
@@ -89,7 +89,7 @@ def build_places_json(
 
     output = {
         "description": _DESCRIPTION,
-        "publication": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
+        "publication": datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%SZ"),
         "sources": _SOURCES,
         "places": records,
     }

src/build/tlds.py

@@ -4,7 +4,7 @@
 import json
 import logging
 from dataclasses import dataclass
-from datetime import datetime, timezone
+from datetime import UTC, datetime
 from pathlib import Path
 from typing import Any
 
@@ -199,7 +199,7 @@ def build_tlds_json(
 
     # Build-time stamp passed to every write; write_json_if_changed skips
     # unchanged artifacts, so per-file publication reads as "last changed".
-    publication = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
+    publication = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%SZ")
     sources = {
         "iana_root_db": IANA_URLS["ROOT_ZONE_DB"],
         "iana_rdap": IANA_URLS["RDAP_BOOTSTRAP"],
@@ -458,9 +458,8 @@ def _build_tld_entry(
         try:
             tld_unicode = tld.encode("ascii").decode("idna")
             entry["tld_unicode"] = tld_unicode
-        except Exception:
-            # If decoding fails, skip unicode field
-            pass
+        except UnicodeError as e:
+            logger.warning("Could not decode IDN %s, omitting tld_unicode: %s", tld, e)
 
     # Add tld_script for IDNs
     if tld in idn_script_mapping:

src/cli.py

@@ -1,4 +1,3 @@
-#!/usr/bin/env python3
 """Command-line interface for IANA data ETL."""
 
 import argparse

src/parse/__init__.py

@@ -24,27 +24,27 @@
 __all__ = [
     "ASNLookup",
     "ASNRecord",
-    "parse_iptoasn_tsv",
     "GtldRecord",
-    "parse_gtlds_json",
-    "parse_manual_annotations",
     "OrgResolver",
     "build_resolver",
-    "parse_organizations_manual",
-    "parse_tlds_txt",
-    "tlds_txt_content_changed",
-    "parse_root_db_html",
-    "parse_root_db_tlds",
-    "root_db_html_content_changed",
     "derive_type_from_iana_tag",
-    "parse_rdap_json",
-    "rdap_json_content_changed",
-    "parse_supplemental_cctld_rdap",
-    "parse_registry_agreement_csv",
-    "parse_agreement_types",
-    "get_normalized_agreement_types",
     "extract_main_content",
+    "get_all_country_mappings",
     "get_country_name",
+    "get_normalized_agreement_types",
     "is_cctld",
-    "get_all_country_mappings",
+    "parse_agreement_types",
+    "parse_gtlds_json",
+    "parse_iptoasn_tsv",
+    "parse_manual_annotations",
+    "parse_organizations_manual",
+    "parse_rdap_json",
+    "parse_registry_agreement_csv",
+    "parse_root_db_html",
+    "parse_root_db_tlds",
+    "parse_supplemental_cctld_rdap",
+    "parse_tlds_txt",
+    "rdap_json_content_changed",
+    "root_db_html_content_changed",
+    "tlds_txt_content_changed",
 ]

src/utilities/__init__.py

@@ -16,8 +16,8 @@
     "download_iptoasn",
     "download_tld_pages",
     "get_iptoasn_path",
+    "is_cache_fresh",
     "load_metadata",
-    "save_metadata",
     "parse_cache_control_max_age",
-    "is_cache_fresh",
+    "save_metadata",
 ]

src/utilities/cache.py

@@ -1,7 +1,7 @@
 """Cache utilities for HTTP responses."""
 
 import re
-from datetime import datetime, timezone
+from datetime import UTC, datetime
 from typing import Any
 
 
@@ -43,6 +43,6 @@ def is_cache_fresh(metadata_entry: dict[str, Any]) -> bool:
     max_age = int(cache_data["cache_max_age"])
 
     # Calculate age in seconds
-    age = (datetime.now(timezone.utc) - download_time).total_seconds()
+    age = (datetime.now(UTC) - download_time).total_seconds()
 
     return age < max_age

src/utilities/download.py

@@ -2,8 +2,8 @@
 
 import logging
 import time
+from collections.abc import Callable
 from pathlib import Path
-from typing import Callable
 
 import httpx
 
@@ -17,8 +17,8 @@
 )
 from .cache import is_cache_fresh, parse_cache_control_max_age
 from .metadata import load_metadata, save_metadata, utc_timestamp
-from .urls import get_tld_file_path, get_tld_page_url
 from .retry import make_request_with_retry
+from .urls import get_tld_file_path, get_tld_page_url
 
 logger = logging.getLogger(__name__)
 

src/utilities/metadata.py

@@ -2,7 +2,7 @@
 
 import json
 import logging
-from datetime import datetime, timezone
+from datetime import UTC, datetime
 from pathlib import Path
 from typing import Any, Final
 
@@ -15,7 +15,7 @@ def utc_timestamp() -> str:
     Returns:
         Timestamp string in format: 2025-11-18T20:23:07Z
     """
-    return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
+    return datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%SZ")
 
 
 GENERATED_DIR: Final[str] = "data/generated"

src/utilities/retry.py

@@ -18,8 +18,6 @@
 class ServerError(Exception):
     """Raised when server returns 5xx error, triggering retry."""
 
-    pass
-
 
 def make_request_with_retry(
     client: httpx.Client,

tests/build/test_idn_script_field.py

@@ -60,7 +60,7 @@ def test_idn_script_mapping_file_format():
     assert len(mappings) > 0
 
     # Check that all keys are IDN TLDs
-    for tld in mappings.keys():
+    for tld in mappings:
         assert tld.startswith("xn--"), f"Non-IDN TLD in mapping: {tld}"
 
     # Check that all values are strings

tests/build/test_tlds.py

@@ -450,7 +450,7 @@ def test_build_tlds_json_gtld_no_country_name(shared_build):
 
     for gtld in test_gtlds:
         if gtld in gtld_tlds:
-            entry = [e for e in gtlds if e["tld"] == gtld][0]
+            entry = next(e for e in gtlds if e["tld"] == gtld)
             if "annotations" in entry:
                 assert "country_name_iso" not in entry["annotations"]
 

tests/parse/test_country.py

@@ -105,7 +105,7 @@ def test_mappings_have_country_names(self):
         tlds = parse_tlds_txt(fixture_path)
         mappings = get_all_country_mappings(tlds)
 
-        for code, name in mappings.items():
+        for name in mappings.values():
             assert isinstance(name, str)
             assert len(name) > 0
 

tests/parse/test_root_db_html.py

@@ -125,16 +125,16 @@ def test_parse_root_db_html_unique_managers():
     delegated = [e for e in entries if e.get("delegated", True)]
 
     # Count unique managers (excluding "Not assigned")
-    unique_managers = set(e["manager"] for e in delegated)
+    unique_managers = {e["manager"] for e in delegated}
     assert len(unique_managers) == 28
 
     # Count unique gTLD managers
     generic_types = ["generic", "sponsored", "infrastructure", "generic-restricted"]
-    gtld_managers = set(e["manager"] for e in delegated if e["type"] in generic_types)
+    gtld_managers = {e["manager"] for e in delegated if e["type"] in generic_types}
     assert len(gtld_managers) == 18
 
     # Count unique ccTLD managers
-    cctld_managers = set(e["manager"] for e in delegated if e["type"] == "country-code")
+    cctld_managers = {e["manager"] for e in delegated if e["type"] == "country-code"}
     assert len(cctld_managers) == 10
 
     # Verify it's counting unique managers, not total TLDs

tests/parse/test_supplemental_cctld_rdap.py

@@ -77,6 +77,6 @@ def test_parse_supplemental_cctld_rdap_default_path():
     # If file exists, should have actual data
     if rdap_lookup:
         # Check structure of first entry
-        first_tld = list(rdap_lookup.keys())[0]
+        first_tld = next(iter(rdap_lookup.keys()))
         assert "rdap_server" in rdap_lookup[first_tld]
         assert "source" in rdap_lookup[first_tld]

tests/test_circular_imports.py

@@ -36,6 +36,7 @@ def test_no_circular_imports_with_pydeps():
         capture_output=True,
         text=True,
         timeout=30,
+        check=False,
     )
 
     # pydeps outputs cycle information to stdout/stderr
@@ -72,6 +73,7 @@ def test_pydeps_is_installed():
         [sys.executable, "-m", "pydeps", "--version"],
         capture_output=True,
         text=True,
+        check=False,
     )
 
     assert result.returncode == 0, "pydeps not installed. Run: uv add --dev pydeps"