Skip to content

test: 16 pre-existing failures on main — ruamel.yaml dep + bundled-catalog/fixture drift #229

@typelicious

Description

@typelicious

Two unrelated baseline failures, both pre-existing on main (verified by stash-popping PR #226 changes)

$ pytest tests/ --ignore=tests/test_wizard.py -q -k "not benchmark"
...
16 failed, 446 passed, 6 deselected, 1 warning in 103.41s

Cluster A — ruamel.yaml missing on Python 3.14 (11 failures)

All 11 failures come from tests/test_dashboard_settings.py. They die with:

ModuleNotFoundError: No module named 'ruamel'
  File ".../faigate/dashboard_settings.py", line 122, in set_default_view
    from ruamel.yaml import YAML

ruamel.yaml >= 0.18.6 is in pyproject.toml and requirements.txt (added in v2.3.0 for the dashboard-settings comment-preserving YAML round-trip). The Python 3.14 wheel resolution path appears to skip it on a fresh pip install -e . — possibly a marker/extras issue or the upstream wheel coverage for py3.14 lagging.

Failing tests:

  • TestGetSettings::test_cockpit_view_has_no_pinned_slug
  • TestGetSettings::test_bad_stored_value_degrades_to_overview
  • TestSetDefaultViewRoundTrip::* (4 tests)
  • TestSettingsApi::test_post_overview / test_post_brand_persists_and_echoes / test_post_cockpit
  • TestDashboardRedirect::test_brand_default_redirects / test_cockpit_default_redirects_offsite

Investigation hint: check whether pyproject.toml needs a python_requires upper bound, an explicit ;python_version<"3.14" marker, or a fallback path in dashboard_settings.py that uses stdlib yaml when ruamel.yaml is unavailable (loses comment preservation but keeps the API working).

Cluster B — bundled catalog.v1.json already V4-aware vs. test fixtures still using deepseek-chat (3 failures)

FAILED tests/test_provider_catalog.py::test_offerings_and_packages_catalog_loading
FAILED tests/test_router_offerings_packages.py::test_pricing_lookup_prefers_offerings
FAILED tests/test_router_offerings_packages.py::test_router_uses_offerings_for_cost_calculation

faigate/assets/metadata/catalog.v1.json was updated (likely by #225 / #223 metadata sync) to map "deepseek-chat""recommended_model": "deepseek-v4-flash". The above tests construct synthetic offerings catalogs in tmp_path/ keyed by deepseek-chat / deepseek/chat and expect their fixture pricing to win — but the bundled catalog now resolves the lookup to a different entry first.

Fix paths:

  • Option 1: Rename the synthetic provider names in the fixtures to something that doesn't collide with the bundled catalog (test-deepseek-flash, etc.).
  • Option 2: Add a test fixture that clears or shadows the bundled catalog inside _get_pricing_for_provider_and_model for the duration of these tests.
  • Option 3: Update the assertions to match the new bundled-catalog-aware lookup behavior — but only if the new behavior is the one we want to lock in.

PR #226 explicitly did NOT touch these tests because their failure mode pre-dates the V4 migration that PR carries.

Why one issue for two clusters

Both are silent baseline noise that hides real regressions: a contributor running pytest on a fresh checkout sees 16 failures and can't tell which (if any) are theirs. Bundling them into one cleanup issue keeps the "is the suite green?" question binary.

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdocumentationImprovements or additions to documentation

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions