Split unit and integration test jobs

[mendral-app]

Summary

• Split the single tests job into unit-tests (required gate) and integration-tests (non-blocking) to stop integration test flakiness from blocking merges on main. The tests job was failing ~85% of pushes due to integration tests hitting live external services (staging API regressions, e-commerce page changes, browser OOM crashes).
• unit-tests runs all tests except tests/integration/, tests/browser/, tests/chapter/, tests/examples/, and tests/mcp/ — these are fast, deterministic tests with no external dependencies. Pre-commit checks and coverage reporting remain here.
• integration-tests runs tests/integration, tests/browser, and tests/chapter with continue-on-error: true and --reruns 3 for flaky test resilience. Failures no longer block merges.

Important

Manual Steps Required Before Merging

• Update branch protection rules: if the tests job is currently a required status check, replace it with unit-tests (the new required gate job name)

Context

Addresses insight: tests job on main failing ~85% of pushes. Three failure modes were identified:

1. xdist worker OOM crashes (Chromium subprocess killed)
2. Individual external-service flakiness (staging API 502s, page content changes)
3. Staging API regressions causing bulk failures (e.g., all 9 test_personas.py tests)

None of these affect unit tests — splitting the jobs isolates the required CI gate from external service instability.

Changes

File	Change
.github/workflows/test-cicd.yml	Split tests → unit-tests + integration-tests jobs
pyproject.toml	Add unit and integration pytest markers

---

Note

Created by Mendral. Tag @mendral-app with feedback or questions.

• View implementation session
• Addresses this insight

[greptile-apps]

Greptile Summary

This PR splits the single tests CI job into unit-tests (required gate, fast/deterministic) and integration-tests (non-blocking, continue-on-error: true) to prevent external-service flakiness from blocking merges. The split is clean: directory-based ignores correctly partition the test suite, pytest-rerunfailures is already a project dependency so --reruns 3 works out of the box, and tests/mcp and tests/examples are correctly included in the integration job.

Confidence Score: 5/5

Safe to merge after updating the required branch protection status check from tests to unit-tests.

Only P2 findings (missing artifact upload for integration results). No logic errors, security issues, or correctness problems in the split. The critical manual step (updating branch protection rules) is already called out in the PR description.

No files require special attention beyond the manual branch-protection update noted in the PR description.

Important Files Changed

Filename	Overview
.github/workflows/test-cicd.yml	Splits tests into unit-tests (required gate) and integration-tests (non-blocking, continue-on-error); integration test results are not uploaded as artifacts.
pyproject.toml	Adds unit and integration pytest marker definitions to suppress PytestUnknownMarkWarning; no other changes.

Prompt To Fix All With AI

This is a comment left during a code review.
Path: .github/workflows/test-cicd.yml
Line: 216-225

Comment:
**No artifact upload for integration test results**

`pytest-integration.xml` and `pytest-integration.txt` are generated but never uploaded. Since integration tests are expected to fail regularly and `continue-on-error: true` hides the failure from the status check UI, there's no easy way to inspect results without digging through raw job logs. Consider adding an artifact upload step so failures remain accessible.

```yaml
      - name: Upload integration test results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: integration-test-results
          path: |
            pytest-integration.xml
            pytest-integration.txt
          retention-days: 7
```

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (2): Last reviewed commit: "ci: add tests/mcp and tests/examples to ..." | Re-trigger Greptile}

[greptile-apps]

tests/mcp and tests/examples silently dropped from CI

unit-tests excludes tests/mcp and tests/examples, but integration-tests only runs tests/integration tests/browser tests/chapter. Neither job will execute tests/mcp or tests/examples, so any regressions in those areas will go completely undetected. The original tests job previously ran tests/mcp (it was not in the ignore list), so this is a regression in coverage.

Add the missing directories to the integration-tests run command (or add them back to unit-tests if they have no external dependencies):

Suggested change

	uv run pytest -n logical \
	tests/integration tests/browser tests/chapter \
	--ignore=tests/integration/test_webvoyager_resolution.py \
	--ignore=tests/integration/test_e2e.py \
	--reruns 3 --reruns-delay 5 \
	--durations=10 --junitxml=pytest-integration.xml \
	| tee pytest-integration.txt
	uv run pytest -n logical \
	tests/integration tests/browser tests/chapter tests/mcp tests/examples \
	--ignore=tests/integration/test_webvoyager_resolution.py \
	--ignore=tests/integration/test_e2e.py \
	--reruns 3 --reruns-delay 5 \
	--durations=10 --junitxml=pytest-integration.xml \
	| tee pytest-integration.txt

Prompt To Fix With AI

This is a comment left during a code review.
Path: .github/workflows/test-cicd.yml
Line: 219-225

Comment:
**`tests/mcp` and `tests/examples` silently dropped from CI**

`unit-tests` excludes `tests/mcp` and `tests/examples`, but `integration-tests` only runs `tests/integration tests/browser tests/chapter`. Neither job will execute `tests/mcp` or `tests/examples`, so any regressions in those areas will go completely undetected. The original `tests` job previously ran `tests/mcp` (it was not in the ignore list), so this is a regression in coverage.

Add the missing directories to the integration-tests run command (or add them back to unit-tests if they have no external dependencies):

```suggestion
          uv run pytest -n logical \
            tests/integration tests/browser tests/chapter tests/mcp tests/examples \
            --ignore=tests/integration/test_webvoyager_resolution.py \
            --ignore=tests/integration/test_e2e.py \
            --reruns 3 --reruns-delay 5 \
            --durations=10 --junitxml=pytest-integration.xml \
            | tee pytest-integration.txt
```

How can I resolve this? If you propose a fix, please make it concise.

[mendral-app]

Good catch — both tests/mcp and tests/examples require external dependencies (MCP server subprocess, browser automation, live API calls, example script execution), so they belong in the integration-tests job. Fixed in 7a6ac8b by adding them to the integration-tests pytest run command.