feat(error-handler): add ErrorHandlerInterceptor plugin

.github/workflows/split-publish.yml

@@ -30,6 +30,7 @@ on:  # yamllint disable-line rule:truthy
       - 'convention-[0-9]*'
       - 'data-[0-9]*'
       - 'facade-[0-9]*'
+      - 'error-handler-[0-9]*'
       - 'filter-[0-9]*'
       - 'inline-[0-9]*'
       - 'lifecycle-[0-9]*'

bridge/rector/FEATURE_PARITY.md

@@ -31,6 +31,7 @@ Conversion coverage across the three directions supported by `testo/bridge-recto
 | **ExpectNoAssertions** (`#[\Testo\Assert\ExpectNoAssertions]` ↔ `#[\PHPUnit\Framework\Attributes\DoesNotPerformAssertions]`) | ✅ *`ExpectNoAssertionsToPhpUnitRector` (attribute rename; both sides method/function-level only — no fan-out)* | ✅ *`DoesNotPerformAssertionsToTestoRector` (attribute rename)* | ➖ |
 | **Mocks** (`createMock`/`createStub` + `expects`/`method`/`will*`/`with`) | ➖ | 🟡 *`CreateMockToDoubleRector` converts onto the Double bridge (`testo/bridge-double`): `createMock`/`createStub` → `Double::for`, `create{Mock,Stub}ForIntersectionOfInterfaces([A, B])` → `Double::for(A, B)`, and the configuration chain onto `expects`/`allows`/`with`/`returns`/`throws`/`resolves` — the invocation matcher moves onto the verb (`once`→`times(1)`, `exactly`→`times`, `never`→`never`, `atLeastOnce`/`atLeast`/`atMost`→`times(minimum:/maximum:)`, `any`→`allows`), the method name off `->method()` onto `expects('m')`, `withAnyParameters()` drops away, the returns (incl. `willReturnArgument($n)`→`resolves(fn (...$a) => $a[$n])`, `willReturnSelf()`→`returns(<the double>)`, and legacy `will(...)` incl. `onConsecutiveCalls`/`returnArgument`/`returnSelf`), the builder chain `getMockBuilder(X)->disableOriginalConstructor()->getMock()`→`Double::for(X)`, and `with()` constraints onto `Argument::*` (`anything`→`any`, `identicalTo`→`same`, `isInstanceOf`/`isType`→`type`, `callback`→`satisfies`, `contains`→`contains`, `matchesRegularExpression`→`matches`; `equalTo($x)`→bare `$x`, `isNull`/`isTrue`/`isFalse`→`null`/`true`/`false`; `greaterThan`/`lessThan`/`…OrEqual`, `isEmpty`, `stringContains`, `stringStartsWith`/`EndsWith`, `arrayHasKey`→`satisfies(fn ($value) => …)`; `logicalNot`→`Argument::not(...)`, `logicalOr`→`Argument::any(...)`). All-or-nothing per chain: `willReturnMap`, a variable matcher, a builder step beyond `disableOriginalConstructor` (or the bare constructor-calling `getMockBuilder(X)->getMock()`), `prophesize`, and `with()` constraints with no faithful form (`logicalAnd`, `equalToWithDelta`/`equalToCanonicalizing`, case-insensitive `stringContains`) leave the statement untouched — see `MockToTestoRector` (stub) and TODO.md* | ➖ |
 | **Memory-leak expectations** | ⛔ *no PHPUnit equivalent* | ➖ | ➖ |
+| **Error-handler declarations** (`#[ExpectErrorHandlerChange]` ↔ `#[WithoutErrorHandler]`) | ⛔ *`ExpectErrorHandlerChangeRector` stub: PHPUnit always flags a test that leaves a handler on the stack as risky and has no attribute to waive that* | ⛔ *`WithoutErrorHandlerRector` stub: Testo core installs no error handler and the `testo/error-handler` plugin has no per-test opt-out; the two attributes mean different things* | ➖ |
 | **Retry / Repeat** (`#[Retry]`/`#[Repeat]`) | 🟡 *`RepeatRetryRector` converts `#[\Testo\Repeat]`/`#[\Testo\Retry]` → PHPUnit `#[Repeat]`/`#[Retry]` (PHPUnit 13.3+): `maxFailures`→`failureThreshold` (+1), Testo defaults made explicit. PHPUnit's are `TARGET_METHOD` only, so a class-level Testo attribute is fanned out onto each test method (a method's own attribute overrides it, not doubled); `markFlaky` is dropped (no PHPUnit equivalent)* | 🟡 *`RepeatRetryToTestoRector` converts `#[Repeat]`/`#[Retry]` → Testo's attributes: `failureThreshold`→`maxFailures` (−1; the default 1 folds to Testo's default 0 and is omitted)* | ➖ |
 | **Fiber** (`#[RunInFiber]`, `Coroutine::spawn/await/concurrently`) | ⛔ *no PHPUnit/Pest equivalent — neither has a fiber/coroutine test attribute or an in-test coroutine scope* | ➖ | ➖ |
 | **HTML report** (`HtmlPlugin`, `--log-html`) | ⛔ *not test code — a reporter configured in `testo.php` or by a flag, with nothing in a test file to convert* | ➖ | ➖ |
@@ -103,7 +104,7 @@ attributes / body statements. It bails (leaves the statement untouched) on a non
 a `use (...)`-capturing closure, or any unrecognised modifier — see `src/PestToTesto/TODO.md`.
 
 The remaining ⛔ rows are intentionally out of scope: a missing target feature (`arch()`,
-memory-leak, PHPUnit `assertThat` constraints), the substring-vs-regex
+memory-leak, the error-handler declarations, PHPUnit `assertThat` constraints), the substring-vs-regex
 exception-message mismatch, or Pest `uses()` (a function has no base class / traits / `$this`).
 Mocks moved off this list: with the Double bridge there is now a target API, so `createMock`/
 `createStub` (and intersection mocks), their `expects`/`method`/`will*` chains, and `with()`

bridge/rector/src/PhpunitToTesto/TODO.md

@@ -20,6 +20,10 @@ exist for each so the intent and blockers are discoverable in code.
   constraint objects (and composites/callbacks) with no Testo equivalent.
 - **ExpectExceptionMessageMatchesRector** — regex message matching; Testo's
   `withMessage()` does literal matching, not PCRE, so conversion would change meaning.
+- **WithoutErrorHandlerRector** — `#[WithoutErrorHandler]` opts one test out of PHPUnit's own
+  error handler; Testo core installs none and the `testo/error-handler` plugin has no per-test
+  opt-out. `#[\Testo\ErrorHandler\ExpectErrorHandlerChange]` is not a counterpart (it declares a
+  deliberate handler-stack change), so the attribute is left for manual handling.
 
 ## Implemented since the first cut
 

bridge/rector/src/PhpunitToTesto/WithoutErrorHandlerRector.php

@@ -0,0 +1,44 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Testo\Bridge\Rector\PhpunitToTesto;
+
+use PhpParser\Node;
+use Rector\Rector\AbstractRector;
+use Symplify\RuleDocGenerator\ValueObject\RuleDefinition;
+
+/**
+ * STUB — not implemented, not registered in the set.
+ *
+ * Intent: convert `#[\PHPUnit\Framework\Attributes\WithoutErrorHandler]` into a Testo equivalent.
+ *
+ * @todo No faithful Testo equivalent exists. The attribute tells PHPUnit not to install its own
+ *   error handler for one test. Testo core installs no error handler at all, and the opt-in
+ *   `testo/error-handler` plugin applies per suite, with no per-test opt-out. In a project without
+ *   the plugin the attribute is a no-op and could be dropped; in a project with it there is nothing
+ *   to map to, and `#[\Testo\ErrorHandler\ExpectErrorHandlerChange]` means something else (the test
+ *   changes the handler stack on purpose). Left unconverted for manual handling.
+ */
+final class WithoutErrorHandlerRector extends AbstractRector
+{
+    public function getRuleDefinition(): RuleDefinition
+    {
+        return new RuleDefinition(
+            'STUB: PHPUnit #[WithoutErrorHandler] (per-test opt-out of the runner error handler) has no faithful Testo equivalent (not implemented)',
+            [],
+        );
+    }
+
+    #[\Override]
+    public function getNodeTypes(): array
+    {
+        return [];
+    }
+
+    #[\Override]
+    public function refactor(Node $node): ?Node
+    {
+        return null;
+    }
+}

bridge/rector/src/TestoToPhpunit/ExpectErrorHandlerChangeRector.php

@@ -0,0 +1,45 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Testo\Bridge\Rector\TestoToPhpunit;
+
+use PhpParser\Node;
+use Rector\Rector\AbstractRector;
+use Symplify\RuleDocGenerator\ValueObject\RuleDefinition;
+
+/**
+ * STUB — not implemented, not registered in the set.
+ *
+ * Intent: convert `#[\Testo\ErrorHandler\ExpectErrorHandlerChange]` into a PHPUnit equivalent.
+ *
+ * @todo No faithful PHPUnit equivalent exists. The attribute declares that a test installs an
+ *   error handler and leaves it in place, so the error-handler plugin keeps the test out of
+ *   `Risky`. PHPUnit performs the same stack check unconditionally and has no attribute that
+ *   waives it: a test leaving a handler behind is always "did not remove its own error handlers".
+ *   `#[WithoutErrorHandler]` is not that switch — it only stops PHPUnit from installing its own
+ *   handler for the test. Dropping the Testo attribute would silently turn a declared, passing
+ *   test into a risky one, so it is left unconverted for manual handling.
+ */
+final class ExpectErrorHandlerChangeRector extends AbstractRector
+{
+    public function getRuleDefinition(): RuleDefinition
+    {
+        return new RuleDefinition(
+            'STUB: Testo #[ExpectErrorHandlerChange] (declared error-handler change) has no faithful PHPUnit equivalent (not implemented)',
+            [],
+        );
+    }
+
+    #[\Override]
+    public function getNodeTypes(): array
+    {
+        return [];
+    }
+
+    #[\Override]
+    public function refactor(Node $node): ?Node
+    {
+        return null;
+    }
+}

bridge/rector/src/TestoToPhpunit/TODO.md

@@ -9,6 +9,10 @@ directory but are **not** registered in `config/testo-to-phpunit.php`.
   memory-leak assertion or post-test object-liveness hook.
 - **`CancelTestRector`** — `CancelTest` is an external interruption signal with no
   PHPUnit equivalent (not the same as skip/incomplete).
+- **`ExpectErrorHandlerChangeRector`** — `#[\Testo\ErrorHandler\ExpectErrorHandlerChange]` waives
+  the error-handler plugin's risky verdict for a test that leaves a handler installed; PHPUnit runs
+  the same stack check unconditionally and has no attribute that waives it (`#[WithoutErrorHandler]`
+  only stops PHPUnit installing its own handler).
 
 ## Implemented since the first cut
 

composer.json

@@ -45,6 +45,7 @@
         "testo/codecov": "^0.2.2",
         "testo/convention": "^0.1.5",
         "testo/data": "^0.1.9",
+        "testo/error-handler": "^0.1",
         "testo/filter": "^0.1.8",
         "testo/inline": "^0.1.9",
         "testo/lifecycle": "^0.1.6",
@@ -102,6 +103,7 @@
             "Tests\\Convention\\": "plugin/convention/tests/",
             "Tests\\Data\\": "plugin/data/tests/",
             "Tests\\Facade\\": "plugin/facade/tests/",
+            "Tests\\ErrorHandler\\": "plugin/error-handler/tests/",
             "Tests\\Filter\\": "plugin/filter/tests/",
             "Tests\\Lifecycle\\": "plugin/lifecycle/tests/",
             "Tests\\Repeat\\": "plugin/repeat/tests/",
@@ -128,6 +130,7 @@
                     "testo/convention": "0.1.x-dev",
                     "testo/data": "0.1.x-dev",
                     "testo/facade": "0.1.x-dev",
+                    "testo/error-handler": "0.1.x-dev",
                     "testo/filter": "0.1.x-dev",
                     "testo/inline": "0.1.x-dev",
                     "testo/lifecycle": "0.1.x-dev",

core/Application/Config/Plugin/SuitePlugins.php

@@ -7,6 +7,7 @@
 use Testo\Assert\AssertPlugin;
 use Testo\Bench\BenchmarkPlugin;
 use Testo\Common\PluginConfigurator;
+use Testo\ErrorHandler\ErrorHandlerPlugin;
 use Testo\Facade\FacadePlugin;
 use Testo\Inline\InlineTestPlugin;
 use Testo\Lifecycle\LifecyclePlugin;
@@ -21,6 +22,7 @@
 \class_exists(LifecyclePlugin::class) and $_[] = new LifecyclePlugin();
 \class_exists(SkipPlugin::class) and $_[] = new SkipPlugin();
 \class_exists(TestPlugin::class) and $_[] = new TestPlugin();
+\class_exists(ErrorHandlerPlugin::class) and $_[] = new ErrorHandlerPlugin();
 
 \define([__NAMESPACE__ . '\DEFAULT_SUITE_PLUGINS'][0], $_);
 unset($_);

docs/spec/skills-regrouping.md

@@ -0,0 +1,82 @@
+# Skills: group by direction, not by plugin
+
+## Problem
+
+`skills/` holds 14 skills; nine of them describe exactly one plugin (`testo-flaky-tests` is
+Retry + Repeat, `testo-coverage` is Codecov, `testo-inline-tests` is Inline, and so on). The agent
+picks a skill by *what it is doing*, not by *which package is installed*, so a per-plugin split
+makes it guess the package first. Every extra skill also adds a permanently loaded description to
+the agent's context, and every new plugin threatens to add one more.
+
+`testo-test-doubles` is the shape to copy: one skill for one direction ("isolate a collaborator"),
+a short `SKILL.md` that routes, and one file per approach under `references/` (Double, Mockery,
+hand-written). The plugin is a detail inside the direction.
+
+## Target layout
+
+| Direction | Skill | `SKILL.md` holds | `references/` |
+|---|---|---|---|
+| Run the suite | `testo-run-tests` | as is | |
+| Configure `testo.php` | `testo-configure` | as is, plus a table of optional plugins pointing at the skill that covers each | |
+| Write a test | `testo-write-tests` | `#[Test]`, `Assert`, `Expect`, skip/cancel, lifecycle hooks, groups | `data-driven.md` (DataSet, DataProvider, DataZip, DataCross), `inline-tests.md` (`#[TestInline]`) |
+| Isolate a collaborator | `testo-test-doubles` | as is | as is |
+| Keep tests reliable and clean | `testo-test-hygiene` (new) | the symptom table below, routing | `flaky.md` (Retry, Repeat), `php-errors.md` (error-handler plugin, `#[ExpectErrorHandlerChange]`), `leaks.md` (`Expect::leaks` / `notLeaks`, process-global state) |
+| Test async code | `testo-async` | as is (already two approaches in one skill) | |
+| Measure performance | `testo-benchmarks` | as is | |
+| Measure test quality | `testo-test-quality` (merge) | routing between the three | `coverage.md` (CodecovPlugin, levels, reports, `#[Covers]`), `increase-coverage.md` + its `scripts/`, `mutation-testing.md` + its `scripts/` |
+| Migrate from PHPUnit | `testo-migrate-from-phpunit` | as is | as is |
+| Extend Testo | `testo-plugin-author` | as is | as is |
+
+Fourteen skills become ten. Skills removed: `testo-data-driven`, `testo-inline-tests`,
+`testo-flaky-tests`, `testo-coverage`, `testo-increase-coverage`, `testo-mutation-testing`,
+`testo-php-errors` (folded into hygiene). `testo-async` and `testo-benchmarks` stay separate: each
+is a direction of its own, and merging them anywhere loses the trigger words.
+
+## Rules for a direction skill
+
+- **`SKILL.md` routes; `references/` explains.** The top file answers "which approach, and why"
+  in one table and states what every branch needs (fetch `llms.txt`, run from the project root).
+  Everything only one branch needs goes into that branch's reference file.
+- **Description carries every trigger the merged skills had.** Merging `testo-flaky-tests` into
+  hygiene must keep "flaky", "intermittent failure", "retry", "rerun" in the description, or the
+  agent stops reaching the material. One trigger per branch; drop synonyms.
+- **Quote the description** in the frontmatter whenever it contains `#[Attr]`: YAML cuts an
+  unquoted value at ` #`.
+- **A reference file is self-contained for its branch.** The agent loads one reference, not all
+  of them; a reference may not depend on a sibling being read first.
+- **Symptom-first routing.** The user names a symptom ("test passes but a warning scrolls by",
+  "green locally, red in CI"), not a plugin. The routing table maps symptoms to references.
+
+## `testo-test-hygiene` routing table (draft)
+
+| Symptom | Reference |
+|---|---|
+| Fails sometimes, passes on rerun; "verify the fix sticks" | `flaky.md` |
+| A PHP warning, notice or deprecation appears during a test and nothing fails; deprecations must fail the build; tested code installs its own error handler | `php-errors.md` |
+| Memory grows across tests; an object must not survive the test; a static or global keeps state | `leaks.md` |
+
+## `testo-test-quality` routing table (draft)
+
+| Intent | Reference |
+|---|---|
+| Turn coverage on, pick a level, produce a report, scope with `#[Covers]` | `coverage.md` |
+| Raise the number: find the least-covered files and write tests for them | `increase-coverage.md` |
+| Check whether the tests would notice a bug: Infection setup, MSI, killing mutants | `mutation-testing.md` |
+
+## Order of work
+
+1. `testo-test-hygiene`: create it from `testo-flaky-tests` and the standalone
+   `testo-php-errors` skill; write `leaks.md` from `Expect::leaks` in `plugin/assert`. Delete the
+   two source skills.
+2. `testo-write-tests`: move the data-driven and inline material into `references/`; keep a
+   one-line pointer per reference in the body where the topic would otherwise appear. Delete the
+   two source skills.
+3. `testo-test-quality`: merge the coverage trio; move each skill's `scripts/` under the merged
+   skill and fix the relative paths the references cite.
+4. `testo-configure`: add the optional-plugin table.
+5. `skills/README.md`: rewrite the table to the ten skills. `AGENTS.md` names
+   `skills/README.md` as the index, so nothing else needs to change.
+
+Each step is one commit and leaves every skill in the table loadable on its own. After each step
+load every touched skill once (`composer install` re-syncs `.agents/skills` through `llm/skills`)
+to confirm its frontmatter still parses and its `references/` paths resolve.

plugin/error-handler/composer.json

@@ -0,0 +1,39 @@
+{
+    "name": "testo/error-handler",
+    "description": "Error handler interceptor plugin for the Testo testing framework.",
+    "license": "BSD-3-Clause",
+    "type": "library",
+    "keywords": [
+        "testo",
+        "error-handler",
+        "test"
+    ],
+    "authors": [
+        {
+            "name": "Aleksei Gagarin (roxblnfk)",
+            "homepage": "https://github.com/roxblnfk"
+        }
+    ],
+    "funding": [
+        {
+            "type": "boosty",
+            "url": "https://boosty.to/roxblnfk"
+        }
+    ],
+    "require": {
+        "php": ">=8.2",
+        "testo/testo": "0.10.34 - 1"
+    },
+    "autoload": {
+        "psr-4": {
+            "Testo\\ErrorHandler\\": "src/"
+        }
+    },
+    "minimum-stability": "dev",
+    "prefer-stable": true,
+    "extra": {
+        "branch-alias": {
+            "dev-1.x": "1.x-dev"
+        }
+    }
+}

plugin/error-handler/src/CapturedError.php

@@ -0,0 +1,47 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Testo\ErrorHandler;
+
+/**
+ * A single PHP error captured during test execution.
+ *
+ * @api
+ */
+final readonly class CapturedError implements \Stringable
+{
+    /**
+     * @param bool $handled The handler installed before the test took the error (returned true), so
+     *        PHP would not have printed it.
+     */
+    public function __construct(
+        public int $severity,
+        public string $message,
+        public string $file,
+        public int $line,
+        public bool $handled = false,
+    ) {}
+
+    /**
+     * The error as PHP itself prints it.
+     *
+     * @return non-empty-string
+     */
+    #[\Override]
+    public function __toString(): string
+    {
+        return \sprintf('%s: %s in %s on line %d', self::label($this->severity), $this->message, $this->file, $this->line);
+    }
+
+    private static function label(int $severity): string
+    {
+        return match ($severity) {
+            \E_WARNING, \E_USER_WARNING, \E_CORE_WARNING, \E_COMPILE_WARNING => 'Warning',
+            \E_NOTICE, \E_USER_NOTICE => 'Notice',
+            \E_DEPRECATED, \E_USER_DEPRECATED => 'Deprecated',
+            \E_RECOVERABLE_ERROR => 'Recoverable fatal error',
+            default => 'Error',
+        };
+    }
+}

plugin/error-handler/src/CapturedErrors.php

@@ -0,0 +1,28 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Testo\ErrorHandler;
+
+/**
+ * Collection of PHP errors accumulated during test execution.
+ *
+ * Stored as a {@see \Testo\Core\Context\TestResult} attribute under the key {@see CapturedErrors::class}.
+ * Renderers that wish to display collected errors should retrieve it from the result.
+ *
+ * @api
+ */
+final readonly class CapturedErrors
+{
+    /**
+     * @param list<CapturedError> $errors
+     */
+    public function __construct(
+        public array $errors,
+    ) {}
+
+    public function isEmpty(): bool
+    {
+        return $this->errors === [];
+    }
+}