test: verify WinCode 0.12.3 against real application evidence - #26
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Version 0.12.3 adds an opt-in real-application acceptance script for six TavernDesk navigation-to-source tasks. Candidate filenames are discovered by bounded native searches; the script then checks unique live controls, literal command assignments and method declaration snippets against current source hashes. It records native/MCP call counts, UTF-16 output characters, call time, repeated displayed lines and failures in local reports. Core and desktop check reports also retain actual TAP test/pass/fail/cancel/skip totals. No new dependencies or runtime feature expansion.
The isolated WPF source-repair test now carries the repository SDK policy and NuGet lock into its temporary checkout, keeping its offline restore locked. Skill guidance documents evidence reuse and fixed-profile testing, and the acceptance matrix distinguishes real application observations, controlled fixtures and unavailable integration conditions.
Validation:
Limits remain explicit: no native-only performance comparison, universal token-saving claim, whole-method coverage, verified runtime binding or real Serena/C# semantic integration. No usable Serena command was found and none was installed. A user-imported card was verified read-only in the fixed test database and visually on the shelf; its description/first-message fields and managed cover were preserved. Two same-name UI text nodes correctly return complete-but-ambiguous without selecting a guessed subtree, also verified through newly loaded 0.12.3 production Gateway/Host handlers. Six navigation tasks passed again with the real card present (67,083 characters). Computer Use still misattributes the window to AiPPT; detail-page actions and chat were not tested. Existing Repomix alert #1 and branch-protection policy remain separate pending decisions. Review was author self-review without new agents. Exact-head Node 22/24 and CodeQL results are required before merge.