test(ci): first engine-free test project (Tests~/EngineFree) + a hosted-ubuntu job

[IvanMurzak]

Unity had zero engine-free tests. Every C# signal came from a game-ci Unity Test Runner leg — licence, editor boot, minutes, and nothing at all on a fork PR. This adds a plain Microsoft.NET.Sdk xUnit project (Unity-MCP-Plugin/Tests~/EngineFree) that compiles a chosen set of the real plugin sources via <Compile Include> and runs in seconds on ubuntu-latest, on the Godot-MCP.Tests model, plus the engine-free-tests job that runs it.

44 tests, ~65 ms.

CI evidence

	run	result
DoD 5 — job plant	33863152821 on 3d276c3e (all tests deleted)	engine-free tests (.NET 8) FAILED — No test is available ... Process completed with exit code 1, while nuget gate, both test-cli legs and the completed Unity legs stayed green. Reverted by 7fbf3527.
DoD 3 — the new job	33863846659 on 519a55fa	engine-free tests (.NET 8) SUCCESS — Passed! - Failed: 0, Passed: 44, Skipped: 0, Total: 44, Duration: 230 ms, whole job 22 s on ubuntu-latest / SDK 8.0.x
baseline for the rest of the workflow	33796052151 — p0-unity-ci-regreen's final run, and 33815830447 11 h later	both success on every leg

test-unity-6000-3-1f1-editmode (base) flaked once on run 33863846659 with Failed to resolve packages: Access to 'https://packages.unity.com/com.unity.ide.visualstudio' was denied. No packages loaded. inside the game-ci container — a Unity package-registry denial, green on both baseline runs above, and re-run to confirm. Two independent facts say this diff cannot reach it: the log contains zero CS#### errors and zero occurrences of EngineFree, i.e. Unity never imported the new folder at all.

DoD 7 — workspace-source override, run rather than assumed

With a repo-root Directory.Build.targets (the 02 §5 fragment shape) + nuget.config pointing at a local feed of repacked -ws.g<sha8> packages and UseWorkspaceSources=true, obj/project.assets.json resolves:

"com.IvanMurzak.McpPlugin/8.3.0-ws.g9c0e11d2": {
"com.IvanMurzak.McpPlugin.Common/8.3.0-ws.g9c0e11d2": {
"com.IvanMurzak.ReflectorNet/5.4.0-ws.g1dff5501": {

All 44 tests pass against those packages, and again with -p:Version=8.3.0-ws.g9c0e11d2 — so the Godot-MCP.Tests-style <Compile Include> links are unaffected by either. The project stays override-compatible by construction: no UseWorkspaceSources property of its own, no packages.lock.json, no RestorePackagesPath, no nuget.config.

One measured detail worth knowing: the pin-parity guard stays GREEN under the override. Directory.Build.targets is imported AFTER the ItemGroup that captures the metadata, so the metadata reports the DECLARED pin, not the resolved one. That is what this guard should do — it is a check about what the release train must bump, and a transient override must not redden it — but it does mean the guard is not a proof of what was restored. Said plainly in the csproj, the test and the README rather than left to be rediscovered.

What is covered

Area	File	What it pins
(a) resolver pins + asmdef gate	NuGetPinGateTests.cs	the issue #957 invariant, with .github/scripts/check_nuget_gate.py's rule re-implemented in C# over the compiled-in NuGetConfig constants (not by shelling out to Python), so the two guards fail for genuinely different reasons: the script parses NuGetConfig.cs as text with regexes, these read what the compiler sees
(b) connection config	ConnectionConfigTests.cs	the cloud authorization-server side — base-URL/endpoint derivation, the mcp:agent LOGIN scope (never narrowed to mcp:plugin, the P0-3 hazard), the RFC 8628/6749 wire forms and documents, base64url JWT subject decoding — plus the plugin log-level gate
(c) tool attribute plumbing	ToolAttributeTests.cs	[AiToolType] / [AiTool] driven through the real McpPluginBuilder, so the System-vs-Standard registry split is observed rather than asserted from the source text

Scope limits, measured rather than assumed

Tests~/EngineFree/README.md records this in full. The short version: Unity's engine-free surface is far smaller than Godot's, and almost all of it funnels through two leaves — Runtime/Logger/UnityLogger.cs (UnityEngine.Debug.*) and Runtime/Unity/Logs/UnityLogCollector.cs (UnityEngine.Application.logMessageReceivedThreaded).

"0 UnityEngine grep hits" is a hypothesis about compilability, not the answer — a file drags in the engine transitively just as fatally. Verified by building, not by reading:

File	First error	Chain
Runtime/Utils/EnvironmentUtils.cs	CS0234: namespace 'Utils' does not exist	UnityLoggerFactory to UnityLoggerProvider to UnityLogger
Runtime/UnityMcpPlugin.cs	CS0246: UnityLogCollector	UnityLogCollector
Runtime/UnityMcpPlugin.Config.cs	CS0103: GeneratePortFromDirectory / EnvironmentUtils	both of the above
Runtime/UnityMcpPluginBuilder.cs	CS0246: UnityMcpPluginRuntime	to UnityMcpPlugin to UnityMcpPlugin.Build.cs (using UnityEngine)
Editor/DependencyResolver/NuGetInstallManifest.cs	CS0246: UnityEngine	Debug.LogWarning, not behind #if UNITY_EDITOR
Editor/DependencyResolver/NuGetPackageRestorer.cs	CS0246: UnityEngine	four Debug.Log* calls, same

So, stated plainly rather than papered over:

• UnityConnectionConfig and EnvironmentUtils.ApplyEnvironmentOverrides are NOT covered here. The args > env > disk ladder stays a game-ci EditMode concern. Area (b) covers the parts of connection configuration that are reachable.
• Only Tool_Ping of the tool families is covered. It is the only [AiToolType] class whose every partial file is engine-free. A partial class assembled from a subset of its files presents a surface that does not exist in production, so a reflection test over it would be measuring the wrong object and reporting green — the csproj comments make that a rule, not a preference.

Nothing is stubbed. A file that does not compile is left out and named.

New pin location — for the release train

EngineFree.csproj consumes com.IvanMurzak.McpPlugin 8.3.0 and com.IvanMurzak.ReflectorNet 5.4.0 — the same pins NuGetConfig.cs:117,124 installs. That makes this csproj a second pin location the train must bump (p3-train-create, update-consumers.py:114-135 B path). Filed as a ROADMAP note for sibling C, not edited here.

It cannot rot silently: PinnedPackageVersions_MatchNuGetConfig compares the declared versions — exported as AssemblyMetadata from the real PackageReference items rather than re-typed as a literal, so the guard cannot drift from the csproj — against NuGetConfig.Packages. (What it does not prove is what restore resolved; see the DoD 7 note above.)

Incidental finding: the mirror-image downgrade (csproj ReflectorNet to 5.3.3) is not possible — McpPlugin 8.3.0 requires >= 5.4.0 transitively and NuGet refuses with NU1605 before any test runs. So the csproj can drift upward but not below the framework floor.

Vacuity guards

• RunConfiguration.TreatNoTestsAsError=true is load-bearing, not decoration. Measured on this project: with all tests deleted, dotnet test exits 0 without the flag and 1 with it. Without it a vacuous suite reports a green job. (Reproduced in CI by the DoD-5 plant run above.)
• The base64url JWT fixture originally used sub = "user_42?/+" — chosen because the literal characters look like the thing under test. Its base64 contains neither + nor /, so it exercised neither Replace() and scored GREEN under the plant with both deleted. The subject is now selected by search for the property that does the work, and the test asserts - and _ are present in the encoded payload before decoding, so it cannot go toothless again.

Plant table — 35 plants, 35 RED, all 32 test methods covered

Each row: an exact one-occurrence source mutation, applied to a clean tree, diff-verified, suite run, then reverted with the tree re-asserted clean. Baseline before every plant: 44/44 passed.

Plant	Mutation (source)	Result	Tests turned RED
P01	csproj McpPlugin pin 8.3.0 -> 8.2.0 (DoD 6)	RED (43/44 passed)	PinnedPackageVersions_MatchNuGetConfig
P02	resolver ReflectorNet pin 5.4.0 -> 5.3.3	RED (42/44 passed)	PinnedPackageVersions_MatchNuGetConfig, PinnedSet_MatchesTheBlessedLock_AndTheLockPairsTheCurrentGenerationDefine
P03	csproj drops the pin-parity AssemblyMetadata export	RED (43/44 passed)	PinnedPackageVersions_MatchNuGetConfig
P04	resolver pin bumped without re-blessing the lock	RED (42/44 passed)	PinnedPackageVersions_MatchNuGetConfig, PinnedSet_MatchesTheBlessedLock_AndTheLockPairsTheCurrentGenerationDefine
P05	generation define bumped without re-blessing the lock	RED (43/44 passed)	PinnedSet_MatchesTheBlessedLock_AndTheLockPairsTheCurrentGenerationDefine
P06	generation define collapsed onto the ready define	RED (42/44 passed)	GenerationDefine_IsPresent_PrefixedAndDistinctFromTheReadyDefine, PinnedSet_MatchesTheBlessedLock_AndTheLockPairsTheCurrentGenerationDefine
P07	GateDefines drops the generation define	RED (43/44 passed)	GateDefines_AreExactlyTheReadyAndGenerationPair
P08	a pin version becomes a non-version string	RED (42/44 passed)	Packages_AreNonEmpty_UniquelyIdentified_AndEveryVersionParses, PinnedSet_MatchesTheBlessedLock_AndTheLockPairsTheCurrentGenerationDefine
P09	a package id is pinned twice	RED (42/44 passed)	Packages_AreNonEmpty_UniquelyIdentified_AndEveryVersionParses, PinnedSet_MatchesTheBlessedLock_AndTheLockPairsTheCurrentGenerationDefine
P10	lock pins[] no longer hashes to its own pinsSha256	RED (43/44 passed)	Digest_AgreesWithThePythonCanonicalForm_ForTheBlessedPinList
P11	digest re-implementation stops sorting (order-sensitive)	RED (41/44 passed)	Digest_AgreesWithThePythonCanonicalForm_ForTheBlessedPinList, Digest_IsOrderInsensitive_ButChangesWhenAnyPinVersionChanges, PinnedSet_MatchesTheBlessedLock_AndTheLockPairsTheCurrentGenerationDefine
P12	digest re-implementation ignores the version	RED (41/44 passed)	Digest_AgreesWithThePythonCanonicalForm_ForTheBlessedPinList, Digest_IsOrderInsensitive_ButChangesWhenAnyPinVersionChanges, PinnedSet_MatchesTheBlessedLock_AndTheLockPairsTheCurrentGenerationDefine
P13	flat-container URL stops lower-casing the id	RED (43/44 passed)	DownloadUrl_IsTheLowerCasedFlatContainerPath
P14	nupkg cache moves into the asset pipeline	RED (43/44 passed)	CachedPath_IsUnderTheConfiguredCacheDirectory
P15	target-framework priority prefers net48 over netstandard2.1	RED (43/44 passed)	TargetFrameworkPriority_PrefersNetstandard21_AndEndsWithTheRootLibFallback
P16	authorization-server base URL stops being trimmed	RED (42/44 passed)	EndpointUrls_AreAppendedToTheTrimmedBase
P17	empty-base-URL guard removed	RED (43/44 passed)	Constructor_TrimsTheBase_DefaultsTheClientId_AndRejectsAnEmptyBase
P18	device LOGIN narrowed to the plugin scope	RED (43/44 passed)	DeviceLogin_RequestsTheAgentScope_AndIsNeverNarrowedToThePluginScope
P19	device_authorization form field renamed	RED (43/44 passed)	DeviceAuthorizeForm_CarriesTheClientIdAndScope_UnderTheRfc8628FieldNames
P20	device-code grant URN typo	RED (43/44 passed)	DeviceTokenForm_RedeemsTheDeviceCodeUnderTheDeviceCodeGrantUrn
P21	device_authorization response json name changed	RED (43/44 passed)	DeviceAuthorizeResponse_ReadsTheSnakeCaseDocument
P22	RFC 6749 error field dropped from the token response	RED (43/44 passed)	DeviceTokenResponse_DistinguishesASuccessFromAnRfc6749PendingError
P23	JWT payload decoded as standard base64, not base64url	RED (43/44 passed)	JwtSubject_IsDecodedFromTheBase64UrlPayload
P24	malformed-JWT catch stops returning null	RED (43/44 passed)	JwtSubject_ReturnsNullOnAnythingMalformed
P25	empty sub claim passed through instead of null	RED (43/44 passed)	JwtSubject_ReturnsNullWhenTheClaimIsAbsentOrNotAString
P26	log-level gate becomes strictly-greater	RED (42/44 passed)	LogLevelGate_EmitsOnlyAtOrAboveTheConfiguredThreshold
P27	log-level None renumbered	RED (43/44 passed)	LogLevelOrder_RunsFromTraceToNone_SoTheGateComparisonMeansWhatItSays
P28	[AiToolType] removed from the tool class	RED (41/44 passed)	AssemblyScan_DiscoversPingAndLandsItOnTheSystemRegistry, EveryAiToolOnACompiledInFamily_DeclaresANameAndAKnownToolType, ToolClass_CarriesAiToolType_SoTheAssemblyScanCanFindIt
P29	tool id constant renamed	RED (43/44 passed)	ToolIdConstant_IsTheNameTheAttributeActuallyRegisters
P30	tool moved from the System registry to Standard	RED (41/44 passed)	AssemblyScan_DiscoversPingAndLandsItOnTheSystemRegistry, ExplicitRegistration_ByType_ProducesTheSameSystemToolEntry, PingIsRegisteredAsASystemTool_NotAStandardOne
P31	ReadOnlyHint dropped	RED (43/44 passed)	PingCarriesTheReadOnlyAndIdempotentHints_AndStaysEnabled
P32	[AiSkillBody] emptied	RED (43/44 passed)	PingCarriesSkillDescriptionAndBody_SoTheGeneratedSkillMdIsUseful
P33	tool registered under an empty name	RED (40/44 passed)	AssemblyScan_DiscoversPingAndLandsItOnTheSystemRegistry, EveryAiToolOnACompiledInFamily_DeclaresANameAndAKnownToolType, ExplicitRegistration_ByType_ProducesTheSameSystemToolEntry, ToolIdConstant_IsTheNameTheAttributeActuallyRegisters
P34	ignore-assembly control removed from the scan-prune test	RED (43/44 passed)	AssemblyScan_RegistersNothing_WhenTheHostingAssemblyIsIgnored
P35	zero-tools-by-default control opts a tool in	RED (43/44 passed)	BuilderWithNoToolsOptedIn_RegistersNothing

Every one of the 32 distinct test methods is turned RED by at least one plant. P11/P12/P34/P35 mutate the test-side code that is the thing under test (the digest re-implementation; the two negative controls) — labelled as such rather than dressed up as product mutations.

Invisible to Unity and to UPM

• git ls-files Unity-MCP-Plugin/Tests~/EngineFree returns the csproj + 3 test files + README + a local .gitignore. Nothing else.
• .gitignore:35 !Tests~/EngineFree/*.csproj re-includes exactly this csproj past the *.csproj rule that exists for Unity's generated project files; git check-ignore -v confirms Assembly-CSharp.csproj is still ignored.
• grep -r EngineFree Unity-MCP-Plugin/Packages/com.ivanmurzak.unity.mcp returns 0 hits (that dir is what release.yml:213,250 packs).
• The folder sits outside Assets/ and Packages/ and ends in ~, so Unity never imports it: find Unity-MCP-Plugin/Tests~ -name '*.meta' returns 0.

One deliberate deviation from the task's file list, flagged for review: the task said to touch Unity-MCP-Plugin/.gitignore and "nothing else" there, which I did — but nothing in the repo ignores bin/, obj/ or TestResults/ under this new folder (the plugin-root /[Oo]bj/ rule is rooted at Unity-MCP-Plugin/), so a local dotnet test followed by git add -A would commit the whole restore output. I added a 6-line Tests~/EngineFree/.gitignore scoped entirely inside the folder this task creates, rather than widening the plugin-root file.

🤖 Generated with Claude Code

https://claude.ai/code/session_01SHsWZUyyCcq4D78nXkXX2T