Skip to content

Sisco/bug fixes#31

Merged
dsisco11 merged 11 commits into
masterfrom
sisco/bug-fixes
Jan 9, 2026
Merged

Sisco/bug fixes#31
dsisco11 merged 11 commits into
masterfrom
sisco/bug-fixes

Conversation

@dsisco11

@dsisco11 dsisco11 commented Jan 9, 2026

Copy link
Copy Markdown
Owner

PR Description: sisco/bug-fixes

Title: Fix keyword query matching & block opener trivia handling


Summary

This PR addresses three bugs discovered in the TinyAst system and introduces a new ISchemaResolvableQuery interface to enable proper keyword matching in syntax definitions.


Bug Fixes

1. Query.Keyword() matches wrong keywords in syntax definitions ⚠️ Breaking behavior fixed

Problem: Query.Keyword("uniform") was incorrectly matching ANY keyword in the same category, not just "uniform". This happened because MatchesGreen() only checked IsKeyword() and ignored the text predicate during green-tree pattern matching.

Root Cause: The previous implementation used AnyKeywordQuery().Where(...) which filtered correctly during red-tree traversal but failed during green-tree syntax binding because predicates weren't evaluated.

Solution: Introduced SpecificKeywordQuery that resolves keyword text → NodeKind via schema before pattern matching, enabling O(1) kind comparison instead of string matching.

2. Block opener trivia incorrectly assigned to first child

Problem: Leading trivia (like indentation) after a block opener's newline was being dropped instead of attached to the first child element.

Fix: Updated GreenLexer.ParseBlock() to properly separate opener's trailing trivia from the first child's leading trivia.


New Features

ISchemaResolvableQuery Interface

A new internal interface for queries that require schema resolution before green-tree matching:

internal interface ISchemaResolvableQuery
{
    void ResolveWithSchema(Schema schema);
    bool IsResolved { get; }
}

Key points:

  • SyntaxBinder automatically calls ResolveWithSchema() before pattern matching
  • SyntaxTree.Select() resolves automatically when a schema is attached
  • Queries return empty results (not errors) when used without a schema
  • Composite queries (SequenceQuery, AnyOfQuery, etc.) propagate resolution recursively

Implementation Details

File Changes
NodeQuery.cs Added ISchemaResolvableQuery interface, SelectRegionsFromTree hook
NodeQueryTypes.cs New SpecificKeywordQuery class with schema resolution
QueryCombinators.cs Made composite queries implement ISchemaResolvableQuery
Query.cs Updated Query.Keyword() to return SpecificKeywordQuery
SyntaxBinder.cs Added schema storage and query resolution before matching
GreenLexer.cs Fixed trivia handling for block opener/first-child boundary

Test Coverage

  • New tests: 830+ lines of test cases added
  • Bug reproduction tests: All 3 original bug cases now pass
  • Keyword tests: Comprehensive coverage for SpecificKeywordQuery behavior
  • Schema edge cases: Schemaless trees, unknown keywords, case sensitivity
TinyTokenizer.Tests/KeywordTests.cs       | 563 ++++++++++++
TinyTokenizer.Tests/SyntaxEditorTests.cs  | 267 ++++++
TinyTokenizer.E2ETests/GlslEditorTests.cs | 135 +++

Breaking Changes

None for public API — The fix changes internal behavior to match documented expectations. Code using Query.Keyword() will now correctly match only the specified keyword instead of all keywords in the same category.


Migration Notes

If you were (incorrectly) relying on Query.Keyword("x") matching other keywords, update to use Query.AnyKeyword or Query.KeywordCategory("CategoryName") instead.

@dsisco11 dsisco11 merged commit b4f80e2 into master Jan 9, 2026
2 checks passed
@dsisco11 dsisco11 deleted the sisco/bug-fixes branch January 9, 2026 03:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant