Skip to content

Обновление парсера#3816

Open
theshadowco wants to merge 3 commits intodevelopfrom
feature/updParser260212
Open

Обновление парсера#3816
theshadowco wants to merge 3 commits intodevelopfrom
feature/updParser260212

Conversation

@theshadowco
Copy link
Member

@theshadowco theshadowco commented Feb 12, 2026

Описание

Доработки, связанные с обновлением bsl parser

  • изменения в лексике SDBL парсера
  • исправление некоторых ошибок
  • новый ANTLR + null check + фиксы из-за нарушения обратной совместимости

Связанные задачи

Closes

Чеклист

Общие

  • Ветка PR обновлена из develop
  • Отладочные, закомментированные и прочие, не имеющие смысла участки кода удалены
  • Изменения покрыты тестами
  • Обязательные действия перед коммитом выполнены (запускал команду gradlew precommit)

Для диагностик

  • Описание диагностики заполнено для обоих языков (присутствуют файлы для обоих языков, для русского заполнено все подробно, перевод на английский можно опустить)

Дополнительно

Summary by CodeRabbit

  • New Features

    • Broadened SDBL tokenization and keyword/join recognition for richer language support.
  • Bug Fixes

    • Improved null-safety across parsing, diagnostics and highlighting to reduce errors.
    • Tightened control-flow graph and query diagnostics to avoid edge-case failures.
  • Chores

    • Upgraded parser dependency and added support configuration library.
  • Tests

    • Updated token/highlight tests to reflect merged multi-token keyword handling.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 12, 2026

Warning

Rate limit exceeded

@theshadowco has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 18 minutes and 0 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📝 Walkthrough

Walkthrough

This PR upgrades the BSL parser and adds supportconf, and applies widespread null-safety and parser-node access changes (adding @Nullable, switching from direct .children to getChildren(), adding null checks) across diagnostics, AST utilities, semantic tokens, providers, and tests.

Changes

Cohort / File(s) Summary
Build / Dependencies
build.gradle.kts
Bump bsl-parser to 0.31.0-rc.1 (removed previous exclusions) and add io.github.1c-syntax:supportconf:0.15.0.
Callback / Sentry
src/main/java/.../aop/sentry/PermissionFilterBeforeSendCallback.java, src/main/java/.../aop/sentry/SentryScopeConfigurer.java
Moved @Nullable annotation placement, added ExecutionException handling in waitForPermission; removed unused ClientInfo import.
CFG & Parse Tree
src/main/java/.../cfg/CfgBuildingParseTreeVisitor.java
Added parent null-guard when detecting top-level preprocessor; minor message formatting adjustments.
Code Actions / Extractors
src/main/java/.../codeactions/ExtractStructureConstructorSupplier.java
Replaced direct .children access with getChildren(), added null-safety checks and Objects usage to avoid NPEs.
Semantic Tokens (core)
src/main/java/.../semantictokens/strings/SdblTokenTypes.java
Large rework of token sets: remove/replace legacy tokens, add composite join/union tokens and many functions/operators; token classification and ordering changed.
Diagnostics (many files)
src/main/java/.../diagnostics/...
Widespread null-safety: many visitor returns annotated @Nullable, parent/context null checks, replace .children with getChildren(), token/union/join detection updates, minor threshold/logic tweaks.
Document Highlighting
src/main/java/.../documenthighlight/...
Add/remove Nullable imports, mark helper methods nullable, move to Optional/getChildren() patterns, and refactor join highlighting to use composite join nodes and BY token.
Utilities & Ranges
src/main/java/.../utils/DiagnosticHelper.java, .../utils/Ranges.java, .../SemanticTokensOptions.java
equalNodes accepts nullable params; Ranges factory methods accept @Nullable and return empty range for null; removed unused imports.
Expression Tree & Terminals
src/main/java/.../utils/expressiontree/..., src/main/java/.../semantictokens/strings/SpecialContextVisitor.java
Switch to getChildren(), add @Nullable to visitMember/makeSubexpression/TerminalSymbolNode params/returns, guard optional methodCall paths.
Providers / SelectionRange
src/main/java/.../providers/SelectionRangeProvider.java
Construct temporary ParserRuleContext with parent in constructor, add null-check for ifStatement before else/elsif access.
Tests
src/test/java/.../documenthighlight/SDBLJoinDocumentHighlightSupplierTest.java, .../documenthighlight/AllKnownTokenTest.java, .../providers/SemanticTokensProviderTest.java
Update expected highlight/token spans to reflect merged multi-token keywords; add new AllKnownTokenTest; minor formatting test change.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Possibly related PRs

Suggested labels

hacktoberfest-accepted

Suggested reviewers

  • nixel2007
  • EvilBeaver

Poem

🐰 I hopped through trees of parse and token,
Added guards so nulls won't be a problem,
Joins now merge and visitors sigh,
Tests updated, CI give a try —
A tiny rabbit patch, happy and solemn.

🚥 Pre-merge checks | ✅ 3 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 22.62% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title "Обновление парсера" (Parser update) is directly related to the main objective of this pull request, which involves updates to the BSL parser including dependency version upgrades, null safety improvements, and ANTLR migration fixes.
Merge Conflict Detection ✅ Passed ✅ No merge conflicts detected when merging into develop

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feature/updParser260212

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/main/java/com/github/_1c_syntax/bsl/languageserver/diagnostics/ParseErrorDiagnostic.java (1)

91-98: ⚠️ Potential issue | 🟡 Minor

@Nullable return may produce "null" text in diagnostic messages.

StringJoiner.add(null) appends the literal string "null". If both getLiteralName and getSymbolicName return null for a token type, the error message will contain "null" as an expected token name. Consider filtering or providing a fallback.

💡 Suggested: filter nulls in the caller
         expectedTokens.getIntervals().stream()
           .flatMapToInt(interval -> IntStream.rangeClosed(interval.a, interval.b))
           .mapToObj(ParseErrorDiagnostic::getTokenName)
+          .filter(Objects::nonNull)
           .forEachOrdered(sj::add);
🤖 Fix all issues with AI agents
In `@build.gradle.kts`:
- Line 85: The build declares a pre-release dependency
api("io.github.1c-syntax", "bsl-parser", "0.31.0-rc.1"); replace the RC with the
latest stable release by changing the version to "0.30.0" in that api(...)
declaration unless you explicitly require RC features—if RC is required, add a
clear TODO comment next to the api(...) line noting it's intentional for the
develop branch and must be upgraded to a stable version before any
production/release build.

In
`@src/main/java/com/github/_1c_syntax/bsl/languageserver/diagnostics/UsingFindElementByStringDiagnostic.java`:
- Around line 61-70: The code in UsingFindElementByStringDiagnostic accesses the
first parameter with callParamList.callParam().get(0) without checking for an
empty list which can throw IndexOutOfBoundsException; modify the lambda to guard
against an empty callParam() (e.g., check !callParamList.callParam().isEmpty()
or use callParamList.callParam().stream().findFirst()) before accessing the
first element, then proceed to inspect the param and call
diagnosticStorage.addDiagnostic(ctx, info.getMessage(matcher.group(0))) only
when a param is present.

In
`@src/main/java/com/github/_1c_syntax/bsl/languageserver/documenthighlight/LoopStatementDocumentHighlightSupplier.java`:
- Around line 231-234: The method addBreakAndContinueFromTryStatement can NPE if
tryStatement.tryCodeBlock() or tryStatement.exceptCodeBlock() is null; add null
guards before calling .codeBlock(): check tryStatement.tryCodeBlock() != null
and call addBreakAndContinueHighlights(highlights,
tryStatement.tryCodeBlock().codeBlock()) only if present, and likewise check
tryStatement.exceptCodeBlock() != null before calling .codeBlock() and
addBreakAndContinueHighlights for the except branch; keep using the existing
addBreakAndContinueHighlights method and ensure both null-checked branches are
executed independently.

In
`@src/main/java/com/github/_1c_syntax/bsl/languageserver/documenthighlight/SDBLJoinDocumentHighlightSupplier.java`:
- Around line 117-118: The code handles JOIN variants incorrectly causing an
NPE: in the branch checking joinCtx.innerJoin() != null it calls
joinCtx.fullJoin().keyword which can be null; change the call to use
joinCtx.innerJoin().keyword instead so addTokenHighlight(highlights, ...)
references the innerJoin token; update the conditional branch that contains
addTokenHighlight to use innerJoin() rather than fullJoin() and run tests for
addTokenHighlight and SDBLJoinDocumentHighlightSupplier to confirm no
regressions.
🧹 Nitpick comments (5)
src/main/java/com/github/_1c_syntax/bsl/languageserver/utils/Ranges.java (1)

107-109: Consider adding @Nullable to the two-arg ParserRuleContext overload as well.

With the parser upgrade introducing more potential nulls, callers of create(startCtx, endCtx) could also receive null contexts. Unlike the single-arg variant (line 93), this overload will NPE if either argument is null. If null inputs are plausible here, the same defensive pattern should be applied.

src/main/java/com/github/_1c_syntax/bsl/languageserver/diagnostics/UnsafeSafeModeMethodCallDiagnostic.java (1)

110-120: Unsafe cast of getChildren() elements to ParserRuleContext.

getChildren() returns List<ParseTree>, which can include TerminalNode instances (e.g., operator tokens between members in an expression). Casting the prev/next sibling directly to ParserRuleContext on Lines 112 and 120 risks a ClassCastException if the sibling is a terminal node.

This appears to be a pre-existing risk carried over from the old children field access, but worth noting since you're already touching these lines.

Safer alternative
-      var prev = (ParserRuleContext) rootExpressionNode.getChildren().get(indexOfCurrentMemberNode - 1);
+      var prevTree = rootExpressionNode.getChildren().get(indexOfCurrentMemberNode - 1);
+      if (!(prevTree instanceof ParserRuleContext prev)) {
+        return false;
+      }

Apply the same pattern on Line 120 for next.

src/main/java/com/github/_1c_syntax/bsl/languageserver/documenthighlight/SDBLJoinDocumentHighlightSupplier.java (1)

111-116: Nit: missing space after if/else if keywords.

Minor style inconsistency: if( should be if ( per Java conventions.

🪧 Style fix
-    if(joinCtx.leftJoin() != null) {
+    if (joinCtx.leftJoin() != null) {
       addTokenHighlight(highlights, joinCtx.leftJoin().keyword);
-    } else if(joinCtx.rightJoin() != null) {
+    } else if (joinCtx.rightJoin() != null) {
       addTokenHighlight(highlights, joinCtx.rightJoin().keyword);
-    } else if(joinCtx.fullJoin() != null) {
+    } else if (joinCtx.fullJoin() != null) {
       addTokenHighlight(highlights, joinCtx.fullJoin().keyword);
-    } else if(joinCtx.innerJoin() != null) {
+    } else if (joinCtx.innerJoin() != null) {
src/test/java/com/github/_1c_syntax/bsl/languageserver/documenthighlight/AllKnownTokenTest.java (2)

39-40: @SpringBootTest is unnecessary here — no Spring beans are injected.

This test only uses static methods (SdblTokenTypes.getTokenTypeAndModifiers, SDBLLexer.VOCABULARY) and doesn't require the Spring application context. Loading the full context slows test execution for no benefit. Also, the class is public while neighboring test classes (e.g., SDBLJoinDocumentHighlightSupplierTest) use package-private visibility.

♻️ Suggested fix
-@SpringBootTest
-public class AllKnownTokenTest {
+class AllKnownTokenTest {

84-92: bslTokens() is a no-op placeholder test that always passes.

The test creates an empty list and asserts it's empty — it can never fail. The TODO on Line 89 indicates this is intentional work-in-progress. Consider either implementing it or annotating with @Disabled("TODO: implement BSL token coverage check") so it's visible as pending work rather than silently passing.

♻️ Suggested change
+  `@org.junit.jupiter.api.Disabled`("TODO: implement analog for BSL lexer")
   `@Test`
   void bslTokens() {

Would you like me to help implement the BSL token coverage check following the same pattern as sdblTokens()?

@github-actions
Copy link
Contributor

github-actions bot commented Feb 12, 2026

Test Results

 2 916 files   2 916 suites   1h 7m 50s ⏱️
 1 235 tests  1 234 ✅ 1 💤 0 ❌
11 115 runs  11 106 ✅ 9 💤 0 ❌

Results for commit 5a11395.

♻️ This comment has been updated with latest results.

}

private void addDiagnostic(BinaryOperationNode node) {
var startToken = Trees.getTokens(node.getParent().getRepresentingAst())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@theshadowco тут node - это кусок cfg, а не parse tree. Оно точно @nullable?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ну parent может быть null... это например рут


@Override
public ParseTree visitIfStatement(BSLParser.IfStatementContext ctx) {
public @Nullable ParseTree visitIfStatement(BSLParser.IfStatementContext ctx) {
Copy link
Member

@nixel2007 nixel2007 Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@theshadowco почему visit-методы стали @nullable для возвращаемого значения? Они же вызываются только, когда ctx не равен null, значит, должны вызвать либо super, либо вернуть себя же, то есть не-null.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

забываешь про рут.
Там схема вызова построена как раз на наличии null в результате - если он есть, то значит дальше нет никто.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ну и да, прервать дальнейшее движение по дереву можно именно с null (кейсов не придумаю, но так есть)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

попробую поковырять antlr

@theshadowco theshadowco force-pushed the feature/updParser260212 branch from 58b9de5 to 5a11395 Compare February 23, 2026 06:41
@sonarqubecloud
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants