Skip to content

Report missing generated source files during sync instead of dropping them silently#400

Closed
Frank Portman (FrankPortman) wants to merge 1 commit into
JetBrains:262from
FrankPortman:fix/report-missing-generated-sources
Closed

Report missing generated source files during sync instead of dropping them silently#400
Frank Portman (FrankPortman) wants to merge 1 commit into
JetBrains:262from
FrankPortman:fix/report-missing-generated-sources

Conversation

@FrankPortman

Copy link
Copy Markdown

Problem

While debugging why code-generated sources do not resolve in the IDE (red squiggles), I traced two symptom classes through the sync pipeline:

  1. Generated Scala does not resolve. A genrule-produced .scala file fed into scala_library.srcs reaches TargetIdeInfo.srcsList with is_source = false, but is not part of any output group in the aspect. A normal Resync (buildProject = false) only materializes the intellij-info and intellij-sync output groups, so the generated file never exists on disk at mapping time.
  2. AspectBazelProjectMapper.resolveSourceSet then drops the file because path.exists() is false — and MissingFilesReporter.add deliberately stays silent for generated sources, so there is no trace of the drop anywhere. (Generated Python via py_library works only because py_info.bzl registers PyInfo.transitive_sources in the sync output group.)

This PR

Makes the failure observable: MissingFilesReporter now also logs a warning for generated source files that were not materialized during sync, instead of silently ignoring them. This would have pointed straight at the root cause.

Companion fix

The root causes live in the aspect (JetBrains/intellij-aspect, consumed via the intellij_aspect_sdk pin in MODULE.bazel):

  • modules/scala_info.bzl detects Scala targets solely by the scala_/thrift_ kind prefix, so custom Scala rules get no scala_target_info (unlike Java/Kotlin/Python, which detect by provider).
  • Generated sources of JVM targets are in no output group, so a sync without a build never materializes them (see above).
  • modules/python_info.bzl only falls back to PyInfo/runfiles when srcs is completely empty, so a custom code-generator rule providing PyInfo whose srcs holds generator inputs (templates/protos) exposes no Python sources at all.

I have those fixes plus regression tests ready (new fixture tests fail before / pass after; //testing/tests/{python,scala,java,kotlin} pass). Once that lands, the intellij_aspect_sdk commit pin here needs a bump to pick it up. Happy to open that PR against intellij-aspect as well.

Missing generated sources were silently dropped by AspectBazelProjectMapper,
which makes it hard to diagnose why generated code (e.g. Scala sources
produced by a genrule) is not indexed by the IDE after a sync. Log a warning
for them, like for missing regular source files.
@FrankPortman

Copy link
Copy Markdown
Author

Companion aspect fix is up: JetBrains/intellij-aspect#143 — once merged, the intellij_aspect_sdk pin in MODULE.bazel needs a bump to pick it up.

@noxvost

Copy link
Copy Markdown
Contributor

Thank you for the proposed PR. Generally I accept it, however we already have this implemented in a bit different way, and the implementation is in the merge queue. I close the PR, and you can await the implementation in a week

@FrankPortman

Copy link
Copy Markdown
Author

thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants