Provide a way to retrieve raw used packages/classes before artifact mapping

[delanym]

New feature, improvement proposal

AI DISCLAIMER: https://gemini.google.com/share/dd9579182da4

Currently, ProjectDependencyAnalyzer immediately aggregates scanned bytecode data into Set. When un-relocated classes exist across duplicate artifacts or fat-jars on the classpath, the artifact-guessing step yields false positives that are impossible for upstream plugins to filter cleanly.

We need an option or an extended analysis result (e.g., a getUnmappedPackages() or passing a raw Set of used types) that records the raw bytecode types detected for which no explicit top-level dependency exists before trying to associate them with a specific GAV coordinate loop.

[elharo]

Maybe, but I don't fully grok your issue. Can you explain what this means for the project being analyzed? I.e. what does a project that triggers this issue looks like? Let's get clear on the exact issue before we dig into the implementation.

[delanym]

Sure. So i recently added this configuration to our multi-module project to require that all used dependencies are declared.
It is no longer acceptable to depend on classes provided by transitive dependencies:

<plugin>
        <artifactId>maven-dependency-plugin</artifactId>
        <version>3.11.0</version>
        <executions>
          <execution>
            <id>enforce-explicit-dependencies</id>
            <goals>
              <goal>analyze-only</goal>
            </goals>
            <phase>verify</phase>
            <configuration>
              <failOnWarning>true</failOnWarning>
              <!-- Temporary -->
              <ignoreAllNonTestScoped>true</ignoreAllNonTestScoped>
              <ignoreNonCompile>true</ignoreNonCompile>
              <ignoreUnusedRuntime>true</ignoreUnusedRuntime>
              <ignoredUsedUndeclaredDependencies>
                <!-- Test infrastructure -->
                <usedUndeclaredDependency>org.junit.jupiter:junit-jupiter-api:jar:*</usedUndeclaredDependency>
                <usedUndeclaredDependency>org.junit.jupiter:junit-jupiter-params:jar:*</usedUndeclaredDependency>
                <usedUndeclaredDependency>com.tngtech.archunit:archunit:jar:*</usedUndeclaredDependency>
                <usedUndeclaredDependency>org.mockito:mockito-core:jar:*</usedUndeclaredDependency>
                <usedUndeclaredDependency>org.slf4j:slf4j-api:jar:*</usedUndeclaredDependency>
                <!-- Deprecated -->
                <usedUndeclaredDependency>com.google.code.findbugs:jsr305:jar:*</usedUndeclaredDependency>
                <!-- Shaded -->
                <usedUndeclaredDependency>org.glassfish.metro:webservices-api:jar:*</usedUndeclaredDependency>
              </ignoredUsedUndeclaredDependencies>
            </configuration>
          </execution>
        </executions>
      </plugin>

You'll notice that webservices-api is ignored. This is because as long as it is on the classpath the dependency plugin will suggest to add it to satisfy this rule when a dependent project requires a class that it has. However, webservices-api is an artifact shading various jakarta micro apis (activation, jws, mail and xml). I would rather my colleagues were prompted to include those dependencies, not the webservices-api dependency.

You might ask: why do we have both dependencies on the classpath in the first place? I agree, we need to sort that out. I was actually hoping the dependency plugin could help with this. When there are more than one potential dependency the plugin should suggest all potential dependencies.

I removed from this config other usedUndeclaredDependencyof shaded company artifacts that had to be ignored. Unlike metro dependencies, these are not going to be easy to remove/replace.

Another example is where shading is unavoidable, but exclusions have not been properly applied.
For example, the kryo persistency project depends on a static logger called minlog.
I want to use kryo, but i want all logs to pass through slf4j.
Until EsotericSoftware/minlog#17 is accepted my only option is to shade the com.esotericsoftware.minlog.Log class. I suspect this would only be a problem if I forgot to exclude minlog from the kryo dependency, and i ended up with some code compiling against it but not declaring the dependency.
Its an unlikely situation, but it does demonstrate another possible grey area that i shouldn't have to concern myself with (or my colleagues, who know and care less).

[elharo]

I think I get it. I need to think about it more. My initial reaction is that if a referenced type is available from more than one transitive dependency (jar) in the tree then the plugin should suggest an explicit declaration be added for the jar from which that class would actually loaded. This isn't what you're asking for, which is that we list all the dependencies the type appears in.

[delanym]

Unfortunately most developers will probably take the path of least resistance and not try to resolve the ambiguity. So yes I think listing all the dependencies would be more helpful. Perhaps it could have an asterisk next to the one that it happened to match first, since this is the expected JVM behaviour. But as you probably know this is no guarantee that the classpath fed to the machine would be constructed in the same manner in the runtime environment.

[elharo]

If developers are not going to try to resolve the ambiguity, then why show it to them at all? Why show them several they won't do anything with?

Classpath construction and class loading is reasonably deterministic in the absence of version ranges.

It's possible what you're trying to do is better handled by the maven-enforcer-plugin and doesn't belong in the maven-dependency-analyzer at all though I do think there is a real issue here we should look into. I just don't think the right fix is what you're asking for.

[delanym]

I'm coming off a bit harsh on developers. All i'm saying is if the dependency plugin wont even show them the ambiguity then they're just going to tick the checkbox and make the warning go away.
And by developers that naturally includes agents too, which will be much more likely to take up the issue of an ambiguous dependency. But not if it doesn't appear in the output.

The maven-dependency-plugin's analyze goal should be the go to place to understand whats going on with the dependencies so it should have all the details.

I will of course turn on the banDuplicateClasses enforcer rule once all the duplicates are actually resolved. Then I'll have no more use for this ability I'm asking for.
You say its something to look into - so I take it my part here is done.