Skip to content

Assembly fails for version >= 0.30.0 with "deduplicate: different file contents found in the following" error #257

Description

@jfuentescpp

In order to reproduce:

build.sbt

name := "assembly_conflict"
version := "0.0.1"
scalaVersion := "2.12.11"

libraryDependencies ++= List(
  "com.lightstep.tracer" % "lightstep-tracer-jre" % "0.30.1",
  "com.lightstep.tracer" % "tracer-grpc" % "0.30.0",
  "io.grpc" % "grpc-netty" % "1.23.0",
  "io.netty" % "netty-tcnative-boringssl-static" % "2.0.25.Final",
  "org.slf4j" % "slf4j-simple" % "1.7.25"
)

project/plugins.sbt

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.15.0")

project/build.properties

sbt.version=1.1.1

then sbt assembly give erros:

[error] 13 errors were encountered during merge
[error] java.lang.RuntimeException: deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-buffer/jars/netty-buffer-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-codec-http/jars/netty-codec-http-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-codec-http2/jars/netty-codec-http2-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-codec-socks/jars/netty-codec-socks-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-codec/jars/netty-codec-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-common/jars/netty-common-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-handler-proxy/jars/netty-handler-proxy-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-handler/jars/netty-handler-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-resolver/jars/netty-resolver-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-tcnative-boringssl-static/jars/netty-tcnative-boringssl-static-2.0.25.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-transport/jars/netty-transport-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Collector$1.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Collector$1.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Collector.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Collector.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue$1.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue$1.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue$2.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue$2.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue$Builder.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue$Builder.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue$ValueCase.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue$ValueCase.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValueOrBuilder.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValueOrBuilder.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Reporter$1.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Reporter$1.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Reporter$Builder.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Reporter$Builder.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Reporter.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Reporter.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/ReporterOrBuilder.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/ReporterOrBuilder.class
[error]         at sbtassembly.Assembly$.applyStrategies(Assembly.scala:161)
[error]         at sbtassembly.Assembly$.x$1$lzycompute$1(Assembly.scala:43)
[error]         at sbtassembly.Assembly$.x$1$1(Assembly.scala:41)
[error]         at sbtassembly.Assembly$.stratMapping$lzycompute$1(Assembly.scala:41)
[error]         at sbtassembly.Assembly$.stratMapping$1(Assembly.scala:41)
[error]         at sbtassembly.Assembly$.inputs$lzycompute$1(Assembly.scala:86)
[error]         at sbtassembly.Assembly$.inputs$1(Assembly.scala:76)
[error]         at sbtassembly.Assembly$.apply(Assembly.scala:103)
[error]         at sbtassembly.Assembly$.$anonfun$assemblyTask$1(Assembly.scala:278)
[error]         at scala.Function1.$anonfun$compose$1(Function1.scala:44)
[error]         at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:39)
[error]         at sbt.std.Transform$$anon$4.work(System.scala:66)
[error]         at sbt.Execute.$anonfun$submit$2(Execute.scala:262)
[error]         at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error]         at sbt.Execute.work(Execute.scala:271)
[error]         at sbt.Execute.$anonfun$submit$1(Execute.scala:262)
[error]         at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:174)
[error]         at sbt.CompletionService$$anon$2.call(CompletionService.scala:36)
[error]         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error]         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error]         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error]         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error]         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error]         at java.lang.Thread.run(Thread.java:748)
[error] (assembly) deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-buffer/jars/netty-buffer-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-codec-http/jars/netty-codec-http-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-codec-http2/jars/netty-codec-http2-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-codec-socks/jars/netty-codec-socks-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-codec/jars/netty-codec-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-common/jars/netty-common-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-handler-proxy/jars/netty-handler-proxy-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-handler/jars/netty-handler-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-resolver/jars/netty-resolver-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-tcnative-boringssl-static/jars/netty-tcnative-boringssl-static-2.0.25.Final.jar:META-INF/io.netty.versions.properties
[error] /Users/jfuentes/.ivy2/cache/io.netty/netty-transport/jars/netty-transport-4.1.38.Final.jar:META-INF/io.netty.versions.properties
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Collector$1.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Collector$1.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Collector.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Collector.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue$1.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue$1.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue$2.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue$2.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue$Builder.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue$Builder.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue$ValueCase.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue$ValueCase.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValue.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValue.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/KeyValueOrBuilder.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/KeyValueOrBuilder.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Reporter$1.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Reporter$1.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Reporter$Builder.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Reporter$Builder.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/Reporter.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/Reporter.class
[error] deduplicate: different file contents found in the following:
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-common/jars/java-common-0.30.0.jar:com/lightstep/tracer/grpc/ReporterOrBuilder.class
[error] /Users/jfuentes/.ivy2/cache/com.lightstep.tracer/java-metrics-reporter/jars/java-metrics-reporter-0.1.4.jar:com/lightstep/tracer/grpc/ReporterOrBuilder.class
[error] Total time: 8 s, completed Aug 12, 2020 12:58:41 PM

Issue can be worked around with something like code below. It worked fine before 0.30.0 release.

assemblyMergeStrategy in assembly := {
  case PathList("com", "lightstep", "tracer", "grpc", xs @ _*) =>
    MergeStrategy.first
  case x => MergeStrategy.defaultMergeStrategy(x)
}

Checking the dependency tree, the conflict is from libraries that lightstep-tracer-jre depends:
Screenshot 2020-08-12 at 13 09 13

Why is this error happening? Content of the source code seems quite similar but I didn't properly check. Some difference in the .class?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions