java.lang.NoClassDefFoundError: scala/reflect/internal/util/Statistics$

[sidharthk]

Hi

I have downloaded the doctorkafka from https://github.com/pinterest/doctorkafka.git and buid completed successfully. However while running KafkaStatMain command i get below error . Please help
Command:

java -server
-Dlog4j.configurationFile=file:./config/log4j2.xml
-cp target/lib/:target//:target/:target/kafkastats-0.2.4.4.jar:/kafka/tools/doctorkafka/kafkastats/target/classes/com/pinterest/doctorkafka/*:target/kafkastats-0.2.4.4-jar-with-dependencies.jar
com.pinterest.doctorkafka.stats.KafkaStatsMain
-broker kafka-poc-4
-jmxport 8888
-topic brokerstats
-zookeeper kafka-poc-4:2181/cluster
-uptimeinseconds 3600
-pollingintervalinseconds 60
-ostrichport 2051
-tsdhostport localhost:18126
-kafka_config /kafka/server.properties
-producer_config /kafka/producer.properties
-primary_network_ifacename eth0

ERROR:
[root@sidharth-kafka-poc-1 kafkastats]# sh dockafka.sh
2019-04-08 18:28:55,208 main DEBUG Apache Log4j Core 2.11.1 initializing configuration XmlConfiguration[location=/kafka/tools/doctorkafka/kafkastats/./config/log4j2.xml]
2019-04-08 18:28:55,414 main DEBUG Installed 2 script engines

Failed to initialize compiler: NoClassDefFoundError.
This is most often remedied by a full clean and recompile.
Otherwise, your classpath may continue bytecode compiled by
different and incompatible versions of scala.

java.lang.NoClassDefFoundError: scala/reflect/internal/util/Statistics$
at scala.tools.nsc.symtab.SymbolLoaders$ClassfileLoader.doComplete(SymbolLoaders.scala:310)
at scala.tools.nsc.symtab.SymbolLoaders$SymbolLoader.complete(SymbolLoaders.scala:213)
at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1530)
at scala.reflect.internal.Symbols$Symbol.initialize(Symbols.scala:1678)
at scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1422)
at scala.tools.nsc.Global$Run.(Global.scala:1140)
at scala.tools.nsc.interpreter.IMain._initialize(IMain.scala:127)
at scala.tools.nsc.interpreter.IMain.initializeSynchronous(IMain.scala:149)
at scala.tools.nsc.interpreter.Scripted.(Scripted.scala:74)
at scala.tools.nsc.interpreter.Scripted$.apply(Scripted.scala:309)
at scala.tools.nsc.interpreter.Scripted$Factory.getScriptEngine(Scripted.scala:302)
at org.apache.logging.log4j.core.script.ScriptManager.(ScriptManager.java:99)
at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:216)
at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:250)
at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:547)
at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:619)
at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:636)
at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:231)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:153)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:45)
at org.apache.logging.log4j.LogManager.getContext(LogManager.java:194)
at org.apache.logging.log4j.LogManager.getLogger(LogManager.java:581)
at com.pinterest.doctorkafka.stats.KafkaStatsMain.(KafkaStatsMain.java:34)
Caused by: java.lang.ClassNotFoundException: scala.reflect.internal.util.Statistics$
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 23 more

Failed to initialize compiler: NoClassDefFoundError.
This is most often remedied by a full clean and recompile.
Otherwise, your classpath may continue bytecode compiled by
different and incompatible versions of scala.

java.lang.NoClassDefFoundError: scala/reflect/internal/util/Statistics$
at scala.tools.nsc.symtab.SymbolLoaders$ClassfileLoader.doComplete(SymbolLoaders.scala:310)
at scala.tools.nsc.symtab.SymbolLoaders$SymbolLoader.complete(SymbolLoaders.scala:213)
at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1530)
at scala.reflect.internal.Symbols$Symbol.initialize(Symbols.scala:1678)
at scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1422)
at scala.tools.nsc.Global$Run.(Global.scala:1140)
at scala.tools.nsc.interpreter.IMain._initialize(IMain.scala:127)
at scala.tools.nsc.interpreter.IMain.global$lzycompute(IMain.scala:156)
at scala.tools.nsc.interpreter.IMain.global(IMain.scala:155)
at scala.tools.nsc.interpreter.IMain.initializeSynchronous(IMain.scala:150)
at scala.tools.nsc.interpreter.Scripted.(Scripted.scala:74)
at scala.tools.nsc.interpreter.Scripted$.apply(Scripted.scala:309)
at scala.tools.nsc.interpreter.Scripted$Factory.getScriptEngine(Scripted.scala:302)
at org.apache.logging.log4j.core.script.ScriptManager.(ScriptManager.java:99)
at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:216)
at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:250)
at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:547)
at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:619)
at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:636)
at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:231)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:153)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:45)
at org.apache.logging.log4j.LogManager.getContext(LogManager.java:194)
at org.apache.logging.log4j.LogManager.getLogger(LogManager.java:581)
at com.pinterest.doctorkafka.stats.KafkaStatsMain.(KafkaStatsMain.java:34)
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Exception in thread "main" java.lang.NullPointerException
at java.util.Hashtable.put(Hashtable.java:460)
at com.pinterest.doctorkafka.util.OperatorUtil.createKafkaProducerProperties(OperatorUtil.java:234)
at com.pinterest.doctorkafka.stats.KafkaAvroPublisher.(KafkaAvroPublisher.java:60)
at com.pinterest.doctorkafka.stats.KafkaStatsMain.main(KafkaStatsMain.java:135)
2019-04-08 18:28:58,428 pool-1-thread-1 DEBUG Stopping LoggerContext[name=31befd9f, org.apache.logging.log4j.core.LoggerContext@6a988392]

[sidharthk]

Hi Yang, Thanks for your reply. Initially I just had target/lib/* and target/kafkastats-0.2.4.4.jar in classpath but due to error and for trail purpose I added other one after another hierarchy but nothing helped. Warm Regards Sidharth Kumar |

…

On Tue, 9 Apr, 2019, 6:02 PM Yu Yang, ***@***.***> wrote: can you exclude /kafka/tools/doctorkafka/kafkastats/target/classes/com/pinterest/doctorkafka/* from the class path and see if that fixes the problem? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#120 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Au8ZIaNcwTdUbP2JAZLIkVDJjpU61J6Uks5vfIhrgaJpZM4ciJ0r> .

[sidharthk]

Any help please. I am not able to understand why this issues as I do see scala-reflect-2.12.x.jar in lib directory as well. Any input will be great help

[yuyang08]

I am not sure if it is because of any issue in 0.2.4.4. I tried with the latest kafkastats binary, and it worked fine on our test cluster. Can you sync with the latest code and try again?

java -server -Dlog4j.configurationFile=file:./config/log4j2.xml . \
  -cp target/lib/:target//:target/:target/kafkastats-0.2.4.5.jar:/kafka/tools/doctorkafka/kafkastats/target/classes/com/pinterest/doctorkafka/*:target/kafkastats-0.2.4.5-jar-with-dependencies.jar   \
  com.pinterest.doctorkafka.stats.KafkaStatsMain -jmxport 9999 \
  -topic brokerstats -zookeeper testzk001:2181/testkafka  \
  -tsdhostport localhost:18126 -ostrichport 2052 \
  -uptimeinseconds 3600 -pollingintervalinseconds 60 \
  -kafka_config /etc/kafka/server.properties \
  -broker testbroker001  \
  -producer_config  /etc/kafka/producer.properties \
  -primary_network_ifacename eth0

[sidharthk]

Thanks Yu Yang for your help. I have Apache Kafka 1.0.2 deployed. So could you please confirm if latest Kafkastats supports 1.0.2 Kafka. It will be great help if by any means you could share the binaries. Warm Regards Sidharth Kumar | Mob: +91 8197 555 599|

…

On Fri, 12 Apr, 2019, 11:54 AM Yu Yang, ***@***.***> wrote: I am not sure if it is because of any issue in 0.2.4.4. I tried with the latest kafkastats binary, and it worked fine on our test cluster: java -server -Dlog4j.configurationFile=file:./config/log4j2.xml . \ -cp target/lib/:target//:target/:target/kafkastats-0.2.4.5.jar:/kafka/tools/doctorkafka/kafkastats/target/classes/com/pinterest/doctorkafka/*:target/kafkastats-0.2.4.5-jar-with-dependencies.jar \ com.pinterest.doctorkafka.stats.KafkaStatsMain -jmxport 9999 \ -topic brokerstats -zookeeper testzk001:2181/testkafka \ -tsdhostport localhost:18126 -ostrichport 2052 \ -uptimeinseconds 3600 -pollingintervalinseconds 60 \ -kafka_config /etc/kafka/server.properties \ -broker testkafka080 \ -producer_config /etc/kafka/producer.properties \ -primary_network_ifacename eth0 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#120 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Au8ZIf9u_vdcI0fAvh92TKUYs5VH_vqhks5vgCaggaJpZM4ciJ0r> .

[yuyang08]

I tested kafkastats 0.2.4.5 with kafka 1.0.2 binary (https://archive.apache.org/dist/kafka/1.0.2/kafka_2.12-1.0.2.tgz) , and it worked fine for me.

Note that kafkastats by default use scala 2.12 packages. If you have scala 2.11 installed on your local machine, that may cause jar conflicts.

[sidharthk]

Thanks for your help,build 0.2.4.5 resolved the issue. However i am getting another error now in 0.2.4.5 #128 . Please let me know where i am going wrong with configurations