You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Tomas Nunez <nu...@pythian.com> on 2014/11/01 04:05:14 UTC
Error using migrationtool for upgrading 0.7 to 0.8
Hi
I'm trying to upgrade a 0.7 kafka cluster, but I'm getting an error:
I created the file migrationToolConsumer.properties with just theinfo to
connect to the old cluster:
_____
zookeeper.connect=zookeeper01:2181,zookeeper02:2181,zookeeper03:2181
group.id=migration.prod
______
Then the producer file
_______
zookeeper.connect=zookeeper01_new:2181,zookeeper02_new:2181,zookeeper03_new:2181
group.id=migration.prod
_______
And then I called the migrationtool, and get this error
______________
$ kafka/bin/kafka-run-class.sh kafka.tools.KafkaMigrationTool
--kafka.07.jar migration-tool/kafka-0.7.2.jar --zkclient.01.jar
migration-tool/zkclient-0.1.jar --num.producers 4
--consumer.config=kafka/config/migrationToolConsumer.properties
-producer.config=kafka/config/migrationToolProducer.properties
--whitelist=.*
Kafka migration tool failed due to:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
at kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
... 5 more
Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
at java.lang.ClassLoader.findClass(ClassLoader.java:531)
at
kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at
kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 7 more
[2014-11-01 02:51:35,124] ERROR Kafka migration tool failed:
(kafka.tools.KafkaMigrationTool)
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
at kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
... 5 more
Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
at java.lang.ClassLoader.findClass(ClassLoader.java:531)
at
kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at
kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 7 more
__________
The files kafka-0.7.2.jar and zkclient-0.1.jar are the exact same ones
that are being used in the old 0.7 cluster.
I also tried downloading and building a new 0.7.2 jar file with scala, but
it had the same error, too.
What can I be doing wrong? Is there any way to get more output to see where
is the error?
Thanks in advance,
--
Tomàs Núñez
Enterprise Infrastructure Consultant
The Pythian Group - Love your data!
Office (international): +1 613 565 8696 x1501
--
--
Re: Error using migrationtool for upgrading 0.7 to 0.8
Posted by Tomas Nunez <nu...@pythian.com>.
Thanks to this last hint we finally got it to work!
One problem was that the new kafka 0.8 cluster, was compiled with scala
2.10, while the old 0.7 one was compiled with scala 2.8.
As the MigrationTool uses 0.8 binaries, but also libraries from 0.7, there
was a mismatch and it wasn't finding the classes it should.
Solution? As the new cluster needs to keep running and changing it to
scala2.8 wasn't an option, I just downloaded the kafka 0.8 version compiled
with scala 2.8 (
https://www.apache.org/dyn/closer.cgi?path=/kafka/0.8.1.1/kafka_2.8.0-0.8.1.1.tgz),
and run kafka-run-class.sh from there. Then there was no more missing
libraries.
But I still had those "Caused by: java.lang.NullPointerException" errors.
After last Gwen's question, I checked the consumer.properties, where the
"zookeeper.connect". Then I thought, although that's the correct value in
0.8 (checked here https://kafka.apache.org/08/configuration.html), if it's
connecting to the 0.7 cluster, to consume messages, maybe I should use 0.7
confing instead, where it is "zk.connect" (checked here
http://kafka.apache.org/07/configuration.html). Changed it and BOOM! It
worked!
So in conclussion: MigrationTool needs to be compiled with the same scala
version as the 0.7 cluster, and consumer config needs to comply with 0.7,
while producer config needs to comply with 0.8
And that's it!
Thanks a lot
Regards,
Tomàs
On Thu, Nov 6, 2014 at 3:16 AM, Gwen Shapira <gs...@cloudera.com> wrote:
> Also, can you post your configs? Especially the "zookeeper.connect" one?
>
> On Wed, Nov 5, 2014 at 6:15 PM, Gwen Shapira <gs...@cloudera.com>
> wrote:
>
> > Regarding more information:
> > Maybe ltrace?
> >
> > If I were you, I'd go to MigrationTool code and start adding LOG lines.
> > because there aren't enough of those to troubleshoot.
> >
> > On Wed, Nov 5, 2014 at 6:13 PM, Gwen Shapira <gs...@cloudera.com>
> > wrote:
> >
> >> org.apache.zookeeper.ClientCnxn is throwing the exception, so I'm 100%
> >> sure it eventually found the class.
> >>
> >> On Wed, Nov 5, 2014 at 5:59 PM, Tomas Nunez <nu...@pythian.com> wrote:
> >>
> >>> Ok, still fighting with the migrationTool here...
> >>>
> >>> That tuple wasn't in the scala-library.jar. It turns out I was using
> >>> scala
> >>> 2.10 for kafka0.8 and scala 2.8 for kafka0.7, and the jar files were
> not
> >>> compatible. So, for the record, it seems that you need both the 0.7 jar
> >>> files and your 0.8 kafka compiled with the same java version.
> >>>
> >>> After fixing that (downloading kafka 0.8 compiled with scala 2.8), I'm
> >>> now
> >>> facing a different error, this time more crypic:
> >>>
> >>> Kafka migration tool failed due to:
> >>> java.lang.reflect.InvocationTargetException
> >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>> at
> >>>
> >>>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >>> at
> >>>
> >>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >>> at java.lang.reflect.Method.invoke(Method.java:606)
> >>> at
> >>> kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:204)
> >>> Caused by: java.lang.NullPointerException
> >>> at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:361)
> >>> at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:332)
> >>> at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:383)
> >>> at
> org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:64)
> >>> at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:872)
> >>> at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
> >>> at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
> >>> at
> >>>
> >>>
> kafka.consumer.ZookeeperConsumerConnector.connectZk(ZookeeperConsumerConnector.scala:152)
> >>> at
> >>>
> >>>
> kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:122)
> >>> at
> >>>
> >>>
> kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:65)
> >>> at
> >>>
> >>>
> kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:67)
> >>> at
> >>>
> >>>
> kafka.consumer.Consumer$.createJavaConsumerConnector(ConsumerConnector.scala:88)
> >>> at
> >>>
> >>>
> kafka.consumer.Consumer.createJavaConsumerConnector(ConsumerConnector.scala)
> >>> ... 5 more
> >>>
> >>> [2014-11-06 01:32:44,362] ERROR Kafka migration tool failed:
> >>> (kafka.tools.KafkaMigrationTool)
> >>> java.lang.reflect.InvocationTargetException
> >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>> at
> >>>
> >>>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >>> at
> >>>
> >>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >>> at java.lang.reflect.Method.invoke(Method.java:606)
> >>> at
> >>> kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:204)
> >>> Caused by: java.lang.NullPointerException
> >>> at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:361)
> >>> at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:332)
> >>> at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:383)
> >>> at
> org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:64)
> >>> at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:872)
> >>> at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
> >>> at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
> >>> at
> >>>
> >>>
> kafka.consumer.ZookeeperConsumerConnector.connectZk(ZookeeperConsumerConnector.scala:152)
> >>> at
> >>>
> >>>
> kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:122)
> >>> at
> >>>
> >>>
> kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:65)
> >>> at
> >>>
> >>>
> kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:67)
> >>> at
> >>>
> >>>
> kafka.consumer.Consumer$.createJavaConsumerConnector(ConsumerConnector.scala:88)
> >>> at
> >>>
> >>>
> kafka.consumer.Consumer.createJavaConsumerConnector(ConsumerConnector.scala)
> >>> ... 5 more
> >>>
> >>> Using strace with it, I could see these errors again:
> >>>
> >>> 13342 stat("/var/migration/kafka/utils/Logging.class", 0x7fc92913c280)
> =
> >>> -1
> >>> ENOENT (No such file or directory)
> >>> 13342 stat("/var/migration/kafka/common/KafkaStorageException.class",
> >>> 0x7fc92913d950) = -1 ENOENT (No such file or directory)
> >>> 13342 stat("/var/migration/kafka/common/KafkaException.class",
> >>> 0x7fc92913d950) = -1 ENOENT (No such file or directory)
> >>> 13342 stat("/var/migration/kafka/utils/Logging$class.class",
> >>> 0x7fc92913de00) = -1 ENOENT (No such file or directory)
> >>> 13342 stat("/var/migration/kafka/utils/Log4jController$.class",
> >>> 0x7fc92913dc60) = -1 ENOENT (No such file or directory)
> >>> 13342 stat("/var/migration/kafka/utils/Log4jController.class",
> >>> 0x7fc92913d7c0) = -1 ENOENT (No such file or directory)
> >>> 13342 stat("/var/migration/kafka/utils/Log4jControllerMBean.class",
> >>> 0x7fc92913b4e0) = -1 ENOENT (No such file or directory)
> >>> 13342
> >>> stat("/var/migration/org/apache/log4j/spi/ThrowableInformation.class",
> >>> 0x7fc92913e860) = -1 ENOENT (No such file or directory)
> >>>
> >>> and after I cd to kafka-0.8.1.1-src/core/build/classes/main/, a lot of
> >>> those ENOENT disappear, but I still get the same "Null Pointer error".
> >>> Grepping "ENOENT" I see:
> >>>
> >>> 13257
> >>>
> >>>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/Environment.class",
> >>> 0x7fee656ae750) = -1 ENOENT (No such file or directory)
> >>> 13257
> >>>
> >>>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/Version.class",
> >>> 0x7fee656ae680) = -1 ENOENT (No such file or directory)
> >>> 13257
> >>>
> >>>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/version/Info.class",
> >>> 0x7fee656ac3a0) = -1 ENOENT (No such file or directory)
> >>> 13257
> >>>
> >>>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/Environment$Entry.class",
> >>> 0x7fee656ae8e0) = -1 ENOENT (No such file or directory)
> >>> 13257
> >>>
> >>>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/ZooKeeper$ZKWatchManager.class",
> >>> 0x7fee656aede0) = -1 ENOENT (No such file or directory)
> >>> 13257
> >>>
> >>>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/ClientCnxn.class",
> >>> 0x7fee656aede0) = -1 ENOENT (No such file or directory)
> >>> 13257
> >>>
> >>>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/proto/RequestHeader.class",
> >>> 0x7fee656ae5f0) = -1 ENOENT (No such file or directory)
> >>> 13257
> >>>
> >>>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/ClientCnxn$1.class",
> >>> 0x7fee656ae9a0) = -1 ENOENT (No such file or directory)
> >>> 13257
> >>>
> >>>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/META-INF/services/java.nio.channels.spi.SelectorProvider",
> >>> 0x7fee656aed20) = -1 ENOENT (No such file or directory)
> >>> 13257
> >>>
> >>>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/log4j/spi/ThrowableInformation.class",
> >>> 0x7fee656af860) = -1 ENOENT (No such file or directory)
> >>>
> >>> I found, for instance ClientCnxn$1.class in zookeeper-3.3.4.jar
> >>>
> >>> $ jar tf ./core/build/dependant-libs-2.8.0/zookeeper-3.3.4.jar|grep
> >>> Client
> >>> org/apache/zookeeper/ClientCnxn$1.class
> >>> org/apache/zookeeper/ClientCnxn$2.class
> >>> org/apache/zookeeper/ClientCnxn$AuthData.class
> >>> org/apache/zookeeper/ClientCnxn$EndOfStreamException.class
> >>> org/apache/zookeeper/ClientCnxn$EventThread.class
> >>> org/apache/zookeeper/ClientCnxn$Packet.class
> >>> org/apache/zookeeper/ClientCnxn$SendThread.class
> >>> org/apache/zookeeper/ClientCnxn$SessionExpiredException.class
> >>> org/apache/zookeeper/ClientCnxn$SessionTimeoutException.class
> >>> org/apache/zookeeper/ClientCnxn$WatcherSetEventPair.class
> >>> org/apache/zookeeper/ClientCnxn.class
> >>> org/apache/zookeeper/ClientWatchManager.class
> >>> org/apache/zookeeper/ServerAdminClient.class
> >>>
> >>> And that jarfile is on the classpath, and I see on strace that it's
> >>> finding
> >>> it... Why is it not finding the class, then?
> >>>
> >>> I'm a bit lost... Is there any way I can run it with more verbose?
> Strace
> >>> is too verbose, "bash -x" is not that much. Something in between?
> >>>
> >>>
> >>>
> >>>
> >>> On Sat, Nov 1, 2014 at 4:53 AM, Gwen Shapira <gs...@cloudera.com>
> >>> wrote:
> >>>
> >>> > The file should be your Scala jar.
> >>> > You should have it in your Kafka lib directory.
> >>> > Do a "jar -t" to make sure it actually contains the Tuple class.
> >>> >
> >>> > Gwen
> >>> >
> >>> > On Fri, Oct 31, 2014 at 8:49 PM, Tomas Nunez <nu...@pythian.com>
> >>> wrote:
> >>> >
> >>> > > Thanks for pointing the error. I fixed it, but I'm still getting
> the
> >>> same
> >>> > > error. The content of migrationToolProducer.properties is now:
> >>> > > _____
> >>> > >
> >>> > >
> >>> >
> >>>
> metadata.broker.list=kafka_new01:9092,kafka_new02:9092,kafka_new03:9092,kafka_new04:9092,kafka_new05:9092,
> >>> > > group.id=migration.prod
> >>> > > _____
> >>> > >
> >>> > > And the error still is:
> >>> > >
> >>> > > Exception in thread "main" java.lang.NoClassDefFoundError:
> >>> > > scala/Tuple2$mcLL$sp
> >>> > > at kafka.utils.Utils.stackTrace(Utils.scala)
> >>> > > at
> >>> > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:268)
> >>> > > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> >>> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> >>> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> >>> > > at java.security.AccessController.doPrivileged(Native
> Method)
> >>> > > at
> java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> >>> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> >>> > > at
> >>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> >>> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> >>> > > ... 2 more
> >>> > >
> >>> > > I have no "Tuple2*" file anywhere in the server. I'm following
> >>> > >
> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/KAFKA/Migrating+from+0.7+to+0.8
> >>> > > and I can't see there anything about downloading classes, and I
> don't
> >>> > find
> >>> > > much people with the same problem, which leads me to think that I'm
> >>> doing
> >>> > > something wrong...
> >>> > >
> >>> > > Any idea on what to try?
> >>> > >
> >>> > > Thanks!
> >>> > >
> >>> > > PD: Of course, I'll tell Alex you say hi :)
> >>> > >
> >>> > > On Sat, Nov 1, 2014 at 4:32 AM, Gwen Shapira <
> gshapira@cloudera.com>
> >>> > > wrote:
> >>> > >
> >>> > > > This is part of Scala, so it should be in the
> scala-library-...jar
> >>> > > >
> >>> > > >
> >>> > > > On Fri, Oct 31, 2014 at 8:26 PM, Tomas Nunez <nu...@pythian.com>
> >>> > wrote:
> >>> > > >
> >>> > > > > Well... I used strace and I found it was looking for some
> >>> classes in
> >>> > a
> >>> > > > > wrong path. I fixed most of them, but there's one that isn't
> >>> > anywhere,
> >>> > > > > neither the new nor the old kafka servers:
> >>> > > > >
> >>> > > > > $ strace -o logfile -s 1000 -f kafka/bin/kafka-run-class.sh
> >>> > > > > kafka.tools.KafkaMigrationTool --kafka.07.jar
> >>> > > > > migration-tool/kafka-0.7.2.jar --zkclient.01.jar
> >>> > > > > migration-tool/zkclient-0.1.jar
> >>> > > > > --consumer.config=kafka/config/migrationToolConsumer.properties
> >>> > > > > -producer.config=kafka/config/migrationToolProducer.properties
> >>> > > > > --whitelist=.*
> >>> > > > > Exception in thread "main" java.lang.NoClassDefFoundError:
> >>> > > > > scala/Tuple2$mcLL$sp
> >>> > > > > at kafka.utils.Utils.stackTrace(Utils.scala)
> >>> > > > > at
> >>> > > > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:268)
> >>> > > > > Caused by: java.lang.ClassNotFoundException:
> scala.Tuple2$mcLL$sp
> >>> > > > > at
> java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> >>> > > > > at
> java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> >>> > > > > at java.security.AccessController.doPrivileged(Native
> >>> Method)
> >>> > > > > at
> >>> java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> >>> > > > > at
> java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> >>> > > > > at
> >>> > > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> >>> > > > > at
> java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> >>> > > > > ... 2 more
> >>> > > > >
> >>> > > > > $ grep ENOENT logfile|tail -1
> >>> > > > > 16484
> >>> > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> stat("migration-tool/kafka07/core/target/scala_2.8.0/classes/scala/Tuple2$mcLL$sp.class",
> >>> > > > > 0x7f59b27f4950) = -1 ENOENT (No such file or directory)
> >>> > > > >
> >>> > > > > I saw someone had a similar problem in this list, but it's not
> >>> clear
> >>> > > (at
> >>> > > > > least to me) how it was fixed in the end
> >>> > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201302.mbox/%3CCANEBM33rMTZ2UGp2dpaFj8xV=n4=Rr9GP3Six0sko++nhQv4kQ@mail.gmail.com%3E
> >>> > > > >
> >>> > > > > The JIRA ticket is still untouched
> >>> > > > > https://issues.apache.org/jira/browse/KAFKA-760
> >>> > > > >
> >>> > > > > May different versions of scala be the cause of this error? I
> >>> just
> >>> > used
> >>> > > > > "./sbt update" to make sure...
> >>> > > > >
> >>> > > > > If not, where should I find that class file?
> >>> > > > >
> >>> > > > > Thanks again!
> >>> > > > >
> >>> > > > > On Sat, Nov 1, 2014 at 4:05 AM, Tomas Nunez <nunez@pythian.com
> >
> >>> > wrote:
> >>> > > > >
> >>> > > > > > Hi
> >>> > > > > >
> >>> > > > > > I'm trying to upgrade a 0.7 kafka cluster, but I'm getting an
> >>> > error:
> >>> > > > > >
> >>> > > > > > I created the file migrationToolConsumer.properties with just
> >>> > theinfo
> >>> > > > to
> >>> > > > > > connect to the old cluster:
> >>> > > > > > _____
> >>> > > > > >
> >>> > zookeeper.connect=zookeeper01:2181,zookeeper02:2181,zookeeper03:2181
> >>> > > > > > group.id=migration.prod
> >>> > > > > > ______
> >>> > > > > >
> >>> > > > > >
> >>> > > > > > Then the producer file
> >>> > > > > > _______
> >>> > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> zookeeper.connect=zookeeper01_new:2181,zookeeper02_new:2181,zookeeper03_new:2181
> >>> > > > > > group.id=migration.prod
> >>> > > > > > _______
> >>> > > > > >
> >>> > > > > > And then I called the migrationtool, and get this error
> >>> > > > > >
> >>> > > > > > ______________
> >>> > > > > > $ kafka/bin/kafka-run-class.sh kafka.tools.KafkaMigrationTool
> >>> > > > > > --kafka.07.jar migration-tool/kafka-0.7.2.jar
> --zkclient.01.jar
> >>> > > > > > migration-tool/zkclient-0.1.jar --num.producers 4
> >>> > > > > >
> --consumer.config=kafka/config/migrationToolConsumer.properties
> >>> > > > > >
> -producer.config=kafka/config/migrationToolProducer.properties
> >>> > > > > > --whitelist=.*
> >>> > > > > > Kafka migration tool failed due to:
> >>> > > > > > java.lang.reflect.InvocationTargetException
> >>> > > > > > at
> >>> > > > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> >>> > > > > > Method)
> >>> > > > > > at
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> >>> > > > > > at
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> >>> > > > > > at
> >>> > > > > java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> >>> > > > > > at
> >>> > > > >
> kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
> >>> > > > > > Caused by: java.lang.NoClassDefFoundError:
> scala/Tuple2$mcLL$sp
> >>> > > > > > at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
> >>> > > > > > at
> >>> > > > kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
> >>> > > > > > ... 5 more
> >>> > > > > > Caused by: java.lang.ClassNotFoundException:
> >>> scala.Tuple2$mcLL$sp
> >>> > > > > > at
> >>> java.lang.ClassLoader.findClass(ClassLoader.java:531)
> >>> > > > > > at
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
> >>> > > > > > at
> >>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> >>> > > > > > at
> >>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> >>> > > > > > at
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
> >>> > > > > > at
> >>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> >>> > > > > > at
> >>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> >>> > > > > > ... 7 more
> >>> > > > > >
> >>> > > > > > [2014-11-01 02:51:35,124] ERROR Kafka migration tool failed:
> >>> > > > > > (kafka.tools.KafkaMigrationTool)
> >>> > > > > > java.lang.reflect.InvocationTargetException
> >>> > > > > > at
> >>> > > > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> >>> > > > > > Method)
> >>> > > > > > at
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> >>> > > > > > at
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> >>> > > > > > at
> >>> > > > > java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> >>> > > > > > at
> >>> > > > >
> kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
> >>> > > > > > Caused by: java.lang.NoClassDefFoundError:
> scala/Tuple2$mcLL$sp
> >>> > > > > > at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
> >>> > > > > > at
> >>> > > > kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
> >>> > > > > > ... 5 more
> >>> > > > > > Caused by: java.lang.ClassNotFoundException:
> >>> scala.Tuple2$mcLL$sp
> >>> > > > > > at
> >>> java.lang.ClassLoader.findClass(ClassLoader.java:531)
> >>> > > > > > at
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
> >>> > > > > > at
> >>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> >>> > > > > > at
> >>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> >>> > > > > > at
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
> >>> > > > > > at
> >>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> >>> > > > > > at
> >>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> >>> > > > > > ... 7 more
> >>> > > > > > __________
> >>> > > > > >
> >>> > > > > >
> >>> > > > > > The files kafka-0.7.2.jar and zkclient-0.1.jar are the exact
> >>> same
> >>> > > ones
> >>> > > > > > that are being used in the old 0.7 cluster.
> >>> > > > > >
> >>> > > > > > I also tried downloading and building a new 0.7.2 jar file
> with
> >>> > > scala,
> >>> > > > > but
> >>> > > > > > it had the same error, too.
> >>> > > > > >
> >>> > > > > > What can I be doing wrong? Is there any way to get more
> output
> >>> to
> >>> > see
> >>> > > > > > where is the error?
> >>> > > > > >
> >>> > > > > > Thanks in advance,
> >>> > > > > >
> >>> > > > > > --
> >>> > > > > > Tomàs Núñez
> >>> > > > > > Enterprise Infrastructure Consultant
> >>> > > > > > The Pythian Group - Love your data!
> >>> > > > > >
> >>> > > > > > Office (international): +1 613 565 8696 x1501
> >>> > > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > > --
> >>> > > > > Tomàs Núñez
> >>> > > > > Enterprise Infrastructure Consultant
> >>> > > > > The Pythian Group - Love your data!
> >>> > > > >
> >>> > > > > Office (international): +1 613 565 8696 x1501
> >>> > > > >
> >>> > > > > --
> >>> > > > >
> >>> > > > >
> >>> > > > > --
> >>> > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> > >
> >>> > >
> >>> > > --
> >>> > > Tomàs Núñez
> >>> > > Enterprise Infrastructure Consultant
> >>> > > The Pythian Group - Love your data!
> >>> > >
> >>> > > Office (international): +1 613 565 8696 x1501
> >>> > >
> >>> > > --
> >>> > >
> >>> > >
> >>> > > --
> >>> > >
> >>> > >
> >>> > >
> >>> > >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> Tomàs Núñez
> >>> Enterprise Infrastructure Consultant
> >>> The Pythian Group - Love your data!
> >>>
> >>> Office (international): +1 613 565 8696 x1501
> >>>
> >>> --
> >>>
> >>>
> >>> --
> >>>
> >>>
> >>>
> >>>
> >>
> >
>
--
Tomàs Núñez
Enterprise Infrastructure Consultant
The Pythian Group - Love your data!
Office (international): +1 613 565 8696 x1501
--
--
Re: Error using migrationtool for upgrading 0.7 to 0.8
Posted by Gwen Shapira <gs...@cloudera.com>.
Also, can you post your configs? Especially the "zookeeper.connect" one?
On Wed, Nov 5, 2014 at 6:15 PM, Gwen Shapira <gs...@cloudera.com> wrote:
> Regarding more information:
> Maybe ltrace?
>
> If I were you, I'd go to MigrationTool code and start adding LOG lines.
> because there aren't enough of those to troubleshoot.
>
> On Wed, Nov 5, 2014 at 6:13 PM, Gwen Shapira <gs...@cloudera.com>
> wrote:
>
>> org.apache.zookeeper.ClientCnxn is throwing the exception, so I'm 100%
>> sure it eventually found the class.
>>
>> On Wed, Nov 5, 2014 at 5:59 PM, Tomas Nunez <nu...@pythian.com> wrote:
>>
>>> Ok, still fighting with the migrationTool here...
>>>
>>> That tuple wasn't in the scala-library.jar. It turns out I was using
>>> scala
>>> 2.10 for kafka0.8 and scala 2.8 for kafka0.7, and the jar files were not
>>> compatible. So, for the record, it seems that you need both the 0.7 jar
>>> files and your 0.8 kafka compiled with the same java version.
>>>
>>> After fixing that (downloading kafka 0.8 compiled with scala 2.8), I'm
>>> now
>>> facing a different error, this time more crypic:
>>>
>>> Kafka migration tool failed due to:
>>> java.lang.reflect.InvocationTargetException
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>>
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>> at
>>> kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:204)
>>> Caused by: java.lang.NullPointerException
>>> at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:361)
>>> at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:332)
>>> at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:383)
>>> at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:64)
>>> at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:872)
>>> at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
>>> at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
>>> at
>>>
>>> kafka.consumer.ZookeeperConsumerConnector.connectZk(ZookeeperConsumerConnector.scala:152)
>>> at
>>>
>>> kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:122)
>>> at
>>>
>>> kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:65)
>>> at
>>>
>>> kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:67)
>>> at
>>>
>>> kafka.consumer.Consumer$.createJavaConsumerConnector(ConsumerConnector.scala:88)
>>> at
>>>
>>> kafka.consumer.Consumer.createJavaConsumerConnector(ConsumerConnector.scala)
>>> ... 5 more
>>>
>>> [2014-11-06 01:32:44,362] ERROR Kafka migration tool failed:
>>> (kafka.tools.KafkaMigrationTool)
>>> java.lang.reflect.InvocationTargetException
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>>
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>> at
>>> kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:204)
>>> Caused by: java.lang.NullPointerException
>>> at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:361)
>>> at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:332)
>>> at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:383)
>>> at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:64)
>>> at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:872)
>>> at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
>>> at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
>>> at
>>>
>>> kafka.consumer.ZookeeperConsumerConnector.connectZk(ZookeeperConsumerConnector.scala:152)
>>> at
>>>
>>> kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:122)
>>> at
>>>
>>> kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:65)
>>> at
>>>
>>> kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:67)
>>> at
>>>
>>> kafka.consumer.Consumer$.createJavaConsumerConnector(ConsumerConnector.scala:88)
>>> at
>>>
>>> kafka.consumer.Consumer.createJavaConsumerConnector(ConsumerConnector.scala)
>>> ... 5 more
>>>
>>> Using strace with it, I could see these errors again:
>>>
>>> 13342 stat("/var/migration/kafka/utils/Logging.class", 0x7fc92913c280) =
>>> -1
>>> ENOENT (No such file or directory)
>>> 13342 stat("/var/migration/kafka/common/KafkaStorageException.class",
>>> 0x7fc92913d950) = -1 ENOENT (No such file or directory)
>>> 13342 stat("/var/migration/kafka/common/KafkaException.class",
>>> 0x7fc92913d950) = -1 ENOENT (No such file or directory)
>>> 13342 stat("/var/migration/kafka/utils/Logging$class.class",
>>> 0x7fc92913de00) = -1 ENOENT (No such file or directory)
>>> 13342 stat("/var/migration/kafka/utils/Log4jController$.class",
>>> 0x7fc92913dc60) = -1 ENOENT (No such file or directory)
>>> 13342 stat("/var/migration/kafka/utils/Log4jController.class",
>>> 0x7fc92913d7c0) = -1 ENOENT (No such file or directory)
>>> 13342 stat("/var/migration/kafka/utils/Log4jControllerMBean.class",
>>> 0x7fc92913b4e0) = -1 ENOENT (No such file or directory)
>>> 13342
>>> stat("/var/migration/org/apache/log4j/spi/ThrowableInformation.class",
>>> 0x7fc92913e860) = -1 ENOENT (No such file or directory)
>>>
>>> and after I cd to kafka-0.8.1.1-src/core/build/classes/main/, a lot of
>>> those ENOENT disappear, but I still get the same "Null Pointer error".
>>> Grepping "ENOENT" I see:
>>>
>>> 13257
>>>
>>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/Environment.class",
>>> 0x7fee656ae750) = -1 ENOENT (No such file or directory)
>>> 13257
>>>
>>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/Version.class",
>>> 0x7fee656ae680) = -1 ENOENT (No such file or directory)
>>> 13257
>>>
>>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/version/Info.class",
>>> 0x7fee656ac3a0) = -1 ENOENT (No such file or directory)
>>> 13257
>>>
>>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/Environment$Entry.class",
>>> 0x7fee656ae8e0) = -1 ENOENT (No such file or directory)
>>> 13257
>>>
>>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/ZooKeeper$ZKWatchManager.class",
>>> 0x7fee656aede0) = -1 ENOENT (No such file or directory)
>>> 13257
>>>
>>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/ClientCnxn.class",
>>> 0x7fee656aede0) = -1 ENOENT (No such file or directory)
>>> 13257
>>>
>>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/proto/RequestHeader.class",
>>> 0x7fee656ae5f0) = -1 ENOENT (No such file or directory)
>>> 13257
>>>
>>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/ClientCnxn$1.class",
>>> 0x7fee656ae9a0) = -1 ENOENT (No such file or directory)
>>> 13257
>>>
>>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/META-INF/services/java.nio.channels.spi.SelectorProvider",
>>> 0x7fee656aed20) = -1 ENOENT (No such file or directory)
>>> 13257
>>>
>>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/log4j/spi/ThrowableInformation.class",
>>> 0x7fee656af860) = -1 ENOENT (No such file or directory)
>>>
>>> I found, for instance ClientCnxn$1.class in zookeeper-3.3.4.jar
>>>
>>> $ jar tf ./core/build/dependant-libs-2.8.0/zookeeper-3.3.4.jar|grep
>>> Client
>>> org/apache/zookeeper/ClientCnxn$1.class
>>> org/apache/zookeeper/ClientCnxn$2.class
>>> org/apache/zookeeper/ClientCnxn$AuthData.class
>>> org/apache/zookeeper/ClientCnxn$EndOfStreamException.class
>>> org/apache/zookeeper/ClientCnxn$EventThread.class
>>> org/apache/zookeeper/ClientCnxn$Packet.class
>>> org/apache/zookeeper/ClientCnxn$SendThread.class
>>> org/apache/zookeeper/ClientCnxn$SessionExpiredException.class
>>> org/apache/zookeeper/ClientCnxn$SessionTimeoutException.class
>>> org/apache/zookeeper/ClientCnxn$WatcherSetEventPair.class
>>> org/apache/zookeeper/ClientCnxn.class
>>> org/apache/zookeeper/ClientWatchManager.class
>>> org/apache/zookeeper/ServerAdminClient.class
>>>
>>> And that jarfile is on the classpath, and I see on strace that it's
>>> finding
>>> it... Why is it not finding the class, then?
>>>
>>> I'm a bit lost... Is there any way I can run it with more verbose? Strace
>>> is too verbose, "bash -x" is not that much. Something in between?
>>>
>>>
>>>
>>>
>>> On Sat, Nov 1, 2014 at 4:53 AM, Gwen Shapira <gs...@cloudera.com>
>>> wrote:
>>>
>>> > The file should be your Scala jar.
>>> > You should have it in your Kafka lib directory.
>>> > Do a "jar -t" to make sure it actually contains the Tuple class.
>>> >
>>> > Gwen
>>> >
>>> > On Fri, Oct 31, 2014 at 8:49 PM, Tomas Nunez <nu...@pythian.com>
>>> wrote:
>>> >
>>> > > Thanks for pointing the error. I fixed it, but I'm still getting the
>>> same
>>> > > error. The content of migrationToolProducer.properties is now:
>>> > > _____
>>> > >
>>> > >
>>> >
>>> metadata.broker.list=kafka_new01:9092,kafka_new02:9092,kafka_new03:9092,kafka_new04:9092,kafka_new05:9092,
>>> > > group.id=migration.prod
>>> > > _____
>>> > >
>>> > > And the error still is:
>>> > >
>>> > > Exception in thread "main" java.lang.NoClassDefFoundError:
>>> > > scala/Tuple2$mcLL$sp
>>> > > at kafka.utils.Utils.stackTrace(Utils.scala)
>>> > > at
>>> > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:268)
>>> > > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
>>> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>>> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>>> > > at java.security.AccessController.doPrivileged(Native Method)
>>> > > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>>> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>> > > at
>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>>> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>> > > ... 2 more
>>> > >
>>> > > I have no "Tuple2*" file anywhere in the server. I'm following
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/Migrating+from+0.7+to+0.8
>>> > > and I can't see there anything about downloading classes, and I don't
>>> > find
>>> > > much people with the same problem, which leads me to think that I'm
>>> doing
>>> > > something wrong...
>>> > >
>>> > > Any idea on what to try?
>>> > >
>>> > > Thanks!
>>> > >
>>> > > PD: Of course, I'll tell Alex you say hi :)
>>> > >
>>> > > On Sat, Nov 1, 2014 at 4:32 AM, Gwen Shapira <gs...@cloudera.com>
>>> > > wrote:
>>> > >
>>> > > > This is part of Scala, so it should be in the scala-library-...jar
>>> > > >
>>> > > >
>>> > > > On Fri, Oct 31, 2014 at 8:26 PM, Tomas Nunez <nu...@pythian.com>
>>> > wrote:
>>> > > >
>>> > > > > Well... I used strace and I found it was looking for some
>>> classes in
>>> > a
>>> > > > > wrong path. I fixed most of them, but there's one that isn't
>>> > anywhere,
>>> > > > > neither the new nor the old kafka servers:
>>> > > > >
>>> > > > > $ strace -o logfile -s 1000 -f kafka/bin/kafka-run-class.sh
>>> > > > > kafka.tools.KafkaMigrationTool --kafka.07.jar
>>> > > > > migration-tool/kafka-0.7.2.jar --zkclient.01.jar
>>> > > > > migration-tool/zkclient-0.1.jar
>>> > > > > --consumer.config=kafka/config/migrationToolConsumer.properties
>>> > > > > -producer.config=kafka/config/migrationToolProducer.properties
>>> > > > > --whitelist=.*
>>> > > > > Exception in thread "main" java.lang.NoClassDefFoundError:
>>> > > > > scala/Tuple2$mcLL$sp
>>> > > > > at kafka.utils.Utils.stackTrace(Utils.scala)
>>> > > > > at
>>> > > > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:268)
>>> > > > > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
>>> > > > > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>>> > > > > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>>> > > > > at java.security.AccessController.doPrivileged(Native
>>> Method)
>>> > > > > at
>>> java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>>> > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>> > > > > at
>>> > > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>>> > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>> > > > > ... 2 more
>>> > > > >
>>> > > > > $ grep ENOENT logfile|tail -1
>>> > > > > 16484
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> stat("migration-tool/kafka07/core/target/scala_2.8.0/classes/scala/Tuple2$mcLL$sp.class",
>>> > > > > 0x7f59b27f4950) = -1 ENOENT (No such file or directory)
>>> > > > >
>>> > > > > I saw someone had a similar problem in this list, but it's not
>>> clear
>>> > > (at
>>> > > > > least to me) how it was fixed in the end
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> http://mail-archives.apache.org/mod_mbox/kafka-dev/201302.mbox/%3CCANEBM33rMTZ2UGp2dpaFj8xV=n4=Rr9GP3Six0sko++nhQv4kQ@mail.gmail.com%3E
>>> > > > >
>>> > > > > The JIRA ticket is still untouched
>>> > > > > https://issues.apache.org/jira/browse/KAFKA-760
>>> > > > >
>>> > > > > May different versions of scala be the cause of this error? I
>>> just
>>> > used
>>> > > > > "./sbt update" to make sure...
>>> > > > >
>>> > > > > If not, where should I find that class file?
>>> > > > >
>>> > > > > Thanks again!
>>> > > > >
>>> > > > > On Sat, Nov 1, 2014 at 4:05 AM, Tomas Nunez <nu...@pythian.com>
>>> > wrote:
>>> > > > >
>>> > > > > > Hi
>>> > > > > >
>>> > > > > > I'm trying to upgrade a 0.7 kafka cluster, but I'm getting an
>>> > error:
>>> > > > > >
>>> > > > > > I created the file migrationToolConsumer.properties with just
>>> > theinfo
>>> > > > to
>>> > > > > > connect to the old cluster:
>>> > > > > > _____
>>> > > > > >
>>> > zookeeper.connect=zookeeper01:2181,zookeeper02:2181,zookeeper03:2181
>>> > > > > > group.id=migration.prod
>>> > > > > > ______
>>> > > > > >
>>> > > > > >
>>> > > > > > Then the producer file
>>> > > > > > _______
>>> > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> zookeeper.connect=zookeeper01_new:2181,zookeeper02_new:2181,zookeeper03_new:2181
>>> > > > > > group.id=migration.prod
>>> > > > > > _______
>>> > > > > >
>>> > > > > > And then I called the migrationtool, and get this error
>>> > > > > >
>>> > > > > > ______________
>>> > > > > > $ kafka/bin/kafka-run-class.sh kafka.tools.KafkaMigrationTool
>>> > > > > > --kafka.07.jar migration-tool/kafka-0.7.2.jar --zkclient.01.jar
>>> > > > > > migration-tool/zkclient-0.1.jar --num.producers 4
>>> > > > > > --consumer.config=kafka/config/migrationToolConsumer.properties
>>> > > > > > -producer.config=kafka/config/migrationToolProducer.properties
>>> > > > > > --whitelist=.*
>>> > > > > > Kafka migration tool failed due to:
>>> > > > > > java.lang.reflect.InvocationTargetException
>>> > > > > > at
>>> > > > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>>> > > > > > Method)
>>> > > > > > at
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>>> > > > > > at
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>> > > > > > at
>>> > > > > java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>>> > > > > > at
>>> > > > > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
>>> > > > > > Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
>>> > > > > > at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
>>> > > > > > at
>>> > > > kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
>>> > > > > > ... 5 more
>>> > > > > > Caused by: java.lang.ClassNotFoundException:
>>> scala.Tuple2$mcLL$sp
>>> > > > > > at
>>> java.lang.ClassLoader.findClass(ClassLoader.java:531)
>>> > > > > > at
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
>>> > > > > > at
>>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>> > > > > > at
>>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>> > > > > > at
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
>>> > > > > > at
>>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>> > > > > > at
>>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>> > > > > > ... 7 more
>>> > > > > >
>>> > > > > > [2014-11-01 02:51:35,124] ERROR Kafka migration tool failed:
>>> > > > > > (kafka.tools.KafkaMigrationTool)
>>> > > > > > java.lang.reflect.InvocationTargetException
>>> > > > > > at
>>> > > > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>>> > > > > > Method)
>>> > > > > > at
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>>> > > > > > at
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>> > > > > > at
>>> > > > > java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>>> > > > > > at
>>> > > > > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
>>> > > > > > Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
>>> > > > > > at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
>>> > > > > > at
>>> > > > kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
>>> > > > > > ... 5 more
>>> > > > > > Caused by: java.lang.ClassNotFoundException:
>>> scala.Tuple2$mcLL$sp
>>> > > > > > at
>>> java.lang.ClassLoader.findClass(ClassLoader.java:531)
>>> > > > > > at
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
>>> > > > > > at
>>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>> > > > > > at
>>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>> > > > > > at
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
>>> > > > > > at
>>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>> > > > > > at
>>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>> > > > > > ... 7 more
>>> > > > > > __________
>>> > > > > >
>>> > > > > >
>>> > > > > > The files kafka-0.7.2.jar and zkclient-0.1.jar are the exact
>>> same
>>> > > ones
>>> > > > > > that are being used in the old 0.7 cluster.
>>> > > > > >
>>> > > > > > I also tried downloading and building a new 0.7.2 jar file with
>>> > > scala,
>>> > > > > but
>>> > > > > > it had the same error, too.
>>> > > > > >
>>> > > > > > What can I be doing wrong? Is there any way to get more output
>>> to
>>> > see
>>> > > > > > where is the error?
>>> > > > > >
>>> > > > > > Thanks in advance,
>>> > > > > >
>>> > > > > > --
>>> > > > > > Tomàs Núñez
>>> > > > > > Enterprise Infrastructure Consultant
>>> > > > > > The Pythian Group - Love your data!
>>> > > > > >
>>> > > > > > Office (international): +1 613 565 8696 x1501
>>> > > > > >
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > --
>>> > > > > Tomàs Núñez
>>> > > > > Enterprise Infrastructure Consultant
>>> > > > > The Pythian Group - Love your data!
>>> > > > >
>>> > > > > Office (international): +1 613 565 8696 x1501
>>> > > > >
>>> > > > > --
>>> > > > >
>>> > > > >
>>> > > > > --
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> > >
>>> > >
>>> > > --
>>> > > Tomàs Núñez
>>> > > Enterprise Infrastructure Consultant
>>> > > The Pythian Group - Love your data!
>>> > >
>>> > > Office (international): +1 613 565 8696 x1501
>>> > >
>>> > > --
>>> > >
>>> > >
>>> > > --
>>> > >
>>> > >
>>> > >
>>> > >
>>> >
>>>
>>>
>>>
>>> --
>>> Tomàs Núñez
>>> Enterprise Infrastructure Consultant
>>> The Pythian Group - Love your data!
>>>
>>> Office (international): +1 613 565 8696 x1501
>>>
>>> --
>>>
>>>
>>> --
>>>
>>>
>>>
>>>
>>
>
Re: Error using migrationtool for upgrading 0.7 to 0.8
Posted by Gwen Shapira <gs...@cloudera.com>.
Regarding more information:
Maybe ltrace?
If I were you, I'd go to MigrationTool code and start adding LOG lines.
because there aren't enough of those to troubleshoot.
On Wed, Nov 5, 2014 at 6:13 PM, Gwen Shapira <gs...@cloudera.com> wrote:
> org.apache.zookeeper.ClientCnxn is throwing the exception, so I'm 100%
> sure it eventually found the class.
>
> On Wed, Nov 5, 2014 at 5:59 PM, Tomas Nunez <nu...@pythian.com> wrote:
>
>> Ok, still fighting with the migrationTool here...
>>
>> That tuple wasn't in the scala-library.jar. It turns out I was using scala
>> 2.10 for kafka0.8 and scala 2.8 for kafka0.7, and the jar files were not
>> compatible. So, for the record, it seems that you need both the 0.7 jar
>> files and your 0.8 kafka compiled with the same java version.
>>
>> After fixing that (downloading kafka 0.8 compiled with scala 2.8), I'm now
>> facing a different error, this time more crypic:
>>
>> Kafka migration tool failed due to:
>> java.lang.reflect.InvocationTargetException
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at
>> kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:204)
>> Caused by: java.lang.NullPointerException
>> at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:361)
>> at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:332)
>> at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:383)
>> at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:64)
>> at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:872)
>> at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
>> at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
>> at
>>
>> kafka.consumer.ZookeeperConsumerConnector.connectZk(ZookeeperConsumerConnector.scala:152)
>> at
>>
>> kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:122)
>> at
>>
>> kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:65)
>> at
>>
>> kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:67)
>> at
>>
>> kafka.consumer.Consumer$.createJavaConsumerConnector(ConsumerConnector.scala:88)
>> at
>>
>> kafka.consumer.Consumer.createJavaConsumerConnector(ConsumerConnector.scala)
>> ... 5 more
>>
>> [2014-11-06 01:32:44,362] ERROR Kafka migration tool failed:
>> (kafka.tools.KafkaMigrationTool)
>> java.lang.reflect.InvocationTargetException
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at
>> kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:204)
>> Caused by: java.lang.NullPointerException
>> at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:361)
>> at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:332)
>> at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:383)
>> at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:64)
>> at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:872)
>> at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
>> at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
>> at
>>
>> kafka.consumer.ZookeeperConsumerConnector.connectZk(ZookeeperConsumerConnector.scala:152)
>> at
>>
>> kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:122)
>> at
>>
>> kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:65)
>> at
>>
>> kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:67)
>> at
>>
>> kafka.consumer.Consumer$.createJavaConsumerConnector(ConsumerConnector.scala:88)
>> at
>>
>> kafka.consumer.Consumer.createJavaConsumerConnector(ConsumerConnector.scala)
>> ... 5 more
>>
>> Using strace with it, I could see these errors again:
>>
>> 13342 stat("/var/migration/kafka/utils/Logging.class", 0x7fc92913c280) =
>> -1
>> ENOENT (No such file or directory)
>> 13342 stat("/var/migration/kafka/common/KafkaStorageException.class",
>> 0x7fc92913d950) = -1 ENOENT (No such file or directory)
>> 13342 stat("/var/migration/kafka/common/KafkaException.class",
>> 0x7fc92913d950) = -1 ENOENT (No such file or directory)
>> 13342 stat("/var/migration/kafka/utils/Logging$class.class",
>> 0x7fc92913de00) = -1 ENOENT (No such file or directory)
>> 13342 stat("/var/migration/kafka/utils/Log4jController$.class",
>> 0x7fc92913dc60) = -1 ENOENT (No such file or directory)
>> 13342 stat("/var/migration/kafka/utils/Log4jController.class",
>> 0x7fc92913d7c0) = -1 ENOENT (No such file or directory)
>> 13342 stat("/var/migration/kafka/utils/Log4jControllerMBean.class",
>> 0x7fc92913b4e0) = -1 ENOENT (No such file or directory)
>> 13342
>> stat("/var/migration/org/apache/log4j/spi/ThrowableInformation.class",
>> 0x7fc92913e860) = -1 ENOENT (No such file or directory)
>>
>> and after I cd to kafka-0.8.1.1-src/core/build/classes/main/, a lot of
>> those ENOENT disappear, but I still get the same "Null Pointer error".
>> Grepping "ENOENT" I see:
>>
>> 13257
>>
>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/Environment.class",
>> 0x7fee656ae750) = -1 ENOENT (No such file or directory)
>> 13257
>>
>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/Version.class",
>> 0x7fee656ae680) = -1 ENOENT (No such file or directory)
>> 13257
>>
>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/version/Info.class",
>> 0x7fee656ac3a0) = -1 ENOENT (No such file or directory)
>> 13257
>>
>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/Environment$Entry.class",
>> 0x7fee656ae8e0) = -1 ENOENT (No such file or directory)
>> 13257
>>
>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/ZooKeeper$ZKWatchManager.class",
>> 0x7fee656aede0) = -1 ENOENT (No such file or directory)
>> 13257
>>
>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/ClientCnxn.class",
>> 0x7fee656aede0) = -1 ENOENT (No such file or directory)
>> 13257
>>
>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/proto/RequestHeader.class",
>> 0x7fee656ae5f0) = -1 ENOENT (No such file or directory)
>> 13257
>>
>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/ClientCnxn$1.class",
>> 0x7fee656ae9a0) = -1 ENOENT (No such file or directory)
>> 13257
>>
>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/META-INF/services/java.nio.channels.spi.SelectorProvider",
>> 0x7fee656aed20) = -1 ENOENT (No such file or directory)
>> 13257
>>
>> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/log4j/spi/ThrowableInformation.class",
>> 0x7fee656af860) = -1 ENOENT (No such file or directory)
>>
>> I found, for instance ClientCnxn$1.class in zookeeper-3.3.4.jar
>>
>> $ jar tf ./core/build/dependant-libs-2.8.0/zookeeper-3.3.4.jar|grep Client
>> org/apache/zookeeper/ClientCnxn$1.class
>> org/apache/zookeeper/ClientCnxn$2.class
>> org/apache/zookeeper/ClientCnxn$AuthData.class
>> org/apache/zookeeper/ClientCnxn$EndOfStreamException.class
>> org/apache/zookeeper/ClientCnxn$EventThread.class
>> org/apache/zookeeper/ClientCnxn$Packet.class
>> org/apache/zookeeper/ClientCnxn$SendThread.class
>> org/apache/zookeeper/ClientCnxn$SessionExpiredException.class
>> org/apache/zookeeper/ClientCnxn$SessionTimeoutException.class
>> org/apache/zookeeper/ClientCnxn$WatcherSetEventPair.class
>> org/apache/zookeeper/ClientCnxn.class
>> org/apache/zookeeper/ClientWatchManager.class
>> org/apache/zookeeper/ServerAdminClient.class
>>
>> And that jarfile is on the classpath, and I see on strace that it's
>> finding
>> it... Why is it not finding the class, then?
>>
>> I'm a bit lost... Is there any way I can run it with more verbose? Strace
>> is too verbose, "bash -x" is not that much. Something in between?
>>
>>
>>
>>
>> On Sat, Nov 1, 2014 at 4:53 AM, Gwen Shapira <gs...@cloudera.com>
>> wrote:
>>
>> > The file should be your Scala jar.
>> > You should have it in your Kafka lib directory.
>> > Do a "jar -t" to make sure it actually contains the Tuple class.
>> >
>> > Gwen
>> >
>> > On Fri, Oct 31, 2014 at 8:49 PM, Tomas Nunez <nu...@pythian.com> wrote:
>> >
>> > > Thanks for pointing the error. I fixed it, but I'm still getting the
>> same
>> > > error. The content of migrationToolProducer.properties is now:
>> > > _____
>> > >
>> > >
>> >
>> metadata.broker.list=kafka_new01:9092,kafka_new02:9092,kafka_new03:9092,kafka_new04:9092,kafka_new05:9092,
>> > > group.id=migration.prod
>> > > _____
>> > >
>> > > And the error still is:
>> > >
>> > > Exception in thread "main" java.lang.NoClassDefFoundError:
>> > > scala/Tuple2$mcLL$sp
>> > > at kafka.utils.Utils.stackTrace(Utils.scala)
>> > > at
>> > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:268)
>> > > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
>> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> > > at java.security.AccessController.doPrivileged(Native Method)
>> > > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> > > at
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>> > > ... 2 more
>> > >
>> > > I have no "Tuple2*" file anywhere in the server. I'm following
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/Migrating+from+0.7+to+0.8
>> > > and I can't see there anything about downloading classes, and I don't
>> > find
>> > > much people with the same problem, which leads me to think that I'm
>> doing
>> > > something wrong...
>> > >
>> > > Any idea on what to try?
>> > >
>> > > Thanks!
>> > >
>> > > PD: Of course, I'll tell Alex you say hi :)
>> > >
>> > > On Sat, Nov 1, 2014 at 4:32 AM, Gwen Shapira <gs...@cloudera.com>
>> > > wrote:
>> > >
>> > > > This is part of Scala, so it should be in the scala-library-...jar
>> > > >
>> > > >
>> > > > On Fri, Oct 31, 2014 at 8:26 PM, Tomas Nunez <nu...@pythian.com>
>> > wrote:
>> > > >
>> > > > > Well... I used strace and I found it was looking for some classes
>> in
>> > a
>> > > > > wrong path. I fixed most of them, but there's one that isn't
>> > anywhere,
>> > > > > neither the new nor the old kafka servers:
>> > > > >
>> > > > > $ strace -o logfile -s 1000 -f kafka/bin/kafka-run-class.sh
>> > > > > kafka.tools.KafkaMigrationTool --kafka.07.jar
>> > > > > migration-tool/kafka-0.7.2.jar --zkclient.01.jar
>> > > > > migration-tool/zkclient-0.1.jar
>> > > > > --consumer.config=kafka/config/migrationToolConsumer.properties
>> > > > > -producer.config=kafka/config/migrationToolProducer.properties
>> > > > > --whitelist=.*
>> > > > > Exception in thread "main" java.lang.NoClassDefFoundError:
>> > > > > scala/Tuple2$mcLL$sp
>> > > > > at kafka.utils.Utils.stackTrace(Utils.scala)
>> > > > > at
>> > > > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:268)
>> > > > > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
>> > > > > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>> > > > > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> > > > > at java.security.AccessController.doPrivileged(Native
>> Method)
>> > > > > at
>> java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> > > > > at
>> > > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>> > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>> > > > > ... 2 more
>> > > > >
>> > > > > $ grep ENOENT logfile|tail -1
>> > > > > 16484
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> stat("migration-tool/kafka07/core/target/scala_2.8.0/classes/scala/Tuple2$mcLL$sp.class",
>> > > > > 0x7f59b27f4950) = -1 ENOENT (No such file or directory)
>> > > > >
>> > > > > I saw someone had a similar problem in this list, but it's not
>> clear
>> > > (at
>> > > > > least to me) how it was fixed in the end
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> http://mail-archives.apache.org/mod_mbox/kafka-dev/201302.mbox/%3CCANEBM33rMTZ2UGp2dpaFj8xV=n4=Rr9GP3Six0sko++nhQv4kQ@mail.gmail.com%3E
>> > > > >
>> > > > > The JIRA ticket is still untouched
>> > > > > https://issues.apache.org/jira/browse/KAFKA-760
>> > > > >
>> > > > > May different versions of scala be the cause of this error? I just
>> > used
>> > > > > "./sbt update" to make sure...
>> > > > >
>> > > > > If not, where should I find that class file?
>> > > > >
>> > > > > Thanks again!
>> > > > >
>> > > > > On Sat, Nov 1, 2014 at 4:05 AM, Tomas Nunez <nu...@pythian.com>
>> > wrote:
>> > > > >
>> > > > > > Hi
>> > > > > >
>> > > > > > I'm trying to upgrade a 0.7 kafka cluster, but I'm getting an
>> > error:
>> > > > > >
>> > > > > > I created the file migrationToolConsumer.properties with just
>> > theinfo
>> > > > to
>> > > > > > connect to the old cluster:
>> > > > > > _____
>> > > > > >
>> > zookeeper.connect=zookeeper01:2181,zookeeper02:2181,zookeeper03:2181
>> > > > > > group.id=migration.prod
>> > > > > > ______
>> > > > > >
>> > > > > >
>> > > > > > Then the producer file
>> > > > > > _______
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> zookeeper.connect=zookeeper01_new:2181,zookeeper02_new:2181,zookeeper03_new:2181
>> > > > > > group.id=migration.prod
>> > > > > > _______
>> > > > > >
>> > > > > > And then I called the migrationtool, and get this error
>> > > > > >
>> > > > > > ______________
>> > > > > > $ kafka/bin/kafka-run-class.sh kafka.tools.KafkaMigrationTool
>> > > > > > --kafka.07.jar migration-tool/kafka-0.7.2.jar --zkclient.01.jar
>> > > > > > migration-tool/zkclient-0.1.jar --num.producers 4
>> > > > > > --consumer.config=kafka/config/migrationToolConsumer.properties
>> > > > > > -producer.config=kafka/config/migrationToolProducer.properties
>> > > > > > --whitelist=.*
>> > > > > > Kafka migration tool failed due to:
>> > > > > > java.lang.reflect.InvocationTargetException
>> > > > > > at
>> > > > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>> > > > > > Method)
>> > > > > > at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>> > > > > > at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>> > > > > > at
>> > > > > java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>> > > > > > at
>> > > > > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
>> > > > > > Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
>> > > > > > at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
>> > > > > > at
>> > > > kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
>> > > > > > ... 5 more
>> > > > > > Caused by: java.lang.ClassNotFoundException:
>> scala.Tuple2$mcLL$sp
>> > > > > > at java.lang.ClassLoader.findClass(ClassLoader.java:531)
>> > > > > > at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
>> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>> > > > > > at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
>> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>> > > > > > ... 7 more
>> > > > > >
>> > > > > > [2014-11-01 02:51:35,124] ERROR Kafka migration tool failed:
>> > > > > > (kafka.tools.KafkaMigrationTool)
>> > > > > > java.lang.reflect.InvocationTargetException
>> > > > > > at
>> > > > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>> > > > > > Method)
>> > > > > > at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>> > > > > > at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>> > > > > > at
>> > > > > java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>> > > > > > at
>> > > > > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
>> > > > > > Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
>> > > > > > at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
>> > > > > > at
>> > > > kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
>> > > > > > ... 5 more
>> > > > > > Caused by: java.lang.ClassNotFoundException:
>> scala.Tuple2$mcLL$sp
>> > > > > > at java.lang.ClassLoader.findClass(ClassLoader.java:531)
>> > > > > > at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
>> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>> > > > > > at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
>> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>> > > > > > ... 7 more
>> > > > > > __________
>> > > > > >
>> > > > > >
>> > > > > > The files kafka-0.7.2.jar and zkclient-0.1.jar are the exact
>> same
>> > > ones
>> > > > > > that are being used in the old 0.7 cluster.
>> > > > > >
>> > > > > > I also tried downloading and building a new 0.7.2 jar file with
>> > > scala,
>> > > > > but
>> > > > > > it had the same error, too.
>> > > > > >
>> > > > > > What can I be doing wrong? Is there any way to get more output
>> to
>> > see
>> > > > > > where is the error?
>> > > > > >
>> > > > > > Thanks in advance,
>> > > > > >
>> > > > > > --
>> > > > > > Tomàs Núñez
>> > > > > > Enterprise Infrastructure Consultant
>> > > > > > The Pythian Group - Love your data!
>> > > > > >
>> > > > > > Office (international): +1 613 565 8696 x1501
>> > > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Tomàs Núñez
>> > > > > Enterprise Infrastructure Consultant
>> > > > > The Pythian Group - Love your data!
>> > > > >
>> > > > > Office (international): +1 613 565 8696 x1501
>> > > > >
>> > > > > --
>> > > > >
>> > > > >
>> > > > > --
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Tomàs Núñez
>> > > Enterprise Infrastructure Consultant
>> > > The Pythian Group - Love your data!
>> > >
>> > > Office (international): +1 613 565 8696 x1501
>> > >
>> > > --
>> > >
>> > >
>> > > --
>> > >
>> > >
>> > >
>> > >
>> >
>>
>>
>>
>> --
>> Tomàs Núñez
>> Enterprise Infrastructure Consultant
>> The Pythian Group - Love your data!
>>
>> Office (international): +1 613 565 8696 x1501
>>
>> --
>>
>>
>> --
>>
>>
>>
>>
>
Re: Error using migrationtool for upgrading 0.7 to 0.8
Posted by Gwen Shapira <gs...@cloudera.com>.
org.apache.zookeeper.ClientCnxn is throwing the exception, so I'm 100% sure
it eventually found the class.
On Wed, Nov 5, 2014 at 5:59 PM, Tomas Nunez <nu...@pythian.com> wrote:
> Ok, still fighting with the migrationTool here...
>
> That tuple wasn't in the scala-library.jar. It turns out I was using scala
> 2.10 for kafka0.8 and scala 2.8 for kafka0.7, and the jar files were not
> compatible. So, for the record, it seems that you need both the 0.7 jar
> files and your 0.8 kafka compiled with the same java version.
>
> After fixing that (downloading kafka 0.8 compiled with scala 2.8), I'm now
> facing a different error, this time more crypic:
>
> Kafka migration tool failed due to:
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:204)
> Caused by: java.lang.NullPointerException
> at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:361)
> at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:332)
> at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:383)
> at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:64)
> at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:872)
> at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
> at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
> at
>
> kafka.consumer.ZookeeperConsumerConnector.connectZk(ZookeeperConsumerConnector.scala:152)
> at
>
> kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:122)
> at
>
> kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:65)
> at
>
> kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:67)
> at
>
> kafka.consumer.Consumer$.createJavaConsumerConnector(ConsumerConnector.scala:88)
> at
>
> kafka.consumer.Consumer.createJavaConsumerConnector(ConsumerConnector.scala)
> ... 5 more
>
> [2014-11-06 01:32:44,362] ERROR Kafka migration tool failed:
> (kafka.tools.KafkaMigrationTool)
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:204)
> Caused by: java.lang.NullPointerException
> at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:361)
> at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:332)
> at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:383)
> at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:64)
> at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:872)
> at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
> at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
> at
>
> kafka.consumer.ZookeeperConsumerConnector.connectZk(ZookeeperConsumerConnector.scala:152)
> at
>
> kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:122)
> at
>
> kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:65)
> at
>
> kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:67)
> at
>
> kafka.consumer.Consumer$.createJavaConsumerConnector(ConsumerConnector.scala:88)
> at
>
> kafka.consumer.Consumer.createJavaConsumerConnector(ConsumerConnector.scala)
> ... 5 more
>
> Using strace with it, I could see these errors again:
>
> 13342 stat("/var/migration/kafka/utils/Logging.class", 0x7fc92913c280) = -1
> ENOENT (No such file or directory)
> 13342 stat("/var/migration/kafka/common/KafkaStorageException.class",
> 0x7fc92913d950) = -1 ENOENT (No such file or directory)
> 13342 stat("/var/migration/kafka/common/KafkaException.class",
> 0x7fc92913d950) = -1 ENOENT (No such file or directory)
> 13342 stat("/var/migration/kafka/utils/Logging$class.class",
> 0x7fc92913de00) = -1 ENOENT (No such file or directory)
> 13342 stat("/var/migration/kafka/utils/Log4jController$.class",
> 0x7fc92913dc60) = -1 ENOENT (No such file or directory)
> 13342 stat("/var/migration/kafka/utils/Log4jController.class",
> 0x7fc92913d7c0) = -1 ENOENT (No such file or directory)
> 13342 stat("/var/migration/kafka/utils/Log4jControllerMBean.class",
> 0x7fc92913b4e0) = -1 ENOENT (No such file or directory)
> 13342
> stat("/var/migration/org/apache/log4j/spi/ThrowableInformation.class",
> 0x7fc92913e860) = -1 ENOENT (No such file or directory)
>
> and after I cd to kafka-0.8.1.1-src/core/build/classes/main/, a lot of
> those ENOENT disappear, but I still get the same "Null Pointer error".
> Grepping "ENOENT" I see:
>
> 13257
>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/Environment.class",
> 0x7fee656ae750) = -1 ENOENT (No such file or directory)
> 13257
>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/Version.class",
> 0x7fee656ae680) = -1 ENOENT (No such file or directory)
> 13257
>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/version/Info.class",
> 0x7fee656ac3a0) = -1 ENOENT (No such file or directory)
> 13257
>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/Environment$Entry.class",
> 0x7fee656ae8e0) = -1 ENOENT (No such file or directory)
> 13257
>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/ZooKeeper$ZKWatchManager.class",
> 0x7fee656aede0) = -1 ENOENT (No such file or directory)
> 13257
>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/ClientCnxn.class",
> 0x7fee656aede0) = -1 ENOENT (No such file or directory)
> 13257
>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/proto/RequestHeader.class",
> 0x7fee656ae5f0) = -1 ENOENT (No such file or directory)
> 13257
>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/ClientCnxn$1.class",
> 0x7fee656ae9a0) = -1 ENOENT (No such file or directory)
> 13257
>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/META-INF/services/java.nio.channels.spi.SelectorProvider",
> 0x7fee656aed20) = -1 ENOENT (No such file or directory)
> 13257
>
> stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/log4j/spi/ThrowableInformation.class",
> 0x7fee656af860) = -1 ENOENT (No such file or directory)
>
> I found, for instance ClientCnxn$1.class in zookeeper-3.3.4.jar
>
> $ jar tf ./core/build/dependant-libs-2.8.0/zookeeper-3.3.4.jar|grep Client
> org/apache/zookeeper/ClientCnxn$1.class
> org/apache/zookeeper/ClientCnxn$2.class
> org/apache/zookeeper/ClientCnxn$AuthData.class
> org/apache/zookeeper/ClientCnxn$EndOfStreamException.class
> org/apache/zookeeper/ClientCnxn$EventThread.class
> org/apache/zookeeper/ClientCnxn$Packet.class
> org/apache/zookeeper/ClientCnxn$SendThread.class
> org/apache/zookeeper/ClientCnxn$SessionExpiredException.class
> org/apache/zookeeper/ClientCnxn$SessionTimeoutException.class
> org/apache/zookeeper/ClientCnxn$WatcherSetEventPair.class
> org/apache/zookeeper/ClientCnxn.class
> org/apache/zookeeper/ClientWatchManager.class
> org/apache/zookeeper/ServerAdminClient.class
>
> And that jarfile is on the classpath, and I see on strace that it's finding
> it... Why is it not finding the class, then?
>
> I'm a bit lost... Is there any way I can run it with more verbose? Strace
> is too verbose, "bash -x" is not that much. Something in between?
>
>
>
>
> On Sat, Nov 1, 2014 at 4:53 AM, Gwen Shapira <gs...@cloudera.com>
> wrote:
>
> > The file should be your Scala jar.
> > You should have it in your Kafka lib directory.
> > Do a "jar -t" to make sure it actually contains the Tuple class.
> >
> > Gwen
> >
> > On Fri, Oct 31, 2014 at 8:49 PM, Tomas Nunez <nu...@pythian.com> wrote:
> >
> > > Thanks for pointing the error. I fixed it, but I'm still getting the
> same
> > > error. The content of migrationToolProducer.properties is now:
> > > _____
> > >
> > >
> >
> metadata.broker.list=kafka_new01:9092,kafka_new02:9092,kafka_new03:9092,kafka_new04:9092,kafka_new05:9092,
> > > group.id=migration.prod
> > > _____
> > >
> > > And the error still is:
> > >
> > > Exception in thread "main" java.lang.NoClassDefFoundError:
> > > scala/Tuple2$mcLL$sp
> > > at kafka.utils.Utils.stackTrace(Utils.scala)
> > > at
> > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:268)
> > > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > > at java.security.AccessController.doPrivileged(Native Method)
> > > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > ... 2 more
> > >
> > > I have no "Tuple2*" file anywhere in the server. I'm following
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Migrating+from+0.7+to+0.8
> > > and I can't see there anything about downloading classes, and I don't
> > find
> > > much people with the same problem, which leads me to think that I'm
> doing
> > > something wrong...
> > >
> > > Any idea on what to try?
> > >
> > > Thanks!
> > >
> > > PD: Of course, I'll tell Alex you say hi :)
> > >
> > > On Sat, Nov 1, 2014 at 4:32 AM, Gwen Shapira <gs...@cloudera.com>
> > > wrote:
> > >
> > > > This is part of Scala, so it should be in the scala-library-...jar
> > > >
> > > >
> > > > On Fri, Oct 31, 2014 at 8:26 PM, Tomas Nunez <nu...@pythian.com>
> > wrote:
> > > >
> > > > > Well... I used strace and I found it was looking for some classes
> in
> > a
> > > > > wrong path. I fixed most of them, but there's one that isn't
> > anywhere,
> > > > > neither the new nor the old kafka servers:
> > > > >
> > > > > $ strace -o logfile -s 1000 -f kafka/bin/kafka-run-class.sh
> > > > > kafka.tools.KafkaMigrationTool --kafka.07.jar
> > > > > migration-tool/kafka-0.7.2.jar --zkclient.01.jar
> > > > > migration-tool/zkclient-0.1.jar
> > > > > --consumer.config=kafka/config/migrationToolConsumer.properties
> > > > > -producer.config=kafka/config/migrationToolProducer.properties
> > > > > --whitelist=.*
> > > > > Exception in thread "main" java.lang.NoClassDefFoundError:
> > > > > scala/Tuple2$mcLL$sp
> > > > > at kafka.utils.Utils.stackTrace(Utils.scala)
> > > > > at
> > > > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:268)
> > > > > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> > > > > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> > > > > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > > > > at java.security.AccessController.doPrivileged(Native
> Method)
> > > > > at
> java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > > > at
> > > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > > > ... 2 more
> > > > >
> > > > > $ grep ENOENT logfile|tail -1
> > > > > 16484
> > > > >
> > > > >
> > > >
> > >
> >
> stat("migration-tool/kafka07/core/target/scala_2.8.0/classes/scala/Tuple2$mcLL$sp.class",
> > > > > 0x7f59b27f4950) = -1 ENOENT (No such file or directory)
> > > > >
> > > > > I saw someone had a similar problem in this list, but it's not
> clear
> > > (at
> > > > > least to me) how it was fixed in the end
> > > > >
> > > > >
> > > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201302.mbox/%3CCANEBM33rMTZ2UGp2dpaFj8xV=n4=Rr9GP3Six0sko++nhQv4kQ@mail.gmail.com%3E
> > > > >
> > > > > The JIRA ticket is still untouched
> > > > > https://issues.apache.org/jira/browse/KAFKA-760
> > > > >
> > > > > May different versions of scala be the cause of this error? I just
> > used
> > > > > "./sbt update" to make sure...
> > > > >
> > > > > If not, where should I find that class file?
> > > > >
> > > > > Thanks again!
> > > > >
> > > > > On Sat, Nov 1, 2014 at 4:05 AM, Tomas Nunez <nu...@pythian.com>
> > wrote:
> > > > >
> > > > > > Hi
> > > > > >
> > > > > > I'm trying to upgrade a 0.7 kafka cluster, but I'm getting an
> > error:
> > > > > >
> > > > > > I created the file migrationToolConsumer.properties with just
> > theinfo
> > > > to
> > > > > > connect to the old cluster:
> > > > > > _____
> > > > > >
> > zookeeper.connect=zookeeper01:2181,zookeeper02:2181,zookeeper03:2181
> > > > > > group.id=migration.prod
> > > > > > ______
> > > > > >
> > > > > >
> > > > > > Then the producer file
> > > > > > _______
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> zookeeper.connect=zookeeper01_new:2181,zookeeper02_new:2181,zookeeper03_new:2181
> > > > > > group.id=migration.prod
> > > > > > _______
> > > > > >
> > > > > > And then I called the migrationtool, and get this error
> > > > > >
> > > > > > ______________
> > > > > > $ kafka/bin/kafka-run-class.sh kafka.tools.KafkaMigrationTool
> > > > > > --kafka.07.jar migration-tool/kafka-0.7.2.jar --zkclient.01.jar
> > > > > > migration-tool/zkclient-0.1.jar --num.producers 4
> > > > > > --consumer.config=kafka/config/migrationToolConsumer.properties
> > > > > > -producer.config=kafka/config/migrationToolProducer.properties
> > > > > > --whitelist=.*
> > > > > > Kafka migration tool failed due to:
> > > > > > java.lang.reflect.InvocationTargetException
> > > > > > at
> > > > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > > > > > Method)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > > > > > at
> > > > > java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> > > > > > at
> > > > > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
> > > > > > Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
> > > > > > at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
> > > > > > at
> > > > kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
> > > > > > ... 5 more
> > > > > > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> > > > > > at java.lang.ClassLoader.findClass(ClassLoader.java:531)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > > > > ... 7 more
> > > > > >
> > > > > > [2014-11-01 02:51:35,124] ERROR Kafka migration tool failed:
> > > > > > (kafka.tools.KafkaMigrationTool)
> > > > > > java.lang.reflect.InvocationTargetException
> > > > > > at
> > > > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > > > > > Method)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > > > > > at
> > > > > java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> > > > > > at
> > > > > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
> > > > > > Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
> > > > > > at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
> > > > > > at
> > > > kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
> > > > > > ... 5 more
> > > > > > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> > > > > > at java.lang.ClassLoader.findClass(ClassLoader.java:531)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > > > > ... 7 more
> > > > > > __________
> > > > > >
> > > > > >
> > > > > > The files kafka-0.7.2.jar and zkclient-0.1.jar are the exact
> same
> > > ones
> > > > > > that are being used in the old 0.7 cluster.
> > > > > >
> > > > > > I also tried downloading and building a new 0.7.2 jar file with
> > > scala,
> > > > > but
> > > > > > it had the same error, too.
> > > > > >
> > > > > > What can I be doing wrong? Is there any way to get more output to
> > see
> > > > > > where is the error?
> > > > > >
> > > > > > Thanks in advance,
> > > > > >
> > > > > > --
> > > > > > Tomàs Núñez
> > > > > > Enterprise Infrastructure Consultant
> > > > > > The Pythian Group - Love your data!
> > > > > >
> > > > > > Office (international): +1 613 565 8696 x1501
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Tomàs Núñez
> > > > > Enterprise Infrastructure Consultant
> > > > > The Pythian Group - Love your data!
> > > > >
> > > > > Office (international): +1 613 565 8696 x1501
> > > > >
> > > > > --
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Tomàs Núñez
> > > Enterprise Infrastructure Consultant
> > > The Pythian Group - Love your data!
> > >
> > > Office (international): +1 613 565 8696 x1501
> > >
> > > --
> > >
> > >
> > > --
> > >
> > >
> > >
> > >
> >
>
>
>
> --
> Tomàs Núñez
> Enterprise Infrastructure Consultant
> The Pythian Group - Love your data!
>
> Office (international): +1 613 565 8696 x1501
>
> --
>
>
> --
>
>
>
>
Re: Error using migrationtool for upgrading 0.7 to 0.8
Posted by Tomas Nunez <nu...@pythian.com>.
Ok, still fighting with the migrationTool here...
That tuple wasn't in the scala-library.jar. It turns out I was using scala
2.10 for kafka0.8 and scala 2.8 for kafka0.7, and the jar files were not
compatible. So, for the record, it seems that you need both the 0.7 jar
files and your 0.8 kafka compiled with the same java version.
After fixing that (downloading kafka 0.8 compiled with scala 2.8), I'm now
facing a different error, this time more crypic:
Kafka migration tool failed due to:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:204)
Caused by: java.lang.NullPointerException
at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:361)
at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:332)
at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:383)
at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:64)
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:872)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
at
kafka.consumer.ZookeeperConsumerConnector.connectZk(ZookeeperConsumerConnector.scala:152)
at
kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:122)
at
kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:65)
at
kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:67)
at
kafka.consumer.Consumer$.createJavaConsumerConnector(ConsumerConnector.scala:88)
at
kafka.consumer.Consumer.createJavaConsumerConnector(ConsumerConnector.scala)
... 5 more
[2014-11-06 01:32:44,362] ERROR Kafka migration tool failed:
(kafka.tools.KafkaMigrationTool)
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:204)
Caused by: java.lang.NullPointerException
at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:361)
at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:332)
at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:383)
at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:64)
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:872)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
at
kafka.consumer.ZookeeperConsumerConnector.connectZk(ZookeeperConsumerConnector.scala:152)
at
kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:122)
at
kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:65)
at
kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:67)
at
kafka.consumer.Consumer$.createJavaConsumerConnector(ConsumerConnector.scala:88)
at
kafka.consumer.Consumer.createJavaConsumerConnector(ConsumerConnector.scala)
... 5 more
Using strace with it, I could see these errors again:
13342 stat("/var/migration/kafka/utils/Logging.class", 0x7fc92913c280) = -1
ENOENT (No such file or directory)
13342 stat("/var/migration/kafka/common/KafkaStorageException.class",
0x7fc92913d950) = -1 ENOENT (No such file or directory)
13342 stat("/var/migration/kafka/common/KafkaException.class",
0x7fc92913d950) = -1 ENOENT (No such file or directory)
13342 stat("/var/migration/kafka/utils/Logging$class.class",
0x7fc92913de00) = -1 ENOENT (No such file or directory)
13342 stat("/var/migration/kafka/utils/Log4jController$.class",
0x7fc92913dc60) = -1 ENOENT (No such file or directory)
13342 stat("/var/migration/kafka/utils/Log4jController.class",
0x7fc92913d7c0) = -1 ENOENT (No such file or directory)
13342 stat("/var/migration/kafka/utils/Log4jControllerMBean.class",
0x7fc92913b4e0) = -1 ENOENT (No such file or directory)
13342
stat("/var/migration/org/apache/log4j/spi/ThrowableInformation.class",
0x7fc92913e860) = -1 ENOENT (No such file or directory)
and after I cd to kafka-0.8.1.1-src/core/build/classes/main/, a lot of
those ENOENT disappear, but I still get the same "Null Pointer error".
Grepping "ENOENT" I see:
13257
stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/Environment.class",
0x7fee656ae750) = -1 ENOENT (No such file or directory)
13257
stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/Version.class",
0x7fee656ae680) = -1 ENOENT (No such file or directory)
13257
stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/version/Info.class",
0x7fee656ac3a0) = -1 ENOENT (No such file or directory)
13257
stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/Environment$Entry.class",
0x7fee656ae8e0) = -1 ENOENT (No such file or directory)
13257
stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/ZooKeeper$ZKWatchManager.class",
0x7fee656aede0) = -1 ENOENT (No such file or directory)
13257
stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/ClientCnxn.class",
0x7fee656aede0) = -1 ENOENT (No such file or directory)
13257
stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/proto/RequestHeader.class",
0x7fee656ae5f0) = -1 ENOENT (No such file or directory)
13257
stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/zookeeper/ClientCnxn$1.class",
0x7fee656ae9a0) = -1 ENOENT (No such file or directory)
13257
stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/META-INF/services/java.nio.channels.spi.SelectorProvider",
0x7fee656aed20) = -1 ENOENT (No such file or directory)
13257
stat("/var/migration/migration-tool/kafka-0.8.1.1-src/core/build/classes/main/org/apache/log4j/spi/ThrowableInformation.class",
0x7fee656af860) = -1 ENOENT (No such file or directory)
I found, for instance ClientCnxn$1.class in zookeeper-3.3.4.jar
$ jar tf ./core/build/dependant-libs-2.8.0/zookeeper-3.3.4.jar|grep Client
org/apache/zookeeper/ClientCnxn$1.class
org/apache/zookeeper/ClientCnxn$2.class
org/apache/zookeeper/ClientCnxn$AuthData.class
org/apache/zookeeper/ClientCnxn$EndOfStreamException.class
org/apache/zookeeper/ClientCnxn$EventThread.class
org/apache/zookeeper/ClientCnxn$Packet.class
org/apache/zookeeper/ClientCnxn$SendThread.class
org/apache/zookeeper/ClientCnxn$SessionExpiredException.class
org/apache/zookeeper/ClientCnxn$SessionTimeoutException.class
org/apache/zookeeper/ClientCnxn$WatcherSetEventPair.class
org/apache/zookeeper/ClientCnxn.class
org/apache/zookeeper/ClientWatchManager.class
org/apache/zookeeper/ServerAdminClient.class
And that jarfile is on the classpath, and I see on strace that it's finding
it... Why is it not finding the class, then?
I'm a bit lost... Is there any way I can run it with more verbose? Strace
is too verbose, "bash -x" is not that much. Something in between?
On Sat, Nov 1, 2014 at 4:53 AM, Gwen Shapira <gs...@cloudera.com> wrote:
> The file should be your Scala jar.
> You should have it in your Kafka lib directory.
> Do a "jar -t" to make sure it actually contains the Tuple class.
>
> Gwen
>
> On Fri, Oct 31, 2014 at 8:49 PM, Tomas Nunez <nu...@pythian.com> wrote:
>
> > Thanks for pointing the error. I fixed it, but I'm still getting the same
> > error. The content of migrationToolProducer.properties is now:
> > _____
> >
> >
> metadata.broker.list=kafka_new01:9092,kafka_new02:9092,kafka_new03:9092,kafka_new04:9092,kafka_new05:9092,
> > group.id=migration.prod
> > _____
> >
> > And the error still is:
> >
> > Exception in thread "main" java.lang.NoClassDefFoundError:
> > scala/Tuple2$mcLL$sp
> > at kafka.utils.Utils.stackTrace(Utils.scala)
> > at
> kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:268)
> > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > ... 2 more
> >
> > I have no "Tuple2*" file anywhere in the server. I'm following
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Migrating+from+0.7+to+0.8
> > and I can't see there anything about downloading classes, and I don't
> find
> > much people with the same problem, which leads me to think that I'm doing
> > something wrong...
> >
> > Any idea on what to try?
> >
> > Thanks!
> >
> > PD: Of course, I'll tell Alex you say hi :)
> >
> > On Sat, Nov 1, 2014 at 4:32 AM, Gwen Shapira <gs...@cloudera.com>
> > wrote:
> >
> > > This is part of Scala, so it should be in the scala-library-...jar
> > >
> > >
> > > On Fri, Oct 31, 2014 at 8:26 PM, Tomas Nunez <nu...@pythian.com>
> wrote:
> > >
> > > > Well... I used strace and I found it was looking for some classes in
> a
> > > > wrong path. I fixed most of them, but there's one that isn't
> anywhere,
> > > > neither the new nor the old kafka servers:
> > > >
> > > > $ strace -o logfile -s 1000 -f kafka/bin/kafka-run-class.sh
> > > > kafka.tools.KafkaMigrationTool --kafka.07.jar
> > > > migration-tool/kafka-0.7.2.jar --zkclient.01.jar
> > > > migration-tool/zkclient-0.1.jar
> > > > --consumer.config=kafka/config/migrationToolConsumer.properties
> > > > -producer.config=kafka/config/migrationToolProducer.properties
> > > > --whitelist=.*
> > > > Exception in thread "main" java.lang.NoClassDefFoundError:
> > > > scala/Tuple2$mcLL$sp
> > > > at kafka.utils.Utils.stackTrace(Utils.scala)
> > > > at
> > > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:268)
> > > > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> > > > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> > > > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > > > at java.security.AccessController.doPrivileged(Native Method)
> > > > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > > at
> > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > > ... 2 more
> > > >
> > > > $ grep ENOENT logfile|tail -1
> > > > 16484
> > > >
> > > >
> > >
> >
> stat("migration-tool/kafka07/core/target/scala_2.8.0/classes/scala/Tuple2$mcLL$sp.class",
> > > > 0x7f59b27f4950) = -1 ENOENT (No such file or directory)
> > > >
> > > > I saw someone had a similar problem in this list, but it's not clear
> > (at
> > > > least to me) how it was fixed in the end
> > > >
> > > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201302.mbox/%3CCANEBM33rMTZ2UGp2dpaFj8xV=n4=Rr9GP3Six0sko++nhQv4kQ@mail.gmail.com%3E
> > > >
> > > > The JIRA ticket is still untouched
> > > > https://issues.apache.org/jira/browse/KAFKA-760
> > > >
> > > > May different versions of scala be the cause of this error? I just
> used
> > > > "./sbt update" to make sure...
> > > >
> > > > If not, where should I find that class file?
> > > >
> > > > Thanks again!
> > > >
> > > > On Sat, Nov 1, 2014 at 4:05 AM, Tomas Nunez <nu...@pythian.com>
> wrote:
> > > >
> > > > > Hi
> > > > >
> > > > > I'm trying to upgrade a 0.7 kafka cluster, but I'm getting an
> error:
> > > > >
> > > > > I created the file migrationToolConsumer.properties with just
> theinfo
> > > to
> > > > > connect to the old cluster:
> > > > > _____
> > > > >
> zookeeper.connect=zookeeper01:2181,zookeeper02:2181,zookeeper03:2181
> > > > > group.id=migration.prod
> > > > > ______
> > > > >
> > > > >
> > > > > Then the producer file
> > > > > _______
> > > > >
> > > > >
> > > >
> > >
> >
> zookeeper.connect=zookeeper01_new:2181,zookeeper02_new:2181,zookeeper03_new:2181
> > > > > group.id=migration.prod
> > > > > _______
> > > > >
> > > > > And then I called the migrationtool, and get this error
> > > > >
> > > > > ______________
> > > > > $ kafka/bin/kafka-run-class.sh kafka.tools.KafkaMigrationTool
> > > > > --kafka.07.jar migration-tool/kafka-0.7.2.jar --zkclient.01.jar
> > > > > migration-tool/zkclient-0.1.jar --num.producers 4
> > > > > --consumer.config=kafka/config/migrationToolConsumer.properties
> > > > > -producer.config=kafka/config/migrationToolProducer.properties
> > > > > --whitelist=.*
> > > > > Kafka migration tool failed due to:
> > > > > java.lang.reflect.InvocationTargetException
> > > > > at
> > > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > > > > Method)
> > > > > at
> > > > >
> > > >
> > >
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> > > > > at
> > > > >
> > > >
> > >
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > > > > at
> > > > java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> > > > > at
> > > > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
> > > > > Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
> > > > > at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
> > > > > at
> > > kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
> > > > > ... 5 more
> > > > > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> > > > > at java.lang.ClassLoader.findClass(ClassLoader.java:531)
> > > > > at
> > > > >
> > > >
> > >
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
> > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > > > at
> > > > >
> > > >
> > >
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
> > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > > > ... 7 more
> > > > >
> > > > > [2014-11-01 02:51:35,124] ERROR Kafka migration tool failed:
> > > > > (kafka.tools.KafkaMigrationTool)
> > > > > java.lang.reflect.InvocationTargetException
> > > > > at
> > > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > > > > Method)
> > > > > at
> > > > >
> > > >
> > >
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> > > > > at
> > > > >
> > > >
> > >
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > > > > at
> > > > java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> > > > > at
> > > > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
> > > > > Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
> > > > > at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
> > > > > at
> > > kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
> > > > > ... 5 more
> > > > > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> > > > > at java.lang.ClassLoader.findClass(ClassLoader.java:531)
> > > > > at
> > > > >
> > > >
> > >
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
> > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > > > at
> > > > >
> > > >
> > >
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
> > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > > > ... 7 more
> > > > > __________
> > > > >
> > > > >
> > > > > The files kafka-0.7.2.jar and zkclient-0.1.jar are the exact same
> > ones
> > > > > that are being used in the old 0.7 cluster.
> > > > >
> > > > > I also tried downloading and building a new 0.7.2 jar file with
> > scala,
> > > > but
> > > > > it had the same error, too.
> > > > >
> > > > > What can I be doing wrong? Is there any way to get more output to
> see
> > > > > where is the error?
> > > > >
> > > > > Thanks in advance,
> > > > >
> > > > > --
> > > > > Tomàs Núñez
> > > > > Enterprise Infrastructure Consultant
> > > > > The Pythian Group - Love your data!
> > > > >
> > > > > Office (international): +1 613 565 8696 x1501
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Tomàs Núñez
> > > > Enterprise Infrastructure Consultant
> > > > The Pythian Group - Love your data!
> > > >
> > > > Office (international): +1 613 565 8696 x1501
> > > >
> > > > --
> > > >
> > > >
> > > > --
> > > >
> > > >
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Tomàs Núñez
> > Enterprise Infrastructure Consultant
> > The Pythian Group - Love your data!
> >
> > Office (international): +1 613 565 8696 x1501
> >
> > --
> >
> >
> > --
> >
> >
> >
> >
>
--
Tomàs Núñez
Enterprise Infrastructure Consultant
The Pythian Group - Love your data!
Office (international): +1 613 565 8696 x1501
--
--
Re: Error using migrationtool for upgrading 0.7 to 0.8
Posted by Gwen Shapira <gs...@cloudera.com>.
The file should be your Scala jar.
You should have it in your Kafka lib directory.
Do a "jar -t" to make sure it actually contains the Tuple class.
Gwen
On Fri, Oct 31, 2014 at 8:49 PM, Tomas Nunez <nu...@pythian.com> wrote:
> Thanks for pointing the error. I fixed it, but I'm still getting the same
> error. The content of migrationToolProducer.properties is now:
> _____
>
> metadata.broker.list=kafka_new01:9092,kafka_new02:9092,kafka_new03:9092,kafka_new04:9092,kafka_new05:9092,
> group.id=migration.prod
> _____
>
> And the error still is:
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> scala/Tuple2$mcLL$sp
> at kafka.utils.Utils.stackTrace(Utils.scala)
> at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:268)
> Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ... 2 more
>
> I have no "Tuple2*" file anywhere in the server. I'm following
> https://cwiki.apache.org/confluence/display/KAFKA/Migrating+from+0.7+to+0.8
> and I can't see there anything about downloading classes, and I don't find
> much people with the same problem, which leads me to think that I'm doing
> something wrong...
>
> Any idea on what to try?
>
> Thanks!
>
> PD: Of course, I'll tell Alex you say hi :)
>
> On Sat, Nov 1, 2014 at 4:32 AM, Gwen Shapira <gs...@cloudera.com>
> wrote:
>
> > This is part of Scala, so it should be in the scala-library-...jar
> >
> >
> > On Fri, Oct 31, 2014 at 8:26 PM, Tomas Nunez <nu...@pythian.com> wrote:
> >
> > > Well... I used strace and I found it was looking for some classes in a
> > > wrong path. I fixed most of them, but there's one that isn't anywhere,
> > > neither the new nor the old kafka servers:
> > >
> > > $ strace -o logfile -s 1000 -f kafka/bin/kafka-run-class.sh
> > > kafka.tools.KafkaMigrationTool --kafka.07.jar
> > > migration-tool/kafka-0.7.2.jar --zkclient.01.jar
> > > migration-tool/zkclient-0.1.jar
> > > --consumer.config=kafka/config/migrationToolConsumer.properties
> > > -producer.config=kafka/config/migrationToolProducer.properties
> > > --whitelist=.*
> > > Exception in thread "main" java.lang.NoClassDefFoundError:
> > > scala/Tuple2$mcLL$sp
> > > at kafka.utils.Utils.stackTrace(Utils.scala)
> > > at
> > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:268)
> > > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > > at java.security.AccessController.doPrivileged(Native Method)
> > > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > ... 2 more
> > >
> > > $ grep ENOENT logfile|tail -1
> > > 16484
> > >
> > >
> >
> stat("migration-tool/kafka07/core/target/scala_2.8.0/classes/scala/Tuple2$mcLL$sp.class",
> > > 0x7f59b27f4950) = -1 ENOENT (No such file or directory)
> > >
> > > I saw someone had a similar problem in this list, but it's not clear
> (at
> > > least to me) how it was fixed in the end
> > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201302.mbox/%3CCANEBM33rMTZ2UGp2dpaFj8xV=n4=Rr9GP3Six0sko++nhQv4kQ@mail.gmail.com%3E
> > >
> > > The JIRA ticket is still untouched
> > > https://issues.apache.org/jira/browse/KAFKA-760
> > >
> > > May different versions of scala be the cause of this error? I just used
> > > "./sbt update" to make sure...
> > >
> > > If not, where should I find that class file?
> > >
> > > Thanks again!
> > >
> > > On Sat, Nov 1, 2014 at 4:05 AM, Tomas Nunez <nu...@pythian.com> wrote:
> > >
> > > > Hi
> > > >
> > > > I'm trying to upgrade a 0.7 kafka cluster, but I'm getting an error:
> > > >
> > > > I created the file migrationToolConsumer.properties with just theinfo
> > to
> > > > connect to the old cluster:
> > > > _____
> > > > zookeeper.connect=zookeeper01:2181,zookeeper02:2181,zookeeper03:2181
> > > > group.id=migration.prod
> > > > ______
> > > >
> > > >
> > > > Then the producer file
> > > > _______
> > > >
> > > >
> > >
> >
> zookeeper.connect=zookeeper01_new:2181,zookeeper02_new:2181,zookeeper03_new:2181
> > > > group.id=migration.prod
> > > > _______
> > > >
> > > > And then I called the migrationtool, and get this error
> > > >
> > > > ______________
> > > > $ kafka/bin/kafka-run-class.sh kafka.tools.KafkaMigrationTool
> > > > --kafka.07.jar migration-tool/kafka-0.7.2.jar --zkclient.01.jar
> > > > migration-tool/zkclient-0.1.jar --num.producers 4
> > > > --consumer.config=kafka/config/migrationToolConsumer.properties
> > > > -producer.config=kafka/config/migrationToolProducer.properties
> > > > --whitelist=.*
> > > > Kafka migration tool failed due to:
> > > > java.lang.reflect.InvocationTargetException
> > > > at
> > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > > > Method)
> > > > at
> > > >
> > >
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> > > > at
> > > >
> > >
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > > > at
> > > java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> > > > at
> > > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
> > > > Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
> > > > at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
> > > > at
> > kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
> > > > ... 5 more
> > > > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> > > > at java.lang.ClassLoader.findClass(ClassLoader.java:531)
> > > > at
> > > >
> > >
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
> > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > > at
> > > >
> > >
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
> > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > > ... 7 more
> > > >
> > > > [2014-11-01 02:51:35,124] ERROR Kafka migration tool failed:
> > > > (kafka.tools.KafkaMigrationTool)
> > > > java.lang.reflect.InvocationTargetException
> > > > at
> > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > > > Method)
> > > > at
> > > >
> > >
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> > > > at
> > > >
> > >
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > > > at
> > > java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> > > > at
> > > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
> > > > Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
> > > > at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
> > > > at
> > kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
> > > > ... 5 more
> > > > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> > > > at java.lang.ClassLoader.findClass(ClassLoader.java:531)
> > > > at
> > > >
> > >
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
> > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > > at
> > > >
> > >
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
> > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > > ... 7 more
> > > > __________
> > > >
> > > >
> > > > The files kafka-0.7.2.jar and zkclient-0.1.jar are the exact same
> ones
> > > > that are being used in the old 0.7 cluster.
> > > >
> > > > I also tried downloading and building a new 0.7.2 jar file with
> scala,
> > > but
> > > > it had the same error, too.
> > > >
> > > > What can I be doing wrong? Is there any way to get more output to see
> > > > where is the error?
> > > >
> > > > Thanks in advance,
> > > >
> > > > --
> > > > Tomàs Núñez
> > > > Enterprise Infrastructure Consultant
> > > > The Pythian Group - Love your data!
> > > >
> > > > Office (international): +1 613 565 8696 x1501
> > > >
> > >
> > >
> > >
> > > --
> > > Tomàs Núñez
> > > Enterprise Infrastructure Consultant
> > > The Pythian Group - Love your data!
> > >
> > > Office (international): +1 613 565 8696 x1501
> > >
> > > --
> > >
> > >
> > > --
> > >
> > >
> > >
> > >
> >
>
>
>
> --
> Tomàs Núñez
> Enterprise Infrastructure Consultant
> The Pythian Group - Love your data!
>
> Office (international): +1 613 565 8696 x1501
>
> --
>
>
> --
>
>
>
>
Re: Error using migrationtool for upgrading 0.7 to 0.8
Posted by Tomas Nunez <nu...@pythian.com>.
Thanks for pointing the error. I fixed it, but I'm still getting the same
error. The content of migrationToolProducer.properties is now:
_____
metadata.broker.list=kafka_new01:9092,kafka_new02:9092,kafka_new03:9092,kafka_new04:9092,kafka_new05:9092,
group.id=migration.prod
_____
And the error still is:
Exception in thread "main" java.lang.NoClassDefFoundError:
scala/Tuple2$mcLL$sp
at kafka.utils.Utils.stackTrace(Utils.scala)
at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:268)
Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 2 more
I have no "Tuple2*" file anywhere in the server. I'm following
https://cwiki.apache.org/confluence/display/KAFKA/Migrating+from+0.7+to+0.8
and I can't see there anything about downloading classes, and I don't find
much people with the same problem, which leads me to think that I'm doing
something wrong...
Any idea on what to try?
Thanks!
PD: Of course, I'll tell Alex you say hi :)
On Sat, Nov 1, 2014 at 4:32 AM, Gwen Shapira <gs...@cloudera.com> wrote:
> This is part of Scala, so it should be in the scala-library-...jar
>
>
> On Fri, Oct 31, 2014 at 8:26 PM, Tomas Nunez <nu...@pythian.com> wrote:
>
> > Well... I used strace and I found it was looking for some classes in a
> > wrong path. I fixed most of them, but there's one that isn't anywhere,
> > neither the new nor the old kafka servers:
> >
> > $ strace -o logfile -s 1000 -f kafka/bin/kafka-run-class.sh
> > kafka.tools.KafkaMigrationTool --kafka.07.jar
> > migration-tool/kafka-0.7.2.jar --zkclient.01.jar
> > migration-tool/zkclient-0.1.jar
> > --consumer.config=kafka/config/migrationToolConsumer.properties
> > -producer.config=kafka/config/migrationToolProducer.properties
> > --whitelist=.*
> > Exception in thread "main" java.lang.NoClassDefFoundError:
> > scala/Tuple2$mcLL$sp
> > at kafka.utils.Utils.stackTrace(Utils.scala)
> > at
> kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:268)
> > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > ... 2 more
> >
> > $ grep ENOENT logfile|tail -1
> > 16484
> >
> >
> stat("migration-tool/kafka07/core/target/scala_2.8.0/classes/scala/Tuple2$mcLL$sp.class",
> > 0x7f59b27f4950) = -1 ENOENT (No such file or directory)
> >
> > I saw someone had a similar problem in this list, but it's not clear (at
> > least to me) how it was fixed in the end
> >
> >
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201302.mbox/%3CCANEBM33rMTZ2UGp2dpaFj8xV=n4=Rr9GP3Six0sko++nhQv4kQ@mail.gmail.com%3E
> >
> > The JIRA ticket is still untouched
> > https://issues.apache.org/jira/browse/KAFKA-760
> >
> > May different versions of scala be the cause of this error? I just used
> > "./sbt update" to make sure...
> >
> > If not, where should I find that class file?
> >
> > Thanks again!
> >
> > On Sat, Nov 1, 2014 at 4:05 AM, Tomas Nunez <nu...@pythian.com> wrote:
> >
> > > Hi
> > >
> > > I'm trying to upgrade a 0.7 kafka cluster, but I'm getting an error:
> > >
> > > I created the file migrationToolConsumer.properties with just theinfo
> to
> > > connect to the old cluster:
> > > _____
> > > zookeeper.connect=zookeeper01:2181,zookeeper02:2181,zookeeper03:2181
> > > group.id=migration.prod
> > > ______
> > >
> > >
> > > Then the producer file
> > > _______
> > >
> > >
> >
> zookeeper.connect=zookeeper01_new:2181,zookeeper02_new:2181,zookeeper03_new:2181
> > > group.id=migration.prod
> > > _______
> > >
> > > And then I called the migrationtool, and get this error
> > >
> > > ______________
> > > $ kafka/bin/kafka-run-class.sh kafka.tools.KafkaMigrationTool
> > > --kafka.07.jar migration-tool/kafka-0.7.2.jar --zkclient.01.jar
> > > migration-tool/zkclient-0.1.jar --num.producers 4
> > > --consumer.config=kafka/config/migrationToolConsumer.properties
> > > -producer.config=kafka/config/migrationToolProducer.properties
> > > --whitelist=.*
> > > Kafka migration tool failed due to:
> > > java.lang.reflect.InvocationTargetException
> > > at
> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > > Method)
> > > at
> > >
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> > > at
> > >
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > > at
> > java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> > > at
> > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
> > > Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
> > > at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
> > > at
> kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
> > > ... 5 more
> > > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> > > at java.lang.ClassLoader.findClass(ClassLoader.java:531)
> > > at
> > >
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > at
> > >
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > ... 7 more
> > >
> > > [2014-11-01 02:51:35,124] ERROR Kafka migration tool failed:
> > > (kafka.tools.KafkaMigrationTool)
> > > java.lang.reflect.InvocationTargetException
> > > at
> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > > Method)
> > > at
> > >
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> > > at
> > >
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > > at
> > java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> > > at
> > kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
> > > Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
> > > at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
> > > at
> kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
> > > ... 5 more
> > > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> > > at java.lang.ClassLoader.findClass(ClassLoader.java:531)
> > > at
> > >
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > at
> > >
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > ... 7 more
> > > __________
> > >
> > >
> > > The files kafka-0.7.2.jar and zkclient-0.1.jar are the exact same ones
> > > that are being used in the old 0.7 cluster.
> > >
> > > I also tried downloading and building a new 0.7.2 jar file with scala,
> > but
> > > it had the same error, too.
> > >
> > > What can I be doing wrong? Is there any way to get more output to see
> > > where is the error?
> > >
> > > Thanks in advance,
> > >
> > > --
> > > Tomàs Núñez
> > > Enterprise Infrastructure Consultant
> > > The Pythian Group - Love your data!
> > >
> > > Office (international): +1 613 565 8696 x1501
> > >
> >
> >
> >
> > --
> > Tomàs Núñez
> > Enterprise Infrastructure Consultant
> > The Pythian Group - Love your data!
> >
> > Office (international): +1 613 565 8696 x1501
> >
> > --
> >
> >
> > --
> >
> >
> >
> >
>
--
Tomàs Núñez
Enterprise Infrastructure Consultant
The Pythian Group - Love your data!
Office (international): +1 613 565 8696 x1501
--
--
Re: Error using migrationtool for upgrading 0.7 to 0.8
Posted by Gwen Shapira <gs...@cloudera.com>.
This is part of Scala, so it should be in the scala-library-...jar
On Fri, Oct 31, 2014 at 8:26 PM, Tomas Nunez <nu...@pythian.com> wrote:
> Well... I used strace and I found it was looking for some classes in a
> wrong path. I fixed most of them, but there's one that isn't anywhere,
> neither the new nor the old kafka servers:
>
> $ strace -o logfile -s 1000 -f kafka/bin/kafka-run-class.sh
> kafka.tools.KafkaMigrationTool --kafka.07.jar
> migration-tool/kafka-0.7.2.jar --zkclient.01.jar
> migration-tool/zkclient-0.1.jar
> --consumer.config=kafka/config/migrationToolConsumer.properties
> -producer.config=kafka/config/migrationToolProducer.properties
> --whitelist=.*
> Exception in thread "main" java.lang.NoClassDefFoundError:
> scala/Tuple2$mcLL$sp
> at kafka.utils.Utils.stackTrace(Utils.scala)
> at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:268)
> Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ... 2 more
>
> $ grep ENOENT logfile|tail -1
> 16484
>
> stat("migration-tool/kafka07/core/target/scala_2.8.0/classes/scala/Tuple2$mcLL$sp.class",
> 0x7f59b27f4950) = -1 ENOENT (No such file or directory)
>
> I saw someone had a similar problem in this list, but it's not clear (at
> least to me) how it was fixed in the end
>
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201302.mbox/%3CCANEBM33rMTZ2UGp2dpaFj8xV=n4=Rr9GP3Six0sko++nhQv4kQ@mail.gmail.com%3E
>
> The JIRA ticket is still untouched
> https://issues.apache.org/jira/browse/KAFKA-760
>
> May different versions of scala be the cause of this error? I just used
> "./sbt update" to make sure...
>
> If not, where should I find that class file?
>
> Thanks again!
>
> On Sat, Nov 1, 2014 at 4:05 AM, Tomas Nunez <nu...@pythian.com> wrote:
>
> > Hi
> >
> > I'm trying to upgrade a 0.7 kafka cluster, but I'm getting an error:
> >
> > I created the file migrationToolConsumer.properties with just theinfo to
> > connect to the old cluster:
> > _____
> > zookeeper.connect=zookeeper01:2181,zookeeper02:2181,zookeeper03:2181
> > group.id=migration.prod
> > ______
> >
> >
> > Then the producer file
> > _______
> >
> >
> zookeeper.connect=zookeeper01_new:2181,zookeeper02_new:2181,zookeeper03_new:2181
> > group.id=migration.prod
> > _______
> >
> > And then I called the migrationtool, and get this error
> >
> > ______________
> > $ kafka/bin/kafka-run-class.sh kafka.tools.KafkaMigrationTool
> > --kafka.07.jar migration-tool/kafka-0.7.2.jar --zkclient.01.jar
> > migration-tool/zkclient-0.1.jar --num.producers 4
> > --consumer.config=kafka/config/migrationToolConsumer.properties
> > -producer.config=kafka/config/migrationToolProducer.properties
> > --whitelist=.*
> > Kafka migration tool failed due to:
> > java.lang.reflect.InvocationTargetException
> > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > Method)
> > at
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> > at
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > at
> java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> > at
> kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
> > Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
> > at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
> > at kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
> > ... 5 more
> > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> > at java.lang.ClassLoader.findClass(ClassLoader.java:531)
> > at
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > at
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > ... 7 more
> >
> > [2014-11-01 02:51:35,124] ERROR Kafka migration tool failed:
> > (kafka.tools.KafkaMigrationTool)
> > java.lang.reflect.InvocationTargetException
> > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > Method)
> > at
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> > at
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > at
> java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> > at
> kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
> > Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
> > at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
> > at kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
> > ... 5 more
> > Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> > at java.lang.ClassLoader.findClass(ClassLoader.java:531)
> > at
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > at
> >
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > ... 7 more
> > __________
> >
> >
> > The files kafka-0.7.2.jar and zkclient-0.1.jar are the exact same ones
> > that are being used in the old 0.7 cluster.
> >
> > I also tried downloading and building a new 0.7.2 jar file with scala,
> but
> > it had the same error, too.
> >
> > What can I be doing wrong? Is there any way to get more output to see
> > where is the error?
> >
> > Thanks in advance,
> >
> > --
> > Tomàs Núñez
> > Enterprise Infrastructure Consultant
> > The Pythian Group - Love your data!
> >
> > Office (international): +1 613 565 8696 x1501
> >
>
>
>
> --
> Tomàs Núñez
> Enterprise Infrastructure Consultant
> The Pythian Group - Love your data!
>
> Office (international): +1 613 565 8696 x1501
>
> --
>
>
> --
>
>
>
>
Re: Error using migrationtool for upgrading 0.7 to 0.8
Posted by Tomas Nunez <nu...@pythian.com>.
Well... I used strace and I found it was looking for some classes in a
wrong path. I fixed most of them, but there's one that isn't anywhere,
neither the new nor the old kafka servers:
$ strace -o logfile -s 1000 -f kafka/bin/kafka-run-class.sh
kafka.tools.KafkaMigrationTool --kafka.07.jar
migration-tool/kafka-0.7.2.jar --zkclient.01.jar
migration-tool/zkclient-0.1.jar
--consumer.config=kafka/config/migrationToolConsumer.properties
-producer.config=kafka/config/migrationToolProducer.properties
--whitelist=.*
Exception in thread "main" java.lang.NoClassDefFoundError:
scala/Tuple2$mcLL$sp
at kafka.utils.Utils.stackTrace(Utils.scala)
at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:268)
Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 2 more
$ grep ENOENT logfile|tail -1
16484
stat("migration-tool/kafka07/core/target/scala_2.8.0/classes/scala/Tuple2$mcLL$sp.class",
0x7f59b27f4950) = -1 ENOENT (No such file or directory)
I saw someone had a similar problem in this list, but it's not clear (at
least to me) how it was fixed in the end
http://mail-archives.apache.org/mod_mbox/kafka-dev/201302.mbox/%3CCANEBM33rMTZ2UGp2dpaFj8xV=n4=Rr9GP3Six0sko++nhQv4kQ@mail.gmail.com%3E
The JIRA ticket is still untouched
https://issues.apache.org/jira/browse/KAFKA-760
May different versions of scala be the cause of this error? I just used
"./sbt update" to make sure...
If not, where should I find that class file?
Thanks again!
On Sat, Nov 1, 2014 at 4:05 AM, Tomas Nunez <nu...@pythian.com> wrote:
> Hi
>
> I'm trying to upgrade a 0.7 kafka cluster, but I'm getting an error:
>
> I created the file migrationToolConsumer.properties with just theinfo to
> connect to the old cluster:
> _____
> zookeeper.connect=zookeeper01:2181,zookeeper02:2181,zookeeper03:2181
> group.id=migration.prod
> ______
>
>
> Then the producer file
> _______
>
> zookeeper.connect=zookeeper01_new:2181,zookeeper02_new:2181,zookeeper03_new:2181
> group.id=migration.prod
> _______
>
> And then I called the migrationtool, and get this error
>
> ______________
> $ kafka/bin/kafka-run-class.sh kafka.tools.KafkaMigrationTool
> --kafka.07.jar migration-tool/kafka-0.7.2.jar --zkclient.01.jar
> migration-tool/zkclient-0.1.jar --num.producers 4
> --consumer.config=kafka/config/migrationToolConsumer.properties
> -producer.config=kafka/config/migrationToolProducer.properties
> --whitelist=.*
> Kafka migration tool failed due to:
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
> Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
> at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
> at kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
> ... 5 more
> Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> at java.lang.ClassLoader.findClass(ClassLoader.java:531)
> at
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ... 7 more
>
> [2014-11-01 02:51:35,124] ERROR Kafka migration tool failed:
> (kafka.tools.KafkaMigrationTool)
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
> Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
> at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
> at kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
> ... 5 more
> Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> at java.lang.ClassLoader.findClass(ClassLoader.java:531)
> at
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ... 7 more
> __________
>
>
> The files kafka-0.7.2.jar and zkclient-0.1.jar are the exact same ones
> that are being used in the old 0.7 cluster.
>
> I also tried downloading and building a new 0.7.2 jar file with scala, but
> it had the same error, too.
>
> What can I be doing wrong? Is there any way to get more output to see
> where is the error?
>
> Thanks in advance,
>
> --
> Tomàs Núñez
> Enterprise Infrastructure Consultant
> The Pythian Group - Love your data!
>
> Office (international): +1 613 565 8696 x1501
>
--
Tomàs Núñez
Enterprise Infrastructure Consultant
The Pythian Group - Love your data!
Office (international): +1 613 565 8696 x1501
--
--
Re: Error using migrationtool for upgrading 0.7 to 0.8
Posted by Gwen Shapira <gs...@cloudera.com>.
The producer configuration should list the kafka brokers, not the zookeeper
quorum.
See here: http://kafka.apache.org/documentation.html#producerconfigs
(and send my regards to Alex Gorbachev ;)
Gwen
On Fri, Oct 31, 2014 at 8:05 PM, Tomas Nunez <nu...@pythian.com> wrote:
> Hi
>
> I'm trying to upgrade a 0.7 kafka cluster, but I'm getting an error:
>
> I created the file migrationToolConsumer.properties with just theinfo to
> connect to the old cluster:
> _____
> zookeeper.connect=zookeeper01:2181,zookeeper02:2181,zookeeper03:2181
> group.id=migration.prod
> ______
>
>
> Then the producer file
> _______
>
> zookeeper.connect=zookeeper01_new:2181,zookeeper02_new:2181,zookeeper03_new:2181
> group.id=migration.prod
> _______
>
> And then I called the migrationtool, and get this error
>
> ______________
> $ kafka/bin/kafka-run-class.sh kafka.tools.KafkaMigrationTool
> --kafka.07.jar migration-tool/kafka-0.7.2.jar --zkclient.01.jar
> migration-tool/zkclient-0.1.jar --num.producers 4
> --consumer.config=kafka/config/migrationToolConsumer.properties
> -producer.config=kafka/config/migrationToolProducer.properties
> --whitelist=.*
> Kafka migration tool failed due to:
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
>
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
> Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
> at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
> at kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
> ... 5 more
> Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> at java.lang.ClassLoader.findClass(ClassLoader.java:531)
> at
>
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
>
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ... 7 more
>
> [2014-11-01 02:51:35,124] ERROR Kafka migration tool failed:
> (kafka.tools.KafkaMigrationTool)
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
>
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:200)
> Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcLL$sp
> at kafka.utils.ZKConfig.<init>(ZkUtils.scala:302)
> at kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:44)
> ... 5 more
> Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcLL$sp
> at java.lang.ClassLoader.findClass(ClassLoader.java:531)
> at
>
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$FindClassClassLoader.findClass(KafkaMigrationTool.java:440)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
>
> kafka.tools.KafkaMigrationTool$ParentLastURLClassLoader$ChildURLClassLoader.findClass(KafkaMigrationTool.java:463)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ... 7 more
> __________
>
>
> The files kafka-0.7.2.jar and zkclient-0.1.jar are the exact same ones
> that are being used in the old 0.7 cluster.
>
> I also tried downloading and building a new 0.7.2 jar file with scala, but
> it had the same error, too.
>
> What can I be doing wrong? Is there any way to get more output to see where
> is the error?
>
> Thanks in advance,
>
> --
> Tomàs Núñez
> Enterprise Infrastructure Consultant
> The Pythian Group - Love your data!
>
> Office (international): +1 613 565 8696 x1501
>
> --
>
>
> --
>
>
>
>