You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Alex Melville <am...@g.hmc.edu> on 2014/12/14 05:11:00 UTC
How To Specify partioner.class for Mirrormaker?
Dear Kafka Devs,
Re: How To Specify partioner.class for Mirrormaker?
Posted by Madhukar Bharti <bh...@gmail.com>.
Dear Alex Melville,
You can put the you partitioner class in an jar file inside kafka/libs.
Now if you will start Mirror maker it will load the jar and you will not
get any exception.
On Sun, Dec 14, 2014 at 9:41 AM, Alex Melville <am...@g.hmc.edu> wrote:
>
> Sorry, the email sent before I typed my question. Please read below:
>
>
> I wrote my own Java Consumer and Producer based off of the Kafka Producer
> API and High Level Consumer. Let's call them MyConsumer and MyProducer.
> MyProducer uses a custom Partitioner class called SimplePartitioner. In the
> producer.config file that I specify when I run the MirrorMaker from the
> command line, there is a parameter "partitioner.class". I keep getting
> "ClassDefNotFoundException exceptions, no matter if I put the absolute path
> to my SimplePartitioner.class file, a relative path, or even when I add
> SimplePartitioner.class to the $CLASSPATH variables created in the
> kafka-run-class.sh script. Here is my output error:
>
> Exception in thread "main" java.lang.ClassNotFoundException:
> SimplePartitioner.class
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:191)
> at kafka.utils.Utils$.createObject(Utils.scala:438)
> at kafka.producer.Producer.<init>(Producer.scala:60)
> at kafka.tools.MirrorMaker$$anonfun$1.apply(MirrorMaker.scala:116)
> at kafka.tools.MirrorMaker$$anonfun$1.apply(MirrorMaker.scala:106)
> at
>
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233)
> at
>
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233)
> at scala.collection.immutable.Range.foreach(Range.scala:81)
> at scala.collection.TraversableLike$class.map(TraversableLike.scala:233)
> at scala.collection.immutable.Range.map(Range.scala:46)
> at kafka.tools.MirrorMaker$.main(MirrorMaker.scala:106)
> at kafka.tools.MirrorMaker.main(MirrorMaker.scala)
>
> What is the correct value for the "partitioner.class" parameter in my
> producer.properties config file?
>
> On Sat, Dec 13, 2014 at 8:11 PM, Alex Melville <am...@g.hmc.edu>
> wrote:
> >
> > Dear Kafka Devs,
> >
> >
> >
>
--
Regards,
Madhukar
Re: How To Specify partioner.class for Mirrormaker?
Posted by Alex Melville <am...@g.hmc.edu>.
Sorry, the email sent before I typed my question. Please read below:
I wrote my own Java Consumer and Producer based off of the Kafka Producer
API and High Level Consumer. Let's call them MyConsumer and MyProducer.
MyProducer uses a custom Partitioner class called SimplePartitioner. In the
producer.config file that I specify when I run the MirrorMaker from the
command line, there is a parameter "partitioner.class". I keep getting
"ClassDefNotFoundException exceptions, no matter if I put the absolute path
to my SimplePartitioner.class file, a relative path, or even when I add
SimplePartitioner.class to the $CLASSPATH variables created in the
kafka-run-class.sh script. Here is my output error:
Exception in thread "main" java.lang.ClassNotFoundException:
SimplePartitioner.class
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:191)
at kafka.utils.Utils$.createObject(Utils.scala:438)
at kafka.producer.Producer.<init>(Producer.scala:60)
at kafka.tools.MirrorMaker$$anonfun$1.apply(MirrorMaker.scala:116)
at kafka.tools.MirrorMaker$$anonfun$1.apply(MirrorMaker.scala:106)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233)
at scala.collection.immutable.Range.foreach(Range.scala:81)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:233)
at scala.collection.immutable.Range.map(Range.scala:46)
at kafka.tools.MirrorMaker$.main(MirrorMaker.scala:106)
at kafka.tools.MirrorMaker.main(MirrorMaker.scala)
What is the correct value for the "partitioner.class" parameter in my
producer.properties config file?
On Sat, Dec 13, 2014 at 8:11 PM, Alex Melville <am...@g.hmc.edu> wrote:
>
> Dear Kafka Devs,
>
>
>