You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by David Montgomery <da...@gmail.com> on 2014/03/03 12:37:20 UTC

Re: How ot had scala libraries to path

I have reverted back to 7.  I can produce and consume using the python lib
brod.






Druid consumer gives this error.

My libs are here.  So..kafka cant find scala?

/var/lib/kafka-0.7.2-incubating-src/core/lib_managed/scala_2.8.0/test/scalatest-1.2.jar
/var/lib/kafka-0.7.2-incubating-src/project/boot/scala-2.7.7/lib/scala-compiler.jar
/var/lib/kafka-0.7.2-incubating-src/project/boot/scala-2.7.7/lib/scala-library.jar
/var/lib/kafka-0.7.2-incubating-src/project/boot/scala-2.8.0/lib/scala-compiler.jar
/var/lib/kafka-0.7.2-incubating-src/project/boot/scala-2.8.0/lib/scala-library.jar


Here is how I install:]'cd kafka-0.7.2-incubating-src
./sbt update
./sbt package
echo 'KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote=true
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false"' | tee -a
/var/lib/kafka-0.7.2-incubating-src/bin/kafka-run-class.sh
echo 'export JMX_PORT=${JMX_PORT:-9999}' | tee -a
/var/lib/kafka-0.7.2-incubating-src/bin/kafka-server-start.sh

Here is how I start
/var/lib/kafka-0.7.2-incubating-src/bin/kafka-server-start.sh
/var/lib/kafka-0.7.2-incubating-src/config/server.properties



ing SASL (unknown error)
2014-03-03 11:19:40,237 INFO [main]
com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking
start method[public void io.druid.segment.realtime.RealtimeManager.start()
throws java.io.IOException] on
object[io.druid.segment.realtime.RealtimeManager@73ed27ed].
2014-03-03 11:19:40,241 INFO [main]
io.druid.segment.realtime.RealtimeManager - Calling the FireDepartment and
getting a Firehose.
2014-03-03 11:19:40,249 ERROR [main] io.druid.cli.CliBroker - Error when
starting up.  Failing.
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at
com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler.start(Lifecycle.java:331)
    at com.metamx.common.lifecycle.Lifecycle.start(Lifecycle.java:250)
    at io.druid.guice.LifecycleModule$2.start(LifecycleModule.java:136)
    at io.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:72)
    at io.druid.cli.ServerRunnable.run(ServerRunnable.java:40)
    at io.druid.cli.Main.main(Main.java:91)
Caused by: java.lang.NoClassDefFoundError: scala/ScalaObject
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
    at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at
io.druid.firehose.kafka.KafkaSevenFirehoseFactory.connect(KafkaSevenFirehoseFactory.java:89)
    at
io.druid.segment.realtime.FireDepartment.connect(FireDepartment.java:90)
    at
io.druid.segment.realtime.RealtimeManager$FireChief.init(RealtimeManager.java:150)
    at
io.druid.segment.realtime.RealtimeManager.start(RealtimeManager.java:87)
    ... 10 more
Caused by: java.lang.ClassNotFoundException: scala.ScalaObject
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    ... 25 more



On Wed, Feb 26, 2014 at 12:40 PM, David Montgomery <
davidmontgomery@gmail.com> wrote:

> Hi,
>
> This is how I start kafka.
>
> command = /var/lib/kafka-<%=@version%>-src/bin/kafka-server-start.sh
> /var/lib/kafka-<%=@version%>-src/config/server.properties
>
> In another application I get teh below error.  The suggestion is to add
> the scalar libraries to the path.  How do I do that?
>
>
> thanks
>
>
> java.lang.reflect.
> InvocationTargetException
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:606)
>     at
> com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler.start(Lifecycle.java:331)
>     at com.metamx.common.lifecycle.Lifecycle.start(Lifecycle.java:250)
>     at io.druid.guice.LifecycleModule$2.start(LifecycleModule.java:136)
>     at io.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:72)
>     at io.druid.cli.ServerRunnable.run(ServerRunnable.java:40)
>     at io.druid.cli.Main.main(Main.java:91)
> Caused by: java.lang.NoClassDefFoundError: scala/ScalaObject
>     at java.lang.ClassLoader.defineClass1(Native Method)
>     at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>     at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>     at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>     at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>     at
> io.druid.firehose.kafka.KafkaEightFirehoseFactory.connect(KafkaEightFirehoseFactory.java:76)
>     at
> io.druid.segment.realtime.FireDepartment.connect(FireDepartment.java:90)
>     at
> io.druid.segment.realtime.RealtimeManager$FireChief.init(RealtimeManager.java:150)
>     at
> io.druid.segment.realtime.RealtimeManager.start(RealtimeManager.java:87)
>     ... 10 more
> Caused by: java.lang.ClassNotFoundException: scala.ScalaObject
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>     ... 25 more
>

Re: How ot had scala libraries to path

Posted by Jun Rao <ju...@gmail.com>.
Hmm, not sure how duid picks up the jars. Perhaps you can ask duid mailing
list? Do you know if duid is on Kafka 0.7 or 0.8?

Thanks,

Jun


On Mon, Mar 3, 2014 at 3:37 AM, David Montgomery
<da...@gmail.com>wrote:

> I have reverted back to 7.  I can produce and consume using the python lib
> brod.
>
>
>
>
>
>
> Druid consumer gives this error.
>
> My libs are here.  So..kafka cant find scala?
>
>
> /var/lib/kafka-0.7.2-incubating-src/core/lib_managed/scala_2.8.0/test/scalatest-1.2.jar
>
> /var/lib/kafka-0.7.2-incubating-src/project/boot/scala-2.7.7/lib/scala-compiler.jar
>
> /var/lib/kafka-0.7.2-incubating-src/project/boot/scala-2.7.7/lib/scala-library.jar
>
> /var/lib/kafka-0.7.2-incubating-src/project/boot/scala-2.8.0/lib/scala-compiler.jar
>
> /var/lib/kafka-0.7.2-incubating-src/project/boot/scala-2.8.0/lib/scala-library.jar
>
>
> Here is how I install:]'cd kafka-0.7.2-incubating-src
> ./sbt update
> ./sbt package
> echo 'KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote=true
> -Dcom.sun.management.jmxremote.authenticate=false
> -Dcom.sun.management.jmxremote.ssl=false"' | tee -a
> /var/lib/kafka-0.7.2-incubating-src/bin/kafka-run-class.sh
> echo 'export JMX_PORT=${JMX_PORT:-9999}' | tee -a
> /var/lib/kafka-0.7.2-incubating-src/bin/kafka-server-start.sh
>
> Here is how I start
> /var/lib/kafka-0.7.2-incubating-src/bin/kafka-server-start.sh
> /var/lib/kafka-0.7.2-incubating-src/config/server.properties
>
>
>
> ing SASL (unknown error)
> 2014-03-03 11:19:40,237 INFO [main]
> com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking
> start method[public void io.druid.segment.realtime.RealtimeManager.start()
> throws java.io.IOException] on
> object[io.druid.segment.realtime.RealtimeManager@73ed27ed].
> 2014-03-03 11:19:40,241 INFO [main]
> io.druid.segment.realtime.RealtimeManager - Calling the FireDepartment and
> getting a Firehose.
> 2014-03-03 11:19:40,249 ERROR [main] io.druid.cli.CliBroker - Error when
> starting up.  Failing.
> java.lang.reflect.InvocationTargetException
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:606)
>     at
>
> com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler.start(Lifecycle.java:331)
>     at com.metamx.common.lifecycle.Lifecycle.start(Lifecycle.java:250)
>     at io.druid.guice.LifecycleModule$2.start(LifecycleModule.java:136)
>     at io.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:72)
>     at io.druid.cli.ServerRunnable.run(ServerRunnable.java:40)
>     at io.druid.cli.Main.main(Main.java:91)
> Caused by: java.lang.NoClassDefFoundError: scala/ScalaObject
>     at java.lang.ClassLoader.defineClass1(Native Method)
>     at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>     at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>     at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>     at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>     at
>
> io.druid.firehose.kafka.KafkaSevenFirehoseFactory.connect(KafkaSevenFirehoseFactory.java:89)
>     at
> io.druid.segment.realtime.FireDepartment.connect(FireDepartment.java:90)
>     at
>
> io.druid.segment.realtime.RealtimeManager$FireChief.init(RealtimeManager.java:150)
>     at
> io.druid.segment.realtime.RealtimeManager.start(RealtimeManager.java:87)
>     ... 10 more
> Caused by: java.lang.ClassNotFoundException: scala.ScalaObject
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>     ... 25 more
>
>
>
> On Wed, Feb 26, 2014 at 12:40 PM, David Montgomery <
> davidmontgomery@gmail.com> wrote:
>
> > Hi,
> >
> > This is how I start kafka.
> >
> > command = /var/lib/kafka-<%=@version%>-src/bin/kafka-server-start.sh
> > /var/lib/kafka-<%=@version%>-src/config/server.properties
> >
> > In another application I get teh below error.  The suggestion is to add
> > the scalar libraries to the path.  How do I do that?
> >
> >
> > thanks
> >
> >
> > java.lang.reflect.
> > InvocationTargetException
> >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >     at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >     at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >     at java.lang.reflect.Method.invoke(Method.java:606)
> >     at
> >
> com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler.start(Lifecycle.java:331)
> >     at com.metamx.common.lifecycle.Lifecycle.start(Lifecycle.java:250)
> >     at io.druid.guice.LifecycleModule$2.start(LifecycleModule.java:136)
> >     at io.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:72)
> >     at io.druid.cli.ServerRunnable.run(ServerRunnable.java:40)
> >     at io.druid.cli.Main.main(Main.java:91)
> > Caused by: java.lang.NoClassDefFoundError: scala/ScalaObject
> >     at java.lang.ClassLoader.defineClass1(Native Method)
> >     at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> >     at
> > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> >     at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> >     at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> >     at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> >     at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> >     at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> >     at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> >     at
> >
> io.druid.firehose.kafka.KafkaEightFirehoseFactory.connect(KafkaEightFirehoseFactory.java:76)
> >     at
> > io.druid.segment.realtime.FireDepartment.connect(FireDepartment.java:90)
> >     at
> >
> io.druid.segment.realtime.RealtimeManager$FireChief.init(RealtimeManager.java:150)
> >     at
> > io.druid.segment.realtime.RealtimeManager.start(RealtimeManager.java:87)
> >     ... 10 more
> > Caused by: java.lang.ClassNotFoundException: scala.ScalaObject
> >     at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> >     at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> >     at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> >     at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> >     ... 25 more
> >
>