You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Joe Stein <cr...@gmail.com> on 2011/11/06 18:46:58 UTC
patched performance testing issues
Hi, I applied kafka-176.patch and got an error while trying to run it
Joseph-Steins-MacBook-Air:kafka joestein$ bin/kafka-producer-perf-test.sh
Exception in thread "main" java.lang.NoClassDefFoundError:
kafka/tools/ProducerPerformance
Caused by: java.lang.ClassNotFoundException: kafka.tools.ProducerPerformance
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
I then grabbed kafka-175 and got an error while applying the patch
Joseph-Steins-MacBook-Air:kafka joestein$ patch -p0 -i ~/kafka-175.patch
patching file perf/config/log4j.properties
patching file perf/bin/consumer-throughput-over-fetch-size.sh
patch: **** malformed patch at line 82:
Should I be patching in some specific order or different commands or
something else I am doing wrong? Let me know please
/*
Joe Stein
http://www.linkedin.com/in/charmalloc
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
*/
Re: patched performance testing issues
Posted by Neha Narkhede <ne...@gmail.com>.
Joe,
I'm sorry for the delay here. I was at ApacheCon last week. I'm working on
these perf testing issues today and will have an update by tomorrow.
Thanks,
Neha
On Fri, Nov 11, 2011 at 7:57 AM, Joe Stein <cr...@gmail.com> wrote:
> Hi, just wanted to ping again on this I still can't run the performance
> test to compare gzip with snappy.
>
> I could just be doing something wrong but unclear. Also was not sure if
> this was still holding up +1 on KAFKA-187 and it getting committed to
> trunk, no rush I can patch the trunk to use it (planning on pushing a
> production cluster next week in either case).
>
> Thanks folks, I know this was a big conference week ontop of all of our day
> jobs, let me know however I can assist.
>
> On Mon, Nov 7, 2011 at 9:32 AM, Joe Stein <cr...@gmail.com> wrote:
>
> > Thanks, the updated patch applied this time, new issues now.
> >
> > fyi, I had to chmod a+x the sh in perf/bin but after I did that still got
> > an error
> >
> > new-host-3:kafka joestein$ perf/bin/kafka-producer-perf-test.sh
> > Exception in thread "main" java.lang.NoClassDefFoundError:
> > kafka/perf/ProducerPerformance
> > Caused by: java.lang.ClassNotFoundException:
> kafka.perf.ProducerPerformance
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> >
> > let me know if I should be running this some other way or how please or
> if
> > still missing a patch (I applied 176 then 175-updated both against trunk
> > with my 187-refactored), thanks!
> >
> > On Sun, Nov 6, 2011 at 1:46 PM, Neha Narkhede <neha.narkhede@gmail.com
> >wrote:
> >
> >> Joe,
> >>
> >> Its a pain that I had to separate the perf patches into separate JIRAs
> >> and have not still checked it in. Sorry for the trouble!
> >>
> >> >> Caused by: java.lang.ClassNotFoundException:
> >> kafka.tools.ProducerPerformance
> >>
> >> Here, the bin/kafka-producer-perf-test.sh is still referring to the
> >> old location of ProducerPerformance. By applying kafka-175's patch,
> >> this will get resolved.
> >>
> >> >> > Joseph-Steins-MacBook-Air:kafka joestein$ patch -p0 -i
> >> ~/kafka-175.patch
> >> > patching file perf/config/log4j.properties
> >> > patching file perf/bin/consumer-throughput-over-fetch-size.sh
> >> > patch: **** malformed patch at line 82:
> >>
> >> That patch seems to have some malformed lines. I've uploaded an
> >> updated patch -
> >>
> >>
> https://issues.apache.org/jira/secure/attachment/12502678/kafka-175-updated.patch
> >>
> >> Thanks,
> >> Neha
> >>
> >>
> >> On Sun, Nov 6, 2011 at 9:46 AM, Joe Stein <cr...@gmail.com> wrote:
> >> > Hi, I applied kafka-176.patch and got an error while trying to run it
> >> >
> >> > Joseph-Steins-MacBook-Air:kafka joestein$
> >> bin/kafka-producer-perf-test.sh
> >> > Exception in thread "main" java.lang.NoClassDefFoundError:
> >> > kafka/tools/ProducerPerformance
> >> > Caused by: java.lang.ClassNotFoundException:
> >> kafka.tools.ProducerPerformance
> >> > at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> >> > at java.security.AccessController.doPrivileged(Native Method)
> >> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> >> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> >> >
> >> > I then grabbed kafka-175 and got an error while applying the patch
> >> >
> >> > Joseph-Steins-MacBook-Air:kafka joestein$ patch -p0 -i
> ~/kafka-175.patch
> >> > patching file perf/config/log4j.properties
> >> > patching file perf/bin/consumer-throughput-over-fetch-size.sh
> >> > patch: **** malformed patch at line 82:
> >> >
> >> > Should I be patching in some specific order or different commands or
> >> > something else I am doing wrong? Let me know please
> >> >
> >> >
> >> > /*
> >> > Joe Stein
> >> > http://www.linkedin.com/in/charmalloc
> >> > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> >> > */
> >> >
> >>
> >
> >
> >
> > --
> >
> > /*
> > Joe Stein
> > http://www.linkedin.com/in/charmalloc
> > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> > */
> >
>
>
>
> --
>
> /*
> Joe Stein
> http://www.linkedin.com/in/charmalloc
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> */
>
Re: patched performance testing issues
Posted by Joe Stein <cr...@gmail.com>.
Hi, just wanted to ping again on this I still can't run the performance
test to compare gzip with snappy.
I could just be doing something wrong but unclear. Also was not sure if
this was still holding up +1 on KAFKA-187 and it getting committed to
trunk, no rush I can patch the trunk to use it (planning on pushing a
production cluster next week in either case).
Thanks folks, I know this was a big conference week ontop of all of our day
jobs, let me know however I can assist.
On Mon, Nov 7, 2011 at 9:32 AM, Joe Stein <cr...@gmail.com> wrote:
> Thanks, the updated patch applied this time, new issues now.
>
> fyi, I had to chmod a+x the sh in perf/bin but after I did that still got
> an error
>
> new-host-3:kafka joestein$ perf/bin/kafka-producer-perf-test.sh
> Exception in thread "main" java.lang.NoClassDefFoundError:
> kafka/perf/ProducerPerformance
> Caused by: java.lang.ClassNotFoundException: kafka.perf.ProducerPerformance
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>
> let me know if I should be running this some other way or how please or if
> still missing a patch (I applied 176 then 175-updated both against trunk
> with my 187-refactored), thanks!
>
> On Sun, Nov 6, 2011 at 1:46 PM, Neha Narkhede <ne...@gmail.com>wrote:
>
>> Joe,
>>
>> Its a pain that I had to separate the perf patches into separate JIRAs
>> and have not still checked it in. Sorry for the trouble!
>>
>> >> Caused by: java.lang.ClassNotFoundException:
>> kafka.tools.ProducerPerformance
>>
>> Here, the bin/kafka-producer-perf-test.sh is still referring to the
>> old location of ProducerPerformance. By applying kafka-175's patch,
>> this will get resolved.
>>
>> >> > Joseph-Steins-MacBook-Air:kafka joestein$ patch -p0 -i
>> ~/kafka-175.patch
>> > patching file perf/config/log4j.properties
>> > patching file perf/bin/consumer-throughput-over-fetch-size.sh
>> > patch: **** malformed patch at line 82:
>>
>> That patch seems to have some malformed lines. I've uploaded an
>> updated patch -
>>
>> https://issues.apache.org/jira/secure/attachment/12502678/kafka-175-updated.patch
>>
>> Thanks,
>> Neha
>>
>>
>> On Sun, Nov 6, 2011 at 9:46 AM, Joe Stein <cr...@gmail.com> wrote:
>> > Hi, I applied kafka-176.patch and got an error while trying to run it
>> >
>> > Joseph-Steins-MacBook-Air:kafka joestein$
>> bin/kafka-producer-perf-test.sh
>> > Exception in thread "main" java.lang.NoClassDefFoundError:
>> > kafka/tools/ProducerPerformance
>> > Caused by: java.lang.ClassNotFoundException:
>> kafka.tools.ProducerPerformance
>> > at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>> > at java.security.AccessController.doPrivileged(Native Method)
>> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>> >
>> > I then grabbed kafka-175 and got an error while applying the patch
>> >
>> > Joseph-Steins-MacBook-Air:kafka joestein$ patch -p0 -i ~/kafka-175.patch
>> > patching file perf/config/log4j.properties
>> > patching file perf/bin/consumer-throughput-over-fetch-size.sh
>> > patch: **** malformed patch at line 82:
>> >
>> > Should I be patching in some specific order or different commands or
>> > something else I am doing wrong? Let me know please
>> >
>> >
>> > /*
>> > Joe Stein
>> > http://www.linkedin.com/in/charmalloc
>> > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>> > */
>> >
>>
>
>
>
> --
>
> /*
> Joe Stein
> http://www.linkedin.com/in/charmalloc
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> */
>
--
/*
Joe Stein
http://www.linkedin.com/in/charmalloc
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
*/
Re: patched performance testing issues
Posted by Joe Stein <cr...@gmail.com>.
Thanks, the updated patch applied this time, new issues now.
fyi, I had to chmod a+x the sh in perf/bin but after I did that still got
an error
new-host-3:kafka joestein$ perf/bin/kafka-producer-perf-test.sh
Exception in thread "main" java.lang.NoClassDefFoundError:
kafka/perf/ProducerPerformance
Caused by: java.lang.ClassNotFoundException: kafka.perf.ProducerPerformance
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
let me know if I should be running this some other way or how please or if
still missing a patch (I applied 176 then 175-updated both against trunk
with my 187-refactored), thanks!
On Sun, Nov 6, 2011 at 1:46 PM, Neha Narkhede <ne...@gmail.com>wrote:
> Joe,
>
> Its a pain that I had to separate the perf patches into separate JIRAs
> and have not still checked it in. Sorry for the trouble!
>
> >> Caused by: java.lang.ClassNotFoundException:
> kafka.tools.ProducerPerformance
>
> Here, the bin/kafka-producer-perf-test.sh is still referring to the
> old location of ProducerPerformance. By applying kafka-175's patch,
> this will get resolved.
>
> >> > Joseph-Steins-MacBook-Air:kafka joestein$ patch -p0 -i
> ~/kafka-175.patch
> > patching file perf/config/log4j.properties
> > patching file perf/bin/consumer-throughput-over-fetch-size.sh
> > patch: **** malformed patch at line 82:
>
> That patch seems to have some malformed lines. I've uploaded an
> updated patch -
>
> https://issues.apache.org/jira/secure/attachment/12502678/kafka-175-updated.patch
>
> Thanks,
> Neha
>
>
> On Sun, Nov 6, 2011 at 9:46 AM, Joe Stein <cr...@gmail.com> wrote:
> > Hi, I applied kafka-176.patch and got an error while trying to run it
> >
> > Joseph-Steins-MacBook-Air:kafka joestein$ bin/kafka-producer-perf-test.sh
> > Exception in thread "main" java.lang.NoClassDefFoundError:
> > kafka/tools/ProducerPerformance
> > Caused by: java.lang.ClassNotFoundException:
> kafka.tools.ProducerPerformance
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> >
> > I then grabbed kafka-175 and got an error while applying the patch
> >
> > Joseph-Steins-MacBook-Air:kafka joestein$ patch -p0 -i ~/kafka-175.patch
> > patching file perf/config/log4j.properties
> > patching file perf/bin/consumer-throughput-over-fetch-size.sh
> > patch: **** malformed patch at line 82:
> >
> > Should I be patching in some specific order or different commands or
> > something else I am doing wrong? Let me know please
> >
> >
> > /*
> > Joe Stein
> > http://www.linkedin.com/in/charmalloc
> > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> > */
> >
>
--
/*
Joe Stein
http://www.linkedin.com/in/charmalloc
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
*/
Re: patched performance testing issues
Posted by Neha Narkhede <ne...@gmail.com>.
Joe,
Its a pain that I had to separate the perf patches into separate JIRAs
and have not still checked it in. Sorry for the trouble!
>> Caused by: java.lang.ClassNotFoundException: kafka.tools.ProducerPerformance
Here, the bin/kafka-producer-perf-test.sh is still referring to the
old location of ProducerPerformance. By applying kafka-175's patch,
this will get resolved.
>> > Joseph-Steins-MacBook-Air:kafka joestein$ patch -p0 -i ~/kafka-175.patch
> patching file perf/config/log4j.properties
> patching file perf/bin/consumer-throughput-over-fetch-size.sh
> patch: **** malformed patch at line 82:
That patch seems to have some malformed lines. I've uploaded an
updated patch -
https://issues.apache.org/jira/secure/attachment/12502678/kafka-175-updated.patch
Thanks,
Neha
On Sun, Nov 6, 2011 at 9:46 AM, Joe Stein <cr...@gmail.com> wrote:
> Hi, I applied kafka-176.patch and got an error while trying to run it
>
> Joseph-Steins-MacBook-Air:kafka joestein$ bin/kafka-producer-perf-test.sh
> Exception in thread "main" java.lang.NoClassDefFoundError:
> kafka/tools/ProducerPerformance
> Caused by: java.lang.ClassNotFoundException: kafka.tools.ProducerPerformance
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>
> I then grabbed kafka-175 and got an error while applying the patch
>
> Joseph-Steins-MacBook-Air:kafka joestein$ patch -p0 -i ~/kafka-175.patch
> patching file perf/config/log4j.properties
> patching file perf/bin/consumer-throughput-over-fetch-size.sh
> patch: **** malformed patch at line 82:
>
> Should I be patching in some specific order or different commands or
> something else I am doing wrong? Let me know please
>
>
> /*
> Joe Stein
> http://www.linkedin.com/in/charmalloc
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> */
>