You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Xinyi Su <xi...@gmail.com> on 2015/02/05 06:06:33 UTC
Kafka producer perf script throw java.io.IOException
Hi,
Today I updated Kafka cluster from 0.8.2-beta to 0.8.2.0 and run kafka
producer performance test.
The test cannot continue because of some exceptions thrown which does not
occur at 0.8.2-beta. My perf library is kafka-perf_2.9.2-0.8.0.jar which is
the latest version on maven repository.
-bash-4.1$ bin/kafka-producer-perf-test.sh --broker-list <broker list>
--topics PerfTopic22 --sync --initial-message-id 1 --messages 200000
--csv-reporter-enabled --metrics-dir /tmp/PerfTopic22_1
--message-send-gap-ms 20 --request-num-acks -1 --batch-size 1
java.io.IOException: Unable to create
/tmp/PerfTopic22_1/ProducerRequestSize.csv
at
com.yammer.metrics.reporting.CsvReporter.createStreamForMetric(CsvReporter.java:141)
at
com.yammer.metrics.reporting.CsvReporter.getPrintStream(CsvReporter.java:257)
at com.yammer.metrics.reporting.CsvReporter.access$000(CsvReporter.java:22)
at
com.yammer.metrics.reporting.CsvReporter$1.getStream(CsvReporter.java:156)
at
com.yammer.metrics.reporting.CsvReporter.processHistogram(CsvReporter.java:194)
at
com.yammer.metrics.reporting.CsvReporter.processHistogram(CsvReporter.java:22)
at com.yammer.metrics.core.Histogram.processWith(Histogram.java:231)
at com.yammer.metrics.reporting.CsvReporter.run(CsvReporter.java:163)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Re: Kafka producer perf script throw java.io.IOException
Posted by Xinyi Su <xi...@gmail.com>.
Hi, Jun
I create a Jira issue to track it. Please have a look at it.
https://issues.apache.org/jira/browse/KAFKA-1939
Thanks.
Xinyi
On 10 February 2015 at 10:38, Jun Rao <ju...@confluent.io> wrote:
> The new producer uses a different built-in metrics package. Currently, it
> only supports a jmx reporter for the metrics. So you will have to get the
> metrics from jmx.
>
> We can add the csv reporter in ProducerPerformance for the new producer by
> using the new metrics api. Could you file a jira for that?
>
> Thanks,
>
> Jun
>
> On Wed, Feb 4, 2015 at 11:48 PM, Xinyi Su <xi...@gmail.com> wrote:
>
> > Hi,
> >
> > You are right. The file is created by Kafka and Kafka producer csv
> reporter
> > append metrics continuously during perf test. It should not be created
> > continuously during test but create once only.
> >
> > Another thing is when I turn on "--new-producer" and "--sync" option,
> > nothing is appended into ProducerRequestSize.csv. From source code, I
> > have not got any NewShinyProducer record stats code. It may be the
> reason.
> >
> > Thanks.
> > Xinyi
> >
> > On 5 February 2015 at 15:26, Jaikiran Pai <ja...@gmail.com>
> > wrote:
> >
> > > >> java.io.IOException: Unable to create /tmp/PerfTopic22_1/
> > > ProducerRequestSize.csv
> > >
> > > It looks like a file with that exact same name already exists which is
> > > causing that file creation request to fail. This indicates that
> probably
> > > the metric name (ProducerRequestSize) from which the file is created,
> is
> > > duplicate for whatever reason.
> > >
> > > -Jaikiran
> > >
> > >
> > > On Thursday 05 February 2015 12:49 PM, Xinyi Su wrote:
> > >
> > >> Hi,
> > >>
> > >> I need to get more metrics from csv reporter. If turn off
> csv-reporter,
> > >> few output is shown.
> > >>
> > >> Thanks.
> > >> Xinyi
> > >>
> > >> On 5 February 2015 at 13:09, tao xiao <xi...@gmail.com> wrote:
> > >>
> > >> Hi,
> > >>>
> > >>> In order to get it work you can turn off csv-reporter.
> > >>>
> > >>> On Thu, Feb 5, 2015 at 1:06 PM, Xinyi Su <xi...@gmail.com> wrote:
> > >>>
> > >>> Hi,
> > >>>>
> > >>>> Today I updated Kafka cluster from 0.8.2-beta to 0.8.2.0 and run
> kafka
> > >>>> producer performance test.
> > >>>>
> > >>>> The test cannot continue because of some exceptions thrown which
> does
> > >>>> not
> > >>>> occur at 0.8.2-beta. My perf library is kafka-perf_2.9.2-0.8.0.jar
> > which
> > >>>>
> > >>> is
> > >>>
> > >>>> the latest version on maven repository.
> > >>>>
> > >>>> -bash-4.1$ bin/kafka-producer-perf-test.sh --broker-list <broker
> > >>>> list>
> > >>>> --topics PerfTopic22 --sync --initial-message-id 1 --messages 200000
> > >>>> --csv-reporter-enabled --metrics-dir /tmp/PerfTopic22_1
> > >>>> --message-send-gap-ms 20 --request-num-acks -1 --batch-size 1
> > >>>>
> > >>>> java.io.IOException: Unable to create
> > >>>> /tmp/PerfTopic22_1/ProducerRequestSize.csv
> > >>>> at
> > >>>>
> > >>>>
> > >>>> com.yammer.metrics.reporting.CsvReporter.createStreamForMetric(
> > >>> CsvReporter.java:141)
> > >>>
> > >>>> at
> > >>>>
> > >>>>
> > >>>> com.yammer.metrics.reporting.CsvReporter.getPrintStream(
> > >>> CsvReporter.java:257)
> > >>>
> > >>>> at
> > >>>>
> > >>>
> > com.yammer.metrics.reporting.CsvReporter.access$000(CsvReporter.java:22)
> > >>>
> > >>>> at
> > >>>>
> > >>>> com.yammer.metrics.reporting.CsvReporter$1.getStream(
> > >>> CsvReporter.java:156)
> > >>>
> > >>>> at
> > >>>>
> > >>>>
> > >>>> com.yammer.metrics.reporting.CsvReporter.processHistogram(
> > >>> CsvReporter.java:194)
> > >>>
> > >>>> at
> > >>>>
> > >>>>
> > >>>> com.yammer.metrics.reporting.CsvReporter.processHistogram(
> > >>> CsvReporter.java:22)
> > >>>
> > >>>> at com.yammer.metrics.core.Histogram.processWith(Histogram.java:231)
> > >>>> at
> com.yammer.metrics.reporting.CsvReporter.run(CsvReporter.java:163)
> > >>>> at
> > >>>>
> > >>>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> > >>>
> > >>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> > >>>> at
> > >>>>
> > >>>>
> > >>>> java.util.concurrent.ScheduledThreadPoolExecutor$
> > >>> ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> > >>>
> > >>>> at
> > >>>>
> > >>>>
> > >>>> java.util.concurrent.ScheduledThreadPoolExecutor$
> > >>> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> > >>>
> > >>>> at
> > >>>>
> > >>>>
> > >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(
> > >>> ThreadPoolExecutor.java:1145)
> > >>>
> > >>>> at
> > >>>>
> > >>>>
> > >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > >>> ThreadPoolExecutor.java:615)
> > >>>
> > >>>> at java.lang.Thread.run(Thread.java:745)
> > >>>>
> > >>>>
> > >>>
> > >>> --
> > >>> Regards,
> > >>> Tao
> > >>>
> > >>>
> > >
> >
>
Re: Kafka producer perf script throw java.io.IOException
Posted by Jun Rao <ju...@confluent.io>.
The new producer uses a different built-in metrics package. Currently, it
only supports a jmx reporter for the metrics. So you will have to get the
metrics from jmx.
We can add the csv reporter in ProducerPerformance for the new producer by
using the new metrics api. Could you file a jira for that?
Thanks,
Jun
On Wed, Feb 4, 2015 at 11:48 PM, Xinyi Su <xi...@gmail.com> wrote:
> Hi,
>
> You are right. The file is created by Kafka and Kafka producer csv reporter
> append metrics continuously during perf test. It should not be created
> continuously during test but create once only.
>
> Another thing is when I turn on "--new-producer" and "--sync" option,
> nothing is appended into ProducerRequestSize.csv. From source code, I
> have not got any NewShinyProducer record stats code. It may be the reason.
>
> Thanks.
> Xinyi
>
> On 5 February 2015 at 15:26, Jaikiran Pai <ja...@gmail.com>
> wrote:
>
> > >> java.io.IOException: Unable to create /tmp/PerfTopic22_1/
> > ProducerRequestSize.csv
> >
> > It looks like a file with that exact same name already exists which is
> > causing that file creation request to fail. This indicates that probably
> > the metric name (ProducerRequestSize) from which the file is created, is
> > duplicate for whatever reason.
> >
> > -Jaikiran
> >
> >
> > On Thursday 05 February 2015 12:49 PM, Xinyi Su wrote:
> >
> >> Hi,
> >>
> >> I need to get more metrics from csv reporter. If turn off csv-reporter,
> >> few output is shown.
> >>
> >> Thanks.
> >> Xinyi
> >>
> >> On 5 February 2015 at 13:09, tao xiao <xi...@gmail.com> wrote:
> >>
> >> Hi,
> >>>
> >>> In order to get it work you can turn off csv-reporter.
> >>>
> >>> On Thu, Feb 5, 2015 at 1:06 PM, Xinyi Su <xi...@gmail.com> wrote:
> >>>
> >>> Hi,
> >>>>
> >>>> Today I updated Kafka cluster from 0.8.2-beta to 0.8.2.0 and run kafka
> >>>> producer performance test.
> >>>>
> >>>> The test cannot continue because of some exceptions thrown which does
> >>>> not
> >>>> occur at 0.8.2-beta. My perf library is kafka-perf_2.9.2-0.8.0.jar
> which
> >>>>
> >>> is
> >>>
> >>>> the latest version on maven repository.
> >>>>
> >>>> -bash-4.1$ bin/kafka-producer-perf-test.sh --broker-list <broker
> >>>> list>
> >>>> --topics PerfTopic22 --sync --initial-message-id 1 --messages 200000
> >>>> --csv-reporter-enabled --metrics-dir /tmp/PerfTopic22_1
> >>>> --message-send-gap-ms 20 --request-num-acks -1 --batch-size 1
> >>>>
> >>>> java.io.IOException: Unable to create
> >>>> /tmp/PerfTopic22_1/ProducerRequestSize.csv
> >>>> at
> >>>>
> >>>>
> >>>> com.yammer.metrics.reporting.CsvReporter.createStreamForMetric(
> >>> CsvReporter.java:141)
> >>>
> >>>> at
> >>>>
> >>>>
> >>>> com.yammer.metrics.reporting.CsvReporter.getPrintStream(
> >>> CsvReporter.java:257)
> >>>
> >>>> at
> >>>>
> >>>
> com.yammer.metrics.reporting.CsvReporter.access$000(CsvReporter.java:22)
> >>>
> >>>> at
> >>>>
> >>>> com.yammer.metrics.reporting.CsvReporter$1.getStream(
> >>> CsvReporter.java:156)
> >>>
> >>>> at
> >>>>
> >>>>
> >>>> com.yammer.metrics.reporting.CsvReporter.processHistogram(
> >>> CsvReporter.java:194)
> >>>
> >>>> at
> >>>>
> >>>>
> >>>> com.yammer.metrics.reporting.CsvReporter.processHistogram(
> >>> CsvReporter.java:22)
> >>>
> >>>> at com.yammer.metrics.core.Histogram.processWith(Histogram.java:231)
> >>>> at com.yammer.metrics.reporting.CsvReporter.run(CsvReporter.java:163)
> >>>> at
> >>>>
> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >>>
> >>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> >>>> at
> >>>>
> >>>>
> >>>> java.util.concurrent.ScheduledThreadPoolExecutor$
> >>> ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> >>>
> >>>> at
> >>>>
> >>>>
> >>>> java.util.concurrent.ScheduledThreadPoolExecutor$
> >>> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> >>>
> >>>> at
> >>>>
> >>>>
> >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(
> >>> ThreadPoolExecutor.java:1145)
> >>>
> >>>> at
> >>>>
> >>>>
> >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(
> >>> ThreadPoolExecutor.java:615)
> >>>
> >>>> at java.lang.Thread.run(Thread.java:745)
> >>>>
> >>>>
> >>>
> >>> --
> >>> Regards,
> >>> Tao
> >>>
> >>>
> >
>
Re: Kafka producer perf script throw java.io.IOException
Posted by Xinyi Su <xi...@gmail.com>.
Hi,
You are right. The file is created by Kafka and Kafka producer csv reporter
append metrics continuously during perf test. It should not be created
continuously during test but create once only.
Another thing is when I turn on "--new-producer" and "--sync" option,
nothing is appended into ProducerRequestSize.csv. From source code, I
have not got any NewShinyProducer record stats code. It may be the reason.
Thanks.
Xinyi
On 5 February 2015 at 15:26, Jaikiran Pai <ja...@gmail.com> wrote:
> >> java.io.IOException: Unable to create /tmp/PerfTopic22_1/
> ProducerRequestSize.csv
>
> It looks like a file with that exact same name already exists which is
> causing that file creation request to fail. This indicates that probably
> the metric name (ProducerRequestSize) from which the file is created, is
> duplicate for whatever reason.
>
> -Jaikiran
>
>
> On Thursday 05 February 2015 12:49 PM, Xinyi Su wrote:
>
>> Hi,
>>
>> I need to get more metrics from csv reporter. If turn off csv-reporter,
>> few output is shown.
>>
>> Thanks.
>> Xinyi
>>
>> On 5 February 2015 at 13:09, tao xiao <xi...@gmail.com> wrote:
>>
>> Hi,
>>>
>>> In order to get it work you can turn off csv-reporter.
>>>
>>> On Thu, Feb 5, 2015 at 1:06 PM, Xinyi Su <xi...@gmail.com> wrote:
>>>
>>> Hi,
>>>>
>>>> Today I updated Kafka cluster from 0.8.2-beta to 0.8.2.0 and run kafka
>>>> producer performance test.
>>>>
>>>> The test cannot continue because of some exceptions thrown which does
>>>> not
>>>> occur at 0.8.2-beta. My perf library is kafka-perf_2.9.2-0.8.0.jar which
>>>>
>>> is
>>>
>>>> the latest version on maven repository.
>>>>
>>>> -bash-4.1$ bin/kafka-producer-perf-test.sh --broker-list <broker
>>>> list>
>>>> --topics PerfTopic22 --sync --initial-message-id 1 --messages 200000
>>>> --csv-reporter-enabled --metrics-dir /tmp/PerfTopic22_1
>>>> --message-send-gap-ms 20 --request-num-acks -1 --batch-size 1
>>>>
>>>> java.io.IOException: Unable to create
>>>> /tmp/PerfTopic22_1/ProducerRequestSize.csv
>>>> at
>>>>
>>>>
>>>> com.yammer.metrics.reporting.CsvReporter.createStreamForMetric(
>>> CsvReporter.java:141)
>>>
>>>> at
>>>>
>>>>
>>>> com.yammer.metrics.reporting.CsvReporter.getPrintStream(
>>> CsvReporter.java:257)
>>>
>>>> at
>>>>
>>> com.yammer.metrics.reporting.CsvReporter.access$000(CsvReporter.java:22)
>>>
>>>> at
>>>>
>>>> com.yammer.metrics.reporting.CsvReporter$1.getStream(
>>> CsvReporter.java:156)
>>>
>>>> at
>>>>
>>>>
>>>> com.yammer.metrics.reporting.CsvReporter.processHistogram(
>>> CsvReporter.java:194)
>>>
>>>> at
>>>>
>>>>
>>>> com.yammer.metrics.reporting.CsvReporter.processHistogram(
>>> CsvReporter.java:22)
>>>
>>>> at com.yammer.metrics.core.Histogram.processWith(Histogram.java:231)
>>>> at com.yammer.metrics.reporting.CsvReporter.run(CsvReporter.java:163)
>>>> at
>>>>
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>
>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>>>> at
>>>>
>>>>
>>>> java.util.concurrent.ScheduledThreadPoolExecutor$
>>> ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>>>
>>>> at
>>>>
>>>>
>>>> java.util.concurrent.ScheduledThreadPoolExecutor$
>>> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>>
>>>> at
>>>>
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(
>>> ThreadPoolExecutor.java:1145)
>>>
>>>> at
>>>>
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(
>>> ThreadPoolExecutor.java:615)
>>>
>>>> at java.lang.Thread.run(Thread.java:745)
>>>>
>>>>
>>>
>>> --
>>> Regards,
>>> Tao
>>>
>>>
>
Re: Kafka producer perf script throw java.io.IOException
Posted by Jaikiran Pai <ja...@gmail.com>.
>> java.io.IOException: Unable to create
/tmp/PerfTopic22_1/ProducerRequestSize.csv
It looks like a file with that exact same name already exists which is
causing that file creation request to fail. This indicates that probably
the metric name (ProducerRequestSize) from which the file is created, is
duplicate for whatever reason.
-Jaikiran
On Thursday 05 February 2015 12:49 PM, Xinyi Su wrote:
> Hi,
>
> I need to get more metrics from csv reporter. If turn off csv-reporter,
> few output is shown.
>
> Thanks.
> Xinyi
>
> On 5 February 2015 at 13:09, tao xiao <xi...@gmail.com> wrote:
>
>> Hi,
>>
>> In order to get it work you can turn off csv-reporter.
>>
>> On Thu, Feb 5, 2015 at 1:06 PM, Xinyi Su <xi...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Today I updated Kafka cluster from 0.8.2-beta to 0.8.2.0 and run kafka
>>> producer performance test.
>>>
>>> The test cannot continue because of some exceptions thrown which does not
>>> occur at 0.8.2-beta. My perf library is kafka-perf_2.9.2-0.8.0.jar which
>> is
>>> the latest version on maven repository.
>>>
>>> -bash-4.1$ bin/kafka-producer-perf-test.sh --broker-list <broker list>
>>> --topics PerfTopic22 --sync --initial-message-id 1 --messages 200000
>>> --csv-reporter-enabled --metrics-dir /tmp/PerfTopic22_1
>>> --message-send-gap-ms 20 --request-num-acks -1 --batch-size 1
>>>
>>> java.io.IOException: Unable to create
>>> /tmp/PerfTopic22_1/ProducerRequestSize.csv
>>> at
>>>
>>>
>> com.yammer.metrics.reporting.CsvReporter.createStreamForMetric(CsvReporter.java:141)
>>> at
>>>
>>>
>> com.yammer.metrics.reporting.CsvReporter.getPrintStream(CsvReporter.java:257)
>>> at
>> com.yammer.metrics.reporting.CsvReporter.access$000(CsvReporter.java:22)
>>> at
>>>
>> com.yammer.metrics.reporting.CsvReporter$1.getStream(CsvReporter.java:156)
>>> at
>>>
>>>
>> com.yammer.metrics.reporting.CsvReporter.processHistogram(CsvReporter.java:194)
>>> at
>>>
>>>
>> com.yammer.metrics.reporting.CsvReporter.processHistogram(CsvReporter.java:22)
>>> at com.yammer.metrics.core.Histogram.processWith(Histogram.java:231)
>>> at com.yammer.metrics.reporting.CsvReporter.run(CsvReporter.java:163)
>>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>>> at
>>>
>>>
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>>> at
>>>
>>>
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>> at
>>>
>>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> at
>>>
>>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> at java.lang.Thread.run(Thread.java:745)
>>>
>>
>>
>> --
>> Regards,
>> Tao
>>
Re: Kafka producer perf script throw java.io.IOException
Posted by Xinyi Su <xi...@gmail.com>.
Hi,
I need to get more metrics from csv reporter. If turn off csv-reporter,
few output is shown.
Thanks.
Xinyi
On 5 February 2015 at 13:09, tao xiao <xi...@gmail.com> wrote:
> Hi,
>
> In order to get it work you can turn off csv-reporter.
>
> On Thu, Feb 5, 2015 at 1:06 PM, Xinyi Su <xi...@gmail.com> wrote:
>
> > Hi,
> >
> > Today I updated Kafka cluster from 0.8.2-beta to 0.8.2.0 and run kafka
> > producer performance test.
> >
> > The test cannot continue because of some exceptions thrown which does not
> > occur at 0.8.2-beta. My perf library is kafka-perf_2.9.2-0.8.0.jar which
> is
> > the latest version on maven repository.
> >
> > -bash-4.1$ bin/kafka-producer-perf-test.sh --broker-list <broker list>
> > --topics PerfTopic22 --sync --initial-message-id 1 --messages 200000
> > --csv-reporter-enabled --metrics-dir /tmp/PerfTopic22_1
> > --message-send-gap-ms 20 --request-num-acks -1 --batch-size 1
> >
> > java.io.IOException: Unable to create
> > /tmp/PerfTopic22_1/ProducerRequestSize.csv
> > at
> >
> >
> com.yammer.metrics.reporting.CsvReporter.createStreamForMetric(CsvReporter.java:141)
> > at
> >
> >
> com.yammer.metrics.reporting.CsvReporter.getPrintStream(CsvReporter.java:257)
> > at
> com.yammer.metrics.reporting.CsvReporter.access$000(CsvReporter.java:22)
> > at
> >
> com.yammer.metrics.reporting.CsvReporter$1.getStream(CsvReporter.java:156)
> > at
> >
> >
> com.yammer.metrics.reporting.CsvReporter.processHistogram(CsvReporter.java:194)
> > at
> >
> >
> com.yammer.metrics.reporting.CsvReporter.processHistogram(CsvReporter.java:22)
> > at com.yammer.metrics.core.Histogram.processWith(Histogram.java:231)
> > at com.yammer.metrics.reporting.CsvReporter.run(CsvReporter.java:163)
> > at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> > at
> >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> > at
> >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > at java.lang.Thread.run(Thread.java:745)
> >
>
>
>
> --
> Regards,
> Tao
>
Re: Kafka producer perf script throw java.io.IOException
Posted by tao xiao <xi...@gmail.com>.
Hi,
In order to get it work you can turn off csv-reporter.
On Thu, Feb 5, 2015 at 1:06 PM, Xinyi Su <xi...@gmail.com> wrote:
> Hi,
>
> Today I updated Kafka cluster from 0.8.2-beta to 0.8.2.0 and run kafka
> producer performance test.
>
> The test cannot continue because of some exceptions thrown which does not
> occur at 0.8.2-beta. My perf library is kafka-perf_2.9.2-0.8.0.jar which is
> the latest version on maven repository.
>
> -bash-4.1$ bin/kafka-producer-perf-test.sh --broker-list <broker list>
> --topics PerfTopic22 --sync --initial-message-id 1 --messages 200000
> --csv-reporter-enabled --metrics-dir /tmp/PerfTopic22_1
> --message-send-gap-ms 20 --request-num-acks -1 --batch-size 1
>
> java.io.IOException: Unable to create
> /tmp/PerfTopic22_1/ProducerRequestSize.csv
> at
>
> com.yammer.metrics.reporting.CsvReporter.createStreamForMetric(CsvReporter.java:141)
> at
>
> com.yammer.metrics.reporting.CsvReporter.getPrintStream(CsvReporter.java:257)
> at com.yammer.metrics.reporting.CsvReporter.access$000(CsvReporter.java:22)
> at
> com.yammer.metrics.reporting.CsvReporter$1.getStream(CsvReporter.java:156)
> at
>
> com.yammer.metrics.reporting.CsvReporter.processHistogram(CsvReporter.java:194)
> at
>
> com.yammer.metrics.reporting.CsvReporter.processHistogram(CsvReporter.java:22)
> at com.yammer.metrics.core.Histogram.processWith(Histogram.java:231)
> at com.yammer.metrics.reporting.CsvReporter.run(CsvReporter.java:163)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
>
--
Regards,
Tao