You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Lior Zeno <li...@gmail.com> on 2016/10/13 15:19:43 UTC
Flume bechmarks
Hi All,
Monitoring Flume's performance over time is an important step in every
production-level application. Benchmarking Flume on a nightly basis has
the following advantages:
* Better understanding of Flume's bottlenecks.
* Allow users to compare the performance of different solutions, such as
Logstash and Fluentd.
* Better understanding of the influence of recent commits on performance.
Logstash already conducts various performance tests, more details in this
link:
http://logstash-benchmarks.elastic.co/
I propose adding a few micro-benchmarks showing Flume's TPS vs date (of
course, in the ideal case where the input and/or output do not bottleneck
the system), e.g. using the SeqGen source.
Thoughts?
Thanks
Re: Flume bechmarks
Posted by Chris Horrocks <ch...@hor.rocks>.
I've got a pretty well resourced pre-production environment that might do the trick quite nicely.
-- Chris Horrocks
On Thu, Oct 13, 2016 at 4:55 pm, Lior Zeno <'liorzino@gmail.com'> wrote:
I think that we can come up with an initial version with little efforts.
The simplest scenario I can think of is running a Flume instance (with a
SeqGen source and a Null sink) for one minute, and then report the average
events per second.
On Thu, Oct 13, 2016 at 6:43 PM, Attila Simon <sa...@cloudera.com> wrote:
> Good idea! What would be required to set up something similar for Flume?
> ie initial time cost for setting up the infrastructure and periodic time
> cost to add new use-cases.
>
> Cheers,
> Attila
>
>
>
> On Thu, Oct 13, 2016 at 5:19 PM, Lior Zeno <li...@gmail.com> wrote:
>
> > Hi All,
> >
> > Monitoring Flume's performance over time is an important step in every
> > production-level application. Benchmarking Flume on a nightly basis has
> > the following advantages:
> >
> > * Better understanding of Flume's bottlenecks.
> > * Allow users to compare the performance of different solutions, such as
> > Logstash and Fluentd.
> > * Better understanding of the influence of recent commits on performance.
> >
> > Logstash already conducts various performance tests, more details in this
> > link:
> > http://logstash-benchmarks.elastic.co/
> >
> > I propose adding a few micro-benchmarks showing Flume's TPS vs date (of
> > course, in the ideal case where the input and/or output do not bottleneck
> > the system), e.g. using the SeqGen source.
> >
> > Thoughts?
> >
> > Thanks
> >
>
Re: Flume bechmarks
Posted by Lior Zeno <li...@gmail.com>.
I think that we can come up with an initial version with little efforts.
The simplest scenario I can think of is running a Flume instance (with a
SeqGen source and a Null sink) for one minute, and then report the average
events per second.
On Thu, Oct 13, 2016 at 6:43 PM, Attila Simon <sa...@cloudera.com> wrote:
> Good idea! What would be required to set up something similar for Flume?
> ie initial time cost for setting up the infrastructure and periodic time
> cost to add new use-cases.
>
> Cheers,
> Attila
>
>
>
> On Thu, Oct 13, 2016 at 5:19 PM, Lior Zeno <li...@gmail.com> wrote:
>
> > Hi All,
> >
> > Monitoring Flume's performance over time is an important step in every
> > production-level application. Benchmarking Flume on a nightly basis has
> > the following advantages:
> >
> > * Better understanding of Flume's bottlenecks.
> > * Allow users to compare the performance of different solutions, such as
> > Logstash and Fluentd.
> > * Better understanding of the influence of recent commits on performance.
> >
> > Logstash already conducts various performance tests, more details in this
> > link:
> > http://logstash-benchmarks.elastic.co/
> >
> > I propose adding a few micro-benchmarks showing Flume's TPS vs date (of
> > course, in the ideal case where the input and/or output do not bottleneck
> > the system), e.g. using the SeqGen source.
> >
> > Thoughts?
> >
> > Thanks
> >
>
Re: Flume bechmarks
Posted by Attila Simon <sa...@cloudera.com>.
Good idea! What would be required to set up something similar for Flume?
ie initial time cost for setting up the infrastructure and periodic time
cost to add new use-cases.
Cheers,
Attila
On Thu, Oct 13, 2016 at 5:19 PM, Lior Zeno <li...@gmail.com> wrote:
> Hi All,
>
> Monitoring Flume's performance over time is an important step in every
> production-level application. Benchmarking Flume on a nightly basis has
> the following advantages:
>
> * Better understanding of Flume's bottlenecks.
> * Allow users to compare the performance of different solutions, such as
> Logstash and Fluentd.
> * Better understanding of the influence of recent commits on performance.
>
> Logstash already conducts various performance tests, more details in this
> link:
> http://logstash-benchmarks.elastic.co/
>
> I propose adding a few micro-benchmarks showing Flume's TPS vs date (of
> course, in the ideal case where the input and/or output do not bottleneck
> the system), e.g. using the SeqGen source.
>
> Thoughts?
>
> Thanks
>
RE: Flume bechmarks
Posted by Jeff Field <jv...@blizzard.com>.
Those benchmarks are definitely useful but I think it'd be helpful to include the specific flume.conf settings used when benchmarking in the future (with host names/credentials cleaned up of course) to make it easier for others to replicate and compare results.
-----Original Message-----
From: Roshan Naik [mailto:roshan@hortonworks.com]
Sent: Thursday, October 13, 2016 12:42 PM
To: dev@flume.apache.org
Subject: Re: Flume bechmarks
You may want to take a look at
-
https://cwiki.apache.org/confluence/display/FLUME/Performance+Measurements+
-+round+2
and the older..
-
https://cwiki.apache.org/confluence/display/FLUME/Flume+NG+Performance+Meas
urements
when coming up with a list of configurations to benchmark.
-roshan
On 10/13/16, 9:12 AM, "Balazs Donat Bessenyei" <be...@cloudera.com> wrote:
>I have just proposed enabling Travis on a different thread. That should
>help with this. (Having a separate machine would be best, but I don't know
>how we could get one. I'll do the homework for this.)
>
>On Oct 13, 2016 5:57 PM, "Lior Zeno" <li...@gmail.com> wrote:
>
>> Maybe getting an isolated environment? The CI environment might be
>>shared
>> among multiple users, adding too much noise to the performance test.
>>
>> On Thu, Oct 13, 2016 at 6:53 PM, Balazs Donat Bessenyei <
>> bessbd@cloudera.com
>> > wrote:
>>
>> > +1
>> >
>> > I think this is a good idea!
>> >
>> > How can I help with setting it up?
>> >
>> > On Oct 13, 2016 5:20 PM, "Lior Zeno" <li...@gmail.com> wrote:
>> >
>> > > Hi All,
>> > >
>> > > Monitoring Flume's performance over time is an important step in
>>every
>> > > production-level application. Benchmarking Flume on a nightly basis
>> has
>> > > the following advantages:
>> > >
>> > > * Better understanding of Flume's bottlenecks.
>> > > * Allow users to compare the performance of different solutions,
>>such
>> as
>> > > Logstash and Fluentd.
>> > > * Better understanding of the influence of recent commits on
>> performance.
>> > >
>> > > Logstash already conducts various performance tests, more details in
>> this
>> > > link:
>> > > http://logstash-benchmarks.elastic.co/
>> > >
>> > > I propose adding a few micro-benchmarks showing Flume's TPS vs date
>>(of
>> > > course, in the ideal case where the input and/or output do not
>> bottleneck
>> > > the system), e.g. using the SeqGen source.
>> > >
>> > > Thoughts?
>> > >
>> > > Thanks
>> > >
>> >
>>
Re: Flume bechmarks
Posted by Roshan Naik <ro...@hortonworks.com>.
You may want to take a look at
-
https://cwiki.apache.org/confluence/display/FLUME/Performance+Measurements+
-+round+2
and the older..
-
https://cwiki.apache.org/confluence/display/FLUME/Flume+NG+Performance+Meas
urements
when coming up with a list of configurations to benchmark.
-roshan
On 10/13/16, 9:12 AM, "Balazs Donat Bessenyei" <be...@cloudera.com> wrote:
>I have just proposed enabling Travis on a different thread. That should
>help with this. (Having a separate machine would be best, but I don't know
>how we could get one. I'll do the homework for this.)
>
>On Oct 13, 2016 5:57 PM, "Lior Zeno" <li...@gmail.com> wrote:
>
>> Maybe getting an isolated environment? The CI environment might be
>>shared
>> among multiple users, adding too much noise to the performance test.
>>
>> On Thu, Oct 13, 2016 at 6:53 PM, Balazs Donat Bessenyei <
>> bessbd@cloudera.com
>> > wrote:
>>
>> > +1
>> >
>> > I think this is a good idea!
>> >
>> > How can I help with setting it up?
>> >
>> > On Oct 13, 2016 5:20 PM, "Lior Zeno" <li...@gmail.com> wrote:
>> >
>> > > Hi All,
>> > >
>> > > Monitoring Flume's performance over time is an important step in
>>every
>> > > production-level application. Benchmarking Flume on a nightly basis
>> has
>> > > the following advantages:
>> > >
>> > > * Better understanding of Flume's bottlenecks.
>> > > * Allow users to compare the performance of different solutions,
>>such
>> as
>> > > Logstash and Fluentd.
>> > > * Better understanding of the influence of recent commits on
>> performance.
>> > >
>> > > Logstash already conducts various performance tests, more details in
>> this
>> > > link:
>> > > http://logstash-benchmarks.elastic.co/
>> > >
>> > > I propose adding a few micro-benchmarks showing Flume's TPS vs date
>>(of
>> > > course, in the ideal case where the input and/or output do not
>> bottleneck
>> > > the system), e.g. using the SeqGen source.
>> > >
>> > > Thoughts?
>> > >
>> > > Thanks
>> > >
>> >
>>
Re: Flume bechmarks
Posted by Balazs Donat Bessenyei <be...@cloudera.com>.
I have just proposed enabling Travis on a different thread. That should
help with this. (Having a separate machine would be best, but I don't know
how we could get one. I'll do the homework for this.)
On Oct 13, 2016 5:57 PM, "Lior Zeno" <li...@gmail.com> wrote:
> Maybe getting an isolated environment? The CI environment might be shared
> among multiple users, adding too much noise to the performance test.
>
> On Thu, Oct 13, 2016 at 6:53 PM, Balazs Donat Bessenyei <
> bessbd@cloudera.com
> > wrote:
>
> > +1
> >
> > I think this is a good idea!
> >
> > How can I help with setting it up?
> >
> > On Oct 13, 2016 5:20 PM, "Lior Zeno" <li...@gmail.com> wrote:
> >
> > > Hi All,
> > >
> > > Monitoring Flume's performance over time is an important step in every
> > > production-level application. Benchmarking Flume on a nightly basis
> has
> > > the following advantages:
> > >
> > > * Better understanding of Flume's bottlenecks.
> > > * Allow users to compare the performance of different solutions, such
> as
> > > Logstash and Fluentd.
> > > * Better understanding of the influence of recent commits on
> performance.
> > >
> > > Logstash already conducts various performance tests, more details in
> this
> > > link:
> > > http://logstash-benchmarks.elastic.co/
> > >
> > > I propose adding a few micro-benchmarks showing Flume's TPS vs date (of
> > > course, in the ideal case where the input and/or output do not
> bottleneck
> > > the system), e.g. using the SeqGen source.
> > >
> > > Thoughts?
> > >
> > > Thanks
> > >
> >
>
Re: Flume bechmarks
Posted by Lior Zeno <li...@gmail.com>.
Maybe getting an isolated environment? The CI environment might be shared
among multiple users, adding too much noise to the performance test.
On Thu, Oct 13, 2016 at 6:53 PM, Balazs Donat Bessenyei <bessbd@cloudera.com
> wrote:
> +1
>
> I think this is a good idea!
>
> How can I help with setting it up?
>
> On Oct 13, 2016 5:20 PM, "Lior Zeno" <li...@gmail.com> wrote:
>
> > Hi All,
> >
> > Monitoring Flume's performance over time is an important step in every
> > production-level application. Benchmarking Flume on a nightly basis has
> > the following advantages:
> >
> > * Better understanding of Flume's bottlenecks.
> > * Allow users to compare the performance of different solutions, such as
> > Logstash and Fluentd.
> > * Better understanding of the influence of recent commits on performance.
> >
> > Logstash already conducts various performance tests, more details in this
> > link:
> > http://logstash-benchmarks.elastic.co/
> >
> > I propose adding a few micro-benchmarks showing Flume's TPS vs date (of
> > course, in the ideal case where the input and/or output do not bottleneck
> > the system), e.g. using the SeqGen source.
> >
> > Thoughts?
> >
> > Thanks
> >
>
Re: Flume bechmarks
Posted by Balazs Donat Bessenyei <be...@cloudera.com>.
+1
I think this is a good idea!
How can I help with setting it up?
On Oct 13, 2016 5:20 PM, "Lior Zeno" <li...@gmail.com> wrote:
> Hi All,
>
> Monitoring Flume's performance over time is an important step in every
> production-level application. Benchmarking Flume on a nightly basis has
> the following advantages:
>
> * Better understanding of Flume's bottlenecks.
> * Allow users to compare the performance of different solutions, such as
> Logstash and Fluentd.
> * Better understanding of the influence of recent commits on performance.
>
> Logstash already conducts various performance tests, more details in this
> link:
> http://logstash-benchmarks.elastic.co/
>
> I propose adding a few micro-benchmarks showing Flume's TPS vs date (of
> course, in the ideal case where the input and/or output do not bottleneck
> the system), e.g. using the SeqGen source.
>
> Thoughts?
>
> Thanks
>