You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jmeter.apache.org by Philippe Mouawad <ph...@gmail.com> on 2013/12/26 00:24:05 UTC
Graphite Listener
Hello,
First Merry Christmas to you !
Then a little note to notify that I created a bugzilla for this feature and
proposed a patch for it:
- https://issues.apache.org/bugzilla/show_bug.cgi?id=55932
Waiting for your remarks, reviewes and agreement to commit it to JMeter
trunk.
We could mark it as beta or alpha for next release if it seems to you risky
although it is totally isolated from the rest.
Regards
Philippe
Re: Graphite Listener
Posted by Antonio Gomes Rodrigues <ra...@gmail.com>.
Hi,
In my opinion, graphite listener must be include in JMeter core for a lot
of reasons
1. All serious competitors (LoadRunner, neoload etc.) have it
2. It's avoid OutOfMemoryError by saving result in graphite and not in
memory
3. It's allow to have result in real time during very long test and not to
wait the end without tips
4. it's allow to have a complete monitoring with jmxtrans, collectd ...
(see my french article in
http://blog.aliecom.com/supervision-facile-avec-jmxtrans-collectd-et-graphite/)
5. It's allow to have result in real time during failure test (e.g. stop
the database, stop the cache during a load test)
6. It's allow to save the results to use it later (compare 2 run, ....)
etc.
2014/1/1 Philippe Mouawad <ph...@gmail.com>
> On Tuesday, December 31, 2013, sebb wrote:
>
> > On 31 December 2013 17:55, Philippe Mouawad <philippe.mouawad@gmail.com
> <javascript:;>>
> > wrote:
> > > On Tue, Dec 31, 2013 at 5:24 PM, Milamber <milamber@apache.org
> <javascript:;>>
> > wrote:
> > >
> > >>
> > >> Le 31/12/2013 16:48, sebb a ecrit :
> > >>
> > >> I'm not sure this belongs in JMeter at all.
> > >>>
> > >>> Just because it is useful to some, does not mean that it should be
> > >>> included.
> > >>>
> > >>
> > > One of the weaknesses of JMeter is the lack of graphs and historization
> > of
> > > these Graphs, Graphite listener is an answer to it.
> > >
> >
> > JMeter is not primarily intended as a reporting tool.
>
>
> Loot at all competitors they all have reporting and imho it is a mistake to
> ignore it.
I am ok with it
> Take the selection process of a load testing tool, what factor would you
> give to reports ?
> I would personally give a high one.
>
Me too
>
>
>
>
> > So long as there is sufficient reporting to be able to check that a
> > test is running OK, I think that is sufficient.
> >
>
> I disagree as currently summariser does not allow filtering on some
> samplers, for example take an ecommerce website, you might have a total of
> 1% error while on detail you have order pipeline with 30% of error but due
> to the low number of threads running it you won't see it until test has
> ended.
>
I am ok with it
The actual reporting is not enough
>
>
> > Proper analysis should take place offline after the test has completed.
>
> I agree but this feature allows also this.
> What about historisation this feature brings ?
>
Sometime we need feedback before the end of the test
And graphite allow it (real time reporting) and allow to save the results
to analyze it after the test has completed.
>
>
> >
> > What is perhaps missing currently is a way to save the output to a
> > database.
>
> But it's not enough yoy would still have to build reporting
>
I am ok with it.
Graphite allow to save and report the results easily.
If we have a way to save the output to a database, we need more works to
have a great report.
And this solution (to save the output to a database) don't allow to have
real time reporting easily
>
>
> >
> > >>> If we are not careful, JMeter is going to become a huge download
> (it's
> > >>> already quite big) containing lots of stuff that is only used by a
> few
> > >>> people.
> > >>>
> > >>
> > > I don't see currently useless elements , only maybe in Listeners:
> > > - Distribution Graph (alpha)
> > > - Spline Visualizer : consumes a lot of resources so not really usable
> > > - Graph Result : same argument
> > >
> > > What features do you think are unused, maybe we could do some cleanup.
> >
> > In retrospect, I think the BeanShell elements were probably unnecessary.
> > The same functionality is available using BSF and now JSR223.
> >
> > I've never used MongoDB so I don't know if that really belongs either.
> >
> > > Regarding my company, as big users of JMeter we nearly use all elements
> > > within JMeter either to script or load test except for LDAP, Mail and
> > > Listeners I mentionned.
> >
> > I've not used LDAP either, but it is a standard server interface.
> > Likewise Mail.
> >
> > However unfortunately once we have released additional code, it is not
> > going to be easy to drop it, as that might break test plans.
> > JMeter strives for upward compatibility.
> > This is another reason why it is important not to add extensions that
> > are not likely to be of long-term value.
> >
> > Yes, we have dropped some stuff that had become obsolete, but it is
> > not something to be undertaken lightly.
> >
> > > Also looking at various blogs I think nearly all elements are used.
> > >
> > > Regarding download Size, it will only increase by 1.2 mb and it is not
> an
> > > argument today taking into account the low price of disk space.
> >
> > It all adds up.
> >
> > >
> > >
> > >>> I am concerned that this is another addition that is specific some
> 3rd
> > >>> party software, rather than a generic solution which can be used with
> > >>> multiple 3rd party code.
> > >>>
> > >>
> > > Try to find generic solution for this, I don't see any.
> > >
> > >>
> > >>> The code is harder to test (because it needs a 3rd party installion)
> > >>> and therefore harder to maintain.
> > >>> It is dependent on the 3rd party API.
> > >>>
> > >> I don't think it is, look at code it is pretty simple in terms of
> > > features, it sends simple data using a simple format.
> > >
> > > Regarding Graphite Listener there is no real third party library , it
> is
> > > just a writing on a socket in a very simple format and it could be unit
> > > tested.
> > > commons-math3 is used to compute Percentile (it would be useful in a
> lot
> > of
> > > places) and commons-pool2 is to enable pooling of socket, it could in
> the
> > > future
> > > replace excalibur deprecated libraries
> > >
> > >>
> > >>> I think this is the sort of plugin that should probably be hosted
> > >>> externally.
> > >>> Not because of license issues, but because of the maintenance and
> size
> > >>> issues.
> > >>>
> > >>
> > >> Perhaps, the Apache Extras can be a good candidate to hosting this
> kind
> > of
> > >> plugins:
> > >> http://code.google.com/a/apache-extras.org/hosting/
> > >>
> > >> http://community.apache.org/apache-extras/faq.html
> > >>
> > >
> > > It would take us a lot of effort to start a project in it. And you
> can't
> > > ask user to go to X number of websites to have a full featured
> solution.
> > >
> > > I think we should open a poll on user mailing list to ask user their
> > > opinion and see what it gives.
> > >
> > > If we get no reaction, then I will propose it to a third party library
> > like
> > > jmeter-plugins as I think it is nearly always a companion of core
> JMeter
> > > due to the nice graphs at least.
> > >
> > >
> > >>
> > >> Milamber
> > >>
> > >>
> > >>
> > >>
> > >>> On 27 December 2013 16:25, Philippe Mouawad <
> > philippe.mouawad@gmail.com>
> > >>> wrote:
> > >>>
> > >>>> ok, sorry for being impatient :)
> > >>>>
> > >>>> On Friday, December 27, 2013, Milamber wrote:
> > >>>>
> > >>>> Le 27/12/2013 17:05, Philippe Mouawad a ecrit :
> > >>>>>
> > >>>>> Hello,
> > >>>>>> Any feedback ?
> > >>>>>> Agree for commit ?
> > >>>>>>
> > >>>>>> Please wait some days. The current period (christmas, new year)
> > means
> > >>>>> often some vacation.
> > >>>>>
> > >>>>> And Graphite isn't easy to install/configure to test after the
> > listener.
> > >>>>>
> > >>>>> (Some characters seems invalid in the french messages properties)
> > >>>>>
> > >>>>>
> > >>>>> Regards
> > >>>>>>
> > >>>>>> On Thursday, December 26, 2013, Philippe Mouawad wrote:
> > >>>>>>
> > >>>>>> Hello,
> > >>>>>>
> > >>>>>>> First Merry Christmas to you !
> > >>>>>>>
> > >>>>>>> Then a little note to notify that I created a bugzilla for this
> > >>>>>>> feature
> > >>>>>>> and proposed a patch for it:
> > >>>>>>> - https://issues.apache.org/bugzilla/show_bug.cgi?id=55932
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Waiting for your remarks, reviewes and agreement to commit it to
> > >>>>>>> JMeter
> > >>>>>>> trunk.
> > >>>>>>>
> > >>>>>>> We could mark it as beta or alpha for next release if it seems to
> > you
> > >>>>>>> risky although it is totally isolated from the rest.
> > >>>>>>>
> > >>>>>>> Regards
> > >>>>>>> Philippe
> > >>>>>>>
> > >>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
Antonio
Re: Graphite Listener
Posted by Philippe Mouawad <ph...@gmail.com>.
On Tuesday, December 31, 2013, sebb wrote:
> On 31 December 2013 17:55, Philippe Mouawad <philippe.mouawad@gmail.com<javascript:;>>
> wrote:
> > On Tue, Dec 31, 2013 at 5:24 PM, Milamber <milamber@apache.org<javascript:;>>
> wrote:
> >
> >>
> >> Le 31/12/2013 16:48, sebb a ecrit :
> >>
> >> I'm not sure this belongs in JMeter at all.
> >>>
> >>> Just because it is useful to some, does not mean that it should be
> >>> included.
> >>>
> >>
> > One of the weaknesses of JMeter is the lack of graphs and historization
> of
> > these Graphs, Graphite listener is an answer to it.
> >
>
> JMeter is not primarily intended as a reporting tool.
Loot at all competitors they all have reporting and imho it is a mistake to
ignore it.
Take the selection process of a load testing tool, what factor would you
give to reports ?
I would personally give a high one.
> So long as there is sufficient reporting to be able to check that a
> test is running OK, I think that is sufficient.
>
I disagree as currently summariser does not allow filtering on some
samplers, for example take an ecommerce website, you might have a total of
1% error while on detail you have order pipeline with 30% of error but due
to the low number of threads running it you won't see it until test has
ended.
> Proper analysis should take place offline after the test has completed.
I agree but this feature allows also this.
What about historisation this feature brings ?
>
> What is perhaps missing currently is a way to save the output to a
> database.
But it's not enough yoy would still have to build reporting
>
> >>> If we are not careful, JMeter is going to become a huge download (it's
> >>> already quite big) containing lots of stuff that is only used by a few
> >>> people.
> >>>
> >>
> > I don't see currently useless elements , only maybe in Listeners:
> > - Distribution Graph (alpha)
> > - Spline Visualizer : consumes a lot of resources so not really usable
> > - Graph Result : same argument
> >
> > What features do you think are unused, maybe we could do some cleanup.
>
> In retrospect, I think the BeanShell elements were probably unnecessary.
> The same functionality is available using BSF and now JSR223.
>
> I've never used MongoDB so I don't know if that really belongs either.
>
> > Regarding my company, as big users of JMeter we nearly use all elements
> > within JMeter either to script or load test except for LDAP, Mail and
> > Listeners I mentionned.
>
> I've not used LDAP either, but it is a standard server interface.
> Likewise Mail.
>
> However unfortunately once we have released additional code, it is not
> going to be easy to drop it, as that might break test plans.
> JMeter strives for upward compatibility.
> This is another reason why it is important not to add extensions that
> are not likely to be of long-term value.
>
> Yes, we have dropped some stuff that had become obsolete, but it is
> not something to be undertaken lightly.
>
> > Also looking at various blogs I think nearly all elements are used.
> >
> > Regarding download Size, it will only increase by 1.2 mb and it is not an
> > argument today taking into account the low price of disk space.
>
> It all adds up.
>
> >
> >
> >>> I am concerned that this is another addition that is specific some 3rd
> >>> party software, rather than a generic solution which can be used with
> >>> multiple 3rd party code.
> >>>
> >>
> > Try to find generic solution for this, I don't see any.
> >
> >>
> >>> The code is harder to test (because it needs a 3rd party installion)
> >>> and therefore harder to maintain.
> >>> It is dependent on the 3rd party API.
> >>>
> >> I don't think it is, look at code it is pretty simple in terms of
> > features, it sends simple data using a simple format.
> >
> > Regarding Graphite Listener there is no real third party library , it is
> > just a writing on a socket in a very simple format and it could be unit
> > tested.
> > commons-math3 is used to compute Percentile (it would be useful in a lot
> of
> > places) and commons-pool2 is to enable pooling of socket, it could in the
> > future
> > replace excalibur deprecated libraries
> >
> >>
> >>> I think this is the sort of plugin that should probably be hosted
> >>> externally.
> >>> Not because of license issues, but because of the maintenance and size
> >>> issues.
> >>>
> >>
> >> Perhaps, the Apache Extras can be a good candidate to hosting this kind
> of
> >> plugins:
> >> http://code.google.com/a/apache-extras.org/hosting/
> >>
> >> http://community.apache.org/apache-extras/faq.html
> >>
> >
> > It would take us a lot of effort to start a project in it. And you can't
> > ask user to go to X number of websites to have a full featured solution.
> >
> > I think we should open a poll on user mailing list to ask user their
> > opinion and see what it gives.
> >
> > If we get no reaction, then I will propose it to a third party library
> like
> > jmeter-plugins as I think it is nearly always a companion of core JMeter
> > due to the nice graphs at least.
> >
> >
> >>
> >> Milamber
> >>
> >>
> >>
> >>
> >>> On 27 December 2013 16:25, Philippe Mouawad <
> philippe.mouawad@gmail.com>
> >>> wrote:
> >>>
> >>>> ok, sorry for being impatient :)
> >>>>
> >>>> On Friday, December 27, 2013, Milamber wrote:
> >>>>
> >>>> Le 27/12/2013 17:05, Philippe Mouawad a ecrit :
> >>>>>
> >>>>> Hello,
> >>>>>> Any feedback ?
> >>>>>> Agree for commit ?
> >>>>>>
> >>>>>> Please wait some days. The current period (christmas, new year)
> means
> >>>>> often some vacation.
> >>>>>
> >>>>> And Graphite isn't easy to install/configure to test after the
> listener.
> >>>>>
> >>>>> (Some characters seems invalid in the french messages properties)
> >>>>>
> >>>>>
> >>>>> Regards
> >>>>>>
> >>>>>> On Thursday, December 26, 2013, Philippe Mouawad wrote:
> >>>>>>
> >>>>>> Hello,
> >>>>>>
> >>>>>>> First Merry Christmas to you !
> >>>>>>>
> >>>>>>> Then a little note to notify that I created a bugzilla for this
> >>>>>>> feature
> >>>>>>> and proposed a patch for it:
> >>>>>>> - https://issues.apache.org/bugzilla/show_bug.cgi?id=55932
> >>>>>>>
> >>>>>>>
> >>>>>>> Waiting for your remarks, reviewes and agreement to commit it to
> >>>>>>> JMeter
> >>>>>>> trunk.
> >>>>>>>
> >>>>>>> We could mark it as beta or alpha for next release if it seems to
> you
> >>>>>>> risky although it is totally isolated from the rest.
> >>>>>>>
> >>>>>>> Regards
> >>>>>>> Philippe
> >>>>>>>
> >>
--
Cordialement.
Philippe Mouawad.
Re: Graphite Listener
Posted by sebb <se...@gmail.com>.
On 31 December 2013 17:55, Philippe Mouawad <ph...@gmail.com> wrote:
> On Tue, Dec 31, 2013 at 5:24 PM, Milamber <mi...@apache.org> wrote:
>
>>
>> Le 31/12/2013 16:48, sebb a ecrit :
>>
>> I'm not sure this belongs in JMeter at all.
>>>
>>> Just because it is useful to some, does not mean that it should be
>>> included.
>>>
>>
> One of the weaknesses of JMeter is the lack of graphs and historization of
> these Graphs, Graphite listener is an answer to it.
>
JMeter is not primarily intended as a reporting tool.
So long as there is sufficient reporting to be able to check that a
test is running OK, I think that is sufficient.
Proper analysis should take place offline after the test has completed.
What is perhaps missing currently is a way to save the output to a database.
>>> If we are not careful, JMeter is going to become a huge download (it's
>>> already quite big) containing lots of stuff that is only used by a few
>>> people.
>>>
>>
> I don't see currently useless elements , only maybe in Listeners:
> - Distribution Graph (alpha)
> - Spline Visualizer : consumes a lot of resources so not really usable
> - Graph Result : same argument
>
> What features do you think are unused, maybe we could do some cleanup.
In retrospect, I think the BeanShell elements were probably unnecessary.
The same functionality is available using BSF and now JSR223.
I've never used MongoDB so I don't know if that really belongs either.
> Regarding my company, as big users of JMeter we nearly use all elements
> within JMeter either to script or load test except for LDAP, Mail and
> Listeners I mentionned.
I've not used LDAP either, but it is a standard server interface.
Likewise Mail.
However unfortunately once we have released additional code, it is not
going to be easy to drop it, as that might break test plans.
JMeter strives for upward compatibility.
This is another reason why it is important not to add extensions that
are not likely to be of long-term value.
Yes, we have dropped some stuff that had become obsolete, but it is
not something to be undertaken lightly.
> Also looking at various blogs I think nearly all elements are used.
>
> Regarding download Size, it will only increase by 1.2 mb and it is not an
> argument today taking into account the low price of disk space.
It all adds up.
>
>
>>> I am concerned that this is another addition that is specific some 3rd
>>> party software, rather than a generic solution which can be used with
>>> multiple 3rd party code.
>>>
>>
> Try to find generic solution for this, I don't see any.
>
>>
>>> The code is harder to test (because it needs a 3rd party installion)
>>> and therefore harder to maintain.
>>> It is dependent on the 3rd party API.
>>>
>> I don't think it is, look at code it is pretty simple in terms of
> features, it sends simple data using a simple format.
>
> Regarding Graphite Listener there is no real third party library , it is
> just a writing on a socket in a very simple format and it could be unit
> tested.
> commons-math3 is used to compute Percentile (it would be useful in a lot of
> places) and commons-pool2 is to enable pooling of socket, it could in the
> future
> replace excalibur deprecated libraries
>
>>
>>> I think this is the sort of plugin that should probably be hosted
>>> externally.
>>> Not because of license issues, but because of the maintenance and size
>>> issues.
>>>
>>
>> Perhaps, the Apache Extras can be a good candidate to hosting this kind of
>> plugins:
>> http://code.google.com/a/apache-extras.org/hosting/
>>
>> http://community.apache.org/apache-extras/faq.html
>>
>
> It would take us a lot of effort to start a project in it. And you can't
> ask user to go to X number of websites to have a full featured solution.
>
> I think we should open a poll on user mailing list to ask user their
> opinion and see what it gives.
>
> If we get no reaction, then I will propose it to a third party library like
> jmeter-plugins as I think it is nearly always a companion of core JMeter
> due to the nice graphs at least.
>
>
>>
>> Milamber
>>
>>
>>
>>
>>> On 27 December 2013 16:25, Philippe Mouawad <ph...@gmail.com>
>>> wrote:
>>>
>>>> ok, sorry for being impatient :)
>>>>
>>>> On Friday, December 27, 2013, Milamber wrote:
>>>>
>>>> Le 27/12/2013 17:05, Philippe Mouawad a ecrit :
>>>>>
>>>>> Hello,
>>>>>> Any feedback ?
>>>>>> Agree for commit ?
>>>>>>
>>>>>> Please wait some days. The current period (christmas, new year) means
>>>>> often some vacation.
>>>>>
>>>>> And Graphite isn't easy to install/configure to test after the listener.
>>>>>
>>>>> (Some characters seems invalid in the french messages properties)
>>>>>
>>>>>
>>>>> Regards
>>>>>>
>>>>>> On Thursday, December 26, 2013, Philippe Mouawad wrote:
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>>> First Merry Christmas to you !
>>>>>>>
>>>>>>> Then a little note to notify that I created a bugzilla for this
>>>>>>> feature
>>>>>>> and proposed a patch for it:
>>>>>>> - https://issues.apache.org/bugzilla/show_bug.cgi?id=55932
>>>>>>>
>>>>>>>
>>>>>>> Waiting for your remarks, reviewes and agreement to commit it to
>>>>>>> JMeter
>>>>>>> trunk.
>>>>>>>
>>>>>>> We could mark it as beta or alpha for next release if it seems to you
>>>>>>> risky although it is totally isolated from the rest.
>>>>>>>
>>>>>>> Regards
>>>>>>> Philippe
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>> Cordialement.
>>>> Philippe Mouawad.
>>>>
>>>
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
Re: Graphite Listener
Posted by sebb <se...@gmail.com>.
On 1 January 2014 20:40, Philippe Mouawad <ph...@gmail.com> wrote:
> Regarding the vote I propose the following.
> -----------------------------------------------------------------------------------------------------------------------------------------------------
> The JMeter team is asking for your opinion regarding the inclusion in
> JMeter of a Graphite Listener.
-1
That is not the wote I thought was being proposed.
I understood the vote to be about latency:
>> >> One can parse raw csv/xml results and upload for the analysis, however
> it
>> >> is likely to create big latency gap between collection and the
>> >> visualization.
> > >
>> sebb> Is that really a problem for most users?
>> How do we measure that?
> A vote would be a way to sort it out.
This is in any case a departure from the original thread topic.
Please start a new thread in such cases, as it gets impossible to
follow otherwise.
Re: Graphite Listener
Posted by Philippe Mouawad <ph...@gmail.com>.
Regarding the vote I propose the following.
-----------------------------------------------------------------------------------------------------------------------------------------------------
The JMeter team is asking for your opinion regarding the inclusion in
JMeter of a Graphite Listener.
Graphite overview can be found here:
- http://graphite.readthedocs.org/en/latest/overview.html
This listener will allow JMeter to dump summary of response time and status
every second:
1) Per second:
- min response time
- max response time
- percentile 90 for response time rolling over last 100 values
- percentile 95 for response time rolling over last 100 values
- successful requests
- failed requests
- total request
2) Listener will also send the following metrics which are not based
on a variation per second:
- ActiveThreads
- Stopped Threads
- Started Threads
More details can be found here:
- https://issues.apache.org/bugzilla/show_bug.cgi?id=55932
To give you opinion, select one of the 3 options:
[+1] I support this inclusion in JMeter core
[0] It should be available within a third-party library
[-1] I vote against any inclusion
Vote is open for 1 week.
Regards
Apache JMeter Team
-----------------------------------------------------------------------------------------------------------------------------------------------------
On Wed, Jan 1, 2014 at 9:28 PM, Philippe Mouawad <philippe.mouawad@gmail.com
> wrote:
>
>
>
> On Wed, Jan 1, 2014 at 8:04 PM, sebb <se...@gmail.com> wrote:
>
>> On 1 January 2014 15:08, Philippe Mouawad <ph...@gmail.com>
>> wrote:
>> > On Tue, Dec 31, 2013 at 9:00 PM, Vladimir Sitnikov <
>> > sitnikov.vladimir@gmail.com> wrote:
>> >
>> >> >> I believe JMeter plugin-like interface between injector and
>> performance
>> >> >> repository would enable live monitoring of sytem under test.
>> >> sebb>Exactly. The JMeter architecture is designed to allow for easy
>> >> integration of 3rd party plugins.
>> >>
>> >> I need to double-check, however I did not manage to specify "path to
>> >> plugins.jar folder" via command line option.
>> >> Ultimately I would love to have JMeter installation and plugins in
>> >> completely separate folders. That simplifies "jmeter upgrade", "check
>> >> what plugins are installed", "compose test harness from maven"
>> >> usecases.
>> >>
>> >> Add this to user.properties:
>> > plugin_dependency_paths which will contains the path to third party jar
>> >
>> > user.classpath which will contain path to dependencies of third party
>> jar
>> >
>> >
>> >
>> >> Are there "asynchronous output" interfaces from JMeter?
>> >>
>> >
>> > in remote testing you are in async mode with:
>> > mode=StrippedAsynch
>> > or
>> > mode= Asynch
>> >
>> >>
>> >> Is there a way to send listener result via regular samplers?
>> >>
>> >
>> > Why would you like to do that ?
>> >
>> > sebb> Proper analysis should take place offline after the test has
>> >> completed.
>> >> Very true.
>> >> However, it is quite important to perform online analysis to be able
>> >> to adjust the test.
>> >> Say, adjust the load, fix bugs in script, correct system configuration,
>> >> etc.
>> >>
>> >> >> One can parse raw csv/xml results and upload for the analysis,
>> however
>> >> it
>> >> >> is likely to create big latency gap between collection and the
>> >> >> visualization.
>> >> >
>> >> sebb> Is that really a problem for most users?
>> >> How do we measure that?
>> >>
>> >
>> > A vote would be a way to sort it out.
>>
>> Perhaps, though it's very difficult to know whether a vote is
>> representative.
>>
>> >>
>> >> Here are the most relevant scenarios for our company:
>> >> 1) Durability testing. Say, you launch 1..7days long test script.
>> >> It is crucial to know if the system is yet stable.
>> >> That includes multiple KPIs (throughput, latency, failrate%, memory
>> >> consumption, gc, cpu%), and the request processing KPIs are not the
>> >> least important ones.
>> >> In case JMeter sampler info is not ready until the whole test is
>> >> finished it is major drawback.
>> >>
>> >
>> > Agree
>>
>> There are ways round this.
>> The Generate Summary Results Listener was designed for just this sort
>> of scenario.
>> I was involved in testing a large system where the test ran for
>> several days, and that was ideal for checking that the test was still
>> running well.
>>
>> Obviously it only provides basic info, but sufficient to check that
>> the test is generally behaving (or not).
>> Its big advantage is the low overhead and simplicity.
>>
>> >
>> >>
>> >> 2) Automated scalability testing. Say you want to identify maximum
>> >> load the system will sustain. One way to identify it is to gradually
>> >> increase the load and see if the system is stable (e.g. queues do not
>> >> build up, failrate is 0, response times are stable, etc).
>> >> Having data displayed in near-realtime helps a lot.
>> >> Especially, when you run test suite in different environment (e.g.
>> >> acceptance of new software/hardware)
>> >>
>> >
>> > Agree
>>
>> We used the Generate Summary Results Listener for that as well.
>>
>> >>
>> >> 3) Tight-schedule testing. When performing load testing at customer
>> >> environment (e.g. acceptance of production environment), it is
>> >> important to make informed decisions. Is good to see if your test
>> >> works as expected when the test is running, not when you've done 4
>> >> hours of testing and analyzing afterwards.
>> >>
>> >
>> > Agree
>> >
>> >>
>> >> 4) Red-green detection during regular 30-60min testing. Our scenarios
>> >> involve multiple jmx scripts and lots of samples.
>> >> We use JMeter GUI only for script development. We use just console for
>> >> load testing to avoid injector slowness/out of memory/etc.
>> >> Currently it is hard to tell if the test goes as expected: failrate%,
>> >> number of samples, response times, etc.
>>
>> Most of that is available from the Generate Summary Results Listener
>> already.
>>
>> >>
>> >> Agree
>> >
>> >
>> >> sebb> That would be a sensible addition to JMeter to allow performance
>> data
>> >> > to be readily saved to an arbitrary repo.
>> >> True.
>> >> OOB samplers might work (e.g. http sampler) might work for the "output
>> >> interfaces".
>> >> For instance "http sampler" under "stats listener" posts the result of
>> >> "stats listener" to the configured end-point.
>> >>
>> >> However, I believe this kind of integration should be asynchronous to
>> >> avoid impact result collection on the test scenario. If trying to use
>> >> sync result posting, we could result into multi-second test hang due
>> >> to lag of performance repository receiver.
>> >>
>> >> Currently the Graphite Listener patch it is implemented this way:
>> > 1) Synchronously, success , failures, min and max are updated (very fast
>> > computations), percentile related data is also computed (Looking at
>> code of
>> > commons-math3 I don't see why it would not perform fine). If further
>> tests
>> > show the percentile computation is costly , then we could delay this to
>> the
>> > 2) thread below. But the tests I did did not show problem.
>> > 2) And in the background, a task runs every minute to send current
>> computed
>> > results to Graphite server , resetting datas except for percentile
>> related
>> > ones.
>>
>> Seems to me all this could be done by having a generic interface to
>> enable the raw results to be saved to whatever backend is required.
>>
>
> We could refactor code in the following way:
> - GraphiteListener => BackendListener
> - Introduce Backend Interface
> - Backend would be called in testStarted/testEnded and within the existing
> run() to send metrics
> - There would be a select box to select backend. Backend implementations
> would be searched within classpaths
> - Graphite part and PickleMetricsManager would be moved to Graphite
> Backend.
>
> We could after that add a JBDC Backend.
>
>>
>> What I don't agree with is yet more code that is very specific to a
>> single 3rd party software solution.
>>
>> Whatever is added needs to be
>> - generally useful to the JMeter population. Niche requirements should
>> be met by 3rd party add-ons developed for the purpose.
>> - use a generic API, for example like JSR223, JDBC etc.
>>
>> I understand your wish but take the history of JMeter. At some time there
> was a need for custom scripting but no standard like BSF of JSR223.
> So you selected Beanshell . Once standard was introduced we moved to it.
> I think we cannot wait for all standards to exist, for example if we take
> NoSQL Databases, it is hard to tell when a JDBC equivalent will be
> available, does it means we should
> wait until they are available ? I don't think so.
>
>
>
>> >
>> >
>> >> Regards,
>> >> Vladimir Sitnikov
>> >>
>> >
>> >
>> >
>> > --
>> > Cordialement.
>> > Philippe Mouawad.
>>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: Graphite Listener
Posted by Philippe Mouawad <ph...@gmail.com>.
opened new discussion for backend listener implementation
On Wed, Jan 1, 2014 at 9:50 PM, sebb <se...@gmail.com> wrote:
> On 1 January 2014 20:46, Philippe Mouawad <ph...@gmail.com>
> wrote:
> > On Wed, Jan 1, 2014 at 9:43 PM, Philippe Mouawad <
> philippe.mouawad@gmail.com
> >> wrote:
> >
> >>
> >>
> >>
> >> On Wed, Jan 1, 2014 at 9:41 PM, sebb <se...@gmail.com> wrote:
> >>
> >>> On 1 January 2014 20:28, Philippe Mouawad <ph...@gmail.com>
> >>> wrote:
> >>> > On Wed, Jan 1, 2014 at 8:04 PM, sebb <se...@gmail.com> wrote:
> >>> >
> >>> >> On 1 January 2014 15:08, Philippe Mouawad <
> philippe.mouawad@gmail.com>
> >>> >> wrote:
> >>>
> >>> <snip/>
> >>>
> >>> >> Seems to me all this could be done by having a generic interface to
> >>> >> enable the raw results to be saved to whatever backend is required.
> >>> >>
> >>> >
> >>> > We could refactor code in the following way:
> >>> > - GraphiteListener => BackendListener
> >>> > - Introduce Backend Interface
> >>> > - Backend would be called in testStarted/testEnded and within the
> >>> existing
> >>> > run() to send metrics
> >>> > - There would be a select box to select backend. Backend
> implementations
> >>> > would be searched within classpaths
> >>> > - Graphite part and PickleMetricsManager would be moved to Graphite
> >>> Backend.
> >>> >
> >>> > We could after that add a JBDC Backend.
> >>>
> >>> That sounds a lot better.
> >>>
> >>> >>
> >>> >> What I don't agree with is yet more code that is very specific to a
> >>> >> single 3rd party software solution.
> >>> >>
> >>> >> Whatever is added needs to be
> >>> >> - generally useful to the JMeter population. Niche requirements
> should
> >>> >> be met by 3rd party add-ons developed for the purpose.
> >>> >> - use a generic API, for example like JSR223, JDBC etc.
> >>> >>
> >>> >> I understand your wish but take the history of JMeter. At some time
> >>> there
> >>> > was a need for custom scripting but no standard like BSF of JSR223.
> >>> > So you selected Beanshell . Once standard was introduced we moved to
> it.
> >>>
> >>> Actually, I think BSF was available, but I did not know about it.
> >>>
> >>
> > So you chose a popular third party at that time, and you did very well I
> > think if you look at JMeter history :-)
>
> Perhaps, but I think it would have worked just as well - if not better
> - if I had chosen to use BSF.
>
> > I think we should accept to make the best choice at the time the choice
> is
> > done, not the best theoretical choice :-)
>
> I think we need to learn from our mistakes.
>
> >
> >>> > I think we cannot wait for all standards to exist, for example if we
> >>> take
> >>> > NoSQL Databases, it is hard to tell when a JDBC equivalent will be
> >>> > available, does it means we should
> >>> > wait until they are available ? I don't think so.
> >>>
> >>> If there is no standard, then we need to look very carefully at what
> >>> JMeter needs as regards an API.
> >>> We should define an API to suit JMeter which can then be implemented
> >>> for specific NoSQL implementations.
> >>>
> >>> It does not have to be a fully fledged NoSQL API - in fact it should
> >>> probably not reference NoSQL at all, but should allow the repository
> >>> to be JDBC as well.
> >>>
> >>> Sorry my note was kind of out of subject, it was refering to last Redis
> >> discussion.
> >> Regarding the Backend discussion, of course if will be totally
> >> independent, could be file, jdbc, jms, no sql, graphite ...
> >>
> >>
> >>> >
> >>> >> >
> >>> >> >
> >>> >> >> Regards,
> >>> >> >> Vladimir Sitnikov
> >>> >> >>
> >>> >> >
> >>> >> >
> >>> >> >
> >>> >> > --
> >>> >> > Cordialement.
> >>> >> > Philippe Mouawad.
> >>> >>
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Cordialement.
> >>> > Philippe Mouawad.
> >>>
> >>
> >>
> >>
> >> --
> >> Cordialement.
> >> Philippe Mouawad.
> >>
> >>
> >>
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
>
--
Cordialement.
Philippe Mouawad.
Re: Graphite Listener
Posted by sebb <se...@gmail.com>.
On 1 January 2014 20:46, Philippe Mouawad <ph...@gmail.com> wrote:
> On Wed, Jan 1, 2014 at 9:43 PM, Philippe Mouawad <philippe.mouawad@gmail.com
>> wrote:
>
>>
>>
>>
>> On Wed, Jan 1, 2014 at 9:41 PM, sebb <se...@gmail.com> wrote:
>>
>>> On 1 January 2014 20:28, Philippe Mouawad <ph...@gmail.com>
>>> wrote:
>>> > On Wed, Jan 1, 2014 at 8:04 PM, sebb <se...@gmail.com> wrote:
>>> >
>>> >> On 1 January 2014 15:08, Philippe Mouawad <ph...@gmail.com>
>>> >> wrote:
>>>
>>> <snip/>
>>>
>>> >> Seems to me all this could be done by having a generic interface to
>>> >> enable the raw results to be saved to whatever backend is required.
>>> >>
>>> >
>>> > We could refactor code in the following way:
>>> > - GraphiteListener => BackendListener
>>> > - Introduce Backend Interface
>>> > - Backend would be called in testStarted/testEnded and within the
>>> existing
>>> > run() to send metrics
>>> > - There would be a select box to select backend. Backend implementations
>>> > would be searched within classpaths
>>> > - Graphite part and PickleMetricsManager would be moved to Graphite
>>> Backend.
>>> >
>>> > We could after that add a JBDC Backend.
>>>
>>> That sounds a lot better.
>>>
>>> >>
>>> >> What I don't agree with is yet more code that is very specific to a
>>> >> single 3rd party software solution.
>>> >>
>>> >> Whatever is added needs to be
>>> >> - generally useful to the JMeter population. Niche requirements should
>>> >> be met by 3rd party add-ons developed for the purpose.
>>> >> - use a generic API, for example like JSR223, JDBC etc.
>>> >>
>>> >> I understand your wish but take the history of JMeter. At some time
>>> there
>>> > was a need for custom scripting but no standard like BSF of JSR223.
>>> > So you selected Beanshell . Once standard was introduced we moved to it.
>>>
>>> Actually, I think BSF was available, but I did not know about it.
>>>
>>
> So you chose a popular third party at that time, and you did very well I
> think if you look at JMeter history :-)
Perhaps, but I think it would have worked just as well - if not better
- if I had chosen to use BSF.
> I think we should accept to make the best choice at the time the choice is
> done, not the best theoretical choice :-)
I think we need to learn from our mistakes.
>
>>> > I think we cannot wait for all standards to exist, for example if we
>>> take
>>> > NoSQL Databases, it is hard to tell when a JDBC equivalent will be
>>> > available, does it means we should
>>> > wait until they are available ? I don't think so.
>>>
>>> If there is no standard, then we need to look very carefully at what
>>> JMeter needs as regards an API.
>>> We should define an API to suit JMeter which can then be implemented
>>> for specific NoSQL implementations.
>>>
>>> It does not have to be a fully fledged NoSQL API - in fact it should
>>> probably not reference NoSQL at all, but should allow the repository
>>> to be JDBC as well.
>>>
>>> Sorry my note was kind of out of subject, it was refering to last Redis
>> discussion.
>> Regarding the Backend discussion, of course if will be totally
>> independent, could be file, jdbc, jms, no sql, graphite ...
>>
>>
>>> >
>>> >> >
>>> >> >
>>> >> >> Regards,
>>> >> >> Vladimir Sitnikov
>>> >> >>
>>> >> >
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Cordialement.
>>> >> > Philippe Mouawad.
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > Cordialement.
>>> > Philippe Mouawad.
>>>
>>
>>
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>>
>>
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
Re: Graphite Listener
Posted by Philippe Mouawad <ph...@gmail.com>.
On Wed, Jan 1, 2014 at 9:43 PM, Philippe Mouawad <philippe.mouawad@gmail.com
> wrote:
>
>
>
> On Wed, Jan 1, 2014 at 9:41 PM, sebb <se...@gmail.com> wrote:
>
>> On 1 January 2014 20:28, Philippe Mouawad <ph...@gmail.com>
>> wrote:
>> > On Wed, Jan 1, 2014 at 8:04 PM, sebb <se...@gmail.com> wrote:
>> >
>> >> On 1 January 2014 15:08, Philippe Mouawad <ph...@gmail.com>
>> >> wrote:
>>
>> <snip/>
>>
>> >> Seems to me all this could be done by having a generic interface to
>> >> enable the raw results to be saved to whatever backend is required.
>> >>
>> >
>> > We could refactor code in the following way:
>> > - GraphiteListener => BackendListener
>> > - Introduce Backend Interface
>> > - Backend would be called in testStarted/testEnded and within the
>> existing
>> > run() to send metrics
>> > - There would be a select box to select backend. Backend implementations
>> > would be searched within classpaths
>> > - Graphite part and PickleMetricsManager would be moved to Graphite
>> Backend.
>> >
>> > We could after that add a JBDC Backend.
>>
>> That sounds a lot better.
>>
>> >>
>> >> What I don't agree with is yet more code that is very specific to a
>> >> single 3rd party software solution.
>> >>
>> >> Whatever is added needs to be
>> >> - generally useful to the JMeter population. Niche requirements should
>> >> be met by 3rd party add-ons developed for the purpose.
>> >> - use a generic API, for example like JSR223, JDBC etc.
>> >>
>> >> I understand your wish but take the history of JMeter. At some time
>> there
>> > was a need for custom scripting but no standard like BSF of JSR223.
>> > So you selected Beanshell . Once standard was introduced we moved to it.
>>
>> Actually, I think BSF was available, but I did not know about it.
>>
>
So you chose a popular third party at that time, and you did very well I
think if you look at JMeter history :-)
I think we should accept to make the best choice at the time the choice is
done, not the best theoretical choice :-)
>> > I think we cannot wait for all standards to exist, for example if we
>> take
>> > NoSQL Databases, it is hard to tell when a JDBC equivalent will be
>> > available, does it means we should
>> > wait until they are available ? I don't think so.
>>
>> If there is no standard, then we need to look very carefully at what
>> JMeter needs as regards an API.
>> We should define an API to suit JMeter which can then be implemented
>> for specific NoSQL implementations.
>>
>> It does not have to be a fully fledged NoSQL API - in fact it should
>> probably not reference NoSQL at all, but should allow the repository
>> to be JDBC as well.
>>
>> Sorry my note was kind of out of subject, it was refering to last Redis
> discussion.
> Regarding the Backend discussion, of course if will be totally
> independent, could be file, jdbc, jms, no sql, graphite ...
>
>
>> >
>> >> >
>> >> >
>> >> >> Regards,
>> >> >> Vladimir Sitnikov
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Cordialement.
>> >> > Philippe Mouawad.
>> >>
>> >
>> >
>> >
>> > --
>> > Cordialement.
>> > Philippe Mouawad.
>>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: Graphite Listener
Posted by Philippe Mouawad <ph...@gmail.com>.
On Wed, Jan 1, 2014 at 9:41 PM, sebb <se...@gmail.com> wrote:
> On 1 January 2014 20:28, Philippe Mouawad <ph...@gmail.com>
> wrote:
> > On Wed, Jan 1, 2014 at 8:04 PM, sebb <se...@gmail.com> wrote:
> >
> >> On 1 January 2014 15:08, Philippe Mouawad <ph...@gmail.com>
> >> wrote:
>
> <snip/>
>
> >> Seems to me all this could be done by having a generic interface to
> >> enable the raw results to be saved to whatever backend is required.
> >>
> >
> > We could refactor code in the following way:
> > - GraphiteListener => BackendListener
> > - Introduce Backend Interface
> > - Backend would be called in testStarted/testEnded and within the
> existing
> > run() to send metrics
> > - There would be a select box to select backend. Backend implementations
> > would be searched within classpaths
> > - Graphite part and PickleMetricsManager would be moved to Graphite
> Backend.
> >
> > We could after that add a JBDC Backend.
>
> That sounds a lot better.
>
> >>
> >> What I don't agree with is yet more code that is very specific to a
> >> single 3rd party software solution.
> >>
> >> Whatever is added needs to be
> >> - generally useful to the JMeter population. Niche requirements should
> >> be met by 3rd party add-ons developed for the purpose.
> >> - use a generic API, for example like JSR223, JDBC etc.
> >>
> >> I understand your wish but take the history of JMeter. At some time
> there
> > was a need for custom scripting but no standard like BSF of JSR223.
> > So you selected Beanshell . Once standard was introduced we moved to it.
>
> Actually, I think BSF was available, but I did not know about it.
>
> > I think we cannot wait for all standards to exist, for example if we take
> > NoSQL Databases, it is hard to tell when a JDBC equivalent will be
> > available, does it means we should
> > wait until they are available ? I don't think so.
>
> If there is no standard, then we need to look very carefully at what
> JMeter needs as regards an API.
> We should define an API to suit JMeter which can then be implemented
> for specific NoSQL implementations.
>
> It does not have to be a fully fledged NoSQL API - in fact it should
> probably not reference NoSQL at all, but should allow the repository
> to be JDBC as well.
>
> Sorry my note was kind of out of subject, it was refering to last Redis
discussion.
Regarding the Backend discussion, of course if will be totally independent,
could be file, jdbc, jms, no sql, graphite ...
> >
> >> >
> >> >
> >> >> Regards,
> >> >> Vladimir Sitnikov
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Cordialement.
> >> > Philippe Mouawad.
> >>
> >
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
>
--
Cordialement.
Philippe Mouawad.
Re: Graphite Listener
Posted by sebb <se...@gmail.com>.
On 1 January 2014 20:28, Philippe Mouawad <ph...@gmail.com> wrote:
> On Wed, Jan 1, 2014 at 8:04 PM, sebb <se...@gmail.com> wrote:
>
>> On 1 January 2014 15:08, Philippe Mouawad <ph...@gmail.com>
>> wrote:
<snip/>
>> Seems to me all this could be done by having a generic interface to
>> enable the raw results to be saved to whatever backend is required.
>>
>
> We could refactor code in the following way:
> - GraphiteListener => BackendListener
> - Introduce Backend Interface
> - Backend would be called in testStarted/testEnded and within the existing
> run() to send metrics
> - There would be a select box to select backend. Backend implementations
> would be searched within classpaths
> - Graphite part and PickleMetricsManager would be moved to Graphite Backend.
>
> We could after that add a JBDC Backend.
That sounds a lot better.
>>
>> What I don't agree with is yet more code that is very specific to a
>> single 3rd party software solution.
>>
>> Whatever is added needs to be
>> - generally useful to the JMeter population. Niche requirements should
>> be met by 3rd party add-ons developed for the purpose.
>> - use a generic API, for example like JSR223, JDBC etc.
>>
>> I understand your wish but take the history of JMeter. At some time there
> was a need for custom scripting but no standard like BSF of JSR223.
> So you selected Beanshell . Once standard was introduced we moved to it.
Actually, I think BSF was available, but I did not know about it.
> I think we cannot wait for all standards to exist, for example if we take
> NoSQL Databases, it is hard to tell when a JDBC equivalent will be
> available, does it means we should
> wait until they are available ? I don't think so.
If there is no standard, then we need to look very carefully at what
JMeter needs as regards an API.
We should define an API to suit JMeter which can then be implemented
for specific NoSQL implementations.
It does not have to be a fully fledged NoSQL API - in fact it should
probably not reference NoSQL at all, but should allow the repository
to be JDBC as well.
>
>> >
>> >
>> >> Regards,
>> >> Vladimir Sitnikov
>> >>
>> >
>> >
>> >
>> > --
>> > Cordialement.
>> > Philippe Mouawad.
>>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
Re: Graphite Listener
Posted by Philippe Mouawad <ph...@gmail.com>.
On Wed, Jan 1, 2014 at 8:04 PM, sebb <se...@gmail.com> wrote:
> On 1 January 2014 15:08, Philippe Mouawad <ph...@gmail.com>
> wrote:
> > On Tue, Dec 31, 2013 at 9:00 PM, Vladimir Sitnikov <
> > sitnikov.vladimir@gmail.com> wrote:
> >
> >> >> I believe JMeter plugin-like interface between injector and
> performance
> >> >> repository would enable live monitoring of sytem under test.
> >> sebb>Exactly. The JMeter architecture is designed to allow for easy
> >> integration of 3rd party plugins.
> >>
> >> I need to double-check, however I did not manage to specify "path to
> >> plugins.jar folder" via command line option.
> >> Ultimately I would love to have JMeter installation and plugins in
> >> completely separate folders. That simplifies "jmeter upgrade", "check
> >> what plugins are installed", "compose test harness from maven"
> >> usecases.
> >>
> >> Add this to user.properties:
> > plugin_dependency_paths which will contains the path to third party jar
> >
> > user.classpath which will contain path to dependencies of third party jar
> >
> >
> >
> >> Are there "asynchronous output" interfaces from JMeter?
> >>
> >
> > in remote testing you are in async mode with:
> > mode=StrippedAsynch
> > or
> > mode= Asynch
> >
> >>
> >> Is there a way to send listener result via regular samplers?
> >>
> >
> > Why would you like to do that ?
> >
> > sebb> Proper analysis should take place offline after the test has
> >> completed.
> >> Very true.
> >> However, it is quite important to perform online analysis to be able
> >> to adjust the test.
> >> Say, adjust the load, fix bugs in script, correct system configuration,
> >> etc.
> >>
> >> >> One can parse raw csv/xml results and upload for the analysis,
> however
> >> it
> >> >> is likely to create big latency gap between collection and the
> >> >> visualization.
> >> >
> >> sebb> Is that really a problem for most users?
> >> How do we measure that?
> >>
> >
> > A vote would be a way to sort it out.
>
> Perhaps, though it's very difficult to know whether a vote is
> representative.
>
> >>
> >> Here are the most relevant scenarios for our company:
> >> 1) Durability testing. Say, you launch 1..7days long test script.
> >> It is crucial to know if the system is yet stable.
> >> That includes multiple KPIs (throughput, latency, failrate%, memory
> >> consumption, gc, cpu%), and the request processing KPIs are not the
> >> least important ones.
> >> In case JMeter sampler info is not ready until the whole test is
> >> finished it is major drawback.
> >>
> >
> > Agree
>
> There are ways round this.
> The Generate Summary Results Listener was designed for just this sort
> of scenario.
> I was involved in testing a large system where the test ran for
> several days, and that was ideal for checking that the test was still
> running well.
>
> Obviously it only provides basic info, but sufficient to check that
> the test is generally behaving (or not).
> Its big advantage is the low overhead and simplicity.
>
> >
> >>
> >> 2) Automated scalability testing. Say you want to identify maximum
> >> load the system will sustain. One way to identify it is to gradually
> >> increase the load and see if the system is stable (e.g. queues do not
> >> build up, failrate is 0, response times are stable, etc).
> >> Having data displayed in near-realtime helps a lot.
> >> Especially, when you run test suite in different environment (e.g.
> >> acceptance of new software/hardware)
> >>
> >
> > Agree
>
> We used the Generate Summary Results Listener for that as well.
>
> >>
> >> 3) Tight-schedule testing. When performing load testing at customer
> >> environment (e.g. acceptance of production environment), it is
> >> important to make informed decisions. Is good to see if your test
> >> works as expected when the test is running, not when you've done 4
> >> hours of testing and analyzing afterwards.
> >>
> >
> > Agree
> >
> >>
> >> 4) Red-green detection during regular 30-60min testing. Our scenarios
> >> involve multiple jmx scripts and lots of samples.
> >> We use JMeter GUI only for script development. We use just console for
> >> load testing to avoid injector slowness/out of memory/etc.
> >> Currently it is hard to tell if the test goes as expected: failrate%,
> >> number of samples, response times, etc.
>
> Most of that is available from the Generate Summary Results Listener
> already.
>
> >>
> >> Agree
> >
> >
> >> sebb> That would be a sensible addition to JMeter to allow performance
> data
> >> > to be readily saved to an arbitrary repo.
> >> True.
> >> OOB samplers might work (e.g. http sampler) might work for the "output
> >> interfaces".
> >> For instance "http sampler" under "stats listener" posts the result of
> >> "stats listener" to the configured end-point.
> >>
> >> However, I believe this kind of integration should be asynchronous to
> >> avoid impact result collection on the test scenario. If trying to use
> >> sync result posting, we could result into multi-second test hang due
> >> to lag of performance repository receiver.
> >>
> >> Currently the Graphite Listener patch it is implemented this way:
> > 1) Synchronously, success , failures, min and max are updated (very fast
> > computations), percentile related data is also computed (Looking at code
> of
> > commons-math3 I don't see why it would not perform fine). If further
> tests
> > show the percentile computation is costly , then we could delay this to
> the
> > 2) thread below. But the tests I did did not show problem.
> > 2) And in the background, a task runs every minute to send current
> computed
> > results to Graphite server , resetting datas except for percentile
> related
> > ones.
>
> Seems to me all this could be done by having a generic interface to
> enable the raw results to be saved to whatever backend is required.
>
We could refactor code in the following way:
- GraphiteListener => BackendListener
- Introduce Backend Interface
- Backend would be called in testStarted/testEnded and within the existing
run() to send metrics
- There would be a select box to select backend. Backend implementations
would be searched within classpaths
- Graphite part and PickleMetricsManager would be moved to Graphite Backend.
We could after that add a JBDC Backend.
>
> What I don't agree with is yet more code that is very specific to a
> single 3rd party software solution.
>
> Whatever is added needs to be
> - generally useful to the JMeter population. Niche requirements should
> be met by 3rd party add-ons developed for the purpose.
> - use a generic API, for example like JSR223, JDBC etc.
>
> I understand your wish but take the history of JMeter. At some time there
was a need for custom scripting but no standard like BSF of JSR223.
So you selected Beanshell . Once standard was introduced we moved to it.
I think we cannot wait for all standards to exist, for example if we take
NoSQL Databases, it is hard to tell when a JDBC equivalent will be
available, does it means we should
wait until they are available ? I don't think so.
> >
> >
> >> Regards,
> >> Vladimir Sitnikov
> >>
> >
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
>
--
Cordialement.
Philippe Mouawad.
Re: Graphite Listener
Posted by sebb <se...@gmail.com>.
On 1 January 2014 15:08, Philippe Mouawad <ph...@gmail.com> wrote:
> On Tue, Dec 31, 2013 at 9:00 PM, Vladimir Sitnikov <
> sitnikov.vladimir@gmail.com> wrote:
>
>> >> I believe JMeter plugin-like interface between injector and performance
>> >> repository would enable live monitoring of sytem under test.
>> sebb>Exactly. The JMeter architecture is designed to allow for easy
>> integration of 3rd party plugins.
>>
>> I need to double-check, however I did not manage to specify "path to
>> plugins.jar folder" via command line option.
>> Ultimately I would love to have JMeter installation and plugins in
>> completely separate folders. That simplifies "jmeter upgrade", "check
>> what plugins are installed", "compose test harness from maven"
>> usecases.
>>
>> Add this to user.properties:
> plugin_dependency_paths which will contains the path to third party jar
>
> user.classpath which will contain path to dependencies of third party jar
>
>
>
>> Are there "asynchronous output" interfaces from JMeter?
>>
>
> in remote testing you are in async mode with:
> mode=StrippedAsynch
> or
> mode= Asynch
>
>>
>> Is there a way to send listener result via regular samplers?
>>
>
> Why would you like to do that ?
>
> sebb> Proper analysis should take place offline after the test has
>> completed.
>> Very true.
>> However, it is quite important to perform online analysis to be able
>> to adjust the test.
>> Say, adjust the load, fix bugs in script, correct system configuration,
>> etc.
>>
>> >> One can parse raw csv/xml results and upload for the analysis, however
>> it
>> >> is likely to create big latency gap between collection and the
>> >> visualization.
>> >
>> sebb> Is that really a problem for most users?
>> How do we measure that?
>>
>
> A vote would be a way to sort it out.
Perhaps, though it's very difficult to know whether a vote is representative.
>>
>> Here are the most relevant scenarios for our company:
>> 1) Durability testing. Say, you launch 1..7days long test script.
>> It is crucial to know if the system is yet stable.
>> That includes multiple KPIs (throughput, latency, failrate%, memory
>> consumption, gc, cpu%), and the request processing KPIs are not the
>> least important ones.
>> In case JMeter sampler info is not ready until the whole test is
>> finished it is major drawback.
>>
>
> Agree
There are ways round this.
The Generate Summary Results Listener was designed for just this sort
of scenario.
I was involved in testing a large system where the test ran for
several days, and that was ideal for checking that the test was still
running well.
Obviously it only provides basic info, but sufficient to check that
the test is generally behaving (or not).
Its big advantage is the low overhead and simplicity.
>
>>
>> 2) Automated scalability testing. Say you want to identify maximum
>> load the system will sustain. One way to identify it is to gradually
>> increase the load and see if the system is stable (e.g. queues do not
>> build up, failrate is 0, response times are stable, etc).
>> Having data displayed in near-realtime helps a lot.
>> Especially, when you run test suite in different environment (e.g.
>> acceptance of new software/hardware)
>>
>
> Agree
We used the Generate Summary Results Listener for that as well.
>>
>> 3) Tight-schedule testing. When performing load testing at customer
>> environment (e.g. acceptance of production environment), it is
>> important to make informed decisions. Is good to see if your test
>> works as expected when the test is running, not when you've done 4
>> hours of testing and analyzing afterwards.
>>
>
> Agree
>
>>
>> 4) Red-green detection during regular 30-60min testing. Our scenarios
>> involve multiple jmx scripts and lots of samples.
>> We use JMeter GUI only for script development. We use just console for
>> load testing to avoid injector slowness/out of memory/etc.
>> Currently it is hard to tell if the test goes as expected: failrate%,
>> number of samples, response times, etc.
Most of that is available from the Generate Summary Results Listener already.
>>
>> Agree
>
>
>> sebb> That would be a sensible addition to JMeter to allow performance data
>> > to be readily saved to an arbitrary repo.
>> True.
>> OOB samplers might work (e.g. http sampler) might work for the "output
>> interfaces".
>> For instance "http sampler" under "stats listener" posts the result of
>> "stats listener" to the configured end-point.
>>
>> However, I believe this kind of integration should be asynchronous to
>> avoid impact result collection on the test scenario. If trying to use
>> sync result posting, we could result into multi-second test hang due
>> to lag of performance repository receiver.
>>
>> Currently the Graphite Listener patch it is implemented this way:
> 1) Synchronously, success , failures, min and max are updated (very fast
> computations), percentile related data is also computed (Looking at code of
> commons-math3 I don't see why it would not perform fine). If further tests
> show the percentile computation is costly , then we could delay this to the
> 2) thread below. But the tests I did did not show problem.
> 2) And in the background, a task runs every minute to send current computed
> results to Graphite server , resetting datas except for percentile related
> ones.
Seems to me all this could be done by having a generic interface to
enable the raw results to be saved to whatever backend is required.
What I don't agree with is yet more code that is very specific to a
single 3rd party software solution.
Whatever is added needs to be
- generally useful to the JMeter population. Niche requirements should
be met by 3rd party add-ons developed for the purpose.
- use a generic API, for example like JSR223, JDBC etc.
>
>
>> Regards,
>> Vladimir Sitnikov
>>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
Re: Graphite Listener
Posted by Philippe Mouawad <ph...@gmail.com>.
On Tue, Dec 31, 2013 at 9:00 PM, Vladimir Sitnikov <
sitnikov.vladimir@gmail.com> wrote:
> >> I believe JMeter plugin-like interface between injector and performance
> >> repository would enable live monitoring of sytem under test.
> sebb>Exactly. The JMeter architecture is designed to allow for easy
> integration of 3rd party plugins.
>
> I need to double-check, however I did not manage to specify "path to
> plugins.jar folder" via command line option.
> Ultimately I would love to have JMeter installation and plugins in
> completely separate folders. That simplifies "jmeter upgrade", "check
> what plugins are installed", "compose test harness from maven"
> usecases.
>
> Add this to user.properties:
plugin_dependency_paths which will contains the path to third party jar
user.classpath which will contain path to dependencies of third party jar
> Are there "asynchronous output" interfaces from JMeter?
>
in remote testing you are in async mode with:
mode=StrippedAsynch
or
mode= Asynch
>
> Is there a way to send listener result via regular samplers?
>
Why would you like to do that ?
sebb> Proper analysis should take place offline after the test has
> completed.
> Very true.
> However, it is quite important to perform online analysis to be able
> to adjust the test.
> Say, adjust the load, fix bugs in script, correct system configuration,
> etc.
>
> >> One can parse raw csv/xml results and upload for the analysis, however
> it
> >> is likely to create big latency gap between collection and the
> >> visualization.
> >
> sebb> Is that really a problem for most users?
> How do we measure that?
>
A vote would be a way to sort it out.
>
> Here are the most relevant scenarios for our company:
> 1) Durability testing. Say, you launch 1..7days long test script.
> It is crucial to know if the system is yet stable.
> That includes multiple KPIs (throughput, latency, failrate%, memory
> consumption, gc, cpu%), and the request processing KPIs are not the
> least important ones.
> In case JMeter sampler info is not ready until the whole test is
> finished it is major drawback.
>
Agree
>
> 2) Automated scalability testing. Say you want to identify maximum
> load the system will sustain. One way to identify it is to gradually
> increase the load and see if the system is stable (e.g. queues do not
> build up, failrate is 0, response times are stable, etc).
> Having data displayed in near-realtime helps a lot.
> Especially, when you run test suite in different environment (e.g.
> acceptance of new software/hardware)
>
Agree
>
> 3) Tight-schedule testing. When performing load testing at customer
> environment (e.g. acceptance of production environment), it is
> important to make informed decisions. Is good to see if your test
> works as expected when the test is running, not when you've done 4
> hours of testing and analyzing afterwards.
>
Agree
>
> 4) Red-green detection during regular 30-60min testing. Our scenarios
> involve multiple jmx scripts and lots of samples.
> We use JMeter GUI only for script development. We use just console for
> load testing to avoid injector slowness/out of memory/etc.
> Currently it is hard to tell if the test goes as expected: failrate%,
> number of samples, response times, etc.
>
> Agree
> sebb> That would be a sensible addition to JMeter to allow performance data
> > to be readily saved to an arbitrary repo.
> True.
> OOB samplers might work (e.g. http sampler) might work for the "output
> interfaces".
> For instance "http sampler" under "stats listener" posts the result of
> "stats listener" to the configured end-point.
>
> However, I believe this kind of integration should be asynchronous to
> avoid impact result collection on the test scenario. If trying to use
> sync result posting, we could result into multi-second test hang due
> to lag of performance repository receiver.
>
> Currently the Graphite Listener patch it is implemented this way:
1) Synchronously, success , failures, min and max are updated (very fast
computations), percentile related data is also computed (Looking at code of
commons-math3 I don't see why it would not perform fine). If further tests
show the percentile computation is costly , then we could delay this to the
2) thread below. But the tests I did did not show problem.
2) And in the background, a task runs every minute to send current computed
results to Graphite server , resetting datas except for percentile related
ones.
> Regards,
> Vladimir Sitnikov
>
--
Cordialement.
Philippe Mouawad.
Re: Graphite Listener
Posted by Vladimir Sitnikov <si...@gmail.com>.
>> I believe JMeter plugin-like interface between injector and performance
>> repository would enable live monitoring of sytem under test.
sebb>Exactly. The JMeter architecture is designed to allow for easy
integration of 3rd party plugins.
I need to double-check, however I did not manage to specify "path to
plugins.jar folder" via command line option.
Ultimately I would love to have JMeter installation and plugins in
completely separate folders. That simplifies "jmeter upgrade", "check
what plugins are installed", "compose test harness from maven"
usecases.
Are there "asynchronous output" interfaces from JMeter?
Is there a way to send listener result via regular samplers?
sebb> Proper analysis should take place offline after the test has completed.
Very true.
However, it is quite important to perform online analysis to be able
to adjust the test.
Say, adjust the load, fix bugs in script, correct system configuration, etc.
>> One can parse raw csv/xml results and upload for the analysis, however it
>> is likely to create big latency gap between collection and the
>> visualization.
>
sebb> Is that really a problem for most users?
How do we measure that?
Here are the most relevant scenarios for our company:
1) Durability testing. Say, you launch 1..7days long test script.
It is crucial to know if the system is yet stable.
That includes multiple KPIs (throughput, latency, failrate%, memory
consumption, gc, cpu%), and the request processing KPIs are not the
least important ones.
In case JMeter sampler info is not ready until the whole test is
finished it is major drawback.
2) Automated scalability testing. Say you want to identify maximum
load the system will sustain. One way to identify it is to gradually
increase the load and see if the system is stable (e.g. queues do not
build up, failrate is 0, response times are stable, etc).
Having data displayed in near-realtime helps a lot.
Especially, when you run test suite in different environment (e.g.
acceptance of new software/hardware)
3) Tight-schedule testing. When performing load testing at customer
environment (e.g. acceptance of production environment), it is
important to make informed decisions. Is good to see if your test
works as expected when the test is running, not when you've done 4
hours of testing and analyzing afterwards.
4) Red-green detection during regular 30-60min testing. Our scenarios
involve multiple jmx scripts and lots of samples.
We use JMeter GUI only for script development. We use just console for
load testing to avoid injector slowness/out of memory/etc.
Currently it is hard to tell if the test goes as expected: failrate%,
number of samples, response times, etc.
sebb> That would be a sensible addition to JMeter to allow performance data
> to be readily saved to an arbitrary repo.
True.
OOB samplers might work (e.g. http sampler) might work for the "output
interfaces".
For instance "http sampler" under "stats listener" posts the result of
"stats listener" to the configured end-point.
However, I believe this kind of integration should be asynchronous to
avoid impact result collection on the test scenario. If trying to use
sync result posting, we could result into multi-second test hang due
to lag of performance repository receiver.
Regards,
Vladimir Sitnikov
Re: Graphite Listener
Posted by sebb <se...@gmail.com>.
On 31 December 2013 18:35, Vladimir Sitnikov
<si...@gmail.com> wrote:
>>>One of the weaknesses of JMeter is the lack of graphs and historization of
> these Graphs, Graphite listener is an answer to it.
>
> I fully agree there is a gap between measurements collection and
> processing/reporting.
> In our company there is a need of performance repository and we are
> weighting the options.
>
> I believe JMeter plugin-like interface between injector and performance
> repository would enable live monitoring of sytem under test.
Exactly. The JMeter architecture is designed to allow for easy
integration of 3rd party plugins.
> One can parse raw csv/xml results and upload for the analysis, however it
> is likely to create big latency gap between collection and the
> visualization.
Is that really a problem for most users?
> In general, I would love to see performance repository integration in
> JMeter itself or in JMeter-plugins.
That would be a sensible addition to JMeter to allow performance data
to be readily saved to an arbitrary repo.
> To be fair, I did not yet look into the proposed listener.
>
> It is not clear if commons-math (and other new dependencies) fits JMeter's
> needs.
> I do not think 1-5MiB download increase make any difference, however
> performance defects or concurrency problems do.
Also maintainability, testing, documentation etc.
> It is not good at all if commons-math/pool/etc breaks under heavy threaded
> load.
>
> Are there benchmarks to see max achieved througput? (e.g. max number of
> listened events using all the cores)
> What latency does listener add? (e.g. max throughput in single thread)
>
> Regards,
> Vladimir Sitnikov
Re: Graphite Listener
Posted by Vladimir Sitnikov <si...@gmail.com>.
>>One of the weaknesses of JMeter is the lack of graphs and historization of
these Graphs, Graphite listener is an answer to it.
I fully agree there is a gap between measurements collection and
processing/reporting.
In our company there is a need of performance repository and we are
weighting the options.
I believe JMeter plugin-like interface between injector and performance
repository would enable live monitoring of sytem under test.
One can parse raw csv/xml results and upload for the analysis, however it
is likely to create big latency gap between collection and the
visualization.
In general, I would love to see performance repository integration in
JMeter itself or in JMeter-plugins.
To be fair, I did not yet look into the proposed listener.
It is not clear if commons-math (and other new dependencies) fits JMeter's
needs.
I do not think 1-5MiB download increase make any difference, however
performance defects or concurrency problems do.
It is not good at all if commons-math/pool/etc breaks under heavy threaded
load.
Are there benchmarks to see max achieved througput? (e.g. max number of
listened events using all the cores)
What latency does listener add? (e.g. max throughput in single thread)
Regards,
Vladimir Sitnikov
Re: Graphite Listener
Posted by Philippe Mouawad <ph...@gmail.com>.
On Tue, Dec 31, 2013 at 5:24 PM, Milamber <mi...@apache.org> wrote:
>
> Le 31/12/2013 16:48, sebb a ecrit :
>
> I'm not sure this belongs in JMeter at all.
>>
>> Just because it is useful to some, does not mean that it should be
>> included.
>>
>
One of the weaknesses of JMeter is the lack of graphs and historization of
these Graphs, Graphite listener is an answer to it.
>> If we are not careful, JMeter is going to become a huge download (it's
>> already quite big) containing lots of stuff that is only used by a few
>> people.
>>
>
I don't see currently useless elements , only maybe in Listeners:
- Distribution Graph (alpha)
- Spline Visualizer : consumes a lot of resources so not really usable
- Graph Result : same argument
What features do you think are unused, maybe we could do some cleanup.
Regarding my company, as big users of JMeter we nearly use all elements
within JMeter either to script or load test except for LDAP, Mail and
Listeners I mentionned.
Also looking at various blogs I think nearly all elements are used.
Regarding download Size, it will only increase by 1.2 mb and it is not an
argument today taking into account the low price of disk space.
>> I am concerned that this is another addition that is specific some 3rd
>> party software, rather than a generic solution which can be used with
>> multiple 3rd party code.
>>
>
Try to find generic solution for this, I don't see any.
>
>> The code is harder to test (because it needs a 3rd party installion)
>> and therefore harder to maintain.
>> It is dependent on the 3rd party API.
>>
> I don't think it is, look at code it is pretty simple in terms of
features, it sends simple data using a simple format.
Regarding Graphite Listener there is no real third party library , it is
just a writing on a socket in a very simple format and it could be unit
tested.
commons-math3 is used to compute Percentile (it would be useful in a lot of
places) and commons-pool2 is to enable pooling of socket, it could in the
future
replace excalibur deprecated libraries
>
>> I think this is the sort of plugin that should probably be hosted
>> externally.
>> Not because of license issues, but because of the maintenance and size
>> issues.
>>
>
> Perhaps, the Apache Extras can be a good candidate to hosting this kind of
> plugins:
> http://code.google.com/a/apache-extras.org/hosting/
>
> http://community.apache.org/apache-extras/faq.html
>
It would take us a lot of effort to start a project in it. And you can't
ask user to go to X number of websites to have a full featured solution.
I think we should open a poll on user mailing list to ask user their
opinion and see what it gives.
If we get no reaction, then I will propose it to a third party library like
jmeter-plugins as I think it is nearly always a companion of core JMeter
due to the nice graphs at least.
>
> Milamber
>
>
>
>
>> On 27 December 2013 16:25, Philippe Mouawad <ph...@gmail.com>
>> wrote:
>>
>>> ok, sorry for being impatient :)
>>>
>>> On Friday, December 27, 2013, Milamber wrote:
>>>
>>> Le 27/12/2013 17:05, Philippe Mouawad a ecrit :
>>>>
>>>> Hello,
>>>>> Any feedback ?
>>>>> Agree for commit ?
>>>>>
>>>>> Please wait some days. The current period (christmas, new year) means
>>>> often some vacation.
>>>>
>>>> And Graphite isn't easy to install/configure to test after the listener.
>>>>
>>>> (Some characters seems invalid in the french messages properties)
>>>>
>>>>
>>>> Regards
>>>>>
>>>>> On Thursday, December 26, 2013, Philippe Mouawad wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>>> First Merry Christmas to you !
>>>>>>
>>>>>> Then a little note to notify that I created a bugzilla for this
>>>>>> feature
>>>>>> and proposed a patch for it:
>>>>>> - https://issues.apache.org/bugzilla/show_bug.cgi?id=55932
>>>>>>
>>>>>>
>>>>>> Waiting for your remarks, reviewes and agreement to commit it to
>>>>>> JMeter
>>>>>> trunk.
>>>>>>
>>>>>> We could mark it as beta or alpha for next release if it seems to you
>>>>>> risky although it is totally isolated from the rest.
>>>>>>
>>>>>> Regards
>>>>>> Philippe
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>> Cordialement.
>>> Philippe Mouawad.
>>>
>>
>
--
Cordialement.
Philippe Mouawad.
Re: Graphite Listener
Posted by Milamber <mi...@apache.org>.
Le 31/12/2013 16:48, sebb a ecrit :
> I'm not sure this belongs in JMeter at all.
>
> Just because it is useful to some, does not mean that it should be included.
>
> If we are not careful, JMeter is going to become a huge download (it's
> already quite big) containing lots of stuff that is only used by a few
> people.
>
> I am concerned that this is another addition that is specific some 3rd
> party software, rather than a generic solution which can be used with
> multiple 3rd party code.
>
> The code is harder to test (because it needs a 3rd party installion)
> and therefore harder to maintain.
> It is dependent on the 3rd party API.
>
> I think this is the sort of plugin that should probably be hosted externally.
> Not because of license issues, but because of the maintenance and size issues.
Perhaps, the Apache Extras can be a good candidate to hosting this kind
of plugins:
http://code.google.com/a/apache-extras.org/hosting/
http://community.apache.org/apache-extras/faq.html
Milamber
>
> On 27 December 2013 16:25, Philippe Mouawad <ph...@gmail.com> wrote:
>> ok, sorry for being impatient :)
>>
>> On Friday, December 27, 2013, Milamber wrote:
>>
>>> Le 27/12/2013 17:05, Philippe Mouawad a ecrit :
>>>
>>>> Hello,
>>>> Any feedback ?
>>>> Agree for commit ?
>>>>
>>> Please wait some days. The current period (christmas, new year) means
>>> often some vacation.
>>>
>>> And Graphite isn't easy to install/configure to test after the listener.
>>>
>>> (Some characters seems invalid in the french messages properties)
>>>
>>>
>>>> Regards
>>>>
>>>> On Thursday, December 26, 2013, Philippe Mouawad wrote:
>>>>
>>>> Hello,
>>>>> First Merry Christmas to you !
>>>>>
>>>>> Then a little note to notify that I created a bugzilla for this feature
>>>>> and proposed a patch for it:
>>>>> - https://issues.apache.org/bugzilla/show_bug.cgi?id=55932
>>>>>
>>>>>
>>>>> Waiting for your remarks, reviewes and agreement to commit it to JMeter
>>>>> trunk.
>>>>>
>>>>> We could mark it as beta or alpha for next release if it seems to you
>>>>> risky although it is totally isolated from the rest.
>>>>>
>>>>> Regards
>>>>> Philippe
>>>>>
>>>>>
>>>>>
>>>>>
>> --
>> Cordialement.
>> Philippe Mouawad.
Re: Graphite Listener
Posted by sebb <se...@gmail.com>.
I'm not sure this belongs in JMeter at all.
Just because it is useful to some, does not mean that it should be included.
If we are not careful, JMeter is going to become a huge download (it's
already quite big) containing lots of stuff that is only used by a few
people.
I am concerned that this is another addition that is specific some 3rd
party software, rather than a generic solution which can be used with
multiple 3rd party code.
The code is harder to test (because it needs a 3rd party installion)
and therefore harder to maintain.
It is dependent on the 3rd party API.
I think this is the sort of plugin that should probably be hosted externally.
Not because of license issues, but because of the maintenance and size issues.
On 27 December 2013 16:25, Philippe Mouawad <ph...@gmail.com> wrote:
> ok, sorry for being impatient :)
>
> On Friday, December 27, 2013, Milamber wrote:
>
>>
>> Le 27/12/2013 17:05, Philippe Mouawad a ecrit :
>>
>>> Hello,
>>> Any feedback ?
>>> Agree for commit ?
>>>
>>
>> Please wait some days. The current period (christmas, new year) means
>> often some vacation.
>>
>> And Graphite isn't easy to install/configure to test after the listener.
>>
>> (Some characters seems invalid in the french messages properties)
>>
>>
>>> Regards
>>>
>>> On Thursday, December 26, 2013, Philippe Mouawad wrote:
>>>
>>> Hello,
>>>> First Merry Christmas to you !
>>>>
>>>> Then a little note to notify that I created a bugzilla for this feature
>>>> and proposed a patch for it:
>>>> - https://issues.apache.org/bugzilla/show_bug.cgi?id=55932
>>>>
>>>>
>>>> Waiting for your remarks, reviewes and agreement to commit it to JMeter
>>>> trunk.
>>>>
>>>> We could mark it as beta or alpha for next release if it seems to you
>>>> risky although it is totally isolated from the rest.
>>>>
>>>> Regards
>>>> Philippe
>>>>
>>>>
>>>>
>>>>
>>
>
> --
> Cordialement.
> Philippe Mouawad.
Re: Graphite Listener
Posted by Philippe Mouawad <ph...@gmail.com>.
ok, sorry for being impatient :)
On Friday, December 27, 2013, Milamber wrote:
>
> Le 27/12/2013 17:05, Philippe Mouawad a ecrit :
>
>> Hello,
>> Any feedback ?
>> Agree for commit ?
>>
>
> Please wait some days. The current period (christmas, new year) means
> often some vacation.
>
> And Graphite isn't easy to install/configure to test after the listener.
>
> (Some characters seems invalid in the french messages properties)
>
>
>> Regards
>>
>> On Thursday, December 26, 2013, Philippe Mouawad wrote:
>>
>> Hello,
>>> First Merry Christmas to you !
>>>
>>> Then a little note to notify that I created a bugzilla for this feature
>>> and proposed a patch for it:
>>> - https://issues.apache.org/bugzilla/show_bug.cgi?id=55932
>>>
>>>
>>> Waiting for your remarks, reviewes and agreement to commit it to JMeter
>>> trunk.
>>>
>>> We could mark it as beta or alpha for next release if it seems to you
>>> risky although it is totally isolated from the rest.
>>>
>>> Regards
>>> Philippe
>>>
>>>
>>>
>>>
>
--
Cordialement.
Philippe Mouawad.
Re: Graphite Listener
Posted by Milamber <mi...@apache.org>.
Le 27/12/2013 17:05, Philippe Mouawad a ecrit :
> Hello,
> Any feedback ?
> Agree for commit ?
Please wait some days. The current period (christmas, new year) means
often some vacation.
And Graphite isn't easy to install/configure to test after the listener.
(Some characters seems invalid in the french messages properties)
>
> Regards
>
> On Thursday, December 26, 2013, Philippe Mouawad wrote:
>
>> Hello,
>> First Merry Christmas to you !
>>
>> Then a little note to notify that I created a bugzilla for this feature
>> and proposed a patch for it:
>> - https://issues.apache.org/bugzilla/show_bug.cgi?id=55932
>>
>>
>> Waiting for your remarks, reviewes and agreement to commit it to JMeter
>> trunk.
>>
>> We could mark it as beta or alpha for next release if it seems to you
>> risky although it is totally isolated from the rest.
>>
>> Regards
>> Philippe
>>
>>
>>
Re: Graphite Listener
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello,
Any feedback ?
Agree for commit ?
Regards
On Thursday, December 26, 2013, Philippe Mouawad wrote:
> Hello,
> First Merry Christmas to you !
>
> Then a little note to notify that I created a bugzilla for this feature
> and proposed a patch for it:
> - https://issues.apache.org/bugzilla/show_bug.cgi?id=55932
>
>
> Waiting for your remarks, reviewes and agreement to commit it to JMeter
> trunk.
>
> We could mark it as beta or alpha for next release if it seems to you
> risky although it is totally isolated from the rest.
>
> Regards
> Philippe
>
>
>
--
Cordialement.
Philippe Mouawad.