You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@jmeter.apache.org by Oliver Erlewein <ol...@erlewein.net> on 2014/08/20 06:01:07 UTC

JMeter and NewRelic

Hi everyone,

Has anyone connected JMeter output up to NewRelic? I know BlazeMeter & Co
are doing it so it is possible. What I am looking for is to suck in
response time data and error data into NewRelic.

I thought I'd ask before I go on and try and develop something myself (and
it doesn't look as simple as I first thought). Any help would be
appreciated.

Cheers Oliver

Re: JMeter and NewRelic

Posted by Oliver Erlewein <ol...@erlewein.net>.
Hey,

Thanks for all the input. I'll have a look.
It would be nice to have simple error data and response times (even if
averaged over some timeframe) in there, just so that there is a reference
point. I already generate my own detailed graphs and amalgamate/extend that
from PRTG graphs. This would go further than that. There is also an option
that might happen, where I can use splunk, which would probably suit
better.

But alas I'll try this first and see how I go.

Cheers!
Oliver
On 23 Aug 2014 08:43, "Tim Koopmans" <ti...@flood.io> wrote:

> We do this in Flood IO https://flood.io/blog/15-new-relic-integration
> except we are pulling metrics from our own reporting engine and pushing
> them to New Relic.
>
> I found the best way to approach this was to create an actual New Relic
> plugin for which the docs are pretty straightforward --
>
> https://docs.newrelic.com/docs/plugins/plugin-developer-resources/developer-reference/working-directly-plugin-api
>
> Once you decide on what to measure it should be fairly simple to extend a
> basic listener as Philippe suggested. You will also need some form of
> aggregation of the metrics to tie in the poll cycle. You can see from our
> blog what basic metrics we push every 60s.
>
> Hope that helps.
>
> Regards,
>
>
> Tim Koopmans
> +61 3 9221 6309
>
> [image: Flood IO Pty Ltd] <https://flood.io>
>
> Level 27, 101 Collins Street
> Melbourne, Vic 3000
>
>
>
>
> On Sat, Aug 23, 2014 at 6:07 AM, Philippe Mouawad <
> philippe.mouawad@gmail.com> wrote:
>
> > Hello Oliver,
> > What exactly do you mean by connecting JMeter output to New Relic ?
> > Is it about putting the results from JMeter into NewRelic to be able to
> see
> > all jmeter indicators (response time, errors, response code ...) within
> New
> > Relic dashboard ?
> > If so, in my understanding NewRelic proposes Api for different languages
> > (Java is one of them) to add custom metrics, I suppose it can be done
> like
> > this using a Listener to do it.
> > But it would be interesting to have the official answer from NewRelic and
> > why not share it with us.
> >
> > @Adrian, were you able to have those infos ? I don't think so, but I
> > suppose you used the JMeter Graphs and compared with New Relice dashboard
> > to give results ?
> >
> > A description or blog about this experience would be great ?
> >
> > Thanks
> > Regards
> > @philmdot
> >
> >
> > On Fri, Aug 22, 2014 at 5:52 PM, Adrian Speteanu <as...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > If you don't get any specific responses from others, I'd recommend
> > > contacting their tech support with confidence, if you're using the
> > > pro/enterprise version.
> > >
> > > We used New Relic only to monitor the applications under test and it
> > proved
> > > a useful way to report performance KPIs, especially since the same tool
> > was
> > > used in production for monitoring. I'd recommend this setup, unless you
> > > need something very specific out of the JMeter logs / graphs.
> > >
> > > Cheers,
> > > Adrian S
> > >
> > >
> > > On Wed, Aug 20, 2014 at 7:01 AM, Oliver Erlewein <ol...@erlewein.net>
> > > wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > Has anyone connected JMeter output up to NewRelic? I know BlazeMeter
> &
> > Co
> > > > are doing it so it is possible. What I am looking for is to suck in
> > > > response time data and error data into NewRelic.
> > > >
> > > > I thought I'd ask before I go on and try and develop something myself
> > > (and
> > > > it doesn't look as simple as I first thought). Any help would be
> > > > appreciated.
> > > >
> > > > Cheers Oliver
> > > >
> > >
> >
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
> >
>

Re: JMeter and NewRelic

Posted by Oliver Erlewein <ol...@erlewein.net>.
Tim, just had a quick read over the doco. Totally doable and probably
enough for my needs. So thanks 4 now!
On 23 Aug 2014 08:43, "Tim Koopmans" <ti...@flood.io> wrote:

> We do this in Flood IO https://flood.io/blog/15-new-relic-integration
> except we are pulling metrics from our own reporting engine and pushing
> them to New Relic.
>
> I found the best way to approach this was to create an actual New Relic
> plugin for which the docs are pretty straightforward --
>
> https://docs.newrelic.com/docs/plugins/plugin-developer-resources/developer-reference/working-directly-plugin-api
>
> Once you decide on what to measure it should be fairly simple to extend a
> basic listener as Philippe suggested. You will also need some form of
> aggregation of the metrics to tie in the poll cycle. You can see from our
> blog what basic metrics we push every 60s.
>
> Hope that helps.
>
> Regards,
>
>
> Tim Koopmans
> +61 3 9221 6309
>
> [image: Flood IO Pty Ltd] <https://flood.io>
>
> Level 27, 101 Collins Street
> Melbourne, Vic 3000
>
>
>
>
> On Sat, Aug 23, 2014 at 6:07 AM, Philippe Mouawad <
> philippe.mouawad@gmail.com> wrote:
>
> > Hello Oliver,
> > What exactly do you mean by connecting JMeter output to New Relic ?
> > Is it about putting the results from JMeter into NewRelic to be able to
> see
> > all jmeter indicators (response time, errors, response code ...) within
> New
> > Relic dashboard ?
> > If so, in my understanding NewRelic proposes Api for different languages
> > (Java is one of them) to add custom metrics, I suppose it can be done
> like
> > this using a Listener to do it.
> > But it would be interesting to have the official answer from NewRelic and
> > why not share it with us.
> >
> > @Adrian, were you able to have those infos ? I don't think so, but I
> > suppose you used the JMeter Graphs and compared with New Relice dashboard
> > to give results ?
> >
> > A description or blog about this experience would be great ?
> >
> > Thanks
> > Regards
> > @philmdot
> >
> >
> > On Fri, Aug 22, 2014 at 5:52 PM, Adrian Speteanu <as...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > If you don't get any specific responses from others, I'd recommend
> > > contacting their tech support with confidence, if you're using the
> > > pro/enterprise version.
> > >
> > > We used New Relic only to monitor the applications under test and it
> > proved
> > > a useful way to report performance KPIs, especially since the same tool
> > was
> > > used in production for monitoring. I'd recommend this setup, unless you
> > > need something very specific out of the JMeter logs / graphs.
> > >
> > > Cheers,
> > > Adrian S
> > >
> > >
> > > On Wed, Aug 20, 2014 at 7:01 AM, Oliver Erlewein <ol...@erlewein.net>
> > > wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > Has anyone connected JMeter output up to NewRelic? I know BlazeMeter
> &
> > Co
> > > > are doing it so it is possible. What I am looking for is to suck in
> > > > response time data and error data into NewRelic.
> > > >
> > > > I thought I'd ask before I go on and try and develop something myself
> > > (and
> > > > it doesn't look as simple as I first thought). Any help would be
> > > > appreciated.
> > > >
> > > > Cheers Oliver
> > > >
> > >
> >
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
> >
>

Re: JMeter and NewRelic

Posted by Tim Koopmans <ti...@flood.io>.
We do this in Flood IO https://flood.io/blog/15-new-relic-integration
except we are pulling metrics from our own reporting engine and pushing
them to New Relic.

I found the best way to approach this was to create an actual New Relic
plugin for which the docs are pretty straightforward --
https://docs.newrelic.com/docs/plugins/plugin-developer-resources/developer-reference/working-directly-plugin-api

Once you decide on what to measure it should be fairly simple to extend a
basic listener as Philippe suggested. You will also need some form of
aggregation of the metrics to tie in the poll cycle. You can see from our
blog what basic metrics we push every 60s.

Hope that helps.

Regards,


Tim Koopmans
+61 3 9221 6309

[image: Flood IO Pty Ltd] <https://flood.io>

Level 27, 101 Collins Street
Melbourne, Vic 3000




On Sat, Aug 23, 2014 at 6:07 AM, Philippe Mouawad <
philippe.mouawad@gmail.com> wrote:

> Hello Oliver,
> What exactly do you mean by connecting JMeter output to New Relic ?
> Is it about putting the results from JMeter into NewRelic to be able to see
> all jmeter indicators (response time, errors, response code ...) within New
> Relic dashboard ?
> If so, in my understanding NewRelic proposes Api for different languages
> (Java is one of them) to add custom metrics, I suppose it can be done like
> this using a Listener to do it.
> But it would be interesting to have the official answer from NewRelic and
> why not share it with us.
>
> @Adrian, were you able to have those infos ? I don't think so, but I
> suppose you used the JMeter Graphs and compared with New Relice dashboard
> to give results ?
>
> A description or blog about this experience would be great ?
>
> Thanks
> Regards
> @philmdot
>
>
> On Fri, Aug 22, 2014 at 5:52 PM, Adrian Speteanu <as...@gmail.com>
> wrote:
>
> > Hi,
> >
> > If you don't get any specific responses from others, I'd recommend
> > contacting their tech support with confidence, if you're using the
> > pro/enterprise version.
> >
> > We used New Relic only to monitor the applications under test and it
> proved
> > a useful way to report performance KPIs, especially since the same tool
> was
> > used in production for monitoring. I'd recommend this setup, unless you
> > need something very specific out of the JMeter logs / graphs.
> >
> > Cheers,
> > Adrian S
> >
> >
> > On Wed, Aug 20, 2014 at 7:01 AM, Oliver Erlewein <ol...@erlewein.net>
> > wrote:
> >
> > > Hi everyone,
> > >
> > > Has anyone connected JMeter output up to NewRelic? I know BlazeMeter &
> Co
> > > are doing it so it is possible. What I am looking for is to suck in
> > > response time data and error data into NewRelic.
> > >
> > > I thought I'd ask before I go on and try and develop something myself
> > (and
> > > it doesn't look as simple as I first thought). Any help would be
> > > appreciated.
> > >
> > > Cheers Oliver
> > >
> >
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>

Re: JMeter and NewRelic

Posted by Adrian Speteanu <as...@gmail.com>.
Hi Philippe,

On Fri, Aug 22, 2014 at 11:07 PM, Philippe Mouawad <
philippe.mouawad@gmail.com> wrote:

> Hello Oliver,
> What exactly do you mean by connecting JMeter output to New Relic ?
> Is it about putting the results from JMeter into NewRelic to be able to see
> all jmeter indicators (response time, errors, response code ...) within New
> Relic dashboard ?
> If so, in my understanding NewRelic proposes Api for different languages
> (Java is one of them) to add custom metrics, I suppose it can be done like
> this using a Listener to do it.
> But it would be interesting to have the official answer from NewRelic and
> why not share it with us.
>
> @Adrian, were you able to have those infos ? I don't think so, but I
> suppose you used the JMeter Graphs and compared with New Relice dashboard
> to give results ?
>

In my case, the projects that required testing have big tech / it ops
support. This is why I'm not familiar with the specific/techie details of
the actual setup. But we did require support from newrelic people for the
initial setup.

In large tests we do very minimal client side monitoring (i.e. jmeter logs,
graphs are out of the question), in either jmeter or the system itself: no
logging apart from console output and minimal machine resources monitoring.
This is needed only to double check test health and results stored in other
tools (ganglia, open tsdb, newrelic, load balancer stats, etc). As such
there is very little to compare. We did double check before running the
test at full speed that results on both sides coincide, but after that
point, monitoring was done only in those tools, whatever was used for each
project, and only on the server under test side! I recommend this setup to
detailed client side monitoring because the first setup allows you more
detailed debugging of the actuall application under test and having both in
place makes little sense when you're confident in your test setup.

I can confirm the results were similar if that's what you need, but I'm not
really sure if the data can be shared publicly... the data is also very
specific to each project (like response times of a particular method, which
is possible in newrelic & open tsdb).

Cheers,
Adrian

>
> A description or blog about this experience would be great ?
>
> Thanks
> Regards
> @philmdot
>
>
> On Fri, Aug 22, 2014 at 5:52 PM, Adrian Speteanu <as...@gmail.com>
> wrote:
>
> > Hi,
> >
> > If you don't get any specific responses from others, I'd recommend
> > contacting their tech support with confidence, if you're using the
> > pro/enterprise version.
> >
> > We used New Relic only to monitor the applications under test and it
> proved
> > a useful way to report performance KPIs, especially since the same tool
> was
> > used in production for monitoring. I'd recommend this setup, unless you
> > need something very specific out of the JMeter logs / graphs.
> >
> > Cheers,
> > Adrian S
> >
> >
> > On Wed, Aug 20, 2014 at 7:01 AM, Oliver Erlewein <ol...@erlewein.net>
> > wrote:
> >
> > > Hi everyone,
> > >
> > > Has anyone connected JMeter output up to NewRelic? I know BlazeMeter &
> Co
> > > are doing it so it is possible. What I am looking for is to suck in
> > > response time data and error data into NewRelic.
> > >
> > > I thought I'd ask before I go on and try and develop something myself
> > (and
> > > it doesn't look as simple as I first thought). Any help would be
> > > appreciated.
> > >
> > > Cheers Oliver
> > >
> >
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>

Re: JMeter and NewRelic

Posted by Philippe Mouawad <ph...@gmail.com>.
Hello Oliver,
What exactly do you mean by connecting JMeter output to New Relic ?
Is it about putting the results from JMeter into NewRelic to be able to see
all jmeter indicators (response time, errors, response code ...) within New
Relic dashboard ?
If so, in my understanding NewRelic proposes Api for different languages
(Java is one of them) to add custom metrics, I suppose it can be done like
this using a Listener to do it.
But it would be interesting to have the official answer from NewRelic and
why not share it with us.

@Adrian, were you able to have those infos ? I don't think so, but I
suppose you used the JMeter Graphs and compared with New Relice dashboard
to give results ?

A description or blog about this experience would be great ?

Thanks
Regards
@philmdot


On Fri, Aug 22, 2014 at 5:52 PM, Adrian Speteanu <as...@gmail.com>
wrote:

> Hi,
>
> If you don't get any specific responses from others, I'd recommend
> contacting their tech support with confidence, if you're using the
> pro/enterprise version.
>
> We used New Relic only to monitor the applications under test and it proved
> a useful way to report performance KPIs, especially since the same tool was
> used in production for monitoring. I'd recommend this setup, unless you
> need something very specific out of the JMeter logs / graphs.
>
> Cheers,
> Adrian S
>
>
> On Wed, Aug 20, 2014 at 7:01 AM, Oliver Erlewein <ol...@erlewein.net>
> wrote:
>
> > Hi everyone,
> >
> > Has anyone connected JMeter output up to NewRelic? I know BlazeMeter & Co
> > are doing it so it is possible. What I am looking for is to suck in
> > response time data and error data into NewRelic.
> >
> > I thought I'd ask before I go on and try and develop something myself
> (and
> > it doesn't look as simple as I first thought). Any help would be
> > appreciated.
> >
> > Cheers Oliver
> >
>



-- 
Cordialement.
Philippe Mouawad.

Re: JMeter and NewRelic

Posted by Adrian Speteanu <as...@gmail.com>.
Hi,

If you don't get any specific responses from others, I'd recommend
contacting their tech support with confidence, if you're using the
pro/enterprise version.

We used New Relic only to monitor the applications under test and it proved
a useful way to report performance KPIs, especially since the same tool was
used in production for monitoring. I'd recommend this setup, unless you
need something very specific out of the JMeter logs / graphs.

Cheers,
Adrian S


On Wed, Aug 20, 2014 at 7:01 AM, Oliver Erlewein <ol...@erlewein.net>
wrote:

> Hi everyone,
>
> Has anyone connected JMeter output up to NewRelic? I know BlazeMeter & Co
> are doing it so it is possible. What I am looking for is to suck in
> response time data and error data into NewRelic.
>
> I thought I'd ask before I go on and try and develop something myself (and
> it doesn't look as simple as I first thought). Any help would be
> appreciated.
>
> Cheers Oliver
>