You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jmeter-dev@jakarta.apache.org by Wolfram Rittmeyer <wo...@web.de> on 2002/08/12 16:55:55 UTC
BUG 11536 - throughput calculation
Hi,
I have looked at the code and at the output of the GraphVisualizer and
GraphModel and I am amazed. I never thought much about it since it
hasn't been of much interest up to now. But I thought it would calculate
the amount of bytes per time. Yet by looking at the code I realize it
calculates the number of request per minute. The funny (but
understandable) thing is that the results depend upon the timers. If you
add a constant timer of 500 milliseconds results will be much lower than
with a timer of 300 milliseconds. Is this really what we want? I doubt
it. At least it is not, what I would like to see, but others may
disagree.
So I could imagine either to fix this so it shows the real throughput of
bytes per amount of time (i would prefer bytes per second) or to add a
further information to the GraphVisualizer.
BTW: The first solution would kind of "solve" the BUG 11536.
What's your opinion about it?
Greetings,
Wolfram Rittmeyer
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: WAS: BUG 11536 - throughput calculation
Posted by Jordi Salvat i Alabart <js...@atg.com>.
Hi Wolfram.
A while ago I submitted an enhancement request to Bugzilla for a Timer
which waits for a certain amount of time minus whatever the previous
request has taken.
I think this would solve your issue, since you can say:
- Repeat
- Get this page
- Wait for 30 seconds (minus the time it took to get the page)
To generate a traffic of exactly 2 requests per minute.
I'm thinking it would be even greater if you could use this with the
time a whole controller took to run:
- Repeat
- Simple controller
- get this page
- get that page
- submit this form
- Wait for 60 seconds (minus bla bla)
To generate a traffic of exactly 1 submission per minute (provided the
whole controller completes in less than 1 minute, of course).
I'll try to find time to add this to the enhancement request... and to
give a crack at implementing it :-)
Salut,
Jordi.
Wolfram Rittmeyer wrote:
> Mike wrote:
>
>
>>But, here's the problem: The server didn't actually serve up 10 requests per second. It was
>>only asked to serve up some fraction of that by JMeter - for 5 seconds between each request,
>>JMeter "gave the server a break", so to speak. I think developers would be incorrect to think
>>that the 10 requests/second calculation represents what their server could have done. You
>>don't know that. If you take away JMeter's 5 second delay, you might find the numbers get
>>worse - it depends since servers can get bogged down unexpectedly, and performance can
>>go in the toilet without warning as you increase number of hits attempted.
>>
>>Although your calculation seems to indicate the server can handle 10 requests per second,
>>you might find that in actuality, it can never reach that. In which case, I have to ask - what is
>>the value of that calculation?
>>
>>The current calculation gives a cold, hard number - this is the number of requests that the
>>server actually handled. It doesn't represent a theoretical max, or even a theoretical max given
>>the limited # of threads. It's the actual. to find your server's max throughput, you have to run
>>multiple tests, changing the number of users. As you increase the number of simultaneous
>>users, the actual throughput will generally increase, until you reach a point at which your
>>server either plateau's or begins to decline. This represents your server's maximum
>>throughput. That is my thinking. Feel free to set me straight :-)
>>
>>-Mike
>>
>
>
> Even this is somewhat hypothetical. At least if some pages need more
> processing-time than others. You would know which is more time-intensive
> of course, but how often which page is really called, that's a thing you
> just get to know when your page (in the case of HTTP-requests) is up and
> real users use it. They decide wether they want the page to be
> recomputed by changing some parameters and so on. If you just request
> this time-intensive page two times in your test but it get's called four
> times on average in reality, your calculated throughput will quickly get
> a number with no counterpart in reality. Therefore, to be honest, every
> throughput calculation in advance is somewhat hypothetical, isn't it.
>
> So, finally, I must admit that changing the GraphVisualizers doesn't
> make much sense ;-) Anyhow I think the discussion was fertile (or
> insightful or whatever one would say in English)...
>
> Nevertheless, as a result of that I want to suggest something else (I
> will post an enhancement request later on when I have my thoughts a bit
> more straight), but since I'd like to discuss the usefullness, I just
> include it in here:
>
> 1. Another Visualizer that visualizes the contents of a
> ThreadGroups-test by each Sampler. So you get the avg, mean, and
> deviation for every HttpRquest. Maybe a BarChart or something like that.
>
> 2. Maybe it could be useful (could it?) to set up a timer or something
> else to ensure that with a given number of users your throughput is met.
> That is you set up the ThreadGroup as usual. But you add something where
> you can input the throughput you want to guarantee. If you run your
> tests you will receive a failure if the throughput cannot be met or a
> value indicating how much sleep-time remains between individual requests
> per user. This one might be helpful in determining if addition to a
> given path. E.g.: I have contributed to a web-app where the user had to
> follow a given path to get a map in the end [which is the most
> time-consuming page of course]. So if I want to add some steps and some
> processing in between, could we probably meet the throughput-goal or
> not. Yet I'm not so sure about that (especially since I guess it would
> be some work to realize it). Hm, after thinking a short while about
> this, I think one would get the result as quickly by the exiting
> GraphVisualizer. But maybe my confused thoughts elicit better and more
> useful thoughts by others...
>
> Greetings,
>
> Wolfram
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: WAS: BUG 11536 - throughput calculation
Posted by Mike Stover <ms...@apache.org>.
On 14 Aug 2002 at 12:08, Wolfram Rittmeyer wrote:
>
> Mike wrote:
>
> > But, here's the problem: The server didn't actually serve up 10 requests per second. It was
> > only asked to serve up some fraction of that by JMeter - for 5 seconds between each request,
> > JMeter "gave the server a break", so to speak. I think developers would be incorrect to think
> > that the 10 requests/second calculation represents what their server could have done. You
> > don't know that. If you take away JMeter's 5 second delay, you might find the numbers get
> > worse - it depends since servers can get bogged down unexpectedly, and performance can
> > go in the toilet without warning as you increase number of hits attempted.
> >
> > Although your calculation seems to indicate the server can handle 10 requests per second,
> > you might find that in actuality, it can never reach that. In which case, I have to ask - what is
> > the value of that calculation?
> >
> > The current calculation gives a cold, hard number - this is the number of requests that the
> > server actually handled. It doesn't represent a theoretical max, or even a theoretical max given
> > the limited # of threads. It's the actual. to find your server's max throughput, you have to run
> > multiple tests, changing the number of users. As you increase the number of simultaneous
> > users, the actual throughput will generally increase, until you reach a point at which your
> > server either plateau's or begins to decline. This represents your server's maximum
> > throughput. That is my thinking. Feel free to set me straight :-)
> >
> > -Mike
> >
>
> Even this is somewhat hypothetical. At least if some pages need more
> processing-time than others. You would know which is more time-intensive
> of course, but how often which page is really called, that's a thing you
> just get to know when your page (in the case of HTTP-requests) is up and
> real users use it. They decide wether they want the page to be
> recomputed by changing some parameters and so on. If you just request
> this time-intensive page two times in your test but it get's called four
> times on average in reality, your calculated throughput will quickly get
> a number with no counterpart in reality. Therefore, to be honest, every
> throughput calculation in advance is somewhat hypothetical, isn't it.
>
> So, finally, I must admit that changing the GraphVisualizers doesn't
> make much sense ;-) Anyhow I think the discussion was fertile (or
> insightful or whatever one would say in English)...
I think there's room in the graph visualizer for a few more numbers at the bottom - like Berin's
"idealized" throughput calculation. I don't know about drawing more lines in the graph itself
though.
Entirely new visualizers are always a possibility to. A useful visualizer might be one that has
tabs across the top to let you quickly switch views between multiple visualizers.
>
> Nevertheless, as a result of that I want to suggest something else (I
> will post an enhancement request later on when I have my thoughts a bit
> more straight), but since I'd like to discuss the usefullness, I just
> include it in here:
>
> 1. Another Visualizer that visualizes the contents of a
> ThreadGroups-test by each Sampler. So you get the avg, mean, and
> deviation for every HttpRquest. Maybe a BarChart or something like that.
Check out the Aggregate Report. It gives you this data per different request. I use it to
pinpoint which requests are giving all the trouble - time wise and error-wise. It could use
enhancement, but it's quite useful as it is.
>
> 2. Maybe it could be useful (could it?) to set up a timer or something
> else to ensure that with a given number of users your throughput is met.
> That is you set up the ThreadGroup as usual. But you add something where
> you can input the throughput you want to guarantee. If you run your
> tests you will receive a failure if the throughput cannot be met or a
> value indicating how much sleep-time remains between individual requests
> per user. This one might be helpful in determining if addition to a
> given path. E.g.: I have contributed to a web-app where the user had to
> follow a given path to get a map in the end [which is the most
> time-consuming page of course]. So if I want to add some steps and some
> processing in between, could we probably meet the throughput-goal or
> not. Yet I'm not so sure about that (especially since I guess it would
> be some work to realize it). Hm, after thinking a short while about
> this, I think one would get the result as quickly by the exiting
> GraphVisualizer. But maybe my confused thoughts elicit better and more
> useful thoughts by others...
Sounds similar to thoughts I'd had about making JMeter adjust test settings to discover various
things about a site. For instance, gradually increasing the number of threads to find the
saturation point of the server. Similarly, decreasing the delays could be part of that, and
Jordi's throughput timer idea is probably most useful in that respect.
-Mike
>
> Greetings,
>
> Wolfram
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
Michael Stover
mstover1@apache.org
Yahoo IM: mstover_ya
ICQ: 152975688
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
WAS: BUG 11536 - throughput calculation
Posted by Wolfram Rittmeyer <wo...@web.de>.
Mike wrote:
> But, here's the problem: The server didn't actually serve up 10 requests per second. It was
> only asked to serve up some fraction of that by JMeter - for 5 seconds between each request,
> JMeter "gave the server a break", so to speak. I think developers would be incorrect to think
> that the 10 requests/second calculation represents what their server could have done. You
> don't know that. If you take away JMeter's 5 second delay, you might find the numbers get
> worse - it depends since servers can get bogged down unexpectedly, and performance can
> go in the toilet without warning as you increase number of hits attempted.
>
> Although your calculation seems to indicate the server can handle 10 requests per second,
> you might find that in actuality, it can never reach that. In which case, I have to ask - what is
> the value of that calculation?
>
> The current calculation gives a cold, hard number - this is the number of requests that the
> server actually handled. It doesn't represent a theoretical max, or even a theoretical max given
> the limited # of threads. It's the actual. to find your server's max throughput, you have to run
> multiple tests, changing the number of users. As you increase the number of simultaneous
> users, the actual throughput will generally increase, until you reach a point at which your
> server either plateau's or begins to decline. This represents your server's maximum
> throughput. That is my thinking. Feel free to set me straight :-)
>
> -Mike
>
Even this is somewhat hypothetical. At least if some pages need more
processing-time than others. You would know which is more time-intensive
of course, but how often which page is really called, that's a thing you
just get to know when your page (in the case of HTTP-requests) is up and
real users use it. They decide wether they want the page to be
recomputed by changing some parameters and so on. If you just request
this time-intensive page two times in your test but it get's called four
times on average in reality, your calculated throughput will quickly get
a number with no counterpart in reality. Therefore, to be honest, every
throughput calculation in advance is somewhat hypothetical, isn't it.
So, finally, I must admit that changing the GraphVisualizers doesn't
make much sense ;-) Anyhow I think the discussion was fertile (or
insightful or whatever one would say in English)...
Nevertheless, as a result of that I want to suggest something else (I
will post an enhancement request later on when I have my thoughts a bit
more straight), but since I'd like to discuss the usefullness, I just
include it in here:
1. Another Visualizer that visualizes the contents of a
ThreadGroups-test by each Sampler. So you get the avg, mean, and
deviation for every HttpRquest. Maybe a BarChart or something like that.
2. Maybe it could be useful (could it?) to set up a timer or something
else to ensure that with a given number of users your throughput is met.
That is you set up the ThreadGroup as usual. But you add something where
you can input the throughput you want to guarantee. If you run your
tests you will receive a failure if the throughput cannot be met or a
value indicating how much sleep-time remains between individual requests
per user. This one might be helpful in determining if addition to a
given path. E.g.: I have contributed to a web-app where the user had to
follow a given path to get a map in the end [which is the most
time-consuming page of course]. So if I want to add some steps and some
processing in between, could we probably meet the throughput-goal or
not. Yet I'm not so sure about that (especially since I guess it would
be some work to realize it). Hm, after thinking a short while about
this, I think one would get the result as quickly by the exiting
GraphVisualizer. But maybe my confused thoughts elicit better and more
useful thoughts by others...
Greetings,
Wolfram
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: BUG 11536 - throughput calculation
Posted by Berin Loritsch <bl...@apache.org>.
> From: Mike Stover [mailto:mstover1@apache.org]
>
> Thank you Berin, that would give a meaningful interpretation
> of throughput that cuts out factors
> internal to JMeter. I certainly wasn't aware of that formula.
>
> I'm wondering how it would relate to reality, however. Say I
> have a test with 10 simultaneous
> users, each delaying 5 seconds between each request. If I
> run the test for 1 hour, and get an
> average response time of 1 sec, the numbers will look like this:
>
> Rn = (1 hour * 10)/1 sec
> = 3600 * 10 / 1 (converted to seconds)
> = 36000, which means 36000 theoretical requests per hour,
> or 10/second
>
> But, here's the problem: The server didn't actually serve up
> 10 requests per second. It was
> only asked to serve up some fraction of that by JMeter - for
> 5 seconds between each request,
> JMeter "gave the server a break", so to speak. I think
> developers would be incorrect to think
> that the 10 requests/second calculation represents what their
> server could have done. You
> don't know that. If you take away JMeter's 5 second delay,
> you might find the numbers get
> worse - it depends since servers can get bogged down
> unexpectedly, and performance can
> go in the toilet without warning as you increase number of
> hits attempted.
That is why the calculation has to be re-evaluated when we change
critical parameters. Also keep in mind that for real throughput
requests, you cannot give the server a break. You have to remove
all delays for the throughput request.
>
> Although your calculation seems to indicate the server can
> handle 10 requests per second,
> you might find that in actuality, it can never reach that.
> In which case, I have to ask - what is
> the value of that calculation?
I use the other formula more often:
Treq = ( Rn / Ttot ) * N
That tells me how fast I need to process requests before I violate
my performance criteria.
The two used together help size your servers. However, to be
used affectively, you have to remove all the delays.
> The current calculation gives a cold, hard number - this is
> the number of requests that the
> server actually handled. It doesn't represent a theoretical
> max, or even a theoretical max given
> the limited # of threads. It's the actual. to find your
> server's max throughput, you have to run
> multiple tests, changing the number of users. As you
> increase the number of simultaneous
> users, the actual throughput will generally increase, until
> you reach a point at which your
> server either plateau's or begins to decline. This
> represents your server's maximum
> throughput. That is my thinking. Feel free to set me straight :-)
True. What you are describing is the Saturation curve that most
servers have. When testing a site, you have to mimic the expeced
load as closely as you possibly can. When you are finding the
saturation point of your server, you have to remove all delays
from the test. If you can handle more requests, start increasing
the number of clients.
However, you do need to put your server's maximum throughput against
the theorhetical maximum to see if adding yet another server to the
mix and clustering them will get you to full bandwidth.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: BUG 11536 - throughput calculation
Posted by Mike Stover <ms...@apache.org>.
Thank you Berin, that would give a meaningful interpretation of throughput that cuts out factors
internal to JMeter. I certainly wasn't aware of that formula.
I'm wondering how it would relate to reality, however. Say I have a test with 10 simultaneous
users, each delaying 5 seconds between each request. If I run the test for 1 hour, and get an
average response time of 1 sec, the numbers will look like this:
Rn = (1 hour * 10)/1 sec
= 3600 * 10 / 1 (converted to seconds)
= 36000, which means 36000 theoretical requests per hour, or 10/second
But, here's the problem: The server didn't actually serve up 10 requests per second. It was
only asked to serve up some fraction of that by JMeter - for 5 seconds between each request,
JMeter "gave the server a break", so to speak. I think developers would be incorrect to think
that the 10 requests/second calculation represents what their server could have done. You
don't know that. If you take away JMeter's 5 second delay, you might find the numbers get
worse - it depends since servers can get bogged down unexpectedly, and performance can
go in the toilet without warning as you increase number of hits attempted.
Although your calculation seems to indicate the server can handle 10 requests per second,
you might find that in actuality, it can never reach that. In which case, I have to ask - what is
the value of that calculation?
The current calculation gives a cold, hard number - this is the number of requests that the
server actually handled. It doesn't represent a theoretical max, or even a theoretical max given
the limited # of threads. It's the actual. to find your server's max throughput, you have to run
multiple tests, changing the number of users. As you increase the number of simultaneous
users, the actual throughput will generally increase, until you reach a point at which your
server either plateau's or begins to decline. This represents your server's maximum
throughput. That is my thinking. Feel free to set me straight :-)
-Mike
On 13 Aug 2002 at 10:43, Berin Loritsch wrote:
>
> Throughput calculations have to take into acount the periodicity
> of the samples (i.e. are we measuring in minutes, milliseconds, etc.),
> the number of threads used to make the requests, the average response
> time, the size of the requests, and the number of requests.
>
> These all work together to help you make the most of your measurements.
> The size of the requests and the speed of the internet connection
> speed determine your theorhetical maximum. Therefore, if you have
> a 10K page and a 100 Mb connection to your intranet, your theorhetical
> maximum is:
>
> T = Sb / Lb
>
> T = ((100Mb * 1000000) / 8 ) / (10K * 1024)
> T = 12500000 / 10240
> T = 1220.703125 Requests/Second
>
> Your actuals will never exceed this throughput. If your numbers are
> close, or hitting this, then the only way to increase throughput is
> to have higher bandwidth.
>
> Your actuals have to consider everything else:
>
> * Time period of the test
> * Average response time
> * Number of simultaneous requests (aka threads)
> * Actual number of requests
>
> Rn = ( Ttot * N ) / Treq
>
> Where:
>
> Ttot = number of ms in time period (like ms/day = 86400000)
> N = number of simultaneous requests
> Treq = time in ms of the average request
> Rn = number of requests processed in the time period
>
> Plugging real numbers in, to get 1,000,000 requests per day
> with 20 threads and average request time of 1728ms:
>
> Rn = ( 86400000 * 20 ) / 1728
> Rn = 1728000000 / 1728
> Rn = 1000000 Requests/day
>
> Plugging the same numbers in, but limiting it to one second
> we get:
>
> Rn = ( 1000 * 20 ) / 1728
> Rn = 20000 / 1728
> Rn = 11.5740740.... Requests/Second
>
> To determine how long the average request needs to be in order
> to saturate the line, we need to reverse the algorithm:
>
> Treq = ( Rn / Ttot ) * N
>
> So plugging in the values from our theorhetical maximum and
> 10 threads we get:
>
> Treq = (1220.703125 / 1000) * 20
> Treq = 1.22070325 * 20
> Treq = 24.414065 ms/Request
>
>
> So you can see that the number of threads directly affects how long
> processing can take to affect either our desired goals, or the total
> throughput.
>
> BTW, if you can do 24.414065 ms/Request sustained for a day it is
> 70,778,872.75 Requests/Day.
>
> The important measurement in throughput is really how long a request
> takes. The size only comes into play when we are figuring out how
> fast our ceiling is, or how long it should take to transfer bytes.
>
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
Michael Stover
mstover1@apache.org
Yahoo IM: mstover_ya
ICQ: 152975688
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: BUG 11536 - throughput calculation
Posted by Berin Loritsch <bl...@apache.org>.
Throughput calculations have to take into acount the periodicity
of the samples (i.e. are we measuring in minutes, milliseconds, etc.),
the number of threads used to make the requests, the average response
time, the size of the requests, and the number of requests.
These all work together to help you make the most of your measurements.
The size of the requests and the speed of the internet connection
speed determine your theorhetical maximum. Therefore, if you have
a 10K page and a 100 Mb connection to your intranet, your theorhetical
maximum is:
T = Sb / Lb
T = ((100Mb * 1000000) / 8 ) / (10K * 1024)
T = 12500000 / 10240
T = 1220.703125 Requests/Second
Your actuals will never exceed this throughput. If your numbers are
close, or hitting this, then the only way to increase throughput is
to have higher bandwidth.
Your actuals have to consider everything else:
* Time period of the test
* Average response time
* Number of simultaneous requests (aka threads)
* Actual number of requests
Rn = ( Ttot * N ) / Treq
Where:
Ttot = number of ms in time period (like ms/day = 86400000)
N = number of simultaneous requests
Treq = time in ms of the average request
Rn = number of requests processed in the time period
Plugging real numbers in, to get 1,000,000 requests per day
with 20 threads and average request time of 1728ms:
Rn = ( 86400000 * 20 ) / 1728
Rn = 1728000000 / 1728
Rn = 1000000 Requests/day
Plugging the same numbers in, but limiting it to one second
we get:
Rn = ( 1000 * 20 ) / 1728
Rn = 20000 / 1728
Rn = 11.5740740.... Requests/Second
To determine how long the average request needs to be in order
to saturate the line, we need to reverse the algorithm:
Treq = ( Rn / Ttot ) * N
So plugging in the values from our theorhetical maximum and
10 threads we get:
Treq = (1220.703125 / 1000) * 20
Treq = 1.22070325 * 20
Treq = 24.414065 ms/Request
So you can see that the number of threads directly affects how long
processing can take to affect either our desired goals, or the total
throughput.
BTW, if you can do 24.414065 ms/Request sustained for a day it is
70,778,872.75 Requests/Day.
The important measurement in throughput is really how long a request
takes. The size only comes into play when we are figuring out how
fast our ceiling is, or how long it should take to transfer bytes.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: BUG 11536 - throughput calculation
Posted by Mike Stover <ms...@apache.org>.
On 13 Aug 2002 at 15:00, Wolfram Rittmeyer wrote:
> Hi Mike, Hi Jordi,
>
> you're right. And I am nearly convinced ;-)
>
> Nevertheless I would like to compute something like a "hypothetical"
> throughput. That is, how many requests the server might have done if no
> timers were set. Of course the real throughput (if really no timers were
> set)
> would be lower (probably even much lower) than the hypothetical in most
> real-world applications. Hm, I have to put some more thought on this
> issue. And of course the time of JMeters processing itself cannot be
> filtered.
>
> So how do I want to do the hypothetical calculation?
>
> I would simply add up the time of each sample in a private attribute and
> calculate based upon this value instead of the overall elapsed time.
> Just a few changes in code. But that would be so simple that I guess, I
> missed something. Maybe this approach doesn't work. Maybe I forgot
> something. Maybe we even do not talk about the same issue. I'm looking
> forward to your comments.
But, if you carry that out, you end up with the following equation:
Throughput = # requests/time /definition of throughput
Total time = (overall average) * (# requests)
(this should be equivalent to the sum of each sample's time)
throughput now = (# requests) / (# requests) * (average time) // simple substitution
and, simplifying = 1/average time
In other words, you would get the simple inverse of the average time for each sample. If the
average was 200 ms, then the "throughput" would be 5/second, 300/minute.
And I don't think that number has much relation to a real throughput measurement.
-Mike
>
> Still I'm not sure yet wether it really yields any reasonable results.
> Could be that the throughput would really be too hypothetical.
>
> Greetings,
>
> Wolfram Rittmeyer
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
Michael Stover
mstover1@apache.org
Yahoo IM: mstover_ya
ICQ: 152975688
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: BUG 11536 - throughput calculation
Posted by Wolfram Rittmeyer <wo...@web.de>.
Hi Mike, Hi Jordi,
you're right. And I am nearly convinced ;-)
Nevertheless I would like to compute something like a "hypothetical"
throughput. That is, how many requests the server might have done if no
timers were set. Of course the real throughput (if really no timers were
set)
would be lower (probably even much lower) than the hypothetical in most
real-world applications. Hm, I have to put some more thought on this
issue. And of course the time of JMeters processing itself cannot be
filtered.
So how do I want to do the hypothetical calculation?
I would simply add up the time of each sample in a private attribute and
calculate based upon this value instead of the overall elapsed time.
Just a few changes in code. But that would be so simple that I guess, I
missed something. Maybe this approach doesn't work. Maybe I forgot
something. Maybe we even do not talk about the same issue. I'm looking
forward to your comments.
Still I'm not sure yet wether it really yields any reasonable results.
Could be that the throughput would really be too hypothetical.
Greetings,
Wolfram Rittmeyer
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: BUG 11536 - throughput calculation
Posted by Mike Stover <ms...@apache.org>.
Bytes/time could easily be accomodated, and I see no reason not to. However, there's a
larger issue, I think, having to do with how the measurement is made. Currently, the
throughput measurement measures how many requests are finished from the start of the
test run till the end. This means all the timers and JMeter's own processing are factoring
in. But, I don't see a better alternative.
If you were to factor out those consideration.. well, firstly, how exactly would you do
that? When you have 10 threads running for 1 hour, you actually have 10 * 1 hours of
"subjective" time measured. But, you want an objective number, # requests/minute of
test run. The fact that each thread runs it's own hour must not interfere with the
objectivity of this number. If anyone can suggest a procedure that will yield up a valid
number, I'm listening.
Also understand that any calculations you attempt to do on the raw data will have a
questionable relationship to reality. The current throughput represents how many
requests the server, in fact, handled per minute.
A bytes per minute number could similarly be determined, but again, it would include the
timers and JMeter's processing time.
-Mike
On 12 Aug 2002 at 18:38, Jordi Salvat i Alabart wrote:
> In my experience, requests per unit of time is a more important value
> than bytes per unit of time when load testing web applications.
>
> The former usually relates to CPU power used (there isn't much
> difference between spitting out 1K or 2K of data) -- and CPU is most
> often the bottleneck in this kind of systems.
>
> The later comes handy when the network over which you're testing is the
> bottleneck -- but that's usually more a "test accident" than a real
> "test result".
>
> Sorry I can't be clearer.
>
> Salut,
>
> Jordi.
>
> Wolfram Rittmeyer wrote:
> > Hi,
> >
> > I have looked at the code and at the output of the GraphVisualizer and
> > GraphModel and I am amazed. I never thought much about it since it
> > hasn't been of much interest up to now. But I thought it would calculate
> > the amount of bytes per time. Yet by looking at the code I realize it
> > calculates the number of request per minute. The funny (but
> > understandable) thing is that the results depend upon the timers. If you
> > add a constant timer of 500 milliseconds results will be much lower than
> > with a timer of 300 milliseconds. Is this really what we want? I doubt
> > it. At least it is not, what I would like to see, but others may
> > disagree.
> >
> > So I could imagine either to fix this so it shows the real throughput of
> > bytes per amount of time (i would prefer bytes per second) or to add a
> > further information to the GraphVisualizer.
> >
> > BTW: The first solution would kind of "solve" the BUG 11536.
> >
> > What's your opinion about it?
> >
> > Greetings,
> >
> > Wolfram Rittmeyer
> >
> > --
> > To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> > For additional commands, e-mail: <ma...@jakarta.apache.org>
> >
> >
>
>
>
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
Michael Stover
mstover1@apache.org
Yahoo IM: mstover_ya
ICQ: 152975688
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: BUG 11536 - throughput calculation
Posted by Jordi Salvat i Alabart <js...@atg.com>.
In my experience, requests per unit of time is a more important value
than bytes per unit of time when load testing web applications.
The former usually relates to CPU power used (there isn't much
difference between spitting out 1K or 2K of data) -- and CPU is most
often the bottleneck in this kind of systems.
The later comes handy when the network over which you're testing is the
bottleneck -- but that's usually more a "test accident" than a real
"test result".
Sorry I can't be clearer.
Salut,
Jordi.
Wolfram Rittmeyer wrote:
> Hi,
>
> I have looked at the code and at the output of the GraphVisualizer and
> GraphModel and I am amazed. I never thought much about it since it
> hasn't been of much interest up to now. But I thought it would calculate
> the amount of bytes per time. Yet by looking at the code I realize it
> calculates the number of request per minute. The funny (but
> understandable) thing is that the results depend upon the timers. If you
> add a constant timer of 500 milliseconds results will be much lower than
> with a timer of 300 milliseconds. Is this really what we want? I doubt
> it. At least it is not, what I would like to see, but others may
> disagree.
>
> So I could imagine either to fix this so it shows the real throughput of
> bytes per amount of time (i would prefer bytes per second) or to add a
> further information to the GraphVisualizer.
>
> BTW: The first solution would kind of "solve" the BUG 11536.
>
> What's your opinion about it?
>
> Greetings,
>
> Wolfram Rittmeyer
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: BUG 11536 - throughput calculation
Posted by Mike Stover <ms...@apache.org>.
I think my understanding of throughput is different. When I look at the throughput number, it
is telling me how many requests the server is handling per unit time (in this case, #
requests/minute). That's what I want to know. I see no way of accurately reporting that info
other than the way I do it, which includes the delay timers and all.
Bytes per time seems useless to me, unless you are particularly interested in your bandwidth
performance. Most web apps these days need to know about their server performance,
however.
-Mike
On 12 Aug 2002 at 16:55, Wolfram Rittmeyer wrote:
> Hi,
>
> I have looked at the code and at the output of the GraphVisualizer and
> GraphModel and I am amazed. I never thought much about it since it
> hasn't been of much interest up to now. But I thought it would calculate
> the amount of bytes per time. Yet by looking at the code I realize it
> calculates the number of request per minute. The funny (but
> understandable) thing is that the results depend upon the timers. If you
> add a constant timer of 500 milliseconds results will be much lower than
> with a timer of 300 milliseconds. Is this really what we want? I doubt
> it. At least it is not, what I would like to see, but others may
> disagree.
>
> So I could imagine either to fix this so it shows the real throughput of
> bytes per amount of time (i would prefer bytes per second) or to add a
> further information to the GraphVisualizer.
>
> BTW: The first solution would kind of "solve" the BUG 11536.
>
> What's your opinion about it?
>
> Greetings,
>
> Wolfram Rittmeyer
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
Michael Stover
mstover1@apache.org
Yahoo IM: mstover_ya
ICQ: 152975688
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>