You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@jmeter.apache.org by Alexandru Rotaru <al...@altom.ro> on 2009/10/19 11:34:47 UTC
JMeter ResponseTime >> Browser ResponseTime
Hi all,
I'm running some simple tests using Jmeter 2.3.4:
1. connect to the website home page
2. go to the login page
3. log into the website
My test plan looks like this:
+ HTTP Cookie Manager
+ HTTP Cache Manager
+ HTTP Request Defaults ("Retrieve all embedded resources from html
files" is ticked)
+ Thread Group (repeat 1 thread for 10 times)
- HTTP Request HomePage-Get
- HTTP Request LoginPage-Get
- HTTp Request LoginPage-Post
+View Results Tree
The response times for the Get requests are:
HomePage - Get -> First run ~ 16.5 seconds, Next 9 runs ~ 9.3 seconds
LoginPage - Get -> First run ~ 14.5 seconds, Next 9 runs ~ 9.3
seconds
If I run the same steps in a browser, I get the following results:
HomePage - Get -> First time (nothing is cached) <10 seconds, Next
runs < 3 seconds
LoginPage - Get -> less than 2 seconds
I ran the same tests using the HttpClient sampler, and got similar
result.
I ran the test both in GUI and non-GUI mode, and the results were
similar.
I was expecting to see a time difference between the browser and
Jmeter, but the other way around (Jmeter time < Browser time) as the
browser does also Java Script interpretation, page rendering ...
I've been looking through the mailing archive, and found out about
YSlow and TamperData, and the Browser times I manually recorder
resemble the ones from these apps.
Is there a way to measure the server response times using JMeter, and
getting results close to real usage?
Thanks,
Alex
Re: JMeter ResponseTime >> Browser ResponseTime
Posted by Deepak Shetty <sh...@gmail.com>.
hi
The thing you should check from YSlow/tamper data is if you added up
individual times for each resource downloaded , it should match the Jmeter
time.
>I'm thinking of a requirement like: "all pages should load within 7 seconds
for 100 concurrent users."
Speaking practically , such a requirement is hard to verify. There are just
too many factors. What machine is the user using, which browser version,
what cache setting, where is the user located. To give you an example in my
current project (before we gzip responses and minified CSS/JS) concurrent
usage gave a 4 second response time if the user was based in california and
a 20 second response time if the user was based in New York.
This is my rule of thumb , I rarely have embedded resources checked for my
tests.
That gives me the pure server speeds. This is the minimum time the browser
will take(if this is close to for e.g. in your case 7 seconds then it most
likely will fail your requirement). Images /CSS/JS downloads are hard to
predict , but if you know that your app rates pretty highly on YSLow (or
static files served from a webserver, minified, gzipped, minimal number of
requests, with correctly cached headers), you can more or less take these
out of the equation and add a fixed time to your request + add some network
latency(and verify from multiple physical locations) + some browser render
time .
If you really, really still want to verify it , then run a constant load
using Jmeter (but dont use response times from here). Concurrently run
something like selenium/QTP which drive the browser and take the response
times from there.
regards
deepak
On Mon, Oct 19, 2009 at 11:37 PM, Alexandru Rotaru <al...@altom.ro>wrote:
> Hi Deepak,
>
> If I compare the times from YSlow and the ones from JMeter, the differences
> are still high (YSlow < JMeter).
> I understand that the browser uses more threads to load a page, and thus
> the load time is shorter.
> Having this in mind, is there a way to benchmark the server response time
> for a specific number of users using JMeter?
>
> I'm thinking of a requirement like: "all pages should load within 7 seconds
> for 100 concurrent users."
> If it loads within 7 seconds in JMeter, then it should also load in the
> browser. But if it doesn't in JMeter, does it mean that there's a
> performance problem?
>
> Thanks,
> Alex
>
>
> On Oct 19, 2009, at 11:47 PM, Deepak Shetty wrote:
>
> And just FYI the number of parallel connections differ per browser
>>
>> http://stackoverflow.com/questions/561046/how-many-concurrent-ajax-xmlhttprequest-requests-are-allowed-in-popular-browser
>>
>>
>> regards
>> deepak
>>
>> On Mon, Oct 19, 2009 at 1:45 PM, Deepak Shetty <sh...@gmail.com> wrote:
>>
>> Hi
>>> as mentioned by another poster, the browser can make requests in parallel
>>> (as far as I know this is max of two threads per domain).
>>> Hence you will see a difference with Jmeter which will download all
>>> requests serially. Also Yslow/tamper data do not measure rendering
>>> times/javascript execution either (I may be wrong). And finally even with
>>> the cache on , a request is still made (you'll just get a 304 response)
>>> hence requests being made in parallel(from the browser) will be faster.
>>> If
>>> you use tamper data , you can see this when you graph the results.
>>>
>>>
>>> regards
>>> deepak
>>>
>>>
>>>
>>>
>>> On Mon, Oct 19, 2009 at 2:34 AM, Alexandru Rotaru <alex.rotaru@altom.ro
>>> >wrote:
>>>
>>> Hi all,
>>>>
>>>> I'm running some simple tests using Jmeter 2.3.4:
>>>> 1. connect to the website home page
>>>> 2. go to the login page
>>>> 3. log into the website
>>>>
>>>> My test plan looks like this:
>>>> + HTTP Cookie Manager
>>>> + HTTP Cache Manager
>>>> + HTTP Request Defaults ("Retrieve all embedded resources from html
>>>> files" is ticked)
>>>> + Thread Group (repeat 1 thread for 10 times)
>>>> - HTTP Request HomePage-Get
>>>> - HTTP Request LoginPage-Get
>>>> - HTTp Request LoginPage-Post
>>>> +View Results Tree
>>>>
>>>> The response times for the Get requests are:
>>>>
>>>> HomePage - Get -> First run ~ 16.5 seconds, Next 9 runs ~ 9.3
>>>> seconds
>>>> LoginPage - Get -> First run ~ 14.5 seconds, Next 9 runs ~ 9.3
>>>> seconds
>>>>
>>>> If I run the same steps in a browser, I get the following results:
>>>> HomePage - Get -> First time (nothing is cached) <10 seconds, Next
>>>> runs < 3 seconds
>>>> LoginPage - Get -> less than 2 seconds
>>>>
>>>> I ran the same tests using the HttpClient sampler, and got similar
>>>> result.
>>>> I ran the test both in GUI and non-GUI mode, and the results were
>>>> similar.
>>>>
>>>> I was expecting to see a time difference between the browser and Jmeter,
>>>> but the other way around (Jmeter time < Browser time) as the browser
>>>> does
>>>> also Java Script interpretation, page rendering ...
>>>>
>>>> I've been looking through the mailing archive, and found out about YSlow
>>>> and TamperData, and the Browser times I manually recorder resemble the
>>>> ones
>>>> from these apps.
>>>>
>>>> Is there a way to measure the server response times using JMeter, and
>>>> getting results close to real usage?
>>>>
>>>> Thanks,
>>>> Alex
>>>>
>>>>
>>>>
>>>
>
Re: JMeter ResponseTime >> Browser ResponseTime
Posted by Alexandru Rotaru <al...@altom.ro>.
Hi Deepak,
If I compare the times from YSlow and the ones from JMeter, the
differences are still high (YSlow < JMeter).
I understand that the browser uses more threads to load a page, and
thus the load time is shorter.
Having this in mind, is there a way to benchmark the server response
time for a specific number of users using JMeter?
I'm thinking of a requirement like: "all pages should load within 7
seconds for 100 concurrent users."
If it loads within 7 seconds in JMeter, then it should also load in
the browser. But if it doesn't in JMeter, does it mean that there's a
performance problem?
Thanks,
Alex
On Oct 19, 2009, at 11:47 PM, Deepak Shetty wrote:
> And just FYI the number of parallel connections differ per browser
> http://stackoverflow.com/questions/561046/how-many-concurrent-ajax-xmlhttprequest-requests-are-allowed-in-popular-browser
>
>
> regards
> deepak
>
> On Mon, Oct 19, 2009 at 1:45 PM, Deepak Shetty <sh...@gmail.com>
> wrote:
>
>> Hi
>> as mentioned by another poster, the browser can make requests in
>> parallel
>> (as far as I know this is max of two threads per domain).
>> Hence you will see a difference with Jmeter which will download all
>> requests serially. Also Yslow/tamper data do not measure rendering
>> times/javascript execution either (I may be wrong). And finally
>> even with
>> the cache on , a request is still made (you'll just get a 304
>> response)
>> hence requests being made in parallel(from the browser) will be
>> faster. If
>> you use tamper data , you can see this when you graph the results.
>>
>>
>> regards
>> deepak
>>
>>
>>
>>
>> On Mon, Oct 19, 2009 at 2:34 AM, Alexandru Rotaru <alex.rotaru@altom.ro
>> >wrote:
>>
>>> Hi all,
>>>
>>> I'm running some simple tests using Jmeter 2.3.4:
>>> 1. connect to the website home page
>>> 2. go to the login page
>>> 3. log into the website
>>>
>>> My test plan looks like this:
>>> + HTTP Cookie Manager
>>> + HTTP Cache Manager
>>> + HTTP Request Defaults ("Retrieve all embedded resources from html
>>> files" is ticked)
>>> + Thread Group (repeat 1 thread for 10 times)
>>> - HTTP Request HomePage-Get
>>> - HTTP Request LoginPage-Get
>>> - HTTp Request LoginPage-Post
>>> +View Results Tree
>>>
>>> The response times for the Get requests are:
>>>
>>> HomePage - Get -> First run ~ 16.5 seconds, Next 9 runs ~ 9.3
>>> seconds
>>> LoginPage - Get -> First run ~ 14.5 seconds, Next 9 runs ~
>>> 9.3
>>> seconds
>>>
>>> If I run the same steps in a browser, I get the following results:
>>> HomePage - Get -> First time (nothing is cached) <10
>>> seconds, Next
>>> runs < 3 seconds
>>> LoginPage - Get -> less than 2 seconds
>>>
>>> I ran the same tests using the HttpClient sampler, and got similar
>>> result.
>>> I ran the test both in GUI and non-GUI mode, and the results were
>>> similar.
>>>
>>> I was expecting to see a time difference between the browser and
>>> Jmeter,
>>> but the other way around (Jmeter time < Browser time) as the
>>> browser does
>>> also Java Script interpretation, page rendering ...
>>>
>>> I've been looking through the mailing archive, and found out about
>>> YSlow
>>> and TamperData, and the Browser times I manually recorder resemble
>>> the ones
>>> from these apps.
>>>
>>> Is there a way to measure the server response times using JMeter,
>>> and
>>> getting results close to real usage?
>>>
>>> Thanks,
>>> Alex
>>>
>>>
>>
Re: JMeter ResponseTime >> Browser ResponseTime
Posted by Deepak Shetty <sh...@gmail.com>.
And just FYI the number of parallel connections differ per browser
http://stackoverflow.com/questions/561046/how-many-concurrent-ajax-xmlhttprequest-requests-are-allowed-in-popular-browser
regards
deepak
On Mon, Oct 19, 2009 at 1:45 PM, Deepak Shetty <sh...@gmail.com> wrote:
> Hi
> as mentioned by another poster, the browser can make requests in parallel
> (as far as I know this is max of two threads per domain).
> Hence you will see a difference with Jmeter which will download all
> requests serially. Also Yslow/tamper data do not measure rendering
> times/javascript execution either (I may be wrong). And finally even with
> the cache on , a request is still made (you'll just get a 304 response)
> hence requests being made in parallel(from the browser) will be faster. If
> you use tamper data , you can see this when you graph the results.
>
>
> regards
> deepak
>
>
>
>
> On Mon, Oct 19, 2009 at 2:34 AM, Alexandru Rotaru <al...@altom.ro>wrote:
>
>> Hi all,
>>
>> I'm running some simple tests using Jmeter 2.3.4:
>> 1. connect to the website home page
>> 2. go to the login page
>> 3. log into the website
>>
>> My test plan looks like this:
>> + HTTP Cookie Manager
>> + HTTP Cache Manager
>> + HTTP Request Defaults ("Retrieve all embedded resources from html
>> files" is ticked)
>> + Thread Group (repeat 1 thread for 10 times)
>> - HTTP Request HomePage-Get
>> - HTTP Request LoginPage-Get
>> - HTTp Request LoginPage-Post
>> +View Results Tree
>>
>> The response times for the Get requests are:
>>
>> HomePage - Get -> First run ~ 16.5 seconds, Next 9 runs ~ 9.3
>> seconds
>> LoginPage - Get -> First run ~ 14.5 seconds, Next 9 runs ~ 9.3
>> seconds
>>
>> If I run the same steps in a browser, I get the following results:
>> HomePage - Get -> First time (nothing is cached) <10 seconds, Next
>> runs < 3 seconds
>> LoginPage - Get -> less than 2 seconds
>>
>> I ran the same tests using the HttpClient sampler, and got similar result.
>> I ran the test both in GUI and non-GUI mode, and the results were similar.
>>
>> I was expecting to see a time difference between the browser and Jmeter,
>> but the other way around (Jmeter time < Browser time) as the browser does
>> also Java Script interpretation, page rendering ...
>>
>> I've been looking through the mailing archive, and found out about YSlow
>> and TamperData, and the Browser times I manually recorder resemble the ones
>> from these apps.
>>
>> Is there a way to measure the server response times using JMeter, and
>> getting results close to real usage?
>>
>> Thanks,
>> Alex
>>
>>
>
Re: JMeter ResponseTime >> Browser ResponseTime
Posted by Deepak Shetty <sh...@gmail.com>.
Hi
as mentioned by another poster, the browser can make requests in parallel
(as far as I know this is max of two threads per domain).
Hence you will see a difference with Jmeter which will download all requests
serially. Also Yslow/tamper data do not measure rendering times/javascript
execution either (I may be wrong). And finally even with the cache on , a
request is still made (you'll just get a 304 response) hence requests being
made in parallel(from the browser) will be faster. If you use tamper data ,
you can see this when you graph the results.
regards
deepak
On Mon, Oct 19, 2009 at 2:34 AM, Alexandru Rotaru <al...@altom.ro>wrote:
> Hi all,
>
> I'm running some simple tests using Jmeter 2.3.4:
> 1. connect to the website home page
> 2. go to the login page
> 3. log into the website
>
> My test plan looks like this:
> + HTTP Cookie Manager
> + HTTP Cache Manager
> + HTTP Request Defaults ("Retrieve all embedded resources from html files"
> is ticked)
> + Thread Group (repeat 1 thread for 10 times)
> - HTTP Request HomePage-Get
> - HTTP Request LoginPage-Get
> - HTTp Request LoginPage-Post
> +View Results Tree
>
> The response times for the Get requests are:
>
> HomePage - Get -> First run ~ 16.5 seconds, Next 9 runs ~ 9.3
> seconds
> LoginPage - Get -> First run ~ 14.5 seconds, Next 9 runs ~ 9.3
> seconds
>
> If I run the same steps in a browser, I get the following results:
> HomePage - Get -> First time (nothing is cached) <10 seconds, Next
> runs < 3 seconds
> LoginPage - Get -> less than 2 seconds
>
> I ran the same tests using the HttpClient sampler, and got similar result.
> I ran the test both in GUI and non-GUI mode, and the results were similar.
>
> I was expecting to see a time difference between the browser and Jmeter,
> but the other way around (Jmeter time < Browser time) as the browser does
> also Java Script interpretation, page rendering ...
>
> I've been looking through the mailing archive, and found out about YSlow
> and TamperData, and the Browser times I manually recorder resemble the ones
> from these apps.
>
> Is there a way to measure the server response times using JMeter, and
> getting results close to real usage?
>
> Thanks,
> Alex
>
>
Re: JMeter ResponseTime >> Browser ResponseTime
Posted by Alexandru Rotaru <al...@altom.ro>.
Hi Carl,
Thanks for your reply.
I wasn't using BrowserMob. I ran the JMeter test from one single
machine. Anyway my test was done only for one single user, so I guess
it was VU.
Thanks,
Alex
On Oct 19, 2009, at 3:12 PM, Carl Shaulis wrote:
> Good morning,
>
> When using BrowserMob did your test use Real Browser users (RBU) or
> Virtual
> Users (VU)?
>
> If they were virtual users I would expect the numbers to be
> similar. If
> they were RBU I would expect the results you observed, since a real
> browser
> uses multiple threads (I think 6 or 7 threads) and a VU uses a
> single thread
> to execute requests.
>
> I greatly appreciate your post because we are about to do the same
> experiment.
>
> Carl
>
>
> On 10/19/09 4:34 AM, "Alexandru Rotaru" <al...@altom.ro> wrote:
>
>> Hi all,
>>
>> I'm running some simple tests using Jmeter 2.3.4:
>> 1. connect to the website home page
>> 2. go to the login page
>> 3. log into the website
>>
>> My test plan looks like this:
>> + HTTP Cookie Manager
>> + HTTP Cache Manager
>> + HTTP Request Defaults ("Retrieve all embedded resources from html
>> files" is ticked)
>> + Thread Group (repeat 1 thread for 10 times)
>> - HTTP Request HomePage-Get
>> - HTTP Request LoginPage-Get
>> - HTTp Request LoginPage-Post
>> +View Results Tree
>>
>> The response times for the Get requests are:
>>
>> HomePage - Get -> First run ~ 16.5 seconds, Next 9 runs ~ 9.3 seconds
>> LoginPage - Get -> First run ~ 14.5 seconds, Next 9 runs ~ 9.3
>> seconds
>>
>> If I run the same steps in a browser, I get the following results:
>> HomePage - Get -> First time (nothing is cached) <10 seconds, Next
>> runs < 3 seconds
>> LoginPage - Get -> less than 2 seconds
>>
>> I ran the same tests using the HttpClient sampler, and got similar
>> result.
>> I ran the test both in GUI and non-GUI mode, and the results were
>> similar.
>>
>> I was expecting to see a time difference between the browser and
>> Jmeter, but the other way around (Jmeter time < Browser time) as the
>> browser does also Java Script interpretation, page rendering ...
>>
>> I've been looking through the mailing archive, and found out about
>> YSlow and TamperData, and the Browser times I manually recorder
>> resemble the ones from these apps.
>>
>> Is there a way to measure the server response times using JMeter, and
>> getting results close to real usage?
>>
>> Thanks,
>> Alex
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
Re: JMeter ResponseTime >> Browser ResponseTime
Posted by Carl Shaulis <cs...@homeaway.com>.
Good morning,
When using BrowserMob did your test use Real Browser users (RBU) or Virtual
Users (VU)?
If they were virtual users I would expect the numbers to be similar. If
they were RBU I would expect the results you observed, since a real browser
uses multiple threads (I think 6 or 7 threads) and a VU uses a single thread
to execute requests.
I greatly appreciate your post because we are about to do the same
experiment.
Carl
On 10/19/09 4:34 AM, "Alexandru Rotaru" <al...@altom.ro> wrote:
> Hi all,
>
> I'm running some simple tests using Jmeter 2.3.4:
> 1. connect to the website home page
> 2. go to the login page
> 3. log into the website
>
> My test plan looks like this:
> + HTTP Cookie Manager
> + HTTP Cache Manager
> + HTTP Request Defaults ("Retrieve all embedded resources from html
> files" is ticked)
> + Thread Group (repeat 1 thread for 10 times)
> - HTTP Request HomePage-Get
> - HTTP Request LoginPage-Get
> - HTTp Request LoginPage-Post
> +View Results Tree
>
> The response times for the Get requests are:
>
> HomePage - Get -> First run ~ 16.5 seconds, Next 9 runs ~ 9.3 seconds
> LoginPage - Get -> First run ~ 14.5 seconds, Next 9 runs ~ 9.3
> seconds
>
> If I run the same steps in a browser, I get the following results:
> HomePage - Get -> First time (nothing is cached) <10 seconds, Next
> runs < 3 seconds
> LoginPage - Get -> less than 2 seconds
>
> I ran the same tests using the HttpClient sampler, and got similar
> result.
> I ran the test both in GUI and non-GUI mode, and the results were
> similar.
>
> I was expecting to see a time difference between the browser and
> Jmeter, but the other way around (Jmeter time < Browser time) as the
> browser does also Java Script interpretation, page rendering ...
>
> I've been looking through the mailing archive, and found out about
> YSlow and TamperData, and the Browser times I manually recorder
> resemble the ones from these apps.
>
> Is there a way to measure the server response times using JMeter, and
> getting results close to real usage?
>
> Thanks,
> Alex
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org