You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@wicket.apache.org by Martijn Dashorst <ma...@gmail.com> on 2011/06/16 15:56:07 UTC
Performance statistics for Wicket 1.5-SNAPSHOT
The application used in
http://www.jtict.com/blog/rails-wicket-grails-play-lift-jsp/ has been
controversial as it pits all popular frameworks against one another.
I've tried to replicate the results on my box with AB, and did some
modifications to the code:
- removed the image from the markup: browsers will hit the
non-existent image causing long load times
- did the categories list in a similar way that the Tapestry
implementation did: using a StringBuilder (saves ~300ms for a single
user with 5000 items)
- randomized the results for each query to the products() method such
that caching the results is not an option
I couldn't replicate the tapestry results with these settings (instead
of being almost constant time, it now is linear but 1.5x faster than
wicket). I'm not sure if the author actually looked at the results of
the tapestry tests, but it seems to me that something went wrong.
The code has been altered with the above changes, including changing
the ListView into a RepeatingView, and rendering the categories in a
StringBuilder. Here are the results of an ab test on my laptop for 200
products and 32 concurrent users:
dashorst$ ab -c 32 -n 10240 http://localhost:8080/wicketapp/products3/200
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 1024 requests
Completed 2048 requests
Completed 3072 requests
Completed 4096 requests
Completed 5120 requests
Completed 6144 requests
Completed 7168 requests
Completed 8192 requests
Completed 9216 requests
Completed 10240 requests
Finished 10240 requests
Server Software: Jetty(6.1.16)
Server Hostname: localhost
Server Port: 8080
Document Path: /wicketapp/products3/200
Document Length: 34689 bytes
Concurrency Level: 32
Time taken for tests: 57.509 seconds
Complete requests: 10240
Failed requests: 9670
(Connect: 0, Receive: 0, Length: 9670, Exceptions: 0)
Write errors: 0
Total transferred: 354991304 bytes
HTML transferred: 352820424 bytes
Requests per second: 178.06 [#/sec] (mean)
Time per request: 179.714 [ms] (mean)
Time per request: 5.616 [ms] (mean, across all concurrent requests)
Transfer rate: 6028.16 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 1.6 0 17
Processing: 11 179 123.2 163 1000
Waiting: 11 171 123.1 154 999
Total: 12 179 123.3 164 1000
Percentage of the requests served within a certain time (ms)
50% 164
66% 218
75% 248
80% 268
90% 337
95% 418
98% 491
99% 552
100% 1000 (longest request)
The same test but then using 16 users:
dashorst$ ab -c 16 -n 10240 http://localhost:8080/wicketapp/products3/200
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 1024 requests
Completed 2048 requests
Completed 3072 requests
Completed 4096 requests
Completed 5120 requests
Completed 6144 requests
Completed 7168 requests
Completed 8192 requests
Completed 9216 requests
Completed 10240 requests
Finished 10240 requests
Server Software: Jetty(6.1.16)
Server Hostname: localhost
Server Port: 8080
Document Path: /wicketapp/products3/200
Document Length: 34290 bytes
Concurrency Level: 16
Time taken for tests: 54.005 seconds
Complete requests: 10240
Failed requests: 6161
(Connect: 0, Receive: 0, Length: 6161, Exceptions: 0)
Write errors: 0
Total transferred: 355066608 bytes
HTML transferred: 352895728 bytes
Requests per second: 189.61 [#/sec] (mean)
Time per request: 84.383 [ms] (mean)
Time per request: 5.274 [ms] (mean, across all concurrent requests)
Transfer rate: 6420.58 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.0 0 16
Processing: 11 84 51.2 74 397
Waiting: 8 77 49.4 66 397
Total: 11 84 51.2 75 398
Percentage of the requests served within a certain time (ms)
50% 75
66% 97
75% 113
80% 124
90% 154
95% 182
98% 217
99% 239
100% 398 (longest request)
Now with 8 users:
dashorst$ ab -c 8 -n 10240 http://localhost:8080/wicketapp/products3/200
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 1024 requests
Completed 2048 requests
Completed 3072 requests
Completed 4096 requests
Completed 5120 requests
Completed 6144 requests
Completed 7168 requests
Completed 8192 requests
Completed 9216 requests
Completed 10240 requests
Finished 10240 requests
Server Software: Jetty(6.1.16)
Server Hostname: localhost
Server Port: 8080
Document Path: /wicketapp/products3/200
Document Length: 34290 bytes
Concurrency Level: 8
Time taken for tests: 48.672 seconds
Complete requests: 10240
Failed requests: 6308
(Connect: 0, Receive: 0, Length: 6308, Exceptions: 0)
Write errors: 0
Total transferred: 355046233 bytes
HTML transferred: 352875353 bytes
Requests per second: 210.39 [#/sec] (mean)
Time per request: 38.025 [ms] (mean)
Time per request: 4.753 [ms] (mean, across all concurrent requests)
Transfer rate: 7123.72 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.9 0 10
Processing: 11 38 19.9 33 203
Waiting: 9 35 18.6 30 203
Total: 11 38 20.0 33 204
Percentage of the requests served within a certain time (ms)
50% 33
66% 41
75% 47
80% 53
90% 65
95% 77
98% 93
99% 105
100% 204 (longest request)
And with 4 users:
dashorst$ ab -c 4 -n 10240 http://localhost:8080/wicketapp/products3/200
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 1024 requests
Completed 2048 requests
Completed 3072 requests
Completed 4096 requests
Completed 5120 requests
Completed 6144 requests
Completed 7168 requests
Completed 8192 requests
Completed 9216 requests
Completed 10240 requests
Finished 10240 requests
Server Software: Jetty(6.1.16)
Server Hostname: localhost
Server Port: 8080
Document Path: /wicketapp/products3/200
Document Length: 34290 bytes
Concurrency Level: 4
Time taken for tests: 45.238 seconds
Complete requests: 10240
Failed requests: 6134
(Connect: 0, Receive: 0, Length: 6134, Exceptions: 0)
Write errors: 0
Total transferred: 355063755 bytes
HTML transferred: 352892875 bytes
Requests per second: 226.36 [#/sec] (mean)
Time per request: 17.671 [ms] (mean)
Time per request: 4.418 [ms] (mean, across all concurrent requests)
Transfer rate: 7664.91 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.2 0 7
Processing: 11 17 2.4 18 43
Waiting: 10 17 2.5 17 41
Total: 11 18 2.4 18 43
Percentage of the requests served within a certain time (ms)
50% 18
66% 18
75% 19
80% 19
90% 20
95% 21
98% 23
99% 25
100% 43 (longest request)
And finally with 2 users:
dashorst$ ab -c 2 -n 10240 http://localhost:8080/wicketapp/products3/200
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 1024 requests
Completed 2048 requests
Completed 3072 requests
Completed 4096 requests
Completed 5120 requests
Completed 6144 requests
Completed 7168 requests
Completed 8192 requests
Completed 9216 requests
Completed 10240 requests
Finished 10240 requests
Server Software: Jetty(6.1.16)
Server Hostname: localhost
Server Port: 8080
Document Path: /wicketapp/products3/200
Document Length: 34691 bytes
Concurrency Level: 2
Time taken for tests: 55.005 seconds
Complete requests: 10240
Failed requests: 7532
(Connect: 0, Receive: 0, Length: 7532, Exceptions: 0)
Write errors: 0
Total transferred: 355060137 bytes
HTML transferred: 352889257 bytes
Requests per second: 186.16 [#/sec] (mean)
Time per request: 10.743 [ms] (mean)
Time per request: 5.372 [ms] (mean, across all concurrent requests)
Transfer rate: 6303.74 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 1
Processing: 6 11 2.8 11 25
Waiting: 6 10 2.8 10 22
Total: 7 11 2.8 11 25
Percentage of the requests served within a certain time (ms)
50% 11
66% 12
75% 13
80% 14
90% 15
95% 15
98% 16
99% 16
100% 25 (longest request)
Looking at the results I think it is safe to say that with 16 users we
hit the limit of my core i7, and with 4 users we hit the sweet spot
(which is probably correct since I have 2 cores with each 2
hyperthreads).
I expected 4 users to perform similar to 2 users, but it appears to be
slower. I'll run it using yourkit to see if there's a blocker
somewhere.
In my overall experience, 1.5-SNAPSHOT performs comparable to 1.4.
Notes from this test:
- we report wrong length in several cases
- exceptions occur when a request is broken off
- performance seems on par with 1.4
Martijn
Re: Performance statistics for Wicket 1.5-SNAPSHOT
Posted by Martin Grigorov <mg...@apache.org>.
You can use wicket-threadtest
On Thu, Jun 16, 2011 at 9:42 PM, Bruno Borges <br...@gmail.com> wrote:
> Martijn, would you mind to share the benchmark suite you have?
>
> I'd like to test a few changes.
>
>
> *Bruno Borges*
> www.brunoborges.com.br
> +55 21 76727099
>
>
>
> On Thu, Jun 16, 2011 at 1:00 PM, Igor Vaynberg <ig...@gmail.com>wrote:
>
>> lol. i was waiting for someone to say that!
>>
>> -igor
>>
>> On Thu, Jun 16, 2011 at 8:56 AM, James Carman
>> <ja...@carmanconsulting.com> wrote:
>> > Perhaps we should re-write it using wicket-velocity! ;)
>> >
>> > On Thu, Jun 16, 2011 at 11:53 AM, Igor Vaynberg <ig...@gmail.com>
>> wrote:
>> >> his test is hand-tailored to fail any true component oriented
>> >> framework. 5000 items per page? really? show me a single website that
>> >> does that. even facebook doesnt show that many comments on the wall
>> >> without making you page. but, after looking at other "i hate wicket"
>> >> posts on his blog i am not surprised with his testing approach :)
>> >>
>> >> if this situation came up in the real life any sane developer would
>> >> wrap the entire product list into a single component that would stream
>> >> html directly using string manipulation. and if you do that then
>> >> wicket will be on par - if not faster - then the other ones. because
>> >> at that point you are measuring the same thing - how fast the jvm
>> >> loops.
>> >>
>> >> -igor
>> >>
>> >>
>> >> On Thu, Jun 16, 2011 at 6:56 AM, Martijn Dashorst
>> >> <ma...@gmail.com> wrote:
>> >>> The application used in
>> >>> http://www.jtict.com/blog/rails-wicket-grails-play-lift-jsp/ has been
>> >>> controversial as it pits all popular frameworks against one another.
>> >>> I've tried to replicate the results on my box with AB, and did some
>> >>> modifications to the code:
>> >>> - removed the image from the markup: browsers will hit the
>> >>> non-existent image causing long load times
>> >>> - did the categories list in a similar way that the Tapestry
>> >>> implementation did: using a StringBuilder (saves ~300ms for a single
>> >>> user with 5000 items)
>> >>> - randomized the results for each query to the products() method such
>> >>> that caching the results is not an option
>> >>>
>> >>> I couldn't replicate the tapestry results with these settings (instead
>> >>> of being almost constant time, it now is linear but 1.5x faster than
>> >>> wicket). I'm not sure if the author actually looked at the results of
>> >>> the tapestry tests, but it seems to me that something went wrong.
>> >>>
>> >>> The code has been altered with the above changes, including changing
>> >>> the ListView into a RepeatingView, and rendering the categories in a
>> >>> StringBuilder. Here are the results of an ab test on my laptop for 200
>> >>> products and 32 concurrent users:
>> >>>
>> >>> dashorst$ ab -c 32 -n 10240
>> http://localhost:8080/wicketapp/products3/200
>> >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
>> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd,
>> http://www.zeustech.net/
>> >>> Licensed to The Apache Software Foundation, http://www.apache.org/
>> >>>
>> >>> Benchmarking localhost (be patient)
>> >>> Completed 1024 requests
>> >>> Completed 2048 requests
>> >>> Completed 3072 requests
>> >>> Completed 4096 requests
>> >>> Completed 5120 requests
>> >>> Completed 6144 requests
>> >>> Completed 7168 requests
>> >>> Completed 8192 requests
>> >>> Completed 9216 requests
>> >>> Completed 10240 requests
>> >>> Finished 10240 requests
>> >>>
>> >>>
>> >>> Server Software: Jetty(6.1.16)
>> >>> Server Hostname: localhost
>> >>> Server Port: 8080
>> >>>
>> >>> Document Path: /wicketapp/products3/200
>> >>> Document Length: 34689 bytes
>> >>>
>> >>> Concurrency Level: 32
>> >>> Time taken for tests: 57.509 seconds
>> >>> Complete requests: 10240
>> >>> Failed requests: 9670
>> >>> (Connect: 0, Receive: 0, Length: 9670, Exceptions: 0)
>> >>> Write errors: 0
>> >>> Total transferred: 354991304 bytes
>> >>> HTML transferred: 352820424 bytes
>> >>> Requests per second: 178.06 [#/sec] (mean)
>> >>> Time per request: 179.714 [ms] (mean)
>> >>> Time per request: 5.616 [ms] (mean, across all concurrent
>> requests)
>> >>> Transfer rate: 6028.16 [Kbytes/sec] received
>> >>>
>> >>> Connection Times (ms)
>> >>> min mean[+/-sd] median max
>> >>> Connect: 0 1 1.6 0 17
>> >>> Processing: 11 179 123.2 163 1000
>> >>> Waiting: 11 171 123.1 154 999
>> >>> Total: 12 179 123.3 164 1000
>> >>>
>> >>> Percentage of the requests served within a certain time (ms)
>> >>> 50% 164
>> >>> 66% 218
>> >>> 75% 248
>> >>> 80% 268
>> >>> 90% 337
>> >>> 95% 418
>> >>> 98% 491
>> >>> 99% 552
>> >>> 100% 1000 (longest request)
>> >>>
>> >>> The same test but then using 16 users:
>> >>>
>> >>> dashorst$ ab -c 16 -n 10240
>> http://localhost:8080/wicketapp/products3/200
>> >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
>> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd,
>> http://www.zeustech.net/
>> >>> Licensed to The Apache Software Foundation, http://www.apache.org/
>> >>>
>> >>> Benchmarking localhost (be patient)
>> >>> Completed 1024 requests
>> >>> Completed 2048 requests
>> >>> Completed 3072 requests
>> >>> Completed 4096 requests
>> >>> Completed 5120 requests
>> >>> Completed 6144 requests
>> >>> Completed 7168 requests
>> >>> Completed 8192 requests
>> >>> Completed 9216 requests
>> >>> Completed 10240 requests
>> >>> Finished 10240 requests
>> >>>
>> >>>
>> >>> Server Software: Jetty(6.1.16)
>> >>> Server Hostname: localhost
>> >>> Server Port: 8080
>> >>>
>> >>> Document Path: /wicketapp/products3/200
>> >>> Document Length: 34290 bytes
>> >>>
>> >>> Concurrency Level: 16
>> >>> Time taken for tests: 54.005 seconds
>> >>> Complete requests: 10240
>> >>> Failed requests: 6161
>> >>> (Connect: 0, Receive: 0, Length: 6161, Exceptions: 0)
>> >>> Write errors: 0
>> >>> Total transferred: 355066608 bytes
>> >>> HTML transferred: 352895728 bytes
>> >>> Requests per second: 189.61 [#/sec] (mean)
>> >>> Time per request: 84.383 [ms] (mean)
>> >>> Time per request: 5.274 [ms] (mean, across all concurrent
>> requests)
>> >>> Transfer rate: 6420.58 [Kbytes/sec] received
>> >>>
>> >>> Connection Times (ms)
>> >>> min mean[+/-sd] median max
>> >>> Connect: 0 0 1.0 0 16
>> >>> Processing: 11 84 51.2 74 397
>> >>> Waiting: 8 77 49.4 66 397
>> >>> Total: 11 84 51.2 75 398
>> >>>
>> >>> Percentage of the requests served within a certain time (ms)
>> >>> 50% 75
>> >>> 66% 97
>> >>> 75% 113
>> >>> 80% 124
>> >>> 90% 154
>> >>> 95% 182
>> >>> 98% 217
>> >>> 99% 239
>> >>> 100% 398 (longest request)
>> >>>
>> >>> Now with 8 users:
>> >>>
>> >>> dashorst$ ab -c 8 -n 10240
>> http://localhost:8080/wicketapp/products3/200
>> >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
>> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd,
>> http://www.zeustech.net/
>> >>> Licensed to The Apache Software Foundation, http://www.apache.org/
>> >>>
>> >>> Benchmarking localhost (be patient)
>> >>> Completed 1024 requests
>> >>> Completed 2048 requests
>> >>> Completed 3072 requests
>> >>> Completed 4096 requests
>> >>> Completed 5120 requests
>> >>> Completed 6144 requests
>> >>> Completed 7168 requests
>> >>> Completed 8192 requests
>> >>> Completed 9216 requests
>> >>> Completed 10240 requests
>> >>> Finished 10240 requests
>> >>>
>> >>>
>> >>> Server Software: Jetty(6.1.16)
>> >>> Server Hostname: localhost
>> >>> Server Port: 8080
>> >>>
>> >>> Document Path: /wicketapp/products3/200
>> >>> Document Length: 34290 bytes
>> >>>
>> >>> Concurrency Level: 8
>> >>> Time taken for tests: 48.672 seconds
>> >>> Complete requests: 10240
>> >>> Failed requests: 6308
>> >>> (Connect: 0, Receive: 0, Length: 6308, Exceptions: 0)
>> >>> Write errors: 0
>> >>> Total transferred: 355046233 bytes
>> >>> HTML transferred: 352875353 bytes
>> >>> Requests per second: 210.39 [#/sec] (mean)
>> >>> Time per request: 38.025 [ms] (mean)
>> >>> Time per request: 4.753 [ms] (mean, across all concurrent
>> requests)
>> >>> Transfer rate: 7123.72 [Kbytes/sec] received
>> >>>
>> >>> Connection Times (ms)
>> >>> min mean[+/-sd] median max
>> >>> Connect: 0 0 0.9 0 10
>> >>> Processing: 11 38 19.9 33 203
>> >>> Waiting: 9 35 18.6 30 203
>> >>> Total: 11 38 20.0 33 204
>> >>>
>> >>> Percentage of the requests served within a certain time (ms)
>> >>> 50% 33
>> >>> 66% 41
>> >>> 75% 47
>> >>> 80% 53
>> >>> 90% 65
>> >>> 95% 77
>> >>> 98% 93
>> >>> 99% 105
>> >>> 100% 204 (longest request)
>> >>>
>> >>> And with 4 users:
>> >>>
>> >>> dashorst$ ab -c 4 -n 10240
>> http://localhost:8080/wicketapp/products3/200
>> >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
>> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd,
>> http://www.zeustech.net/
>> >>> Licensed to The Apache Software Foundation, http://www.apache.org/
>> >>>
>> >>> Benchmarking localhost (be patient)
>> >>> Completed 1024 requests
>> >>> Completed 2048 requests
>> >>> Completed 3072 requests
>> >>> Completed 4096 requests
>> >>> Completed 5120 requests
>> >>> Completed 6144 requests
>> >>> Completed 7168 requests
>> >>> Completed 8192 requests
>> >>> Completed 9216 requests
>> >>> Completed 10240 requests
>> >>> Finished 10240 requests
>> >>>
>> >>>
>> >>> Server Software: Jetty(6.1.16)
>> >>> Server Hostname: localhost
>> >>> Server Port: 8080
>> >>>
>> >>> Document Path: /wicketapp/products3/200
>> >>> Document Length: 34290 bytes
>> >>>
>> >>> Concurrency Level: 4
>> >>> Time taken for tests: 45.238 seconds
>> >>> Complete requests: 10240
>> >>> Failed requests: 6134
>> >>> (Connect: 0, Receive: 0, Length: 6134, Exceptions: 0)
>> >>> Write errors: 0
>> >>> Total transferred: 355063755 bytes
>> >>> HTML transferred: 352892875 bytes
>> >>> Requests per second: 226.36 [#/sec] (mean)
>> >>> Time per request: 17.671 [ms] (mean)
>> >>> Time per request: 4.418 [ms] (mean, across all concurrent
>> requests)
>> >>> Transfer rate: 7664.91 [Kbytes/sec] received
>> >>>
>> >>> Connection Times (ms)
>> >>> min mean[+/-sd] median max
>> >>> Connect: 0 0 0.2 0 7
>> >>> Processing: 11 17 2.4 18 43
>> >>> Waiting: 10 17 2.5 17 41
>> >>> Total: 11 18 2.4 18 43
>> >>>
>> >>> Percentage of the requests served within a certain time (ms)
>> >>> 50% 18
>> >>> 66% 18
>> >>> 75% 19
>> >>> 80% 19
>> >>> 90% 20
>> >>> 95% 21
>> >>> 98% 23
>> >>> 99% 25
>> >>> 100% 43 (longest request)
>> >>>
>> >>> And finally with 2 users:
>> >>>
>> >>> dashorst$ ab -c 2 -n 10240
>> http://localhost:8080/wicketapp/products3/200
>> >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
>> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd,
>> http://www.zeustech.net/
>> >>> Licensed to The Apache Software Foundation, http://www.apache.org/
>> >>>
>> >>> Benchmarking localhost (be patient)
>> >>> Completed 1024 requests
>> >>> Completed 2048 requests
>> >>> Completed 3072 requests
>> >>> Completed 4096 requests
>> >>> Completed 5120 requests
>> >>> Completed 6144 requests
>> >>> Completed 7168 requests
>> >>> Completed 8192 requests
>> >>> Completed 9216 requests
>> >>> Completed 10240 requests
>> >>> Finished 10240 requests
>> >>>
>> >>>
>> >>> Server Software: Jetty(6.1.16)
>> >>> Server Hostname: localhost
>> >>> Server Port: 8080
>> >>>
>> >>> Document Path: /wicketapp/products3/200
>> >>> Document Length: 34691 bytes
>> >>>
>> >>> Concurrency Level: 2
>> >>> Time taken for tests: 55.005 seconds
>> >>> Complete requests: 10240
>> >>> Failed requests: 7532
>> >>> (Connect: 0, Receive: 0, Length: 7532, Exceptions: 0)
>> >>> Write errors: 0
>> >>> Total transferred: 355060137 bytes
>> >>> HTML transferred: 352889257 bytes
>> >>> Requests per second: 186.16 [#/sec] (mean)
>> >>> Time per request: 10.743 [ms] (mean)
>> >>> Time per request: 5.372 [ms] (mean, across all concurrent
>> requests)
>> >>> Transfer rate: 6303.74 [Kbytes/sec] received
>> >>>
>> >>> Connection Times (ms)
>> >>> min mean[+/-sd] median max
>> >>> Connect: 0 0 0.0 0 1
>> >>> Processing: 6 11 2.8 11 25
>> >>> Waiting: 6 10 2.8 10 22
>> >>> Total: 7 11 2.8 11 25
>> >>>
>> >>> Percentage of the requests served within a certain time (ms)
>> >>> 50% 11
>> >>> 66% 12
>> >>> 75% 13
>> >>> 80% 14
>> >>> 90% 15
>> >>> 95% 15
>> >>> 98% 16
>> >>> 99% 16
>> >>> 100% 25 (longest request)
>> >>>
>> >>> Looking at the results I think it is safe to say that with 16 users we
>> >>> hit the limit of my core i7, and with 4 users we hit the sweet spot
>> >>> (which is probably correct since I have 2 cores with each 2
>> >>> hyperthreads).
>> >>>
>> >>> I expected 4 users to perform similar to 2 users, but it appears to be
>> >>> slower. I'll run it using yourkit to see if there's a blocker
>> >>> somewhere.
>> >>>
>> >>> In my overall experience, 1.5-SNAPSHOT performs comparable to 1.4.
>> >>>
>> >>> Notes from this test:
>> >>> - we report wrong length in several cases
>> >>> - exceptions occur when a request is broken off
>> >>> - performance seems on par with 1.4
>> >>>
>> >>> Martijn
>> >>>
>> >>
>> >
>>
>
--
Martin Grigorov
jWeekend
Training, Consulting, Development
http://jWeekend.com
Re: Performance statistics for Wicket 1.5-SNAPSHOT
Posted by Bruno Borges <br...@gmail.com>.
Martijn, would you mind to share the benchmark suite you have?
I'd like to test a few changes.
*Bruno Borges*
www.brunoborges.com.br
+55 21 76727099
On Thu, Jun 16, 2011 at 1:00 PM, Igor Vaynberg <ig...@gmail.com>wrote:
> lol. i was waiting for someone to say that!
>
> -igor
>
> On Thu, Jun 16, 2011 at 8:56 AM, James Carman
> <ja...@carmanconsulting.com> wrote:
> > Perhaps we should re-write it using wicket-velocity! ;)
> >
> > On Thu, Jun 16, 2011 at 11:53 AM, Igor Vaynberg <ig...@gmail.com>
> wrote:
> >> his test is hand-tailored to fail any true component oriented
> >> framework. 5000 items per page? really? show me a single website that
> >> does that. even facebook doesnt show that many comments on the wall
> >> without making you page. but, after looking at other "i hate wicket"
> >> posts on his blog i am not surprised with his testing approach :)
> >>
> >> if this situation came up in the real life any sane developer would
> >> wrap the entire product list into a single component that would stream
> >> html directly using string manipulation. and if you do that then
> >> wicket will be on par - if not faster - then the other ones. because
> >> at that point you are measuring the same thing - how fast the jvm
> >> loops.
> >>
> >> -igor
> >>
> >>
> >> On Thu, Jun 16, 2011 at 6:56 AM, Martijn Dashorst
> >> <ma...@gmail.com> wrote:
> >>> The application used in
> >>> http://www.jtict.com/blog/rails-wicket-grails-play-lift-jsp/ has been
> >>> controversial as it pits all popular frameworks against one another.
> >>> I've tried to replicate the results on my box with AB, and did some
> >>> modifications to the code:
> >>> - removed the image from the markup: browsers will hit the
> >>> non-existent image causing long load times
> >>> - did the categories list in a similar way that the Tapestry
> >>> implementation did: using a StringBuilder (saves ~300ms for a single
> >>> user with 5000 items)
> >>> - randomized the results for each query to the products() method such
> >>> that caching the results is not an option
> >>>
> >>> I couldn't replicate the tapestry results with these settings (instead
> >>> of being almost constant time, it now is linear but 1.5x faster than
> >>> wicket). I'm not sure if the author actually looked at the results of
> >>> the tapestry tests, but it seems to me that something went wrong.
> >>>
> >>> The code has been altered with the above changes, including changing
> >>> the ListView into a RepeatingView, and rendering the categories in a
> >>> StringBuilder. Here are the results of an ab test on my laptop for 200
> >>> products and 32 concurrent users:
> >>>
> >>> dashorst$ ab -c 32 -n 10240
> http://localhost:8080/wicketapp/products3/200
> >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd,
> http://www.zeustech.net/
> >>> Licensed to The Apache Software Foundation, http://www.apache.org/
> >>>
> >>> Benchmarking localhost (be patient)
> >>> Completed 1024 requests
> >>> Completed 2048 requests
> >>> Completed 3072 requests
> >>> Completed 4096 requests
> >>> Completed 5120 requests
> >>> Completed 6144 requests
> >>> Completed 7168 requests
> >>> Completed 8192 requests
> >>> Completed 9216 requests
> >>> Completed 10240 requests
> >>> Finished 10240 requests
> >>>
> >>>
> >>> Server Software: Jetty(6.1.16)
> >>> Server Hostname: localhost
> >>> Server Port: 8080
> >>>
> >>> Document Path: /wicketapp/products3/200
> >>> Document Length: 34689 bytes
> >>>
> >>> Concurrency Level: 32
> >>> Time taken for tests: 57.509 seconds
> >>> Complete requests: 10240
> >>> Failed requests: 9670
> >>> (Connect: 0, Receive: 0, Length: 9670, Exceptions: 0)
> >>> Write errors: 0
> >>> Total transferred: 354991304 bytes
> >>> HTML transferred: 352820424 bytes
> >>> Requests per second: 178.06 [#/sec] (mean)
> >>> Time per request: 179.714 [ms] (mean)
> >>> Time per request: 5.616 [ms] (mean, across all concurrent
> requests)
> >>> Transfer rate: 6028.16 [Kbytes/sec] received
> >>>
> >>> Connection Times (ms)
> >>> min mean[+/-sd] median max
> >>> Connect: 0 1 1.6 0 17
> >>> Processing: 11 179 123.2 163 1000
> >>> Waiting: 11 171 123.1 154 999
> >>> Total: 12 179 123.3 164 1000
> >>>
> >>> Percentage of the requests served within a certain time (ms)
> >>> 50% 164
> >>> 66% 218
> >>> 75% 248
> >>> 80% 268
> >>> 90% 337
> >>> 95% 418
> >>> 98% 491
> >>> 99% 552
> >>> 100% 1000 (longest request)
> >>>
> >>> The same test but then using 16 users:
> >>>
> >>> dashorst$ ab -c 16 -n 10240
> http://localhost:8080/wicketapp/products3/200
> >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd,
> http://www.zeustech.net/
> >>> Licensed to The Apache Software Foundation, http://www.apache.org/
> >>>
> >>> Benchmarking localhost (be patient)
> >>> Completed 1024 requests
> >>> Completed 2048 requests
> >>> Completed 3072 requests
> >>> Completed 4096 requests
> >>> Completed 5120 requests
> >>> Completed 6144 requests
> >>> Completed 7168 requests
> >>> Completed 8192 requests
> >>> Completed 9216 requests
> >>> Completed 10240 requests
> >>> Finished 10240 requests
> >>>
> >>>
> >>> Server Software: Jetty(6.1.16)
> >>> Server Hostname: localhost
> >>> Server Port: 8080
> >>>
> >>> Document Path: /wicketapp/products3/200
> >>> Document Length: 34290 bytes
> >>>
> >>> Concurrency Level: 16
> >>> Time taken for tests: 54.005 seconds
> >>> Complete requests: 10240
> >>> Failed requests: 6161
> >>> (Connect: 0, Receive: 0, Length: 6161, Exceptions: 0)
> >>> Write errors: 0
> >>> Total transferred: 355066608 bytes
> >>> HTML transferred: 352895728 bytes
> >>> Requests per second: 189.61 [#/sec] (mean)
> >>> Time per request: 84.383 [ms] (mean)
> >>> Time per request: 5.274 [ms] (mean, across all concurrent
> requests)
> >>> Transfer rate: 6420.58 [Kbytes/sec] received
> >>>
> >>> Connection Times (ms)
> >>> min mean[+/-sd] median max
> >>> Connect: 0 0 1.0 0 16
> >>> Processing: 11 84 51.2 74 397
> >>> Waiting: 8 77 49.4 66 397
> >>> Total: 11 84 51.2 75 398
> >>>
> >>> Percentage of the requests served within a certain time (ms)
> >>> 50% 75
> >>> 66% 97
> >>> 75% 113
> >>> 80% 124
> >>> 90% 154
> >>> 95% 182
> >>> 98% 217
> >>> 99% 239
> >>> 100% 398 (longest request)
> >>>
> >>> Now with 8 users:
> >>>
> >>> dashorst$ ab -c 8 -n 10240
> http://localhost:8080/wicketapp/products3/200
> >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd,
> http://www.zeustech.net/
> >>> Licensed to The Apache Software Foundation, http://www.apache.org/
> >>>
> >>> Benchmarking localhost (be patient)
> >>> Completed 1024 requests
> >>> Completed 2048 requests
> >>> Completed 3072 requests
> >>> Completed 4096 requests
> >>> Completed 5120 requests
> >>> Completed 6144 requests
> >>> Completed 7168 requests
> >>> Completed 8192 requests
> >>> Completed 9216 requests
> >>> Completed 10240 requests
> >>> Finished 10240 requests
> >>>
> >>>
> >>> Server Software: Jetty(6.1.16)
> >>> Server Hostname: localhost
> >>> Server Port: 8080
> >>>
> >>> Document Path: /wicketapp/products3/200
> >>> Document Length: 34290 bytes
> >>>
> >>> Concurrency Level: 8
> >>> Time taken for tests: 48.672 seconds
> >>> Complete requests: 10240
> >>> Failed requests: 6308
> >>> (Connect: 0, Receive: 0, Length: 6308, Exceptions: 0)
> >>> Write errors: 0
> >>> Total transferred: 355046233 bytes
> >>> HTML transferred: 352875353 bytes
> >>> Requests per second: 210.39 [#/sec] (mean)
> >>> Time per request: 38.025 [ms] (mean)
> >>> Time per request: 4.753 [ms] (mean, across all concurrent
> requests)
> >>> Transfer rate: 7123.72 [Kbytes/sec] received
> >>>
> >>> Connection Times (ms)
> >>> min mean[+/-sd] median max
> >>> Connect: 0 0 0.9 0 10
> >>> Processing: 11 38 19.9 33 203
> >>> Waiting: 9 35 18.6 30 203
> >>> Total: 11 38 20.0 33 204
> >>>
> >>> Percentage of the requests served within a certain time (ms)
> >>> 50% 33
> >>> 66% 41
> >>> 75% 47
> >>> 80% 53
> >>> 90% 65
> >>> 95% 77
> >>> 98% 93
> >>> 99% 105
> >>> 100% 204 (longest request)
> >>>
> >>> And with 4 users:
> >>>
> >>> dashorst$ ab -c 4 -n 10240
> http://localhost:8080/wicketapp/products3/200
> >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd,
> http://www.zeustech.net/
> >>> Licensed to The Apache Software Foundation, http://www.apache.org/
> >>>
> >>> Benchmarking localhost (be patient)
> >>> Completed 1024 requests
> >>> Completed 2048 requests
> >>> Completed 3072 requests
> >>> Completed 4096 requests
> >>> Completed 5120 requests
> >>> Completed 6144 requests
> >>> Completed 7168 requests
> >>> Completed 8192 requests
> >>> Completed 9216 requests
> >>> Completed 10240 requests
> >>> Finished 10240 requests
> >>>
> >>>
> >>> Server Software: Jetty(6.1.16)
> >>> Server Hostname: localhost
> >>> Server Port: 8080
> >>>
> >>> Document Path: /wicketapp/products3/200
> >>> Document Length: 34290 bytes
> >>>
> >>> Concurrency Level: 4
> >>> Time taken for tests: 45.238 seconds
> >>> Complete requests: 10240
> >>> Failed requests: 6134
> >>> (Connect: 0, Receive: 0, Length: 6134, Exceptions: 0)
> >>> Write errors: 0
> >>> Total transferred: 355063755 bytes
> >>> HTML transferred: 352892875 bytes
> >>> Requests per second: 226.36 [#/sec] (mean)
> >>> Time per request: 17.671 [ms] (mean)
> >>> Time per request: 4.418 [ms] (mean, across all concurrent
> requests)
> >>> Transfer rate: 7664.91 [Kbytes/sec] received
> >>>
> >>> Connection Times (ms)
> >>> min mean[+/-sd] median max
> >>> Connect: 0 0 0.2 0 7
> >>> Processing: 11 17 2.4 18 43
> >>> Waiting: 10 17 2.5 17 41
> >>> Total: 11 18 2.4 18 43
> >>>
> >>> Percentage of the requests served within a certain time (ms)
> >>> 50% 18
> >>> 66% 18
> >>> 75% 19
> >>> 80% 19
> >>> 90% 20
> >>> 95% 21
> >>> 98% 23
> >>> 99% 25
> >>> 100% 43 (longest request)
> >>>
> >>> And finally with 2 users:
> >>>
> >>> dashorst$ ab -c 2 -n 10240
> http://localhost:8080/wicketapp/products3/200
> >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd,
> http://www.zeustech.net/
> >>> Licensed to The Apache Software Foundation, http://www.apache.org/
> >>>
> >>> Benchmarking localhost (be patient)
> >>> Completed 1024 requests
> >>> Completed 2048 requests
> >>> Completed 3072 requests
> >>> Completed 4096 requests
> >>> Completed 5120 requests
> >>> Completed 6144 requests
> >>> Completed 7168 requests
> >>> Completed 8192 requests
> >>> Completed 9216 requests
> >>> Completed 10240 requests
> >>> Finished 10240 requests
> >>>
> >>>
> >>> Server Software: Jetty(6.1.16)
> >>> Server Hostname: localhost
> >>> Server Port: 8080
> >>>
> >>> Document Path: /wicketapp/products3/200
> >>> Document Length: 34691 bytes
> >>>
> >>> Concurrency Level: 2
> >>> Time taken for tests: 55.005 seconds
> >>> Complete requests: 10240
> >>> Failed requests: 7532
> >>> (Connect: 0, Receive: 0, Length: 7532, Exceptions: 0)
> >>> Write errors: 0
> >>> Total transferred: 355060137 bytes
> >>> HTML transferred: 352889257 bytes
> >>> Requests per second: 186.16 [#/sec] (mean)
> >>> Time per request: 10.743 [ms] (mean)
> >>> Time per request: 5.372 [ms] (mean, across all concurrent
> requests)
> >>> Transfer rate: 6303.74 [Kbytes/sec] received
> >>>
> >>> Connection Times (ms)
> >>> min mean[+/-sd] median max
> >>> Connect: 0 0 0.0 0 1
> >>> Processing: 6 11 2.8 11 25
> >>> Waiting: 6 10 2.8 10 22
> >>> Total: 7 11 2.8 11 25
> >>>
> >>> Percentage of the requests served within a certain time (ms)
> >>> 50% 11
> >>> 66% 12
> >>> 75% 13
> >>> 80% 14
> >>> 90% 15
> >>> 95% 15
> >>> 98% 16
> >>> 99% 16
> >>> 100% 25 (longest request)
> >>>
> >>> Looking at the results I think it is safe to say that with 16 users we
> >>> hit the limit of my core i7, and with 4 users we hit the sweet spot
> >>> (which is probably correct since I have 2 cores with each 2
> >>> hyperthreads).
> >>>
> >>> I expected 4 users to perform similar to 2 users, but it appears to be
> >>> slower. I'll run it using yourkit to see if there's a blocker
> >>> somewhere.
> >>>
> >>> In my overall experience, 1.5-SNAPSHOT performs comparable to 1.4.
> >>>
> >>> Notes from this test:
> >>> - we report wrong length in several cases
> >>> - exceptions occur when a request is broken off
> >>> - performance seems on par with 1.4
> >>>
> >>> Martijn
> >>>
> >>
> >
>
Re: Performance statistics for Wicket 1.5-SNAPSHOT
Posted by Igor Vaynberg <ig...@gmail.com>.
lol. i was waiting for someone to say that!
-igor
On Thu, Jun 16, 2011 at 8:56 AM, James Carman
<ja...@carmanconsulting.com> wrote:
> Perhaps we should re-write it using wicket-velocity! ;)
>
> On Thu, Jun 16, 2011 at 11:53 AM, Igor Vaynberg <ig...@gmail.com> wrote:
>> his test is hand-tailored to fail any true component oriented
>> framework. 5000 items per page? really? show me a single website that
>> does that. even facebook doesnt show that many comments on the wall
>> without making you page. but, after looking at other "i hate wicket"
>> posts on his blog i am not surprised with his testing approach :)
>>
>> if this situation came up in the real life any sane developer would
>> wrap the entire product list into a single component that would stream
>> html directly using string manipulation. and if you do that then
>> wicket will be on par - if not faster - then the other ones. because
>> at that point you are measuring the same thing - how fast the jvm
>> loops.
>>
>> -igor
>>
>>
>> On Thu, Jun 16, 2011 at 6:56 AM, Martijn Dashorst
>> <ma...@gmail.com> wrote:
>>> The application used in
>>> http://www.jtict.com/blog/rails-wicket-grails-play-lift-jsp/ has been
>>> controversial as it pits all popular frameworks against one another.
>>> I've tried to replicate the results on my box with AB, and did some
>>> modifications to the code:
>>> - removed the image from the markup: browsers will hit the
>>> non-existent image causing long load times
>>> - did the categories list in a similar way that the Tapestry
>>> implementation did: using a StringBuilder (saves ~300ms for a single
>>> user with 5000 items)
>>> - randomized the results for each query to the products() method such
>>> that caching the results is not an option
>>>
>>> I couldn't replicate the tapestry results with these settings (instead
>>> of being almost constant time, it now is linear but 1.5x faster than
>>> wicket). I'm not sure if the author actually looked at the results of
>>> the tapestry tests, but it seems to me that something went wrong.
>>>
>>> The code has been altered with the above changes, including changing
>>> the ListView into a RepeatingView, and rendering the categories in a
>>> StringBuilder. Here are the results of an ab test on my laptop for 200
>>> products and 32 concurrent users:
>>>
>>> dashorst$ ab -c 32 -n 10240 http://localhost:8080/wicketapp/products3/200
>>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
>>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
>>> Licensed to The Apache Software Foundation, http://www.apache.org/
>>>
>>> Benchmarking localhost (be patient)
>>> Completed 1024 requests
>>> Completed 2048 requests
>>> Completed 3072 requests
>>> Completed 4096 requests
>>> Completed 5120 requests
>>> Completed 6144 requests
>>> Completed 7168 requests
>>> Completed 8192 requests
>>> Completed 9216 requests
>>> Completed 10240 requests
>>> Finished 10240 requests
>>>
>>>
>>> Server Software: Jetty(6.1.16)
>>> Server Hostname: localhost
>>> Server Port: 8080
>>>
>>> Document Path: /wicketapp/products3/200
>>> Document Length: 34689 bytes
>>>
>>> Concurrency Level: 32
>>> Time taken for tests: 57.509 seconds
>>> Complete requests: 10240
>>> Failed requests: 9670
>>> (Connect: 0, Receive: 0, Length: 9670, Exceptions: 0)
>>> Write errors: 0
>>> Total transferred: 354991304 bytes
>>> HTML transferred: 352820424 bytes
>>> Requests per second: 178.06 [#/sec] (mean)
>>> Time per request: 179.714 [ms] (mean)
>>> Time per request: 5.616 [ms] (mean, across all concurrent requests)
>>> Transfer rate: 6028.16 [Kbytes/sec] received
>>>
>>> Connection Times (ms)
>>> min mean[+/-sd] median max
>>> Connect: 0 1 1.6 0 17
>>> Processing: 11 179 123.2 163 1000
>>> Waiting: 11 171 123.1 154 999
>>> Total: 12 179 123.3 164 1000
>>>
>>> Percentage of the requests served within a certain time (ms)
>>> 50% 164
>>> 66% 218
>>> 75% 248
>>> 80% 268
>>> 90% 337
>>> 95% 418
>>> 98% 491
>>> 99% 552
>>> 100% 1000 (longest request)
>>>
>>> The same test but then using 16 users:
>>>
>>> dashorst$ ab -c 16 -n 10240 http://localhost:8080/wicketapp/products3/200
>>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
>>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
>>> Licensed to The Apache Software Foundation, http://www.apache.org/
>>>
>>> Benchmarking localhost (be patient)
>>> Completed 1024 requests
>>> Completed 2048 requests
>>> Completed 3072 requests
>>> Completed 4096 requests
>>> Completed 5120 requests
>>> Completed 6144 requests
>>> Completed 7168 requests
>>> Completed 8192 requests
>>> Completed 9216 requests
>>> Completed 10240 requests
>>> Finished 10240 requests
>>>
>>>
>>> Server Software: Jetty(6.1.16)
>>> Server Hostname: localhost
>>> Server Port: 8080
>>>
>>> Document Path: /wicketapp/products3/200
>>> Document Length: 34290 bytes
>>>
>>> Concurrency Level: 16
>>> Time taken for tests: 54.005 seconds
>>> Complete requests: 10240
>>> Failed requests: 6161
>>> (Connect: 0, Receive: 0, Length: 6161, Exceptions: 0)
>>> Write errors: 0
>>> Total transferred: 355066608 bytes
>>> HTML transferred: 352895728 bytes
>>> Requests per second: 189.61 [#/sec] (mean)
>>> Time per request: 84.383 [ms] (mean)
>>> Time per request: 5.274 [ms] (mean, across all concurrent requests)
>>> Transfer rate: 6420.58 [Kbytes/sec] received
>>>
>>> Connection Times (ms)
>>> min mean[+/-sd] median max
>>> Connect: 0 0 1.0 0 16
>>> Processing: 11 84 51.2 74 397
>>> Waiting: 8 77 49.4 66 397
>>> Total: 11 84 51.2 75 398
>>>
>>> Percentage of the requests served within a certain time (ms)
>>> 50% 75
>>> 66% 97
>>> 75% 113
>>> 80% 124
>>> 90% 154
>>> 95% 182
>>> 98% 217
>>> 99% 239
>>> 100% 398 (longest request)
>>>
>>> Now with 8 users:
>>>
>>> dashorst$ ab -c 8 -n 10240 http://localhost:8080/wicketapp/products3/200
>>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
>>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
>>> Licensed to The Apache Software Foundation, http://www.apache.org/
>>>
>>> Benchmarking localhost (be patient)
>>> Completed 1024 requests
>>> Completed 2048 requests
>>> Completed 3072 requests
>>> Completed 4096 requests
>>> Completed 5120 requests
>>> Completed 6144 requests
>>> Completed 7168 requests
>>> Completed 8192 requests
>>> Completed 9216 requests
>>> Completed 10240 requests
>>> Finished 10240 requests
>>>
>>>
>>> Server Software: Jetty(6.1.16)
>>> Server Hostname: localhost
>>> Server Port: 8080
>>>
>>> Document Path: /wicketapp/products3/200
>>> Document Length: 34290 bytes
>>>
>>> Concurrency Level: 8
>>> Time taken for tests: 48.672 seconds
>>> Complete requests: 10240
>>> Failed requests: 6308
>>> (Connect: 0, Receive: 0, Length: 6308, Exceptions: 0)
>>> Write errors: 0
>>> Total transferred: 355046233 bytes
>>> HTML transferred: 352875353 bytes
>>> Requests per second: 210.39 [#/sec] (mean)
>>> Time per request: 38.025 [ms] (mean)
>>> Time per request: 4.753 [ms] (mean, across all concurrent requests)
>>> Transfer rate: 7123.72 [Kbytes/sec] received
>>>
>>> Connection Times (ms)
>>> min mean[+/-sd] median max
>>> Connect: 0 0 0.9 0 10
>>> Processing: 11 38 19.9 33 203
>>> Waiting: 9 35 18.6 30 203
>>> Total: 11 38 20.0 33 204
>>>
>>> Percentage of the requests served within a certain time (ms)
>>> 50% 33
>>> 66% 41
>>> 75% 47
>>> 80% 53
>>> 90% 65
>>> 95% 77
>>> 98% 93
>>> 99% 105
>>> 100% 204 (longest request)
>>>
>>> And with 4 users:
>>>
>>> dashorst$ ab -c 4 -n 10240 http://localhost:8080/wicketapp/products3/200
>>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
>>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
>>> Licensed to The Apache Software Foundation, http://www.apache.org/
>>>
>>> Benchmarking localhost (be patient)
>>> Completed 1024 requests
>>> Completed 2048 requests
>>> Completed 3072 requests
>>> Completed 4096 requests
>>> Completed 5120 requests
>>> Completed 6144 requests
>>> Completed 7168 requests
>>> Completed 8192 requests
>>> Completed 9216 requests
>>> Completed 10240 requests
>>> Finished 10240 requests
>>>
>>>
>>> Server Software: Jetty(6.1.16)
>>> Server Hostname: localhost
>>> Server Port: 8080
>>>
>>> Document Path: /wicketapp/products3/200
>>> Document Length: 34290 bytes
>>>
>>> Concurrency Level: 4
>>> Time taken for tests: 45.238 seconds
>>> Complete requests: 10240
>>> Failed requests: 6134
>>> (Connect: 0, Receive: 0, Length: 6134, Exceptions: 0)
>>> Write errors: 0
>>> Total transferred: 355063755 bytes
>>> HTML transferred: 352892875 bytes
>>> Requests per second: 226.36 [#/sec] (mean)
>>> Time per request: 17.671 [ms] (mean)
>>> Time per request: 4.418 [ms] (mean, across all concurrent requests)
>>> Transfer rate: 7664.91 [Kbytes/sec] received
>>>
>>> Connection Times (ms)
>>> min mean[+/-sd] median max
>>> Connect: 0 0 0.2 0 7
>>> Processing: 11 17 2.4 18 43
>>> Waiting: 10 17 2.5 17 41
>>> Total: 11 18 2.4 18 43
>>>
>>> Percentage of the requests served within a certain time (ms)
>>> 50% 18
>>> 66% 18
>>> 75% 19
>>> 80% 19
>>> 90% 20
>>> 95% 21
>>> 98% 23
>>> 99% 25
>>> 100% 43 (longest request)
>>>
>>> And finally with 2 users:
>>>
>>> dashorst$ ab -c 2 -n 10240 http://localhost:8080/wicketapp/products3/200
>>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
>>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
>>> Licensed to The Apache Software Foundation, http://www.apache.org/
>>>
>>> Benchmarking localhost (be patient)
>>> Completed 1024 requests
>>> Completed 2048 requests
>>> Completed 3072 requests
>>> Completed 4096 requests
>>> Completed 5120 requests
>>> Completed 6144 requests
>>> Completed 7168 requests
>>> Completed 8192 requests
>>> Completed 9216 requests
>>> Completed 10240 requests
>>> Finished 10240 requests
>>>
>>>
>>> Server Software: Jetty(6.1.16)
>>> Server Hostname: localhost
>>> Server Port: 8080
>>>
>>> Document Path: /wicketapp/products3/200
>>> Document Length: 34691 bytes
>>>
>>> Concurrency Level: 2
>>> Time taken for tests: 55.005 seconds
>>> Complete requests: 10240
>>> Failed requests: 7532
>>> (Connect: 0, Receive: 0, Length: 7532, Exceptions: 0)
>>> Write errors: 0
>>> Total transferred: 355060137 bytes
>>> HTML transferred: 352889257 bytes
>>> Requests per second: 186.16 [#/sec] (mean)
>>> Time per request: 10.743 [ms] (mean)
>>> Time per request: 5.372 [ms] (mean, across all concurrent requests)
>>> Transfer rate: 6303.74 [Kbytes/sec] received
>>>
>>> Connection Times (ms)
>>> min mean[+/-sd] median max
>>> Connect: 0 0 0.0 0 1
>>> Processing: 6 11 2.8 11 25
>>> Waiting: 6 10 2.8 10 22
>>> Total: 7 11 2.8 11 25
>>>
>>> Percentage of the requests served within a certain time (ms)
>>> 50% 11
>>> 66% 12
>>> 75% 13
>>> 80% 14
>>> 90% 15
>>> 95% 15
>>> 98% 16
>>> 99% 16
>>> 100% 25 (longest request)
>>>
>>> Looking at the results I think it is safe to say that with 16 users we
>>> hit the limit of my core i7, and with 4 users we hit the sweet spot
>>> (which is probably correct since I have 2 cores with each 2
>>> hyperthreads).
>>>
>>> I expected 4 users to perform similar to 2 users, but it appears to be
>>> slower. I'll run it using yourkit to see if there's a blocker
>>> somewhere.
>>>
>>> In my overall experience, 1.5-SNAPSHOT performs comparable to 1.4.
>>>
>>> Notes from this test:
>>> - we report wrong length in several cases
>>> - exceptions occur when a request is broken off
>>> - performance seems on par with 1.4
>>>
>>> Martijn
>>>
>>
>
Re: Performance statistics for Wicket 1.5-SNAPSHOT
Posted by James Carman <ja...@carmanconsulting.com>.
Perhaps we should re-write it using wicket-velocity! ;)
On Thu, Jun 16, 2011 at 11:53 AM, Igor Vaynberg <ig...@gmail.com> wrote:
> his test is hand-tailored to fail any true component oriented
> framework. 5000 items per page? really? show me a single website that
> does that. even facebook doesnt show that many comments on the wall
> without making you page. but, after looking at other "i hate wicket"
> posts on his blog i am not surprised with his testing approach :)
>
> if this situation came up in the real life any sane developer would
> wrap the entire product list into a single component that would stream
> html directly using string manipulation. and if you do that then
> wicket will be on par - if not faster - then the other ones. because
> at that point you are measuring the same thing - how fast the jvm
> loops.
>
> -igor
>
>
> On Thu, Jun 16, 2011 at 6:56 AM, Martijn Dashorst
> <ma...@gmail.com> wrote:
>> The application used in
>> http://www.jtict.com/blog/rails-wicket-grails-play-lift-jsp/ has been
>> controversial as it pits all popular frameworks against one another.
>> I've tried to replicate the results on my box with AB, and did some
>> modifications to the code:
>> - removed the image from the markup: browsers will hit the
>> non-existent image causing long load times
>> - did the categories list in a similar way that the Tapestry
>> implementation did: using a StringBuilder (saves ~300ms for a single
>> user with 5000 items)
>> - randomized the results for each query to the products() method such
>> that caching the results is not an option
>>
>> I couldn't replicate the tapestry results with these settings (instead
>> of being almost constant time, it now is linear but 1.5x faster than
>> wicket). I'm not sure if the author actually looked at the results of
>> the tapestry tests, but it seems to me that something went wrong.
>>
>> The code has been altered with the above changes, including changing
>> the ListView into a RepeatingView, and rendering the categories in a
>> StringBuilder. Here are the results of an ab test on my laptop for 200
>> products and 32 concurrent users:
>>
>> dashorst$ ab -c 32 -n 10240 http://localhost:8080/wicketapp/products3/200
>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
>> Licensed to The Apache Software Foundation, http://www.apache.org/
>>
>> Benchmarking localhost (be patient)
>> Completed 1024 requests
>> Completed 2048 requests
>> Completed 3072 requests
>> Completed 4096 requests
>> Completed 5120 requests
>> Completed 6144 requests
>> Completed 7168 requests
>> Completed 8192 requests
>> Completed 9216 requests
>> Completed 10240 requests
>> Finished 10240 requests
>>
>>
>> Server Software: Jetty(6.1.16)
>> Server Hostname: localhost
>> Server Port: 8080
>>
>> Document Path: /wicketapp/products3/200
>> Document Length: 34689 bytes
>>
>> Concurrency Level: 32
>> Time taken for tests: 57.509 seconds
>> Complete requests: 10240
>> Failed requests: 9670
>> (Connect: 0, Receive: 0, Length: 9670, Exceptions: 0)
>> Write errors: 0
>> Total transferred: 354991304 bytes
>> HTML transferred: 352820424 bytes
>> Requests per second: 178.06 [#/sec] (mean)
>> Time per request: 179.714 [ms] (mean)
>> Time per request: 5.616 [ms] (mean, across all concurrent requests)
>> Transfer rate: 6028.16 [Kbytes/sec] received
>>
>> Connection Times (ms)
>> min mean[+/-sd] median max
>> Connect: 0 1 1.6 0 17
>> Processing: 11 179 123.2 163 1000
>> Waiting: 11 171 123.1 154 999
>> Total: 12 179 123.3 164 1000
>>
>> Percentage of the requests served within a certain time (ms)
>> 50% 164
>> 66% 218
>> 75% 248
>> 80% 268
>> 90% 337
>> 95% 418
>> 98% 491
>> 99% 552
>> 100% 1000 (longest request)
>>
>> The same test but then using 16 users:
>>
>> dashorst$ ab -c 16 -n 10240 http://localhost:8080/wicketapp/products3/200
>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
>> Licensed to The Apache Software Foundation, http://www.apache.org/
>>
>> Benchmarking localhost (be patient)
>> Completed 1024 requests
>> Completed 2048 requests
>> Completed 3072 requests
>> Completed 4096 requests
>> Completed 5120 requests
>> Completed 6144 requests
>> Completed 7168 requests
>> Completed 8192 requests
>> Completed 9216 requests
>> Completed 10240 requests
>> Finished 10240 requests
>>
>>
>> Server Software: Jetty(6.1.16)
>> Server Hostname: localhost
>> Server Port: 8080
>>
>> Document Path: /wicketapp/products3/200
>> Document Length: 34290 bytes
>>
>> Concurrency Level: 16
>> Time taken for tests: 54.005 seconds
>> Complete requests: 10240
>> Failed requests: 6161
>> (Connect: 0, Receive: 0, Length: 6161, Exceptions: 0)
>> Write errors: 0
>> Total transferred: 355066608 bytes
>> HTML transferred: 352895728 bytes
>> Requests per second: 189.61 [#/sec] (mean)
>> Time per request: 84.383 [ms] (mean)
>> Time per request: 5.274 [ms] (mean, across all concurrent requests)
>> Transfer rate: 6420.58 [Kbytes/sec] received
>>
>> Connection Times (ms)
>> min mean[+/-sd] median max
>> Connect: 0 0 1.0 0 16
>> Processing: 11 84 51.2 74 397
>> Waiting: 8 77 49.4 66 397
>> Total: 11 84 51.2 75 398
>>
>> Percentage of the requests served within a certain time (ms)
>> 50% 75
>> 66% 97
>> 75% 113
>> 80% 124
>> 90% 154
>> 95% 182
>> 98% 217
>> 99% 239
>> 100% 398 (longest request)
>>
>> Now with 8 users:
>>
>> dashorst$ ab -c 8 -n 10240 http://localhost:8080/wicketapp/products3/200
>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
>> Licensed to The Apache Software Foundation, http://www.apache.org/
>>
>> Benchmarking localhost (be patient)
>> Completed 1024 requests
>> Completed 2048 requests
>> Completed 3072 requests
>> Completed 4096 requests
>> Completed 5120 requests
>> Completed 6144 requests
>> Completed 7168 requests
>> Completed 8192 requests
>> Completed 9216 requests
>> Completed 10240 requests
>> Finished 10240 requests
>>
>>
>> Server Software: Jetty(6.1.16)
>> Server Hostname: localhost
>> Server Port: 8080
>>
>> Document Path: /wicketapp/products3/200
>> Document Length: 34290 bytes
>>
>> Concurrency Level: 8
>> Time taken for tests: 48.672 seconds
>> Complete requests: 10240
>> Failed requests: 6308
>> (Connect: 0, Receive: 0, Length: 6308, Exceptions: 0)
>> Write errors: 0
>> Total transferred: 355046233 bytes
>> HTML transferred: 352875353 bytes
>> Requests per second: 210.39 [#/sec] (mean)
>> Time per request: 38.025 [ms] (mean)
>> Time per request: 4.753 [ms] (mean, across all concurrent requests)
>> Transfer rate: 7123.72 [Kbytes/sec] received
>>
>> Connection Times (ms)
>> min mean[+/-sd] median max
>> Connect: 0 0 0.9 0 10
>> Processing: 11 38 19.9 33 203
>> Waiting: 9 35 18.6 30 203
>> Total: 11 38 20.0 33 204
>>
>> Percentage of the requests served within a certain time (ms)
>> 50% 33
>> 66% 41
>> 75% 47
>> 80% 53
>> 90% 65
>> 95% 77
>> 98% 93
>> 99% 105
>> 100% 204 (longest request)
>>
>> And with 4 users:
>>
>> dashorst$ ab -c 4 -n 10240 http://localhost:8080/wicketapp/products3/200
>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
>> Licensed to The Apache Software Foundation, http://www.apache.org/
>>
>> Benchmarking localhost (be patient)
>> Completed 1024 requests
>> Completed 2048 requests
>> Completed 3072 requests
>> Completed 4096 requests
>> Completed 5120 requests
>> Completed 6144 requests
>> Completed 7168 requests
>> Completed 8192 requests
>> Completed 9216 requests
>> Completed 10240 requests
>> Finished 10240 requests
>>
>>
>> Server Software: Jetty(6.1.16)
>> Server Hostname: localhost
>> Server Port: 8080
>>
>> Document Path: /wicketapp/products3/200
>> Document Length: 34290 bytes
>>
>> Concurrency Level: 4
>> Time taken for tests: 45.238 seconds
>> Complete requests: 10240
>> Failed requests: 6134
>> (Connect: 0, Receive: 0, Length: 6134, Exceptions: 0)
>> Write errors: 0
>> Total transferred: 355063755 bytes
>> HTML transferred: 352892875 bytes
>> Requests per second: 226.36 [#/sec] (mean)
>> Time per request: 17.671 [ms] (mean)
>> Time per request: 4.418 [ms] (mean, across all concurrent requests)
>> Transfer rate: 7664.91 [Kbytes/sec] received
>>
>> Connection Times (ms)
>> min mean[+/-sd] median max
>> Connect: 0 0 0.2 0 7
>> Processing: 11 17 2.4 18 43
>> Waiting: 10 17 2.5 17 41
>> Total: 11 18 2.4 18 43
>>
>> Percentage of the requests served within a certain time (ms)
>> 50% 18
>> 66% 18
>> 75% 19
>> 80% 19
>> 90% 20
>> 95% 21
>> 98% 23
>> 99% 25
>> 100% 43 (longest request)
>>
>> And finally with 2 users:
>>
>> dashorst$ ab -c 2 -n 10240 http://localhost:8080/wicketapp/products3/200
>> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
>> Licensed to The Apache Software Foundation, http://www.apache.org/
>>
>> Benchmarking localhost (be patient)
>> Completed 1024 requests
>> Completed 2048 requests
>> Completed 3072 requests
>> Completed 4096 requests
>> Completed 5120 requests
>> Completed 6144 requests
>> Completed 7168 requests
>> Completed 8192 requests
>> Completed 9216 requests
>> Completed 10240 requests
>> Finished 10240 requests
>>
>>
>> Server Software: Jetty(6.1.16)
>> Server Hostname: localhost
>> Server Port: 8080
>>
>> Document Path: /wicketapp/products3/200
>> Document Length: 34691 bytes
>>
>> Concurrency Level: 2
>> Time taken for tests: 55.005 seconds
>> Complete requests: 10240
>> Failed requests: 7532
>> (Connect: 0, Receive: 0, Length: 7532, Exceptions: 0)
>> Write errors: 0
>> Total transferred: 355060137 bytes
>> HTML transferred: 352889257 bytes
>> Requests per second: 186.16 [#/sec] (mean)
>> Time per request: 10.743 [ms] (mean)
>> Time per request: 5.372 [ms] (mean, across all concurrent requests)
>> Transfer rate: 6303.74 [Kbytes/sec] received
>>
>> Connection Times (ms)
>> min mean[+/-sd] median max
>> Connect: 0 0 0.0 0 1
>> Processing: 6 11 2.8 11 25
>> Waiting: 6 10 2.8 10 22
>> Total: 7 11 2.8 11 25
>>
>> Percentage of the requests served within a certain time (ms)
>> 50% 11
>> 66% 12
>> 75% 13
>> 80% 14
>> 90% 15
>> 95% 15
>> 98% 16
>> 99% 16
>> 100% 25 (longest request)
>>
>> Looking at the results I think it is safe to say that with 16 users we
>> hit the limit of my core i7, and with 4 users we hit the sweet spot
>> (which is probably correct since I have 2 cores with each 2
>> hyperthreads).
>>
>> I expected 4 users to perform similar to 2 users, but it appears to be
>> slower. I'll run it using yourkit to see if there's a blocker
>> somewhere.
>>
>> In my overall experience, 1.5-SNAPSHOT performs comparable to 1.4.
>>
>> Notes from this test:
>> - we report wrong length in several cases
>> - exceptions occur when a request is broken off
>> - performance seems on par with 1.4
>>
>> Martijn
>>
>
Re: Performance statistics for Wicket 1.5-SNAPSHOT
Posted by Igor Vaynberg <ig...@gmail.com>.
his test is hand-tailored to fail any true component oriented
framework. 5000 items per page? really? show me a single website that
does that. even facebook doesnt show that many comments on the wall
without making you page. but, after looking at other "i hate wicket"
posts on his blog i am not surprised with his testing approach :)
if this situation came up in the real life any sane developer would
wrap the entire product list into a single component that would stream
html directly using string manipulation. and if you do that then
wicket will be on par - if not faster - then the other ones. because
at that point you are measuring the same thing - how fast the jvm
loops.
-igor
On Thu, Jun 16, 2011 at 6:56 AM, Martijn Dashorst
<ma...@gmail.com> wrote:
> The application used in
> http://www.jtict.com/blog/rails-wicket-grails-play-lift-jsp/ has been
> controversial as it pits all popular frameworks against one another.
> I've tried to replicate the results on my box with AB, and did some
> modifications to the code:
> - removed the image from the markup: browsers will hit the
> non-existent image causing long load times
> - did the categories list in a similar way that the Tapestry
> implementation did: using a StringBuilder (saves ~300ms for a single
> user with 5000 items)
> - randomized the results for each query to the products() method such
> that caching the results is not an option
>
> I couldn't replicate the tapestry results with these settings (instead
> of being almost constant time, it now is linear but 1.5x faster than
> wicket). I'm not sure if the author actually looked at the results of
> the tapestry tests, but it seems to me that something went wrong.
>
> The code has been altered with the above changes, including changing
> the ListView into a RepeatingView, and rendering the categories in a
> StringBuilder. Here are the results of an ab test on my laptop for 200
> products and 32 concurrent users:
>
> dashorst$ ab -c 32 -n 10240 http://localhost:8080/wicketapp/products3/200
> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
> Licensed to The Apache Software Foundation, http://www.apache.org/
>
> Benchmarking localhost (be patient)
> Completed 1024 requests
> Completed 2048 requests
> Completed 3072 requests
> Completed 4096 requests
> Completed 5120 requests
> Completed 6144 requests
> Completed 7168 requests
> Completed 8192 requests
> Completed 9216 requests
> Completed 10240 requests
> Finished 10240 requests
>
>
> Server Software: Jetty(6.1.16)
> Server Hostname: localhost
> Server Port: 8080
>
> Document Path: /wicketapp/products3/200
> Document Length: 34689 bytes
>
> Concurrency Level: 32
> Time taken for tests: 57.509 seconds
> Complete requests: 10240
> Failed requests: 9670
> (Connect: 0, Receive: 0, Length: 9670, Exceptions: 0)
> Write errors: 0
> Total transferred: 354991304 bytes
> HTML transferred: 352820424 bytes
> Requests per second: 178.06 [#/sec] (mean)
> Time per request: 179.714 [ms] (mean)
> Time per request: 5.616 [ms] (mean, across all concurrent requests)
> Transfer rate: 6028.16 [Kbytes/sec] received
>
> Connection Times (ms)
> min mean[+/-sd] median max
> Connect: 0 1 1.6 0 17
> Processing: 11 179 123.2 163 1000
> Waiting: 11 171 123.1 154 999
> Total: 12 179 123.3 164 1000
>
> Percentage of the requests served within a certain time (ms)
> 50% 164
> 66% 218
> 75% 248
> 80% 268
> 90% 337
> 95% 418
> 98% 491
> 99% 552
> 100% 1000 (longest request)
>
> The same test but then using 16 users:
>
> dashorst$ ab -c 16 -n 10240 http://localhost:8080/wicketapp/products3/200
> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
> Licensed to The Apache Software Foundation, http://www.apache.org/
>
> Benchmarking localhost (be patient)
> Completed 1024 requests
> Completed 2048 requests
> Completed 3072 requests
> Completed 4096 requests
> Completed 5120 requests
> Completed 6144 requests
> Completed 7168 requests
> Completed 8192 requests
> Completed 9216 requests
> Completed 10240 requests
> Finished 10240 requests
>
>
> Server Software: Jetty(6.1.16)
> Server Hostname: localhost
> Server Port: 8080
>
> Document Path: /wicketapp/products3/200
> Document Length: 34290 bytes
>
> Concurrency Level: 16
> Time taken for tests: 54.005 seconds
> Complete requests: 10240
> Failed requests: 6161
> (Connect: 0, Receive: 0, Length: 6161, Exceptions: 0)
> Write errors: 0
> Total transferred: 355066608 bytes
> HTML transferred: 352895728 bytes
> Requests per second: 189.61 [#/sec] (mean)
> Time per request: 84.383 [ms] (mean)
> Time per request: 5.274 [ms] (mean, across all concurrent requests)
> Transfer rate: 6420.58 [Kbytes/sec] received
>
> Connection Times (ms)
> min mean[+/-sd] median max
> Connect: 0 0 1.0 0 16
> Processing: 11 84 51.2 74 397
> Waiting: 8 77 49.4 66 397
> Total: 11 84 51.2 75 398
>
> Percentage of the requests served within a certain time (ms)
> 50% 75
> 66% 97
> 75% 113
> 80% 124
> 90% 154
> 95% 182
> 98% 217
> 99% 239
> 100% 398 (longest request)
>
> Now with 8 users:
>
> dashorst$ ab -c 8 -n 10240 http://localhost:8080/wicketapp/products3/200
> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
> Licensed to The Apache Software Foundation, http://www.apache.org/
>
> Benchmarking localhost (be patient)
> Completed 1024 requests
> Completed 2048 requests
> Completed 3072 requests
> Completed 4096 requests
> Completed 5120 requests
> Completed 6144 requests
> Completed 7168 requests
> Completed 8192 requests
> Completed 9216 requests
> Completed 10240 requests
> Finished 10240 requests
>
>
> Server Software: Jetty(6.1.16)
> Server Hostname: localhost
> Server Port: 8080
>
> Document Path: /wicketapp/products3/200
> Document Length: 34290 bytes
>
> Concurrency Level: 8
> Time taken for tests: 48.672 seconds
> Complete requests: 10240
> Failed requests: 6308
> (Connect: 0, Receive: 0, Length: 6308, Exceptions: 0)
> Write errors: 0
> Total transferred: 355046233 bytes
> HTML transferred: 352875353 bytes
> Requests per second: 210.39 [#/sec] (mean)
> Time per request: 38.025 [ms] (mean)
> Time per request: 4.753 [ms] (mean, across all concurrent requests)
> Transfer rate: 7123.72 [Kbytes/sec] received
>
> Connection Times (ms)
> min mean[+/-sd] median max
> Connect: 0 0 0.9 0 10
> Processing: 11 38 19.9 33 203
> Waiting: 9 35 18.6 30 203
> Total: 11 38 20.0 33 204
>
> Percentage of the requests served within a certain time (ms)
> 50% 33
> 66% 41
> 75% 47
> 80% 53
> 90% 65
> 95% 77
> 98% 93
> 99% 105
> 100% 204 (longest request)
>
> And with 4 users:
>
> dashorst$ ab -c 4 -n 10240 http://localhost:8080/wicketapp/products3/200
> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
> Licensed to The Apache Software Foundation, http://www.apache.org/
>
> Benchmarking localhost (be patient)
> Completed 1024 requests
> Completed 2048 requests
> Completed 3072 requests
> Completed 4096 requests
> Completed 5120 requests
> Completed 6144 requests
> Completed 7168 requests
> Completed 8192 requests
> Completed 9216 requests
> Completed 10240 requests
> Finished 10240 requests
>
>
> Server Software: Jetty(6.1.16)
> Server Hostname: localhost
> Server Port: 8080
>
> Document Path: /wicketapp/products3/200
> Document Length: 34290 bytes
>
> Concurrency Level: 4
> Time taken for tests: 45.238 seconds
> Complete requests: 10240
> Failed requests: 6134
> (Connect: 0, Receive: 0, Length: 6134, Exceptions: 0)
> Write errors: 0
> Total transferred: 355063755 bytes
> HTML transferred: 352892875 bytes
> Requests per second: 226.36 [#/sec] (mean)
> Time per request: 17.671 [ms] (mean)
> Time per request: 4.418 [ms] (mean, across all concurrent requests)
> Transfer rate: 7664.91 [Kbytes/sec] received
>
> Connection Times (ms)
> min mean[+/-sd] median max
> Connect: 0 0 0.2 0 7
> Processing: 11 17 2.4 18 43
> Waiting: 10 17 2.5 17 41
> Total: 11 18 2.4 18 43
>
> Percentage of the requests served within a certain time (ms)
> 50% 18
> 66% 18
> 75% 19
> 80% 19
> 90% 20
> 95% 21
> 98% 23
> 99% 25
> 100% 43 (longest request)
>
> And finally with 2 users:
>
> dashorst$ ab -c 2 -n 10240 http://localhost:8080/wicketapp/products3/200
> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
> Licensed to The Apache Software Foundation, http://www.apache.org/
>
> Benchmarking localhost (be patient)
> Completed 1024 requests
> Completed 2048 requests
> Completed 3072 requests
> Completed 4096 requests
> Completed 5120 requests
> Completed 6144 requests
> Completed 7168 requests
> Completed 8192 requests
> Completed 9216 requests
> Completed 10240 requests
> Finished 10240 requests
>
>
> Server Software: Jetty(6.1.16)
> Server Hostname: localhost
> Server Port: 8080
>
> Document Path: /wicketapp/products3/200
> Document Length: 34691 bytes
>
> Concurrency Level: 2
> Time taken for tests: 55.005 seconds
> Complete requests: 10240
> Failed requests: 7532
> (Connect: 0, Receive: 0, Length: 7532, Exceptions: 0)
> Write errors: 0
> Total transferred: 355060137 bytes
> HTML transferred: 352889257 bytes
> Requests per second: 186.16 [#/sec] (mean)
> Time per request: 10.743 [ms] (mean)
> Time per request: 5.372 [ms] (mean, across all concurrent requests)
> Transfer rate: 6303.74 [Kbytes/sec] received
>
> Connection Times (ms)
> min mean[+/-sd] median max
> Connect: 0 0 0.0 0 1
> Processing: 6 11 2.8 11 25
> Waiting: 6 10 2.8 10 22
> Total: 7 11 2.8 11 25
>
> Percentage of the requests served within a certain time (ms)
> 50% 11
> 66% 12
> 75% 13
> 80% 14
> 90% 15
> 95% 15
> 98% 16
> 99% 16
> 100% 25 (longest request)
>
> Looking at the results I think it is safe to say that with 16 users we
> hit the limit of my core i7, and with 4 users we hit the sweet spot
> (which is probably correct since I have 2 cores with each 2
> hyperthreads).
>
> I expected 4 users to perform similar to 2 users, but it appears to be
> slower. I'll run it using yourkit to see if there's a blocker
> somewhere.
>
> In my overall experience, 1.5-SNAPSHOT performs comparable to 1.4.
>
> Notes from this test:
> - we report wrong length in several cases
> - exceptions occur when a request is broken off
> - performance seems on par with 1.4
>
> Martijn
>
Re: Performance statistics for Wicket 1.5-SNAPSHOT
Posted by Martijn Dashorst <ma...@gmail.com>.
Oh, and I removed the new ArrayList<Category>(product.getCategories()) as well.
Martijn
On Thu, Jun 16, 2011 at 3:56 PM, Martijn Dashorst
<ma...@gmail.com> wrote:
> The application used in
> http://www.jtict.com/blog/rails-wicket-grails-play-lift-jsp/ has been
> controversial as it pits all popular frameworks against one another.
> I've tried to replicate the results on my box with AB, and did some
> modifications to the code:
> - removed the image from the markup: browsers will hit the
> non-existent image causing long load times
> - did the categories list in a similar way that the Tapestry
> implementation did: using a StringBuilder (saves ~300ms for a single
> user with 5000 items)
> - randomized the results for each query to the products() method such
> that caching the results is not an option
>
> I couldn't replicate the tapestry results with these settings (instead
> of being almost constant time, it now is linear but 1.5x faster than
> wicket). I'm not sure if the author actually looked at the results of
> the tapestry tests, but it seems to me that something went wrong.
>
> The code has been altered with the above changes, including changing
> the ListView into a RepeatingView, and rendering the categories in a
> StringBuilder. Here are the results of an ab test on my laptop for 200
> products and 32 concurrent users:
>
> dashorst$ ab -c 32 -n 10240 http://localhost:8080/wicketapp/products3/200
> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
> Licensed to The Apache Software Foundation, http://www.apache.org/
>
> Benchmarking localhost (be patient)
> Completed 1024 requests
> Completed 2048 requests
> Completed 3072 requests
> Completed 4096 requests
> Completed 5120 requests
> Completed 6144 requests
> Completed 7168 requests
> Completed 8192 requests
> Completed 9216 requests
> Completed 10240 requests
> Finished 10240 requests
>
>
> Server Software: Jetty(6.1.16)
> Server Hostname: localhost
> Server Port: 8080
>
> Document Path: /wicketapp/products3/200
> Document Length: 34689 bytes
>
> Concurrency Level: 32
> Time taken for tests: 57.509 seconds
> Complete requests: 10240
> Failed requests: 9670
> (Connect: 0, Receive: 0, Length: 9670, Exceptions: 0)
> Write errors: 0
> Total transferred: 354991304 bytes
> HTML transferred: 352820424 bytes
> Requests per second: 178.06 [#/sec] (mean)
> Time per request: 179.714 [ms] (mean)
> Time per request: 5.616 [ms] (mean, across all concurrent requests)
> Transfer rate: 6028.16 [Kbytes/sec] received
>
> Connection Times (ms)
> min mean[+/-sd] median max
> Connect: 0 1 1.6 0 17
> Processing: 11 179 123.2 163 1000
> Waiting: 11 171 123.1 154 999
> Total: 12 179 123.3 164 1000
>
> Percentage of the requests served within a certain time (ms)
> 50% 164
> 66% 218
> 75% 248
> 80% 268
> 90% 337
> 95% 418
> 98% 491
> 99% 552
> 100% 1000 (longest request)
>
> The same test but then using 16 users:
>
> dashorst$ ab -c 16 -n 10240 http://localhost:8080/wicketapp/products3/200
> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
> Licensed to The Apache Software Foundation, http://www.apache.org/
>
> Benchmarking localhost (be patient)
> Completed 1024 requests
> Completed 2048 requests
> Completed 3072 requests
> Completed 4096 requests
> Completed 5120 requests
> Completed 6144 requests
> Completed 7168 requests
> Completed 8192 requests
> Completed 9216 requests
> Completed 10240 requests
> Finished 10240 requests
>
>
> Server Software: Jetty(6.1.16)
> Server Hostname: localhost
> Server Port: 8080
>
> Document Path: /wicketapp/products3/200
> Document Length: 34290 bytes
>
> Concurrency Level: 16
> Time taken for tests: 54.005 seconds
> Complete requests: 10240
> Failed requests: 6161
> (Connect: 0, Receive: 0, Length: 6161, Exceptions: 0)
> Write errors: 0
> Total transferred: 355066608 bytes
> HTML transferred: 352895728 bytes
> Requests per second: 189.61 [#/sec] (mean)
> Time per request: 84.383 [ms] (mean)
> Time per request: 5.274 [ms] (mean, across all concurrent requests)
> Transfer rate: 6420.58 [Kbytes/sec] received
>
> Connection Times (ms)
> min mean[+/-sd] median max
> Connect: 0 0 1.0 0 16
> Processing: 11 84 51.2 74 397
> Waiting: 8 77 49.4 66 397
> Total: 11 84 51.2 75 398
>
> Percentage of the requests served within a certain time (ms)
> 50% 75
> 66% 97
> 75% 113
> 80% 124
> 90% 154
> 95% 182
> 98% 217
> 99% 239
> 100% 398 (longest request)
>
> Now with 8 users:
>
> dashorst$ ab -c 8 -n 10240 http://localhost:8080/wicketapp/products3/200
> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
> Licensed to The Apache Software Foundation, http://www.apache.org/
>
> Benchmarking localhost (be patient)
> Completed 1024 requests
> Completed 2048 requests
> Completed 3072 requests
> Completed 4096 requests
> Completed 5120 requests
> Completed 6144 requests
> Completed 7168 requests
> Completed 8192 requests
> Completed 9216 requests
> Completed 10240 requests
> Finished 10240 requests
>
>
> Server Software: Jetty(6.1.16)
> Server Hostname: localhost
> Server Port: 8080
>
> Document Path: /wicketapp/products3/200
> Document Length: 34290 bytes
>
> Concurrency Level: 8
> Time taken for tests: 48.672 seconds
> Complete requests: 10240
> Failed requests: 6308
> (Connect: 0, Receive: 0, Length: 6308, Exceptions: 0)
> Write errors: 0
> Total transferred: 355046233 bytes
> HTML transferred: 352875353 bytes
> Requests per second: 210.39 [#/sec] (mean)
> Time per request: 38.025 [ms] (mean)
> Time per request: 4.753 [ms] (mean, across all concurrent requests)
> Transfer rate: 7123.72 [Kbytes/sec] received
>
> Connection Times (ms)
> min mean[+/-sd] median max
> Connect: 0 0 0.9 0 10
> Processing: 11 38 19.9 33 203
> Waiting: 9 35 18.6 30 203
> Total: 11 38 20.0 33 204
>
> Percentage of the requests served within a certain time (ms)
> 50% 33
> 66% 41
> 75% 47
> 80% 53
> 90% 65
> 95% 77
> 98% 93
> 99% 105
> 100% 204 (longest request)
>
> And with 4 users:
>
> dashorst$ ab -c 4 -n 10240 http://localhost:8080/wicketapp/products3/200
> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
> Licensed to The Apache Software Foundation, http://www.apache.org/
>
> Benchmarking localhost (be patient)
> Completed 1024 requests
> Completed 2048 requests
> Completed 3072 requests
> Completed 4096 requests
> Completed 5120 requests
> Completed 6144 requests
> Completed 7168 requests
> Completed 8192 requests
> Completed 9216 requests
> Completed 10240 requests
> Finished 10240 requests
>
>
> Server Software: Jetty(6.1.16)
> Server Hostname: localhost
> Server Port: 8080
>
> Document Path: /wicketapp/products3/200
> Document Length: 34290 bytes
>
> Concurrency Level: 4
> Time taken for tests: 45.238 seconds
> Complete requests: 10240
> Failed requests: 6134
> (Connect: 0, Receive: 0, Length: 6134, Exceptions: 0)
> Write errors: 0
> Total transferred: 355063755 bytes
> HTML transferred: 352892875 bytes
> Requests per second: 226.36 [#/sec] (mean)
> Time per request: 17.671 [ms] (mean)
> Time per request: 4.418 [ms] (mean, across all concurrent requests)
> Transfer rate: 7664.91 [Kbytes/sec] received
>
> Connection Times (ms)
> min mean[+/-sd] median max
> Connect: 0 0 0.2 0 7
> Processing: 11 17 2.4 18 43
> Waiting: 10 17 2.5 17 41
> Total: 11 18 2.4 18 43
>
> Percentage of the requests served within a certain time (ms)
> 50% 18
> 66% 18
> 75% 19
> 80% 19
> 90% 20
> 95% 21
> 98% 23
> 99% 25
> 100% 43 (longest request)
>
> And finally with 2 users:
>
> dashorst$ ab -c 2 -n 10240 http://localhost:8080/wicketapp/products3/200
> This is ApacheBench, Version 2.3 <$Revision: 655654 $>
> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
> Licensed to The Apache Software Foundation, http://www.apache.org/
>
> Benchmarking localhost (be patient)
> Completed 1024 requests
> Completed 2048 requests
> Completed 3072 requests
> Completed 4096 requests
> Completed 5120 requests
> Completed 6144 requests
> Completed 7168 requests
> Completed 8192 requests
> Completed 9216 requests
> Completed 10240 requests
> Finished 10240 requests
>
>
> Server Software: Jetty(6.1.16)
> Server Hostname: localhost
> Server Port: 8080
>
> Document Path: /wicketapp/products3/200
> Document Length: 34691 bytes
>
> Concurrency Level: 2
> Time taken for tests: 55.005 seconds
> Complete requests: 10240
> Failed requests: 7532
> (Connect: 0, Receive: 0, Length: 7532, Exceptions: 0)
> Write errors: 0
> Total transferred: 355060137 bytes
> HTML transferred: 352889257 bytes
> Requests per second: 186.16 [#/sec] (mean)
> Time per request: 10.743 [ms] (mean)
> Time per request: 5.372 [ms] (mean, across all concurrent requests)
> Transfer rate: 6303.74 [Kbytes/sec] received
>
> Connection Times (ms)
> min mean[+/-sd] median max
> Connect: 0 0 0.0 0 1
> Processing: 6 11 2.8 11 25
> Waiting: 6 10 2.8 10 22
> Total: 7 11 2.8 11 25
>
> Percentage of the requests served within a certain time (ms)
> 50% 11
> 66% 12
> 75% 13
> 80% 14
> 90% 15
> 95% 15
> 98% 16
> 99% 16
> 100% 25 (longest request)
>
> Looking at the results I think it is safe to say that with 16 users we
> hit the limit of my core i7, and with 4 users we hit the sweet spot
> (which is probably correct since I have 2 cores with each 2
> hyperthreads).
>
> I expected 4 users to perform similar to 2 users, but it appears to be
> slower. I'll run it using yourkit to see if there's a blocker
> somewhere.
>
> In my overall experience, 1.5-SNAPSHOT performs comparable to 1.4.
>
> Notes from this test:
> - we report wrong length in several cases
> - exceptions occur when a request is broken off
> - performance seems on par with 1.4
>
> Martijn
>
--
Become a Wicket expert, learn from the best: http://wicketinaction.com