You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tapestry.apache.org by engmail <en...@wrapsidy.com> on 2004/04/30 03:04:35 UTC
Performance Under Load
Please forgive my general ignorance, as I'm quite new to Tapestry, and
really know very little about it.
I'm wondering if anyone can recommend some best practices for getting
Tapestry to perform well under load. I'm interested in Tapestry-specific
tips, as opposed to general best practices for scalable web application
development.
My employer is evaluating Tapestry as a framework for a major project, the
main goal of which is to produce quite dense reports with many layers of
components. We love the framework, as it is extremely well designed and
fits very well with the programming model we're developing. But we need the
app to perform under heavy load, so I've been asked to do load testing
before we commit to Tapestry.
Unfortunately, we found that in our test Tapestry performed very poorly
compared to a JSP alternative. It did acceptably well in single threaded
tests, but scaled much more poorly when we fired off multiple threads
(throughput only 19% that of JSP). I therefore figured there were probably
synchronization bottlenecks in the framework. When we profiled the
application, we found that it was spending about 78% of its time in OGNL
code, so that's the first place I looked for synchronization issues. I
found 3 significant ones in the OGNL code, and was able to get a big
performance boost by eliminating them (if anyone is interested, I would be
glad to discuss how in another message).
But, even with the OGNL improvements, Tapestry's speed under load was only
42% that of JSP.
Our test case is basically a page with a bunch of two column tables on it.
We use forEach components to iterate through a set of data and output one
field in a <td> tag. We also keep track of the index count of how many
items we've output so we can put out a </tr> tag after every other record.
So, the test involves gets and sets of page properties. We built up an
equivalent version using JSP.
The key code in the template that gets repeated many times looks like this
(later we wrote a component that does the same thing):
<tr>
<span jwcid="@Foreach" source="ognl:dataSources" value="ognl:token"
index="ognl:index">
<td width="324"> <span jwcid="@Insert" value="ognl:token.desc"/></td>
<!- Output a closing tr if the index is an odd number -->
<span jwcid="@Insert" value="ognl:endTR" raw="true"/>
</span>
<!-Output a 2nd td and closing tr if we ended with an even index -->
<span jwcid="@Conditional" condition="ognl:index % 2 == 0">
<td> </td><span jwcid="@Insert" value="</tr>" raw="true"/>
</span>
The data that drives the page is created in a static block when the page
class is loaded and is then reused, so the creation of the data is not a
factor in the test.
We set up JMeter to make repeated calls to the page (i.e. invoke
http://.../test/app?service=page/Test).
Here are our results:
First, we ran a single threaded test (i.e. JMeter only fires of one request
at a time, no pause between requests).
JSP: 2500 requests, 8 ms per request avg, throughput = 5854
per minute
Tapestry 3.0: 2500 requests, 13 ms per request avg, throughput = 3687
per minute
Here we see the Tapestry solution is slower than JSP, but still performs
well enough for our needs. Tapestry's throughput is 63% of the JSP
solution. I assume this is mostly due to OGNL's use of reflection.
Then we ran a multi-threaded test. JMeter fires off 100 threads that each
request the page 25 times. Each thread pauses 300 ms between requests.
Takes 3 seconds to ramp up. Cookies are enabled, so once a session is
established it is used by that thread for the duration of the test.
JSP:
2500 requests, 457 ms per request avg, throughput = 6799 per minute
Tapestry 3.0 using OGNL 2.6.3 (distributed with Tapestry):
2500 requests, 4214 ms per request avg, throughput = 1285 per minute
Tapestry 3.0 using OGNL 2.6.5:
2500 requests, 4151 ms per request avg, throughput = 1279 per minute
Tapestry 3.0 using OGNL 2.6.5 with synch & object pooling removed:
2500 requests, 1635 ms per request avg, throughput = 2881 per minute
Under load, the Tapestry solution's throughput is 19% of JSP's. :-( With
synchronization and object pooling removed from OGNL, Tapestry's speed is
only 42% that of Tapestry. :-) The fact that JSP is still relatively faster
under load tells me there are likely synchronization bottlenecks in the
Tapestry code as well.
I also ran tests with 50 and 25 threads, and got similar results - the more
threads involved, the worse the Tapestry solution performed compared to JSP.
The test server was a 2.8GHz Win XP box running Tomcat 5.0.19 on Sun JDK
1.4.2_03. The test client was an equivalent machine.
I'm wondering if the Tapestry folks have any thoughts on this, or can point
me in a direction of further ways to improve performance. We really want to
use Tapestry!
Best,
Brian Stansberry
Re: Performance Under Load
Posted by Eric Schneider <er...@centralparksoftware.com>.
No complaints from the NHL people. Our performance has been great.
Keep in mind, the majority of our applications are stateless and we avoid
using complex ognl expressions in bindings. Probably makes a difference.
Eric
----- Original Message -----
From: "stephen smithstone" <sk...@lchost.co.uk>
To: "Tapestry users" <ta...@jakarta.apache.org>
Sent: Friday, April 30, 2004 6:31 AM
Subject: Re: Performance Under Load
> one of the people on the list points this to the nhl.com website which
> is based on tapestry and reports that tapesty handles it well there is
> a section on the wiki about it
> On 30 Apr 2004, at 02:04, engmail wrote:
>
> > Please forgive my general ignorance, as I'm quite new to Tapestry, and
> > really know very little about it.
> >
> >
> >
> > I'm wondering if anyone can recommend some best practices for getting
> > Tapestry to perform well under load. I'm interested in
> > Tapestry-specific
> > tips, as opposed to general best practices for scalable web application
> > development.
> >
> >
> >
> > My employer is evaluating Tapestry as a framework for a major project,
> > the
> > main goal of which is to produce quite dense reports with many layers
> > of
> > components. We love the framework, as it is extremely well designed
> > and
> > fits very well with the programming model we're developing. But we
> > need the
> > app to perform under heavy load, so I've been asked to do load testing
> > before we commit to Tapestry.
> >
> >
> >
> > Unfortunately, we found that in our test Tapestry performed very poorly
> > compared to a JSP alternative. It did acceptably well in single
> > threaded
> > tests, but scaled much more poorly when we fired off multiple threads
> > (throughput only 19% that of JSP). I therefore figured there were
> > probably
> > synchronization bottlenecks in the framework. When we profiled the
> > application, we found that it was spending about 78% of its time in
> > OGNL
> > code, so that's the first place I looked for synchronization issues. I
> > found 3 significant ones in the OGNL code, and was able to get a big
> > performance boost by eliminating them (if anyone is interested, I
> > would be
> > glad to discuss how in another message).
> >
> >
> >
> > But, even with the OGNL improvements, Tapestry's speed under load was
> > only
> > 42% that of JSP.
> >
> >
> >
> > Our test case is basically a page with a bunch of two column tables on
> > it.
> > We use forEach components to iterate through a set of data and output
> > one
> > field in a <td> tag. We also keep track of the index count of how many
> > items we've output so we can put out a </tr> tag after every other
> > record.
> > So, the test involves gets and sets of page properties. We built up an
> > equivalent version using JSP.
> >
> >
> >
> > The key code in the template that gets repeated many times looks like
> > this
> > (later we wrote a component that does the same thing):
> >
> >
> >
> > <tr>
> >
> > <span jwcid="@Foreach" source="ognl:dataSources" value="ognl:token"
> > index="ognl:index">
> >
> > <td width="324"> <span jwcid="@Insert"
> > value="ognl:token.desc"/></td>
> >
> > <!- Output a closing tr if the index is an odd number -->
> >
> > <span jwcid="@Insert" value="ognl:endTR" raw="true"/>
> >
> > </span>
> >
> > <!-Output a 2nd td and closing tr if we ended with an even index -->
> >
> > <span jwcid="@Conditional" condition="ognl:index % 2 == 0">
> >
> > <td> </td><span jwcid="@Insert" value="</tr>" raw="true"/>
> >
> > </span>
> >
> >
> >
> > The data that drives the page is created in a static block when the
> > page
> > class is loaded and is then reused, so the creation of the data is not
> > a
> > factor in the test.
> >
> >
> >
> > We set up JMeter to make repeated calls to the page (i.e. invoke
> > http://.../test/app?service=page/Test).
> >
> >
> >
> >
> >
> > Here are our results:
> >
> >
> >
> > First, we ran a single threaded test (i.e. JMeter only fires of one
> > request
> > at a time, no pause between requests).
> >
> >
> >
> > JSP: 2500 requests, 8 ms per request avg, throughput =
> > 5854
> > per minute
> >
> > Tapestry 3.0: 2500 requests, 13 ms per request avg, throughput =
> > 3687
> > per minute
> >
> >
> >
> > Here we see the Tapestry solution is slower than JSP, but still
> > performs
> > well enough for our needs. Tapestry's throughput is 63% of the JSP
> > solution. I assume this is mostly due to OGNL's use of reflection.
> >
> >
> >
> > Then we ran a multi-threaded test. JMeter fires off 100 threads that
> > each
> > request the page 25 times. Each thread pauses 300 ms between requests.
> > Takes 3 seconds to ramp up. Cookies are enabled, so once a session is
> > established it is used by that thread for the duration of the test.
> >
> >
> >
> > JSP:
> > 2500 requests, 457 ms per request avg, throughput = 6799 per minute
> >
> > Tapestry 3.0 using OGNL 2.6.3 (distributed with Tapestry):
> > 2500 requests, 4214 ms per request avg, throughput = 1285 per minute
> >
> > Tapestry 3.0 using OGNL 2.6.5:
> > 2500 requests, 4151 ms per request avg, throughput = 1279 per minute
> >
> > Tapestry 3.0 using OGNL 2.6.5 with synch & object pooling removed:
> > 2500 requests, 1635 ms per request avg, throughput = 2881 per minute
> >
> >
> >
> > Under load, the Tapestry solution's throughput is 19% of JSP's. :-(
> > With
> > synchronization and object pooling removed from OGNL, Tapestry's speed
> > is
> > only 42% that of Tapestry. :-) The fact that JSP is still relatively
> > faster
> > under load tells me there are likely synchronization bottlenecks in the
> > Tapestry code as well.
> >
> >
> >
> > I also ran tests with 50 and 25 threads, and got similar results - the
> > more
> > threads involved, the worse the Tapestry solution performed compared
> > to JSP.
> >
> >
> >
> > The test server was a 2.8GHz Win XP box running Tomcat 5.0.19 on Sun
> > JDK
> > 1.4.2_03. The test client was an equivalent machine.
> >
> >
> >
> > I'm wondering if the Tapestry folks have any thoughts on this, or can
> > point
> > me in a direction of further ways to improve performance. We really
> > want to
> > use Tapestry!
> >
> >
> >
> > Best,
> >
> > Brian Stansberry
> >
> >
> >
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tapestry-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tapestry-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: tapestry-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tapestry-user-help@jakarta.apache.org
RE: Performance Under Load
Posted by "Howard M. Lewis Ship" <hl...@comcast.net>.
These are interesting, and dissapointing, results. Looks like OGNL is gradually taking over
Tapestry. It is a difficult domain ... people keep demanding more, more, more flexibility,
complexity and power but also wanting more, more, more performance. Fortunately, geniuses like Drew
are around; he's taken up the bytecode enhancement hammer to improve OGNL preformance about 10x.
--
Howard M. Lewis Ship
Independent J2EE / Open-Source Java Consultant
Creator, Tapestry: Java Web Components
Creator, HiveMind
http://howardlewisship.com
> -----Original Message-----
> From: stephen smithstone [mailto:skullboxhouse@lchost.co.uk]
> Sent: Friday, April 30, 2004 6:31 AM
> To: Tapestry users
> Subject: Re: Performance Under Load
>
>
> one of the people on the list points this to the nhl.com
> website which
> is based on tapestry and reports that tapesty handles it well
> there is
> a section on the wiki about it
> On 30 Apr 2004, at 02:04, engmail wrote:
>
> > Please forgive my general ignorance, as I'm quite new to
> Tapestry, and
> > really know very little about it.
> >
> >
> >
> > I'm wondering if anyone can recommend some best practices
> for getting
> > Tapestry to perform well under load. I'm interested in
> > Tapestry-specific
> > tips, as opposed to general best practices for scalable web
> application
> > development.
> >
> >
> >
> > My employer is evaluating Tapestry as a framework for a
> major project,
> > the
> > main goal of which is to produce quite dense reports with
> many layers
> > of
> > components. We love the framework, as it is extremely well
> designed
> > and
> > fits very well with the programming model we're developing. But we
> > need the
> > app to perform under heavy load, so I've been asked to do
> load testing
> > before we commit to Tapestry.
> >
> >
> >
> > Unfortunately, we found that in our test Tapestry performed
> very poorly
> > compared to a JSP alternative. It did acceptably well in single
> > threaded
> > tests, but scaled much more poorly when we fired off
> multiple threads
> > (throughput only 19% that of JSP). I therefore figured there were
> > probably
> > synchronization bottlenecks in the framework. When we profiled the
> > application, we found that it was spending about 78% of its time in
> > OGNL
> > code, so that's the first place I looked for
> synchronization issues. I
> > found 3 significant ones in the OGNL code, and was able to get a big
> > performance boost by eliminating them (if anyone is interested, I
> > would be
> > glad to discuss how in another message).
> >
> >
> >
> > But, even with the OGNL improvements, Tapestry's speed
> under load was
> > only
> > 42% that of JSP.
> >
> >
> >
> > Our test case is basically a page with a bunch of two
> column tables on
> > it.
> > We use forEach components to iterate through a set of data
> and output
> > one
> > field in a <td> tag. We also keep track of the index count
> of how many
> > items we've output so we can put out a </tr> tag after every other
> > record.
> > So, the test involves gets and sets of page properties. We
> built up an
> > equivalent version using JSP.
> >
> >
> >
> > The key code in the template that gets repeated many times
> looks like
> > this
> > (later we wrote a component that does the same thing):
> >
> >
> >
> > <tr>
> >
> > <span jwcid="@Foreach" source="ognl:dataSources" value="ognl:token"
> > index="ognl:index">
> >
> > <td width="324"> <span jwcid="@Insert"
> > value="ognl:token.desc"/></td>
> >
> > <!- Output a closing tr if the index is an odd number -->
> >
> > <span jwcid="@Insert" value="ognl:endTR" raw="true"/>
> >
> > </span>
> >
> > <!-Output a 2nd td and closing tr if we ended with an even index -->
> >
> > <span jwcid="@Conditional" condition="ognl:index % 2 == 0">
> >
> > <td> </td><span jwcid="@Insert" value="</tr>"
> raw="true"/>
> >
> > </span>
> >
> >
> >
> > The data that drives the page is created in a static block when the
> > page
> > class is loaded and is then reused, so the creation of the
> data is not
> > a
> > factor in the test.
> >
> >
> >
> > We set up JMeter to make repeated calls to the page (i.e. invoke
> > http://.../test/app?service=page/Test).
> >
> >
> >
> >
> >
> > Here are our results:
> >
> >
> >
> > First, we ran a single threaded test (i.e. JMeter only fires of one
> > request
> > at a time, no pause between requests).
> >
> >
> >
> > JSP: 2500 requests, 8 ms per request avg,
> throughput =
> > 5854
> > per minute
> >
> > Tapestry 3.0: 2500 requests, 13 ms per request avg,
> throughput =
> > 3687
> > per minute
> >
> >
> >
> > Here we see the Tapestry solution is slower than JSP, but still
> > performs
> > well enough for our needs. Tapestry's throughput is 63% of the JSP
> > solution. I assume this is mostly due to OGNL's use of reflection.
> >
> >
> >
> > Then we ran a multi-threaded test. JMeter fires off 100
> threads that
> > each
> > request the page 25 times. Each thread pauses 300 ms
> between requests.
> > Takes 3 seconds to ramp up. Cookies are enabled, so once a
> session is
> > established it is used by that thread for the duration of the test.
> >
> >
> >
> > JSP:
> > 2500 requests, 457 ms per request avg, throughput = 6799 per minute
> >
> > Tapestry 3.0 using OGNL 2.6.3 (distributed with Tapestry):
> > 2500 requests, 4214 ms per request avg, throughput = 1285 per minute
> >
> > Tapestry 3.0 using OGNL 2.6.5:
> > 2500 requests, 4151 ms per request avg, throughput = 1279 per minute
> >
> > Tapestry 3.0 using OGNL 2.6.5 with synch & object pooling removed:
> > 2500 requests, 1635 ms per request avg, throughput = 2881 per minute
> >
> >
> >
> > Under load, the Tapestry solution's throughput is 19% of
> JSP's. :-(
> > With
> > synchronization and object pooling removed from OGNL,
> Tapestry's speed
> > is
> > only 42% that of Tapestry. :-) The fact that JSP is still
> relatively
> > faster
> > under load tells me there are likely synchronization
> bottlenecks in the
> > Tapestry code as well.
> >
> >
> >
> > I also ran tests with 50 and 25 threads, and got similar
> results - the
> > more
> > threads involved, the worse the Tapestry solution performed
> compared
> > to JSP.
> >
> >
> >
> > The test server was a 2.8GHz Win XP box running Tomcat
> 5.0.19 on Sun
> > JDK
> > 1.4.2_03. The test client was an equivalent machine.
> >
> >
> >
> > I'm wondering if the Tapestry folks have any thoughts on
> this, or can
> > point
> > me in a direction of further ways to improve performance.
> We really
> > want to
> > use Tapestry!
> >
> >
> >
> > Best,
> >
> > Brian Stansberry
> >
> >
> >
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tapestry-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tapestry-user-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tapestry-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tapestry-user-help@jakarta.apache.org
Re: Performance Under Load
Posted by stephen smithstone <sk...@lchost.co.uk>.
one of the people on the list points this to the nhl.com website which
is based on tapestry and reports that tapesty handles it well there is
a section on the wiki about it
On 30 Apr 2004, at 02:04, engmail wrote:
> Please forgive my general ignorance, as I'm quite new to Tapestry, and
> really know very little about it.
>
>
>
> I'm wondering if anyone can recommend some best practices for getting
> Tapestry to perform well under load. I'm interested in
> Tapestry-specific
> tips, as opposed to general best practices for scalable web application
> development.
>
>
>
> My employer is evaluating Tapestry as a framework for a major project,
> the
> main goal of which is to produce quite dense reports with many layers
> of
> components. We love the framework, as it is extremely well designed
> and
> fits very well with the programming model we're developing. But we
> need the
> app to perform under heavy load, so I've been asked to do load testing
> before we commit to Tapestry.
>
>
>
> Unfortunately, we found that in our test Tapestry performed very poorly
> compared to a JSP alternative. It did acceptably well in single
> threaded
> tests, but scaled much more poorly when we fired off multiple threads
> (throughput only 19% that of JSP). I therefore figured there were
> probably
> synchronization bottlenecks in the framework. When we profiled the
> application, we found that it was spending about 78% of its time in
> OGNL
> code, so that's the first place I looked for synchronization issues. I
> found 3 significant ones in the OGNL code, and was able to get a big
> performance boost by eliminating them (if anyone is interested, I
> would be
> glad to discuss how in another message).
>
>
>
> But, even with the OGNL improvements, Tapestry's speed under load was
> only
> 42% that of JSP.
>
>
>
> Our test case is basically a page with a bunch of two column tables on
> it.
> We use forEach components to iterate through a set of data and output
> one
> field in a <td> tag. We also keep track of the index count of how many
> items we've output so we can put out a </tr> tag after every other
> record.
> So, the test involves gets and sets of page properties. We built up an
> equivalent version using JSP.
>
>
>
> The key code in the template that gets repeated many times looks like
> this
> (later we wrote a component that does the same thing):
>
>
>
> <tr>
>
> <span jwcid="@Foreach" source="ognl:dataSources" value="ognl:token"
> index="ognl:index">
>
> <td width="324"> <span jwcid="@Insert"
> value="ognl:token.desc"/></td>
>
> <!- Output a closing tr if the index is an odd number -->
>
> <span jwcid="@Insert" value="ognl:endTR" raw="true"/>
>
> </span>
>
> <!-Output a 2nd td and closing tr if we ended with an even index -->
>
> <span jwcid="@Conditional" condition="ognl:index % 2 == 0">
>
> <td> </td><span jwcid="@Insert" value="</tr>" raw="true"/>
>
> </span>
>
>
>
> The data that drives the page is created in a static block when the
> page
> class is loaded and is then reused, so the creation of the data is not
> a
> factor in the test.
>
>
>
> We set up JMeter to make repeated calls to the page (i.e. invoke
> http://.../test/app?service=page/Test).
>
>
>
>
>
> Here are our results:
>
>
>
> First, we ran a single threaded test (i.e. JMeter only fires of one
> request
> at a time, no pause between requests).
>
>
>
> JSP: 2500 requests, 8 ms per request avg, throughput =
> 5854
> per minute
>
> Tapestry 3.0: 2500 requests, 13 ms per request avg, throughput =
> 3687
> per minute
>
>
>
> Here we see the Tapestry solution is slower than JSP, but still
> performs
> well enough for our needs. Tapestry's throughput is 63% of the JSP
> solution. I assume this is mostly due to OGNL's use of reflection.
>
>
>
> Then we ran a multi-threaded test. JMeter fires off 100 threads that
> each
> request the page 25 times. Each thread pauses 300 ms between requests.
> Takes 3 seconds to ramp up. Cookies are enabled, so once a session is
> established it is used by that thread for the duration of the test.
>
>
>
> JSP:
> 2500 requests, 457 ms per request avg, throughput = 6799 per minute
>
> Tapestry 3.0 using OGNL 2.6.3 (distributed with Tapestry):
> 2500 requests, 4214 ms per request avg, throughput = 1285 per minute
>
> Tapestry 3.0 using OGNL 2.6.5:
> 2500 requests, 4151 ms per request avg, throughput = 1279 per minute
>
> Tapestry 3.0 using OGNL 2.6.5 with synch & object pooling removed:
> 2500 requests, 1635 ms per request avg, throughput = 2881 per minute
>
>
>
> Under load, the Tapestry solution's throughput is 19% of JSP's. :-(
> With
> synchronization and object pooling removed from OGNL, Tapestry's speed
> is
> only 42% that of Tapestry. :-) The fact that JSP is still relatively
> faster
> under load tells me there are likely synchronization bottlenecks in the
> Tapestry code as well.
>
>
>
> I also ran tests with 50 and 25 threads, and got similar results - the
> more
> threads involved, the worse the Tapestry solution performed compared
> to JSP.
>
>
>
> The test server was a 2.8GHz Win XP box running Tomcat 5.0.19 on Sun
> JDK
> 1.4.2_03. The test client was an equivalent machine.
>
>
>
> I'm wondering if the Tapestry folks have any thoughts on this, or can
> point
> me in a direction of further ways to improve performance. We really
> want to
> use Tapestry!
>
>
>
> Best,
>
> Brian Stansberry
>
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tapestry-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tapestry-user-help@jakarta.apache.org