You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@esme.apache.org by "Hirsch, Richard" <ri...@siemens.com> on 2009/01/08 05:54:15 UTC
Details from Stax Load test
Here are some details for a very first performance test on Stax for the ESME server. Daniel tried with 1000 concurrent connections and then the server started having some problems. Take a look at the enclosed stack trace and you will see that towards the end there were problems with the threads. I'm also enclosing a picture of the Stax performance indicators. I don't know the exact dimensions of the test but I'm sure Daniel will provide them soon.
Load tests are critical if we are to succeed in enterprises. They are also critical when customers need sizing information. I assume that they should also be useful for the lift framework.
D.
________________________________
From: Spike Washburn [mailto:spike@staxnetworks.com]
Sent: Wed 1/7/2009 23:13
To: Hirsch, Richard
Subject: Re: Stax account
The activity died back down for a bit, but then the app started sucking up memory and CPU like it was stuck in a loop. When we checked the logs, we saw it was throwing out of memory exceptions. Since the app was clearly in a death spiral, we took a JVM stack dump and then restarted the app. I have attached the last part of the appserver log if you want to review it.
Also I noticed from the log that your app is getting warnings about including the servlet-api-2.5.jar in WEB-INF/lib. This is not necessary since the Servlet API classes are part of the classpath provided by the appserver.
Before your app died we were seeing upwards of 1000 concurrent connections to your app. Please let me know if you were expecting this load or if it was some kind of external attack against your app.
Thanks,
Spike
On Wed, Jan 7, 2009 at 1:09 PM, Spike Washburn <sp...@staxnetworks.com> wrote:
Hi Dick,
We just noticed a major spike in activity on your application (id: DickHirsch/esmecloudserver). I just wanted to check with you to see if you were doing some load testing or if this was some kind of external attack on your webapp.
Thanks,
Spike
RE: Details from Stax Load test
Posted by "Hirsch, Richard" <ri...@siemens.com>.
I'm unfamilar with jetty. I'll ping Stax if they have jetty-based environments. What I'd like to be able to do is offer customers various sizing and hardware configuration options. Ideal would be a some sort of a matrix where users can describe what their requirements are (# of users, use of container features, etc.) and have them know whether to use jetty, tomcat or another application server.
D.
________________________________
From: David Pollak [mailto:feeder.of.the.bears@gmail.com]
Sent: Fri 1/9/2009 05:54
To: esme-dev@incubator.apache.org
Subject: Re: Details from Stax Load test
Dick,
Unfortunately, Tomcat is very weak when it comes to support for long
polling. One way or another, that's going to hurt us until Tomcat supports
the Servlet 3.0 spec. I prefer using Jetty because the Jetty team has led
the way in support of long polling style coding. This year, NetWeaver's
going to catch up. I certainly hope Tomcat catches up soon too, but I think
it's important to demonstrate how ESME will perform using the right
environment (one that's been designed for applications like ESME) rather
than a lowest common denominator, like Tomcat.
Thanks,
David
On Wed, Jan 7, 2009 at 11:01 PM, Hirsch, Richard <richard.hirsch@siemens.com
> wrote:
> I'd like to try load tests later in the Stax Environment when Scala 2.7.3
> is released with the new ESME Core from apache. Since Daniel's tests are
> based on the REST-API, we don't have to wait for the UI to be stable or
> finished.
>
> Regarding the use of jetty vs tomcat in such tests: The question is which
> tool is more likely to be used by potential users. I'm assuming tomcat,
> primarily because I've never seen jetty in a productive system in the
> enterprise. I also don't know if Stax supports jetty. Ideal would be a
> comparison between the two.
>
> What I think is great is the ability to use Stax to do load tests. The
> environment is perfect for such tasks. We should probably use a cluster the
> next time we test to see how that influences test results.
>
> We will publish our results from these tests. I know of no other
> microblogging tool (or many other tools irregardless of type) that
> proactively publishes such results. By publishing such reports, we will
> enhance our legitimacy in the community and the marketplace. We also
> spotlight the use of cloud computing as a potential hosting environment for
> potential users.
>
> I'm in contact with Spike Washburn (CEO from Stax) and I think we have a
> good contact there for future reference.
>
> D.
>
> ________________________________
>
> From: David Pollak [mailto:feeder.of.the.bears@gmail.com]
> Sent: Thu 1/8/2009 06:32
> To: esme-dev@incubator.apache.org
> Subject: Re: Details from Stax Load test
>
>
>
> Dick,
> Tomcat is a less than optimal platform for high concurrent load. It does
> not have the same continuations mechanism that Jetty has. All my high load
> tests are done on Jetty. With that being said, ESME's long polling for the
> HTTP APIs does not take advantage of Jetty's continuations yet. That's on
> my to-do list, but to date has not been a high priority.
>
> Another issue is that there's a problem with Scala Actors and memory in
> Scala 2.7.2. The Scala team is releasing Scala 2.7.3 this week or next to
> cure the memory problems.
>
> Also, the continuations that Jetty currently supports are part of the
> Servlet 3.0 spec and should be part of NetWeaver this year.
>
> Thanks,
>
> David
>
> On Wed, Jan 7, 2009 at 8:54 PM, Hirsch, Richard
> <ri...@siemens.com>wrote:
>
> > Here are some details for a very first performance test on Stax for the
> > ESME server. Daniel tried with 1000 concurrent connections and then the
> > server started having some problems. Take a look at the enclosed stack
> trace
> > and you will see that towards the end there were problems with the
> threads.
> > I'm also enclosing a picture of the Stax performance indicators. I don't
> > know the exact dimensions of the test but I'm sure Daniel will provide
> them
> > soon.
> >
> > Load tests are critical if we are to succeed in enterprises. They are
> also
> > critical when customers need sizing information. I assume that they
> should
> > also be useful for the lift framework.
> >
> > D.
> >
> > ________________________________
> >
> > From: Spike Washburn [mailto:spike@staxnetworks.com]
> > Sent: Wed 1/7/2009 23:13
> > To: Hirsch, Richard
> > Subject: Re: Stax account
> >
> >
> > The activity died back down for a bit, but then the app started sucking
> up
> > memory and CPU like it was stuck in a loop. When we checked the logs, we
> > saw it was throwing out of memory exceptions. Since the app was clearly
> in
> > a death spiral, we took a JVM stack dump and then restarted the app. I
> have
> > attached the last part of the appserver log if you want to review it.
> >
> > Also I noticed from the log that your app is getting warnings about
> > including the servlet-api-2.5.jar in WEB-INF/lib. This is not necessary
> > since the Servlet API classes are part of the classpath provided by the
> > appserver.
> >
> > Before your app died we were seeing upwards of 1000 concurrent
> connections
> > to your app. Please let me know if you were expecting this load or if it
> > was some kind of external attack against your app.
> >
> > Thanks,
> > Spike
> >
> >
> > On Wed, Jan 7, 2009 at 1:09 PM, Spike Washburn <sp...@staxnetworks.com>
> > wrote:
> >
> >
> > Hi Dick,
> >
> > We just noticed a major spike in activity on your application (id:
> > DickHirsch/esmecloudserver). I just wanted to check with you to see if
> you
> > were doing some load testing or if this was some kind of external attack
> on
> > your webapp.
> >
> > Thanks,
> > Spike
> >
> >
> >
> >
>
>
> --
> Lift, the simply functional web framework http://liftweb.net <http://liftweb.net/> <
> http://liftweb.net/>
> Collaborative Task Management http://much4.us <http://much4.us/> <http://much4.us/>
> Follow me: http://twitter.com/dpp
> Git some: http://github.com/dpp
>
>
>
--
Lift, the simply functional web framework http://liftweb.net <http://liftweb.net/>
Collaborative Task Management http://much4.us <http://much4.us/>
Follow me: http://twitter.com/dpp
Git some: http://github.com/dpp
Re: Details from Stax Load test
Posted by David Pollak <fe...@gmail.com>.
Dick,
Unfortunately, Tomcat is very weak when it comes to support for long
polling. One way or another, that's going to hurt us until Tomcat supports
the Servlet 3.0 spec. I prefer using Jetty because the Jetty team has led
the way in support of long polling style coding. This year, NetWeaver's
going to catch up. I certainly hope Tomcat catches up soon too, but I think
it's important to demonstrate how ESME will perform using the right
environment (one that's been designed for applications like ESME) rather
than a lowest common denominator, like Tomcat.
Thanks,
David
On Wed, Jan 7, 2009 at 11:01 PM, Hirsch, Richard <richard.hirsch@siemens.com
> wrote:
> I'd like to try load tests later in the Stax Environment when Scala 2.7.3
> is released with the new ESME Core from apache. Since Daniel's tests are
> based on the REST-API, we don't have to wait for the UI to be stable or
> finished.
>
> Regarding the use of jetty vs tomcat in such tests: The question is which
> tool is more likely to be used by potential users. I'm assuming tomcat,
> primarily because I've never seen jetty in a productive system in the
> enterprise. I also don't know if Stax supports jetty. Ideal would be a
> comparison between the two.
>
> What I think is great is the ability to use Stax to do load tests. The
> environment is perfect for such tasks. We should probably use a cluster the
> next time we test to see how that influences test results.
>
> We will publish our results from these tests. I know of no other
> microblogging tool (or many other tools irregardless of type) that
> proactively publishes such results. By publishing such reports, we will
> enhance our legitimacy in the community and the marketplace. We also
> spotlight the use of cloud computing as a potential hosting environment for
> potential users.
>
> I'm in contact with Spike Washburn (CEO from Stax) and I think we have a
> good contact there for future reference.
>
> D.
>
> ________________________________
>
> From: David Pollak [mailto:feeder.of.the.bears@gmail.com]
> Sent: Thu 1/8/2009 06:32
> To: esme-dev@incubator.apache.org
> Subject: Re: Details from Stax Load test
>
>
>
> Dick,
> Tomcat is a less than optimal platform for high concurrent load. It does
> not have the same continuations mechanism that Jetty has. All my high load
> tests are done on Jetty. With that being said, ESME's long polling for the
> HTTP APIs does not take advantage of Jetty's continuations yet. That's on
> my to-do list, but to date has not been a high priority.
>
> Another issue is that there's a problem with Scala Actors and memory in
> Scala 2.7.2. The Scala team is releasing Scala 2.7.3 this week or next to
> cure the memory problems.
>
> Also, the continuations that Jetty currently supports are part of the
> Servlet 3.0 spec and should be part of NetWeaver this year.
>
> Thanks,
>
> David
>
> On Wed, Jan 7, 2009 at 8:54 PM, Hirsch, Richard
> <ri...@siemens.com>wrote:
>
> > Here are some details for a very first performance test on Stax for the
> > ESME server. Daniel tried with 1000 concurrent connections and then the
> > server started having some problems. Take a look at the enclosed stack
> trace
> > and you will see that towards the end there were problems with the
> threads.
> > I'm also enclosing a picture of the Stax performance indicators. I don't
> > know the exact dimensions of the test but I'm sure Daniel will provide
> them
> > soon.
> >
> > Load tests are critical if we are to succeed in enterprises. They are
> also
> > critical when customers need sizing information. I assume that they
> should
> > also be useful for the lift framework.
> >
> > D.
> >
> > ________________________________
> >
> > From: Spike Washburn [mailto:spike@staxnetworks.com]
> > Sent: Wed 1/7/2009 23:13
> > To: Hirsch, Richard
> > Subject: Re: Stax account
> >
> >
> > The activity died back down for a bit, but then the app started sucking
> up
> > memory and CPU like it was stuck in a loop. When we checked the logs, we
> > saw it was throwing out of memory exceptions. Since the app was clearly
> in
> > a death spiral, we took a JVM stack dump and then restarted the app. I
> have
> > attached the last part of the appserver log if you want to review it.
> >
> > Also I noticed from the log that your app is getting warnings about
> > including the servlet-api-2.5.jar in WEB-INF/lib. This is not necessary
> > since the Servlet API classes are part of the classpath provided by the
> > appserver.
> >
> > Before your app died we were seeing upwards of 1000 concurrent
> connections
> > to your app. Please let me know if you were expecting this load or if it
> > was some kind of external attack against your app.
> >
> > Thanks,
> > Spike
> >
> >
> > On Wed, Jan 7, 2009 at 1:09 PM, Spike Washburn <sp...@staxnetworks.com>
> > wrote:
> >
> >
> > Hi Dick,
> >
> > We just noticed a major spike in activity on your application (id:
> > DickHirsch/esmecloudserver). I just wanted to check with you to see if
> you
> > were doing some load testing or if this was some kind of external attack
> on
> > your webapp.
> >
> > Thanks,
> > Spike
> >
> >
> >
> >
>
>
> --
> Lift, the simply functional web framework http://liftweb.net <
> http://liftweb.net/>
> Collaborative Task Management http://much4.us <http://much4.us/>
> Follow me: http://twitter.com/dpp
> Git some: http://github.com/dpp
>
>
>
--
Lift, the simply functional web framework http://liftweb.net
Collaborative Task Management http://much4.us
Follow me: http://twitter.com/dpp
Git some: http://github.com/dpp
RE: Details from Stax Load test
Posted by "Hirsch, Richard" <ri...@siemens.com>.
I'd like to try load tests later in the Stax Environment when Scala 2.7.3 is released with the new ESME Core from apache. Since Daniel's tests are based on the REST-API, we don't have to wait for the UI to be stable or finished.
Regarding the use of jetty vs tomcat in such tests: The question is which tool is more likely to be used by potential users. I'm assuming tomcat, primarily because I've never seen jetty in a productive system in the enterprise. I also don't know if Stax supports jetty. Ideal would be a comparison between the two.
What I think is great is the ability to use Stax to do load tests. The environment is perfect for such tasks. We should probably use a cluster the next time we test to see how that influences test results.
We will publish our results from these tests. I know of no other microblogging tool (or many other tools irregardless of type) that proactively publishes such results. By publishing such reports, we will enhance our legitimacy in the community and the marketplace. We also spotlight the use of cloud computing as a potential hosting environment for potential users.
I'm in contact with Spike Washburn (CEO from Stax) and I think we have a good contact there for future reference.
D.
________________________________
From: David Pollak [mailto:feeder.of.the.bears@gmail.com]
Sent: Thu 1/8/2009 06:32
To: esme-dev@incubator.apache.org
Subject: Re: Details from Stax Load test
Dick,
Tomcat is a less than optimal platform for high concurrent load. It does
not have the same continuations mechanism that Jetty has. All my high load
tests are done on Jetty. With that being said, ESME's long polling for the
HTTP APIs does not take advantage of Jetty's continuations yet. That's on
my to-do list, but to date has not been a high priority.
Another issue is that there's a problem with Scala Actors and memory in
Scala 2.7.2. The Scala team is releasing Scala 2.7.3 this week or next to
cure the memory problems.
Also, the continuations that Jetty currently supports are part of the
Servlet 3.0 spec and should be part of NetWeaver this year.
Thanks,
David
On Wed, Jan 7, 2009 at 8:54 PM, Hirsch, Richard
<ri...@siemens.com>wrote:
> Here are some details for a very first performance test on Stax for the
> ESME server. Daniel tried with 1000 concurrent connections and then the
> server started having some problems. Take a look at the enclosed stack trace
> and you will see that towards the end there were problems with the threads.
> I'm also enclosing a picture of the Stax performance indicators. I don't
> know the exact dimensions of the test but I'm sure Daniel will provide them
> soon.
>
> Load tests are critical if we are to succeed in enterprises. They are also
> critical when customers need sizing information. I assume that they should
> also be useful for the lift framework.
>
> D.
>
> ________________________________
>
> From: Spike Washburn [mailto:spike@staxnetworks.com]
> Sent: Wed 1/7/2009 23:13
> To: Hirsch, Richard
> Subject: Re: Stax account
>
>
> The activity died back down for a bit, but then the app started sucking up
> memory and CPU like it was stuck in a loop. When we checked the logs, we
> saw it was throwing out of memory exceptions. Since the app was clearly in
> a death spiral, we took a JVM stack dump and then restarted the app. I have
> attached the last part of the appserver log if you want to review it.
>
> Also I noticed from the log that your app is getting warnings about
> including the servlet-api-2.5.jar in WEB-INF/lib. This is not necessary
> since the Servlet API classes are part of the classpath provided by the
> appserver.
>
> Before your app died we were seeing upwards of 1000 concurrent connections
> to your app. Please let me know if you were expecting this load or if it
> was some kind of external attack against your app.
>
> Thanks,
> Spike
>
>
> On Wed, Jan 7, 2009 at 1:09 PM, Spike Washburn <sp...@staxnetworks.com>
> wrote:
>
>
> Hi Dick,
>
> We just noticed a major spike in activity on your application (id:
> DickHirsch/esmecloudserver). I just wanted to check with you to see if you
> were doing some load testing or if this was some kind of external attack on
> your webapp.
>
> Thanks,
> Spike
>
>
>
>
--
Lift, the simply functional web framework http://liftweb.net <http://liftweb.net/>
Collaborative Task Management http://much4.us <http://much4.us/>
Follow me: http://twitter.com/dpp
Git some: http://github.com/dpp
Re: Details from Stax Load test
Posted by David Pollak <fe...@gmail.com>.
On Thu, Jan 8, 2009 at 8:54 PM, Hirsch, Richard
<ri...@siemens.com>wrote:
> @David: is the REST-API functional in the present apache version?.
Yes.
> Could we use this for performance tests? I'
Yes.
> d like to wait for the new scala library before testing but I'd curious to
> know if the REST-API has changed in any way.
No, not yet.
>
>
> D.
>
> ________________________________
>
> From: David Pollak [mailto:feeder.of.the.bears@gmail.com]
> Sent: Thu 1/8/2009 06:32
> To: esme-dev@incubator.apache.org
> Subject: Re: Details from Stax Load test
>
>
>
> Dick,
> Tomcat is a less than optimal platform for high concurrent load. It does
> not have the same continuations mechanism that Jetty has. All my high load
> tests are done on Jetty. With that being said, ESME's long polling for the
> HTTP APIs does not take advantage of Jetty's continuations yet. That's on
> my to-do list, but to date has not been a high priority.
>
> Another issue is that there's a problem with Scala Actors and memory in
> Scala 2.7.2. The Scala team is releasing Scala 2.7.3 this week or next to
> cure the memory problems.
>
> Also, the continuations that Jetty currently supports are part of the
> Servlet 3.0 spec and should be part of NetWeaver this year.
>
> Thanks,
>
> David
>
> On Wed, Jan 7, 2009 at 8:54 PM, Hirsch, Richard
> <ri...@siemens.com>wrote:
>
> > Here are some details for a very first performance test on Stax for the
> > ESME server. Daniel tried with 1000 concurrent connections and then the
> > server started having some problems. Take a look at the enclosed stack
> trace
> > and you will see that towards the end there were problems with the
> threads.
> > I'm also enclosing a picture of the Stax performance indicators. I don't
> > know the exact dimensions of the test but I'm sure Daniel will provide
> them
> > soon.
> >
> > Load tests are critical if we are to succeed in enterprises. They are
> also
> > critical when customers need sizing information. I assume that they
> should
> > also be useful for the lift framework.
> >
> > D.
> >
> > ________________________________
> >
> > From: Spike Washburn [mailto:spike@staxnetworks.com]
> > Sent: Wed 1/7/2009 23:13
> > To: Hirsch, Richard
> > Subject: Re: Stax account
> >
> >
> > The activity died back down for a bit, but then the app started sucking
> up
> > memory and CPU like it was stuck in a loop. When we checked the logs, we
> > saw it was throwing out of memory exceptions. Since the app was clearly
> in
> > a death spiral, we took a JVM stack dump and then restarted the app. I
> have
> > attached the last part of the appserver log if you want to review it.
> >
> > Also I noticed from the log that your app is getting warnings about
> > including the servlet-api-2.5.jar in WEB-INF/lib. This is not necessary
> > since the Servlet API classes are part of the classpath provided by the
> > appserver.
> >
> > Before your app died we were seeing upwards of 1000 concurrent
> connections
> > to your app. Please let me know if you were expecting this load or if it
> > was some kind of external attack against your app.
> >
> > Thanks,
> > Spike
> >
> >
> > On Wed, Jan 7, 2009 at 1:09 PM, Spike Washburn <sp...@staxnetworks.com>
> > wrote:
> >
> >
> > Hi Dick,
> >
> > We just noticed a major spike in activity on your application (id:
> > DickHirsch/esmecloudserver). I just wanted to check with you to see if
> you
> > were doing some load testing or if this was some kind of external attack
> on
> > your webapp.
> >
> > Thanks,
> > Spike
> >
> >
> >
> >
>
>
> --
> Lift, the simply functional web framework http://liftweb.net <
> http://liftweb.net/>
> Collaborative Task Management http://much4.us <http://much4.us/>
> Follow me: http://twitter.com/dpp
> Git some: http://github.com/dpp
>
>
>
--
Lift, the simply functional web framework http://liftweb.net
Collaborative Task Management http://much4.us
Follow me: http://twitter.com/dpp
Git some: http://github.com/dpp
RE: Details from Stax Load test
Posted by "Hirsch, Richard" <ri...@siemens.com>.
@David: is the REST-API functional in the present apache version?. Could we use this for performance tests? I'd like to wait for the new scala library before testing but I'd curious to know if the REST-API has changed in any way.
D.
________________________________
From: David Pollak [mailto:feeder.of.the.bears@gmail.com]
Sent: Thu 1/8/2009 06:32
To: esme-dev@incubator.apache.org
Subject: Re: Details from Stax Load test
Dick,
Tomcat is a less than optimal platform for high concurrent load. It does
not have the same continuations mechanism that Jetty has. All my high load
tests are done on Jetty. With that being said, ESME's long polling for the
HTTP APIs does not take advantage of Jetty's continuations yet. That's on
my to-do list, but to date has not been a high priority.
Another issue is that there's a problem with Scala Actors and memory in
Scala 2.7.2. The Scala team is releasing Scala 2.7.3 this week or next to
cure the memory problems.
Also, the continuations that Jetty currently supports are part of the
Servlet 3.0 spec and should be part of NetWeaver this year.
Thanks,
David
On Wed, Jan 7, 2009 at 8:54 PM, Hirsch, Richard
<ri...@siemens.com>wrote:
> Here are some details for a very first performance test on Stax for the
> ESME server. Daniel tried with 1000 concurrent connections and then the
> server started having some problems. Take a look at the enclosed stack trace
> and you will see that towards the end there were problems with the threads.
> I'm also enclosing a picture of the Stax performance indicators. I don't
> know the exact dimensions of the test but I'm sure Daniel will provide them
> soon.
>
> Load tests are critical if we are to succeed in enterprises. They are also
> critical when customers need sizing information. I assume that they should
> also be useful for the lift framework.
>
> D.
>
> ________________________________
>
> From: Spike Washburn [mailto:spike@staxnetworks.com]
> Sent: Wed 1/7/2009 23:13
> To: Hirsch, Richard
> Subject: Re: Stax account
>
>
> The activity died back down for a bit, but then the app started sucking up
> memory and CPU like it was stuck in a loop. When we checked the logs, we
> saw it was throwing out of memory exceptions. Since the app was clearly in
> a death spiral, we took a JVM stack dump and then restarted the app. I have
> attached the last part of the appserver log if you want to review it.
>
> Also I noticed from the log that your app is getting warnings about
> including the servlet-api-2.5.jar in WEB-INF/lib. This is not necessary
> since the Servlet API classes are part of the classpath provided by the
> appserver.
>
> Before your app died we were seeing upwards of 1000 concurrent connections
> to your app. Please let me know if you were expecting this load or if it
> was some kind of external attack against your app.
>
> Thanks,
> Spike
>
>
> On Wed, Jan 7, 2009 at 1:09 PM, Spike Washburn <sp...@staxnetworks.com>
> wrote:
>
>
> Hi Dick,
>
> We just noticed a major spike in activity on your application (id:
> DickHirsch/esmecloudserver). I just wanted to check with you to see if you
> were doing some load testing or if this was some kind of external attack on
> your webapp.
>
> Thanks,
> Spike
>
>
>
>
--
Lift, the simply functional web framework http://liftweb.net <http://liftweb.net/>
Collaborative Task Management http://much4.us <http://much4.us/>
Follow me: http://twitter.com/dpp
Git some: http://github.com/dpp
Re: Details from Stax Load test
Posted by David Pollak <fe...@gmail.com>.
On Wed, Jan 7, 2009 at 11:28 PM, Hirsch, Richard <richard.hirsch@siemens.com
> wrote:
> Once Daniel has composed the figures, I will blog about them on
> blog.esme.us.
>
> Anyone got any problems with that?
That a good thing.
>
>
> D.
>
> ________________________________
>
> From: Daniel Koller [mailto:dakoller@googlemail.com]
> Sent: Thu 1/8/2009 08:21
> To: esme-dev@incubator.apache.org
> Subject: Re: Details from Stax Load test
>
>
>
> Good morning.
>
> I want to give you a short overview about the load test I did yesterday
> evening:
>
> ESME@Stax was approached by up to nine parallel Java applications threads,
> which used the REST api to logon, to post a message and then to logoff
> again. This happens for each of the threads about 250-500 times. This test
> is shown in the pic attached by Dick: it is the high level plateau shown at
> process memory. I measured the time for each of the mentioned steps, and
> also the total time for the call.
>
> I'll post facts and figures, when I compiled them and changed the test
> scripts, but here are the first
> Observations:
> - We had the exceptions Dick mentioned, besides from these errors the
> response times were on the same (good) level for a long time.
> - Sometimes there are subsequent calls which take longer (about 3-5x), but
> performance comes back to the previous level after short time. The reasons
> for these longer function calls has to be investigated. (maybe related to
> the mentioned Scala topics in Davids mail before)
> - During earlier tests, I found that the performance of the ESME Web UI
> goes
> down under heavy REST load.
>
> Next steps:
> - I will include in the test scripts a test step to simulate a call the the
> Web UI (to get the complete picture of ESME performance)
> - Yesterday I tested one user which is logging in over and over again, but
> this user is not followed by anybody. Also the receiving messages step is
> not tested. --> this means creation of a test script, which impersonates
> more different users, which are also followed by a different number of
> other
> users.
> - Also receiving messages hast to be incorporated into the test script.
>
> That's it for now,
>
> Daniel
>
>
>
>
>
> On Thu, Jan 8, 2009 at 6:32 AM, David Pollak
> <fe...@gmail.com>wrote:
>
> > Dick,
> > Tomcat is a less than optimal platform for high concurrent load. It does
> > not have the same continuations mechanism that Jetty has. All my high
> load
> > tests are done on Jetty. With that being said, ESME's long polling for
> the
> > HTTP APIs does not take advantage of Jetty's continuations yet. That's
> on
> > my to-do list, but to date has not been a high priority.
> >
> > Another issue is that there's a problem with Scala Actors and memory in
> > Scala 2.7.2. The Scala team is releasing Scala 2.7.3 this week or next
> to
> > cure the memory problems.
> >
> > Also, the continuations that Jetty currently supports are part of the
> > Servlet 3.0 spec and should be part of NetWeaver this year.
> >
> > Thanks,
> >
> > David
> >
> > On Wed, Jan 7, 2009 at 8:54 PM, Hirsch, Richard
> > <ri...@siemens.com>wrote:
> >
> > > Here are some details for a very first performance test on Stax for the
> > > ESME server. Daniel tried with 1000 concurrent connections and then the
> > > server started having some problems. Take a look at the enclosed stack
> > trace
> > > and you will see that towards the end there were problems with the
> > threads.
> > > I'm also enclosing a picture of the Stax performance indicators. I
> don't
> > > know the exact dimensions of the test but I'm sure Daniel will provide
> > them
> > > soon.
> > >
> > > Load tests are critical if we are to succeed in enterprises. They are
> > also
> > > critical when customers need sizing information. I assume that they
> > should
> > > also be useful for the lift framework.
> > >
> > > D.
> > >
> > > ________________________________
> > >
> > > From: Spike Washburn [mailto:spike@staxnetworks.com]
> > > Sent: Wed 1/7/2009 23:13
> > > To: Hirsch, Richard
> > > Subject: Re: Stax account
> > >
> > >
> > > The activity died back down for a bit, but then the app started sucking
> > up
> > > memory and CPU like it was stuck in a loop. When we checked the logs,
> we
> > > saw it was throwing out of memory exceptions. Since the app was
> clearly
> > in
> > > a death spiral, we took a JVM stack dump and then restarted the app. I
> > have
> > > attached the last part of the appserver log if you want to review it.
> > >
> > > Also I noticed from the log that your app is getting warnings about
> > > including the servlet-api-2.5.jar in WEB-INF/lib. This is not necessary
> > > since the Servlet API classes are part of the classpath provided by the
> > > appserver.
> > >
> > > Before your app died we were seeing upwards of 1000 concurrent
> > connections
> > > to your app. Please let me know if you were expecting this load or if
> it
> > > was some kind of external attack against your app.
> > >
> > > Thanks,
> > > Spike
> > >
> > >
> > > On Wed, Jan 7, 2009 at 1:09 PM, Spike Washburn <spike@staxnetworks.com
> >
> > > wrote:
> > >
> > >
> > > Hi Dick,
> > >
> > > We just noticed a major spike in activity on your application
> (id:
> > > DickHirsch/esmecloudserver). I just wanted to check with you to see if
> > you
> > > were doing some load testing or if this was some kind of external
> attack
> > on
> > > your webapp.
> > >
> > > Thanks,
> > > Spike
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Lift, the simply functional web framework http://liftweb.net <
> http://liftweb.net/>
> > Collaborative Task Management http://much4.us <http://much4.us/>
> > Follow me: http://twitter.com/dpp
> > Git some: http://github.com/dpp
> >
>
>
>
> --
> ---
> Daniel Koller
> Jahnstrasse 20
> 80469 München * dakoller@googlemail.com
>
>
>
--
Lift, the simply functional web framework http://liftweb.net
Collaborative Task Management http://much4.us
Follow me: http://twitter.com/dpp
Git some: http://github.com/dpp
RE: Details from Stax Load test
Posted by "Hirsch, Richard" <ri...@siemens.com>.
Once Daniel has composed the figures, I will blog about them on blog.esme.us.
Anyone got any problems with that?
D.
________________________________
From: Daniel Koller [mailto:dakoller@googlemail.com]
Sent: Thu 1/8/2009 08:21
To: esme-dev@incubator.apache.org
Subject: Re: Details from Stax Load test
Good morning.
I want to give you a short overview about the load test I did yesterday
evening:
ESME@Stax was approached by up to nine parallel Java applications threads,
which used the REST api to logon, to post a message and then to logoff
again. This happens for each of the threads about 250-500 times. This test
is shown in the pic attached by Dick: it is the high level plateau shown at
process memory. I measured the time for each of the mentioned steps, and
also the total time for the call.
I'll post facts and figures, when I compiled them and changed the test
scripts, but here are the first
Observations:
- We had the exceptions Dick mentioned, besides from these errors the
response times were on the same (good) level for a long time.
- Sometimes there are subsequent calls which take longer (about 3-5x), but
performance comes back to the previous level after short time. The reasons
for these longer function calls has to be investigated. (maybe related to
the mentioned Scala topics in Davids mail before)
- During earlier tests, I found that the performance of the ESME Web UI goes
down under heavy REST load.
Next steps:
- I will include in the test scripts a test step to simulate a call the the
Web UI (to get the complete picture of ESME performance)
- Yesterday I tested one user which is logging in over and over again, but
this user is not followed by anybody. Also the receiving messages step is
not tested. --> this means creation of a test script, which impersonates
more different users, which are also followed by a different number of other
users.
- Also receiving messages hast to be incorporated into the test script.
That's it for now,
Daniel
On Thu, Jan 8, 2009 at 6:32 AM, David Pollak
<fe...@gmail.com>wrote:
> Dick,
> Tomcat is a less than optimal platform for high concurrent load. It does
> not have the same continuations mechanism that Jetty has. All my high load
> tests are done on Jetty. With that being said, ESME's long polling for the
> HTTP APIs does not take advantage of Jetty's continuations yet. That's on
> my to-do list, but to date has not been a high priority.
>
> Another issue is that there's a problem with Scala Actors and memory in
> Scala 2.7.2. The Scala team is releasing Scala 2.7.3 this week or next to
> cure the memory problems.
>
> Also, the continuations that Jetty currently supports are part of the
> Servlet 3.0 spec and should be part of NetWeaver this year.
>
> Thanks,
>
> David
>
> On Wed, Jan 7, 2009 at 8:54 PM, Hirsch, Richard
> <ri...@siemens.com>wrote:
>
> > Here are some details for a very first performance test on Stax for the
> > ESME server. Daniel tried with 1000 concurrent connections and then the
> > server started having some problems. Take a look at the enclosed stack
> trace
> > and you will see that towards the end there were problems with the
> threads.
> > I'm also enclosing a picture of the Stax performance indicators. I don't
> > know the exact dimensions of the test but I'm sure Daniel will provide
> them
> > soon.
> >
> > Load tests are critical if we are to succeed in enterprises. They are
> also
> > critical when customers need sizing information. I assume that they
> should
> > also be useful for the lift framework.
> >
> > D.
> >
> > ________________________________
> >
> > From: Spike Washburn [mailto:spike@staxnetworks.com]
> > Sent: Wed 1/7/2009 23:13
> > To: Hirsch, Richard
> > Subject: Re: Stax account
> >
> >
> > The activity died back down for a bit, but then the app started sucking
> up
> > memory and CPU like it was stuck in a loop. When we checked the logs, we
> > saw it was throwing out of memory exceptions. Since the app was clearly
> in
> > a death spiral, we took a JVM stack dump and then restarted the app. I
> have
> > attached the last part of the appserver log if you want to review it.
> >
> > Also I noticed from the log that your app is getting warnings about
> > including the servlet-api-2.5.jar in WEB-INF/lib. This is not necessary
> > since the Servlet API classes are part of the classpath provided by the
> > appserver.
> >
> > Before your app died we were seeing upwards of 1000 concurrent
> connections
> > to your app. Please let me know if you were expecting this load or if it
> > was some kind of external attack against your app.
> >
> > Thanks,
> > Spike
> >
> >
> > On Wed, Jan 7, 2009 at 1:09 PM, Spike Washburn <sp...@staxnetworks.com>
> > wrote:
> >
> >
> > Hi Dick,
> >
> > We just noticed a major spike in activity on your application (id:
> > DickHirsch/esmecloudserver). I just wanted to check with you to see if
> you
> > were doing some load testing or if this was some kind of external attack
> on
> > your webapp.
> >
> > Thanks,
> > Spike
> >
> >
> >
> >
>
>
> --
> Lift, the simply functional web framework http://liftweb.net <http://liftweb.net/>
> Collaborative Task Management http://much4.us <http://much4.us/>
> Follow me: http://twitter.com/dpp
> Git some: http://github.com/dpp
>
--
---
Daniel Koller
Jahnstrasse 20
80469 München * dakoller@googlemail.com
Re: Details from Stax Load test
Posted by Daniel Koller <da...@googlemail.com>.
Good morning.
I want to give you a short overview about the load test I did yesterday
evening:
ESME@Stax was approached by up to nine parallel Java applications threads,
which used the REST api to logon, to post a message and then to logoff
again. This happens for each of the threads about 250-500 times. This test
is shown in the pic attached by Dick: it is the high level plateau shown at
process memory. I measured the time for each of the mentioned steps, and
also the total time for the call.
I'll post facts and figures, when I compiled them and changed the test
scripts, but here are the first
Observations:
- We had the exceptions Dick mentioned, besides from these errors the
response times were on the same (good) level for a long time.
- Sometimes there are subsequent calls which take longer (about 3-5x), but
performance comes back to the previous level after short time. The reasons
for these longer function calls has to be investigated. (maybe related to
the mentioned Scala topics in Davids mail before)
- During earlier tests, I found that the performance of the ESME Web UI goes
down under heavy REST load.
Next steps:
- I will include in the test scripts a test step to simulate a call the the
Web UI (to get the complete picture of ESME performance)
- Yesterday I tested one user which is logging in over and over again, but
this user is not followed by anybody. Also the receiving messages step is
not tested. --> this means creation of a test script, which impersonates
more different users, which are also followed by a different number of other
users.
- Also receiving messages hast to be incorporated into the test script.
That's it for now,
Daniel
On Thu, Jan 8, 2009 at 6:32 AM, David Pollak
<fe...@gmail.com>wrote:
> Dick,
> Tomcat is a less than optimal platform for high concurrent load. It does
> not have the same continuations mechanism that Jetty has. All my high load
> tests are done on Jetty. With that being said, ESME's long polling for the
> HTTP APIs does not take advantage of Jetty's continuations yet. That's on
> my to-do list, but to date has not been a high priority.
>
> Another issue is that there's a problem with Scala Actors and memory in
> Scala 2.7.2. The Scala team is releasing Scala 2.7.3 this week or next to
> cure the memory problems.
>
> Also, the continuations that Jetty currently supports are part of the
> Servlet 3.0 spec and should be part of NetWeaver this year.
>
> Thanks,
>
> David
>
> On Wed, Jan 7, 2009 at 8:54 PM, Hirsch, Richard
> <ri...@siemens.com>wrote:
>
> > Here are some details for a very first performance test on Stax for the
> > ESME server. Daniel tried with 1000 concurrent connections and then the
> > server started having some problems. Take a look at the enclosed stack
> trace
> > and you will see that towards the end there were problems with the
> threads.
> > I'm also enclosing a picture of the Stax performance indicators. I don't
> > know the exact dimensions of the test but I'm sure Daniel will provide
> them
> > soon.
> >
> > Load tests are critical if we are to succeed in enterprises. They are
> also
> > critical when customers need sizing information. I assume that they
> should
> > also be useful for the lift framework.
> >
> > D.
> >
> > ________________________________
> >
> > From: Spike Washburn [mailto:spike@staxnetworks.com]
> > Sent: Wed 1/7/2009 23:13
> > To: Hirsch, Richard
> > Subject: Re: Stax account
> >
> >
> > The activity died back down for a bit, but then the app started sucking
> up
> > memory and CPU like it was stuck in a loop. When we checked the logs, we
> > saw it was throwing out of memory exceptions. Since the app was clearly
> in
> > a death spiral, we took a JVM stack dump and then restarted the app. I
> have
> > attached the last part of the appserver log if you want to review it.
> >
> > Also I noticed from the log that your app is getting warnings about
> > including the servlet-api-2.5.jar in WEB-INF/lib. This is not necessary
> > since the Servlet API classes are part of the classpath provided by the
> > appserver.
> >
> > Before your app died we were seeing upwards of 1000 concurrent
> connections
> > to your app. Please let me know if you were expecting this load or if it
> > was some kind of external attack against your app.
> >
> > Thanks,
> > Spike
> >
> >
> > On Wed, Jan 7, 2009 at 1:09 PM, Spike Washburn <sp...@staxnetworks.com>
> > wrote:
> >
> >
> > Hi Dick,
> >
> > We just noticed a major spike in activity on your application (id:
> > DickHirsch/esmecloudserver). I just wanted to check with you to see if
> you
> > were doing some load testing or if this was some kind of external attack
> on
> > your webapp.
> >
> > Thanks,
> > Spike
> >
> >
> >
> >
>
>
> --
> Lift, the simply functional web framework http://liftweb.net
> Collaborative Task Management http://much4.us
> Follow me: http://twitter.com/dpp
> Git some: http://github.com/dpp
>
--
---
Daniel Koller
Jahnstrasse 20
80469 München * dakoller@googlemail.com
Re: Details from Stax Load test
Posted by David Pollak <fe...@gmail.com>.
Dick,
Tomcat is a less than optimal platform for high concurrent load. It does
not have the same continuations mechanism that Jetty has. All my high load
tests are done on Jetty. With that being said, ESME's long polling for the
HTTP APIs does not take advantage of Jetty's continuations yet. That's on
my to-do list, but to date has not been a high priority.
Another issue is that there's a problem with Scala Actors and memory in
Scala 2.7.2. The Scala team is releasing Scala 2.7.3 this week or next to
cure the memory problems.
Also, the continuations that Jetty currently supports are part of the
Servlet 3.0 spec and should be part of NetWeaver this year.
Thanks,
David
On Wed, Jan 7, 2009 at 8:54 PM, Hirsch, Richard
<ri...@siemens.com>wrote:
> Here are some details for a very first performance test on Stax for the
> ESME server. Daniel tried with 1000 concurrent connections and then the
> server started having some problems. Take a look at the enclosed stack trace
> and you will see that towards the end there were problems with the threads.
> I'm also enclosing a picture of the Stax performance indicators. I don't
> know the exact dimensions of the test but I'm sure Daniel will provide them
> soon.
>
> Load tests are critical if we are to succeed in enterprises. They are also
> critical when customers need sizing information. I assume that they should
> also be useful for the lift framework.
>
> D.
>
> ________________________________
>
> From: Spike Washburn [mailto:spike@staxnetworks.com]
> Sent: Wed 1/7/2009 23:13
> To: Hirsch, Richard
> Subject: Re: Stax account
>
>
> The activity died back down for a bit, but then the app started sucking up
> memory and CPU like it was stuck in a loop. When we checked the logs, we
> saw it was throwing out of memory exceptions. Since the app was clearly in
> a death spiral, we took a JVM stack dump and then restarted the app. I have
> attached the last part of the appserver log if you want to review it.
>
> Also I noticed from the log that your app is getting warnings about
> including the servlet-api-2.5.jar in WEB-INF/lib. This is not necessary
> since the Servlet API classes are part of the classpath provided by the
> appserver.
>
> Before your app died we were seeing upwards of 1000 concurrent connections
> to your app. Please let me know if you were expecting this load or if it
> was some kind of external attack against your app.
>
> Thanks,
> Spike
>
>
> On Wed, Jan 7, 2009 at 1:09 PM, Spike Washburn <sp...@staxnetworks.com>
> wrote:
>
>
> Hi Dick,
>
> We just noticed a major spike in activity on your application (id:
> DickHirsch/esmecloudserver). I just wanted to check with you to see if you
> were doing some load testing or if this was some kind of external attack on
> your webapp.
>
> Thanks,
> Spike
>
>
>
>
--
Lift, the simply functional web framework http://liftweb.net
Collaborative Task Management http://much4.us
Follow me: http://twitter.com/dpp
Git some: http://github.com/dpp