You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomee.apache.org by David Blevins <da...@gmail.com> on 2012/04/25 16:48:32 UTC
Potential improvements and fixes to 1.0.0 binaries
Would be great to at least some of the performance work into the 1.0.0 binaries as there's a bit over 3x increase in Eclipse startup time and a bit over 2x in regular startup time.
Aside from performance I did find two bugs:
- Web.xml metadata-complete effectively ignore
https://issues.apache.org/jira/browse/TOMEE-166
- Omitting <ejb-name> from xml may result in failed deployment
https://issues.apache.org/jira/browse/OPENEJB-1830
OPENEJB-1830 is not terrible, but TOMEE-166 is pretty bad. The point of metadata-complete is to avoid scanning and quite simply it doesn't result in scanning being avoided.
We pass the related TCK tests because we correctly ignore the data we've scanned, but applications still have to pay the full price of scanning regardless.
A lot of the overall improvements were in the .tld file area. Those were the changes that bought us the most time (over a second off of startup and redeploys)
Those are fantastic optimizations, not the "our stuff will be fast, but yours will be slow" kind. If you add a ton of tld libraries in tomcat/lib/, your apps will start much faster than they even do in plain Tomcat because of these optimizations.
In short, we cache the results of the scan (just the urls, not the file contents) and report that back anytime someone asks. If you have 5 apps, you get a 5x improvement on scanning of shared tld libs. This works regardless of how they're shared too. So if you add a common classloader, for example, children of that classloader won't have to each individually scan the common classloader on each deploy.
-David
Re: Potential improvements and fixes to 1.0.0 binaries
Posted by Mark Struberg <st...@yahoo.de>.
> For example, we *always* construct an OWB WebContext for each and every
> application and *use* it in injection and fire all CDI lifecycle events even
> when the application is not CDI enabled. If the application isn't CDI
> enabled then simply no beans are listening for the events and the BeanManager
> will be nearly empty minus a few built-in beans, but our code doesn't see
> any difference. This was done intentionally to avoid the scenario where someone
> adds a beans.xml to an application and suddenly things that worked now blow up,
> resulting in the person being incorrectly mad at CDI or (correctly) mad at
> TomEE.
I know it's only an example.
The bad news: doing those improvements can sometimes be tricky:
How do you detect if the app is a CDI app?
First we scan all META-INF/beans.xml.
If we don't find any, there might STILL be beans!
Any Extension is free to add additional AnnotatedTypes via
http://docs.oracle.com/javaee/6/api/javax/enterprise/inject/spi/BeforeBeanDiscovery.html#addAnnotatedType%28javax.enterprise.inject.spi.AnnotatedType%29
and similar methods!
Next thing is to scan the classpath for the BDAs with META-INF/beans.xml. Even if you don't have a single CDI annotation, all teh beans will still get picked up as @Dependent! Also there are quite a few CDI Lifecycle hooks where you can add your own Beans etc.
The good news: we've done this already for you ;)
Well here it goes: BeanManagerImpl#isInUse()
I guess I should document this a _bit_ better in the future :)
At least I documented the member variable itself:
/**
* This flag will get set to <code>true</code> if a custom bean
* (all non-internal beans like {@link org.apache.webbeans.component.BeanManagerBean;} etc)
* gets set.
*/
All you need to do is to check this flag.
LieGrue,
strub
----- Original Message -----
> From: David Blevins <da...@gmail.com>
> To: dev@openejb.apache.org
> Cc:
> Sent: Thursday, April 26, 2012 2:02 AM
> Subject: Re: Potential improvements and fixes to 1.0.0 binaries
>
>G reen TCK run and green build from all the changes.
>
> Going to start merging code into the branch, get that through the TCK and a
> green build in the CI system.
>
> With any luck we'll have some binaries up by tonight!
>
> Now that I've had some sleep, some details on "faster"....
>
> - The functionality we're running with these much faster numbers is the
> *same* functionality we were running with the much slower numbers.
>
> It is definitely not the case that these improvements came from yanking things
> out we needed. I find that kind of "optimization" to be cheap and
> meaningless -- I'd rather have an honest slow time than a dishonest fast
> one.
>
> We'd never really done heavy performance optimization on TomEE, so there
> were a lot of nails sticking up. I hammered as many down as I could in the time
> we had.
>
> I think of the benefits of optimizations in these categories (in my order of
> preference):
>
> - Variable
> - Fixed
> - Conditional
>
> Variable is where the benefit of the optimization depends on the app size. A
> small app might see nothing and a large app might see huge benefit. Say you add
> an optimization that shaves off 10% of deploy time, our 40 second deploy becomes
> 36 second while our 2 second deploy still feels the same. The percentage of
> time you save is almost never constant, so saying "10%" improvement is
> usually a rough indicator at best.
>
> Fixed is like shaving say 1s off of a deploy time and no matter what you'll
> only get the one second; a 100MB app and a 2MB app would get the same flat 1
> second. If the deploy times were 40 seconds and 2 seconds, after then
> optimization you'd see 39 seconds and 1 second.
>
> Conditional is where the benefit effectively goes away because the boost is
> predicate on your app not needing some particular kind of functionality and that
> functionality. These are of the "if you're not using JSF, why enable
> MyFaces" kind. I really don't like these -- more on that later.
>
> The last month's worth of performance tuning had all been in Variable
> benefit optimizations as I personally find these to have much greater value.
>
> The last two days have largely been Fixed benefit boosts, some Variable benefit
> boosts (yay) around tld scanning, and exactly one Conditional benefit
> optimization. The Fixed benefit optimization were split between the server
> itself and actual app deploy times. The server Fixed optimizations you get
> once, the app deploy optimizations you get once per deploy. I tried to focus on
> app deploy times as much as possible and indeed this is where we saved the most
> on "startup" time. Who really cares how fast your empty server
> starts.
>
> On the Conditional benefit optimization, that was actually a bad bug. The spec
> defined "metadata-complete" is essentially a Conditional benefit
> optimization required as part of certification meant to avoid scanning. We were
> still scanning.
>
> On that, an explicit note to Neale. I was aware of this bug over the last month
> of our performance tuning, but intentionally didn't fix it :) I really
> wanted to tune large, real-world apps and so that's what we did. We made
> big improvements in actual deploy times of actual apps under the scenario that
> "everything was on", when in fact, for the apps we were using, not
> everything should have been on :) I recall you making note of that -- you were
> spot on. Now that the metadata complete flag is properly functioning, these
> apps will boot much faster and probably at a speed that is very on par with
> Tomcat, but if we want to get real performance numbers against beta-2 with the
> latest changes we should edit the web.xml to be
> "metadata-complete=false".
>
> So final note that aside from "metadata-complete", there were no
> conditional benefit optimizations added. They have a place, but I don't
> really care for that kind of thing and think it is short-sighted. Things tend
> to slow down or break when you add bits to your app and suddenly you get
> different behavior. We're still executing all critical paths all the time,
> period. That's really the only way you can keep them well oiled and moving
> smoothly. This is actually at the core of OpenEJB/TomEE an I think a major
> reason for the speed and quality.
>
> For example, we *always* construct an OWB WebContext for each and every
> application and *use* it in injection and fire all CDI lifecycle events even
> when the application is not CDI enabled. If the application isn't CDI
> enabled then simply no beans are listening for the events and the BeanManager
> will be nearly empty minus a few built-in beans, but our code doesn't see
> any difference. This was done intentionally to avoid the scenario where someone
> adds a beans.xml to an application and suddenly things that worked now blow up,
> resulting in the person being incorrectly mad at CDI or (correctly) mad at
> TomEE.
>
> In terms of future perf work, I see room for more Variable benefit optimizations
> and not really too much around Fixed benefit optimizations. Still some scanning
> (variable) work we can do. There are some JAXB (fixed) optimizations we could
> do as well. Overall, though, I'd expect things to lean towards the Variable
> side as they should.
>
> So, yes, we are "fast", but hopefully the above will break that word
> down into a more useful description of what fast means for us.
>
>
> -David
>
> On Apr 25, 2012, at 7:48 AM, David Blevins wrote:
>
>> Would be great to at least some of the performance work into the 1.0.0
> binaries as there's a bit over 3x increase in Eclipse startup time and a bit
> over 2x in regular startup time.
>>
>> Aside from performance I did find two bugs:
>>
>> - Web.xml metadata-complete effectively ignore
>> https://issues.apache.org/jira/browse/TOMEE-166
>>
>> - Omitting <ejb-name> from xml may result in failed deployment
>> https://issues.apache.org/jira/browse/OPENEJB-1830
>>
>> OPENEJB-1830 is not terrible, but TOMEE-166 is pretty bad. The point of
> metadata-complete is to avoid scanning and quite simply it doesn't result in
> scanning being avoided.
>>
>> We pass the related TCK tests because we correctly ignore the data
> we've scanned, but applications still have to pay the full price of scanning
> regardless.
>>
>> A lot of the overall improvements were in the .tld file area. Those were
> the changes that bought us the most time (over a second off of startup and
> redeploys)
>>
>> Those are fantastic optimizations, not the "our stuff will be fast,
> but yours will be slow" kind. If you add a ton of tld libraries in
> tomcat/lib/, your apps will start much faster than they even do in plain Tomcat
> because of these optimizations.
>>
>> In short, we cache the results of the scan (just the urls, not the file
> contents) and report that back anytime someone asks. If you have 5 apps, you
> get a 5x improvement on scanning of shared tld libs. This works regardless of
> how they're shared too. So if you add a common classloader, for example,
> children of that classloader won't have to each individually scan the common
> classloader on each deploy.
>>
>>
>>
>> -David
>>
>
Re: Potential improvements and fixes to 1.0.0 binaries
Posted by David Blevins <da...@gmail.com>.
Green TCK run and green build from all the changes.
Going to start merging code into the branch, get that through the TCK and a green build in the CI system.
With any luck we'll have some binaries up by tonight!
Now that I've had some sleep, some details on "faster"....
- The functionality we're running with these much faster numbers is the *same* functionality we were running with the much slower numbers.
It is definitely not the case that these improvements came from yanking things out we needed. I find that kind of "optimization" to be cheap and meaningless -- I'd rather have an honest slow time than a dishonest fast one.
We'd never really done heavy performance optimization on TomEE, so there were a lot of nails sticking up. I hammered as many down as I could in the time we had.
I think of the benefits of optimizations in these categories (in my order of preference):
- Variable
- Fixed
- Conditional
Variable is where the benefit of the optimization depends on the app size. A small app might see nothing and a large app might see huge benefit. Say you add an optimization that shaves off 10% of deploy time, our 40 second deploy becomes 36 second while our 2 second deploy still feels the same. The percentage of time you save is almost never constant, so saying "10%" improvement is usually a rough indicator at best.
Fixed is like shaving say 1s off of a deploy time and no matter what you'll only get the one second; a 100MB app and a 2MB app would get the same flat 1 second. If the deploy times were 40 seconds and 2 seconds, after then optimization you'd see 39 seconds and 1 second.
Conditional is where the benefit effectively goes away because the boost is predicate on your app not needing some particular kind of functionality and that functionality. These are of the "if you're not using JSF, why enable MyFaces" kind. I really don't like these -- more on that later.
The last month's worth of performance tuning had all been in Variable benefit optimizations as I personally find these to have much greater value.
The last two days have largely been Fixed benefit boosts, some Variable benefit boosts (yay) around tld scanning, and exactly one Conditional benefit optimization. The Fixed benefit optimization were split between the server itself and actual app deploy times. The server Fixed optimizations you get once, the app deploy optimizations you get once per deploy. I tried to focus on app deploy times as much as possible and indeed this is where we saved the most on "startup" time. Who really cares how fast your empty server starts.
On the Conditional benefit optimization, that was actually a bad bug. The spec defined "metadata-complete" is essentially a Conditional benefit optimization required as part of certification meant to avoid scanning. We were still scanning.
On that, an explicit note to Neale. I was aware of this bug over the last month of our performance tuning, but intentionally didn't fix it :) I really wanted to tune large, real-world apps and so that's what we did. We made big improvements in actual deploy times of actual apps under the scenario that "everything was on", when in fact, for the apps we were using, not everything should have been on :) I recall you making note of that -- you were spot on. Now that the metadata complete flag is properly functioning, these apps will boot much faster and probably at a speed that is very on par with Tomcat, but if we want to get real performance numbers against beta-2 with the latest changes we should edit the web.xml to be "metadata-complete=false".
So final note that aside from "metadata-complete", there were no conditional benefit optimizations added. They have a place, but I don't really care for that kind of thing and think it is short-sighted. Things tend to slow down or break when you add bits to your app and suddenly you get different behavior. We're still executing all critical paths all the time, period. That's really the only way you can keep them well oiled and moving smoothly. This is actually at the core of OpenEJB/TomEE an I think a major reason for the speed and quality.
For example, we *always* construct an OWB WebContext for each and every application and *use* it in injection and fire all CDI lifecycle events even when the application is not CDI enabled. If the application isn't CDI enabled then simply no beans are listening for the events and the BeanManager will be nearly empty minus a few built-in beans, but our code doesn't see any difference. This was done intentionally to avoid the scenario where someone adds a beans.xml to an application and suddenly things that worked now blow up, resulting in the person being incorrectly mad at CDI or (correctly) mad at TomEE.
In terms of future perf work, I see room for more Variable benefit optimizations and not really too much around Fixed benefit optimizations. Still some scanning (variable) work we can do. There are some JAXB (fixed) optimizations we could do as well. Overall, though, I'd expect things to lean towards the Variable side as they should.
So, yes, we are "fast", but hopefully the above will break that word down into a more useful description of what fast means for us.
-David
On Apr 25, 2012, at 7:48 AM, David Blevins wrote:
> Would be great to at least some of the performance work into the 1.0.0 binaries as there's a bit over 3x increase in Eclipse startup time and a bit over 2x in regular startup time.
>
> Aside from performance I did find two bugs:
>
> - Web.xml metadata-complete effectively ignore
> https://issues.apache.org/jira/browse/TOMEE-166
>
> - Omitting <ejb-name> from xml may result in failed deployment
> https://issues.apache.org/jira/browse/OPENEJB-1830
>
> OPENEJB-1830 is not terrible, but TOMEE-166 is pretty bad. The point of metadata-complete is to avoid scanning and quite simply it doesn't result in scanning being avoided.
>
> We pass the related TCK tests because we correctly ignore the data we've scanned, but applications still have to pay the full price of scanning regardless.
>
> A lot of the overall improvements were in the .tld file area. Those were the changes that bought us the most time (over a second off of startup and redeploys)
>
> Those are fantastic optimizations, not the "our stuff will be fast, but yours will be slow" kind. If you add a ton of tld libraries in tomcat/lib/, your apps will start much faster than they even do in plain Tomcat because of these optimizations.
>
> In short, we cache the results of the scan (just the urls, not the file contents) and report that back anytime someone asks. If you have 5 apps, you get a 5x improvement on scanning of shared tld libs. This works regardless of how they're shared too. So if you add a common classloader, for example, children of that classloader won't have to each individually scan the common classloader on each deploy.
>
>
>
> -David
>