You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geronimo.apache.org by Jules Gosnell <ju...@coredevelopers.net> on 2003/08/08 23:02:36 UTC
[components] - lazy loading...
a chat with a friend, Shane, at work today produced these thoughts....
Imagine that rather than having a set number of configurations which all
load every service at startup, Geronimo could load just the components
needed to provide the services required by your deployables...
e.g. you are doing web development, so when you start Geronimo it just
loads the webcontainer component and deploys your webapp. You webapp
decides to load a javax.naming class so the JNDI service is brought on
line, it loads a javax.mail class and looks up a mail service in JNDI -
the mail service is brought up... etc
If we could lazily load services as required, startup time would be
greatly reduced, footprint as well, and the distributions complexity too
- since we could just ship everything and only what was needed would be
used.
just a thought to file away for a rainy day :-)
Jules
--
/*************************************
* Jules Gosnell
* Partner
* Core Developers Network (Europe)
* http://www.coredevelopers.net
*************************************/
RE: [components] - lazy loading...
Posted by "Noel J. Bergman" <no...@devtech.com>.
> That is an excellent idea.
http://nagoya.apache.org/wiki/apachewiki.cgi?ApacheJ2EE/WishList
LOL And soon enough it will be come automatic, and I won't mention it again.
--- Noel
Re: [components] - lazy loading...
Posted by Myron Tuang <tu...@yahoo.co.uk>.
That is an excellent idea.
+1
--- Jules Gosnell <ju...@coredevelopers.net> wrote: >
a chat with a friend, Shane, at work today produced
> these thoughts....
>
> Imagine that rather than having a set number of
> configurations which all
> load every service at startup, Geronimo could load
> just the components
> needed to provide the services required by your
> deployables...
>
> e.g. you are doing web development, so when you
> start Geronimo it just
> loads the webcontainer component and deploys your
> webapp. You webapp
> decides to load a javax.naming class so the JNDI
> service is brought on
> line, it loads a javax.mail class and looks up a
> mail service in JNDI -
> the mail service is brought up... etc
>
> If we could lazily load services as required,
> startup time would be
> greatly reduced, footprint as well, and the
> distributions complexity too
> - since we could just ship everything and only what
> was needed would be
> used.
>
> just a thought to file away for a rainy day :-)
>
>
> Jules
>
> --
> /*************************************
> * Jules Gosnell
> * Partner
> * Core Developers Network (Europe)
> * http://www.coredevelopers.net
> *************************************/
>
>
________________________________________________________________________
Want to chat instantly with your online friends? Get the FREE Yahoo!
Messenger http://uk.messenger.yahoo.com/
Re[2]: [components] - lazy loading...
Posted by Igor Lobanov <cr...@mail.ru>.
Hi Howard and fellow geronimos,
Another idea.
During deployment/updating of the application, its classes could be
analyzed to produce dependencies list, let's say, at package
granularity. This list could be persisted and later used to determine
wich of services are needed by deployed application.
Reluctant loading :)
Igor Lobanov
Original Message (Saturday, August 9, 2003, 6:30:09 AM)
> Are you saying, a specialized class loader that links loading
> of certain classes to activation of related services?
> Or the simpler proposition, that services be lazily created as needed.
> --
> Howard M. Lewis Ship
> Creator, Tapestry: Java Web Components
> http://jakarta.apache.org/tapestry
>> -----Original Message-----
>> From: Jules Gosnell [mailto:jules@coredevelopers.net]
>> Sent: Friday, August 08, 2003 5:03 PM
>> To: Geronimo Developers List
>> Subject: [components] - lazy loading...
>>
>>
>> a chat with a friend, Shane, at work today produced these thoughts....
>>
>> Imagine that rather than having a set number of
>> configurations which all
>> load every service at startup, Geronimo could load just the
>> components
>> needed to provide the services required by your deployables...
>>
>> e.g. you are doing web development, so when you start
>> Geronimo it just
>> loads the webcontainer component and deploys your webapp. You webapp
>> decides to load a javax.naming class so the JNDI service is
>> brought on
>> line, it loads a javax.mail class and looks up a mail service
>> in JNDI -
>> the mail service is brought up... etc
>>
>> If we could lazily load services as required, startup time would be
>> greatly reduced, footprint as well, and the distributions
>> complexity too
>> - since we could just ship everything and only what was
>> needed would be
>> used.
>>
>> just a thought to file away for a rainy day :-)
>>
>>
>> Jules
>>
>> --
>> /*************************************
>> * Jules Gosnell
>> * Partner
>> * Core Developers Network (Europe)
>> * http://www.coredevelopers.net
>> *************************************/
>>
>>
RE: [components] - lazy loading...
Posted by "Howard M. Lewis Ship" <hl...@comcast.net>.
Are you saying, a specialized class loader that links loading of certain classes to activation of
related services?
Or the simpler proposition, that services be lazily created as needed.
--
Howard M. Lewis Ship
Creator, Tapestry: Java Web Components
http://jakarta.apache.org/tapestry
> -----Original Message-----
> From: Jules Gosnell [mailto:jules@coredevelopers.net]
> Sent: Friday, August 08, 2003 5:03 PM
> To: Geronimo Developers List
> Subject: [components] - lazy loading...
>
>
> a chat with a friend, Shane, at work today produced these thoughts....
>
> Imagine that rather than having a set number of
> configurations which all
> load every service at startup, Geronimo could load just the
> components
> needed to provide the services required by your deployables...
>
> e.g. you are doing web development, so when you start
> Geronimo it just
> loads the webcontainer component and deploys your webapp. You webapp
> decides to load a javax.naming class so the JNDI service is
> brought on
> line, it loads a javax.mail class and looks up a mail service
> in JNDI -
> the mail service is brought up... etc
>
> If we could lazily load services as required, startup time would be
> greatly reduced, footprint as well, and the distributions
> complexity too
> - since we could just ship everything and only what was
> needed would be
> used.
>
> just a thought to file away for a rainy day :-)
>
>
> Jules
>
> --
> /*************************************
> * Jules Gosnell
> * Partner
> * Core Developers Network (Europe)
> * http://www.coredevelopers.net
> *************************************/
>
>
Re: [components] - lazy loading...
Posted by Berin Loritsch <bl...@apache.org>.
Alex Blewitt wrote:
>> My GUI app framework at D-Haven.org uses Avalon Fortress, which employs
>> asynchronous component management--no extra code necessary. I also have
>> a distributed print server on my harddrive that can take advantage of it,
>> but it is all about what makes sense for this project.
>
>
> I'd like to find out more about what you've done, if we can take this
> discussion out of the group, and then perhaps present a way of how we
> might use these kind of ideas to provide a micro-kernel architecture for
> the system.
Sounds good.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
Concerns for current release (was Re: [components] - lazy loading...)
Posted by Berin Loritsch <bl...@apache.org>.
Jeremy Boynes wrote:
>>I'd like to find out more about what you've done, if we can take this
>>discussion out of the group, and then perhaps present a way of how we
>>might use these kind of ideas to provide a micro-kernel architecture
>>for the system.
>>
>
>
> Please don't take it out of this forum - this is a useful discussion for
> everyone.
>
> We are currently using a JMX kernel for historical reasons, and because it
> lets us get something running really quickly. At this time I think getting
> something running is a better plan than getting the best thing designed as
> it lets other subprojects (like the verifier) get going. This is a project
> management thing, not a technical one.
I agree that this approach is probably the best in the short term. I do have
one major concern that I would like to be addressed in the current release
though:
Please make all server components use the following semantics:
* Separate interface from implementation
* Allow all runtime artifacts to be given to the container during init time
(Inversion of Control principle).
* Don't finalize the contracts too early.
Using these three principles we can avoid static accessors to runtime
artifacts like JNDI (passing in a Context object is better than using the
InitialContext class for the core components).
This will give us the widest flexibility when it comes to choosing our
server component container. The solution will allow us to swap out the
container for something else without forcing us into one approach too
early.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
Re: [components] - lazy loading...
Posted by Alex Blewitt <Al...@ioshq.com>.
On Monday, Aug 11, 2003, at 16:55 Europe/London, Jeremy Boynes wrote:
>> I'd like to find out more about what you've done, if we can take this
>> discussion out of the group, and then perhaps present a way of how we
>> might use these kind of ideas to provide a micro-kernel architecture
>> for the system.
>>
>
> Please don't take it out of this forum - this is a useful discussion
> for
> everyone.
>
> We are currently using a JMX kernel for historical reasons, and
> because it
> lets us get something running really quickly. At this time I think
> getting
> something running is a better plan than getting the best thing
> designed as
> it lets other subprojects (like the verifier) get going. This is a
> project
> management thing, not a technical one.
I agree that getting a working anything is a good starting point.
However, there do seem to be two modes of thought; JMX and not JMX
(various threads, Config/JNDI/One True Way) have already discussed the
benefits of either approach.
Provided that the current implementation is not heavily dependent on
JMX (i.e. the classes don't unnecessarily pretend to be MBeans) then we
should be able to migrate anything that evolves. However, in order for
such changes to occur then it will be necessary for the configuration
file not to be too dependent on JMX, or at least make it easy to
refactor later.
Alex.
RE: [components] - lazy loading...
Posted by Jeremy Boynes <je...@coredevelopers.net>.
> I'd like to find out more about what you've done, if we can take this
> discussion out of the group, and then perhaps present a way of how we
> might use these kind of ideas to provide a micro-kernel architecture
> for the system.
>
Please don't take it out of this forum - this is a useful discussion for
everyone.
We are currently using a JMX kernel for historical reasons, and because it
lets us get something running really quickly. At this time I think getting
something running is a better plan than getting the best thing designed as
it lets other subprojects (like the verifier) get going. This is a project
management thing, not a technical one.
--
Jeremy
Re: [components] - lazy loading...
Posted by Alex Blewitt <Al...@ioshq.com>.
On Monday, Aug 11, 2003, at 16:21 Europe/London, Berin Loritsch wrote:
> Alex Blewitt wrote:
>
>> I really like this idea. Berin seems to think that it's eminantly
>> doable, Jason sounds like he needs to be convinced.
>> If this is already provided as part of Avalon, or Avalon-esque
>> systems, then perhaps this is worth investigating in more detail?
>> Could Berin provide this?
>> It might not be doable using the current JMX interceptor stack, but
>> if it can be done using another mechanism (and demonstrably so) then
>> perhaps that's a good reason to look at using a different mechanism.
> I'll be honest about my ignorance of JMX as a whole. I know some
> rudimentary
> support for it has been added to the Avalon Phoenix container code,
> but I
> would like a better idea on how it is desired to be used in the
> Geronimo/
> whatever we want to call this project codebase.
I'm not sure that you /would/ achieve this using JMX. Instead, the
lazy-loader approach as per the Pheonix (or Merlin) kernels would
probably be more demonstrative of the ideas.
> In Phoenix, JMX is relegated to providing an exposed interface to
> Avalon
> components that are meant to be managed externally. It isn't used by
> the
> kernel to create and tear down component instances.
There has been ongoing discussions in this mailing list as to whether
or not JMX should be used to do this anyway. Current feeling seems to
indicate that (a) At present, the initial code base uses JMX to do
everything, but (b) It may be replaced at a later stage if someone else
presents itself, but only with good reason.
> IMO a management interface like JMX/SNMP should only be used for an
> administrator's console to customize or manage aspects of the running
> system. That fits in with the Phoenix approach.
I concur completely, as do some others on this list. An example of how
we can use this to set up services without having to use JMX would
therefore be of considerable interest.
> That means that the Avalon components can already be loaded on
> demand--without
> any additional coding necessary--once the component is initialized,
> the JMX
> MBean can be created and published.
Indeed, this is the way that JMX should be used, as a management
interface -- not as a kernel implementation.
> My GUI app framework at D-Haven.org uses Avalon Fortress, which employs
> asynchronous component management--no extra code necessary. I also
> have
> a distributed print server on my harddrive that can take advantage of
> it,
> but it is all about what makes sense for this project.
I'd like to find out more about what you've done, if we can take this
discussion out of the group, and then perhaps present a way of how we
might use these kind of ideas to provide a micro-kernel architecture
for the system.
> Alex.
Re: [components] - lazy loading...
Posted by Berin Loritsch <bl...@apache.org>.
Alex Blewitt wrote:
> I really like this idea. Berin seems to think that it's eminantly
> doable, Jason sounds like he needs to be convinced.
>
> If this is already provided as part of Avalon, or Avalon-esque systems,
> then perhaps this is worth investigating in more detail? Could Berin
> provide this?
>
> It might not be doable using the current JMX interceptor stack, but if
> it can be done using another mechanism (and demonstrably so) then
> perhaps that's a good reason to look at using a different mechanism.
How do you want to prove it, or do it?
I'll be honest about my ignorance of JMX as a whole. I know some rudimentary
support for it has been added to the Avalon Phoenix container code, but I
would like a better idea on how it is desired to be used in the Geronimo/
whatever we want to call this project codebase.
In Phoenix, JMX is relegated to providing an exposed interface to Avalon
components that are meant to be managed externally. It isn't used by the
kernel to create and tear down component instances.
IMO a management interface like JMX/SNMP should only be used for an
administrator's console to customize or manage aspects of the running
system. THat fits in with the Phoenix approach.
That means that the Avalon components can already be loaded on demand--without
any additional coding necessary--once the component is initialized, the JMX
MBean can be created and published.
My GUI app framework at D-Haven.org uses Avalon Fortress, which employs
asynchronous component management--no extra code necessary. I also have
a distributed print server on my harddrive that can take advantage of it,
but it is all about what makes sense for this project.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
Re: [components] - lazy loading...
Posted by Alex Blewitt <Al...@ioshq.com>.
I really like this idea. Berin seems to think that it's eminantly
doable, Jason sounds like he needs to be convinced.
If this is already provided as part of Avalon, or Avalon-esque systems,
then perhaps this is worth investigating in more detail? Could Berin
provide this?
It might not be doable using the current JMX interceptor stack, but if
it can be done using another mechanism (and demonstrably so) then
perhaps that's a good reason to look at using a different mechanism.
Alex.
On Monday, Aug 11, 2003, at 15:32 Europe/London, Berin Loritsch wrote:
> Jason Dillon wrote:
>
>>> Imagine that rather than having a set number of configurations which
>>> all load every service at startup, Geronimo could load just the
>>> components needed to provide the services required by your
>>> deployables...
>> I like the concept, but I am not sure how easy this will be to
>> actually implement. To make something like this work, we would need
>> to load configuration metadata for everything on boot, so that we
>> knew what components we had to work with. I mean, how else would the
>> component container know which components it had to work with to
>> start the services required by the deployable?
>
> It's not that hard, really. I have done it a couple of times for the
> containers at Avalon.
>
>
>> Sounds like a lot of complex magic inside to achieve lazy loading.
>
> There is nothing written in stone that the central container-container
> should require JNDI. There are some very simple approaches that does
> not
> require complex logic to achieve on-demand loading.
>
>>> If we could lazily load services as required, startup time would be
>>> greatly reduced, footprint as well, and the distributions complexity
>>> too - since we could just ship everything and only what was needed
>>> would be used.
>> Initial startup time may be reduced, but if an app requires JNDI,
>> JavaMail or whatever, then the time initialize these will not change,
>> we have just moved where that time is spent.
>
> Another approach to minimize startup time that also works quite well
> is asynchronous component management. I have designed the Avalon
> Fortress
> container to load components in a controlled set of background threads
> so
> that the initialization load would be spread among all available
> processors.
> In fact, it adapts the number of threads to the number of processors
> available.
>
> That said, as long as you can guarantee that any requested instance is
> available and fully initialized on first use, then all is well. The
> approach
> uses a library inspired by SEDA principles (Avalon Event) that
> provides a
> CommandManager. The CommandManager executes Command Events from an
> event
> queue. The commands get executed in order, and it pulls the
> initialization
> logic and overall component management logic away from the critical
> path.
>
> The advantage of the approach over regular inline initialization and
> component
> management is at least 5:1, and in some cases 15:1. Anything is
> doable.
>
>
>> I do like the concept, but I think this may not be feasible in the
>> short-term... though it would depend a lot on what we use for
>> component management I would say.
>
> It does. Most of these things are already available with some
> containers.
>
> The advantage of lazy loading is that you can make sure you aren't
> loading
> what you don't need ahead of time--reducing the memory footprint, which
> has an impact on scalability (larger margin before GCs are necessary).
>
> --
>
> "They that give up essential liberty to obtain a little temporary
> safety
> deserve neither liberty nor safety."
> - Benjamin Franklin
>
Re: [components] - lazy loading...
Posted by Berin Loritsch <bl...@apache.org>.
Jeremy Boynes wrote:
> In repsonse to Berin Loritsch:
>
> Fast startup is primarily a benefit for developers and less of an issue in
> production environments. Production configs need scalability and stability,
> and will typically use all the services they ultimately need from the
> outset.
>
> I think a reasonable balance between the two is to aggressively load just
> what is needed for any deployment at deployment time.
>
> This handles the development case, where only the necessary services get
> loaded on startup, and the service mix stays constant as applications are
> redeployed by the developer.
>
> It also handles the production case, in that all services needed are loaded
> together giving a deterministic failure point, but only the ones needed by
> the apps are actually loaded conserving system resources.
>
> The loading pricipals you described would work well, its just that
> "first-use" becomes deployment time rather than run time.
>
> The caveat I would throw in is that if the system is online, I like have the
> option of either dedicating one thread to loading a new application (leaving
> the other threads to service requests) or of dedicating all threads to
> redeploying an existing application (to reduce outage time).
As it is currently written there are three component instantation options:
* Inline--component is always initialized in the critical path.
* Asynchronous--component is always initialized in a set of background threads
(no need to restrict to one or multiple) The command manager is
configurable to use X number of threads per processor.
* On demand--component is only initialized when looked up.
As a safety measure, all components can opperate with "On demand" semantics.
That way we can avoid any race conditions with the Asynchronous model or the
mix of asynchronous and inline components. If a component is requested before
it has been initialized, the container is smart enough to initialize it on
demand and protect against double-initialization.
So if you want to option to force "Inline" you have it. If you want the option
to put some components with asynchronous initialization and some with on demand,
then you have it. No additional coding necessary.
Also, the asynchronous command manager really helps in controlling the container
and other components without hurting the critical path. If you need to
periodically notify certain components of a particular event, the CommandManager
supports both delayed and repeated commands. Using this approach, we can keep
our pool of threads free until we really need them. The monitoring command will
be called as necessary and invoked with any free thread. It is a powerful way
to go.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
RE: [components] - lazy loading...
Posted by Jeremy Boynes <je...@coredevelopers.net>.
In repsonse to Berin Loritsch:
Fast startup is primarily a benefit for developers and less of an issue in
production environments. Production configs need scalability and stability,
and will typically use all the services they ultimately need from the
outset.
I think a reasonable balance between the two is to aggressively load just
what is needed for any deployment at deployment time.
This handles the development case, where only the necessary services get
loaded on startup, and the service mix stays constant as applications are
redeployed by the developer.
It also handles the production case, in that all services needed are loaded
together giving a deterministic failure point, but only the ones needed by
the apps are actually loaded conserving system resources.
The loading pricipals you described would work well, its just that
"first-use" becomes deployment time rather than run time.
The caveat I would throw in is that if the system is online, I like have the
option of either dedicating one thread to loading a new application (leaving
the other threads to service requests) or of dedicating all threads to
redeploying an existing application (to reduce outage time).
--
Jeremy
Re: [components] - lazy loading...
Posted by Berin Loritsch <bl...@apache.org>.
Jason Dillon wrote:
>> Imagine that rather than having a set number of configurations which
>> all load every service at startup, Geronimo could load just the
>> components needed to provide the services required by your deployables...
>
>
> I like the concept, but I am not sure how easy this will be to actually
> implement. To make something like this work, we would need to load
> configuration metadata for everything on boot, so that we knew what
> components we had to work with. I mean, how else would the component
> container know which components it had to work with to start the
> services required by the deployable?
It's not that hard, really. I have done it a couple of times for the
containers at Avalon.
>
> Sounds like a lot of complex magic inside to achieve lazy loading.
There is nothing written in stone that the central container-container
should require JNDI. There are some very simple approaches that does not
require complex logic to achieve on-demand loading.
>> If we could lazily load services as required, startup time would be
>> greatly reduced, footprint as well, and the distributions complexity
>> too - since we could just ship everything and only what was needed
>> would be used.
>
>
> Initial startup time may be reduced, but if an app requires JNDI,
> JavaMail or whatever, then the time initialize these will not change, we
> have just moved where that time is spent.
Another approach to minimize startup time that also works quite well
is asynchronous component management. I have designed the Avalon Fortress
container to load components in a controlled set of background threads so
that the initialization load would be spread among all available processors.
In fact, it adapts the number of threads to the number of processors available.
That said, as long as you can guarantee that any requested instance is
available and fully initialized on first use, then all is well. The approach
uses a library inspired by SEDA principles (Avalon Event) that provides a
CommandManager. The CommandManager executes Command Events from an event
queue. The commands get executed in order, and it pulls the initialization
logic and overall component management logic away from the critical path.
The advantage of the approach over regular inline initialization and component
management is at least 5:1, and in some cases 15:1. Anything is doable.
> I do like the concept, but I think this may not be feasible in the
> short-term... though it would depend a lot on what we use for component
> management I would say.
It does. Most of these things are already available with some containers.
The advantage of lazy loading is that you can make sure you aren't loading
what you don't need ahead of time--reducing the memory footprint, which
has an impact on scalability (larger margin before GCs are necessary).
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
Re: [components] - lazy loading...
Posted by Jules Gosnell <ju...@coredevelopers.net>.
initially you could just do this at low granularity, i.e. deployer,
level - only load a deployer when you find something that you need it to
deploy, although once it is loaded you keep it, rather than unloading it
if e.g. the webapp that you loaded it for is undeployed.
more inline...
Jules Gosnell wrote:
> Jason,
>
> I followed pretty much the same though path as you have just outlined
> after posting - then I hung in there and thought some more :-)
>
> Yes it makes things more complex, however:
>
> In development, which is where we have already proven you need to
> capture hearts and minds first, startup time and footprint are
> important. You would be amazed how many people I have come across in
> my JBoss dealings who still have to do a lot of cold deployment...
> Besides as Geronimo developers, therefore the people who will be
> restarting it most, this would speed our own development cycle.
>
> J2EE decomposes nicely into a bunch of APIs javax.xxx.*. We just need
> a persistant registration service. The first time you run up Geronimo,
> all services start, declaring that e.g. they implement javax.naming or
> javax.servlet etc... (package granularity should be enough). Deployers
> also register their interest in e.g. WebContainer:war etc...
>
> These deps are saved away, and next time you start Geronimo, put to
> good use - the dependency tree is shaken out and only services
> required are loaded. This behaviour could be switchable, to load
> lazily in development and proactively in production.
>
> I was thinking that the ClassLoader could do something similar.
> Instead of sitting there watching hundreds of jars being loaded from
> server/x/lib, it could load the index that it saved from the last run
> and then load these classes lazily as and when required. Further
> startup time speedups...
rather than being 'loaded' from server/x/lib, I should have said that
the jars are being reindexed for the nth time by the classloader, so
that it knows where to lazily load a class from when required. I am just
suggesting hanging on to that index between runs, so we don't have to
build it again on every restart...
I expect a fully loaded Geronimo to become HUGE :-), anything we can do
to mitigate bloat in a running image would be good. Making static
decisions about what is needed has been done. We should be considering a
dynamic mechanism...
OK - I'll shut up now :-)
Jules
>
>
> If you upgrade or add new services, jars, their presence would
> invalidate the relevant part of the cached dependency tree or index.
>
> Complex, yes, innovative and possibly valuable - yes.
>
> I don't want something like this to get in the way of getting a server
> out of the door ASAP, but I'd like it to be considered if anyone
> starts working, out on branch, on the ultimate plug-in component
> system for Geronimo services...
>
>
> What do you say :-)
>
>
> Jules
>
>
> Jason Dillon wrote:
>
>>> Imagine that rather than having a set number of configurations which
>>> all load every service at startup, Geronimo could load just the
>>> components needed to provide the services required by your
>>> deployables...
>>
>>
>>
>> I like the concept, but I am not sure how easy this will be to
>> actually implement. To make something like this work, we would need
>> to load configuration metadata for everything on boot, so that we
>> knew what components we had to work with. I mean, how else would the
>> component container know which components it had to work with to
>> start the services required by the deployable?
>>
>>
>>> e.g. you are doing web development, so when you start Geronimo it
>>> just loads the webcontainer component and deploys your webapp. You
>>> webapp decides to load a javax.naming class so the JNDI service is
>>> brought on line, it loads a javax.mail class and looks up a mail
>>> service in JNDI - the mail service is brought up... etc
>>
>>
>>
>> Sounds like a lot of complex magic inside to achieve lazy loading.
>>
>>
>>> If we could lazily load services as required, startup time would be
>>> greatly reduced, footprint as well, and the distributions complexity
>>> too - since we could just ship everything and only what was needed
>>> would be used.
>>
>>
>>
>> Initial startup time may be reduced, but if an app requires JNDI,
>> JavaMail or whatever, then the time initialize these will not change,
>> we have just moved where that time is spent.
>>
>> I do like the concept, but I think this may not be feasible in the
>> short-term... though it would depend a lot on what we use for
>> component management I would say.
>>
>> Personally I do not believe that this buys us all that much. While
>> lazy loading may (or may not) be nice during development, in a
>> production environment I would rather have a slow startup where all
>> of my components are initialized and connected.
>>
>> On the other hand, if the component container had knowledge of the
>> available components/services, then was asked to deploy a web-app,
>> which requires a web-container, JNDI, whatever, then then container
>> could initialize and start these components automatically with out
>> having to explicitly list them as components to be started when the
>> server comes up.
>>
>> I think to make something like this work it would have to be done at
>> a component connection/dependency level and will depend much on the
>> component container we use or write.
>>
>> --jason
>>
>
>
--
/*************************************
* Jules Gosnell
* Partner
* Core Developers Network (Europe)
* http://www.coredevelopers.net
*************************************/
Re: [components] - lazy loading...
Posted by Richard Monson-Haefel <Ri...@Monson-Haefel.com>.
Jules Gosnell wrote:
> I maintain that it's proper place is in here where the
> code only need be written once and will mature quicker. Since the
> container is also in charge of service lifecycle and interservice
> dependencies, it can make optimisations that the service cannot. (Of
> course the service has advantages over the container (self-knowledge),
> but could take advantage of them as well).
>
> Of course if you wish to override it in particular services, I wouldn't
> see a problem with that. You would simply register them as mandatory and
> let them take control of their own laziness.
I really like this Lazy Service Loading idea as a start up option. Obviouisly,
you still need to also support an Aggresive Loading strategy which would result
in a slower start up, but less latency and more throughput in production.
It seems to me that the logic required to support lazy loading of services is
the same, regardless of the service. Basically, if you access it, than it
should be loaded. And indexed and load it again on the next start up. That
applies to everything from J2EE Connectors to instance pools. If this type of
logic can be isolated than it can be placed to in the requests path all
services and designed to handle anything, without regard to the type of
service. You could also tweak it using a configuration that says, Load these
services at start up, but load everything else lazily. One algorithm and a
single point of control. Easy maintenance.
--
Richard Monson-Haefel
Author of J2EE Web Services (Addison-Wesley 2003)
Author of Enterprise JavaBeans, 3rd Edition (O'Reilly 2001)
Co-Author of Java Message Service (O'Reilly 2000)
http://www.Monson-Haefel.com
Re: [components] - lazy loading...
Posted by Jules Gosnell <ju...@coredevelopers.net>.
Aaron Mulder wrote:
> If you deploy a typical EAR, this won't save you a whole lot,
>because the server itself will start darn fast, then try to deploy an app,
>which loads the deployer, the web & EJB containers, DB pools, JNDI,
>transactions, security, web connectors, "EJB connectors", and so on -- the
>majority of available services. You may save the effort of JMS and J2CA,
>depending on the app, but it's not the huge advantage that it initially
>sounds like.
>
not every ear contains both war and ejb.jar
there are many other services that make up an enterprise strength AppServer
There are many dependencies between them
I expect there to be many more in Geronimo
Different people will use Geronimo for very different things
but...
Your solution is similar to mine. You have just shifted the
responsibilty for lazy loading the bulk of a service to the service
itself, rather than the underlying container.
Seeing as a container's job is to isolate and abstract away commonality
such as this, I maintain that it's proper place is in here where the
code only need be written once and will mature quicker. Since the
container is also in charge of service lifecycle and interservice
dependencies, it can make optimisations that the service cannot. (Of
course the service has advantages over the container (self-knowledge),
but could take advantage of them as well).
Of course if you wish to override it in particular services, I wouldn't
see a problem with that. You would simply register them as mandatory and
let them take control of their own laziness.
Jules
>
> I think we'd get more mileage out of aggressively ensuring that
>components start up quickly. You'd get a lot more (let's say) by ensuring
>that all 20 components together start up in under a second that you would
>by not starting the small number of them that an application required.
>
>Aaron
>
>On Sat, 9 Aug 2003, Jules Gosnell wrote:
>
>
>>Jason,
>>
>>I followed pretty much the same though path as you have just outlined
>>after posting - then I hung in there and thought some more :-)
>>
>>Yes it makes things more complex, however:
>>
>>In development, which is where we have already proven you need to
>>capture hearts and minds first, startup time and footprint are
>>important. You would be amazed how many people I have come across in my
>>JBoss dealings who still have to do a lot of cold deployment... Besides
>>as Geronimo developers, therefore the people who will be restarting it
>>most, this would speed our own development cycle.
>>
>>J2EE decomposes nicely into a bunch of APIs javax.xxx.*. We just need a
>>persistant registration service. The first time you run up Geronimo, all
>>services start, declaring that e.g. they implement javax.naming or
>>javax.servlet etc... (package granularity should be enough). Deployers
>>also register their interest in e.g. WebContainer:war etc...
>>
>>These deps are saved away, and next time you start Geronimo, put to good
>>use - the dependency tree is shaken out and only services required are
>>loaded. This behaviour could be switchable, to load lazily in
>>development and proactively in production.
>>
>>I was thinking that the ClassLoader could do something similar. Instead
>>of sitting there watching hundreds of jars being loaded from
>>server/x/lib, it could load the index that it saved from the last run
>>and then load these classes lazily as and when required. Further startup
>>time speedups...
>>
>>If you upgrade or add new services, jars, their presence would
>>invalidate the relevant part of the cached dependency tree or index.
>>
>>Complex, yes, innovative and possibly valuable - yes.
>>
>>I don't want something like this to get in the way of getting a server
>>out of the door ASAP, but I'd like it to be considered if anyone starts
>>working, out on branch, on the ultimate plug-in component system for
>>Geronimo services...
>>
>>
>>What do you say :-)
>>
>>
>>Jules
>>
>>
>>Jason Dillon wrote:
>>
>>
>>
>>>>Imagine that rather than having a set number of configurations which
>>>>all load every service at startup, Geronimo could load just the
>>>>components needed to provide the services required by your
>>>>deployables...
>>>>
>>>>
>>>I like the concept, but I am not sure how easy this will be to
>>>actually implement. To make something like this work, we would need
>>>to load configuration metadata for everything on boot, so that we knew
>>>what components we had to work with. I mean, how else would the
>>>component container know which components it had to work with to start
>>>the services required by the deployable?
>>>
>>>
>>>
>>>
>>>>e.g. you are doing web development, so when you start Geronimo it
>>>>just loads the webcontainer component and deploys your webapp. You
>>>>webapp decides to load a javax.naming class so the JNDI service is
>>>>brought on line, it loads a javax.mail class and looks up a mail
>>>>service in JNDI - the mail service is brought up... etc
>>>>
>>>>
>>>Sounds like a lot of complex magic inside to achieve lazy loading.
>>>
>>>
>>>
>>>
>>>>If we could lazily load services as required, startup time would be
>>>>greatly reduced, footprint as well, and the distributions complexity
>>>>too - since we could just ship everything and only what was needed
>>>>would be used.
>>>>
>>>>
>>>Initial startup time may be reduced, but if an app requires JNDI,
>>>JavaMail or whatever, then the time initialize these will not change,
>>>we have just moved where that time is spent.
>>>
>>>I do like the concept, but I think this may not be feasible in the
>>>short-term... though it would depend a lot on what we use for
>>>component management I would say.
>>>
>>>Personally I do not believe that this buys us all that much. While
>>>lazy loading may (or may not) be nice during development, in a
>>>production environment I would rather have a slow startup where all of
>>>my components are initialized and connected.
>>>
>>>On the other hand, if the component container had knowledge of the
>>>available components/services, then was asked to deploy a web-app,
>>>which requires a web-container, JNDI, whatever, then then container
>>>could initialize and start these components automatically with out
>>>having to explicitly list them as components to be started when the
>>>server comes up.
>>>
>>>I think to make something like this work it would have to be done at a
>>>component connection/dependency level and will depend much on the
>>>component container we use or write.
>>>
>>>--jason
>>>
>>>
>>>
>>
>>
>>
>
>
>
>
--
/*************************************
* Jules Gosnell
* Partner
* Core Developers Network (Europe)
* http://www.coredevelopers.net
*************************************/
Re: [components] - lazy loading...
Posted by Aaron Mulder <am...@alumni.princeton.edu>.
If you deploy a typical EAR, this won't save you a whole lot,
because the server itself will start darn fast, then try to deploy an app,
which loads the deployer, the web & EJB containers, DB pools, JNDI,
transactions, security, web connectors, "EJB connectors", and so on -- the
majority of available services. You may save the effort of JMS and J2CA,
depending on the app, but it's not the huge advantage that it initially
sounds like.
I think we'd get more mileage out of aggressively ensuring that
components start up quickly. You'd get a lot more (let's say) by ensuring
that all 20 components together start up in under a second that you would
by not starting the small number of them that an application required.
Aaron
On Sat, 9 Aug 2003, Jules Gosnell wrote:
> Jason,
>
> I followed pretty much the same though path as you have just outlined
> after posting - then I hung in there and thought some more :-)
>
> Yes it makes things more complex, however:
>
> In development, which is where we have already proven you need to
> capture hearts and minds first, startup time and footprint are
> important. You would be amazed how many people I have come across in my
> JBoss dealings who still have to do a lot of cold deployment... Besides
> as Geronimo developers, therefore the people who will be restarting it
> most, this would speed our own development cycle.
>
> J2EE decomposes nicely into a bunch of APIs javax.xxx.*. We just need a
> persistant registration service. The first time you run up Geronimo, all
> services start, declaring that e.g. they implement javax.naming or
> javax.servlet etc... (package granularity should be enough). Deployers
> also register their interest in e.g. WebContainer:war etc...
>
> These deps are saved away, and next time you start Geronimo, put to good
> use - the dependency tree is shaken out and only services required are
> loaded. This behaviour could be switchable, to load lazily in
> development and proactively in production.
>
> I was thinking that the ClassLoader could do something similar. Instead
> of sitting there watching hundreds of jars being loaded from
> server/x/lib, it could load the index that it saved from the last run
> and then load these classes lazily as and when required. Further startup
> time speedups...
>
> If you upgrade or add new services, jars, their presence would
> invalidate the relevant part of the cached dependency tree or index.
>
> Complex, yes, innovative and possibly valuable - yes.
>
> I don't want something like this to get in the way of getting a server
> out of the door ASAP, but I'd like it to be considered if anyone starts
> working, out on branch, on the ultimate plug-in component system for
> Geronimo services...
>
>
> What do you say :-)
>
>
> Jules
>
>
> Jason Dillon wrote:
>
> >> Imagine that rather than having a set number of configurations which
> >> all load every service at startup, Geronimo could load just the
> >> components needed to provide the services required by your
> >> deployables...
> >
> >
> > I like the concept, but I am not sure how easy this will be to
> > actually implement. To make something like this work, we would need
> > to load configuration metadata for everything on boot, so that we knew
> > what components we had to work with. I mean, how else would the
> > component container know which components it had to work with to start
> > the services required by the deployable?
> >
> >
> >> e.g. you are doing web development, so when you start Geronimo it
> >> just loads the webcontainer component and deploys your webapp. You
> >> webapp decides to load a javax.naming class so the JNDI service is
> >> brought on line, it loads a javax.mail class and looks up a mail
> >> service in JNDI - the mail service is brought up... etc
> >
> >
> > Sounds like a lot of complex magic inside to achieve lazy loading.
> >
> >
> >> If we could lazily load services as required, startup time would be
> >> greatly reduced, footprint as well, and the distributions complexity
> >> too - since we could just ship everything and only what was needed
> >> would be used.
> >
> >
> > Initial startup time may be reduced, but if an app requires JNDI,
> > JavaMail or whatever, then the time initialize these will not change,
> > we have just moved where that time is spent.
> >
> > I do like the concept, but I think this may not be feasible in the
> > short-term... though it would depend a lot on what we use for
> > component management I would say.
> >
> > Personally I do not believe that this buys us all that much. While
> > lazy loading may (or may not) be nice during development, in a
> > production environment I would rather have a slow startup where all of
> > my components are initialized and connected.
> >
> > On the other hand, if the component container had knowledge of the
> > available components/services, then was asked to deploy a web-app,
> > which requires a web-container, JNDI, whatever, then then container
> > could initialize and start these components automatically with out
> > having to explicitly list them as components to be started when the
> > server comes up.
> >
> > I think to make something like this work it would have to be done at a
> > component connection/dependency level and will depend much on the
> > component container we use or write.
> >
> > --jason
> >
>
>
>
Re: [components] [JNDI] [Config] - lazy loading...
Posted by Alex Blewitt <Al...@ioshq.com>.
> J2EE decomposes nicely into a bunch of APIs javax.xxx.*. We just need
> a persistant registration service. The first time you run up Geronimo,
> all services start, declaring that e.g. they implement javax.naming or
> javax.servlet etc... (package granularity should be enough). Deployers
> also register their interest in e.g. WebContainer:war etc...
I think that JNDI would provide an excellent registration service,
especially to follow with previous suggestions about using JNDI for
storing configuration data. [Of course, it pre-supposes that the JNDI
service is one of the 'core' services that is already up :-)]
You could then set aside a context, such as geronimo/startup/services/
and when installing a new service for the first time (such as during
application install, or plugin install at a later stage) then it can
write an entry in there, such as
geronimo/startup/services/JavaMail=org.apache.mail.Xxxx which is then
used to register the service and what it can do.
> These deps are saved away, and next time you start Geronimo, put to
> good use - the dependency tree is shaken out and only services
> required are loaded. This behaviour could be switchable, to load
> lazily in development and proactively in production.
It would probably just be a matter of a startup switch that configured
one behaviour over another. In fact, a generic switch to say 'run in
development mode' or 'run in server mode' would probably have wider
ramifications to other services as well, such as whether they cached
information or dynamically obtained it each time.
> If you upgrade or add new services, jars, their presence would
> invalidate the relevant part of the cached dependency tree or index.
Installing a plug-in could be a matter of writing a few entries into
this list :-)
> I don't want something like this to get in the way of getting a server
> out of the door ASAP, but I'd like it to be considered if anyone
> starts working, out on branch, on the ultimate plug-in component
> system for Geronimo services...
I can see a very easy transition towards this if we go with the JNDI
representation layer with a configuration interface. We could then move
the definition of these services into that registry layer ...
Alex.
Re: [components] - lazy loading...
Posted by Jules Gosnell <ju...@coredevelopers.net>.
Jason,
I followed pretty much the same though path as you have just outlined
after posting - then I hung in there and thought some more :-)
Yes it makes things more complex, however:
In development, which is where we have already proven you need to
capture hearts and minds first, startup time and footprint are
important. You would be amazed how many people I have come across in my
JBoss dealings who still have to do a lot of cold deployment... Besides
as Geronimo developers, therefore the people who will be restarting it
most, this would speed our own development cycle.
J2EE decomposes nicely into a bunch of APIs javax.xxx.*. We just need a
persistant registration service. The first time you run up Geronimo, all
services start, declaring that e.g. they implement javax.naming or
javax.servlet etc... (package granularity should be enough). Deployers
also register their interest in e.g. WebContainer:war etc...
These deps are saved away, and next time you start Geronimo, put to good
use - the dependency tree is shaken out and only services required are
loaded. This behaviour could be switchable, to load lazily in
development and proactively in production.
I was thinking that the ClassLoader could do something similar. Instead
of sitting there watching hundreds of jars being loaded from
server/x/lib, it could load the index that it saved from the last run
and then load these classes lazily as and when required. Further startup
time speedups...
If you upgrade or add new services, jars, their presence would
invalidate the relevant part of the cached dependency tree or index.
Complex, yes, innovative and possibly valuable - yes.
I don't want something like this to get in the way of getting a server
out of the door ASAP, but I'd like it to be considered if anyone starts
working, out on branch, on the ultimate plug-in component system for
Geronimo services...
What do you say :-)
Jules
Jason Dillon wrote:
>> Imagine that rather than having a set number of configurations which
>> all load every service at startup, Geronimo could load just the
>> components needed to provide the services required by your
>> deployables...
>
>
> I like the concept, but I am not sure how easy this will be to
> actually implement. To make something like this work, we would need
> to load configuration metadata for everything on boot, so that we knew
> what components we had to work with. I mean, how else would the
> component container know which components it had to work with to start
> the services required by the deployable?
>
>
>> e.g. you are doing web development, so when you start Geronimo it
>> just loads the webcontainer component and deploys your webapp. You
>> webapp decides to load a javax.naming class so the JNDI service is
>> brought on line, it loads a javax.mail class and looks up a mail
>> service in JNDI - the mail service is brought up... etc
>
>
> Sounds like a lot of complex magic inside to achieve lazy loading.
>
>
>> If we could lazily load services as required, startup time would be
>> greatly reduced, footprint as well, and the distributions complexity
>> too - since we could just ship everything and only what was needed
>> would be used.
>
>
> Initial startup time may be reduced, but if an app requires JNDI,
> JavaMail or whatever, then the time initialize these will not change,
> we have just moved where that time is spent.
>
> I do like the concept, but I think this may not be feasible in the
> short-term... though it would depend a lot on what we use for
> component management I would say.
>
> Personally I do not believe that this buys us all that much. While
> lazy loading may (or may not) be nice during development, in a
> production environment I would rather have a slow startup where all of
> my components are initialized and connected.
>
> On the other hand, if the component container had knowledge of the
> available components/services, then was asked to deploy a web-app,
> which requires a web-container, JNDI, whatever, then then container
> could initialize and start these components automatically with out
> having to explicitly list them as components to be started when the
> server comes up.
>
> I think to make something like this work it would have to be done at a
> component connection/dependency level and will depend much on the
> component container we use or write.
>
> --jason
>
--
/*************************************
* Jules Gosnell
* Partner
* Core Developers Network (Europe)
* http://www.coredevelopers.net
*************************************/
Re: [components] - lazy loading...
Posted by Stephen McConnell <mc...@apache.org>.
Jules Gosnell wrote:
> Stephen McConnell wrote:
>
>>
>>
>> Jason Dillon wrote:
>>
>>>> Imagine that rather than having a set number of configurations
>>>> which all load every service at startup, Geronimo could load just
>>>> the components needed to provide the services required by your
>>>> deployables...
>>>
>>>
>>>
>>>
>>> I like the concept, but I am not sure how easy this will be to
>>> actually implement. To make something like this work, we would need
>>> to load configuration metadata for everything on boot, so that we
>>> knew what components we had to work with. I mean, how else would
>>> the component container know which components it had to work with to
>>> start the services required by the deployable?
>>
>>
>>
>>
>>
>> Within the Avalon Merlin system this is managed using a meta-data
>> model that enables the construction of component type managers. The
>> manager (an Appliance in Merlin terminology) represents a one-to-one
>> mapping with a identifiable deployment scenario. The manager is also
>> responsible for instance creation - so its here that you can get the
>> benefits of lazy activation.
>>
>>
>>>
>>>
>>>
>>>> e.g. you are doing web development, so when you start Geronimo it
>>>> just loads the webcontainer component and deploys your webapp. You
>>>> webapp decides to load a javax.naming class so the JNDI service is
>>>> brought on line, it loads a javax.mail class and looks up a mail
>>>> service in JNDI - the mail service is brought up... etc
>>>
>>>
>>>
>>>
>>> Sounds like a lot of complex magic inside to achieve lazy loading.
>>
>>
>>
>>
>>
>> My guess is that you would end up with a set of notions covering type
>> descriptors, deployment directives, a containment model, and
>> management services. These facilities will be needed anyway - so in
>> principal the notion of lazy activation should be rather trivial if
>> you have the infrastructure in place.
>>
>>>
>>>
>>>> If we could lazily load services as required, startup time would be
>>>> greatly reduced, footprint as well, and the distributions
>>>> complexity too - since we could just ship everything and only what
>>>> was needed would be used.
>>>
>>>
>>>
>>>
>>> Initial startup time may be reduced, but if an app requires JNDI,
>>> JavaMail or whatever, then the time initialize these will not
>>> change, we have just moved where that time is spent.
>>>
>>> I do like the concept, but I think this may not be feasible in the
>>> short-term... though it would depend a lot on what we use for
>>> component management I would say.
>>
>>
>>
>>
>> There is the other-side of the lazy activation equation - namely
>> aggressive disposal. Lazy activation is basically making sure that a
>> component that implements a service is not introduced until it is
>> actually needed. Aggressive disposal is about getting rid of service
>> implementations that are no longer needed.
>>
>>>
>>>
>>> Personally I do not believe that this buys us all that much. While
>>> lazy loading may (or may not) be nice during development, in a
>>> production environment I would rather have a slow startup where all
>>> of my components are initialized and connected.
>>
>>
>>
>>
>> Naturally - putting lazy activation and aggressive disposal together
>> makes for an interesting combination in a production environment.
>>
>>>
>>>
>>> On the other hand, if the component container had knowledge of the
>>> available components/services, then was asked to deploy a web-app,
>>> which requires a web-container, JNDI, whatever, then then container
>>> could initialize and start these components automatically with out
>>> having to explicitly list them as components to be started when the
>>> server comes up.
>>
>>
>>
>>
>> Just a note here - using this approach (sucking in automatically) can
>> be problamatic because you can easily end up with an implementation
>> which effectively delays deployment errors until a service is
>> requested. One way of minimizing this to to do a complete validation
>> of the feasibility of a potential deployment scenario ahead of
>> deployment. This can get really interesting because you can
>> introduce tools that simulate deployment at build time.
>
>
> see my point about development vs production mode - you just flick a
> switch which forces everything to be loaded proactively (c.f.
> precompiling JSPs instead of shipping JSP src to production). You do
> this periodiacally during your dev cycle to ensure all is OK.
Yep - I read you post on this.
Something to throw into the equation here is the potential application
of Maven in developer cycles. Maven provides support for plugins - and
a container plugin can deliver a full or partial deployment. In effect
I think this is where developers (as in developers using a J2EE
solution) should be seeing distinctions between dev/production modes.
Anyway - there are working demonstrations of this sort of thing that we
quickly pull in as needed.
Cheers, Steve.
>
>
> Of course, this would not test the lazy loading framework itself, but
> would test everything else.
>
> Jules
>
>>
>>
>>>
>>>
>>> I think to make something like this work it would have to be done at
>>> a component connection/dependency level and will depend much on the
>>> component container we use or write.
>>
>>
>>
>>
>> Absolutely.
>>
>> Stephen.
>>
>>>
>>> --jason
>>>
>>>
>>>
>>
>
>
--
Stephen J. McConnell
mailto:mcconnell@apache.org
http://www.osm.net
Sent via James running under Merlin as an NT service.
http://avalon.apache.org/sandbox/merlin
Re: [components] - lazy loading...
Posted by Jules Gosnell <ju...@coredevelopers.net>.
Stephen McConnell wrote:
>
>
> Jason Dillon wrote:
>
>>> Imagine that rather than having a set number of configurations which
>>> all load every service at startup, Geronimo could load just the
>>> components needed to provide the services required by your
>>> deployables...
>>
>>
>>
>> I like the concept, but I am not sure how easy this will be to
>> actually implement. To make something like this work, we would need
>> to load configuration metadata for everything on boot, so that we
>> knew what components we had to work with. I mean, how else would the
>> component container know which components it had to work with to
>> start the services required by the deployable?
>
>
>
>
> Within the Avalon Merlin system this is managed using a meta-data
> model that enables the construction of component type managers. The
> manager (an Appliance in Merlin terminology) represents a one-to-one
> mapping with a identifiable deployment scenario. The manager is also
> responsible for instance creation - so its here that you can get the
> benefits of lazy activation.
>
>
>>
>>
>>
>>> e.g. you are doing web development, so when you start Geronimo it
>>> just loads the webcontainer component and deploys your webapp. You
>>> webapp decides to load a javax.naming class so the JNDI service is
>>> brought on line, it loads a javax.mail class and looks up a mail
>>> service in JNDI - the mail service is brought up... etc
>>
>>
>>
>> Sounds like a lot of complex magic inside to achieve lazy loading.
>
>
>
>
> My guess is that you would end up with a set of notions covering type
> descriptors, deployment directives, a containment model, and
> management services. These facilities will be needed anyway - so in
> principal the notion of lazy activation should be rather trivial if
> you have the infrastructure in place.
>
>>
>>
>>> If we could lazily load services as required, startup time would be
>>> greatly reduced, footprint as well, and the distributions complexity
>>> too - since we could just ship everything and only what was needed
>>> would be used.
>>
>>
>>
>> Initial startup time may be reduced, but if an app requires JNDI,
>> JavaMail or whatever, then the time initialize these will not change,
>> we have just moved where that time is spent.
>>
>> I do like the concept, but I think this may not be feasible in the
>> short-term... though it would depend a lot on what we use for
>> component management I would say.
>
>
>
> There is the other-side of the lazy activation equation - namely
> aggressive disposal. Lazy activation is basically making sure that a
> component that implements a service is not introduced until it is
> actually needed. Aggressive disposal is about getting rid of service
> implementations that are no longer needed.
>
>>
>>
>> Personally I do not believe that this buys us all that much. While
>> lazy loading may (or may not) be nice during development, in a
>> production environment I would rather have a slow startup where all
>> of my components are initialized and connected.
>
>
>
> Naturally - putting lazy activation and aggressive disposal together
> makes for an interesting combination in a production environment.
>
>>
>>
>> On the other hand, if the component container had knowledge of the
>> available components/services, then was asked to deploy a web-app,
>> which requires a web-container, JNDI, whatever, then then container
>> could initialize and start these components automatically with out
>> having to explicitly list them as components to be started when the
>> server comes up.
>
>
>
> Just a note here - using this approach (sucking in automatically) can
> be problamatic because you can easily end up with an implementation
> which effectively delays deployment errors until a service is
> requested. One way of minimizing this to to do a complete validation
> of the feasibility of a potential deployment scenario ahead of
> deployment. This can get really interesting because you can introduce
> tools that simulate deployment at build time.
see my point about development vs production mode - you just flick a
switch which forces everything to be loaded proactively (c.f.
precompiling JSPs instead of shipping JSP src to production). You do
this periodiacally during your dev cycle to ensure all is OK.
Of course, this would not test the lazy loading framework itself, but
would test everything else.
Jules
>
>
>>
>>
>> I think to make something like this work it would have to be done at
>> a component connection/dependency level and will depend much on the
>> component container we use or write.
>
>
>
> Absolutely.
>
> Stephen.
>
>>
>> --jason
>>
>>
>>
>
--
/*************************************
* Jules Gosnell
* Partner
* Core Developers Network (Europe)
* http://www.coredevelopers.net
*************************************/
Re: [components] - lazy loading...
Posted by Stephen McConnell <mc...@apache.org>.
Jason Dillon wrote:
>> Imagine that rather than having a set number of configurations which
>> all load every service at startup, Geronimo could load just the
>> components needed to provide the services required by your
>> deployables...
>
>
> I like the concept, but I am not sure how easy this will be to
> actually implement. To make something like this work, we would need
> to load configuration metadata for everything on boot, so that we knew
> what components we had to work with. I mean, how else would the
> component container know which components it had to work with to start
> the services required by the deployable?
Within the Avalon Merlin system this is managed using a meta-data model
that enables the construction of component type managers. The manager
(an Appliance in Merlin terminology) represents a one-to-one mapping
with a identifiable deployment scenario. The manager is also
responsible for instance creation - so its here that you can get the
benefits of lazy activation.
>
>
>
>> e.g. you are doing web development, so when you start Geronimo it
>> just loads the webcontainer component and deploys your webapp. You
>> webapp decides to load a javax.naming class so the JNDI service is
>> brought on line, it loads a javax.mail class and looks up a mail
>> service in JNDI - the mail service is brought up... etc
>
>
> Sounds like a lot of complex magic inside to achieve lazy loading.
My guess is that you would end up with a set of notions covering type
descriptors, deployment directives, a containment model, and management
services. These facilities will be needed anyway - so in principal the
notion of lazy activation should be rather trivial if you have the
infrastructure in place.
>
>
>> If we could lazily load services as required, startup time would be
>> greatly reduced, footprint as well, and the distributions complexity
>> too - since we could just ship everything and only what was needed
>> would be used.
>
>
> Initial startup time may be reduced, but if an app requires JNDI,
> JavaMail or whatever, then the time initialize these will not change,
> we have just moved where that time is spent.
>
> I do like the concept, but I think this may not be feasible in the
> short-term... though it would depend a lot on what we use for
> component management I would say.
There is the other-side of the lazy activation equation - namely
aggressive disposal. Lazy activation is basically making sure that a
component that implements a service is not introduced until it is
actually needed. Aggressive disposal is about getting rid of service
implementations that are no longer needed.
>
>
> Personally I do not believe that this buys us all that much. While
> lazy loading may (or may not) be nice during development, in a
> production environment I would rather have a slow startup where all of
> my components are initialized and connected.
Naturally - putting lazy activation and aggressive disposal together
makes for an interesting combination in a production environment.
>
>
> On the other hand, if the component container had knowledge of the
> available components/services, then was asked to deploy a web-app,
> which requires a web-container, JNDI, whatever, then then container
> could initialize and start these components automatically with out
> having to explicitly list them as components to be started when the
> server comes up.
Just a note here - using this approach (sucking in automatically) can be
problamatic because you can easily end up with an implementation which
effectively delays deployment errors until a service is requested. One
way of minimizing this to to do a complete validation of the feasibility
of a potential deployment scenario ahead of deployment. This can get
really interesting because you can introduce tools that simulate
deployment at build time.
>
>
> I think to make something like this work it would have to be done at a
> component connection/dependency level and will depend much on the
> component container we use or write.
Absolutely.
Stephen.
>
> --jason
>
>
>
--
Stephen J. McConnell
mailto:mcconnell@apache.org
http://www.osm.net
Sent via James running under Merlin as an NT service.
http://avalon.apache.org/sandbox/merlin
Re: [components] - lazy loading...
Posted by Jason Dillon <ja...@coredevelopers.net>.
> Imagine that rather than having a set number of configurations which
> all load every service at startup, Geronimo could load just the
> components needed to provide the services required by your
> deployables...
I like the concept, but I am not sure how easy this will be to actually
implement. To make something like this work, we would need to load
configuration metadata for everything on boot, so that we knew what
components we had to work with. I mean, how else would the component
container know which components it had to work with to start the
services required by the deployable?
> e.g. you are doing web development, so when you start Geronimo it just
> loads the webcontainer component and deploys your webapp. You webapp
> decides to load a javax.naming class so the JNDI service is brought on
> line, it loads a javax.mail class and looks up a mail service in JNDI
> - the mail service is brought up... etc
Sounds like a lot of complex magic inside to achieve lazy loading.
> If we could lazily load services as required, startup time would be
> greatly reduced, footprint as well, and the distributions complexity
> too - since we could just ship everything and only what was needed
> would be used.
Initial startup time may be reduced, but if an app requires JNDI,
JavaMail or whatever, then the time initialize these will not change,
we have just moved where that time is spent.
I do like the concept, but I think this may not be feasible in the
short-term... though it would depend a lot on what we use for component
management I would say.
Personally I do not believe that this buys us all that much. While
lazy loading may (or may not) be nice during development, in a
production environment I would rather have a slow startup where all of
my components are initialized and connected.
On the other hand, if the component container had knowledge of the
available components/services, then was asked to deploy a web-app,
which requires a web-container, JNDI, whatever, then then container
could initialize and start these components automatically with out
having to explicitly list them as components to be started when the
server comes up.
I think to make something like this work it would have to be done at a
component connection/dependency level and will depend much on the
component container we use or write.
--jason