Who's in charge of deploying EARs?

[Jacek Laskowski <ja...@hp.com>]

Hi,

Unless I'm mistaken, there're 4 planners: Service, Jetty, Connector and 
EJBModule. Their names are selfexplanatory as to what they can plan to 
deploy. The missing piece is a EAR deployer. Once a scanner (FSS or WDS) 
finds a META-INF/MANIFEST.MF it adds the resource which contains the 
manifest to a list of resources to be deployed. As none of the planners 
are willing to take over planning of EAR deployment (as they're looking 
for other files in META-INF as indication of its correctness), the only 
way to deploy that EAR is to remove its META-INF directory.

I've been thinking about how to achieve EAR deployment and what the 
planner could do. The main thing the planner must do would be to plan to 
create a class-space for the EAR and then ask other planners for 
deployment of the rest and tell them what classspace it should belong 
to. I think other EJBs/WARs/RARs that are not part of EAR should belong 
to DEFAULT EAR. It would then become a reserved name, so no DEFAULT.ear 
may get deployed.

I think I'm ready to tackle the issue and prepare something in the 
coming days. Before I do it, though, I'd like to find out if it's 
already being worked out. Also, your comments/recomendation are highly 
appreciated as while I'm reading the message I don't see any problems to 
  sort it out very easily (I know, the devil is in details).

BTW: What is org.apache.geronimo.kernel.deployment.ApplicationDeployer 
for? Would it be of help somehow?

We could also talk it over on IRC tonight.

Jacek

Re: Who's in charge of deploying EARs?

[Jeremy Boynes <je...@coredevelopers.net>]

Aaron Mulder wrote:

> On Tue, 6 Jan 2004, Jeremy Boynes wrote:
> 
>>Yeah, sorry - it's taking longer than expected but I have to work as 
>>well :-)
> 
> 
> 	If you think it's going to be a problem, we could always put it in 
> a branch so more people can work on it before it's "done".
> 
> 

I'm trying to get to that stage. The issue isn't really with my stuff 
but that I am dependent on changes that Dain is making to the GBean that 
would break everything.


>>It is very close to the 88 model.
>>
>>There are three phases: "deploy", "distribute" and "run"
>>
>>In the "deploy" phase, a ConfigurationBuilder takes a set of URLs and 
>>runs them by a set of Deployers to build a Configuration. This is a 
>>jar-formatted file containing code plus a set of persisted GBeans. I'm 
>>thinking of calling it a "CAR" file - any thoughts?
>>
>>Because it is a single file it is easy to copy between nodes (somehow). 
>>The "distribute" phase does just that; it is copied to each target which 
>>then stores in a local ConfigurationStore.
>>
>>Once all the distribution has happened, then the various targets can be 
>>started by bringing the Configuration online. This will create a new 
>>classloader, unpersist the GBeans and then start them all (with the 
>>DependencyService sorting out the start order).
> 
> 
> 	I guess my concern is that the JSR-88 "distribute" stage (which
> here seems to be deploy + distribute) should do validation.  So around the
> time of CAR assembly, we want to make sure that all the DDs are valid, all
> the required code is present, the classes have the right methods and
> superclasses, all resource refs are resolved, etc.  That can be done once,
> by the ConfigurationBuilder (but it needs a live ClassLoader to do it).  

Yes, this is all part of "deployment" as J2EE defines it.

The ConfigurationBuilder actually runs in three steps:
* work out the combined classpath for the configuration and build a
   ClassLoader for it
* build the GBeans
* construct the CAR, including any needed classpath entries and the
   persisted form of the GBean state

> Then when we distribute it to servers (however), they may do more
> validation if there's anything different from server to server (i.e. if
> there are different security realms per server, they could validate the
> security mappings, but there may be nothing to do here).

The intention is that the CAR is distributable from server to server so 
no further validation would be performed by the configuration system. 
However, the GBeans themselves can fail to start (for any number of 
reasons). If this happens, then the intention is the configuration 
change gets rolled back - the new configuration would be unloaded and if 
it was replacing one that would be restarted (which could also fail)).

> 	Bottom line, when you get to the "run" stage (which I assume 
> corresponds to calling "start" on the JSR-77 MBean)

It involves creating a classloader for the code in the CAR, loading the 
persisted GBeans and then calling "start".

> I want the only 
> possible failure to be that some resource that we lazily resolve isn't 
> present at runtime.  I don't want you to get that far only to discover 
> that you misspelled the EJB class name in your DD, or whatever.
> 	Is that pretty much what you have in mind?
> 

Yes

-- 
Jeremy

Re: Who's in charge of deploying EARs?

[Aaron Mulder <am...@alumni.princeton.edu>]

On Tue, 6 Jan 2004, Jeremy Boynes wrote:
> Yeah, sorry - it's taking longer than expected but I have to work as 
> well :-)

	If you think it's going to be a problem, we could always put it in 
a branch so more people can work on it before it's "done".

> It is very close to the 88 model.
> 
> There are three phases: "deploy", "distribute" and "run"
> 
> In the "deploy" phase, a ConfigurationBuilder takes a set of URLs and 
> runs them by a set of Deployers to build a Configuration. This is a 
> jar-formatted file containing code plus a set of persisted GBeans. I'm 
> thinking of calling it a "CAR" file - any thoughts?
> 
> Because it is a single file it is easy to copy between nodes (somehow). 
> The "distribute" phase does just that; it is copied to each target which 
> then stores in a local ConfigurationStore.
> 
> Once all the distribution has happened, then the various targets can be 
> started by bringing the Configuration online. This will create a new 
> classloader, unpersist the GBeans and then start them all (with the 
> DependencyService sorting out the start order).

	I guess my concern is that the JSR-88 "distribute" stage (which
here seems to be deploy + distribute) should do validation.  So around the
time of CAR assembly, we want to make sure that all the DDs are valid, all
the required code is present, the classes have the right methods and
superclasses, all resource refs are resolved, etc.  That can be done once,
by the ConfigurationBuilder (but it needs a live ClassLoader to do it).  
Then when we distribute it to servers (however), they may do more
validation if there's anything different from server to server (i.e. if
there are different security realms per server, they could validate the
security mappings, but there may be nothing to do here).
	Bottom line, when you get to the "run" stage (which I assume 
corresponds to calling "start" on the JSR-77 MBean), I want the only 
possible failure to be that some resource that we lazily resolve isn't 
present at runtime.  I don't want you to get that far only to discover 
that you misspelled the EJB class name in your DD, or whatever.
	Is that pretty much what you have in mind?

Aaron

Re: Who's in charge of deploying EARs?

[Jeremy Boynes <je...@coredevelopers.net>]

Aaron Mulder wrote:

> 	I believe Jeremy is in the midst of some significant changes to 
> the deployment process.  Perhaps he can comment, but I'm guessing this 
> isn't the best time to work on new deployment features.
> 

Yeah, sorry - it's taking longer than expected but I have to work as 
well :-)

> 	The ApplicationDeployer is the JSR-88 server side.  All the 
> unimplemented methods are basically there to support JSR-88 client calls 
> that we have to support.  The implemented methods actually set up 
> deployment operations, and are used for everything that's deployed 
> (services, apps, etc.).
> 
> 	The main shortcoming as far as JSR-88 goes is currently that
> JSR-88 uses a distribute-then-start lifecycle, whereas the current
> planners roll everything into a single "deploy" step.  Jeremy, do you also
> have thoughts on whether the revisions will make it easier to explicitly
> split out the "distribute" step?  My sense is that it will, because we'll
> be handling the ClassSpace in a better way and so on, so that we'll be
> able to do more work during "distribution" and we can simply flick a 
> switch for the "start".
> 

It is very close to the 88 model.

There are three phases: "deploy", "distribute" and "run"

In the "deploy" phase, a ConfigurationBuilder takes a set of URLs and 
runs them by a set of Deployers to build a Configuration. This is a 
jar-formatted file containing code plus a set of persisted GBeans. I'm 
thinking of calling it a "CAR" file - any thoughts?

Because it is a single file it is easy to copy between nodes (somehow). 
The "distribute" phase does just that; it is copied to each target which 
then stores in a local ConfigurationStore.

Once all the distribution has happened, then the various targets can be 
started by bringing the Configuration online. This will create a new 
classloader, unpersist the GBeans and then start them all (with the 
DependencyService sorting out the start order).


So far I have the configuration stuff working but only have around 75% 
of the service deployer working.

--
Jeremy

Re: Who's in charge of deploying EARs?

[Aaron Mulder <am...@alumni.princeton.edu>]

	I believe Jeremy is in the midst of some significant changes to 
the deployment process.  Perhaps he can comment, but I'm guessing this 
isn't the best time to work on new deployment features.

	The ApplicationDeployer is the JSR-88 server side.  All the 
unimplemented methods are basically there to support JSR-88 client calls 
that we have to support.  The implemented methods actually set up 
deployment operations, and are used for everything that's deployed 
(services, apps, etc.).

	The main shortcoming as far as JSR-88 goes is currently that
JSR-88 uses a distribute-then-start lifecycle, whereas the current
planners roll everything into a single "deploy" step.  Jeremy, do you also
have thoughts on whether the revisions will make it easier to explicitly
split out the "distribute" step?  My sense is that it will, because we'll
be handling the ClassSpace in a better way and so on, so that we'll be
able to do more work during "distribution" and we can simply flick a 
switch for the "start".

Aaron

On Tue, 6 Jan 2004, Jacek Laskowski wrote:
> Hi,
> 
> Unless I'm mistaken, there're 4 planners: Service, Jetty, Connector and 
> EJBModule. Their names are selfexplanatory as to what they can plan to 
> deploy. The missing piece is a EAR deployer. Once a scanner (FSS or WDS) 
> finds a META-INF/MANIFEST.MF it adds the resource which contains the 
> manifest to a list of resources to be deployed. As none of the planners 
> are willing to take over planning of EAR deployment (as they're looking 
> for other files in META-INF as indication of its correctness), the only 
> way to deploy that EAR is to remove its META-INF directory.
> 
> I've been thinking about how to achieve EAR deployment and what the 
> planner could do. The main thing the planner must do would be to plan to 
> create a class-space for the EAR and then ask other planners for 
> deployment of the rest and tell them what classspace it should belong 
> to. I think other EJBs/WARs/RARs that are not part of EAR should belong 
> to DEFAULT EAR. It would then become a reserved name, so no DEFAULT.ear 
> may get deployed.
> 
> I think I'm ready to tackle the issue and prepare something in the 
> coming days. Before I do it, though, I'd like to find out if it's 
> already being worked out. Also, your comments/recomendation are highly 
> appreciated as while I'm reading the message I don't see any problems to 
>   sort it out very easily (I know, the devil is in details).
> 
> BTW: What is org.apache.geronimo.kernel.deployment.ApplicationDeployer 
> for? Would it be of help somehow?
> 
> We could also talk it over on IRC tonight.
> 
> Jacek
>