TC4 - Avalon Wrapper.

[Paul Hammant <Pa...@yahoo.com>]

Hi Folks,

I have written a wrapper for Catalina that allows it to be deployed as a 
"sar" file into Apache's Avalon/Phoenix server application framework. 
 To some it seems unecessary, but Phoenix can serve multiple server 
applications from one JVM.  Currently these could include :
JAMES (the mail / news server) plus it's DNS server,
FtpServer (new Apache micro-project that's part of Avalon),
HypersonicSQL (waiting in the wings until the HSQLDB team at Sourceforge 
commit some changes).
A SOAP server - publish any API as WSDL (courtesy of Glue, until Axis is 
finished).
Jesktop (my desktop layer)
Acme.Serve (another http server)
HTTP Proxy server (part of Avalon as a demo)

Soon we hope to be followed by an EJB container...

Anyway,  I am here to list some compromises I had to make when I wrap 
Catalina and launch it inside the context of Avalon/Phoenix.  I've 
simulated the processing of "catalina.sh embedded" which launches the 
org.apache.catalina.startup.Embedded class.  All bar the shutdown part 
of main() has been used as inspiration for the wrapper logic.  Problems :

1) Env vars "catalina.home" and "catalina.base" suck (sorry).  It 
basically means that there can't be more than one instace of catalina 
running in the same VM.  Yes, ordinarily that would be undesirable, but 
we don't place such restrictions on other apps (bind-to-port issues 
considered).  Is there anyway that CatalinaHome could be an object that 
it passed into Embedded's constructor?  It then passed around to all 
that needs it.

2) Primordial classloader.  Some components like StandardEngine assume 
they must be loaded from the primordial classloader.  That's not the 
case for the wrapped Catalina.  It's my feeling that that type of thing 
contributes to Catalina not being able to find servlet classes (that are 
definately visible to the launched catalina).  To overcome the issue I 
had to put servlet.jar in Phoenix's lib directory.  That kinda sucks as 
the "sar" is no longer drag and drop installable.  I'd hope that was 
easy to fix, but it's a bit of a black hole for me to pinpoint the 
actual line of code that's wrong.

3) Connection handling.  We're proud to say that the connection manager 
in Phoenix is stolen from Catalina.  We'd like to be able to use that 
connection manager to listen on ports and pass sockets to the innards of 
Catalina.  In so doing it does it's own threadpooling.  Would it be 
possible for us to be inspired by your HttpConnector and have a 
connector that implements Connector and have one that takes Sockets 
handed from our ConnectionManager?

Lastly congrats on TC4, it's in all other senses a really excellent bit 
of coding.  Quality work.

Regards,

- Paul H

Re: TC4 - Avalon Wrapper.

[Paul Hammant <Pa...@yahoo.com>]

Craig,

>[ .. Avalon intro, and existing blocks snipped .. ]
>One general question -- are you creating a class loader hierarchy (like
>the Bootstrap class does), or are you loading everything from a single
>class loader?  The latter approach might cause you some grief, because
>it's never been tested that way.
>
Well, Phoenix is loaded by the primordial.  Phoenix unpacks each server 
application and then loads the blocks and support libs in a seperate 
classloader (which via parent, can see the phoenix classes).  In the 
case of the wrapped Catalina, I am loading a new classloader that loads 
server/lib/*.jar, common/lib/*jar and bin/bootstrap.jar together.  The 
block I have written is not compiled against catalina classes, it 
emulates a few named by catalina's Embedded and uses reflection (very 
hard coded) to invoke methods therin.  It would have been more glamorous 
for me to use BSF etc, but I chose not to because of the expected tight 
coupling and single need.  Have a look at 
http://cvs.apache.org/viewcvs/jakarta-avalon-cornerstone/apps/bay/src/java/org/apache/avalon/bay/delegate/CatalinaClassLoader.java 
for the aweful truth.

>In the particular case of servlet.jar, these classes *must* be the same
>for both internal Catalina access and web application access.  In
>practice, that means it must be loaded from a common parent class loader
>(which is what the standard installation does), or the webapp class
>loader's parent must be the one that includes servlet.jar and all the
>Catalina internal classes (not desireable from a security point of view).
>
Yup, logical.

>>1) Env vars "catalina.home" and "catalina.base" suck (sorry).  It
>>basically means that there can't be more than one instace of catalina
>>running in the same VM.  Yes, ordinarily that would be undesirable, but
>>we don't place such restrictions on other apps (bind-to-port issues
>>considered).  Is there anyway that CatalinaHome could be an object that
>>it passed into Embedded's constructor?  It then passed around to all
>>that needs it.
>>
>
>It's true that the current design never contemplated more than one
>instance in the same VM.  I'd be happy to look at what it would take to
>change this, but *not* on the day we're trying to get the 4.0 final
>release out (i.e. today).  There are quite a few references to these
>system properties, and we'd need to deal with all of them.
>
Indeed, I can wait.  Tomcat 4.0's release is far more important than 
needs of the Avalon team.  Privately, we'd thought we'd only be able to 
mount Tomcat as an Avalon component at 5.0 and are very pleased to see 
it well designed and built enough for us to do it more or less now. 
 4.01, 4.1 or 4.5 for a more perfect solution will still be very pleasing.

>>2) Primordial classloader.  Some components like StandardEngine assume
>>they must be loaded from the primordial classloader.  That's not the
>>case for the wrapped Catalina.  It's my feeling that that type of thing
>>contributes to Catalina not being able to find servlet classes (that are
>>definately visible to the launched catalina).  To overcome the issue I
>>had to put servlet.jar in Phoenix's lib directory.  That kinda sucks as
>>the "sar" is no longer drag and drop installable.  I'd hope that was
>>easy to fix, but it's a bit of a black hole for me to pinpoint the
>>actual line of code that's wrong.
>>
>
>I'd be interested in where you find StandardEngine making an assumption
>like that.  Conceptually, the internal components generally assume that
>they are being loaded from the *same* class loader (because they use the
>"new" operator to create new instances), but I don't see where any
>assumption about the primoridal class loader is happening.
>

I could be wrong, but the blocked setParent of StandardEngine prevents 
me setting a Container that would allow the getParentClassLoader() to 
indicate that the primordial classloader is not next, it's the custom 
rolled one (see url above).  Just a guess, I've no interactive debugger 
that works for Avalon loading SAR apps loading Catalina.

>>3) Connection handling.  We're proud to say that the connection manager
>>in Phoenix is stolen from Catalina.
>>
>
>I'd normally make a couple of snide jokes about "beggar" and "thief",
>but it's kinda hard to type smiley faces after last week ...
>
Indeed.

>> We'd like to be able to use that
>>connection manager to listen on ports and pass sockets to the innards of
>>Catalina.  In so doing it does it's own threadpooling.  Would it be
>>possible for us to be inspired by your HttpConnector and have a
>>connector that implements Connector and have one that takes Sockets
>>handed from our ConnectionManager?
>>
>
>Conceptually, this seems pretty doable -- you could subclass the existing
>classes (so as to reuse most of the protocol processing) but change how
>the connections are grabbed, and how they are communicated to the
>corresponding processor.
>
I've looked at subclassing HttpConnector, it feels wrong to extend 
something that want's to use Thread and then deliberatly not use Thread. 
 Possible but wrong.  I think the best plan would be to create an 
intermediate class below HttpConnector called, say, HttpBase and have 
that extended by the non threaded, non socket server delegate for avalon 
too.

Noted that this is not now.  4.01 or 4.1 as mentioned before.

>Remy is also working on a design for a higher performance HTTP/1.1
>connector.  I saw that he just posted some initial work, in the "coyote"
>directory of the "jakarta-tomcat-connectors" repository.  It would be
>useful to get you plugged in to the development effort to make sure that
>the abstractions you would need will be there.
>
I'll take the hint and dive in that connection pool.

Regards,

- Paul H

Re: TC4 - Avalon Wrapper.

["Craig R. McClanahan" <cr...@apache.org>]

Hi Paul,

Thanks for the comments.  See embedded.

On Mon, 17 Sep 2001, Paul Hammant wrote:

> Date: Mon, 17 Sep 2001 09:41:17 +0000
> From: Paul Hammant <Pa...@yahoo.com>
> Reply-To: tomcat-dev@jakarta.apache.org
> To: tomcat-dev@jakarta.apache.org
> Subject: TC4 - Avalon Wrapper.
>
> Hi Folks,
>
> I have written a wrapper for Catalina that allows it to be deployed as a
> "sar" file into Apache's Avalon/Phoenix server application framework.
>  To some it seems unecessary, but Phoenix can serve multiple server
> applications from one JVM.  Currently these could include :
> JAMES (the mail / news server) plus it's DNS server,
> FtpServer (new Apache micro-project that's part of Avalon),
> HypersonicSQL (waiting in the wings until the HSQLDB team at Sourceforge
> commit some changes).
> A SOAP server - publish any API as WSDL (courtesy of Glue, until Axis is
> finished).
> Jesktop (my desktop layer)
> Acme.Serve (another http server)
> HTTP Proxy server (part of Avalon as a demo)
>
> Soon we hope to be followed by an EJB container...
>
> Anyway,  I am here to list some compromises I had to make when I wrap
> Catalina and launch it inside the context of Avalon/Phoenix.  I've
> simulated the processing of "catalina.sh embedded" which launches the
> org.apache.catalina.startup.Embedded class.  All bar the shutdown part
> of main() has been used as inspiration for the wrapper logic.  Problems :
>

One general question -- are you creating a class loader hierarchy (like
the Bootstrap class does), or are you loading everything from a single
class loader?  The latter approach might cause you some grief, because
it's never been tested that way.

In the particular case of servlet.jar, these classes *must* be the same
for both internal Catalina access and web application access.  In
practice, that means it must be loaded from a common parent class loader
(which is what the standard installation does), or the webapp class
loader's parent must be the one that includes servlet.jar and all the
Catalina internal classes (not desireable from a security point of view).

> 1) Env vars "catalina.home" and "catalina.base" suck (sorry).  It
> basically means that there can't be more than one instace of catalina
> running in the same VM.  Yes, ordinarily that would be undesirable, but
> we don't place such restrictions on other apps (bind-to-port issues
> considered).  Is there anyway that CatalinaHome could be an object that
> it passed into Embedded's constructor?  It then passed around to all
> that needs it.
>

It's true that the current design never contemplated more than one
instance in the same VM.  I'd be happy to look at what it would take to
change this, but *not* on the day we're trying to get the 4.0 final
release out (i.e. today).  There are quite a few references to these
system properties, and we'd need to deal with all of them.

> 2) Primordial classloader.  Some components like StandardEngine assume
> they must be loaded from the primordial classloader.  That's not the
> case for the wrapped Catalina.  It's my feeling that that type of thing
> contributes to Catalina not being able to find servlet classes (that are
> definately visible to the launched catalina).  To overcome the issue I
> had to put servlet.jar in Phoenix's lib directory.  That kinda sucks as
> the "sar" is no longer drag and drop installable.  I'd hope that was
> easy to fix, but it's a bit of a black hole for me to pinpoint the
> actual line of code that's wrong.
>

I'd be interested in where you find StandardEngine making an assumption
like that.  Conceptually, the internal components generally assume that
they are being loaded from the *same* class loader (because they use the
"new" operator to create new instances), but I don't see where any
assumption about the primoridal class loader is happening.

> 3) Connection handling.  We're proud to say that the connection manager
> in Phoenix is stolen from Catalina.

I'd normally make a couple of snide jokes about "beggar" and "thief",
but it's kinda hard to type smiley faces after last week ...

>  We'd like to be able to use that
> connection manager to listen on ports and pass sockets to the innards of
> Catalina.  In so doing it does it's own threadpooling.  Would it be
> possible for us to be inspired by your HttpConnector and have a
> connector that implements Connector and have one that takes Sockets
> handed from our ConnectionManager?
>

Conceptually, this seems pretty doable -- you could subclass the existing
classes (so as to reuse most of the protocol processing) but change how
the connections are grabbed, and how they are communicated to the
corresponding processor.

Remy is also working on a design for a higher performance HTTP/1.1
connector.  I saw that he just posted some initial work, in the "coyote"
directory of the "jakarta-tomcat-connectors" repository.  It would be
useful to get you plugged in to the development effort to make sure that
the abstractions you would need will be there.

> Lastly congrats on TC4, it's in all other senses a really excellent bit
> of coding.  Quality work.
>
> Regards,
>
> - Paul H
>
>

Craig