Tomcat3.3

[cm...@yahoo.com]

Hi,

As you know, 1-2 months ago, when Tomcat3.2 was frozen, I started to work
on all the long-delayed things that were too "dangerous" to implement in
tomcat3.2 time frame ( the list is at the end ).

Most of that has been implemented ( I sent quite a few mails in addition
to the normal CVS commits describing what's done ). The result is ( IMO
) pretty good, but as expected the code have many bugs ( after you read
the lists of changes, you'll understand why :-)

Unfurtunately, I had far less time than I expected, and the release of 3.2
took far more than I believed. A big thanks to Larry, Nacho and all the
other who fixed so many bugs during the last months, sorry I was unable to
help ( I do have a job and it eats a lot of free time ). I tried to at
least touch the targets and implement the features that were on my
list for far too long.

Tomcat3.2 had a large number of bugfixes that were not incorporated
in 3.3, and 3.3 has a probably a (possibly large) number
of new bugs resulting from the big changes and refactoring. 

Now, that I have a bit more time ( I hope :-), I can start spending 
more time working on tomcat. There are 2 possible ways to do that:

1. In the current "main branch", incorporate all the fixes from tomcat3.2,
plus fixes for the new bugs. 

2. Start with tomcat3.2, and re-do all the changes. 


My preference is toward 2 - even if it will probably take a longer
time. I think this is the normal "evolution", i.e. building on top of the
stable release. Because of the long release cycle 3.3 and 3.2 are out of
sync, and I don't want to loose any of the "evolutions" and fixes that
happened. 


Regarding the changes that happened so far in 3.3 ( and will have to be
re-implemented if we go with (2) ) - I have a list ( probably incomplete
).

Most of them were sent as proposals before - you can send feedback again.

The goal is of course to continue the evolution and produce a better
servlet container, and also to prevent a fragmentation - I'll try to be as
flexible as possible and let any "controversial" item at the end.  As
usually, if you want to -1 any of the changes, I'm sure you'll provide a
better solution and volunteer to implement it :-) 

( this mail is related with the development of tomcat3 - it has nothing to
do with other servlet containers and designs - it's just a plan to
continue and enhance the current jakarta-tomcat servlet container. !) 


CHANGES 3.3
=================

CORE

- Only one type of Interceptor - remove the separation between Context/Request
Interceptor

- Use introspection to detect the hooks implemented by an interceptor, call
only the interceptors that are implementing the hooks.

- Added handleError hook, refactoring/modularization of error handling - it 
is now implemented by a module that can be changed ( see Larry's requirements )

- Added documentation and code to enforce "states" and the order in wich the
hooks are called based on tomcat and context state. For example addContext()
can't be called before tomcat is initialized ( all engineInit done ), etc

- Optional separation of tomcat class loader and webapps classloaders, 
 similar with Catalina's 

- Rewrite of class reloading, the check for expire is no longer done per/request,
but in a background thread. Added DependManager, that allows checks for all
resources, not only classes and works with any class loader ( including 
URLClassLoader )

- use of URLClassLoader if JDK1.2 is used ( faster, more secure )

- removal of all deprecated methods and calls

- more documentation

- removal of ServletLoader interface, it's functionality is already provided
in interceptors 

- insure org.apache.tomcat.core and facade are layered and remove all spaghetti 
calls between layers. 

- rewrite of ServletWrapper, separate base ( core ) functionality from the 
jsp and servlet-specific.

- removal of ServletConnector, it's functionality can be provided in standard
interceptors. By using normal interceptors a Connector can be better configured
( no more <property name/value /> ), and it can affect all request processing
stages ( to allow future server-based authentication, more performance, etc ) 

- Move back to the original Request/Response base classes instead of interfaces.
The interfaces were introduced in an effort to merge with Catalina ( by using
common interfaces), but it's a 2-sided process. 

- Removal of FacadeManager - it's not needed as the same functionality can be 
provided by Interceptors ( to plug the facade ) and simpler mechanisms to
access to internal and check "trusted" apps.

- consolidation of error messages in tomcat.res, consistent with other
apache projects ( and simpler to maintain )

- Removal of Constants - each object will store it's own constants
( tomcat name/version -> ContextManager, 
  request attributes -> Request,  context attributes -> context,
  static file resources -> StaticInterceptor, etc ). This also 
prevent future inconsistencies ( most of the Constants are not used
in 3.2 anyway )

- Make the "Buffer" a top level element, remove BufferedInputStream/BufferedOutputStream.
That allow a big increase in performance and allow future improvements by eliminating
multiple "copy" and better reuse of objects.

- Refactoring for all connectors - use more Interceptor features, separate the
tomcat-specific code from the protocol-specific code ( for possible future reuse )

- Use of MessageBytes for all request components

- major simplification of the overal design - fewer and simpler "contracts"
( Interceptor, 6 core objects ) instead of ( Interceptor, Loader, Connector, 12 
core objects ), real layering ( instead of spaghetti ), almost all functionality
is now in modules ( like Apache !)  

- refactoring of session - use Interceptors for full access to all request
stages, keep tomcat-independent code separated and reusable.

UTILS

- consolidation/refactoring of all tomcat-independent and general-purpose/
reusable code into org.apache.tomcat.util.[log, etc ]

- new utils

BUILD/STARTUP

- generate separate jar files for each component ( utils, core, facade, modules )
That insures the layers are properly separated, utils can be reused.

- use JDK1.2 Main-Class: and a custom starter - deprecate the shell/bat 
scripts and use java to process arguments, guess tomcat home, etc


PLANNED ( not yet implemented )

- the EncodingInterceptor that uses MessageBytes and find the right charset 

- Use custom-code for Parameters, Cookies ( faster data strucures, like in
MimeHeaders )

- Merge of all modules -> org.apache.tomcat.modules.[config,server,request,etc]
instead of org.apache.tomcat.context/request

- A new configuration mechanism that deals with virtual hosts, allow simpler
installation and configuration for contexts

- Improve Nacho's per/context interceptors, allow "profiles" ( sets of interceptors
that are shared by multiple contexts )

- removal of helpers - use either util ( for tomcat-independent, reusable functionality,
or Interceptors for tomcat-specific code )



-------
Costin






 

Re: Tomcat3.3

[Glenn Nielsen <gl...@voyager.apg.more.net>]

cmanolache@yahoo.com wrote:
> 
> Tomcat3.2 had a large number of bugfixes that were not incorporated
> in 3.3, and 3.3 has a probably a (possibly large) number
> of new bugs resulting from the big changes and refactoring.
> 
> Now, that I have a bit more time ( I hope :-), I can start spending
> more time working on tomcat. There are 2 possible ways to do that:
> 
> 1. In the current "main branch", incorporate all the fixes from tomcat3.2,
> plus fixes for the new bugs.
> 
> 2. Start with tomcat3.2, and re-do all the changes.
> 

I think choice 1) is the better approach.  When 3.2 is released, harvest
all the commits to 32 since the 32 release fork and merge them into the
3.3 dev branch if they still apply.  Then track any bug fixes to the
32 release.  There is still alot of work to be done on 33, and at some
point we may want to release bug fixes for 3.2 as a 3.2.x release.

Regards,

Glenn

----------------------------------------------------------------------
Glenn Nielsen             glenn@more.net | /* Spelin donut madder    |
MOREnet System Programming               |  * if iz ina coment.      |
Missouri Research and Education Network  |  */                       |
----------------------------------------------------------------------

RE: Tomcat3.3

[Paulo Gaspar <pa...@krankikom.de>]

> -----Original Message-----
> From: Hans Bergsten [mailto:hans@gefionsoftware.com]
> Sent: Saturday, November 18, 2000 00:34
>
> Regarding the proposed changes, I'm +0. I'd like to see 3.3 (or ideally it
> should be called 3.2.x if you decide to start from scratch with the 3.2
> base) as the continuation of 3.2, with focus on bug fixes and minor
> improvements (e.g. performance, admin, etc.) in support of the
> 2.2/1.1 APIs
> as opposed to refactoring to a completely new architecture. From the mails
> on this list, it seems like the majority wants a stable 3.2.x code base
> that's good enough to be used in production. Another important reason
> to keep the basic architecture intact is to give people a chance to learn
> how it works and be able to contribute to the 3.2.x code base. If it keeps
> changing, there will be very few that has the time to keep up.

I am one of the guys that wants a stable production solution. BUT most
(all?)
of the changes that Costin lists:
 1. Simplify code;
 2. Improve modularity;
 3. Fix functionality.

3 has to be done in order to have a production product that behaves has
expected according to the Servlet standard version it implements.

1 and 2 always pay-off quite fast in terms of maintenance and code reuse.
And this is important because:
 - Tomcat will always have some maintenance work to be done while it is a
   production product. So, it would be better to freeze it at a more
   maintainable form;
 - Tomcat is a product that tends to be reused quite often in other
products.
   Examples I know better: Borland's JBuilder 4 and Oracle's JDeveloper 3.


And let's consider this factors:
 a) It is still not clear when will Catalina be production ready - one can
    never be 100% sure about such things before they are done.
    HOWEVER most of the work on 3.3 and most of the 3.2 bug fixes are done.
    This makes merging them and getting 3.3 production ready a much more
    predictable task than getting Catalina/4.0 there;

 b) And then, Costin wants to do most of the work!


So, why not taking advantage of b) to reach the advantages of the better
and easier to maintain 3.3 that, according to a), is "more sure" to be
there soon than 4.0?


Conclusion:
 - Costin is doing most of the work;
 - His changes really look to be an improvement;
 - And if he is wrong (which I doubt) everybody can stick with 3.2 until
   4.0, even if having to make minor (3.3.x?) fixes...
 - ...since no one else looks to be willing to do all the stuff that
   Costin wants to take care of, anyway.

(BOTTOM LINE: let's take advantage of Costin! (o;=
He is also having his fun!)


> > - consolidation/refactoring of all tomcat-independent and
> general-purpose/
> > reusable code into org.apache.tomcat.util.[log, etc ]
> >
> > - new utils
>
> If they are truly reusable (e.g. used in both Tomcat and Jasper), wouldn't
> org.apache.util be a better package name?

Wouldn't that be a bit intrusive and demand for agreement with other Apache
projects?


Have fun,
Paulo

Re: Tomcat3.3

[cm...@yahoo.com]

> I agree, +1 on option 2).
> 
> Regarding the proposed changes, I'm +0. I'd like to see 3.3 (or ideally it
> should be called 3.2.x if you decide to start from scratch with the 3.2
> base) as the continuation of 3.2, with focus on bug fixes and minor
> improvements (e.g. performance, admin, etc.) in support of the 2.2/1.1 APIs 
> as opposed to refactoring to a completely new architecture. From the mails
> on this list, it seems like the majority wants a stable 3.2.x code base
> that's good enough to be used in production. Another important reason
> to keep the basic architecture intact is to give people a chance to learn 
> how it works and be able to contribute to the 3.2.x code base. If it keeps
> changing, there will be very few that has the time to keep up.

The basic architecture/design will remain intact, it's just that we need
to remove all the code that is deprecated since 3.1, and to do the changes
that we have to do in order to have a better container.

If you look at the list, most of the "architecture" changes are
simplifying the object model, removing what it's not needed. 
We couldn't do that in 3.2 - because of the available time, and the
risks.

Some of the "architecture" changes ( like OutputBuffer, MessageBytes ) are
required for enhancing the performance and fixing the internationalization
bugs ( that can't be fixed with 3.2 ).

Few other changes ( documenting and enforcing the server "state" ) are
again absolutely needed ( IMHO ). 

That's what happened in 3.2 ( compared with 3.1 ), and that's
(IMHO) normal evolution of the code. 


> Without analyzing all the proposals in detail (which I don't have the time 
> to do), it's hard to say which category they belong to. If some of them
> imply a major refactoring, turning it to a new architecture, I suggest you
> do them as a revolution instead as we discussed earlier.

There are few major refactorings - but probably less than we had between
3.1 and 3.2.

Again, it's normal evolution of tomcat3 - if you are only interested in
3.2 bug fixes that can be done in 3.2.1, etc - but I can't see why/how
tomcat3.3 can't evolve in the same way as 3.1 did.



> > - new utils
> 
> If they are truly reusable (e.g. used in both Tomcat and Jasper), wouldn't
> org.apache.util be a better package name?

I would love to - but it's not that easy :-)

Costin

Re: Tomcat3.3

[Hans Bergsten <ha...@gefionsoftware.com>]

cmanolache@yahoo.com wrote:
> [...]
> Now, that I have a bit more time ( I hope :-), I can start spending
> more time working on tomcat. There are 2 possible ways to do that:
> 
> 1. In the current "main branch", incorporate all the fixes from tomcat3.2,
> plus fixes for the new bugs.
> 
> 2. Start with tomcat3.2, and re-do all the changes.
> 
> My preference is toward 2 - even if it will probably take a longer
> time. I think this is the normal "evolution", i.e. building on top of the
> stable release. Because of the long release cycle 3.3 and 3.2 are out of
> sync, and I don't want to loose any of the "evolutions" and fixes that
> happened.

I agree, +1 on option 2).

Regarding the proposed changes, I'm +0. I'd like to see 3.3 (or ideally it
should be called 3.2.x if you decide to start from scratch with the 3.2
base) as the continuation of 3.2, with focus on bug fixes and minor
improvements (e.g. performance, admin, etc.) in support of the 2.2/1.1 APIs 
as opposed to refactoring to a completely new architecture. From the mails
on this list, it seems like the majority wants a stable 3.2.x code base
that's good enough to be used in production. Another important reason
to keep the basic architecture intact is to give people a chance to learn 
how it works and be able to contribute to the 3.2.x code base. If it keeps
changing, there will be very few that has the time to keep up.

Without analyzing all the proposals in detail (which I don't have the time 
to do), it's hard to say which category they belong to. If some of them
imply a major refactoring, turning it to a new architecture, I suggest you
do them as a revolution instead as we discussed earlier.

> [...]
> UTILS
> 
> - consolidation/refactoring of all tomcat-independent and general-purpose/
> reusable code into org.apache.tomcat.util.[log, etc ]
> 
> - new utils

If they are truly reusable (e.g. used in both Tomcat and Jasper), wouldn't
org.apache.util be a better package name?

Hans
-- 
Hans Bergsten		hans@gefionsoftware.com
Gefion Software		http://www.gefionsoftware.com