You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by co...@eng.sun.com on 2000/01/31 11:16:03 UTC
Re: Discussion: AJP next
I think any protocol is ok, but we should first find out what API
we want to expose - that's related with Stefano's proposal for a relay
module.
The main limitation of Ajp is the lack of callbacks. We need for example a
'getRealPath' - since tomcat has no way to guess how Apache will map a
certain URL, and only apache can do that.
The problem is of course that Apache 1.3 is not multithreaded, and we
can't use a standard solution for callbacks. One possible solution
is to work with 1 Apache thread and one tomcat thread waiting for each
other. That mean:
1. Apache starts and open a connection with tomcat ( that will remain open
for the live of the httpd process - avoid TCP open delay/request). Tomcat
thread will be blocked on read, waiting for apache message.
2. HTTP request received by Apache. Send all relevant information in a
packed format to Tomcat, then wait for response message. ( blocked read ).
3. Tomcat receive the request message, and start processing the request (
in the same thread).
4.The servlet might need a getRealPath() or another callback to Apache.
Tomcat will send a callback message and block again waiting response.
5. Apache receive the message, execute it and send the response, then wait
another message.
In particular writeHeader(), writeBody() will be implemented as callbacks
( Apache might want to add it's own headers _and_ maybe use HTTP/1.1),
probably getRemoteHost() will also be a callback ( require an expensive
lookup that is not needed always).
I don't know how do you plan to deal with callbacks ( from a thread point
of view ), everything else is fine in your proposal, but I think
callbacks need to be addressed and explained better.
Costin
Re: Discussion: AJP next
Posted by co...@costin.dnt.ro.
> Of course this problem is more related to general policies of Tomcat
> development. IMHO folding Tomcat into non-expandable environment, which
> is very restrictively limited to current specification is bad. It's
> safer to assume that current specification is not the ultimate solution,
> and it will have to evolve.
There is a price you have to pay for standards.
You should be very carefull with using proprietary APIs - you're code will
no longer be portable, and then I see no reason for using Servlets
anyway, since your code will run only on a particular engine.
There are many things I don't like in the Servlet API, and it will be very
hard to find someone ( anyone ) that think it is "perfect".
On the other side, what you want can be implemented in tomcat ( low level)
and will be _very_ usefull for JSP ( or other template engines ). There is
nothing wrong in having a JSP adapter that is specific to tomcat and takes
advantage of Apache-tomcat cache and optimizations.
It is usefull to have the "caching" API implemented in tomcat, use it on
JSP ( where at least you have a very clear separation between static
and dynamic content).
Costin
Re: Discussion: AJP next
Posted by Michal Mosiewicz <mi...@interdata.com.pl>.
costin@eng.sun.com wrote:
> [...]
> Wait, I don't get it - how can you guess what portion doesn't change ???
> The servlet API doesn't have any support for that - in a templating system
> like JSP you might detect if a portion changed, but for a general Servlet
> I don't have any idea...
True that servlet API doesn't solve it. But I'm looking toward future
specifications. I was experimenting on JServ to test potential gains. In
that case I was able to insert additional methods to mark static body in
JServConnection and then I could cast request to JServConnection, and I
could use those methods. That's how I tested my ideas. That's is hardly
possible in reference implementation.
As for now, you can only use some custom content encoding to accomplish
this features. So that means that some custom protocol have to be
layered over http response.
Another simple solution would be using dispatchers as the mean of
marking different content cacheability. That is I think implemented in
Resin. However in case of resin, cache is stored and managed by the java
engine, not the apache.
Of course this problem is more related to general policies of Tomcat
development. IMHO folding Tomcat into non-expandable environment, which
is very restrictively limited to current specification is bad. It's
safer to assume that current specification is not the ultimate solution,
and it will have to evolve.
So my point is, that smart information caching may be useful right now,
but currently it can be only limited to full resource content. However
it is better to give it more power, that could be used in the future.
-- Mike
Re: Discussion: AJP next
Posted by co...@eng.sun.com.
> costin@eng.sun.com wrote:
> > [...]
> > 1. If a response _does_ set expire headers, it should be easy to cache it
> > on apache side, either in memory or in file system. ( similar with using a
> > squid accelerator). We don't need any API, just an apache module ( that
> > can be used for CGIs for example ). It would be really _cool_.
>
> But you are talking about the whole response, while I'm talking about
> fragments. To get the idea, just take a look at some larger portal site
> - you've got several common elements, like w3 catalogs, stock indexes,
> latest news, and finally - some advertising. Usually you can cache those
> elements with different expiration times - for example hours for w3
> catalog is usually fine, latest news may be delayed by minutes, stock
> indexes by seconds or minutes. Ads cannot be cached at all, cause you
> will usually need some personal profiling methods to serve them. So -
> the common denominator for the whole page is not to cache it at all,
> becouse it is partially served on a per-user basis.
Wait, I don't get it - how can you guess what portion doesn't change ???
The servlet API doesn't have any support for that - in a templating system
like JSP you might detect if a portion changed, but for a general Servlet
I don't have any idea...
Costin
Re: Discussion: AJP next
Posted by Michal Mosiewicz <mi...@interdata.com.pl>.
costin@eng.sun.com wrote:
> [...]
> 1. If a response _does_ set expire headers, it should be easy to cache it
> on apache side, either in memory or in file system. ( similar with using a
> squid accelerator). We don't need any API, just an apache module ( that
> can be used for CGIs for example ). It would be really _cool_.
But you are talking about the whole response, while I'm talking about
fragments. To get the idea, just take a look at some larger portal site
- you've got several common elements, like w3 catalogs, stock indexes,
latest news, and finally - some advertising. Usually you can cache those
elements with different expiration times - for example hours for w3
catalog is usually fine, latest news may be delayed by minutes, stock
indexes by seconds or minutes. Ads cannot be cached at all, cause you
will usually need some personal profiling methods to serve them. So -
the common denominator for the whole page is not to cache it at all,
becouse it is partially served on a per-user basis.
> 2. For JSPs we have a great advantage - we know that a portion of the page
> is constant. ( as a matter of fact, it's one of the main reasons I like
> JSPs ! ). It is very easy to mark the constant body.
>
> In API terms, the response from tomcat will consist in fragements. Because
> of the 2.2 buffering we already have that implemented, and that might
> help a lot in another area ( HTTP/1.1 and HTTP connection reuse ).
That also could help in lowering communication costs. Sometimes it
appears to be very easy to push 10kB response in one tcp packet that is
fraction of that size.
Mike
Re: Discussion: AJP next
Posted by co...@eng.sun.com.
> How it works (or is intended to work) - basically when I send the
> response to apache, I can mark some part of the response data with a
> unique reference id and the expiration time. Then - once the response is
> received in apache, this marked data is stored in a local heap and
> indexed using the reference id.
1. If a response _does_ set expire headers, it should be easy to cache it
on apache side, either in memory or in file system. ( similar with using a
squid accelerator). We don't need any API, just an apache module ( that
can be used for CGIs for example ). It would be really _cool_.
2. For JSPs we have a great advantage - we know that a portion of the page
is constant. ( as a matter of fact, it's one of the main reasons I like
JSPs ! ). It is very easy to mark the constant body.
In API terms, the response from tomcat will consist in fragements. Because
of the 2.2 buffering we already have that implemented, and that might
help a lot in another area ( HTTP/1.1 and HTTP connection reuse ).
It would be very easy to send a fragment type, that can be either
"constant" ( with Apache caching it using (1) - either in memory or on
disk), or a special fragment "page fragment - with a file name and 2
offsets" - then Apache will read the fragment from the file directly,
and use the OS caches or mmap or any other optimizations it has for files.
( apache needs to be able to send file fragments anyway - I think it's in
HTTP/1.1).
That's also great for JSP for "large-file" case - the "tomcat" overhead
will be minimal for pages with a lot of text and little dynamic content.
I think we should spend some time in this area, it's really interesting.
( just one more month to 3.1, and we can have fun with optimizations :-)
Costin
Re: Discussion: AJP next
Posted by Michal Mosiewicz <mi...@interdata.com.pl>.
Jean-Luc Rochat wrote:
> [...]
> Feedback welcome.
Just one more thing...
I've been pretty busy these days and I'm catching up with this
discussion.
In september I've been researching something that in my projects would
give me about 3 to 20 times speedups - a decent caching algorithm.
Something similiar already exists in resin. You can see it's benchmarks
to get the figures. The main idea is that even in case of fully dynamic
content, you may point areas that are more and less dynamic. Some data
may be very volatille and you may need to read them from database or
other storage each new request comes, but some can be valid for several
seconds or minutes.
At first I tried to accomplish my data caching by using dispatchers and
routing some dispatch requests through apache, so some subrequests would
be cached. But then I noticed that it is not that good idea.
Finally I tested a technique to 'bracket' some parts of the content, and
identify them through references.
How it works (or is intended to work) - basically when I send the
response to apache, I can mark some part of the response data with a
unique reference id and the expiration time. Then - once the response is
received in apache, this marked data is stored in a local heap and
indexed using the reference id.
Of course, the java backends also stores locally this reference to the
data fragment that has just been sent to apache, so it knows that it
doesn't need to send the whole content again. Instead it's sufficient
only to send the reference with the next request. Apache would be able
to use this reference, to get the data from the heap and include it in
the response.
I didn't finish implementing it cause there was (and still are)
architectural issues making it hard to use. But still the figures was
very promising. For example, that time I was developing a kind of portal
website. The most frequently viewed pages involved a lot of data taken
from database. On a pretty reasonable hardware, I couldn't serve those
pages faster than in 150-250ms. However for most parts of those pages it
would be harmless if I cached them for several seconds, sometimes even
minutes wouldn't make much difference. I tested some code using hacked
JServ. It appeared, that the above delay could be easily shorted to less
than 15ms. Of course - sometimes I could cache the whole page, and I
could serve it even faster - but that's pretty obvious. However there
are many cases, that you just can't cache the whole page, while you are
free to cache some/most parts.
The more difficult part of this is how to implement such a mechanism in
JSDK. It's really bad idea that subrequests, i.e. included dispatches
are not able to pass the information useful for caching (like
expiration). It could be the easiest just to use them.
But the real gain would be if you could just mark some cacheable areas
while sending the response. It's easy to imagine how powerful it could
be in XML based tools like Cocoon, if you could just mark some cacheable
areas using custom tags.
Anyhow, foreseeing it in the protocol would be a good start to introduce
some API hooks to use it in JSDK.
-- Mike
Re: Discussion: AJP next
Posted by Jean-Luc Rochat <jn...@cybercable.fr>.
costin@eng.sun.com wrote:
>
> > The getRealPath callback is a good example for something that may be a
> > problem, not because of the callback that we will have to do but because
> > of the servlet APIs. We have getRealPath in two places - the request and
> > the context objects, we can use a callback only during a request, how are
> > we going to resolve the difference? Same thing applys when we want to map
> > a suffic to a mime type using the web server.
>
> It's a problem only if tomcat will call getRealPath outside of request
> processing. It's a very special case - most of the time it's a servlet
> that calls the methods, and the servlet is executed as a result of a
> request ( i.e. you have an open communication). Doesn't matter if Context
> or Request is called for realPath - the implementation of the method will
> just send a message to Apache and get the response.
>
> In the special case, when you have a non-servlet thread calling back, you
> can do some tricks ( like a HTTP connection back to apache ?), but again,
> it's not the normal case and probably will not happen too much ( so we can
> have something complex for this case ).
>
> > > 1. Apache starts and open a connection with tomcat ( that will remain open
> > > for the live of the httpd process - avoid TCP open delay/request). Tomcat
> > > thread will be blocked on read, waiting for apache message.
> > >
> >
> > Actually, since (sadly) Apache is multiprocess and can open up to 256
> > processes, one socket for the duration of the process is too much.
> >
> > Also such policy opens the door for denial of service attacks.
> >
> > I though that we should keep the connection open long enough but not for
> > the duration of the process. Not if we are going to allocate a tomcat
> > thread per connection.
>
> Opening a TCP connection is slow, probably we can open it at first request
> to a servlet and close it after a while.
>
> DOS can happen anyway - I don't think 1 connection/request is a solution.
>
> ( also, TIME_WAIT is a big problem sometimes!)
>
> We can discuss that more, but it's not so important - it's easy to make it
> optional.
>
> > > I don't know how do you plan to deal with callbacks ( from a thread point
> > > of view ), everything else is fine in your proposal, but I think
> > > callbacks need to be addressed and explained better.
> > >
> >
> > I am going to deal with them similar to your description... I was also
> > thinking about writing some native code on the tomcat side so I will be
> > able to use select (no select in Java !!!). This way I can reduce the
> > number of Java threads and leave more connections open for a longer time.
>
> I don't know if it's a good ideea - it will be benefical for high-loaded
> sites ( where you have a large number of concurent connections, that mean
> milions of hits/day ) - in this case load balancing is much better ( and
> using more boxes). Also, JNI and Invocation are faster and probably a
> better way to spend the time.
>
> Costin
>
At that point I'll try to give my opinion and possibly offer some ideas.
This topic has been one of the problems that have not been addressed in
Jserv, but we tried to imagine solutions. I am happy to see that a lot
of my previous ideas are expressed by others, and we are here only
missing some glue, which I hope I could contribute to bring.
1 - integrating a JVM in an Apache module is a _bad thing_ (tm) IMO.
. from the architecture point of view. (n-tier is better for
scalability than monolithic servers)
. from the security point of view (small pieces of code means less
bugs, user permissions problems)
. from the portability point of view (example: Apache runs very well
on BSDs. JVMs don't).
2 - Opening a TCP connection for each request brings an extra overhead.
. connections should be reused if possible (keepalive feature).
3 - one socket / httpd process could be too much, especially if more
than 1 Apache send requests to 1 servlet engine.
. sockets can be closed at any time by the servlet engine, so we can
use one socket/httpd
4 - sockets cannot be easily shared between httpd processes (on Unix at
least, and with Apache 1.x).
. true if the servlet engine stops and restarts while the Apache
server is still up. (load-balanced configuration). So don't multiplex
data on one socket, or offer it as an optional feature.
. asking for the Apache father to re-create the socket and re-fork
all the children would be a bad idea.
5 - Using any other protocol than ajpv11 is faster.
. yes but ajpv11 brings everything needed. So keep the "verbose"
protocol a possible choice.(for ben-ssl/mod_ssl by example).
6 - Callbacks to Apache are not mandatory. Or maybe they are. or will
be.
. let's keep the protocol pluggable, so we can change it.
7 - All of this has to pass firewalls (callbacks ?)
8 - This protocol implementation should allow the basic load-balancing
mechanisms used in JServ, take care of keepalive enabled sockets,
authentication, without problem.
The solution seems to split the initial problem in 2 parts :
1 : the keepalive feature.
. every httpd opens a socket -> one servlet engine (default server if
load-balanced config)
. the socket is not closed by the httpd. it is reused and reused.
. if the socket is closed by the servlet engine, httpd opens a new
one when needed.
. the servlet engine accepts incoming connections from httpd
processes (belonging to 1..n Apache servers)
. once accepted, a thread block on the socket to read new requests
(keepalive).
authentication is performed once, and the load-balancing parameter
(the servername to add to the cookie) is sent once.
. if a servlet engine whishes, some of the opened sockets can be
closed.
This should give a boost to performances, as the socket creation has
a huge cost.
2 : the new protocol (possibly) bi-directionnal protocol), built on top
of the the above keepalive enabled connections.
. ajpv11 like. (not very clever, but everything is there).
. ajpv21 like (with multiple packet exchanges in one single request
(yes, including requests servlet engine -> Apache).
Proposal :
Step 1 : I am going to start working on the keepalive feature, and will
bring C & java code ASAP. This will be tested with JServ, as I'm far
more familiar with the code.
This will be for Apache only (I don't care IIS, or NS for the moment)
and especially U**x, (I don't care NT/W95 for the moment).
Step 2 : see the code & talk.
Step 3 : protocol enhancements if people like it.
Feedback welcome.
Jean-Luc
http://ma-planete.net
Re: Discussion: AJP next
Posted by Stefano Mazzocchi <st...@apache.org>.
costin@eng.sun.com wrote:
>
> > The getRealPath callback is a good example for something that may be a
> > problem, not because of the callback that we will have to do but because
> > of the servlet APIs. We have getRealPath in two places - the request and
> > the context objects, we can use a callback only during a request, how are
> > we going to resolve the difference? Same thing applys when we want to map
> > a suffic to a mime type using the web server.
>
> It's a problem only if tomcat will call getRealPath outside of request
> processing. It's a very special case - most of the time it's a servlet
> that calls the methods, and the servlet is executed as a result of a
> request ( i.e. you have an open communication). Doesn't matter if Context
> or Request is called for realPath - the implementation of the method will
> just send a message to Apache and get the response.
>
> In the special case, when you have a non-servlet thread calling back, you
> can do some tricks ( like a HTTP connection back to apache ?), but again,
> it's not the normal case and probably will not happen too much ( so we can
> have something complex for this case ).
Knowing this, I proposed to deprecate getRealPath from _all_ the servlet
API (since there is getResource() which is much better) but I got
overruled for the 2.2 spec.
Maybe we can try this again for the next version.
What do you think?
--
Stefano Mazzocchi One must still have chaos in oneself to be
able to give birth to a dancing star.
<st...@apache.org> Friedrich Nietzsche
--------------------------------------------------------------------
Come to the first official Apache Software Foundation Conference!
------------------------- http://ApacheCon.Com ---------------------
Re: Discussion: AJP next
Posted by co...@eng.sun.com.
> The getRealPath callback is a good example for something that may be a
> problem, not because of the callback that we will have to do but because
> of the servlet APIs. We have getRealPath in two places - the request and
> the context objects, we can use a callback only during a request, how are
> we going to resolve the difference? Same thing applys when we want to map
> a suffic to a mime type using the web server.
It's a problem only if tomcat will call getRealPath outside of request
processing. It's a very special case - most of the time it's a servlet
that calls the methods, and the servlet is executed as a result of a
request ( i.e. you have an open communication). Doesn't matter if Context
or Request is called for realPath - the implementation of the method will
just send a message to Apache and get the response.
In the special case, when you have a non-servlet thread calling back, you
can do some tricks ( like a HTTP connection back to apache ?), but again,
it's not the normal case and probably will not happen too much ( so we can
have something complex for this case ).
> > 1. Apache starts and open a connection with tomcat ( that will remain open
> > for the live of the httpd process - avoid TCP open delay/request). Tomcat
> > thread will be blocked on read, waiting for apache message.
> >
>
> Actually, since (sadly) Apache is multiprocess and can open up to 256
> processes, one socket for the duration of the process is too much.
>
> Also such policy opens the door for denial of service attacks.
>
> I though that we should keep the connection open long enough but not for
> the duration of the process. Not if we are going to allocate a tomcat
> thread per connection.
Opening a TCP connection is slow, probably we can open it at first request
to a servlet and close it after a while.
DOS can happen anyway - I don't think 1 connection/request is a solution.
( also, TIME_WAIT is a big problem sometimes!)
We can discuss that more, but it's not so important - it's easy to make it
optional.
> > I don't know how do you plan to deal with callbacks ( from a thread point
> > of view ), everything else is fine in your proposal, but I think
> > callbacks need to be addressed and explained better.
> >
>
> I am going to deal with them similar to your description... I was also
> thinking about writing some native code on the tomcat side so I will be
> able to use select (no select in Java !!!). This way I can reduce the
> number of Java threads and leave more connections open for a longer time.
I don't know if it's a good ideea - it will be benefical for high-loaded
sites ( where you have a large number of concurent connections, that mean
milions of hits/day ) - in this case load balancing is much better ( and
using more boxes). Also, JNI and Invocation are faster and probably a
better way to spend the time.
Costin
Re: Discussion: AJP next
Posted by Shachor Gal <sh...@techunix.technion.ac.il>.
On Mon, 31 Jan 2000 costin@eng.sun.com wrote:
>
> I think any protocol is ok, but we should first find out what API
> we want to expose - that's related with Stefano's proposal for a relay
> module.
>
> The main limitation of Ajp is the lack of callbacks. We need for example a
> 'getRealPath' - since tomcat has no way to guess how Apache will map a
> certain URL, and only apache can do that.
>
The getRealPath callback is a good example for something that may be a
problem, not because of the callback that we will have to do but because
of the servlet APIs. We have getRealPath in two places - the request and
the context objects, we can use a callback only during a request, how are
we going to resolve the difference? Same thing applys when we want to map
a suffic to a mime type using the web server.
We have a problem here... No one wants to configure two places with the
same data...
>
> The problem is of course that Apache 1.3 is not multithreaded, and we
> can't use a standard solution for callbacks. One possible solution
> is to work with 1 Apache thread and one tomcat thread waiting for each
> other. That mean:
>
> 1. Apache starts and open a connection with tomcat ( that will remain open
> for the live of the httpd process - avoid TCP open delay/request). Tomcat
> thread will be blocked on read, waiting for apache message.
>
Actually, since (sadly) Apache is multiprocess and can open up to 256
processes, one socket for the duration of the process is too much.
Also such policy opens the door for denial of service attacks.
I though that we should keep the connection open long enough but not for
the duration of the process. Not if we are going to allocate a tomcat
thread per connection.
>
> 2. HTTP request received by Apache. Send all relevant information in a
> packed format to Tomcat, then wait for response message. ( blocked read ).
>
> 3. Tomcat receive the request message, and start processing the request (
> in the same thread).
>
> 4.The servlet might need a getRealPath() or another callback to Apache.
> Tomcat will send a callback message and block again waiting response.
>
> 5. Apache receive the message, execute it and send the response, then wait
> another message.
>
Agreed.
> In particular writeHeader(), writeBody() will be implemented as callbacks
> ( Apache might want to add it's own headers _and_ maybe use HTTP/1.1),
> probably getRemoteHost() will also be a callback ( require an expensive
> lookup that is not needed always).
>
> I don't know how do you plan to deal with callbacks ( from a thread point
> of view ), everything else is fine in your proposal, but I think
> callbacks need to be addressed and explained better.
>
I am going to deal with them similar to your description... I was also
thinking about writing some native code on the tomcat side so I will be
able to use select (no select in Java !!!). This way I can reduce the
number of Java threads and leave more connections open for a longer time.
Gal Shachor
> Costin
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
>
>