You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Mladen Turk <mt...@apache.org> on 2006/07/09 12:16:13 UTC
JK and AJP protocol enhancements
You see how desperate I am when writing this on Sunday :)
Anyhow, we are pretty close to the new JK release that I
hope will be the most usable and stable whatsoever.
The things we agreed so many times before, but having
obviously too little resources to actually create are
the 1.3 branch (aka JK3) and the AJP protocol stuff.
Now there is the problem with that. Henri even created
a AJP1.4 protocol enhancements with all that login,
discovery etc... (never implement but thats another story).
Although we got close to the AJP 1.4 protocol conclusion
last year, nowadays all that looks strange to me.
All those things might be implemented, but IMHO only
as a AJP1.5 protocol.
What we desperately need right now are 3 things:
1. Allowing to have +8K headers
2. Allowing to have +0x9999 single header limit
3. Mechanism to tell the Tomcat to gracefully close
the connection.
Now, the number 3 is very easy. A simple message
like we have for SHUTDOW, but instead shutting down
the entire Tomcat instance, closing down the socket/channel.
OTOH first two are little bit tricky :)
I have some ideas:
1. Larger headers can be treated as we handle the POST data.
If there are more headers then 8K, then a servlet container
should send GIVE_ME_MORE_HEADERS message.
2. If the single header is larger then 39321 bytes, then it
should be send as POST data, with servlet container requesting
8K packets. Those headers would be treated as multiple POST
sequences, after the initial header(s) packet(s) have been
read and before the actual POST data is read.
Any comments?
Regards,
Mladen.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: JK and AJP protocol enhancements
Posted by Jean-frederic Clere <jf...@gmail.com>.
Costin Manolache wrote:
> What's the status with mod_proxy ?
I think Mladen idea is to make the same enhancements in mod_proxy (and I
will try to help him :-)
>
> It seems this kind of change would break backward compatibility, and
> if this
> happens - maybe it's better to fix the protocol marshalling
> limitations or
> change it completely.
> I hate the idea of patching an old and mostly broken marshalling model.
>
> The only thing we still need is the alternating client/server role
> using a
> single tcp connection. For the actual marshalling, there are plenty of
> better choices - with more extensibility, self-describing, easy and
> fast to
> parse, etc.
> Examples:
> - json
> - http://www.caucho.com/resin-3.0/protocols/hessian-1.0-spec.xtp
> - yaml.org
> - wbxml if we want very compact representation
>
> ( every time someone is discussing extending AJP I feel the need to point
> that ajp is a very old and not so well designed protocol, it was not
> supposed to live that longer and there is no need for a proprietary,
> tomcat-specific marshalling mechanism )
>
> I realize that replacing the serialization code would be a large
> effort, but
> I think using a proper format, with existing implementations ( to
> allow easy
> testing ) is far simpler than patching ajp and having all kind of
> workarounds and tricks.
I am quite reluctant to use a new (for our community) protocol... I
rememeber the WARP protocol years ago.
Probably patching the old crapy protocol will have more success than
doing a nice new thing :-(
Cheers
Jean-Frederic
>
> Costin
>
> On 7/9/06, Mladen Turk <mt...@apache.org> wrote:
>
>>
>> You see how desperate I am when writing this on Sunday :)
>>
>> Anyhow, we are pretty close to the new JK release that I
>> hope will be the most usable and stable whatsoever.
>>
>> The things we agreed so many times before, but having
>> obviously too little resources to actually create are
>> the 1.3 branch (aka JK3) and the AJP protocol stuff.
>>
>> Now there is the problem with that. Henri even created
>> a AJP1.4 protocol enhancements with all that login,
>> discovery etc... (never implement but thats another story).
>> Although we got close to the AJP 1.4 protocol conclusion
>> last year, nowadays all that looks strange to me.
>> All those things might be implemented, but IMHO only
>> as a AJP1.5 protocol.
>>
>> What we desperately need right now are 3 things:
>> 1. Allowing to have +8K headers
>> 2. Allowing to have +0x9999 single header limit
>> 3. Mechanism to tell the Tomcat to gracefully close
>> the connection.
>>
>> Now, the number 3 is very easy. A simple message
>> like we have for SHUTDOW, but instead shutting down
>> the entire Tomcat instance, closing down the socket/channel.
>>
>> OTOH first two are little bit tricky :)
>> I have some ideas:
>> 1. Larger headers can be treated as we handle the POST data.
>> If there are more headers then 8K, then a servlet container
>> should send GIVE_ME_MORE_HEADERS message.
>> 2. If the single header is larger then 39321 bytes, then it
>> should be send as POST data, with servlet container requesting
>> 8K packets. Those headers would be treated as multiple POST
>> sequences, after the initial header(s) packet(s) have been
>> read and before the actual POST data is read.
>>
>> Any comments?
>>
>> Regards,
>> Mladen.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: dev-help@tomcat.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: JK and AJP protocol enhancements
Posted by Jean-frederic Clere <jf...@gmail.com>.
Henri Gomez wrote:
> Well there was some provision in mind in AJP 1.4 :
>
> Context informations forwarding for Servlet engine to Web Server
>
> With this kind of information requested by webserver, we could
> determine the version of Servlet AJP implementation and as such use
> 32/64k buffers.
>
> the jk could send question like 'do you support 32k buffers ?'
In fact it should start the first dialog by asking the version of AJP TC
is supporting, otherwise we will never be able to improve the protocol.
Probably a CPING answered by a PONG + version before the first request.
Cheers
Jean-Frederic
>
> In fact, the idea was to help jk side to discover information on the
> Servlet engine implementation, and act accordingly.
>
> 2006/7/10, Mladen Turk <mt...@apache.org>:
>
>> Costin Manolache wrote:
>> > What's the status with mod_proxy ?
>> >
>> > It seems this kind of change would break backward compatibility,
>> and if
>> > this
>> > happens - maybe it's better to fix the protocol marshalling
>> limitations or
>> > change it completely.
>> > I hate the idea of patching an old and mostly broken marshalling
>> model.
>> >
>>
>> The point is that it will be backward compatible.
>> It means that for headers < 8K any current AJP 1.3 protocol
>> could be used as is.
>> If there is a larger header, then the current AJP1.3 implementation
>> will return 'error message' (as now), while the new one will
>> handle it by requesting additional header packets.
>> Also if the old 1.3 consumer is in front of the new 1.4 producer
>> nothing will change.
>>
>> Resolving 8K header limitation and allowing a single header to be
>> larger then 33K, while presere the backwards compatibility and it can
>> be done. It would make receiver and producer a little bit more
>> complex, but not that much. It would still maintain the zero-copy
>> failover, that IMHO is the major advantage of AJP protocol, without
>> the need to prefetch the entire header data.
>>
>> Regards,
>> Mladen
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: dev-help@tomcat.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: JK and AJP protocol enhancements
Posted by Henri Gomez <he...@gmail.com>.
Well there was some provision in mind in AJP 1.4 :
Context informations forwarding for Servlet engine to Web Server
With this kind of information requested by webserver, we could
determine the version of Servlet AJP implementation and as such use
32/64k buffers.
the jk could send question like 'do you support 32k buffers ?'
In fact, the idea was to help jk side to discover information on the
Servlet engine implementation, and act accordingly.
2006/7/10, Mladen Turk <mt...@apache.org>:
> Costin Manolache wrote:
> > What's the status with mod_proxy ?
> >
> > It seems this kind of change would break backward compatibility, and if
> > this
> > happens - maybe it's better to fix the protocol marshalling limitations or
> > change it completely.
> > I hate the idea of patching an old and mostly broken marshalling model.
> >
>
> The point is that it will be backward compatible.
> It means that for headers < 8K any current AJP 1.3 protocol
> could be used as is.
> If there is a larger header, then the current AJP1.3 implementation
> will return 'error message' (as now), while the new one will
> handle it by requesting additional header packets.
> Also if the old 1.3 consumer is in front of the new 1.4 producer
> nothing will change.
>
> Resolving 8K header limitation and allowing a single header to be
> larger then 33K, while presere the backwards compatibility and it can
> be done. It would make receiver and producer a little bit more
> complex, but not that much. It would still maintain the zero-copy
> failover, that IMHO is the major advantage of AJP protocol, without
> the need to prefetch the entire header data.
>
> Regards,
> Mladen
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: JK and AJP protocol enhancements
Posted by Mladen Turk <mt...@apache.org>.
Costin Manolache wrote:
> What's the status with mod_proxy ?
>
> It seems this kind of change would break backward compatibility, and if
> this
> happens - maybe it's better to fix the protocol marshalling limitations or
> change it completely.
> I hate the idea of patching an old and mostly broken marshalling model.
>
The point is that it will be backward compatible.
It means that for headers < 8K any current AJP 1.3 protocol
could be used as is.
If there is a larger header, then the current AJP1.3 implementation
will return 'error message' (as now), while the new one will
handle it by requesting additional header packets.
Also if the old 1.3 consumer is in front of the new 1.4 producer
nothing will change.
Resolving 8K header limitation and allowing a single header to be
larger then 33K, while presere the backwards compatibility and it can
be done. It would make receiver and producer a little bit more
complex, but not that much. It would still maintain the zero-copy
failover, that IMHO is the major advantage of AJP protocol, without
the need to prefetch the entire header data.
Regards,
Mladen
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: JK and AJP protocol enhancements
Posted by Costin Manolache <co...@gmail.com>.
What's the status with mod_proxy ?
It seems this kind of change would break backward compatibility, and if this
happens - maybe it's better to fix the protocol marshalling limitations or
change it completely.
I hate the idea of patching an old and mostly broken marshalling model.
The only thing we still need is the alternating client/server role using a
single tcp connection. For the actual marshalling, there are plenty of
better choices - with more extensibility, self-describing, easy and fast to
parse, etc.
Examples:
- json
- http://www.caucho.com/resin-3.0/protocols/hessian-1.0-spec.xtp
- yaml.org
- wbxml if we want very compact representation
( every time someone is discussing extending AJP I feel the need to point
that ajp is a very old and not so well designed protocol, it was not
supposed to live that longer and there is no need for a proprietary,
tomcat-specific marshalling mechanism )
I realize that replacing the serialization code would be a large effort, but
I think using a proper format, with existing implementations ( to allow easy
testing ) is far simpler than patching ajp and having all kind of
workarounds and tricks.
Costin
On 7/9/06, Mladen Turk <mt...@apache.org> wrote:
>
> You see how desperate I am when writing this on Sunday :)
>
> Anyhow, we are pretty close to the new JK release that I
> hope will be the most usable and stable whatsoever.
>
> The things we agreed so many times before, but having
> obviously too little resources to actually create are
> the 1.3 branch (aka JK3) and the AJP protocol stuff.
>
> Now there is the problem with that. Henri even created
> a AJP1.4 protocol enhancements with all that login,
> discovery etc... (never implement but thats another story).
> Although we got close to the AJP 1.4 protocol conclusion
> last year, nowadays all that looks strange to me.
> All those things might be implemented, but IMHO only
> as a AJP1.5 protocol.
>
> What we desperately need right now are 3 things:
> 1. Allowing to have +8K headers
> 2. Allowing to have +0x9999 single header limit
> 3. Mechanism to tell the Tomcat to gracefully close
> the connection.
>
> Now, the number 3 is very easy. A simple message
> like we have for SHUTDOW, but instead shutting down
> the entire Tomcat instance, closing down the socket/channel.
>
> OTOH first two are little bit tricky :)
> I have some ideas:
> 1. Larger headers can be treated as we handle the POST data.
> If there are more headers then 8K, then a servlet container
> should send GIVE_ME_MORE_HEADERS message.
> 2. If the single header is larger then 39321 bytes, then it
> should be send as POST data, with servlet container requesting
> 8K packets. Those headers would be treated as multiple POST
> sequences, after the initial header(s) packet(s) have been
> read and before the actual POST data is read.
>
> Any comments?
>
> Regards,
> Mladen.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
Re: JK and AJP protocol enhancements
Posted by Bill Barker <wb...@wilshire.com>.
"Mladen Turk" <mt...@apache.org> wrote in message
news:44B0D76D.9040804@apache.org...
> You see how desperate I am when writing this on Sunday :)
>
> Anyhow, we are pretty close to the new JK release that I
> hope will be the most usable and stable whatsoever.
>
> The things we agreed so many times before, but having
> obviously too little resources to actually create are
> the 1.3 branch (aka JK3) and the AJP protocol stuff.
>
> Now there is the problem with that. Henri even created
> a AJP1.4 protocol enhancements with all that login,
> discovery etc... (never implement but thats another story).
> Although we got close to the AJP 1.4 protocol conclusion
> last year, nowadays all that looks strange to me.
> All those things might be implemented, but IMHO only
> as a AJP1.5 protocol.
>
> What we desperately need right now are 3 things:
> 1. Allowing to have +8K headers
> 2. Allowing to have +0x9999 single header limit
> 3. Mechanism to tell the Tomcat to gracefully close
> the connection.
>
I'd like to see as well:
4. Separate the SSL variables into a separate message, since these are
rarely actually looked at and we should start sending the entire cert
chain.
5. Implement some method to let Httpd notify Tomcat that the client has
dropped the connection, rather than rudely closing the connection.
Although it is possible that having an ACK for 5. would turn out to be more
expensive then the occational having to re-open the socket connection.
> Now, the number 3 is very easy. A simple message
> like we have for SHUTDOW, but instead shutting down
> the entire Tomcat instance, closing down the socket/channel.
>
> OTOH first two are little bit tricky :)
> I have some ideas:
> 1. Larger headers can be treated as we handle the POST data.
> If there are more headers then 8K, then a servlet container
> should send GIVE_ME_MORE_HEADERS message.
+1
> 2. If the single header is larger then 39321 bytes, then it
> should be send as POST data, with servlet container requesting
> 8K packets. Those headers would be treated as multiple POST
> sequences, after the initial header(s) packet(s) have been
> read and before the actual POST data is read.
>
Or, simpiler (but less compatible :) whould be to change the String length
field to be 4 bytes instead of 2, and Tomcat would know that the beginning
of the next GMMH message is the continuation of the previous String.
> Any comments?
>
> Regards,
> Mladen.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: JK and AJP protocol enhancements
Posted by Jess Holle <je...@ptc.com>.
Mladen Turk wrote:
> You see how desperate I am when writing this on Sunday :)
>
> Anyhow, we are pretty close to the new JK release that I
> hope will be the most usable and stable whatsoever.
>
> The things we agreed so many times before, but having
> obviously too little resources to actually create are
> the 1.3 branch (aka JK3) and the AJP protocol stuff.
>
> Now there is the problem with that. Henri even created
> a AJP1.4 protocol enhancements with all that login,
> discovery etc... (never implement but thats another story).
> Although we got close to the AJP 1.4 protocol conclusion
> last year, nowadays all that looks strange to me.
> All those things might be implemented, but IMHO only
> as a AJP1.5 protocol.
>
> What we desperately need right now are 3 things:
> 1. Allowing to have +8K headers
> 2. Allowing to have +0x9999 single header limit
> 3. Mechanism to tell the Tomcat to gracefully close
> the connection.
>
> Now, the number 3 is very easy. A simple message
> like we have for SHUTDOW, but instead shutting down
> the entire Tomcat instance, closing down the socket/channel.
>
> OTOH first two are little bit tricky :)
> I have some ideas:
> 1. Larger headers can be treated as we handle the POST data.
> If there are more headers then 8K, then a servlet container
> should send GIVE_ME_MORE_HEADERS message.
> 2. If the single header is larger then 39321 bytes, then it
> should be send as POST data, with servlet container requesting
> 8K packets. Those headers would be treated as multiple POST
> sequences, after the initial header(s) packet(s) have been
> read and before the actual POST data is read.
>
> Any comments?
Only that item (1) is a short-term must for some...
--
Jess Holle
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: JK and AJP protocol enhancements
Posted by Jean-frederic Clere <jf...@gmail.com>.
Mladen Turk wrote:
> You see how desperate I am when writing this on Sunday :)
Was it too hot on the sea side?
>
> Anyhow, we are pretty close to the new JK release that I
> hope will be the most usable and stable whatsoever.
>
> The things we agreed so many times before, but having
> obviously too little resources to actually create are
> the 1.3 branch (aka JK3) and the AJP protocol stuff.
>
> Now there is the problem with that. Henri even created
> a AJP1.4 protocol enhancements with all that login,
> discovery etc... (never implement but thats another story).
> Although we got close to the AJP 1.4 protocol conclusion
> last year, nowadays all that looks strange to me.
> All those things might be implemented, but IMHO only
> as a AJP1.5 protocol.
What every protocol has it the detection of the version of its partner,
we have to add it.
We also need to write a state diagram of the protocol: something if the
webserver sends bli applicationserver (TC) will answer bla, next state
is state_1 if not next state is state_2.
That is easy the existing protocol and that will help to see how to
improve it.
Cheers
Jean-Frederic
>
> What we desperately need right now are 3 things:
> 1. Allowing to have +8K headers
> 2. Allowing to have +0x9999 single header limit
> 3. Mechanism to tell the Tomcat to gracefully close
> the connection.
>
> Now, the number 3 is very easy. A simple message
> like we have for SHUTDOW, but instead shutting down
> the entire Tomcat instance, closing down the socket/channel.
>
> OTOH first two are little bit tricky :)
> I have some ideas:
> 1. Larger headers can be treated as we handle the POST data.
> If there are more headers then 8K, then a servlet container
> should send GIVE_ME_MORE_HEADERS message.
> 2. If the single header is larger then 39321 bytes, then it
> should be send as POST data, with servlet container requesting
> 8K packets. Those headers would be treated as multiple POST
> sequences, after the initial header(s) packet(s) have been
> read and before the actual POST data is read.
>
> Any comments?
>
> Regards,
> Mladen.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org