You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Mark Thomas <ma...@apache.org> on 2010/11/06 02:08:39 UTC
[PATCH] Parallel deployment
All,
The clean-up is in place and I have a patch [1] that adds support for
parallel deployment. There is still some work to do before the feature
can be used (Manager app needs to be made version aware when listing
apps, need to test with clustering, need to write some docs) but I'd
like to apply this patch sooner rather than later so the code is in
place for the next release.
Behaviour should remain unchanged unless someone deploys an app with the
version marker (##<version>) at the end of the WAR / dir / context.xml
name so even if the todo list above isn't complete it shouldn't get in
the way of the next release.
Before I apply the patch I wanted to give folks an opportunity to review
and comment on whether or not it was too big a change for the 7.0.x
branch at this stage. My own view is that it is not.
Cheers,
Mark
[1]
http://people.apache.org/~markt/patches/2010-11-05-parallel-deployment.patch
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: [PATCH] Parallel deployment
Posted by Rainer Jung <ra...@kippdata.de>.
On 06.11.2010 14:00, Rainer Jung wrote:
> On 06.11.2010 12:57, Tim Funk wrote:
>> When running mod_jk with sticky session, but not using tomcat clustering
>> ... Will adding a new version append the version number to the end of
>> the session cookie AFTER the engineId (used by mod_jk for determining
>> where to route the request)
>
> Tried it and the jvmRoute comes last. Since it is likely that people put
> dots into the webappVersion, I'll add a loop to mod_jk that keeps
> looking for dots if it can't match the worker name to something it knows.
Added in r1032065. Needs testing though. It should loop through all the
dot separated suffixes until it finds one that fits the name of an LB
member.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: [PATCH] Parallel deployment
Posted by Mark Thomas <ma...@apache.org>.
On 08/11/2010 18:05, Remy Maucherat wrote:
> On Mon, 2010-11-08 at 18:26 +0100, Rainer Jung wrote:
>> On 08.11.2010 18:05, Remy Maucherat wrote:
>>> I'll up that to -1, with another concern I've just thought about: if
>>> using a session id path which includes more than one webapp, the version
>>> numbers will have problems matching between the two webapps.
>>
>> You mean situations like portals / emptysessionpath?
>
> It is replaced by direct session path configuration in Servlet 3, but
> the problem is the same as with the old portal workaround from previous
> versions.
I have updated the proposed patch to match the session ID using the
session manager:
http://people.apache.org/~markt/patches/2010-11-15-parallel-deployment.patch
The docs, manager app work and clustering tests still need to be
completed but I wanted to give folks sight of where I was up to.
As always, feedback welcome.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: [PATCH] Parallel deployment
Posted by Remy Maucherat <re...@apache.org>.
On Mon, 2010-11-08 at 18:26 +0100, Rainer Jung wrote:
> On 08.11.2010 18:05, Remy Maucherat wrote:
> > I'll up that to -1, with another concern I've just thought about: if
> > using a session id path which includes more than one webapp, the version
> > numbers will have problems matching between the two webapps.
>
> You mean situations like portals / emptysessionpath?
It is replaced by direct session path configuration in Servlet 3, but
the problem is the same as with the old portal workaround from previous
versions.
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: [PATCH] Parallel deployment
Posted by Mark Thomas <ma...@apache.org>.
On 08/11/2010 17:26, Rainer Jung wrote:
> On 08.11.2010 18:05, Remy Maucherat wrote:
>> I'll up that to -1, with another concern I've just thought about: if
>> using a session id path which includes more than one webapp, the version
>> numbers will have problems matching between the two webapps.
>
> You mean situations like portals / emptysessionpath?
Yeah - that is going to cause problems. Calling the manager is looking
increasingly like a better option.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: [PATCH] Parallel deployment
Posted by Rainer Jung <ra...@kippdata.de>.
On 08.11.2010 18:05, Remy Maucherat wrote:
> I'll up that to -1, with another concern I've just thought about: if
> using a session id path which includes more than one webapp, the version
> numbers will have problems matching between the two webapps.
You mean situations like portals / emptysessionpath?
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: [PATCH] Parallel deployment
Posted by Remy Maucherat <re...@apache.org>.
On Mon, 2010-11-08 at 15:46 +0000, Mark Thomas wrote:
> I did consider that approach but rejected it for a couple of reasons:
> - Mapper needs to be manager aware - it isn't currently
Add a one method interface to be able to perform a session lookup.
> - Performance
Likely, having to do String manipulation/parsing is much worse than a
map lookup.
> Now the clean-up is in place and we can focus on the actual changes I'll
> probably go back and take another look at this option. It is almost
> certain I didn't fully explore all the possible ways of doing this.
>
> > I don't agree with adding a version number/id to the session id, or
> > changing its format, this will create legacy issues, as we can see right
> > now.
>
> An understandable concern.
I'll up that to -1, with another concern I've just thought about: if
using a session id path which includes more than one webapp, the version
numbers will have problems matching between the two webapps.
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: [PATCH] Parallel deployment
Posted by Mark Thomas <ma...@apache.org>.
On 08/11/2010 15:40, Remy Maucherat wrote:
> On Mon, 2010-11-08 at 14:31 +0000, Mark Thomas wrote:
>> We might be able to avoid that be limiting the version to just integers.
>> I think that is reasonable but would like to hear some feedback from others.
>>
>> That does raise the issue of whether to convert the provided version to
>> a (zero padded) string and continue with String compares or to convert
>> the version parsed from the session ID to an int. Performance is the
>> thing that worries me but I might be over-thinking this. Some
>> micro-benchmarks are probably called for.
>
> Ah, it seems this bad feature is spawning ugly hacks :( Why is it a good
> idea to append one-more-thing to the session id ? Already with the JVM
> route it is costly, and has spawned a ton of bugs (updates not being
> made when switching to a new node, etc etc).
>
> Isn't it better to have the mapper simply query the existing managers if
> multiple versions exist ?
>
> The algorithm:
> - if the Context has old versions active, the mapper will query the
> managers to find the right Context
> - of match, then you know the context
> - otherwise it belongs to the latest version
> - if no old versions of a context, no manager lookups, so no cost
I did consider that approach but rejected it for a couple of reasons:
- Mapper needs to be manager aware - it isn't currently
- Performance
Now the clean-up is in place and we can focus on the actual changes I'll
probably go back and take another look at this option. It is almost
certain I didn't fully explore all the possible ways of doing this.
> I don't agree with adding a version number/id to the session id, or
> changing its format, this will create legacy issues, as we can see right
> now.
An understandable concern.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: [PATCH] Parallel deployment
Posted by Remy Maucherat <re...@apache.org>.
On Mon, 2010-11-08 at 14:31 +0000, Mark Thomas wrote:
> We might be able to avoid that be limiting the version to just integers.
> I think that is reasonable but would like to hear some feedback from others.
>
> That does raise the issue of whether to convert the provided version to
> a (zero padded) string and continue with String compares or to convert
> the version parsed from the session ID to an int. Performance is the
> thing that worries me but I might be over-thinking this. Some
> micro-benchmarks are probably called for.
Ah, it seems this bad feature is spawning ugly hacks :( Why is it a good
idea to append one-more-thing to the session id ? Already with the JVM
route it is costly, and has spawned a ton of bugs (updates not being
made when switching to a new node, etc etc).
Isn't it better to have the mapper simply query the existing managers if
multiple versions exist ?
The algorithm:
- if the Context has old versions active, the mapper will query the
managers to find the right Context
- of match, then you know the context
- otherwise it belongs to the latest version
- if no old versions of a context, no manager lookups, so no cost
I don't agree with adding a version number/id to the session id, or
changing its format, this will create legacy issues, as we can see right
now.
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: [PATCH] Parallel deployment
Posted by Mark Thomas <ma...@apache.org>.
On 06/11/2010 13:00, Rainer Jung wrote:
> On 06.11.2010 12:57, Tim Funk wrote:
>> When running mod_jk with sticky session, but not using tomcat clustering
>> ... Will adding a new version append the version number to the end of
>> the session cookie AFTER the engineId (used by mod_jk for determining
>> where to route the request)
>
> Tried it and the jvmRoute comes last. Since it is likely that people put
> dots into the webappVersion, I'll add a loop to mod_jk that keeps
> looking for dots if it can't match the worker name to something it knows.
We might be able to avoid that be limiting the version to just integers.
I think that is reasonable but would like to hear some feedback from others.
That does raise the issue of whether to convert the provided version to
a (zero padded) string and continue with String compares or to convert
the version parsed from the session ID to an int. Performance is the
thing that worries me but I might be over-thinking this. Some
micro-benchmarks are probably called for.
>> Since the session id tomcat generates is always all uppercase. Should
>> the 'V' for the append be a lower case v? Even though session id
>> collision is *rare*, but using a char that isn't in the session id name
>> space would eliminate that problem.
>
> The ids are actualy hex, so "V" is safe at least concrning ur own id
> generators. Nevertheless I also wonder whether a more obvious character
> would make it easier to understand (like "-" or "_"). Of course we are
> not safe from collissions with webappVersion characters or jvmRoute
> characters.
Indeed. Hence my thoughts around integers.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: [PATCH] Parallel deployment
Posted by Rainer Jung <ra...@kippdata.de>.
On 06.11.2010 12:57, Tim Funk wrote:
> When running mod_jk with sticky session, but not using tomcat clustering
> ... Will adding a new version append the version number to the end of
> the session cookie AFTER the engineId (used by mod_jk for determining
> where to route the request)
Tried it and the jvmRoute comes last. Since it is likely that people put
dots into the webappVersion, I'll add a loop to mod_jk that keeps
looking for dots if it can't match the worker name to something it knows.
> Since the session id tomcat generates is always all uppercase. Should
> the 'V' for the append be a lower case v? Even though session id
> collision is *rare*, but using a char that isn't in the session id name
> space would eliminate that problem.
The ids are actualy hex, so "V" is safe at least concrning ur own id
generators. Nevertheless I also wonder whether a more obvious character
would make it easier to understand (like "-" or "_"). Of course we are
not safe from collissions with webappVersion characters or jvmRoute
characters.
Still need to read the patch, just strated playing around with a patched
trunk.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: [PATCH] Parallel deployment
Posted by Pid <pi...@pidster.com>.
On 08/11/2010 15:09, Rainer Jung wrote:
> Furthermore we could as a convenience convert all dots to underscores and
Or s/dots/non-alphanumerics/
p
Re: [PATCH] Parallel deployment
Posted by Rainer Jung <ra...@kippdata.de>.
On 08.11.2010 15:27, Mark Thomas wrote:
> Yep - that looks like a left-over from when I was playing with int
> rather than String for version. I'm still in two minds about that bit.
> There are places internally where using String makes it easier but
> limiting it to int would allow us to avoid a bunch of issues.
If we limit to int we would need to either reject any webapp with
invalid version or give it an automatic version and log a fat message
about that. That's a bit problematic. Human beings typically like to
think about application versions in terms of some hierarchical
structured number, possibly composed with textual suffixes.
So why not making a compromise, like allowing everything apart from the
already used dot in a webapp version, so the session id would be
session id = tradition hex id "-" webapp version "." jvmRoute
Furthermore we could as a convenience convert all dots to underscores
and log a fat warning about that. As a result, we would know, that
everything behind the first dot is the jvmRoute, and everything before
but after the first dash is the webapp version.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: [PATCH] Parallel deployment
Posted by Mark Thomas <ma...@apache.org>.
On 06/11/2010 11:57, Tim Funk wrote:
> This might cause a problem of using == instead of equals() for strcmp
>
> if (version == (request.getContext().getWebappVersion())) {
> mapRequired = false;
> }
Yep - that looks like a left-over from when I was playing with int
rather than String for version. I'm still in two minds about that bit.
There are places internally where using String makes it easier but
limiting it to int would allow us to avoid a bunch of issues.
> When running mod_jk with sticky session, but not using tomcat clustering
> ... Will adding a new version append the version number to the end of
> the session cookie AFTER the engineId (used by mod_jk for determining
> where to route the request)
Should be <sessionID>V<version>.jvmRoute
> Since the session id tomcat generates is always all uppercase. Should
> the 'V' for the append be a lower case v? Even though session id
> collision is *rare*, but using a char that isn't in the session id name
> space would eliminate that problem.
V was chosen since it isn;t a valid HEX digit.
> Otherwise +1. For those who are uninterested in the feature, it appears
> the extra overhead is minimal. (Some micro benchmarks would be nice for
> any skeptics ... or those just curious :) )
Thanks.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: [PATCH] Parallel deployment
Posted by Tim Funk <fu...@apache.org>.
This might cause a problem of using == instead of equals() for strcmp
if (version == (request.getContext().getWebappVersion())) {
mapRequired = false;
}
When running mod_jk with sticky session, but not using tomcat clustering
... Will adding a new version append the version number to the end of
the session cookie AFTER the engineId (used by mod_jk for determining
where to route the request)
Since the session id tomcat generates is always all uppercase. Should
the 'V' for the append be a lower case v? Even though session id
collision is *rare*, but using a char that isn't in the session id name
space would eliminate that problem.
Otherwise +1. For those who are uninterested in the feature, it appears
the extra overhead is minimal. (Some micro benchmarks would be nice for
any skeptics ... or those just curious :) )
-Tim
On 11/5/2010 9:08 PM, Mark Thomas wrote:
> All,
>
> The clean-up is in place and I have a patch [1] that adds support for
> parallel deployment. There is still some work to do before the feature
> can be used (Manager app needs to be made version aware when listing
> apps, need to test with clustering, need to write some docs) but I'd
> like to apply this patch sooner rather than later so the code is in
> place for the next release.
>
> Behaviour should remain unchanged unless someone deploys an app with the
> version marker (##<version>) at the end of the WAR / dir / context.xml
> name so even if the todo list above isn't complete it shouldn't get in
> the way of the next release.
>
> Before I apply the patch I wanted to give folks an opportunity to review
> and comment on whether or not it was too big a change for the 7.0.x
> branch at this stage. My own view is that it is not.
>
> Cheers,
>
> Mark
>
>
> [1]
> http://people.apache.org/~markt/patches/2010-11-05-parallel-deployment.patch
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org