You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geronimo.apache.org by Dain Sundstrom <da...@iq80.com> on 2006/05/11 20:39:16 UTC
Artifact patch system [was Re: replacing tomcat classes]
I think that technically this is a very easy problem to solve. The
difficult problem will be our recommendation to users on how to use
the feature.
Technical
---------
I think the best way to implement this is to add another method
getPatches(Artifact artifact) the Repository interface (this is
analogous to the getDependecies method which appends to the class
path). Then we just modify the class loader building code in
Configuration to put the patch jars before the artifact in the
classpath.
One thing to note, this proposed patch system will only address class
loading of artifacts. If a user wants to patch a library inside of a
web application, they will need to modify the web application
directly. This is particularly tricky since the load order of jars
in WEB-INF/lib is not specified and updating it requires a full
redeploy (not a restart as most would expect). Additionally, this
system would not address patching resources inside of a war. For
that, the user would have to overwrite the files in the unpacked
deployment.
The only tricky part of implementing this system will be deciding how
we want to associate patches with artifacts. A single flat directory
is easiest for users, but it difficult to avoid name collisions. It
would be very easy for us to have some sort of foo-1.1-23456.patch
files in the normal repository structure, but that requires an
administrator to know where to put files which is error prone. I'm
personally leaning toward the single patch directory simply because
it will make it easier for admins to see which patches the server has
installed.
Finally, this system will impact any tool that is using the
repository. I'm specifically thinking of the plugin packaging and
download code which will have to be modified to grab the patches. I
also suspect it will effect the eclipse tooling also.
Recomendations
--------------
I agree with David that it is a bad idea to replace only a few
classes in a jar. The process is inherently error prone, and only
provides a very risky stop gap measure. I also agree with Matt that
it is important be able to patch just a few classes in an emergency,
and as soon as the emergency is over, work should start to roll the
changes into a full jar update.
I think we should recommend that our users don't use the patch
feature unless there is an emergency. Further, I don't think this
project should ever ship class level patches, since it is so easy for
us to ship a whole jar.
BTW, does anyone know if maven has a patch system in the pipeline?
-dain
On May 11, 2006, at 9:44 AM, David Jencks wrote:
>
> On May 11, 2006, at 9:16 AM, Joe Bohn wrote:
>
>>
>> Bumping up the version should work for the jar approach. However,
>> I was still trying to figure out a way to honor the tomcat
>> recommendation of replacing just the modified classes. Is there
>> some way to make the version independent classloaders pick up
>> individual classes rather than entire jars?
>
> No, and I think that's a good thing. I think the tomcat team is
> giving bad advice.
>
> thanks
> david jencks
>
>>
>> Joe
>>
>>
>> David Jencks wrote:
>>> On May 11, 2006, at 8:29 AM, Joe Bohn wrote:
>>>>
>>>> Thanks for the quick response Jeff.
>>>>
>>>> I like the idea of a "system patch" location in the classpath
>>>> where we can pick up patches for anything we might include in a
>>>> geronimo assembly.
>>> I think this "system patch" idea will only work in environments
>>> with only one classloader, i.e. not geronimo. The problem is
>>> that the patched classes need to get into the correct
>>> classloader, "before" the normal versions. We'd need a patch
>>> directory for each module. I also think any solution that
>>> relies on the order of stuff in a classpath is inherently
>>> unstable and unreliable.
>>> Basically I think this is a terrible idea and we should avoid it
>>> at all costs. I think instead we should use our new version
>>> independence and replace jars with patched jars with slightly
>>> higher version numbers. IIUC this is what you propose doing
>>> below. This should not require removing the standard tomcat
>>> jars: the hight version number should be enough to get the
>>> correct version picked up.
>>> thanks
>>> david jencks
>>>>
>>>> I too was confused by the tomcat recommendation but it does
>>>> seem that they have a strategy for addressing necessary changes
>>>> with minimal interference in tomcat. I have also noticed some
>>>> things that make me wonder if my local tomcat build of 5.5.15
>>>> really does match the official 5.5.15 build. For example, the
>>>> only source for 5.5.15 that I could find was a zip file rather
>>>> than a svn branch or tag. I am not able to build from the
>>>> unpacked zip without making a change to move the contents of
>>>> jasper/jasper2 into the jasper directory itself. And the
>>>> version that is displayed when I hit tomcat with my rebuilt
>>>> image is 5.5 rather than 5.5.15 as with the official image.
>>>>
>>>> Until we figure out the correct approach for Geronimo I'm
>>>> thinking of using a compromise solution. The changes I need
>>>> in tomcat result in 4 of the 13 tomcat jars getting rebuilt.
>>>> Rather than replacing all of the tomcat jars with my local
>>>> build I have verified that replacing just the 4 changed jars
>>>> appears to work fine. I'm hoping this hybrid solution keeps
>>>> most of the official tomcat image and our local changes. I
>>>> haven't noticed any problems. Assuming the source is mostly
>>>> identical (apart from our changes) does anybody know of a
>>>> reason that I should definitely not take this approach?
>>>>
>>>> Joe
>>>>
>>>>
>>>> Jeff Genender wrote:
>>>>
>>>>> Ultimately, we probably would need to somehow build a "patch"
>>>>> directory
>>>>> or lib directory where we can ensure the URLClassLoader picks
>>>>> that up
>>>>> before all other classes. I think this is probably a good idea
>>>>> to have
>>>>> as well, so that we could release "service paks" or patches.
>>>>> I would be
>>>>> interested in others' thoughts on this, but I think this would
>>>>> be a nice
>>>>> feature to have.
>>>>> Right now I think your only choices are to either hard set a
>>>>> classpath
>>>>> to be sure the patches get picked up first or build a hacked
>>>>> Tomcat
>>>>> version, or rebuild Tomcat. Dain or David Jencks may be able
>>>>> to verify
>>>>> if the classpath solution would work or not as I have not dug
>>>>> into the
>>>>> new G classloaders to know if this would even be possible.
>>>>> The best solution right now may be to just build TC. I am a little
>>>>> confused as to why the TC guys say not to build the Tomcat
>>>>> from source
>>>>> (after its hacked). It seems like just an ant build script, so
>>>>> I don't
>>>>> understand why this is being discouraged. This way you can
>>>>> replace the
>>>>> Tomcat jars in the repo and you are good to go.
>>>>> Jeff
>>>>> Joe Bohn wrote:
>>>>>
>>>>>> Jeff,
>>>>>>
>>>>>> I am working with a user that is moving some applications
>>>>>> from tomcat to
>>>>>> geronimo. Due to some problems they have had to modify
>>>>>> tomcat source.
>>>>>> I was chatting with jasonb on the tomcat irc channel and he
>>>>>> recommended
>>>>>> that we only build the classes rather than rebuilding all of
>>>>>> tomcat. He
>>>>>> discouraged rebuilding all of tomcat because there are many
>>>>>> permutations
>>>>>> that can result in different build images and we should run
>>>>>> with as much
>>>>>> of the official tomcat build as possible to avoid problems.
>>>>>> He also
>>>>>> indicated that Tomcat's directory structure provides a place
>>>>>> to put
>>>>>> these "patch classes" in CATALINA_HOME/server/classes .
>>>>>>
>>>>>> Is there a similar place that we can put classes when tomcat
>>>>>> is running
>>>>>> under geronimo to have them picked up? Adding the tomcat
>>>>>> classes to our
>>>>>> new sharedlib doesn't seem to be the right place because it would
>>>>>> require a dependency from the tomcat config on sharelib. The
>>>>>> net result
>>>>>> would be that all tomcat apps would potentially pick up user
>>>>>> classes
>>>>>> added in sharedlib even if the user only intended these
>>>>>> classes for some
>>>>>> subset of the apps.
>>>>>>
>>>>>> Joe
>>>>>>
>>>>
>>>> --
>>>> Joe Bohn
>>>> joe.bohn at earthlink.net
>>>>
>>>> "He is no fool who gives what he cannot keep, to gain what he
>>>> cannot lose." -- Jim Elliot
>>
>> --
>> Joe Bohn
>> joe.bohn at earthlink.net
>>
>> "He is no fool who gives what he cannot keep, to gain what he
>> cannot lose." -- Jim Elliot
Re: Artifact patch system [was Re: replacing tomcat classes]
Posted by John Sisson <jr...@gmail.com>.
Dain Sundstrom wrote:
> I think that technically this is a very easy problem to solve. The
> difficult problem will be our recommendation to users on how to use
> the feature.
>
> Technical
> ---------
> I think the best way to implement this is to add another method
> getPatches(Artifact artifact) the Repository interface (this is
> analogous to the getDependecies method which appends to the class
> path). Then we just modify the class loader building code in
> Configuration to put the patch jars before the artifact in the classpath.
>
> One thing to note, this proposed patch system will only address class
> loading of artifacts. If a user wants to patch a library inside of a
> web application, they will need to modify the web application
> directly. This is particularly tricky since the load order of jars in
> WEB-INF/lib is not specified and updating it requires a full redeploy
> (not a restart as most would expect). Additionally, this system would
> not address patching resources inside of a war. For that, the user
> would have to overwrite the files in the unpacked deployment.
>
> The only tricky part of implementing this system will be deciding how
> we want to associate patches with artifacts. A single flat directory
> is easiest for users, but it difficult to avoid name collisions. It
> would be very easy for us to have some sort of foo-1.1-23456.patch
> files in the normal repository structure, but that requires an
> administrator to know where to put files which is error prone. I'm
> personally leaning toward the single patch directory simply because it
> will make it easier for admins to see which patches the server has
> installed.
>
Instead of having a single flat directory, would it be possible to have
a file that maps an existing artifact to a patched artifact in the
repository so that the patched artifact can be manipulated using
standard maven tools. See link below to Maven proposal on artifact
naming for patches etc.
If a user wants to see what patched files are currently in use they can
just look at this file.
If a patch needs to be sent to a user manually, it could be sent to them
in a tar/zip file that contains the appropriate directories under the
repository directory that can be extracted to the repository directory.
This would allow a number of different versions of the patch to be in
the repository and the user can easily switch between patch versions or
back-out the patch to a previous version that worked more reliably (in
case a number of patch iterations were produced in an attempt to resolve
a problem).
> Finally, this system will impact any tool that is using the
> repository. I'm specifically thinking of the plugin packaging and
> download code which will have to be modified to grab the patches. I
> also suspect it will effect the eclipse tooling also.
>
> Recomendations
> --------------
> I agree with David that it is a bad idea to replace only a few classes
> in a jar. The process is inherently error prone, and only provides a
> very risky stop gap measure. I also agree with Matt that it is
> important be able to patch just a few classes in an emergency, and as
> soon as the emergency is over, work should start to roll the changes
> into a full jar update.
>
> I think we should recommend that our users don't use the patch feature
> unless there is an emergency. Further, I don't think this project
> should ever ship class level patches, since it is so easy for us to
> ship a whole jar.
>
> BTW, does anyone know if maven has a patch system in the pipeline?
>
I'm not aware of a patch system as such, but have seen this proposal on
artifact naming for patches/service packs etc. Not sure what the status
of this proposal is:
http://docs.codehaus.org/display/MAVEN/Extending+Maven+2.0+Dependencies
Regards,
John
> -dain
>
> On May 11, 2006, at 9:44 AM, David Jencks wrote:
>
>>
>> On May 11, 2006, at 9:16 AM, Joe Bohn wrote:
>>
>>>
>>> Bumping up the version should work for the jar approach. However, I
>>> was still trying to figure out a way to honor the tomcat
>>> recommendation of replacing just the modified classes. Is there
>>> some way to make the version independent classloaders pick up
>>> individual classes rather than entire jars?
>>
>> No, and I think that's a good thing. I think the tomcat team is
>> giving bad advice.
>>
>> thanks
>> david jencks
>>
>>>
>>> Joe
>>>
>>>
>>> David Jencks wrote:
>>>> On May 11, 2006, at 8:29 AM, Joe Bohn wrote:
>>>>>
>>>>> Thanks for the quick response Jeff.
>>>>>
>>>>> I like the idea of a "system patch" location in the classpath
>>>>> where we can pick up patches for anything we might include in a
>>>>> geronimo assembly.
>>>> I think this "system patch" idea will only work in environments
>>>> with only one classloader, i.e. not geronimo. The problem is that
>>>> the patched classes need to get into the correct classloader,
>>>> "before" the normal versions. We'd need a patch directory for
>>>> each module. I also think any solution that relies on the order
>>>> of stuff in a classpath is inherently unstable and unreliable.
>>>> Basically I think this is a terrible idea and we should avoid it
>>>> at all costs. I think instead we should use our new version
>>>> independence and replace jars with patched jars with slightly
>>>> higher version numbers. IIUC this is what you propose doing
>>>> below. This should not require removing the standard tomcat jars:
>>>> the hight version number should be enough to get the correct
>>>> version picked up.
>>>> thanks
>>>> david jencks
>>>>>
>>>>> I too was confused by the tomcat recommendation but it does seem
>>>>> that they have a strategy for addressing necessary changes with
>>>>> minimal interference in tomcat. I have also noticed some things
>>>>> that make me wonder if my local tomcat build of 5.5.15 really
>>>>> does match the official 5.5.15 build. For example, the only
>>>>> source for 5.5.15 that I could find was a zip file rather than a
>>>>> svn branch or tag. I am not able to build from the unpacked zip
>>>>> without making a change to move the contents of jasper/jasper2
>>>>> into the jasper directory itself. And the version that is
>>>>> displayed when I hit tomcat with my rebuilt image is 5.5 rather
>>>>> than 5.5.15 as with the official image.
>>>>>
>>>>> Until we figure out the correct approach for Geronimo I'm
>>>>> thinking of using a compromise solution. The changes I need in
>>>>> tomcat result in 4 of the 13 tomcat jars getting rebuilt.
>>>>> Rather than replacing all of the tomcat jars with my local build
>>>>> I have verified that replacing just the 4 changed jars appears to
>>>>> work fine. I'm hoping this hybrid solution keeps most of the
>>>>> official tomcat image and our local changes. I haven't noticed
>>>>> any problems. Assuming the source is mostly identical (apart
>>>>> from our changes) does anybody know of a reason that I should
>>>>> definitely not take this approach?
>>>>>
>>>>> Joe
>>>>>
>>>>>
>>>>> Jeff Genender wrote:
>>>>>
>>>>>> Ultimately, we probably would need to somehow build a "patch"
>>>>>> directory
>>>>>> or lib directory where we can ensure the URLClassLoader picks
>>>>>> that up
>>>>>> before all other classes. I think this is probably a good idea
>>>>>> to have
>>>>>> as well, so that we could release "service paks" or patches. I
>>>>>> would be
>>>>>> interested in others' thoughts on this, but I think this would
>>>>>> be a nice
>>>>>> feature to have.
>>>>>> Right now I think your only choices are to either hard set a
>>>>>> classpath
>>>>>> to be sure the patches get picked up first or build a hacked Tomcat
>>>>>> version, or rebuild Tomcat. Dain or David Jencks may be able to
>>>>>> verify
>>>>>> if the classpath solution would work or not as I have not dug
>>>>>> into the
>>>>>> new G classloaders to know if this would even be possible.
>>>>>> The best solution right now may be to just build TC. I am a little
>>>>>> confused as to why the TC guys say not to build the Tomcat from
>>>>>> source
>>>>>> (after its hacked). It seems like just an ant build script, so
>>>>>> I don't
>>>>>> understand why this is being discouraged. This way you can
>>>>>> replace the
>>>>>> Tomcat jars in the repo and you are good to go.
>>>>>> Jeff
>>>>>> Joe Bohn wrote:
>>>>>>
>>>>>>> Jeff,
>>>>>>>
>>>>>>> I am working with a user that is moving some applications from
>>>>>>> tomcat to
>>>>>>> geronimo. Due to some problems they have had to modify tomcat
>>>>>>> source.
>>>>>>> I was chatting with jasonb on the tomcat irc channel and he
>>>>>>> recommended
>>>>>>> that we only build the classes rather than rebuilding all of
>>>>>>> tomcat. He
>>>>>>> discouraged rebuilding all of tomcat because there are many
>>>>>>> permutations
>>>>>>> that can result in different build images and we should run
>>>>>>> with as much
>>>>>>> of the official tomcat build as possible to avoid problems. He
>>>>>>> also
>>>>>>> indicated that Tomcat's directory structure provides a place to put
>>>>>>> these "patch classes" in CATALINA_HOME/server/classes .
>>>>>>>
>>>>>>> Is there a similar place that we can put classes when tomcat is
>>>>>>> running
>>>>>>> under geronimo to have them picked up? Adding the tomcat
>>>>>>> classes to our
>>>>>>> new sharedlib doesn't seem to be the right place because it would
>>>>>>> require a dependency from the tomcat config on sharelib. The
>>>>>>> net result
>>>>>>> would be that all tomcat apps would potentially pick up user
>>>>>>> classes
>>>>>>> added in sharedlib even if the user only intended these classes
>>>>>>> for some
>>>>>>> subset of the apps.
>>>>>>>
>>>>>>> Joe
>>>>>>>
>>>>>
>>>>> --Joe Bohn
>>>>> joe.bohn at earthlink.net
>>>>>
>>>>> "He is no fool who gives what he cannot keep, to gain what he
>>>>> cannot lose." -- Jim Elliot
>>>
>>> --Joe Bohn
>>> joe.bohn at earthlink.net
>>>
>>> "He is no fool who gives what he cannot keep, to gain what he cannot
>>> lose." -- Jim Elliot
>
>