You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geronimo.apache.org by Jason Dillon <ja...@planet57.com> on 2006/12/09 00:06:00 UTC
No legacy repos for Geronimo projects using Maven2
Maven does not behave well with a mix of default and legacy repos. I
have gone through and moved all legacy repos only to the modules
where they are used, and in some cases imported a module-local repo
to hold those artifacts so that the build does not need to include a
legacy repo. A few weeks ago I finally removed all the legacy
repos.... and now they are starting to creep back in.
Specifically the java.net repo, which is a legacy repo for jstl muck,
was added. Something needs to be done about this, so this repo can
be moved out of the project root, or removed altogether. The
addition of the legacy repo is known to cause problems with SNAPSHOT
resolution, and can get itself into a state with local metadata what
other artifacts will start to resolve in very, very, very strange
ways. So... don't use them.
This is compounded even more by the poor network connectivity (and
maybe connection limiting) done by the java.net repo, which is
causing builds to timeout all over the place.
* * *
Basically... maven remote repos suck ass... and should be limited in
use as much as possible if we want a stable and repeatable build for
any of our projects.
In a corporate environment, this problem is easily solved by managing
a local repo which has copies of all of the artifacts which are
required to build, often times stored in version control and labeled
with project using it.
If we are going to continue to use Maven (which I'm starting to
really wonder if it is worth it), then we really need to address this
problem... otherwise it will be an ongoing head-ache for the
foreseeable future... and really builds will never reach any level of
stability and will almost never have any ability to be reproduced.
Ugh, I've already spent so many hours debugging other peoples
reported issues, which most of the time end up resolve to problems
with Maven. I've been away for a little bit working on build
automation and in the few weeks I'm gone the build system has already
regressed in a few areas, and IMO its well on its way to becoming out
of control again while quickly. I am starting to believe that Maven
promotes that chaos, especially so for larger projects. I also
believe that Maven promotes build instability, where at times someone
might change some dependency which could completely hose our build
with out anyone really knowing why or having any paper trail (change
logs) to debug it.
IMO... the *ONLY* way to resolve these issues with Maven it so have
*ONE* repository which holds all artifacts used by our projects, and
have that repository under SVN control.
So, for this example of jstl on java.net, those artifacts would be
checked into the *ONE* repository and life goes on, no new pom config
to configure a new repo, no side-effects of poor network connectivity
to remote repositories, no strange behavior due to legacy vs. default
layout metadata muck.
If we were using Ant, then we would have needed to implement
something like this already. Though its more cumbersome with Maven
since most of the crap that is downloaded is for Maven itself, not
for our dependencies, our dependencies are much, much, much easier to
manage than the stew of jars required to make Maven and its horde of
plugins work.
The more I use Maven, the more I dislike it... and I don't really see
a light at the end of the tunnel either... mvn is almost as slow
moving as Geronimo has been for the past year, so not sure how viable
it will be as a build tool for the future if they do not provide more
bug fixes sooner and faster. At this point... and ya, I may be in a
bad mood now... I don't think that mvn is an appropriate tool for
building production quality products... period.
--jason
Re: No legacy repos for Geronimo projects using Maven2
Posted by Donald Woods <dr...@yahoo.com>.
Is that an open invitation to help recreate those scripts?
Do others see a need for this? I certainly would like to see us put
them back in geronimo/server/trunk (or genesis) and enhance them to
allow svn checkout/update and orderly building of genesis, specs, server
and even handle openejb and devtools.....
-Donald
Jason Dillon wrote:
> You don't need to add this to settings.xml, you can simply:
>
> mvn -Dmaven.repo.local=some/path
>
> But you are right, to make this work, we need those native scripts to
> bootstrap the build again and set the correct value for the repo.
>
> --jason
>
>
> On Jan 2, 2007, at 8:32 AM, Donald Woods wrote:
>
>> You can create a settings.xml in the default user/.m2 directory to
>> point to an alternate local repo dir, like -
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>> <settings>
>> <localRepository>${LOCAL_GERONIMO_M2_REPO}</localRepository>
>> </settings>
>>
>> But the downside, is you're leaving it up to each developer to either
>> supply this environment variable or manually update the file and it
>> would be used by all Maven2 builds.... So, you would really need to
>> restore the bootstrap scripts to setup a local Geronimo repo dir
>> (checkout or update the needed files from svn) and then run a
>> multistage server build (genesis, maven plugins, modules, configs and
>> assemblies)....
>>
>>
>> -Donald
>>
>>
>> Jason Dillon wrote:
>>> Unfortunately this would have to be externally controlled, can not
>>> specify maven.repo.local in a pom, and if you could, can't root it to
>>> the project, as ${pom.basedir} will always keep changing.
>>> This was possible in m1 though... for what its worth. Only way to do
>>> this in m2 is to use native scripts.
>>> :-(
>>> --jason
>>> On Dec 11, 2006, at 2:49 PM, Guillaume Nodet wrote:
>>>> Is there a way to specify that in the pom ?
>>>> Or you have to rely on users to specify it on the command
>>>> line everytime they build (or use batch files).
>>>>
>>>> On 12/11/06, Jason Dillon <ja...@planet57.com> wrote:
>>>>> What do you mean? If you specify -Dmaven.repo.local=./svn-repo (or
>>>>> where ever the svn checkout is) and run the build offline, then the
>>>>> repo won't get modified, and thus only chance a bad artifact would
>>>>> get in there would be if someone checked in something bad, or if the
>>>>> local `mvn install` got messed up, but generally `mvn install`
>>>>> artifacts will never be checked into that repo.
>>>>>
>>>>> It is really too bad that there is not a local repo for deps and then
>>>>> a local repo for generated artifacts. The whole local repo thing is
>>>>> lossy IMO and makes it very difficult to assure quality of releases.
>>>>>
>>>>> --jason
>>>>>
>>>>>
>>>>> On Dec 11, 2006, at 7:55 AM, Guillaume Nodet wrote:
>>>>>
>>>>> > Also, keep in mind that there is no way to bypass the
>>>>> > local repository afaik. So if a bad artifact goes into the
>>>>> > user local repo, it may disturb Geronimo's build, even
>>>>> > if Geronimo build only use a single svn based remote
>>>>> > repo. In such a case, the only way to ensure that the
>>>>> > build will work is to start from a clean local repo.
>>>>> >
>>>>> > On 12/9/06, Jason Dillon <ja...@planet57.com> wrote:
>>>>> >> On Dec 8, 2006, at 6:41 PM, Prasad Kashyap wrote:
>>>>> >> >> ... At this point... and ya, I may be in a
>>>>> >> >> bad mood now... I don't think that mvn is an appropriate tool
>>>>> for
>>>>> >> >> building production quality products... period.
>>>>> >> >
>>>>> >> > I hear ye. I share the pain. But I fear the alternative -
>>>>> spending
>>>>> >> > considerable time migrating to another build system.
>>>>> >>
>>>>> >> Ya, I know... I'm not suggesting that we change any time soon.
>>>>> But I
>>>>> >> do fear that there is going to be some serious ongoing pain.
>>>>> >>
>>>>> >>
>>>>> >> > When you return from your bad mood to your jolly good ole' self
>>>>> >> again,
>>>>> >>
>>>>> >> I dunno... I'm jaded now... good ole jolly jason was eaten by
>>>>> the big
>>>>> >> angry maven monster... :-P
>>>>> >>
>>>>> >>
>>>>> >> > can you please shed more light on what it would take to have this
>>>>> >> > *ONE* repo; it's pros and cons and such..
>>>>> >>
>>>>> >> I've sent a few emails about this in the past. Major hurtles to
>>>>> this
>>>>> >> are going to be sysadmin/network overheads, ASF infra politics, and
>>>>> >> of course keeping the artifacts in sync. There are just way to
>>>>> many
>>>>> >> things that need to get downloaded, making the window for problems
>>>>> >> really quite massive.
>>>>> >>
>>>>> >> I'm still trying to figure out how to effectively workaround this
>>>>> >> problem for an open community... in a corporate setting this is
>>>>> a no
>>>>> >> brainer, setup a machine, back it up, setup proximity or
>>>>> maven-proxy
>>>>> >> to aggregate remote repos, then create a few local repos backed by
>>>>> >> svn to hold custom artifacts or specific versions to help reduce
>>>>> risk
>>>>> >> incurred by remote artifact stability. Then each project just
>>>>> lists
>>>>> >> that one repo.
>>>>> >>
>>>>> >> This works well, but due to the way maven works, other dependencies
>>>>> >> may list repos, which will then get picked up and used for
>>>>> artifacts
>>>>> >> selection, which tends to pollute the sanity and stability, but
>>>>> >> usually not too much. But its yet another flaw in maven's
>>>>> >> architecture which while its flexible and easy for smaller
>>>>> projects,
>>>>> >> its nearly impossible to make any sort of assurances for larger
>>>>> more
>>>>> >> complicated projects. Actually even for smaller ones it makes it
>>>>> >> very very difficult to ensure build stability over the life of the
>>>>> >> project (past build repeatability and assumed future compatibility,
>>>>> >> as at any time someone could publish a plugin or artifact which
>>>>> >> completely breaks your build, often times requiring days to debug
>>>>> >> why).
>>>>> >>
>>>>> >> The only way around this is to have total ownership of imported
>>>>> build
>>>>> >> artifacts and an effective paper trail for changes (ie svn change
>>>>> >> logs).
>>>>> >>
>>>>> >> While maven has made many things simpler... it really has made it a
>>>>> >> lot harder to implement stable, reliable and durable builds. :-(
>>>>> >>
>>>>> >> Anyways, all I can really think of to step around this problem,
>>>>> is to
>>>>> >> checkin all of the artifacts which are needed into svn,
>>>>> configure the
>>>>> >> build to use a checkout of that repo for its local and then always
>>>>> >> run offline. And periodically update the svn repo from remotes as
>>>>> >> well as manage some artifacts by hand. Essentially removing any
>>>>> >> remoteness from Maven, which is IMO key to making builds stable,
>>>>> >> reliable and durable.
>>>>> >>
>>>>> >> Svn has all the artifacts needed, so svn co will get you the right
>>>>> >> bits, svn up will make sure its the latest, so no need to keep
>>>>> making
>>>>> >> all those network calls to check for artifacts, which will speed
>>>>> the
>>>>> >> build up dramatically. Svn will always have a trail of who changed
>>>>> >> what when which can be easily correlated to build failures using
>>>>> a CI
>>>>> >> tool. Mysterious dependency download, metadata corruption, bad
>>>>> >> network connections basically go a way from the list of normal
>>>>> >> problems we run into. The repository gets labeled when the
>>>>> software
>>>>> >> gets labeled, so you can *always* go back in time, checkout an old
>>>>> >> release and build it... and have a very, very, very high chance
>>>>> that
>>>>> >> it will work with no fuss, only things which may break it would be
>>>>> >> environment related (deep windows folder, wrong jdk version,
>>>>> missing
>>>>> >> heap settings, etc).
>>>>> >>
>>>>> >> Dunno if there are other options really... maybe... but I can't
>>>>> think
>>>>> >> of it at the moment.
>>>>> >>
>>>>> >> I think the mvn plugin system is good, getting better once they fix
>>>>> >> some of the annoying bugs... and even better once they document the
>>>>> >> apis more. Wish the dang pom was not so verbose... or need to
>>>>> carry
>>>>> >> version details into each and every pom... but those are all minor.
>>>>> >> The major issue is the remote repo. Once you eliminate that, then
>>>>> >> mvn starts to look a whole lot more attractive for serious
>>>>> production
>>>>> >> builds.
>>>>> >>
>>>>> >> --jason
>>>>> >>
>>>>> >>
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Cheers,
>>>>> > Guillaume Nodet
>>>>>
>>>>>
>>>>
>>>>
>>>> --Cheers,
>>>> Guillaume Nodet
>
>
>
Re: No legacy repos for Geronimo projects using Maven2
Posted by Jason Dillon <ja...@planet57.com>.
You don't need to add this to settings.xml, you can simply:
mvn -Dmaven.repo.local=some/path
But you are right, to make this work, we need those native scripts to
bootstrap the build again and set the correct value for the repo.
--jason
On Jan 2, 2007, at 8:32 AM, Donald Woods wrote:
> You can create a settings.xml in the default user/.m2 directory to
> point to an alternate local repo dir, like -
>
> <?xml version="1.0" encoding="UTF-8"?>
> <settings>
> <localRepository>${LOCAL_GERONIMO_M2_REPO}</localRepository>
> </settings>
>
> But the downside, is you're leaving it up to each developer to
> either supply this environment variable or manually update the file
> and it would be used by all Maven2 builds.... So, you would really
> need to restore the bootstrap scripts to setup a local Geronimo
> repo dir (checkout or update the needed files from svn) and then
> run a multistage server build (genesis, maven plugins, modules,
> configs and assemblies)....
>
>
> -Donald
>
>
> Jason Dillon wrote:
>> Unfortunately this would have to be externally controlled, can not
>> specify maven.repo.local in a pom, and if you could, can't root it
>> to the project, as ${pom.basedir} will always keep changing.
>> This was possible in m1 though... for what its worth. Only way to
>> do this in m2 is to use native scripts.
>> :-(
>> --jason
>> On Dec 11, 2006, at 2:49 PM, Guillaume Nodet wrote:
>>> Is there a way to specify that in the pom ?
>>> Or you have to rely on users to specify it on the command
>>> line everytime they build (or use batch files).
>>>
>>> On 12/11/06, Jason Dillon <ja...@planet57.com> wrote:
>>>> What do you mean? If you specify -Dmaven.repo.local=./svn-repo (or
>>>> where ever the svn checkout is) and run the build offline, then the
>>>> repo won't get modified, and thus only chance a bad artifact would
>>>> get in there would be if someone checked in something bad, or if
>>>> the
>>>> local `mvn install` got messed up, but generally `mvn install`
>>>> artifacts will never be checked into that repo.
>>>>
>>>> It is really too bad that there is not a local repo for deps and
>>>> then
>>>> a local repo for generated artifacts. The whole local repo
>>>> thing is
>>>> lossy IMO and makes it very difficult to assure quality of
>>>> releases.
>>>>
>>>> --jason
>>>>
>>>>
>>>> On Dec 11, 2006, at 7:55 AM, Guillaume Nodet wrote:
>>>>
>>>> > Also, keep in mind that there is no way to bypass the
>>>> > local repository afaik. So if a bad artifact goes into the
>>>> > user local repo, it may disturb Geronimo's build, even
>>>> > if Geronimo build only use a single svn based remote
>>>> > repo. In such a case, the only way to ensure that the
>>>> > build will work is to start from a clean local repo.
>>>> >
>>>> > On 12/9/06, Jason Dillon <ja...@planet57.com> wrote:
>>>> >> On Dec 8, 2006, at 6:41 PM, Prasad Kashyap wrote:
>>>> >> >> ... At this point... and ya, I may be in a
>>>> >> >> bad mood now... I don't think that mvn is an appropriate
>>>> tool for
>>>> >> >> building production quality products... period.
>>>> >> >
>>>> >> > I hear ye. I share the pain. But I fear the alternative -
>>>> spending
>>>> >> > considerable time migrating to another build system.
>>>> >>
>>>> >> Ya, I know... I'm not suggesting that we change any time
>>>> soon. But I
>>>> >> do fear that there is going to be some serious ongoing pain.
>>>> >>
>>>> >>
>>>> >> > When you return from your bad mood to your jolly good ole'
>>>> self
>>>> >> again,
>>>> >>
>>>> >> I dunno... I'm jaded now... good ole jolly jason was eaten by
>>>> the big
>>>> >> angry maven monster... :-P
>>>> >>
>>>> >>
>>>> >> > can you please shed more light on what it would take to
>>>> have this
>>>> >> > *ONE* repo; it's pros and cons and such..
>>>> >>
>>>> >> I've sent a few emails about this in the past. Major hurtles
>>>> to this
>>>> >> are going to be sysadmin/network overheads, ASF infra
>>>> politics, and
>>>> >> of course keeping the artifacts in sync. There are just way
>>>> to many
>>>> >> things that need to get downloaded, making the window for
>>>> problems
>>>> >> really quite massive.
>>>> >>
>>>> >> I'm still trying to figure out how to effectively workaround
>>>> this
>>>> >> problem for an open community... in a corporate setting this
>>>> is a no
>>>> >> brainer, setup a machine, back it up, setup proximity or
>>>> maven-proxy
>>>> >> to aggregate remote repos, then create a few local repos
>>>> backed by
>>>> >> svn to hold custom artifacts or specific versions to help
>>>> reduce risk
>>>> >> incurred by remote artifact stability. Then each project
>>>> just lists
>>>> >> that one repo.
>>>> >>
>>>> >> This works well, but due to the way maven works, other
>>>> dependencies
>>>> >> may list repos, which will then get picked up and used for
>>>> artifacts
>>>> >> selection, which tends to pollute the sanity and stability, but
>>>> >> usually not too much. But its yet another flaw in maven's
>>>> >> architecture which while its flexible and easy for smaller
>>>> projects,
>>>> >> its nearly impossible to make any sort of assurances for
>>>> larger more
>>>> >> complicated projects. Actually even for smaller ones it
>>>> makes it
>>>> >> very very difficult to ensure build stability over the life
>>>> of the
>>>> >> project (past build repeatability and assumed future
>>>> compatibility,
>>>> >> as at any time someone could publish a plugin or artifact which
>>>> >> completely breaks your build, often times requiring days to
>>>> debug
>>>> >> why).
>>>> >>
>>>> >> The only way around this is to have total ownership of
>>>> imported build
>>>> >> artifacts and an effective paper trail for changes (ie svn
>>>> change
>>>> >> logs).
>>>> >>
>>>> >> While maven has made many things simpler... it really has
>>>> made it a
>>>> >> lot harder to implement stable, reliable and durable builds. :-(
>>>> >>
>>>> >> Anyways, all I can really think of to step around this
>>>> problem, is to
>>>> >> checkin all of the artifacts which are needed into svn,
>>>> configure the
>>>> >> build to use a checkout of that repo for its local and then
>>>> always
>>>> >> run offline. And periodically update the svn repo from
>>>> remotes as
>>>> >> well as manage some artifacts by hand. Essentially removing any
>>>> >> remoteness from Maven, which is IMO key to making builds stable,
>>>> >> reliable and durable.
>>>> >>
>>>> >> Svn has all the artifacts needed, so svn co will get you the
>>>> right
>>>> >> bits, svn up will make sure its the latest, so no need to
>>>> keep making
>>>> >> all those network calls to check for artifacts, which will
>>>> speed the
>>>> >> build up dramatically. Svn will always have a trail of who
>>>> changed
>>>> >> what when which can be easily correlated to build failures
>>>> using a CI
>>>> >> tool. Mysterious dependency download, metadata corruption, bad
>>>> >> network connections basically go a way from the list of normal
>>>> >> problems we run into. The repository gets labeled when the
>>>> software
>>>> >> gets labeled, so you can *always* go back in time, checkout
>>>> an old
>>>> >> release and build it... and have a very, very, very high
>>>> chance that
>>>> >> it will work with no fuss, only things which may break it
>>>> would be
>>>> >> environment related (deep windows folder, wrong jdk version,
>>>> missing
>>>> >> heap settings, etc).
>>>> >>
>>>> >> Dunno if there are other options really... maybe... but I
>>>> can't think
>>>> >> of it at the moment.
>>>> >>
>>>> >> I think the mvn plugin system is good, getting better once
>>>> they fix
>>>> >> some of the annoying bugs... and even better once they
>>>> document the
>>>> >> apis more. Wish the dang pom was not so verbose... or need
>>>> to carry
>>>> >> version details into each and every pom... but those are all
>>>> minor.
>>>> >> The major issue is the remote repo. Once you eliminate that,
>>>> then
>>>> >> mvn starts to look a whole lot more attractive for serious
>>>> production
>>>> >> builds.
>>>> >>
>>>> >> --jason
>>>> >>
>>>> >>
>>>> >
>>>> >
>>>> > --
>>>> > Cheers,
>>>> > Guillaume Nodet
>>>>
>>>>
>>>
>>>
>>> --Cheers,
>>> Guillaume Nodet
Re: No legacy repos for Geronimo projects using Maven2
Posted by Donald Woods <dr...@yahoo.com>.
You can create a settings.xml in the default user/.m2 directory to point
to an alternate local repo dir, like -
<?xml version="1.0" encoding="UTF-8"?>
<settings>
<localRepository>${LOCAL_GERONIMO_M2_REPO}</localRepository>
</settings>
But the downside, is you're leaving it up to each developer to either
supply this environment variable or manually update the file and it
would be used by all Maven2 builds.... So, you would really need to
restore the bootstrap scripts to setup a local Geronimo repo dir
(checkout or update the needed files from svn) and then run a multistage
server build (genesis, maven plugins, modules, configs and assemblies)....
-Donald
Jason Dillon wrote:
> Unfortunately this would have to be externally controlled, can not
> specify maven.repo.local in a pom, and if you could, can't root it to
> the project, as ${pom.basedir} will always keep changing.
>
> This was possible in m1 though... for what its worth. Only way to do
> this in m2 is to use native scripts.
>
> :-(
>
> --jason
>
>
> On Dec 11, 2006, at 2:49 PM, Guillaume Nodet wrote:
>
>> Is there a way to specify that in the pom ?
>> Or you have to rely on users to specify it on the command
>> line everytime they build (or use batch files).
>>
>> On 12/11/06, Jason Dillon <ja...@planet57.com> wrote:
>>> What do you mean? If you specify -Dmaven.repo.local=./svn-repo (or
>>> where ever the svn checkout is) and run the build offline, then the
>>> repo won't get modified, and thus only chance a bad artifact would
>>> get in there would be if someone checked in something bad, or if the
>>> local `mvn install` got messed up, but generally `mvn install`
>>> artifacts will never be checked into that repo.
>>>
>>> It is really too bad that there is not a local repo for deps and then
>>> a local repo for generated artifacts. The whole local repo thing is
>>> lossy IMO and makes it very difficult to assure quality of releases.
>>>
>>> --jason
>>>
>>>
>>> On Dec 11, 2006, at 7:55 AM, Guillaume Nodet wrote:
>>>
>>> > Also, keep in mind that there is no way to bypass the
>>> > local repository afaik. So if a bad artifact goes into the
>>> > user local repo, it may disturb Geronimo's build, even
>>> > if Geronimo build only use a single svn based remote
>>> > repo. In such a case, the only way to ensure that the
>>> > build will work is to start from a clean local repo.
>>> >
>>> > On 12/9/06, Jason Dillon <ja...@planet57.com> wrote:
>>> >> On Dec 8, 2006, at 6:41 PM, Prasad Kashyap wrote:
>>> >> >> ... At this point... and ya, I may be in a
>>> >> >> bad mood now... I don't think that mvn is an appropriate tool for
>>> >> >> building production quality products... period.
>>> >> >
>>> >> > I hear ye. I share the pain. But I fear the alternative - spending
>>> >> > considerable time migrating to another build system.
>>> >>
>>> >> Ya, I know... I'm not suggesting that we change any time soon. But I
>>> >> do fear that there is going to be some serious ongoing pain.
>>> >>
>>> >>
>>> >> > When you return from your bad mood to your jolly good ole' self
>>> >> again,
>>> >>
>>> >> I dunno... I'm jaded now... good ole jolly jason was eaten by the big
>>> >> angry maven monster... :-P
>>> >>
>>> >>
>>> >> > can you please shed more light on what it would take to have this
>>> >> > *ONE* repo; it's pros and cons and such..
>>> >>
>>> >> I've sent a few emails about this in the past. Major hurtles to this
>>> >> are going to be sysadmin/network overheads, ASF infra politics, and
>>> >> of course keeping the artifacts in sync. There are just way to many
>>> >> things that need to get downloaded, making the window for problems
>>> >> really quite massive.
>>> >>
>>> >> I'm still trying to figure out how to effectively workaround this
>>> >> problem for an open community... in a corporate setting this is a no
>>> >> brainer, setup a machine, back it up, setup proximity or maven-proxy
>>> >> to aggregate remote repos, then create a few local repos backed by
>>> >> svn to hold custom artifacts or specific versions to help reduce risk
>>> >> incurred by remote artifact stability. Then each project just lists
>>> >> that one repo.
>>> >>
>>> >> This works well, but due to the way maven works, other dependencies
>>> >> may list repos, which will then get picked up and used for artifacts
>>> >> selection, which tends to pollute the sanity and stability, but
>>> >> usually not too much. But its yet another flaw in maven's
>>> >> architecture which while its flexible and easy for smaller projects,
>>> >> its nearly impossible to make any sort of assurances for larger more
>>> >> complicated projects. Actually even for smaller ones it makes it
>>> >> very very difficult to ensure build stability over the life of the
>>> >> project (past build repeatability and assumed future compatibility,
>>> >> as at any time someone could publish a plugin or artifact which
>>> >> completely breaks your build, often times requiring days to debug
>>> >> why).
>>> >>
>>> >> The only way around this is to have total ownership of imported build
>>> >> artifacts and an effective paper trail for changes (ie svn change
>>> >> logs).
>>> >>
>>> >> While maven has made many things simpler... it really has made it a
>>> >> lot harder to implement stable, reliable and durable builds. :-(
>>> >>
>>> >> Anyways, all I can really think of to step around this problem, is to
>>> >> checkin all of the artifacts which are needed into svn, configure the
>>> >> build to use a checkout of that repo for its local and then always
>>> >> run offline. And periodically update the svn repo from remotes as
>>> >> well as manage some artifacts by hand. Essentially removing any
>>> >> remoteness from Maven, which is IMO key to making builds stable,
>>> >> reliable and durable.
>>> >>
>>> >> Svn has all the artifacts needed, so svn co will get you the right
>>> >> bits, svn up will make sure its the latest, so no need to keep making
>>> >> all those network calls to check for artifacts, which will speed the
>>> >> build up dramatically. Svn will always have a trail of who changed
>>> >> what when which can be easily correlated to build failures using a CI
>>> >> tool. Mysterious dependency download, metadata corruption, bad
>>> >> network connections basically go a way from the list of normal
>>> >> problems we run into. The repository gets labeled when the software
>>> >> gets labeled, so you can *always* go back in time, checkout an old
>>> >> release and build it... and have a very, very, very high chance that
>>> >> it will work with no fuss, only things which may break it would be
>>> >> environment related (deep windows folder, wrong jdk version, missing
>>> >> heap settings, etc).
>>> >>
>>> >> Dunno if there are other options really... maybe... but I can't think
>>> >> of it at the moment.
>>> >>
>>> >> I think the mvn plugin system is good, getting better once they fix
>>> >> some of the annoying bugs... and even better once they document the
>>> >> apis more. Wish the dang pom was not so verbose... or need to carry
>>> >> version details into each and every pom... but those are all minor.
>>> >> The major issue is the remote repo. Once you eliminate that, then
>>> >> mvn starts to look a whole lot more attractive for serious production
>>> >> builds.
>>> >>
>>> >> --jason
>>> >>
>>> >>
>>> >
>>> >
>>> > --
>>> > Cheers,
>>> > Guillaume Nodet
>>>
>>>
>>
>>
>> --Cheers,
>> Guillaume Nodet
>
>
>
Re: No legacy repos for Geronimo projects using Maven2
Posted by Jason Dillon <ja...@planet57.com>.
Unfortunately this would have to be externally controlled, can not
specify maven.repo.local in a pom, and if you could, can't root it to
the project, as ${pom.basedir} will always keep changing.
This was possible in m1 though... for what its worth. Only way to do
this in m2 is to use native scripts.
:-(
--jason
On Dec 11, 2006, at 2:49 PM, Guillaume Nodet wrote:
> Is there a way to specify that in the pom ?
> Or you have to rely on users to specify it on the command
> line everytime they build (or use batch files).
>
> On 12/11/06, Jason Dillon <ja...@planet57.com> wrote:
>> What do you mean? If you specify -Dmaven.repo.local=./svn-repo (or
>> where ever the svn checkout is) and run the build offline, then the
>> repo won't get modified, and thus only chance a bad artifact would
>> get in there would be if someone checked in something bad, or if the
>> local `mvn install` got messed up, but generally `mvn install`
>> artifacts will never be checked into that repo.
>>
>> It is really too bad that there is not a local repo for deps and then
>> a local repo for generated artifacts. The whole local repo thing is
>> lossy IMO and makes it very difficult to assure quality of releases.
>>
>> --jason
>>
>>
>> On Dec 11, 2006, at 7:55 AM, Guillaume Nodet wrote:
>>
>> > Also, keep in mind that there is no way to bypass the
>> > local repository afaik. So if a bad artifact goes into the
>> > user local repo, it may disturb Geronimo's build, even
>> > if Geronimo build only use a single svn based remote
>> > repo. In such a case, the only way to ensure that the
>> > build will work is to start from a clean local repo.
>> >
>> > On 12/9/06, Jason Dillon <ja...@planet57.com> wrote:
>> >> On Dec 8, 2006, at 6:41 PM, Prasad Kashyap wrote:
>> >> >> ... At this point... and ya, I may be in a
>> >> >> bad mood now... I don't think that mvn is an appropriate
>> tool for
>> >> >> building production quality products... period.
>> >> >
>> >> > I hear ye. I share the pain. But I fear the alternative -
>> spending
>> >> > considerable time migrating to another build system.
>> >>
>> >> Ya, I know... I'm not suggesting that we change any time soon.
>> But I
>> >> do fear that there is going to be some serious ongoing pain.
>> >>
>> >>
>> >> > When you return from your bad mood to your jolly good ole' self
>> >> again,
>> >>
>> >> I dunno... I'm jaded now... good ole jolly jason was eaten by
>> the big
>> >> angry maven monster... :-P
>> >>
>> >>
>> >> > can you please shed more light on what it would take to have
>> this
>> >> > *ONE* repo; it's pros and cons and such..
>> >>
>> >> I've sent a few emails about this in the past. Major hurtles
>> to this
>> >> are going to be sysadmin/network overheads, ASF infra politics,
>> and
>> >> of course keeping the artifacts in sync. There are just way to
>> many
>> >> things that need to get downloaded, making the window for problems
>> >> really quite massive.
>> >>
>> >> I'm still trying to figure out how to effectively workaround this
>> >> problem for an open community... in a corporate setting this is
>> a no
>> >> brainer, setup a machine, back it up, setup proximity or maven-
>> proxy
>> >> to aggregate remote repos, then create a few local repos backed by
>> >> svn to hold custom artifacts or specific versions to help
>> reduce risk
>> >> incurred by remote artifact stability. Then each project just
>> lists
>> >> that one repo.
>> >>
>> >> This works well, but due to the way maven works, other
>> dependencies
>> >> may list repos, which will then get picked up and used for
>> artifacts
>> >> selection, which tends to pollute the sanity and stability, but
>> >> usually not too much. But its yet another flaw in maven's
>> >> architecture which while its flexible and easy for smaller
>> projects,
>> >> its nearly impossible to make any sort of assurances for larger
>> more
>> >> complicated projects. Actually even for smaller ones it makes it
>> >> very very difficult to ensure build stability over the life of the
>> >> project (past build repeatability and assumed future
>> compatibility,
>> >> as at any time someone could publish a plugin or artifact which
>> >> completely breaks your build, often times requiring days to debug
>> >> why).
>> >>
>> >> The only way around this is to have total ownership of imported
>> build
>> >> artifacts and an effective paper trail for changes (ie svn change
>> >> logs).
>> >>
>> >> While maven has made many things simpler... it really has made
>> it a
>> >> lot harder to implement stable, reliable and durable builds. :-(
>> >>
>> >> Anyways, all I can really think of to step around this problem,
>> is to
>> >> checkin all of the artifacts which are needed into svn,
>> configure the
>> >> build to use a checkout of that repo for its local and then always
>> >> run offline. And periodically update the svn repo from remotes as
>> >> well as manage some artifacts by hand. Essentially removing any
>> >> remoteness from Maven, which is IMO key to making builds stable,
>> >> reliable and durable.
>> >>
>> >> Svn has all the artifacts needed, so svn co will get you the right
>> >> bits, svn up will make sure its the latest, so no need to keep
>> making
>> >> all those network calls to check for artifacts, which will
>> speed the
>> >> build up dramatically. Svn will always have a trail of who
>> changed
>> >> what when which can be easily correlated to build failures
>> using a CI
>> >> tool. Mysterious dependency download, metadata corruption, bad
>> >> network connections basically go a way from the list of normal
>> >> problems we run into. The repository gets labeled when the
>> software
>> >> gets labeled, so you can *always* go back in time, checkout an old
>> >> release and build it... and have a very, very, very high chance
>> that
>> >> it will work with no fuss, only things which may break it would be
>> >> environment related (deep windows folder, wrong jdk version,
>> missing
>> >> heap settings, etc).
>> >>
>> >> Dunno if there are other options really... maybe... but I can't
>> think
>> >> of it at the moment.
>> >>
>> >> I think the mvn plugin system is good, getting better once they
>> fix
>> >> some of the annoying bugs... and even better once they document
>> the
>> >> apis more. Wish the dang pom was not so verbose... or need to
>> carry
>> >> version details into each and every pom... but those are all
>> minor.
>> >> The major issue is the remote repo. Once you eliminate that, then
>> >> mvn starts to look a whole lot more attractive for serious
>> production
>> >> builds.
>> >>
>> >> --jason
>> >>
>> >>
>> >
>> >
>> > --
>> > Cheers,
>> > Guillaume Nodet
>>
>>
>
>
> --
> Cheers,
> Guillaume Nodet
Re: No legacy repos for Geronimo projects using Maven2
Posted by Guillaume Nodet <gn...@gmail.com>.
Is there a way to specify that in the pom ?
Or you have to rely on users to specify it on the command
line everytime they build (or use batch files).
On 12/11/06, Jason Dillon <ja...@planet57.com> wrote:
> What do you mean? If you specify -Dmaven.repo.local=./svn-repo (or
> where ever the svn checkout is) and run the build offline, then the
> repo won't get modified, and thus only chance a bad artifact would
> get in there would be if someone checked in something bad, or if the
> local `mvn install` got messed up, but generally `mvn install`
> artifacts will never be checked into that repo.
>
> It is really too bad that there is not a local repo for deps and then
> a local repo for generated artifacts. The whole local repo thing is
> lossy IMO and makes it very difficult to assure quality of releases.
>
> --jason
>
>
> On Dec 11, 2006, at 7:55 AM, Guillaume Nodet wrote:
>
> > Also, keep in mind that there is no way to bypass the
> > local repository afaik. So if a bad artifact goes into the
> > user local repo, it may disturb Geronimo's build, even
> > if Geronimo build only use a single svn based remote
> > repo. In such a case, the only way to ensure that the
> > build will work is to start from a clean local repo.
> >
> > On 12/9/06, Jason Dillon <ja...@planet57.com> wrote:
> >> On Dec 8, 2006, at 6:41 PM, Prasad Kashyap wrote:
> >> >> ... At this point... and ya, I may be in a
> >> >> bad mood now... I don't think that mvn is an appropriate tool for
> >> >> building production quality products... period.
> >> >
> >> > I hear ye. I share the pain. But I fear the alternative - spending
> >> > considerable time migrating to another build system.
> >>
> >> Ya, I know... I'm not suggesting that we change any time soon. But I
> >> do fear that there is going to be some serious ongoing pain.
> >>
> >>
> >> > When you return from your bad mood to your jolly good ole' self
> >> again,
> >>
> >> I dunno... I'm jaded now... good ole jolly jason was eaten by the big
> >> angry maven monster... :-P
> >>
> >>
> >> > can you please shed more light on what it would take to have this
> >> > *ONE* repo; it's pros and cons and such..
> >>
> >> I've sent a few emails about this in the past. Major hurtles to this
> >> are going to be sysadmin/network overheads, ASF infra politics, and
> >> of course keeping the artifacts in sync. There are just way to many
> >> things that need to get downloaded, making the window for problems
> >> really quite massive.
> >>
> >> I'm still trying to figure out how to effectively workaround this
> >> problem for an open community... in a corporate setting this is a no
> >> brainer, setup a machine, back it up, setup proximity or maven-proxy
> >> to aggregate remote repos, then create a few local repos backed by
> >> svn to hold custom artifacts or specific versions to help reduce risk
> >> incurred by remote artifact stability. Then each project just lists
> >> that one repo.
> >>
> >> This works well, but due to the way maven works, other dependencies
> >> may list repos, which will then get picked up and used for artifacts
> >> selection, which tends to pollute the sanity and stability, but
> >> usually not too much. But its yet another flaw in maven's
> >> architecture which while its flexible and easy for smaller projects,
> >> its nearly impossible to make any sort of assurances for larger more
> >> complicated projects. Actually even for smaller ones it makes it
> >> very very difficult to ensure build stability over the life of the
> >> project (past build repeatability and assumed future compatibility,
> >> as at any time someone could publish a plugin or artifact which
> >> completely breaks your build, often times requiring days to debug
> >> why).
> >>
> >> The only way around this is to have total ownership of imported build
> >> artifacts and an effective paper trail for changes (ie svn change
> >> logs).
> >>
> >> While maven has made many things simpler... it really has made it a
> >> lot harder to implement stable, reliable and durable builds. :-(
> >>
> >> Anyways, all I can really think of to step around this problem, is to
> >> checkin all of the artifacts which are needed into svn, configure the
> >> build to use a checkout of that repo for its local and then always
> >> run offline. And periodically update the svn repo from remotes as
> >> well as manage some artifacts by hand. Essentially removing any
> >> remoteness from Maven, which is IMO key to making builds stable,
> >> reliable and durable.
> >>
> >> Svn has all the artifacts needed, so svn co will get you the right
> >> bits, svn up will make sure its the latest, so no need to keep making
> >> all those network calls to check for artifacts, which will speed the
> >> build up dramatically. Svn will always have a trail of who changed
> >> what when which can be easily correlated to build failures using a CI
> >> tool. Mysterious dependency download, metadata corruption, bad
> >> network connections basically go a way from the list of normal
> >> problems we run into. The repository gets labeled when the software
> >> gets labeled, so you can *always* go back in time, checkout an old
> >> release and build it... and have a very, very, very high chance that
> >> it will work with no fuss, only things which may break it would be
> >> environment related (deep windows folder, wrong jdk version, missing
> >> heap settings, etc).
> >>
> >> Dunno if there are other options really... maybe... but I can't think
> >> of it at the moment.
> >>
> >> I think the mvn plugin system is good, getting better once they fix
> >> some of the annoying bugs... and even better once they document the
> >> apis more. Wish the dang pom was not so verbose... or need to carry
> >> version details into each and every pom... but those are all minor.
> >> The major issue is the remote repo. Once you eliminate that, then
> >> mvn starts to look a whole lot more attractive for serious production
> >> builds.
> >>
> >> --jason
> >>
> >>
> >
> >
> > --
> > Cheers,
> > Guillaume Nodet
>
>
--
Cheers,
Guillaume Nodet
Re: No legacy repos for Geronimo projects using Maven2
Posted by Jason Dillon <ja...@planet57.com>.
What do you mean? If you specify -Dmaven.repo.local=./svn-repo (or
where ever the svn checkout is) and run the build offline, then the
repo won't get modified, and thus only chance a bad artifact would
get in there would be if someone checked in something bad, or if the
local `mvn install` got messed up, but generally `mvn install`
artifacts will never be checked into that repo.
It is really too bad that there is not a local repo for deps and then
a local repo for generated artifacts. The whole local repo thing is
lossy IMO and makes it very difficult to assure quality of releases.
--jason
On Dec 11, 2006, at 7:55 AM, Guillaume Nodet wrote:
> Also, keep in mind that there is no way to bypass the
> local repository afaik. So if a bad artifact goes into the
> user local repo, it may disturb Geronimo's build, even
> if Geronimo build only use a single svn based remote
> repo. In such a case, the only way to ensure that the
> build will work is to start from a clean local repo.
>
> On 12/9/06, Jason Dillon <ja...@planet57.com> wrote:
>> On Dec 8, 2006, at 6:41 PM, Prasad Kashyap wrote:
>> >> ... At this point... and ya, I may be in a
>> >> bad mood now... I don't think that mvn is an appropriate tool for
>> >> building production quality products... period.
>> >
>> > I hear ye. I share the pain. But I fear the alternative - spending
>> > considerable time migrating to another build system.
>>
>> Ya, I know... I'm not suggesting that we change any time soon. But I
>> do fear that there is going to be some serious ongoing pain.
>>
>>
>> > When you return from your bad mood to your jolly good ole' self
>> again,
>>
>> I dunno... I'm jaded now... good ole jolly jason was eaten by the big
>> angry maven monster... :-P
>>
>>
>> > can you please shed more light on what it would take to have this
>> > *ONE* repo; it's pros and cons and such..
>>
>> I've sent a few emails about this in the past. Major hurtles to this
>> are going to be sysadmin/network overheads, ASF infra politics, and
>> of course keeping the artifacts in sync. There are just way to many
>> things that need to get downloaded, making the window for problems
>> really quite massive.
>>
>> I'm still trying to figure out how to effectively workaround this
>> problem for an open community... in a corporate setting this is a no
>> brainer, setup a machine, back it up, setup proximity or maven-proxy
>> to aggregate remote repos, then create a few local repos backed by
>> svn to hold custom artifacts or specific versions to help reduce risk
>> incurred by remote artifact stability. Then each project just lists
>> that one repo.
>>
>> This works well, but due to the way maven works, other dependencies
>> may list repos, which will then get picked up and used for artifacts
>> selection, which tends to pollute the sanity and stability, but
>> usually not too much. But its yet another flaw in maven's
>> architecture which while its flexible and easy for smaller projects,
>> its nearly impossible to make any sort of assurances for larger more
>> complicated projects. Actually even for smaller ones it makes it
>> very very difficult to ensure build stability over the life of the
>> project (past build repeatability and assumed future compatibility,
>> as at any time someone could publish a plugin or artifact which
>> completely breaks your build, often times requiring days to debug
>> why).
>>
>> The only way around this is to have total ownership of imported build
>> artifacts and an effective paper trail for changes (ie svn change
>> logs).
>>
>> While maven has made many things simpler... it really has made it a
>> lot harder to implement stable, reliable and durable builds. :-(
>>
>> Anyways, all I can really think of to step around this problem, is to
>> checkin all of the artifacts which are needed into svn, configure the
>> build to use a checkout of that repo for its local and then always
>> run offline. And periodically update the svn repo from remotes as
>> well as manage some artifacts by hand. Essentially removing any
>> remoteness from Maven, which is IMO key to making builds stable,
>> reliable and durable.
>>
>> Svn has all the artifacts needed, so svn co will get you the right
>> bits, svn up will make sure its the latest, so no need to keep making
>> all those network calls to check for artifacts, which will speed the
>> build up dramatically. Svn will always have a trail of who changed
>> what when which can be easily correlated to build failures using a CI
>> tool. Mysterious dependency download, metadata corruption, bad
>> network connections basically go a way from the list of normal
>> problems we run into. The repository gets labeled when the software
>> gets labeled, so you can *always* go back in time, checkout an old
>> release and build it... and have a very, very, very high chance that
>> it will work with no fuss, only things which may break it would be
>> environment related (deep windows folder, wrong jdk version, missing
>> heap settings, etc).
>>
>> Dunno if there are other options really... maybe... but I can't think
>> of it at the moment.
>>
>> I think the mvn plugin system is good, getting better once they fix
>> some of the annoying bugs... and even better once they document the
>> apis more. Wish the dang pom was not so verbose... or need to carry
>> version details into each and every pom... but those are all minor.
>> The major issue is the remote repo. Once you eliminate that, then
>> mvn starts to look a whole lot more attractive for serious production
>> builds.
>>
>> --jason
>>
>>
>
>
> --
> Cheers,
> Guillaume Nodet
Re: No legacy repos for Geronimo projects using Maven2
Posted by Guillaume Nodet <gn...@gmail.com>.
Also, keep in mind that there is no way to bypass the
local repository afaik. So if a bad artifact goes into the
user local repo, it may disturb Geronimo's build, even
if Geronimo build only use a single svn based remote
repo. In such a case, the only way to ensure that the
build will work is to start from a clean local repo.
On 12/9/06, Jason Dillon <ja...@planet57.com> wrote:
> On Dec 8, 2006, at 6:41 PM, Prasad Kashyap wrote:
> >> ... At this point... and ya, I may be in a
> >> bad mood now... I don't think that mvn is an appropriate tool for
> >> building production quality products... period.
> >
> > I hear ye. I share the pain. But I fear the alternative - spending
> > considerable time migrating to another build system.
>
> Ya, I know... I'm not suggesting that we change any time soon. But I
> do fear that there is going to be some serious ongoing pain.
>
>
> > When you return from your bad mood to your jolly good ole' self again,
>
> I dunno... I'm jaded now... good ole jolly jason was eaten by the big
> angry maven monster... :-P
>
>
> > can you please shed more light on what it would take to have this
> > *ONE* repo; it's pros and cons and such..
>
> I've sent a few emails about this in the past. Major hurtles to this
> are going to be sysadmin/network overheads, ASF infra politics, and
> of course keeping the artifacts in sync. There are just way to many
> things that need to get downloaded, making the window for problems
> really quite massive.
>
> I'm still trying to figure out how to effectively workaround this
> problem for an open community... in a corporate setting this is a no
> brainer, setup a machine, back it up, setup proximity or maven-proxy
> to aggregate remote repos, then create a few local repos backed by
> svn to hold custom artifacts or specific versions to help reduce risk
> incurred by remote artifact stability. Then each project just lists
> that one repo.
>
> This works well, but due to the way maven works, other dependencies
> may list repos, which will then get picked up and used for artifacts
> selection, which tends to pollute the sanity and stability, but
> usually not too much. But its yet another flaw in maven's
> architecture which while its flexible and easy for smaller projects,
> its nearly impossible to make any sort of assurances for larger more
> complicated projects. Actually even for smaller ones it makes it
> very very difficult to ensure build stability over the life of the
> project (past build repeatability and assumed future compatibility,
> as at any time someone could publish a plugin or artifact which
> completely breaks your build, often times requiring days to debug why).
>
> The only way around this is to have total ownership of imported build
> artifacts and an effective paper trail for changes (ie svn change logs).
>
> While maven has made many things simpler... it really has made it a
> lot harder to implement stable, reliable and durable builds. :-(
>
> Anyways, all I can really think of to step around this problem, is to
> checkin all of the artifacts which are needed into svn, configure the
> build to use a checkout of that repo for its local and then always
> run offline. And periodically update the svn repo from remotes as
> well as manage some artifacts by hand. Essentially removing any
> remoteness from Maven, which is IMO key to making builds stable,
> reliable and durable.
>
> Svn has all the artifacts needed, so svn co will get you the right
> bits, svn up will make sure its the latest, so no need to keep making
> all those network calls to check for artifacts, which will speed the
> build up dramatically. Svn will always have a trail of who changed
> what when which can be easily correlated to build failures using a CI
> tool. Mysterious dependency download, metadata corruption, bad
> network connections basically go a way from the list of normal
> problems we run into. The repository gets labeled when the software
> gets labeled, so you can *always* go back in time, checkout an old
> release and build it... and have a very, very, very high chance that
> it will work with no fuss, only things which may break it would be
> environment related (deep windows folder, wrong jdk version, missing
> heap settings, etc).
>
> Dunno if there are other options really... maybe... but I can't think
> of it at the moment.
>
> I think the mvn plugin system is good, getting better once they fix
> some of the annoying bugs... and even better once they document the
> apis more. Wish the dang pom was not so verbose... or need to carry
> version details into each and every pom... but those are all minor.
> The major issue is the remote repo. Once you eliminate that, then
> mvn starts to look a whole lot more attractive for serious production
> builds.
>
> --jason
>
>
--
Cheers,
Guillaume Nodet
Re: No legacy repos for Geronimo projects using Maven2
Posted by Jason Dillon <ja...@planet57.com>.
On Dec 8, 2006, at 6:41 PM, Prasad Kashyap wrote:
>> ... At this point... and ya, I may be in a
>> bad mood now... I don't think that mvn is an appropriate tool for
>> building production quality products... period.
>
> I hear ye. I share the pain. But I fear the alternative - spending
> considerable time migrating to another build system.
Ya, I know... I'm not suggesting that we change any time soon. But I
do fear that there is going to be some serious ongoing pain.
> When you return from your bad mood to your jolly good ole' self again,
I dunno... I'm jaded now... good ole jolly jason was eaten by the big
angry maven monster... :-P
> can you please shed more light on what it would take to have this
> *ONE* repo; it's pros and cons and such..
I've sent a few emails about this in the past. Major hurtles to this
are going to be sysadmin/network overheads, ASF infra politics, and
of course keeping the artifacts in sync. There are just way to many
things that need to get downloaded, making the window for problems
really quite massive.
I'm still trying to figure out how to effectively workaround this
problem for an open community... in a corporate setting this is a no
brainer, setup a machine, back it up, setup proximity or maven-proxy
to aggregate remote repos, then create a few local repos backed by
svn to hold custom artifacts or specific versions to help reduce risk
incurred by remote artifact stability. Then each project just lists
that one repo.
This works well, but due to the way maven works, other dependencies
may list repos, which will then get picked up and used for artifacts
selection, which tends to pollute the sanity and stability, but
usually not too much. But its yet another flaw in maven's
architecture which while its flexible and easy for smaller projects,
its nearly impossible to make any sort of assurances for larger more
complicated projects. Actually even for smaller ones it makes it
very very difficult to ensure build stability over the life of the
project (past build repeatability and assumed future compatibility,
as at any time someone could publish a plugin or artifact which
completely breaks your build, often times requiring days to debug why).
The only way around this is to have total ownership of imported build
artifacts and an effective paper trail for changes (ie svn change logs).
While maven has made many things simpler... it really has made it a
lot harder to implement stable, reliable and durable builds. :-(
Anyways, all I can really think of to step around this problem, is to
checkin all of the artifacts which are needed into svn, configure the
build to use a checkout of that repo for its local and then always
run offline. And periodically update the svn repo from remotes as
well as manage some artifacts by hand. Essentially removing any
remoteness from Maven, which is IMO key to making builds stable,
reliable and durable.
Svn has all the artifacts needed, so svn co will get you the right
bits, svn up will make sure its the latest, so no need to keep making
all those network calls to check for artifacts, which will speed the
build up dramatically. Svn will always have a trail of who changed
what when which can be easily correlated to build failures using a CI
tool. Mysterious dependency download, metadata corruption, bad
network connections basically go a way from the list of normal
problems we run into. The repository gets labeled when the software
gets labeled, so you can *always* go back in time, checkout an old
release and build it... and have a very, very, very high chance that
it will work with no fuss, only things which may break it would be
environment related (deep windows folder, wrong jdk version, missing
heap settings, etc).
Dunno if there are other options really... maybe... but I can't think
of it at the moment.
I think the mvn plugin system is good, getting better once they fix
some of the annoying bugs... and even better once they document the
apis more. Wish the dang pom was not so verbose... or need to carry
version details into each and every pom... but those are all minor.
The major issue is the remote repo. Once you eliminate that, then
mvn starts to look a whole lot more attractive for serious production
builds.
--jason
Re: No legacy repos for Geronimo projects using Maven2
Posted by Prasad Kashyap <go...@gmail.com>.
On 12/8/06, Jason Dillon <ja...@planet57.com> wrote:
>
> If we are going to continue to use Maven (which I'm starting to
> really wonder if it is worth it),....
>
.. (content clipped)...
>
> ... At this point... and ya, I may be in a
> bad mood now... I don't think that mvn is an appropriate tool for
> building production quality products... period.
>
> --jason
>
I hear ye. I share the pain. But I fear the alternative - spending
considerable time migrating to another build system.
When you return from your bad mood to your jolly good ole' self again,
can you please shed more light on what it would take to have this
*ONE* repo; it's pros and cons and such..
Cheers
Prasad.
Re: No legacy repos for Geronimo projects using Maven2
Posted by Matt Hogstrom <ma...@hogstrom.org>.
Jason V,
Copying you for your feedback and awareness.
On Dec 8, 2006, at 6:06 PM, Jason Dillon wrote:
> Maven does not behave well with a mix of default and legacy repos.
> I have gone through and moved all legacy repos only to the modules
> where they are used, and in some cases imported a module-local repo
> to hold those artifacts so that the build does not need to include
> a legacy repo. A few weeks ago I finally removed all the legacy
> repos.... and now they are starting to creep back in.
>
> Specifically the java.net repo, which is a legacy repo for jstl
> muck, was added. Something needs to be done about this, so this
> repo can be moved out of the project root, or removed altogether.
> The addition of the legacy repo is known to cause problems with
> SNAPSHOT resolution, and can get itself into a state with local
> metadata what other artifacts will start to resolve in very, very,
> very strange ways. So... don't use them.
>
> This is compounded even more by the poor network connectivity (and
> maybe connection limiting) done by the java.net repo, which is
> causing builds to timeout all over the place.
>
> * * *
>
> Basically... maven remote repos suck ass... and should be limited
> in use as much as possible if we want a stable and repeatable build
> for any of our projects.
>
> In a corporate environment, this problem is easily solved by
> managing a local repo which has copies of all of the artifacts
> which are required to build, often times stored in version control
> and labeled with project using it.
>
> If we are going to continue to use Maven (which I'm starting to
> really wonder if it is worth it), then we really need to address
> this problem... otherwise it will be an ongoing head-ache for the
> foreseeable future... and really builds will never reach any level
> of stability and will almost never have any ability to be reproduced.
>
> Ugh, I've already spent so many hours debugging other peoples
> reported issues, which most of the time end up resolve to problems
> with Maven. I've been away for a little bit working on build
> automation and in the few weeks I'm gone the build system has
> already regressed in a few areas, and IMO its well on its way to
> becoming out of control again while quickly. I am starting to
> believe that Maven promotes that chaos, especially so for larger
> projects. I also believe that Maven promotes build instability,
> where at times someone might change some dependency which could
> completely hose our build with out anyone really knowing why or
> having any paper trail (change logs) to debug it.
>
> IMO... the *ONLY* way to resolve these issues with Maven it so have
> *ONE* repository which holds all artifacts used by our projects,
> and have that repository under SVN control.
>
> So, for this example of jstl on java.net, those artifacts would be
> checked into the *ONE* repository and life goes on, no new pom
> config to configure a new repo, no side-effects of poor network
> connectivity to remote repositories, no strange behavior due to
> legacy vs. default layout metadata muck.
>
> If we were using Ant, then we would have needed to implement
> something like this already. Though its more cumbersome with Maven
> since most of the crap that is downloaded is for Maven itself, not
> for our dependencies, our dependencies are much, much, much easier
> to manage than the stew of jars required to make Maven and its
> horde of plugins work.
>
> The more I use Maven, the more I dislike it... and I don't really
> see a light at the end of the tunnel either... mvn is almost as
> slow moving as Geronimo has been for the past year, so not sure how
> viable it will be as a build tool for the future if they do not
> provide more bug fixes sooner and faster. At this point... and ya,
> I may be in a bad mood now... I don't think that mvn is an
> appropriate tool for building production quality products... period.
>
> --jason
>
Matt Hogstrom
matt@hogstrom.org