You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Phil Steitz <ph...@gmail.com> on 2006/07/04 06:50:04 UTC
[all] Nightlies are back...mostly
Nightlies are back up and running for most components. I still think
maven2 + continuum is the way to go long term, but to get the
nightlies back up, I hacked together a little bash script based on
Craig's and set it up to run as a crontab on vmbuild.apache.org.
It would be great if people could have a look at the nightlies
themselves as well as the script, which I checked into commons-build
as "commons_nightly.sh." I have no shame regarding my limited bash
(or Java, actually ;-) skills, so feel free to make suggestions for
improvement or submit patches. Be careful checking in the script,
though, as it is executed via a crontab wrapper that svn ups it before
executing it on vmbuild.apache.org. You can test it locally by
changing the config to make your local host the deploy host. You have
to gen an ssh key (if you don't already have one) and trust it locally
to do this.
The script reads lists of components from commons-build txt files like
"nightly_sandbox_maven_list.txt" and executes maven clean dist for the
ones in the maven lists and ant clean dist for those in the ant lists
(after svn up). I split them up based on whether or not I could get
"maven clean dist" to work. The ant lists consist of components for
which the maven build did not work, but the ant build did. I have left
a few things in the maven list even though they fail, because the
reason seems trivial (e.g. clover license, or missing sun jars - I
will fix the second). It svn ups the lists before executing, so if
you want to move components from ant -> maven, or vice-versa, just
make the changes to the files and check them in. Remember to delete
as well as add; otherwise the script will run both.
Logs for each component build are for the moment being written out to
http://people.apache.org/~psteitz/commons-nightlies/
under dated directories. I didn't want to clutter the normal nightly
location with logs, but that could be changed if we like having them
there.
I ommitted dormant components. If anyone feels strongly that we
should be doing nightlies for any of these, the script can be patched
to do that.
There are no sigs or hashes. Another easy patch, but since this all
runs as me, I don't want to sign. Adding hashes is no problem. Some
of the maven builds do this already. Best would probably be to have
the maven builds all do it and the nightly script just copy out. I am
OK either way, but the script needs to be patched in either case to
scp the hashes.
More warts:
1) The files are not group writable on the deploy host
(people.apache.org). I set the umask in the script, but somehow the
permissions are not getting set right. Any pointers / patches on how
to get the wrapper or script itself to do this correctly would be
appreciated.
2) Ubuntu on vmbuild seems to have no zip command, so the ant builds
are not getting zips generated. I guess I could break down and fork
Ant again to do the zips, but maybe there is an easier way?
3) While it would be easy to get jar:deploy done from within the maven
loop, that seems like running with scissors. Also, what would be
deployed would be a SNAPSHOT that just over-wrote the previous (I
think). This exposes another wart - though the names are munged to be
dated, the internally bundled jars are not. For anyone wanting to
play with this, be forwarned that Ubuntu rename is picky - you need to
use the perl regexp form like the script does. Could be there is a
simple maven way to get this right.
Thanks in advance for feedback / patches / review of the output and
help getting the failing builds restored.
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Brett Porter <br...@apache.org>.
On 4/07/2006 2:50 PM, Phil Steitz wrote:
> 2) Ubuntu on vmbuild seems to have no zip command, so the ant builds
> are not getting zips generated. I guess I could break down and fork
> Ant again to do the zips, but maybe there is an easier way?
sudo apt-get install zip
Done :)
- Brett
--
Apache Maven - http://maven.apache.org/
Better Builds with Maven - http://library.mergere.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Niall Pemberton <ni...@gmail.com>.
On 7/9/06, Phil Steitz <ph...@gmail.com> wrote:
> On 7/7/06, Niall Pemberton <ni...@gmail.com> wrote:
> > On 7/7/06, Phil Steitz <ph...@gmail.com> wrote:
> > > On 7/7/06, Niall Pemberton <ni...@gmail.com> wrote:
> > > > Is it just the jar or the whole distro you want dated? If its just the
> > > > jar then set "maven.jar.final.name" - if its the distro (including
> > > > jar) then set "maven.final.name"
> > > >
> > > > Not sure the best way to set it - putting it in a "build.properties"
> > > > file would do it - or specifying it when you run the command:
> > > >
> > > > maven -Dmaven.final.name=commons-validator-20060707 dist
> > > >
> > > Thanks, Niall!
> > >
> > > That eliminates the need for name-munging in the script and also
> > > handles the jar.
> >
> > I tried "maven.jar.final.name" after I sent this, but didn't seem to
> > work - "maven.final.name" did though. If you use "maven.final.name"
> > and run the "dist" target theres a number of components which the
> > "checkstyle" generation won't work (validator included) - I can change
> > those later.
> >
>
> I did not see the validator failure when I did this.
I changed validator (and other components) yesterday:
http://svn.apache.org/viewvc?view=rev&revision=420094
> Neither the jar.final.name nor maven.final.name seem to be respected
> by the artifact plugin (1.5.2, the last version that works with maven
> 1.0.2), so I just added scp commands to deploy the dated snaps to the
> snapshot repo manually.
Thanks for doing this Phil, much appreciated.
Niall
> Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Phil Steitz <ph...@gmail.com>.
On 7/7/06, Niall Pemberton <ni...@gmail.com> wrote:
> On 7/7/06, Phil Steitz <ph...@gmail.com> wrote:
> > On 7/7/06, Niall Pemberton <ni...@gmail.com> wrote:
> > > Is it just the jar or the whole distro you want dated? If its just the
> > > jar then set "maven.jar.final.name" - if its the distro (including
> > > jar) then set "maven.final.name"
> > >
> > > Not sure the best way to set it - putting it in a "build.properties"
> > > file would do it - or specifying it when you run the command:
> > >
> > > maven -Dmaven.final.name=commons-validator-20060707 dist
> > >
> > Thanks, Niall!
> >
> > That eliminates the need for name-munging in the script and also
> > handles the jar.
>
> I tried "maven.jar.final.name" after I sent this, but didn't seem to
> work - "maven.final.name" did though. If you use "maven.final.name"
> and run the "dist" target theres a number of components which the
> "checkstyle" generation won't work (validator included) - I can change
> those later.
>
I did not see the validator failure when I did this.
Neither the jar.final.name nor maven.final.name seem to be respected
by the artifact plugin (1.5.2, the last version that works with maven
1.0.2), so I just added scp commands to deploy the dated snaps to the
snapshot repo manually.
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Niall Pemberton <ni...@gmail.com>.
On 7/7/06, Phil Steitz <ph...@gmail.com> wrote:
> On 7/7/06, Niall Pemberton <ni...@gmail.com> wrote:
> > Is it just the jar or the whole distro you want dated? If its just the
> > jar then set "maven.jar.final.name" - if its the distro (including
> > jar) then set "maven.final.name"
> >
> > Not sure the best way to set it - putting it in a "build.properties"
> > file would do it - or specifying it when you run the command:
> >
> > maven -Dmaven.final.name=commons-validator-20060707 dist
> >
> Thanks, Niall!
>
> That eliminates the need for name-munging in the script and also
> handles the jar.
I tried "maven.jar.final.name" after I sent this, but didn't seem to
work - "maven.final.name" did though. If you use "maven.final.name"
and run the "dist" target theres a number of components which the
"checkstyle" generation won't work (validator included) - I can change
those later.
Niall
> Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Phil Steitz <ph...@gmail.com>.
On 7/7/06, Niall Pemberton <ni...@gmail.com> wrote:
> Is it just the jar or the whole distro you want dated? If its just the
> jar then set "maven.jar.final.name" - if its the distro (including
> jar) then set "maven.final.name"
>
> Not sure the best way to set it - putting it in a "build.properties"
> file would do it - or specifying it when you run the command:
>
> maven -Dmaven.final.name=commons-validator-20060707 dist
>
Thanks, Niall!
That eliminates the need for name-munging in the script and also
handles the jar.
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Niall Pemberton <ni...@gmail.com>.
Is it just the jar or the whole distro you want dated? If its just the
jar then set "maven.jar.final.name" - if its the distro (including
jar) then set "maven.final.name"
Not sure the best way to set it - putting it in a "build.properties"
file would do it - or specifying it when you run the command:
maven -Dmaven.final.name=commons-validator-20060707 dist
Niall
On 7/7/06, Phil Steitz <ph...@gmail.com> wrote:
> On 7/6/06, Dion Gillard <di...@gmail.com> wrote:
> > I also prefer the dated jars approach.
>
> How do I get maven 1 to change the name of the jar to include the timestamp?
>
> Phil
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Phil Steitz <ph...@gmail.com>.
On 7/6/06, Dion Gillard <di...@gmail.com> wrote:
> I also prefer the dated jars approach.
How do I get maven 1 to change the name of the jar to include the timestamp?
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Dion Gillard <di...@gmail.com>.
I also prefer the dated jars approach.
On 7/7/06, Phil Steitz <ph...@gmail.com> wrote:
> On 7/6/06, Phil Steitz <ph...@gmail.com> wrote:
> > On 7/6/06, Brett Porter <br...@apache.org> wrote:
> > > On 7/07/2006 2:27 AM, Phil Steitz wrote:
> > > > 2) Need to decide whether we want dated snaps or just overwriting. If
> > > > the latter, we would need a way to purge old stuff (like the crontab
> > > > that runs on Minotaur that purges out the old nightly distros).
> > >
> > > I generally prefer dated (just in case something is broken with today's,
> > > you can try yesterdays). However, it does give you the obligation to
> > > clean up afterwards (not necessarily a bad thing).
> > >
> > > Neither work in Maven 2 if you are downloading from a Maven 1 repo at
> > > the moment (dated because there was never any facility for that,
> > > SNAPSHOT because of the bug Jorg mentioned).
> > >
> > > >
> > > > 3) Ensure that the deploy target hosts are set correctly in all of the
> > > > POMs - maybe modify the script to check to avoid accidental
> > > > deployments to java-repository.
> > >
> > > +1, and also check that currentVersion endsWith -SNAPSHOT.
> > >
> > > >
> > > > 4) Someone needs to remind me (or just patch the script) of the
> > > > correct maven 1 target to execute. (jar:snapshot-deploy or somesuch).
> > >
> > > With the current plugins installed, jar:deploy will do a dated snapshot
> > > by default I think, based on the end of the version.
> > >
> > > >
> > > > 5) I either need to get over reservations about signing myself, or be
> > > > OK with no sigs. Is it OK to deploy unsigned stuff to the snapshot
> > > > repo? The same actually applies to the zips, tars being generated by
> > > > the script now.
> > >
> > > As Craig said, no need to sign them.
> > >
> > > It's a false sense of security when they aren't any more secure (anyone
> > > that can compromise the machine can compromise the key being used to
> > > sign them if it is automated).
> >
> > So hashing (which maven does) is as much as we can really practically
> > do automated.
> >
> > Looks also like we have consensus on the "snapshot" semantics - i.e.,
> > just overwrite with a new snap each night. If there are no objections
> > in the next couple of days, I will go ahead and make the change to
> > support this, or someone else can.
>
> Sorry, I missed Brett's stated preference for dated jars, which I
> actually share. Is this possible using maven 1 somehow? I vaguely
> remember doing this long ago and m1 being smart enough to grab the
> latest jar when dated snaps were deployed. We could set up a cron to
> clean up the repo, like we do the nightlies.
>
> Phil
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
>
--
http://www.multitask.com.au/people/dion/
"If you even dream of beating me you'd better wake up and apologize" -
Muhammad Ali
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Brett Porter <br...@apache.org>.
On 7/07/2006 4:09 PM, Phil Steitz wrote:
> Sorry, I missed Brett's stated preference for dated jars, which I
> actually share. Is this possible using maven 1 somehow? I vaguely
> remember doing this long ago and m1 being smart enough to grab the
> latest jar when dated snaps were deployed. We could set up a cron to
> clean up the repo, like we do the nightlies.
It certainly could deploy it through jar:deploy (previously
jar:deploy-snapshot was required).
I don't think Maven 1.x ever got smart enough to grab this automatically
(but I may be misremembering).
This isn't necessarily such a bad thing, allowing folks to always pick
the version they want.
- Brett
--
Apache Maven - http://maven.apache.org/
Better Builds with Maven - http://library.mergere.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Phil Steitz <ph...@gmail.com>.
On 7/6/06, Phil Steitz <ph...@gmail.com> wrote:
> On 7/6/06, Brett Porter <br...@apache.org> wrote:
> > On 7/07/2006 2:27 AM, Phil Steitz wrote:
> > > 2) Need to decide whether we want dated snaps or just overwriting. If
> > > the latter, we would need a way to purge old stuff (like the crontab
> > > that runs on Minotaur that purges out the old nightly distros).
> >
> > I generally prefer dated (just in case something is broken with today's,
> > you can try yesterdays). However, it does give you the obligation to
> > clean up afterwards (not necessarily a bad thing).
> >
> > Neither work in Maven 2 if you are downloading from a Maven 1 repo at
> > the moment (dated because there was never any facility for that,
> > SNAPSHOT because of the bug Jorg mentioned).
> >
> > >
> > > 3) Ensure that the deploy target hosts are set correctly in all of the
> > > POMs - maybe modify the script to check to avoid accidental
> > > deployments to java-repository.
> >
> > +1, and also check that currentVersion endsWith -SNAPSHOT.
> >
> > >
> > > 4) Someone needs to remind me (or just patch the script) of the
> > > correct maven 1 target to execute. (jar:snapshot-deploy or somesuch).
> >
> > With the current plugins installed, jar:deploy will do a dated snapshot
> > by default I think, based on the end of the version.
> >
> > >
> > > 5) I either need to get over reservations about signing myself, or be
> > > OK with no sigs. Is it OK to deploy unsigned stuff to the snapshot
> > > repo? The same actually applies to the zips, tars being generated by
> > > the script now.
> >
> > As Craig said, no need to sign them.
> >
> > It's a false sense of security when they aren't any more secure (anyone
> > that can compromise the machine can compromise the key being used to
> > sign them if it is automated).
>
> So hashing (which maven does) is as much as we can really practically
> do automated.
>
> Looks also like we have consensus on the "snapshot" semantics - i.e.,
> just overwrite with a new snap each night. If there are no objections
> in the next couple of days, I will go ahead and make the change to
> support this, or someone else can.
Sorry, I missed Brett's stated preference for dated jars, which I
actually share. Is this possible using maven 1 somehow? I vaguely
remember doing this long ago and m1 being smart enough to grab the
latest jar when dated snaps were deployed. We could set up a cron to
clean up the repo, like we do the nightlies.
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Craig McClanahan <cr...@apache.org>.
On 7/6/06, Phil Steitz <ph...@gmail.com> wrote:
>
> On 7/6/06, Brett Porter <br...@apache.org> wrote:
> > On 7/07/2006 2:11 PM, Phil Steitz wrote:
> > > So hashing (which maven does) is as much as we can really practically
> > > do automated.
> >
> > Yep (and this is only for transport integrity, not the authenticity of
> > the source).
> >
> > > # Repository to deploy snapshots
> > > maven.repo.apache.snapshots=scp://cvs.apache.org
>
> Just noticed I forgot to change this. This should be people, right?
Yes, and it should be changed in any POMs that reference it directly, too.
Otherwise, people who try to look up dependencies at cvs.apache.org are
going to have problems.
Phil
Craig
---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
>
Re: [all] Nightlies are back...mostly
Posted by Phil Steitz <ph...@gmail.com>.
On 7/6/06, Brett Porter <br...@apache.org> wrote:
> On 7/07/2006 2:11 PM, Phil Steitz wrote:
> > So hashing (which maven does) is as much as we can really practically
> > do automated.
>
> Yep (and this is only for transport integrity, not the authenticity of
> the source).
>
> > # Repository to deploy snapshots
> > maven.repo.apache.snapshots=scp://cvs.apache.org
Just noticed I forgot to change this. This should be people, right?
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Brett Porter <br...@apache.org>.
On 7/07/2006 2:11 PM, Phil Steitz wrote:
> So hashing (which maven does) is as much as we can really practically
> do automated.
Yep (and this is only for transport integrity, not the authenticity of
the source).
> # Repository to deploy snapshots
> maven.repo.apache.snapshots=scp://cvs.apache.org
> maven.repo.apache.snapshots.directory=/www/people.apache.org/repository
>
> I assume I can also specify these with -D on the m1 command line?
Looks right, might also need a private key and username if the defaults
aren't what you wanted.
>
> Also, the target is "jar:deploy" right (assuming all have -SNAPSHOT in
> trunk version)?
Yes.
> This last bit raises an issue that I had to hack around for [scxml]
> and that we might want to consider standardizing - all trunk POMs have
> -SNAPSHOT version names. This means that RCs *must* be built from
> branches. Personally, I think that is a good idea.
Pretty much. The release plugin actually puts the release rev# on trunk,
but immediately bumps it up to the next snapshot and commits again. Both
are good alternatives as long as trunk stays as a snapshot when it has
the possibility to change.
- Brett
--
Apache Maven - http://maven.apache.org/
Better Builds with Maven - http://library.mergere.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Phil Steitz <ph...@gmail.com>.
On 7/6/06, Brett Porter <br...@apache.org> wrote:
> On 7/07/2006 2:27 AM, Phil Steitz wrote:
> > 2) Need to decide whether we want dated snaps or just overwriting. If
> > the latter, we would need a way to purge old stuff (like the crontab
> > that runs on Minotaur that purges out the old nightly distros).
>
> I generally prefer dated (just in case something is broken with today's,
> you can try yesterdays). However, it does give you the obligation to
> clean up afterwards (not necessarily a bad thing).
>
> Neither work in Maven 2 if you are downloading from a Maven 1 repo at
> the moment (dated because there was never any facility for that,
> SNAPSHOT because of the bug Jorg mentioned).
>
> >
> > 3) Ensure that the deploy target hosts are set correctly in all of the
> > POMs - maybe modify the script to check to avoid accidental
> > deployments to java-repository.
>
> +1, and also check that currentVersion endsWith -SNAPSHOT.
>
> >
> > 4) Someone needs to remind me (or just patch the script) of the
> > correct maven 1 target to execute. (jar:snapshot-deploy or somesuch).
>
> With the current plugins installed, jar:deploy will do a dated snapshot
> by default I think, based on the end of the version.
>
> >
> > 5) I either need to get over reservations about signing myself, or be
> > OK with no sigs. Is it OK to deploy unsigned stuff to the snapshot
> > repo? The same actually applies to the zips, tars being generated by
> > the script now.
>
> As Craig said, no need to sign them.
>
> It's a false sense of security when they aren't any more secure (anyone
> that can compromise the machine can compromise the key being used to
> sign them if it is automated).
So hashing (which maven does) is as much as we can really practically
do automated.
Looks also like we have consensus on the "snapshot" semantics - i.e.,
just overwrite with a new snap each night. If there are no objections
in the next couple of days, I will go ahead and make the change to
support this, or someone else can.
I assume the following is correct for maven 1:
maven.repo.list=apache.snapshots
# Repository to deploy snapshots
maven.repo.apache.snapshots=scp://cvs.apache.org
maven.repo.apache.snapshots.directory=/www/people.apache.org/repository
I assume I can also specify these with -D on the m1 command line?
Also, the target is "jar:deploy" right (assuming all have -SNAPSHOT in
trunk version)?
This last bit raises an issue that I had to hack around for [scxml]
and that we might want to consider standardizing - all trunk POMs have
-SNAPSHOT version names. This means that RCs *must* be built from
branches. Personally, I think that is a good idea.
Thanks!
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Brett Porter <br...@apache.org>.
On 7/07/2006 2:27 AM, Phil Steitz wrote:
> 2) Need to decide whether we want dated snaps or just overwriting. If
> the latter, we would need a way to purge old stuff (like the crontab
> that runs on Minotaur that purges out the old nightly distros).
I generally prefer dated (just in case something is broken with today's,
you can try yesterdays). However, it does give you the obligation to
clean up afterwards (not necessarily a bad thing).
Neither work in Maven 2 if you are downloading from a Maven 1 repo at
the moment (dated because there was never any facility for that,
SNAPSHOT because of the bug Jorg mentioned).
>
> 3) Ensure that the deploy target hosts are set correctly in all of the
> POMs - maybe modify the script to check to avoid accidental
> deployments to java-repository.
+1, and also check that currentVersion endsWith -SNAPSHOT.
>
> 4) Someone needs to remind me (or just patch the script) of the
> correct maven 1 target to execute. (jar:snapshot-deploy or somesuch).
With the current plugins installed, jar:deploy will do a dated snapshot
by default I think, based on the end of the version.
>
> 5) I either need to get over reservations about signing myself, or be
> OK with no sigs. Is it OK to deploy unsigned stuff to the snapshot
> repo? The same actually applies to the zips, tars being generated by
> the script now.
As Craig said, no need to sign them.
It's a false sense of security when they aren't any more secure (anyone
that can compromise the machine can compromise the key being used to
sign them if it is automated).
- Brett
--
Apache Maven - http://maven.apache.org/
Better Builds with Maven - http://library.mergere.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Phil Steitz <ph...@gmail.com>.
On 7/6/06, Craig McClanahan <cr...@apache.org> wrote:
> On 7/6/06, Phil Steitz <ph...@gmail.com> wrote:
> >
> > On 7/6/06, Henri Yandell <fl...@gmail.com> wrote:
> > > On 7/6/06, Phil Steitz <ph...@gmail.com> wrote:
> > > > On 7/5/06, Henri Yandell <fl...@gmail.com> wrote:
> > > > > Have added csv. How do I go about hooking it up, doesn't look as
> > > > > though it worked last night?
> > > > >
> > > >
> > > > The script was not creating directories on the deploy host for new
> > > > components, so it failed the first time. I fixed that and it ran
> > > > successfully last night.
> > >
> > > Thanks Phil.
> > >
> > > Any thoughts on having the Maven ones do a snapshot deploy to the
> > > snapshot repository?
> > >
> > Would be easy to do, but before doing it, we need
> >
> > 1) Verify that this is OK from repository@ standpoint. I have sort of
> > lost the thread on that list and don't want to do anything
> > inconsistent / illegal. Probably should ask there and get agreement,
> > unless I am just out of the loop and others are already doing this.
>
>
> When we (Struts and Shale) asked, we got back basically "that's what it's
> for" :-). We deploy snapshots to <
> http://people.apache.org/maven-snapshot-repository>.
>
> 2) Need to decide whether we want dated snaps or just overwriting. If
> > the latter, we would need a way to purge old stuff (like the crontab
> > that runs on Minotaur that purges out the old nightly distros).
>
>
> For uploaded artifacts, I historically modified the tar.gz and zip file
> names to include the date. I have the following types of commands in my
> crontab on people.apache.org to clean up afterwards (all on one line):
>
> 03 04 * * * find /www/cvs.apache.org/builds/jakarta-commons/nightly -mtime
> +7 -exec rm \{\} \; >/dev/null 2>&1
>
> Which reminds me ... I need to clean out the now obsolete lines like this.
>
OK, so now I have to add it to mine ;-)
>
> 3) Ensure that the deploy target hosts are set correctly in all of the
> > POMs - maybe modify the script to check to avoid accidental
> > deployments to java-repository.
>
>
> Do the commons POMs all inherit from a master? That's the easiest way to
> record things like repositories in one place, although it requires a release
> process for the master POM when it gets updated.
No, we eliminated the POM inheritence a while back because it made it
impossible to build the projects by themselves. Should be able to
force this on command line in any case, so this is not really a
problem.
>
> 4) Someone needs to remind me (or just patch the script) of the
> > correct maven 1 target to execute. (jar:snapshot-deploy or somesuch).
>
>
> Maven 1? For m2 I do "mvn install deploy" on each module, to install it
> locally and deploy it to the snapshot directory (the latter works that way
> automatically because the version numbers are something like "1.0.3-SNAPSHOT
> ").
Currently, almost all commons components are still using Maven 1.
>
> 5) I either need to get over reservations about signing myself, or be
> > OK with no sigs. Is it OK to deploy unsigned stuff to the snapshot
> > repo? The same actually applies to the zips, tars being generated by
> > the script now.
>
>
> Nightly builds and snapshots are, IMHO, use at your own risk and don't need
> signing.
>
> Phil
>
>
> Craig
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Craig McClanahan <cr...@apache.org>.
On 7/6/06, Phil Steitz <ph...@gmail.com> wrote:
>
> On 7/6/06, Henri Yandell <fl...@gmail.com> wrote:
> > On 7/6/06, Phil Steitz <ph...@gmail.com> wrote:
> > > On 7/5/06, Henri Yandell <fl...@gmail.com> wrote:
> > > > Have added csv. How do I go about hooking it up, doesn't look as
> > > > though it worked last night?
> > > >
> > >
> > > The script was not creating directories on the deploy host for new
> > > components, so it failed the first time. I fixed that and it ran
> > > successfully last night.
> >
> > Thanks Phil.
> >
> > Any thoughts on having the Maven ones do a snapshot deploy to the
> > snapshot repository?
> >
> Would be easy to do, but before doing it, we need
>
> 1) Verify that this is OK from repository@ standpoint. I have sort of
> lost the thread on that list and don't want to do anything
> inconsistent / illegal. Probably should ask there and get agreement,
> unless I am just out of the loop and others are already doing this.
When we (Struts and Shale) asked, we got back basically "that's what it's
for" :-). We deploy snapshots to <
http://people.apache.org/maven-snapshot-repository>.
2) Need to decide whether we want dated snaps or just overwriting. If
> the latter, we would need a way to purge old stuff (like the crontab
> that runs on Minotaur that purges out the old nightly distros).
For uploaded artifacts, I historically modified the tar.gz and zip file
names to include the date. I have the following types of commands in my
crontab on people.apache.org to clean up afterwards (all on one line):
03 04 * * * find /www/cvs.apache.org/builds/jakarta-commons/nightly -mtime
+7 -exec rm \{\} \; >/dev/null 2>&1
Which reminds me ... I need to clean out the now obsolete lines like this.
3) Ensure that the deploy target hosts are set correctly in all of the
> POMs - maybe modify the script to check to avoid accidental
> deployments to java-repository.
Do the commons POMs all inherit from a master? That's the easiest way to
record things like repositories in one place, although it requires a release
process for the master POM when it gets updated.
4) Someone needs to remind me (or just patch the script) of the
> correct maven 1 target to execute. (jar:snapshot-deploy or somesuch).
Maven 1? For m2 I do "mvn install deploy" on each module, to install it
locally and deploy it to the snapshot directory (the latter works that way
automatically because the version numbers are something like "1.0.3-SNAPSHOT
").
5) I either need to get over reservations about signing myself, or be
> OK with no sigs. Is it OK to deploy unsigned stuff to the snapshot
> repo? The same actually applies to the zips, tars being generated by
> the script now.
Nightly builds and snapshots are, IMHO, use at your own risk and don't need
signing.
Phil
Craig
Re: [all] Nightlies are back...mostly
Posted by Brett Porter <br...@apache.org>.
On 7/07/2006 5:05 AM, Jörg Schaible wrote:
> Does currently not work. A SNAPSHOT is *never* downloaded, if there is
> already sitting one in the local repo - which is true after the first
> fetch ...
>
> MNG-1908 ... vote for it !
>
They're talking about Maven 1, where it does work.
- Brett
--
Apache Maven - http://maven.apache.org/
Better Builds with Maven - http://library.mergere.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Jörg Schaible <jo...@gmx.de>.
Henri Yandell wrote:
> On 7/6/06, Phil Steitz <ph...@gmail.com> wrote:
>> On 7/6/06, Henri Yandell <fl...@gmail.com> wrote:
>> > On 7/6/06, Phil Steitz <ph...@gmail.com> wrote:
>> > > On 7/5/06, Henri Yandell <fl...@gmail.com> wrote:
>> > > > Have added csv. How do I go about hooking it up, doesn't look as
>> > > > though it worked last night?
>> > > >
>> > >
>> > > The script was not creating directories on the deploy host for new
>> > > components, so it failed the first time. I fixed that and it ran
>> > > successfully last night.
>> >
>> > Thanks Phil.
>> >
>> > Any thoughts on having the Maven ones do a snapshot deploy to the
>> > snapshot repository?
>> >
>> Would be easy to do, but before doing it, we need
>
>> 2) Need to decide whether we want dated snaps or just overwriting. If
>> the latter, we would need a way to purge old stuff (like the crontab
>> that runs on Minotaur that purges out the old nightly distros).
>
> Overwriting I think, else the repository is going to explode in size.
Does currently not work. A SNAPSHOT is *never* downloaded, if there is
already sitting one in the local repo - which is true after the first
fetch ...
MNG-1908 ... vote for it !
>
> Otherwise - agreement with Craig's email.
>
> Hen
- Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Henri Yandell <fl...@gmail.com>.
On 7/6/06, Phil Steitz <ph...@gmail.com> wrote:
> On 7/6/06, Henri Yandell <fl...@gmail.com> wrote:
> > On 7/6/06, Phil Steitz <ph...@gmail.com> wrote:
> > > On 7/5/06, Henri Yandell <fl...@gmail.com> wrote:
> > > > Have added csv. How do I go about hooking it up, doesn't look as
> > > > though it worked last night?
> > > >
> > >
> > > The script was not creating directories on the deploy host for new
> > > components, so it failed the first time. I fixed that and it ran
> > > successfully last night.
> >
> > Thanks Phil.
> >
> > Any thoughts on having the Maven ones do a snapshot deploy to the
> > snapshot repository?
> >
> Would be easy to do, but before doing it, we need
> 2) Need to decide whether we want dated snaps or just overwriting. If
> the latter, we would need a way to purge old stuff (like the crontab
> that runs on Minotaur that purges out the old nightly distros).
Overwriting I think, else the repository is going to explode in size.
Otherwise - agreement with Craig's email.
Hen
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Phil Steitz <ph...@gmail.com>.
On 7/6/06, Henri Yandell <fl...@gmail.com> wrote:
> On 7/6/06, Phil Steitz <ph...@gmail.com> wrote:
> > On 7/5/06, Henri Yandell <fl...@gmail.com> wrote:
> > > Have added csv. How do I go about hooking it up, doesn't look as
> > > though it worked last night?
> > >
> >
> > The script was not creating directories on the deploy host for new
> > components, so it failed the first time. I fixed that and it ran
> > successfully last night.
>
> Thanks Phil.
>
> Any thoughts on having the Maven ones do a snapshot deploy to the
> snapshot repository?
>
Would be easy to do, but before doing it, we need
1) Verify that this is OK from repository@ standpoint. I have sort of
lost the thread on that list and don't want to do anything
inconsistent / illegal. Probably should ask there and get agreement,
unless I am just out of the loop and others are already doing this.
2) Need to decide whether we want dated snaps or just overwriting. If
the latter, we would need a way to purge old stuff (like the crontab
that runs on Minotaur that purges out the old nightly distros).
3) Ensure that the deploy target hosts are set correctly in all of the
POMs - maybe modify the script to check to avoid accidental
deployments to java-repository.
4) Someone needs to remind me (or just patch the script) of the
correct maven 1 target to execute. (jar:snapshot-deploy or somesuch).
5) I either need to get over reservations about signing myself, or be
OK with no sigs. Is it OK to deploy unsigned stuff to the snapshot
repo? The same actually applies to the zips, tars being generated by
the script now.
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Henri Yandell <fl...@gmail.com>.
On 7/6/06, Phil Steitz <ph...@gmail.com> wrote:
> On 7/5/06, Henri Yandell <fl...@gmail.com> wrote:
> > Have added csv. How do I go about hooking it up, doesn't look as
> > though it worked last night?
> >
>
> The script was not creating directories on the deploy host for new
> components, so it failed the first time. I fixed that and it ran
> successfully last night.
Thanks Phil.
Any thoughts on having the Maven ones do a snapshot deploy to the
snapshot repository?
Hen
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Phil Steitz <ph...@gmail.com>.
On 7/5/06, Henri Yandell <fl...@gmail.com> wrote:
> Have added csv. How do I go about hooking it up, doesn't look as
> though it worked last night?
>
The script was not creating directories on the deploy host for new
components, so it failed the first time. I fixed that and it ran
successfully last night.
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Henri Yandell <fl...@gmail.com>.
Have added csv. How do I go about hooking it up, doesn't look as
though it worked last night?
Hen
On 7/3/06, Phil Steitz <ph...@gmail.com> wrote:
> Nightlies are back up and running for most components. I still think
> maven2 + continuum is the way to go long term, but to get the
> nightlies back up, I hacked together a little bash script based on
> Craig's and set it up to run as a crontab on vmbuild.apache.org.
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Brett Porter <br...@apache.org>.
If the fix you put in doesn't work, the -Djava.awt.headless=true
property might work.
- Brett
On 7/07/2006 6:18 AM, Phil Steitz wrote:
> On 7/6/06, Oliver Heger <ol...@oliver-heger.de> wrote:
>> Phil,
>>
>> thank you very much for your work! Finally I can see, which test fails
>> for [configuration].
>>
>> Hm, it's a test dealing with Applets. I guess, you run the tests in a
>> non GUI environment, where creating an Applet is not allowed? I modified
>> the test and hope that the problem is now solved.
>
> Yes, the script just runs maven from the command line under Ubuntu
> Linux, JDK 1.5. Thanks for looking into this.
>
> Phil
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
--
Apache Maven - http://maven.apache.org/
Better Builds with Maven - http://library.mergere.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Phil Steitz <ph...@gmail.com>.
On 7/6/06, Oliver Heger <ol...@oliver-heger.de> wrote:
> Phil,
>
> thank you very much for your work! Finally I can see, which test fails
> for [configuration].
>
> Hm, it's a test dealing with Applets. I guess, you run the tests in a
> non GUI environment, where creating an Applet is not allowed? I modified
> the test and hope that the problem is now solved.
Yes, the script just runs maven from the command line under Ubuntu
Linux, JDK 1.5. Thanks for looking into this.
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [all] Nightlies are back...mostly
Posted by Oliver Heger <ol...@oliver-heger.de>.
Phil,
thank you very much for your work! Finally I can see, which test fails
for [configuration].
Hm, it's a test dealing with Applets. I guess, you run the tests in a
non GUI environment, where creating an Applet is not allowed? I modified
the test and hope that the problem is now solved.
Oliver
Phil Steitz wrote:
> Nightlies are back up and running for most components. I still think
> maven2 + continuum is the way to go long term, but to get the
> nightlies back up, I hacked together a little bash script based on
> Craig's and set it up to run as a crontab on vmbuild.apache.org.
>
> It would be great if people could have a look at the nightlies
> themselves as well as the script, which I checked into commons-build
> as "commons_nightly.sh." I have no shame regarding my limited bash
> (or Java, actually ;-) skills, so feel free to make suggestions for
> improvement or submit patches. Be careful checking in the script,
> though, as it is executed via a crontab wrapper that svn ups it before
> executing it on vmbuild.apache.org. You can test it locally by
> changing the config to make your local host the deploy host. You have
> to gen an ssh key (if you don't already have one) and trust it locally
> to do this.
>
> The script reads lists of components from commons-build txt files like
> "nightly_sandbox_maven_list.txt" and executes maven clean dist for the
> ones in the maven lists and ant clean dist for those in the ant lists
> (after svn up). I split them up based on whether or not I could get
> "maven clean dist" to work. The ant lists consist of components for
> which the maven build did not work, but the ant build did. I have left
> a few things in the maven list even though they fail, because the
> reason seems trivial (e.g. clover license, or missing sun jars - I
> will fix the second). It svn ups the lists before executing, so if
> you want to move components from ant -> maven, or vice-versa, just
> make the changes to the files and check them in. Remember to delete
> as well as add; otherwise the script will run both.
>
> Logs for each component build are for the moment being written out to
> http://people.apache.org/~psteitz/commons-nightlies/
> under dated directories. I didn't want to clutter the normal nightly
> location with logs, but that could be changed if we like having them
> there.
>
> I ommitted dormant components. If anyone feels strongly that we
> should be doing nightlies for any of these, the script can be patched
> to do that.
>
> There are no sigs or hashes. Another easy patch, but since this all
> runs as me, I don't want to sign. Adding hashes is no problem. Some
> of the maven builds do this already. Best would probably be to have
> the maven builds all do it and the nightly script just copy out. I am
> OK either way, but the script needs to be patched in either case to
> scp the hashes.
>
> More warts:
>
> 1) The files are not group writable on the deploy host
> (people.apache.org). I set the umask in the script, but somehow the
> permissions are not getting set right. Any pointers / patches on how
> to get the wrapper or script itself to do this correctly would be
> appreciated.
>
> 2) Ubuntu on vmbuild seems to have no zip command, so the ant builds
> are not getting zips generated. I guess I could break down and fork
> Ant again to do the zips, but maybe there is an easier way?
>
> 3) While it would be easy to get jar:deploy done from within the maven
> loop, that seems like running with scissors. Also, what would be
> deployed would be a SNAPSHOT that just over-wrote the previous (I
> think). This exposes another wart - though the names are munged to be
> dated, the internally bundled jars are not. For anyone wanting to
> play with this, be forwarned that Ubuntu rename is picky - you need to
> use the perl regexp form like the script does. Could be there is a
> simple maven way to get this right.
>
> Thanks in advance for feedback / patches / review of the output and
> help getting the failing builds restored.
>
> Phil
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org