You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Stephen Connolly <st...@gmail.com> on 2016/05/04 19:20:01 UTC
An idea I had for one way of doing Continuous Delivery with Maven
Here's an embryonic idea I had for doing Continuous Delivery with Maven
https://www.cloudbees.com/blog/new-way-do-continuous-delivery-maven-and-jenkins-pipeline
Probably needs a little fleshing out.
-Stephen
Re: An idea I had for one way of doing Continuous Delivery with Maven
Posted by Ben Podgursky <bp...@gmail.com>.
Of course, we tag the MANIFEST of all jars with the git commit. We
actually go a bit farther and print out all dependency git SHA1s during the
build process, so you only have to refer to the jenkins build log to do a
bisect and not dig through jars.
I describe how we did that here:
http://liveramp.com/engineering/identifying-maven-snapshot-artifacts-by-git-revision/
On Thu, May 5, 2016 at 6:32 PM, Barrie Treloar <ba...@gmail.com> wrote:
> On 6 May 2016 at 09:48, Ben Podgursky <bp...@gmail.com> wrote:
>
> > I see these discussions often, and I wanted to jump in and mention how we
> > handle continuous deploys, because I feel that it avoids many of the
> > downsides mentioned here, albeit with some (IMO) minor costs:
> >
> > - all of our internal dependencies are SNAPSHOT. we do not do releases,
> > ever, or ever depend on a fixed version
> >
> > - we attach to the package phase of our builds an assembly which packages
> > the artifact and all of the dependencies it tested against into a tar.gz
> >
> > - we deploy the tar.gz to our artifact server with a snapshot version
> (ex,
> > MY_PROJECT-1.0-deploy.tar.gz)
> >
> > - when deploying we simply take the last snapshot deploy artifact and
> unzip
> > it in production (+ some config files etc).
> >
> > so while you use SNAPSHOT versions, you are guaranteed that you are
> > deploying against tested dependencies. and you never have to commit
> > updated dependency versions, do releases, etc. the only real cost is a
> bit
> > of artifact server disk space… but disk space is incredibly cheap.
>
>
> You left out tagging the source code in the built jars.
>
> How are you supposed to pull up the code in production?
> Or git bisect, or diff between versions to track down when a defect was
> introduced?
>
Re: An idea I had for one way of doing Continuous Delivery with Maven
Posted by Barrie Treloar <ba...@gmail.com>.
On 6 May 2016 at 09:48, Ben Podgursky <bp...@gmail.com> wrote:
> I see these discussions often, and I wanted to jump in and mention how we
> handle continuous deploys, because I feel that it avoids many of the
> downsides mentioned here, albeit with some (IMO) minor costs:
>
> - all of our internal dependencies are SNAPSHOT. we do not do releases,
> ever, or ever depend on a fixed version
>
> - we attach to the package phase of our builds an assembly which packages
> the artifact and all of the dependencies it tested against into a tar.gz
>
> - we deploy the tar.gz to our artifact server with a snapshot version (ex,
> MY_PROJECT-1.0-deploy.tar.gz)
>
> - when deploying we simply take the last snapshot deploy artifact and unzip
> it in production (+ some config files etc).
>
> so while you use SNAPSHOT versions, you are guaranteed that you are
> deploying against tested dependencies. and you never have to commit
> updated dependency versions, do releases, etc. the only real cost is a bit
> of artifact server disk space… but disk space is incredibly cheap.
You left out tagging the source code in the built jars.
How are you supposed to pull up the code in production?
Or git bisect, or diff between versions to track down when a defect was
introduced?
Re: An idea I had for one way of doing Continuous Delivery with Maven
Posted by Ben Podgursky <bp...@gmail.com>.
sorry, meant " MY_PROJECT-1.0-SNAPSHOT-deploy.tar.gz"
On Thu, May 5, 2016 at 5:18 PM, Ben Podgursky <bp...@gmail.com> wrote:
> I see these discussions often, and I wanted to jump in and mention how we
> handle continuous deploys, because I feel that it avoids many of the
> downsides mentioned here, albeit with some (IMO) minor costs:
>
> - all of our internal dependencies are SNAPSHOT. we do not do releases,
> ever, or ever depend on a fixed version
>
> - we attach to the package phase of our builds an assembly which packages
> the artifact and all of the dependencies it tested against into a tar.gz
>
> - we deploy the tar.gz to our artifact server with a snapshot version (ex,
> MY_PROJECT-1.0-deploy.tar.gz)
>
> - when deploying we simply take the last snapshot deploy artifact and
> unzip it in production (+ some config files etc).
>
> so while you use SNAPSHOT versions, you are guaranteed that you are
> deploying against tested dependencies. and you never have to commit
> updated dependency versions, do releases, etc. the only real cost is a bit
> of artifact server disk space… but disk space is incredibly cheap.
>
> On Thu, May 5, 2016 at 2:13 AM, Stephen Connolly <
> stephen.alan.connolly@gmail.com> wrote:
>
>> On 5 May 2016 at 10:00, Robert Scholte <rf...@apache.org> wrote:
>>
>> > Hi Stephen, nice blog.
>> >
>> > There are a few things I'd like to share:
>> > - The maven-release-plugin does several checks which ensures that you
>> have
>> > a reproducible artifact, where checking on SNAPSHOT dependencies is one
>> of
>> > the most important ones. (you can find a lot patterns based on the
>> > versions-maven-plugin on the web, but they forget these checks)
>> >
>>
>> This is one of the reasons why my suggestion is to actually cut a release
>> for every commit (just only push the tag - and just the tag - if the
>> release goes anywhere)
>>
>>
>> > - I still have this wish to have support a couple of strategies within
>> the
>> > maven-release-plugin, this could be one as well.
>> >
>>
>> Yes, I think it would be a valuable one to have in the default plugin, for
>> sure... and it may need some assistance from the scm plugin too
>>
>>
>> >
>> > In a continuous environment is becomes more and more important to be
>> able
>> > to reproduce issues, which can be hard when things keep changing. This
>> > might even work with SNAPSHOT dependencies if your can get the
>> > distributable (either the deployable or executable artifact, assuming
>> they
>> > contain all the required dependencies) and have a testing setup which
>> can
>> > test with this artifact. If you can reproduce it here, you can
>> (re)write it
>> > as an appropriate test and fix it.
>> > So next to all the current tests you need to do regression tests on the
>> > distributable as well.
>> > As long as this is not possible or secured in your pipeline, you should
>> > NEVER EVER want to depend on SNAPSHOT-dependencies.
>> >
>>
>> +1
>>
>> One that we gain with the approach I suggest is that we can actually use
>> versions:resolve-ranges as we know that the pom will not be pushed back to
>> master.
>>
>> Thus developers can keep version ranges in the pom and the CI build will
>> resolve the ranges for each "release"
>>
>> I'd like to find a way to allow resolve-ranges to work with the current
>> release plugin flow... my working idea has been to inject e.g. XML PI
>> entities into the pom so that we know what range to restore, e.g.
>> pre-release pom has
>>
>> <dependency>
>> ...
>> <version>[1.2,2.0)</version>
>> </dependency>
>>
>> which would get resolved into the release pom as
>>
>> <dependency>
>> ...
>> <version>1.5<?versions-maven-plugin original="[1.2,2.0)"?></version>
>> </dependency>
>>
>> so that when we want to un-pin we can then see the original range and
>> up-lift it so that the development pom becomes
>>
>> <dependency>
>> ...
>> <version>[1.5,2.0)</version>
>> </dependency>
>>
>>
>>
>> >
>> > thanks,
>> > Robert
>> >
>> >
>> > On Wed, 04 May 2016 21:20:01 +0200, Stephen Connolly <
>> > stephen.alan.connolly@gmail.com> wrote:
>> >
>> > Here's an embryonic idea I had for doing Continuous Delivery with Maven
>> >>
>> >>
>> >>
>> https://www.cloudbees.com/blog/new-way-do-continuous-delivery-maven-and-jenkins-pipeline
>> >>
>> >> Probably needs a little fleshing out.
>> >>
>> >> -Stephen
>> >>
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> > For additional commands, e-mail: users-help@maven.apache.org
>> >
>> >
>>
>
>
Re: An idea I had for one way of doing Continuous Delivery with Maven
Posted by Ben Podgursky <bp...@gmail.com>.
I see these discussions often, and I wanted to jump in and mention how we
handle continuous deploys, because I feel that it avoids many of the
downsides mentioned here, albeit with some (IMO) minor costs:
- all of our internal dependencies are SNAPSHOT. we do not do releases,
ever, or ever depend on a fixed version
- we attach to the package phase of our builds an assembly which packages
the artifact and all of the dependencies it tested against into a tar.gz
- we deploy the tar.gz to our artifact server with a snapshot version (ex,
MY_PROJECT-1.0-deploy.tar.gz)
- when deploying we simply take the last snapshot deploy artifact and unzip
it in production (+ some config files etc).
so while you use SNAPSHOT versions, you are guaranteed that you are
deploying against tested dependencies. and you never have to commit
updated dependency versions, do releases, etc. the only real cost is a bit
of artifact server disk space… but disk space is incredibly cheap.
On Thu, May 5, 2016 at 2:13 AM, Stephen Connolly <
stephen.alan.connolly@gmail.com> wrote:
> On 5 May 2016 at 10:00, Robert Scholte <rf...@apache.org> wrote:
>
> > Hi Stephen, nice blog.
> >
> > There are a few things I'd like to share:
> > - The maven-release-plugin does several checks which ensures that you
> have
> > a reproducible artifact, where checking on SNAPSHOT dependencies is one
> of
> > the most important ones. (you can find a lot patterns based on the
> > versions-maven-plugin on the web, but they forget these checks)
> >
>
> This is one of the reasons why my suggestion is to actually cut a release
> for every commit (just only push the tag - and just the tag - if the
> release goes anywhere)
>
>
> > - I still have this wish to have support a couple of strategies within
> the
> > maven-release-plugin, this could be one as well.
> >
>
> Yes, I think it would be a valuable one to have in the default plugin, for
> sure... and it may need some assistance from the scm plugin too
>
>
> >
> > In a continuous environment is becomes more and more important to be able
> > to reproduce issues, which can be hard when things keep changing. This
> > might even work with SNAPSHOT dependencies if your can get the
> > distributable (either the deployable or executable artifact, assuming
> they
> > contain all the required dependencies) and have a testing setup which can
> > test with this artifact. If you can reproduce it here, you can (re)write
> it
> > as an appropriate test and fix it.
> > So next to all the current tests you need to do regression tests on the
> > distributable as well.
> > As long as this is not possible or secured in your pipeline, you should
> > NEVER EVER want to depend on SNAPSHOT-dependencies.
> >
>
> +1
>
> One that we gain with the approach I suggest is that we can actually use
> versions:resolve-ranges as we know that the pom will not be pushed back to
> master.
>
> Thus developers can keep version ranges in the pom and the CI build will
> resolve the ranges for each "release"
>
> I'd like to find a way to allow resolve-ranges to work with the current
> release plugin flow... my working idea has been to inject e.g. XML PI
> entities into the pom so that we know what range to restore, e.g.
> pre-release pom has
>
> <dependency>
> ...
> <version>[1.2,2.0)</version>
> </dependency>
>
> which would get resolved into the release pom as
>
> <dependency>
> ...
> <version>1.5<?versions-maven-plugin original="[1.2,2.0)"?></version>
> </dependency>
>
> so that when we want to un-pin we can then see the original range and
> up-lift it so that the development pom becomes
>
> <dependency>
> ...
> <version>[1.5,2.0)</version>
> </dependency>
>
>
>
> >
> > thanks,
> > Robert
> >
> >
> > On Wed, 04 May 2016 21:20:01 +0200, Stephen Connolly <
> > stephen.alan.connolly@gmail.com> wrote:
> >
> > Here's an embryonic idea I had for doing Continuous Delivery with Maven
> >>
> >>
> >>
> https://www.cloudbees.com/blog/new-way-do-continuous-delivery-maven-and-jenkins-pipeline
> >>
> >> Probably needs a little fleshing out.
> >>
> >> -Stephen
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> > For additional commands, e-mail: users-help@maven.apache.org
> >
> >
>
Re: An idea I had for one way of doing Continuous Delivery with Maven
Posted by Stephen Connolly <st...@gmail.com>.
On 5 May 2016 at 10:00, Robert Scholte <rf...@apache.org> wrote:
> Hi Stephen, nice blog.
>
> There are a few things I'd like to share:
> - The maven-release-plugin does several checks which ensures that you have
> a reproducible artifact, where checking on SNAPSHOT dependencies is one of
> the most important ones. (you can find a lot patterns based on the
> versions-maven-plugin on the web, but they forget these checks)
>
This is one of the reasons why my suggestion is to actually cut a release
for every commit (just only push the tag - and just the tag - if the
release goes anywhere)
> - I still have this wish to have support a couple of strategies within the
> maven-release-plugin, this could be one as well.
>
Yes, I think it would be a valuable one to have in the default plugin, for
sure... and it may need some assistance from the scm plugin too
>
> In a continuous environment is becomes more and more important to be able
> to reproduce issues, which can be hard when things keep changing. This
> might even work with SNAPSHOT dependencies if your can get the
> distributable (either the deployable or executable artifact, assuming they
> contain all the required dependencies) and have a testing setup which can
> test with this artifact. If you can reproduce it here, you can (re)write it
> as an appropriate test and fix it.
> So next to all the current tests you need to do regression tests on the
> distributable as well.
> As long as this is not possible or secured in your pipeline, you should
> NEVER EVER want to depend on SNAPSHOT-dependencies.
>
+1
One that we gain with the approach I suggest is that we can actually use
versions:resolve-ranges as we know that the pom will not be pushed back to
master.
Thus developers can keep version ranges in the pom and the CI build will
resolve the ranges for each "release"
I'd like to find a way to allow resolve-ranges to work with the current
release plugin flow... my working idea has been to inject e.g. XML PI
entities into the pom so that we know what range to restore, e.g.
pre-release pom has
<dependency>
...
<version>[1.2,2.0)</version>
</dependency>
which would get resolved into the release pom as
<dependency>
...
<version>1.5<?versions-maven-plugin original="[1.2,2.0)"?></version>
</dependency>
so that when we want to un-pin we can then see the original range and
up-lift it so that the development pom becomes
<dependency>
...
<version>[1.5,2.0)</version>
</dependency>
>
> thanks,
> Robert
>
>
> On Wed, 04 May 2016 21:20:01 +0200, Stephen Connolly <
> stephen.alan.connolly@gmail.com> wrote:
>
> Here's an embryonic idea I had for doing Continuous Delivery with Maven
>>
>>
>> https://www.cloudbees.com/blog/new-way-do-continuous-delivery-maven-and-jenkins-pipeline
>>
>> Probably needs a little fleshing out.
>>
>> -Stephen
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
Re: An idea I had for one way of doing Continuous Delivery with Maven
Posted by Robert Scholte <rf...@apache.org>.
Hi Stephen, nice blog.
There are a few things I'd like to share:
- The maven-release-plugin does several checks which ensures that you have
a reproducible artifact, where checking on SNAPSHOT dependencies is one of
the most important ones. (you can find a lot patterns based on the
versions-maven-plugin on the web, but they forget these checks)
- I still have this wish to have support a couple of strategies within the
maven-release-plugin, this could be one as well.
In a continuous environment is becomes more and more important to be able
to reproduce issues, which can be hard when things keep changing. This
might even work with SNAPSHOT dependencies if your can get the
distributable (either the deployable or executable artifact, assuming they
contain all the required dependencies) and have a testing setup which can
test with this artifact. If you can reproduce it here, you can (re)write
it as an appropriate test and fix it.
So next to all the current tests you need to do regression tests on the
distributable as well.
As long as this is not possible or secured in your pipeline, you should
NEVER EVER want to depend on SNAPSHOT-dependencies.
thanks,
Robert
On Wed, 04 May 2016 21:20:01 +0200, Stephen Connolly
<st...@gmail.com> wrote:
> Here's an embryonic idea I had for doing Continuous Delivery with Maven
>
> https://www.cloudbees.com/blog/new-way-do-continuous-delivery-maven-and-jenkins-pipeline
>
> Probably needs a little fleshing out.
>
> -Stephen
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: An idea I had for one way of doing Continuous Delivery with Maven
Posted by Dan Tran <da...@gmail.com>.
Very promising, may need to do some scm provider work to support other SCM
like Perforce?
Thanks
-D
On Wed, May 4, 2016 at 12:20 PM, Stephen Connolly <
stephen.alan.connolly@gmail.com> wrote:
> Here's an embryonic idea I had for doing Continuous Delivery with Maven
>
>
> https://www.cloudbees.com/blog/new-way-do-continuous-delivery-maven-and-jenkins-pipeline
>
> Probably needs a little fleshing out.
>
> -Stephen
>