You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Bernd Eckenfels <ec...@zusammenkunft.net> on 2018/02/14 15:37:57 UTC
Re: Looking for recommendations how to best use Maven in a
muti-stagePipeline build to only deploy at the end
I am not sure why Nexus does not Support staging repos, I thought this is one of its defining Features? Certainly the ASF is using Nexus this way.
Anyway, another solution is to use a Feature of the CI to run a whole Pipeline on a single executor and have this executor have a local maven repo instance (this is a good idea anyway). Then you can avoid to deploy at all (only install) until your last step.
In my cases it Comes up not so often as the last Pipeline steps to release a product depend on Long ago released artifact versions with their own lifecylce.
Gruss
Bernd
--
http://bernd.eckenfels.net
Von: Robert Patrick
Gesendet: Mittwoch, 14. Februar 2018 16:01
An: Maven Users List
Betreff: RE: Looking for recommendations how to best use Maven in a muti-stagePipeline build to only deploy at the end
While Nexus may not support "staging repositories", it certainly supports more than one repository so why not just create one or more repositories that serve as staging repositories. For example,
Pipeline Steps:
1.) Trigger a build based on source check-in and push to stage1 repo if build "succeeds"
2.) Pull artifacts from stage1 repo, run stage 2 tests, and push to stage2 repo if tests succeed.
3.) Pull artifacts from stage2 repo, run stage 3 tests, and push to stage3 repo if tests succeed.
4.) Pull artifacts from stage3 repo, run UAT tests, and push to release repo if tests succeed.
-----Original Message-----
From: Eric B [mailto:ebenzacar@gmail.com]
Sent: Wednesday, February 14, 2018 8:30 AM
To: Maven Users List <us...@maven.apache.org>
Subject: Looking for recommendations how to best use Maven in a muti-stage Pipeline build to only deploy at the end
I'm looking for recommendations for the best way to use Maven in a multi-stage Jenkins pipeline build to deploy only at the end. At the moment, I'm using Sonatype Nexus 3.x, which means i don't have the benefit
of staging repos. Consequently, I have to ensure that the only released
versions of my libraries/application are final - they've passed QA.
Additionally, the team wants to ensure that the version numbers are always incremental and every version in the repo is a consumable version (ie: I cannot deploy a version 1.2.3 which has not passed QA/Acceptance Tests, and furthermore, I cannot deploy a 1.2.2 followed by a 1.2.4).
What that requirement translates to is that I have to ensure that the binary built is fully tested before promoting it to Nexus. (and that I shouldn't be appending build numbers to the maven version number).
In my mind, I would like to do something the following in a Pipeline build:
stage('build') { steps { sh 'mvn clean install'} }
stage('Confirm deploy to QA'){
steps {
checkpoint 'test server deployed'
script {
env.DEPLOY_TO_QA_TEST = input message: 'User input required',
submitter: 'authenticated',
parameters: [choice(name: 'Deploy to acceptance test server', choices:
'no\nyes', description: 'Choose "yes" if you want to deploy the QA test server')]
}
}
}
stage('deployQA') {
when {
environment name: 'DEPLOY_TO_QA_TEST', value: 'yes'
}
steps{
/* deploy the build to a QA environment */ }
}
stage('Confirm deploy to UAT'){ // prompt user }
stage {'deployUAT') { /* deploy the build to a PreProd/User Acceptance Testing enviornment */}
stage('Confirm publish to Nexus'){ // prompt user }
stage('publish') {
steps {
// mvn deploy -DskipTests (just deploy - don't rebuild)
}
}
Basically, I want to design my Jenkins pipeline to be my staging process.
The problem I have is I'm not sure how I can only deploy at the end of the pipeline. When maven runs the deploy lifecycle, it will run through all the other stages and reassemble my binaries, which technically are not longer the same as those that were approved. So consequently, my binary hashes that were approved earlier in the pipeline are not the same hashes that are deployed in Nexus.
I realize that i can probably do some work and use the Reproducible Build plugin (https://urldefense.proofpoint.com/v2/url?u=https-3A__zlika.github.io_reproducible-2Dbuild-2Dmaven-2Dplugin_&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=nSxyAsyxa1Izff8ULe7vW8u084madbci-hLPsiLpxeU&m=Og2S17jc02JRwm-oHae8UQmiIg_ygbBRL0EQoB_Wvuw&s=zipmYyPLpmFYv1RsnquZtQMf0-HoYoix12SZj6gD2jM&e=), but that too comes with drawbacks (I want build timestamps in my Manifest files, and zip entries, etc).
Am I sunk? Is my only hope to wait until Sonatype releases Staging repos for Nexus 3.x sometime in Q2? Or is there some other way I can work around this?
How is everyone else handling this situation?
Thanks for sharing.
Eric
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Looking for recommendations how to best use Maven in a
muti-stagePipeline build to only deploy at the end
Posted by Brian Fox <br...@infinity.nu>.
On Wed, Feb 14, 2018 at 9:31 PM, Eric B <eb...@gmail.com> wrote:
> Bernd,
>
> Nexus 3.x does not support staging repos b/c they are rewriting the entire
> platform to support not just Maven artifacts, but any type of repo-based
> artifact. Ex: docker images, npm dependencies, etc...
This is true and it's getting close to being done. Also included is a
pipeline plugin for Jenkins to help you effect the promotions
directly.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Looking for recommendations how to best use Maven in a
muti-stagePipeline build to only deploy at the end
Posted by Eric B <eb...@gmail.com>.
Bernd,
Nexus 3.x does not support staging repos b/c they are rewriting the entire
platform to support not just Maven artifacts, but any type of repo-based
artifact. Ex: docker images, npm dependencies, etc... But the new feature
won't be available until at least April or May.
With respect to running the whole Pipeline on a single executor with a
local maven repo was what I was thinking of doing. But I still don't know
how to get it to deploy only at the last step without rebuilding again. If
my last step just calls the deploy goal, then it will still rebuild the
entire artifact - not just grab whatever is in the local repo and deploy
it. And I'm looking to avoid rebuilding the artifact since I want the
original artifact (the one sitting in the local m2 folder) to be the one
pushed to the repo.
Any suggestions?
Thanks,
Eric
On Wed, Feb 14, 2018 at 10:37 AM, Bernd Eckenfels <ec...@zusammenkunft.net>
wrote:
> I am not sure why Nexus does not Support staging repos, I thought this is
> one of its defining Features? Certainly the ASF is using Nexus this way.
>
> Anyway, another solution is to use a Feature of the CI to run a whole
> Pipeline on a single executor and have this executor have a local maven
> repo instance (this is a good idea anyway). Then you can avoid to deploy at
> all (only install) until your last step.
>
> In my cases it Comes up not so often as the last Pipeline steps to release
> a product depend on Long ago released artifact versions with their own
> lifecylce.
>
> Gruss
> Bernd
> --
> http://bernd.eckenfels.net
>
> Von: Robert Patrick
> Gesendet: Mittwoch, 14. Februar 2018 16:01
> An: Maven Users List
> Betreff: RE: Looking for recommendations how to best use Maven in a
> muti-stagePipeline build to only deploy at the end
>
> While Nexus may not support "staging repositories", it certainly supports
> more than one repository so why not just create one or more repositories
> that serve as staging repositories. For example,
>
> Pipeline Steps:
> 1.) Trigger a build based on source check-in and push to stage1 repo if
> build "succeeds"
> 2.) Pull artifacts from stage1 repo, run stage 2 tests, and push to stage2
> repo if tests succeed.
> 3.) Pull artifacts from stage2 repo, run stage 3 tests, and push to stage3
> repo if tests succeed.
> 4.) Pull artifacts from stage3 repo, run UAT tests, and push to release
> repo if tests succeed.
>
>
>
>
> -----Original Message-----
> From: Eric B [mailto:ebenzacar@gmail.com]
> Sent: Wednesday, February 14, 2018 8:30 AM
> To: Maven Users List <us...@maven.apache.org>
> Subject: Looking for recommendations how to best use Maven in a muti-stage
> Pipeline build to only deploy at the end
>
> I'm looking for recommendations for the best way to use Maven in a
> multi-stage Jenkins pipeline build to deploy only at the end. At the
> moment, I'm using Sonatype Nexus 3.x, which means i don't have the benefit
> of staging repos. Consequently, I have to ensure that the only released
> versions of my libraries/application are final - they've passed QA.
> Additionally, the team wants to ensure that the version numbers are always
> incremental and every version in the repo is a consumable version (ie: I
> cannot deploy a version 1.2.3 which has not passed QA/Acceptance Tests, and
> furthermore, I cannot deploy a 1.2.2 followed by a 1.2.4).
>
> What that requirement translates to is that I have to ensure that the
> binary built is fully tested before promoting it to Nexus. (and that I
> shouldn't be appending build numbers to the maven version number).
>
> In my mind, I would like to do something the following in a Pipeline build:
>
> stage('build') { steps { sh 'mvn clean install'} }
>
> stage('Confirm deploy to QA'){
>
> steps {
>
> checkpoint 'test server deployed'
>
> script {
>
> env.DEPLOY_TO_QA_TEST = input message: 'User input required',
>
> submitter: 'authenticated',
>
> parameters: [choice(name: 'Deploy to acceptance test server', choices:
> 'no\nyes', description: 'Choose "yes" if you want to deploy the QA test
> server')]
>
> }
>
> }
>
> }
>
> stage('deployQA') {
>
> when {
>
> environment name: 'DEPLOY_TO_QA_TEST', value: 'yes'
>
> }
>
> steps{
>
> /* deploy the build to a QA environment */ }
>
> }
>
>
> stage('Confirm deploy to UAT'){ // prompt user }
>
> stage {'deployUAT') { /* deploy the build to a PreProd/User Acceptance
> Testing enviornment */}
>
>
> stage('Confirm publish to Nexus'){ // prompt user }
>
> stage('publish') {
>
> steps {
>
> // mvn deploy -DskipTests (just deploy - don't rebuild)
>
> }
>
> }
>
>
> Basically, I want to design my Jenkins pipeline to be my staging process.
> The problem I have is I'm not sure how I can only deploy at the end of the
> pipeline. When maven runs the deploy lifecycle, it will run through all
> the other stages and reassemble my binaries, which technically are not
> longer the same as those that were approved. So consequently, my binary
> hashes that were approved earlier in the pipeline are not the same hashes
> that are deployed in Nexus.
>
> I realize that i can probably do some work and use the Reproducible Build
> plugin (https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__zlika.github.io_reproducible-2Dbuild-2Dmaven-2Dplugin_&d=DwIBaQ&c=
> RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=
> nSxyAsyxa1Izff8ULe7vW8u084madbci-hLPsiLpxeU&m=Og2S17jc02JRwm-oHae8UQmiIg_
> ygbBRL0EQoB_Wvuw&s=zipmYyPLpmFYv1RsnquZtQMf0-HoYoix12SZj6gD2jM&e=), but
> that too comes with drawbacks (I want build timestamps in my Manifest
> files, and zip entries, etc).
>
> Am I sunk? Is my only hope to wait until Sonatype releases Staging repos
> for Nexus 3.x sometime in Q2? Or is there some other way I can work around
> this?
>
> How is everyone else handling this situation?
>
> Thanks for sharing.
>
> Eric
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
>