You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geode.apache.org by Robert Houghton <rh...@pivotal.io> on 2020/04/27 22:21:10 UTC
[DISCUSS] Publish Builds, not Snapshots
Hello to the community,
tl;dr - Lets publish builds, not snapshots, for repeatable CI builds, as
GEODE-8016[1]. Communicate desired artifact version via the existing
'UpdatePassingTokens' job.
I have been working on the Geode build and CI systems for a long time, and
it has irked me that the geode-examples pipeline[2] does not build and test
against the latest artifacts from the develop pipeline. Some work has been
done already to allow this via "composite" builds for local testing without
needing to publish Geode to your local Maven repository.
From a Concourse CI perspective, composite builds are costly due to the
rebuild of the upstream artifacts. They allow repeatable builds, but only
by rebuilding those dependencies. Better would be to point to upstream
artifacts as concrete build versions. SNAPSHOT builds can and do roll
(invisibly) as new versions are published. Discrete, numbered builds do
not. Downstream consumers can use greedy version specifiers to get their
current behavior of "latest".
Gradle: 'org.apache.geode:geode-core:1.13.0+'
Maven: '<groupId>org.apache.geode</group>'
'<artifactId>geode-core</name>'
'<version>[1.13.0,1.14.0)</version>'
What do you all think? Discuss!
-Robert Houghton
[1] https://issues.apache.org/jira/browse/GEODE-8016
[2]
https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-examples
Re: [DISCUSS] Publish Builds, not Snapshots
Posted by Owen Nichols <on...@pivotal.io>.
Not sure that’s true. Internally maven snapshot repos are versioned, and the maven-resource is able to download any desired version still on the server, even if maven itself doesn’t expose that capability. You just need to ensure your job uses the version put into the local maven cache by the maven-resource, rather than bypassing that and re-downloading.
Mostly I’m just trying to separate out which problems with geode-examples are helped by your proposal, and which are orthogonal. I assume you just brought up geode-examples as one example of something that consumes Geode. All consumers, including geode-examples, will benefit from having traceable versioning :)
> On Apr 27, 2020, at 4:40 PM, Robert Houghton <rh...@pivotal.io> wrote:
>
> But the "maven-resource" type is still non-repeatable for SNAPSHOTs.
> Resource pinning will not work, and identifying the actual GEODE build
> causing the failure won't work.
>
> On Mon, Apr 27, 2020 at 4:39 PM Owen Nichols <on...@pivotal.io> wrote:
>
>> That sounds like a problem with geode-examples regardless of whether we
>> use -SNAPSHOT or -build.123 … we should replace the 24hr trigger with a
>> maven-resource trigger (nulldriver, linkyard, or similar)
>>
>>> On Apr 27, 2020, at 4:34 PM, Robert Houghton <rh...@pivotal.io>
>> wrote:
>>>
>>> @Owen: What geode-examples is doing, is a perfect case study of the wrong
>>> behaviors predicated on 'SNAPSHOT' with Concourse.
>>>
>>> Concourse CI is built on repeatable builds with defined input. The
>> current
>>> CI for examples is using a 24hr trigger because it has no valid
>> dependency
>>> on the Geode develop pipeline.
>>>
>>>
>>> On Mon, Apr 27, 2020 at 3:55 PM Owen Nichols <on...@pivotal.io>
>> wrote:
>>>
>>>> Sounds good to me, based on the repeatability argument. Will the build
>>>> number output by gfsh version --full match the maven artifact spec?
>>>>
>>>> I am unclear what geode-examples has to do with this. Aside from
>>>> repeatability, there is no reason geode-examples should be having any
>>>> problem with -SNAPSHOT. As far as I can tell, geode-examples develop is
>>>> correctly resolving geodeDistribution from
>>>> org.apache.geode:apache-geode:1.13.0-SNAPSHOT, and geode-examples
>> master is
>>>> correctly resolving geodeDistribution from
>>>> org.apache.geode:apache-geode:1.12.0
>>>>
>>>>> On Apr 27, 2020, at 3:41 PM, Anthony Baker <ab...@pivotal.io> wrote:
>>>>>
>>>>> @Robert, can you show some examples of what the build number would be
>>>> under this proposal? Does 1.13.0-SNAPSHOT become 1.13.0.N where N
>>>> increments every build?
>>>>>
>>>>> Seems reasonable. Since the consumers of pre-release artifacts are
>>>> either a) this project or b) close related projects for
>>>> integration-testing-purposes-only I’m not super worried about the ugly
>>>> syntax.
>>>>>
>>>>>
>>>>> Anthony
>>>>>
>>>>>
>>>>>> On Apr 27, 2020, at 3:25 PM, Jacob Barrett <jb...@pivotal.io>
>> wrote:
>>>>>>
>>>>>> It is unfortunate that the Maven/Gradle community hasn’t addressed
>> this
>>>> glaring issue with SNAPSHOT for decades now (well maybe not decades but
>>>> certainly decade). It is also unfortunate that the Maven version
>> coordinate
>>>> is ugly. Aside from that I am totally onboard. Yay for reproducible
>> builds
>>>> and predictable downstream builds!
>>>>>>
>>>>>> With SNAPSHOTS in a repo the repository automatically prunes back old
>>>> builds. Do we have any concerns about having a plethora of builds
>> filling
>>>> up this new pre-release repository?
>>>>>>
>>>>>> -Jake
>>>>>>
>>>>>>> On Apr 27, 2020, at 3:21 PM, Robert Houghton <rh...@pivotal.io>
>>>> wrote:
>>>>>>>
>>>>>>> Hello to the community,
>>>>>>>
>>>>>>> tl;dr - Lets publish builds, not snapshots, for repeatable CI builds,
>>>> as
>>>>>>> GEODE-8016[1]. Communicate desired artifact version via the existing
>>>>>>> 'UpdatePassingTokens' job.
>>>>>>>
>>>>>>> I have been working on the Geode build and CI systems for a long
>> time,
>>>> and
>>>>>>> it has irked me that the geode-examples pipeline[2] does not build
>> and
>>>> test
>>>>>>> against the latest artifacts from the develop pipeline. Some work has
>>>> been
>>>>>>> done already to allow this via "composite" builds for local testing
>>>> without
>>>>>>> needing to publish Geode to your local Maven repository.
>>>>>>>
>>>>>>> From a Concourse CI perspective, composite builds are costly due to
>> the
>>>>>>> rebuild of the upstream artifacts. They allow repeatable builds, but
>>>> only
>>>>>>> by rebuilding those dependencies. Better would be to point to
>> upstream
>>>>>>> artifacts as concrete build versions. SNAPSHOT builds can and do roll
>>>>>>> (invisibly) as new versions are published. Discrete, numbered builds
>> do
>>>>>>> not. Downstream consumers can use greedy version specifiers to get
>>>> their
>>>>>>> current behavior of "latest".
>>>>>>>
>>>>>>> Gradle: 'org.apache.geode:geode-core:1.13.0+'
>>>>>>> Maven: '<groupId>org.apache.geode</group>'
>>>>>>> '<artifactId>geode-core</name>'
>>>>>>> '<version>[1.13.0,1.14.0)</version>'
>>>>>>>
>>>>>>> What do you all think? Discuss!
>>>>>>> -Robert Houghton
>>>>>>>
>>>>>>> [1] https://issues.apache.org/jira/browse/GEODE-8016
>>>>>>> [2]
>>>>>>>
>>>>
>> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-examples
>>>>>>
>>>>>
>>>>
>>>>
>>
>>
Re: [DISCUSS] Publish Builds, not Snapshots
Posted by Robert Houghton <rh...@pivotal.io>.
But the "maven-resource" type is still non-repeatable for SNAPSHOTs.
Resource pinning will not work, and identifying the actual GEODE build
causing the failure won't work.
On Mon, Apr 27, 2020 at 4:39 PM Owen Nichols <on...@pivotal.io> wrote:
> That sounds like a problem with geode-examples regardless of whether we
> use -SNAPSHOT or -build.123 … we should replace the 24hr trigger with a
> maven-resource trigger (nulldriver, linkyard, or similar)
>
> > On Apr 27, 2020, at 4:34 PM, Robert Houghton <rh...@pivotal.io>
> wrote:
> >
> > @Owen: What geode-examples is doing, is a perfect case study of the wrong
> > behaviors predicated on 'SNAPSHOT' with Concourse.
> >
> > Concourse CI is built on repeatable builds with defined input. The
> current
> > CI for examples is using a 24hr trigger because it has no valid
> dependency
> > on the Geode develop pipeline.
> >
> >
> > On Mon, Apr 27, 2020 at 3:55 PM Owen Nichols <on...@pivotal.io>
> wrote:
> >
> >> Sounds good to me, based on the repeatability argument. Will the build
> >> number output by gfsh version --full match the maven artifact spec?
> >>
> >> I am unclear what geode-examples has to do with this. Aside from
> >> repeatability, there is no reason geode-examples should be having any
> >> problem with -SNAPSHOT. As far as I can tell, geode-examples develop is
> >> correctly resolving geodeDistribution from
> >> org.apache.geode:apache-geode:1.13.0-SNAPSHOT, and geode-examples
> master is
> >> correctly resolving geodeDistribution from
> >> org.apache.geode:apache-geode:1.12.0
> >>
> >>> On Apr 27, 2020, at 3:41 PM, Anthony Baker <ab...@pivotal.io> wrote:
> >>>
> >>> @Robert, can you show some examples of what the build number would be
> >> under this proposal? Does 1.13.0-SNAPSHOT become 1.13.0.N where N
> >> increments every build?
> >>>
> >>> Seems reasonable. Since the consumers of pre-release artifacts are
> >> either a) this project or b) close related projects for
> >> integration-testing-purposes-only I’m not super worried about the ugly
> >> syntax.
> >>>
> >>>
> >>> Anthony
> >>>
> >>>
> >>>> On Apr 27, 2020, at 3:25 PM, Jacob Barrett <jb...@pivotal.io>
> wrote:
> >>>>
> >>>> It is unfortunate that the Maven/Gradle community hasn’t addressed
> this
> >> glaring issue with SNAPSHOT for decades now (well maybe not decades but
> >> certainly decade). It is also unfortunate that the Maven version
> coordinate
> >> is ugly. Aside from that I am totally onboard. Yay for reproducible
> builds
> >> and predictable downstream builds!
> >>>>
> >>>> With SNAPSHOTS in a repo the repository automatically prunes back old
> >> builds. Do we have any concerns about having a plethora of builds
> filling
> >> up this new pre-release repository?
> >>>>
> >>>> -Jake
> >>>>
> >>>>> On Apr 27, 2020, at 3:21 PM, Robert Houghton <rh...@pivotal.io>
> >> wrote:
> >>>>>
> >>>>> Hello to the community,
> >>>>>
> >>>>> tl;dr - Lets publish builds, not snapshots, for repeatable CI builds,
> >> as
> >>>>> GEODE-8016[1]. Communicate desired artifact version via the existing
> >>>>> 'UpdatePassingTokens' job.
> >>>>>
> >>>>> I have been working on the Geode build and CI systems for a long
> time,
> >> and
> >>>>> it has irked me that the geode-examples pipeline[2] does not build
> and
> >> test
> >>>>> against the latest artifacts from the develop pipeline. Some work has
> >> been
> >>>>> done already to allow this via "composite" builds for local testing
> >> without
> >>>>> needing to publish Geode to your local Maven repository.
> >>>>>
> >>>>> From a Concourse CI perspective, composite builds are costly due to
> the
> >>>>> rebuild of the upstream artifacts. They allow repeatable builds, but
> >> only
> >>>>> by rebuilding those dependencies. Better would be to point to
> upstream
> >>>>> artifacts as concrete build versions. SNAPSHOT builds can and do roll
> >>>>> (invisibly) as new versions are published. Discrete, numbered builds
> do
> >>>>> not. Downstream consumers can use greedy version specifiers to get
> >> their
> >>>>> current behavior of "latest".
> >>>>>
> >>>>> Gradle: 'org.apache.geode:geode-core:1.13.0+'
> >>>>> Maven: '<groupId>org.apache.geode</group>'
> >>>>> '<artifactId>geode-core</name>'
> >>>>> '<version>[1.13.0,1.14.0)</version>'
> >>>>>
> >>>>> What do you all think? Discuss!
> >>>>> -Robert Houghton
> >>>>>
> >>>>> [1] https://issues.apache.org/jira/browse/GEODE-8016
> >>>>> [2]
> >>>>>
> >>
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-examples
> >>>>
> >>>
> >>
> >>
>
>
Re: [DISCUSS] Publish Builds, not Snapshots
Posted by Owen Nichols <on...@pivotal.io>.
That sounds like a problem with geode-examples regardless of whether we use -SNAPSHOT or -build.123 … we should replace the 24hr trigger with a maven-resource trigger (nulldriver, linkyard, or similar)
> On Apr 27, 2020, at 4:34 PM, Robert Houghton <rh...@pivotal.io> wrote:
>
> @Owen: What geode-examples is doing, is a perfect case study of the wrong
> behaviors predicated on 'SNAPSHOT' with Concourse.
>
> Concourse CI is built on repeatable builds with defined input. The current
> CI for examples is using a 24hr trigger because it has no valid dependency
> on the Geode develop pipeline.
>
>
> On Mon, Apr 27, 2020 at 3:55 PM Owen Nichols <on...@pivotal.io> wrote:
>
>> Sounds good to me, based on the repeatability argument. Will the build
>> number output by gfsh version --full match the maven artifact spec?
>>
>> I am unclear what geode-examples has to do with this. Aside from
>> repeatability, there is no reason geode-examples should be having any
>> problem with -SNAPSHOT. As far as I can tell, geode-examples develop is
>> correctly resolving geodeDistribution from
>> org.apache.geode:apache-geode:1.13.0-SNAPSHOT, and geode-examples master is
>> correctly resolving geodeDistribution from
>> org.apache.geode:apache-geode:1.12.0
>>
>>> On Apr 27, 2020, at 3:41 PM, Anthony Baker <ab...@pivotal.io> wrote:
>>>
>>> @Robert, can you show some examples of what the build number would be
>> under this proposal? Does 1.13.0-SNAPSHOT become 1.13.0.N where N
>> increments every build?
>>>
>>> Seems reasonable. Since the consumers of pre-release artifacts are
>> either a) this project or b) close related projects for
>> integration-testing-purposes-only I’m not super worried about the ugly
>> syntax.
>>>
>>>
>>> Anthony
>>>
>>>
>>>> On Apr 27, 2020, at 3:25 PM, Jacob Barrett <jb...@pivotal.io> wrote:
>>>>
>>>> It is unfortunate that the Maven/Gradle community hasn’t addressed this
>> glaring issue with SNAPSHOT for decades now (well maybe not decades but
>> certainly decade). It is also unfortunate that the Maven version coordinate
>> is ugly. Aside from that I am totally onboard. Yay for reproducible builds
>> and predictable downstream builds!
>>>>
>>>> With SNAPSHOTS in a repo the repository automatically prunes back old
>> builds. Do we have any concerns about having a plethora of builds filling
>> up this new pre-release repository?
>>>>
>>>> -Jake
>>>>
>>>>> On Apr 27, 2020, at 3:21 PM, Robert Houghton <rh...@pivotal.io>
>> wrote:
>>>>>
>>>>> Hello to the community,
>>>>>
>>>>> tl;dr - Lets publish builds, not snapshots, for repeatable CI builds,
>> as
>>>>> GEODE-8016[1]. Communicate desired artifact version via the existing
>>>>> 'UpdatePassingTokens' job.
>>>>>
>>>>> I have been working on the Geode build and CI systems for a long time,
>> and
>>>>> it has irked me that the geode-examples pipeline[2] does not build and
>> test
>>>>> against the latest artifacts from the develop pipeline. Some work has
>> been
>>>>> done already to allow this via "composite" builds for local testing
>> without
>>>>> needing to publish Geode to your local Maven repository.
>>>>>
>>>>> From a Concourse CI perspective, composite builds are costly due to the
>>>>> rebuild of the upstream artifacts. They allow repeatable builds, but
>> only
>>>>> by rebuilding those dependencies. Better would be to point to upstream
>>>>> artifacts as concrete build versions. SNAPSHOT builds can and do roll
>>>>> (invisibly) as new versions are published. Discrete, numbered builds do
>>>>> not. Downstream consumers can use greedy version specifiers to get
>> their
>>>>> current behavior of "latest".
>>>>>
>>>>> Gradle: 'org.apache.geode:geode-core:1.13.0+'
>>>>> Maven: '<groupId>org.apache.geode</group>'
>>>>> '<artifactId>geode-core</name>'
>>>>> '<version>[1.13.0,1.14.0)</version>'
>>>>>
>>>>> What do you all think? Discuss!
>>>>> -Robert Houghton
>>>>>
>>>>> [1] https://issues.apache.org/jira/browse/GEODE-8016
>>>>> [2]
>>>>>
>> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-examples
>>>>
>>>
>>
>>
Re: [DISCUSS] Publish Builds, not Snapshots
Posted by Robert Houghton <rh...@pivotal.io>.
@Owen: What geode-examples is doing, is a perfect case study of the wrong
behaviors predicated on 'SNAPSHOT' with Concourse.
Concourse CI is built on repeatable builds with defined input. The current
CI for examples is using a 24hr trigger because it has no valid dependency
on the Geode develop pipeline.
On Mon, Apr 27, 2020 at 3:55 PM Owen Nichols <on...@pivotal.io> wrote:
> Sounds good to me, based on the repeatability argument. Will the build
> number output by gfsh version --full match the maven artifact spec?
>
> I am unclear what geode-examples has to do with this. Aside from
> repeatability, there is no reason geode-examples should be having any
> problem with -SNAPSHOT. As far as I can tell, geode-examples develop is
> correctly resolving geodeDistribution from
> org.apache.geode:apache-geode:1.13.0-SNAPSHOT, and geode-examples master is
> correctly resolving geodeDistribution from
> org.apache.geode:apache-geode:1.12.0
>
> > On Apr 27, 2020, at 3:41 PM, Anthony Baker <ab...@pivotal.io> wrote:
> >
> > @Robert, can you show some examples of what the build number would be
> under this proposal? Does 1.13.0-SNAPSHOT become 1.13.0.N where N
> increments every build?
> >
> > Seems reasonable. Since the consumers of pre-release artifacts are
> either a) this project or b) close related projects for
> integration-testing-purposes-only I’m not super worried about the ugly
> syntax.
> >
> >
> > Anthony
> >
> >
> >> On Apr 27, 2020, at 3:25 PM, Jacob Barrett <jb...@pivotal.io> wrote:
> >>
> >> It is unfortunate that the Maven/Gradle community hasn’t addressed this
> glaring issue with SNAPSHOT for decades now (well maybe not decades but
> certainly decade). It is also unfortunate that the Maven version coordinate
> is ugly. Aside from that I am totally onboard. Yay for reproducible builds
> and predictable downstream builds!
> >>
> >> With SNAPSHOTS in a repo the repository automatically prunes back old
> builds. Do we have any concerns about having a plethora of builds filling
> up this new pre-release repository?
> >>
> >> -Jake
> >>
> >>> On Apr 27, 2020, at 3:21 PM, Robert Houghton <rh...@pivotal.io>
> wrote:
> >>>
> >>> Hello to the community,
> >>>
> >>> tl;dr - Lets publish builds, not snapshots, for repeatable CI builds,
> as
> >>> GEODE-8016[1]. Communicate desired artifact version via the existing
> >>> 'UpdatePassingTokens' job.
> >>>
> >>> I have been working on the Geode build and CI systems for a long time,
> and
> >>> it has irked me that the geode-examples pipeline[2] does not build and
> test
> >>> against the latest artifacts from the develop pipeline. Some work has
> been
> >>> done already to allow this via "composite" builds for local testing
> without
> >>> needing to publish Geode to your local Maven repository.
> >>>
> >>> From a Concourse CI perspective, composite builds are costly due to the
> >>> rebuild of the upstream artifacts. They allow repeatable builds, but
> only
> >>> by rebuilding those dependencies. Better would be to point to upstream
> >>> artifacts as concrete build versions. SNAPSHOT builds can and do roll
> >>> (invisibly) as new versions are published. Discrete, numbered builds do
> >>> not. Downstream consumers can use greedy version specifiers to get
> their
> >>> current behavior of "latest".
> >>>
> >>> Gradle: 'org.apache.geode:geode-core:1.13.0+'
> >>> Maven: '<groupId>org.apache.geode</group>'
> >>> '<artifactId>geode-core</name>'
> >>> '<version>[1.13.0,1.14.0)</version>'
> >>>
> >>> What do you all think? Discuss!
> >>> -Robert Houghton
> >>>
> >>> [1] https://issues.apache.org/jira/browse/GEODE-8016
> >>> [2]
> >>>
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-examples
> >>
> >
>
>
Re: [DISCUSS] Publish Builds, not Snapshots
Posted by Owen Nichols <on...@pivotal.io>.
Sounds good to me, based on the repeatability argument. Will the build number output by gfsh version --full match the maven artifact spec?
I am unclear what geode-examples has to do with this. Aside from repeatability, there is no reason geode-examples should be having any problem with -SNAPSHOT. As far as I can tell, geode-examples develop is correctly resolving geodeDistribution from org.apache.geode:apache-geode:1.13.0-SNAPSHOT, and geode-examples master is correctly resolving geodeDistribution from org.apache.geode:apache-geode:1.12.0
> On Apr 27, 2020, at 3:41 PM, Anthony Baker <ab...@pivotal.io> wrote:
>
> @Robert, can you show some examples of what the build number would be under this proposal? Does 1.13.0-SNAPSHOT become 1.13.0.N where N increments every build?
>
> Seems reasonable. Since the consumers of pre-release artifacts are either a) this project or b) close related projects for integration-testing-purposes-only I’m not super worried about the ugly syntax.
>
>
> Anthony
>
>
>> On Apr 27, 2020, at 3:25 PM, Jacob Barrett <jb...@pivotal.io> wrote:
>>
>> It is unfortunate that the Maven/Gradle community hasn’t addressed this glaring issue with SNAPSHOT for decades now (well maybe not decades but certainly decade). It is also unfortunate that the Maven version coordinate is ugly. Aside from that I am totally onboard. Yay for reproducible builds and predictable downstream builds!
>>
>> With SNAPSHOTS in a repo the repository automatically prunes back old builds. Do we have any concerns about having a plethora of builds filling up this new pre-release repository?
>>
>> -Jake
>>
>>> On Apr 27, 2020, at 3:21 PM, Robert Houghton <rh...@pivotal.io> wrote:
>>>
>>> Hello to the community,
>>>
>>> tl;dr - Lets publish builds, not snapshots, for repeatable CI builds, as
>>> GEODE-8016[1]. Communicate desired artifact version via the existing
>>> 'UpdatePassingTokens' job.
>>>
>>> I have been working on the Geode build and CI systems for a long time, and
>>> it has irked me that the geode-examples pipeline[2] does not build and test
>>> against the latest artifacts from the develop pipeline. Some work has been
>>> done already to allow this via "composite" builds for local testing without
>>> needing to publish Geode to your local Maven repository.
>>>
>>> From a Concourse CI perspective, composite builds are costly due to the
>>> rebuild of the upstream artifacts. They allow repeatable builds, but only
>>> by rebuilding those dependencies. Better would be to point to upstream
>>> artifacts as concrete build versions. SNAPSHOT builds can and do roll
>>> (invisibly) as new versions are published. Discrete, numbered builds do
>>> not. Downstream consumers can use greedy version specifiers to get their
>>> current behavior of "latest".
>>>
>>> Gradle: 'org.apache.geode:geode-core:1.13.0+'
>>> Maven: '<groupId>org.apache.geode</group>'
>>> '<artifactId>geode-core</name>'
>>> '<version>[1.13.0,1.14.0)</version>'
>>>
>>> What do you all think? Discuss!
>>> -Robert Houghton
>>>
>>> [1] https://issues.apache.org/jira/browse/GEODE-8016
>>> [2]
>>> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-examples
>>
>
Re: [DISCUSS] Publish Builds, not Snapshots
Posted by Donal Evans <do...@pivotal.io>.
That just means we get to be trailblazers, I guess.
On Tue, Apr 28, 2020 at 9:02 AM Robert Houghton <rh...@pivotal.io>
wrote:
> @donal I do not have an example of a Java project that builds using a
> Gradle multi-module build, that cares as much as we do about re-running CI
> tests with identical inputs. I wish I did, I would crib from their answer
> sheet without shame or remorse.
>
> On Mon, Apr 27, 2020 at 4:37 PM Donal Evans <do...@vmware.com> wrote:
>
> > Do we know if this is an issue that other open source projects have dealt
> > with? And if so, is this proposed solution similar to what they might
> have
> > done to remedy it?
> > ________________________________
> > From: Robert Houghton <rh...@pivotal.io>
> > Sent: Monday, April 27, 2020 4:31 PM
> > To: dev@geode.apache.org <de...@geode.apache.org>
> > Subject: Re: [DISCUSS] Publish Builds, not Snapshots
> >
> > The artifact would change from "1.13.0-SNAPSHOT" to "1.13.0-build.123".
> >
> > The number after the "build" slug is auto-incremented by our CI system
> > anyway, as the "geode-build-version" semver resource. We are actually
> doing
> > *more* work in Gradle to truncate that number from the current "SNAPSHOT"
> > value.
> >
> > On Mon, Apr 27, 2020 at 3:41 PM Anthony Baker <ab...@pivotal.io> wrote:
> >
> > > @Robert, can you show some examples of what the build number would be
> > > under this proposal? Does 1.13.0-SNAPSHOT become 1.13.0.N where N
> > > increments every build?
> > >
> > > Seems reasonable. Since the consumers of pre-release artifacts are
> > either
> > > a) this project or b) close related projects for
> > > integration-testing-purposes-only I’m not super worried about the ugly
> > > syntax.
> > >
> > >
> > > Anthony
> > >
> > >
> > > > On Apr 27, 2020, at 3:25 PM, Jacob Barrett <jb...@pivotal.io>
> > wrote:
> > > >
> > > > It is unfortunate that the Maven/Gradle community hasn’t addressed
> this
> > > glaring issue with SNAPSHOT for decades now (well maybe not decades but
> > > certainly decade). It is also unfortunate that the Maven version
> > coordinate
> > > is ugly. Aside from that I am totally onboard. Yay for reproducible
> > builds
> > > and predictable downstream builds!
> > > >
> > > > With SNAPSHOTS in a repo the repository automatically prunes back old
> > > builds. Do we have any concerns about having a plethora of builds
> filling
> > > up this new pre-release repository?
> > > >
> > > > -Jake
> > > >
> > > >> On Apr 27, 2020, at 3:21 PM, Robert Houghton <rh...@pivotal.io>
> > > wrote:
> > > >>
> > > >> Hello to the community,
> > > >>
> > > >> tl;dr - Lets publish builds, not snapshots, for repeatable CI
> builds,
> > as
> > > >> GEODE-8016[1]. Communicate desired artifact version via the existing
> > > >> 'UpdatePassingTokens' job.
> > > >>
> > > >> I have been working on the Geode build and CI systems for a long
> time,
> > > and
> > > >> it has irked me that the geode-examples pipeline[2] does not build
> and
> > > test
> > > >> against the latest artifacts from the develop pipeline. Some work
> has
> > > been
> > > >> done already to allow this via "composite" builds for local testing
> > > without
> > > >> needing to publish Geode to your local Maven repository.
> > > >>
> > > >> From a Concourse CI perspective, composite builds are costly due to
> > the
> > > >> rebuild of the upstream artifacts. They allow repeatable builds, but
> > > only
> > > >> by rebuilding those dependencies. Better would be to point to
> upstream
> > > >> artifacts as concrete build versions. SNAPSHOT builds can and do
> roll
> > > >> (invisibly) as new versions are published. Discrete, numbered builds
> > do
> > > >> not. Downstream consumers can use greedy version specifiers to get
> > their
> > > >> current behavior of "latest".
> > > >>
> > > >> Gradle: 'org.apache.geode:geode-core:1.13.0+'
> > > >> Maven: '<groupId>org.apache.geode</group>'
> > > >> '<artifactId>geode-core</name>'
> > > >> '<version>[1.13.0,1.14.0)</version>'
> > > >>
> > > >> What do you all think? Discuss!
> > > >> -Robert Houghton
> > > >>
> > > >> [1]
> >
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FGEODE-8016&data=02%7C01%7Cdoevans%40vmware.com%7Cf1b924b4bfe04438773a08d7eb033b08%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637236271422916825&sdata=gRQFRLGueu5x8FxRIsdXeKM5PmZLBT6uW8Dgh7FAx2s%3D&reserved=0
> > > >> [2]
> > > >>
> > >
> >
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fconcourse.apachegeode-ci.info%2Fteams%2Fmain%2Fpipelines%2Fapache-develop-examples&data=02%7C01%7Cdoevans%40vmware.com%7Cf1b924b4bfe04438773a08d7eb033b08%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637236271422916825&sdata=g6mfjOLQrkbpS9UWC9QBQL37rcFASIUo5PG0rCs1eAU%3D&reserved=0
> > > >
> > >
> > >
> >
>
Re: [DISCUSS] Publish Builds, not Snapshots
Posted by Robert Houghton <rh...@pivotal.io>.
@donal I do not have an example of a Java project that builds using a
Gradle multi-module build, that cares as much as we do about re-running CI
tests with identical inputs. I wish I did, I would crib from their answer
sheet without shame or remorse.
On Mon, Apr 27, 2020 at 4:37 PM Donal Evans <do...@vmware.com> wrote:
> Do we know if this is an issue that other open source projects have dealt
> with? And if so, is this proposed solution similar to what they might have
> done to remedy it?
> ________________________________
> From: Robert Houghton <rh...@pivotal.io>
> Sent: Monday, April 27, 2020 4:31 PM
> To: dev@geode.apache.org <de...@geode.apache.org>
> Subject: Re: [DISCUSS] Publish Builds, not Snapshots
>
> The artifact would change from "1.13.0-SNAPSHOT" to "1.13.0-build.123".
>
> The number after the "build" slug is auto-incremented by our CI system
> anyway, as the "geode-build-version" semver resource. We are actually doing
> *more* work in Gradle to truncate that number from the current "SNAPSHOT"
> value.
>
> On Mon, Apr 27, 2020 at 3:41 PM Anthony Baker <ab...@pivotal.io> wrote:
>
> > @Robert, can you show some examples of what the build number would be
> > under this proposal? Does 1.13.0-SNAPSHOT become 1.13.0.N where N
> > increments every build?
> >
> > Seems reasonable. Since the consumers of pre-release artifacts are
> either
> > a) this project or b) close related projects for
> > integration-testing-purposes-only I’m not super worried about the ugly
> > syntax.
> >
> >
> > Anthony
> >
> >
> > > On Apr 27, 2020, at 3:25 PM, Jacob Barrett <jb...@pivotal.io>
> wrote:
> > >
> > > It is unfortunate that the Maven/Gradle community hasn’t addressed this
> > glaring issue with SNAPSHOT for decades now (well maybe not decades but
> > certainly decade). It is also unfortunate that the Maven version
> coordinate
> > is ugly. Aside from that I am totally onboard. Yay for reproducible
> builds
> > and predictable downstream builds!
> > >
> > > With SNAPSHOTS in a repo the repository automatically prunes back old
> > builds. Do we have any concerns about having a plethora of builds filling
> > up this new pre-release repository?
> > >
> > > -Jake
> > >
> > >> On Apr 27, 2020, at 3:21 PM, Robert Houghton <rh...@pivotal.io>
> > wrote:
> > >>
> > >> Hello to the community,
> > >>
> > >> tl;dr - Lets publish builds, not snapshots, for repeatable CI builds,
> as
> > >> GEODE-8016[1]. Communicate desired artifact version via the existing
> > >> 'UpdatePassingTokens' job.
> > >>
> > >> I have been working on the Geode build and CI systems for a long time,
> > and
> > >> it has irked me that the geode-examples pipeline[2] does not build and
> > test
> > >> against the latest artifacts from the develop pipeline. Some work has
> > been
> > >> done already to allow this via "composite" builds for local testing
> > without
> > >> needing to publish Geode to your local Maven repository.
> > >>
> > >> From a Concourse CI perspective, composite builds are costly due to
> the
> > >> rebuild of the upstream artifacts. They allow repeatable builds, but
> > only
> > >> by rebuilding those dependencies. Better would be to point to upstream
> > >> artifacts as concrete build versions. SNAPSHOT builds can and do roll
> > >> (invisibly) as new versions are published. Discrete, numbered builds
> do
> > >> not. Downstream consumers can use greedy version specifiers to get
> their
> > >> current behavior of "latest".
> > >>
> > >> Gradle: 'org.apache.geode:geode-core:1.13.0+'
> > >> Maven: '<groupId>org.apache.geode</group>'
> > >> '<artifactId>geode-core</name>'
> > >> '<version>[1.13.0,1.14.0)</version>'
> > >>
> > >> What do you all think? Discuss!
> > >> -Robert Houghton
> > >>
> > >> [1]
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FGEODE-8016&data=02%7C01%7Cdoevans%40vmware.com%7Cf1b924b4bfe04438773a08d7eb033b08%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637236271422916825&sdata=gRQFRLGueu5x8FxRIsdXeKM5PmZLBT6uW8Dgh7FAx2s%3D&reserved=0
> > >> [2]
> > >>
> >
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fconcourse.apachegeode-ci.info%2Fteams%2Fmain%2Fpipelines%2Fapache-develop-examples&data=02%7C01%7Cdoevans%40vmware.com%7Cf1b924b4bfe04438773a08d7eb033b08%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637236271422916825&sdata=g6mfjOLQrkbpS9UWC9QBQL37rcFASIUo5PG0rCs1eAU%3D&reserved=0
> > >
> >
> >
>
Re: [DISCUSS] Publish Builds, not Snapshots
Posted by Donal Evans <do...@vmware.com>.
Do we know if this is an issue that other open source projects have dealt with? And if so, is this proposed solution similar to what they might have done to remedy it?
________________________________
From: Robert Houghton <rh...@pivotal.io>
Sent: Monday, April 27, 2020 4:31 PM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Publish Builds, not Snapshots
The artifact would change from "1.13.0-SNAPSHOT" to "1.13.0-build.123".
The number after the "build" slug is auto-incremented by our CI system
anyway, as the "geode-build-version" semver resource. We are actually doing
*more* work in Gradle to truncate that number from the current "SNAPSHOT"
value.
On Mon, Apr 27, 2020 at 3:41 PM Anthony Baker <ab...@pivotal.io> wrote:
> @Robert, can you show some examples of what the build number would be
> under this proposal? Does 1.13.0-SNAPSHOT become 1.13.0.N where N
> increments every build?
>
> Seems reasonable. Since the consumers of pre-release artifacts are either
> a) this project or b) close related projects for
> integration-testing-purposes-only I’m not super worried about the ugly
> syntax.
>
>
> Anthony
>
>
> > On Apr 27, 2020, at 3:25 PM, Jacob Barrett <jb...@pivotal.io> wrote:
> >
> > It is unfortunate that the Maven/Gradle community hasn’t addressed this
> glaring issue with SNAPSHOT for decades now (well maybe not decades but
> certainly decade). It is also unfortunate that the Maven version coordinate
> is ugly. Aside from that I am totally onboard. Yay for reproducible builds
> and predictable downstream builds!
> >
> > With SNAPSHOTS in a repo the repository automatically prunes back old
> builds. Do we have any concerns about having a plethora of builds filling
> up this new pre-release repository?
> >
> > -Jake
> >
> >> On Apr 27, 2020, at 3:21 PM, Robert Houghton <rh...@pivotal.io>
> wrote:
> >>
> >> Hello to the community,
> >>
> >> tl;dr - Lets publish builds, not snapshots, for repeatable CI builds, as
> >> GEODE-8016[1]. Communicate desired artifact version via the existing
> >> 'UpdatePassingTokens' job.
> >>
> >> I have been working on the Geode build and CI systems for a long time,
> and
> >> it has irked me that the geode-examples pipeline[2] does not build and
> test
> >> against the latest artifacts from the develop pipeline. Some work has
> been
> >> done already to allow this via "composite" builds for local testing
> without
> >> needing to publish Geode to your local Maven repository.
> >>
> >> From a Concourse CI perspective, composite builds are costly due to the
> >> rebuild of the upstream artifacts. They allow repeatable builds, but
> only
> >> by rebuilding those dependencies. Better would be to point to upstream
> >> artifacts as concrete build versions. SNAPSHOT builds can and do roll
> >> (invisibly) as new versions are published. Discrete, numbered builds do
> >> not. Downstream consumers can use greedy version specifiers to get their
> >> current behavior of "latest".
> >>
> >> Gradle: 'org.apache.geode:geode-core:1.13.0+'
> >> Maven: '<groupId>org.apache.geode</group>'
> >> '<artifactId>geode-core</name>'
> >> '<version>[1.13.0,1.14.0)</version>'
> >>
> >> What do you all think? Discuss!
> >> -Robert Houghton
> >>
> >> [1] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FGEODE-8016&data=02%7C01%7Cdoevans%40vmware.com%7Cf1b924b4bfe04438773a08d7eb033b08%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637236271422916825&sdata=gRQFRLGueu5x8FxRIsdXeKM5PmZLBT6uW8Dgh7FAx2s%3D&reserved=0
> >> [2]
> >>
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fconcourse.apachegeode-ci.info%2Fteams%2Fmain%2Fpipelines%2Fapache-develop-examples&data=02%7C01%7Cdoevans%40vmware.com%7Cf1b924b4bfe04438773a08d7eb033b08%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637236271422916825&sdata=g6mfjOLQrkbpS9UWC9QBQL37rcFASIUo5PG0rCs1eAU%3D&reserved=0
> >
>
>
Re: [DISCUSS] Publish Builds, not Snapshots
Posted by Robert Houghton <rh...@pivotal.io>.
I have no current plans on backporting, though it might make the continued
Concourse testing of those branches easier.
There is a PR about this available:
https://github.com/apache/geode/pull/5057
On Thu, Apr 30, 2020 at 9:25 PM Owen Nichols <on...@pivotal.io> wrote:
> It sounds like the only impact is that downstream projects that consume
> -SNAPSHOT builds will need to make one small change to their maven config;
> downstream projects that consume release versions will be unaffected. So
> just let us know when to make that change.
>
> Anytime soon after support/1.13 is cut would be great to bring this change
> to develop. Assuming it goes smoothly on develop, would you envision
> backporting to support/1.13 and support/1.12 as well?
>
> > On Apr 30, 2020, at 8:38 PM, Robert Houghton <rh...@pivotal.io>
> wrote:
> >
> > I don't read any strong negatives to this change. I would like to start
> > polishing a PR on this work, unless anyone has strong opposing feelings.
> > Does anyone feel that this change requires a VOTE?
> >
> > On Tue, Apr 28, 2020 at 10:36 AM Jacob Barrett <jb...@pivotal.io>
> wrote:
> >
> >> Probably for a separate discussion but I would opt for a future where
> the
> >> release manager validates and signs a specific build from produced by
> the
> >> CI. This takes a lot of error away in the what the release manager
> >> currently does, which has resulted in incorrect artifacts being voted
> on.
> >>
> >> In this model the release manager would be certifying a release with
> full
> >> version number, or whatever we choose for the release version number.
> >>
> >> Until then the release manager could just use the short number. In
> >> Gradle/Maven versioning a version without qualifier is always newer than
> >> one with. So 1.13.0 is newer than any 1.13.0-build.123.
> >>
> >> -Jake
> >>
> >>
> >>> On Apr 28, 2020, at 10:24 AM, Anthony Baker <ab...@pivotal.io> wrote:
> >>>
> >>> Note that I’m asking about building on my local system. Will the
> >> version listed in gradle.properties use the full 1.13.0-build.123
> syntax?
> >> How would it get bumped?
> >>>
> >>> Would a release manager end up using just 1.13.0? Hoping for yes :-)
> >>>
> >>> Anthony
> >>>
> >>>> On Apr 28, 2020, at 9:24 AM, Robert Houghton <rh...@pivotal.io>
> >> wrote:
> >>>>
> >>>> @anthony
> >>>> /develop would say 1.13.0-build.230 (as of this morning).
> >>>> Once cut, /release/1.13 would say 1.13.0-build.231, and /develop would
> >>>> switch to 1.14.0-build.1.
> >>>>
> >>>> gfsh would return that full value. That is the artifact version.
> >>>>
> >>>> On Mon, Apr 27, 2020 at 4:38 PM Anthony Baker <ba...@vmware.com>
> >> wrote:
> >>>>
> >>>>> If I build the /develop branch and run `gfsh version` what will it
> >> print?
> >>>>>
> >>>>> If I build the soon-to-be /release/1.13 branch and run `gfsh version`
> >> what
> >>>>> will it print?
> >>>>>
> >>>>> Anthony
> >>>>>
> >>>>>
> >>>>> On 4/27/20, 4:32 PM, "Robert Houghton" <rh...@pivotal.io>
> wrote:
> >>>>>
> >>>>> The artifact would change from "1.13.0-SNAPSHOT" to
> >> "1.13.0-build.123".
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>
> >>
> >>
>
>
Re: [DISCUSS] Publish Builds, not Snapshots
Posted by Owen Nichols <on...@pivotal.io>.
It sounds like the only impact is that downstream projects that consume -SNAPSHOT builds will need to make one small change to their maven config; downstream projects that consume release versions will be unaffected. So just let us know when to make that change.
Anytime soon after support/1.13 is cut would be great to bring this change to develop. Assuming it goes smoothly on develop, would you envision backporting to support/1.13 and support/1.12 as well?
> On Apr 30, 2020, at 8:38 PM, Robert Houghton <rh...@pivotal.io> wrote:
>
> I don't read any strong negatives to this change. I would like to start
> polishing a PR on this work, unless anyone has strong opposing feelings.
> Does anyone feel that this change requires a VOTE?
>
> On Tue, Apr 28, 2020 at 10:36 AM Jacob Barrett <jb...@pivotal.io> wrote:
>
>> Probably for a separate discussion but I would opt for a future where the
>> release manager validates and signs a specific build from produced by the
>> CI. This takes a lot of error away in the what the release manager
>> currently does, which has resulted in incorrect artifacts being voted on.
>>
>> In this model the release manager would be certifying a release with full
>> version number, or whatever we choose for the release version number.
>>
>> Until then the release manager could just use the short number. In
>> Gradle/Maven versioning a version without qualifier is always newer than
>> one with. So 1.13.0 is newer than any 1.13.0-build.123.
>>
>> -Jake
>>
>>
>>> On Apr 28, 2020, at 10:24 AM, Anthony Baker <ab...@pivotal.io> wrote:
>>>
>>> Note that I’m asking about building on my local system. Will the
>> version listed in gradle.properties use the full 1.13.0-build.123 syntax?
>> How would it get bumped?
>>>
>>> Would a release manager end up using just 1.13.0? Hoping for yes :-)
>>>
>>> Anthony
>>>
>>>> On Apr 28, 2020, at 9:24 AM, Robert Houghton <rh...@pivotal.io>
>> wrote:
>>>>
>>>> @anthony
>>>> /develop would say 1.13.0-build.230 (as of this morning).
>>>> Once cut, /release/1.13 would say 1.13.0-build.231, and /develop would
>>>> switch to 1.14.0-build.1.
>>>>
>>>> gfsh would return that full value. That is the artifact version.
>>>>
>>>> On Mon, Apr 27, 2020 at 4:38 PM Anthony Baker <ba...@vmware.com>
>> wrote:
>>>>
>>>>> If I build the /develop branch and run `gfsh version` what will it
>> print?
>>>>>
>>>>> If I build the soon-to-be /release/1.13 branch and run `gfsh version`
>> what
>>>>> will it print?
>>>>>
>>>>> Anthony
>>>>>
>>>>>
>>>>> On 4/27/20, 4:32 PM, "Robert Houghton" <rh...@pivotal.io> wrote:
>>>>>
>>>>> The artifact would change from "1.13.0-SNAPSHOT" to
>> "1.13.0-build.123".
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>
>>
Re: [DISCUSS] Publish Builds, not Snapshots
Posted by Robert Houghton <rh...@pivotal.io>.
I don't read any strong negatives to this change. I would like to start
polishing a PR on this work, unless anyone has strong opposing feelings.
Does anyone feel that this change requires a VOTE?
On Tue, Apr 28, 2020 at 10:36 AM Jacob Barrett <jb...@pivotal.io> wrote:
> Probably for a separate discussion but I would opt for a future where the
> release manager validates and signs a specific build from produced by the
> CI. This takes a lot of error away in the what the release manager
> currently does, which has resulted in incorrect artifacts being voted on.
>
> In this model the release manager would be certifying a release with full
> version number, or whatever we choose for the release version number.
>
> Until then the release manager could just use the short number. In
> Gradle/Maven versioning a version without qualifier is always newer than
> one with. So 1.13.0 is newer than any 1.13.0-build.123.
>
> -Jake
>
>
> > On Apr 28, 2020, at 10:24 AM, Anthony Baker <ab...@pivotal.io> wrote:
> >
> > Note that I’m asking about building on my local system. Will the
> version listed in gradle.properties use the full 1.13.0-build.123 syntax?
> How would it get bumped?
> >
> > Would a release manager end up using just 1.13.0? Hoping for yes :-)
> >
> > Anthony
> >
> >> On Apr 28, 2020, at 9:24 AM, Robert Houghton <rh...@pivotal.io>
> wrote:
> >>
> >> @anthony
> >> /develop would say 1.13.0-build.230 (as of this morning).
> >> Once cut, /release/1.13 would say 1.13.0-build.231, and /develop would
> >> switch to 1.14.0-build.1.
> >>
> >> gfsh would return that full value. That is the artifact version.
> >>
> >> On Mon, Apr 27, 2020 at 4:38 PM Anthony Baker <ba...@vmware.com>
> wrote:
> >>
> >>> If I build the /develop branch and run `gfsh version` what will it
> print?
> >>>
> >>> If I build the soon-to-be /release/1.13 branch and run `gfsh version`
> what
> >>> will it print?
> >>>
> >>> Anthony
> >>>
> >>>
> >>> On 4/27/20, 4:32 PM, "Robert Houghton" <rh...@pivotal.io> wrote:
> >>>
> >>> The artifact would change from "1.13.0-SNAPSHOT" to
> "1.13.0-build.123".
> >>>
> >>>
> >>>
> >>>
> >>>
> >
>
>
Re: [DISCUSS] Publish Builds, not Snapshots
Posted by Jacob Barrett <jb...@pivotal.io>.
Probably for a separate discussion but I would opt for a future where the release manager validates and signs a specific build from produced by the CI. This takes a lot of error away in the what the release manager currently does, which has resulted in incorrect artifacts being voted on.
In this model the release manager would be certifying a release with full version number, or whatever we choose for the release version number.
Until then the release manager could just use the short number. In Gradle/Maven versioning a version without qualifier is always newer than one with. So 1.13.0 is newer than any 1.13.0-build.123.
-Jake
> On Apr 28, 2020, at 10:24 AM, Anthony Baker <ab...@pivotal.io> wrote:
>
> Note that I’m asking about building on my local system. Will the version listed in gradle.properties use the full 1.13.0-build.123 syntax? How would it get bumped?
>
> Would a release manager end up using just 1.13.0? Hoping for yes :-)
>
> Anthony
>
>> On Apr 28, 2020, at 9:24 AM, Robert Houghton <rh...@pivotal.io> wrote:
>>
>> @anthony
>> /develop would say 1.13.0-build.230 (as of this morning).
>> Once cut, /release/1.13 would say 1.13.0-build.231, and /develop would
>> switch to 1.14.0-build.1.
>>
>> gfsh would return that full value. That is the artifact version.
>>
>> On Mon, Apr 27, 2020 at 4:38 PM Anthony Baker <ba...@vmware.com> wrote:
>>
>>> If I build the /develop branch and run `gfsh version` what will it print?
>>>
>>> If I build the soon-to-be /release/1.13 branch and run `gfsh version` what
>>> will it print?
>>>
>>> Anthony
>>>
>>>
>>> On 4/27/20, 4:32 PM, "Robert Houghton" <rh...@pivotal.io> wrote:
>>>
>>> The artifact would change from "1.13.0-SNAPSHOT" to "1.13.0-build.123".
>>>
>>>
>>>
>>>
>>>
>
Re: [DISCUSS] Publish Builds, not Snapshots
Posted by Anthony Baker <ab...@pivotal.io>.
Note that I’m asking about building on my local system. Will the version listed in gradle.properties use the full 1.13.0-build.123 syntax? How would it get bumped?
Would a release manager end up using just 1.13.0? Hoping for yes :-)
Anthony
> On Apr 28, 2020, at 9:24 AM, Robert Houghton <rh...@pivotal.io> wrote:
>
> @anthony
> /develop would say 1.13.0-build.230 (as of this morning).
> Once cut, /release/1.13 would say 1.13.0-build.231, and /develop would
> switch to 1.14.0-build.1.
>
> gfsh would return that full value. That is the artifact version.
>
> On Mon, Apr 27, 2020 at 4:38 PM Anthony Baker <ba...@vmware.com> wrote:
>
>> If I build the /develop branch and run `gfsh version` what will it print?
>>
>> If I build the soon-to-be /release/1.13 branch and run `gfsh version` what
>> will it print?
>>
>> Anthony
>>
>>
>> On 4/27/20, 4:32 PM, "Robert Houghton" <rh...@pivotal.io> wrote:
>>
>> The artifact would change from "1.13.0-SNAPSHOT" to "1.13.0-build.123".
>>
>>
>>
>>
>>
Re: [DISCUSS] Publish Builds, not Snapshots
Posted by Robert Houghton <rh...@pivotal.io>.
@anthony
/develop would say 1.13.0-build.230 (as of this morning).
Once cut, /release/1.13 would say 1.13.0-build.231, and /develop would
switch to 1.14.0-build.1.
gfsh would return that full value. That is the artifact version.
On Mon, Apr 27, 2020 at 4:38 PM Anthony Baker <ba...@vmware.com> wrote:
> If I build the /develop branch and run `gfsh version` what will it print?
>
> If I build the soon-to-be /release/1.13 branch and run `gfsh version` what
> will it print?
>
> Anthony
>
>
> On 4/27/20, 4:32 PM, "Robert Houghton" <rh...@pivotal.io> wrote:
>
> The artifact would change from "1.13.0-SNAPSHOT" to "1.13.0-build.123".
>
>
>
>
>
Re: [DISCUSS] Publish Builds, not Snapshots
Posted by Anthony Baker <ba...@vmware.com>.
If I build the /develop branch and run `gfsh version` what will it print?
If I build the soon-to-be /release/1.13 branch and run `gfsh version` what will it print?
Anthony
On 4/27/20, 4:32 PM, "Robert Houghton" <rh...@pivotal.io> wrote:
The artifact would change from "1.13.0-SNAPSHOT" to "1.13.0-build.123".
Re: [DISCUSS] Publish Builds, not Snapshots
Posted by Robert Houghton <rh...@pivotal.io>.
The artifact would change from "1.13.0-SNAPSHOT" to "1.13.0-build.123".
The number after the "build" slug is auto-incremented by our CI system
anyway, as the "geode-build-version" semver resource. We are actually doing
*more* work in Gradle to truncate that number from the current "SNAPSHOT"
value.
On Mon, Apr 27, 2020 at 3:41 PM Anthony Baker <ab...@pivotal.io> wrote:
> @Robert, can you show some examples of what the build number would be
> under this proposal? Does 1.13.0-SNAPSHOT become 1.13.0.N where N
> increments every build?
>
> Seems reasonable. Since the consumers of pre-release artifacts are either
> a) this project or b) close related projects for
> integration-testing-purposes-only I’m not super worried about the ugly
> syntax.
>
>
> Anthony
>
>
> > On Apr 27, 2020, at 3:25 PM, Jacob Barrett <jb...@pivotal.io> wrote:
> >
> > It is unfortunate that the Maven/Gradle community hasn’t addressed this
> glaring issue with SNAPSHOT for decades now (well maybe not decades but
> certainly decade). It is also unfortunate that the Maven version coordinate
> is ugly. Aside from that I am totally onboard. Yay for reproducible builds
> and predictable downstream builds!
> >
> > With SNAPSHOTS in a repo the repository automatically prunes back old
> builds. Do we have any concerns about having a plethora of builds filling
> up this new pre-release repository?
> >
> > -Jake
> >
> >> On Apr 27, 2020, at 3:21 PM, Robert Houghton <rh...@pivotal.io>
> wrote:
> >>
> >> Hello to the community,
> >>
> >> tl;dr - Lets publish builds, not snapshots, for repeatable CI builds, as
> >> GEODE-8016[1]. Communicate desired artifact version via the existing
> >> 'UpdatePassingTokens' job.
> >>
> >> I have been working on the Geode build and CI systems for a long time,
> and
> >> it has irked me that the geode-examples pipeline[2] does not build and
> test
> >> against the latest artifacts from the develop pipeline. Some work has
> been
> >> done already to allow this via "composite" builds for local testing
> without
> >> needing to publish Geode to your local Maven repository.
> >>
> >> From a Concourse CI perspective, composite builds are costly due to the
> >> rebuild of the upstream artifacts. They allow repeatable builds, but
> only
> >> by rebuilding those dependencies. Better would be to point to upstream
> >> artifacts as concrete build versions. SNAPSHOT builds can and do roll
> >> (invisibly) as new versions are published. Discrete, numbered builds do
> >> not. Downstream consumers can use greedy version specifiers to get their
> >> current behavior of "latest".
> >>
> >> Gradle: 'org.apache.geode:geode-core:1.13.0+'
> >> Maven: '<groupId>org.apache.geode</group>'
> >> '<artifactId>geode-core</name>'
> >> '<version>[1.13.0,1.14.0)</version>'
> >>
> >> What do you all think? Discuss!
> >> -Robert Houghton
> >>
> >> [1] https://issues.apache.org/jira/browse/GEODE-8016
> >> [2]
> >>
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-examples
> >
>
>
Re: [DISCUSS] Publish Builds, not Snapshots
Posted by Anthony Baker <ab...@pivotal.io>.
@Robert, can you show some examples of what the build number would be under this proposal? Does 1.13.0-SNAPSHOT become 1.13.0.N where N increments every build?
Seems reasonable. Since the consumers of pre-release artifacts are either a) this project or b) close related projects for integration-testing-purposes-only I’m not super worried about the ugly syntax.
Anthony
> On Apr 27, 2020, at 3:25 PM, Jacob Barrett <jb...@pivotal.io> wrote:
>
> It is unfortunate that the Maven/Gradle community hasn’t addressed this glaring issue with SNAPSHOT for decades now (well maybe not decades but certainly decade). It is also unfortunate that the Maven version coordinate is ugly. Aside from that I am totally onboard. Yay for reproducible builds and predictable downstream builds!
>
> With SNAPSHOTS in a repo the repository automatically prunes back old builds. Do we have any concerns about having a plethora of builds filling up this new pre-release repository?
>
> -Jake
>
>> On Apr 27, 2020, at 3:21 PM, Robert Houghton <rh...@pivotal.io> wrote:
>>
>> Hello to the community,
>>
>> tl;dr - Lets publish builds, not snapshots, for repeatable CI builds, as
>> GEODE-8016[1]. Communicate desired artifact version via the existing
>> 'UpdatePassingTokens' job.
>>
>> I have been working on the Geode build and CI systems for a long time, and
>> it has irked me that the geode-examples pipeline[2] does not build and test
>> against the latest artifacts from the develop pipeline. Some work has been
>> done already to allow this via "composite" builds for local testing without
>> needing to publish Geode to your local Maven repository.
>>
>> From a Concourse CI perspective, composite builds are costly due to the
>> rebuild of the upstream artifacts. They allow repeatable builds, but only
>> by rebuilding those dependencies. Better would be to point to upstream
>> artifacts as concrete build versions. SNAPSHOT builds can and do roll
>> (invisibly) as new versions are published. Discrete, numbered builds do
>> not. Downstream consumers can use greedy version specifiers to get their
>> current behavior of "latest".
>>
>> Gradle: 'org.apache.geode:geode-core:1.13.0+'
>> Maven: '<groupId>org.apache.geode</group>'
>> '<artifactId>geode-core</name>'
>> '<version>[1.13.0,1.14.0)</version>'
>>
>> What do you all think? Discuss!
>> -Robert Houghton
>>
>> [1] https://issues.apache.org/jira/browse/GEODE-8016
>> [2]
>> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-examples
>
Re: [DISCUSS] Publish Builds, not Snapshots
Posted by Jacob Barrett <jb...@pivotal.io>.
It is unfortunate that the Maven/Gradle community hasn’t addressed this glaring issue with SNAPSHOT for decades now (well maybe not decades but certainly decade). It is also unfortunate that the Maven version coordinate is ugly. Aside from that I am totally onboard. Yay for reproducible builds and predictable downstream builds!
With SNAPSHOTS in a repo the repository automatically prunes back old builds. Do we have any concerns about having a plethora of builds filling up this new pre-release repository?
-Jake
> On Apr 27, 2020, at 3:21 PM, Robert Houghton <rh...@pivotal.io> wrote:
>
> Hello to the community,
>
> tl;dr - Lets publish builds, not snapshots, for repeatable CI builds, as
> GEODE-8016[1]. Communicate desired artifact version via the existing
> 'UpdatePassingTokens' job.
>
> I have been working on the Geode build and CI systems for a long time, and
> it has irked me that the geode-examples pipeline[2] does not build and test
> against the latest artifacts from the develop pipeline. Some work has been
> done already to allow this via "composite" builds for local testing without
> needing to publish Geode to your local Maven repository.
>
> From a Concourse CI perspective, composite builds are costly due to the
> rebuild of the upstream artifacts. They allow repeatable builds, but only
> by rebuilding those dependencies. Better would be to point to upstream
> artifacts as concrete build versions. SNAPSHOT builds can and do roll
> (invisibly) as new versions are published. Discrete, numbered builds do
> not. Downstream consumers can use greedy version specifiers to get their
> current behavior of "latest".
>
> Gradle: 'org.apache.geode:geode-core:1.13.0+'
> Maven: '<groupId>org.apache.geode</group>'
> '<artifactId>geode-core</name>'
> '<version>[1.13.0,1.14.0)</version>'
>
> What do you all think? Discuss!
> -Robert Houghton
>
> [1] https://issues.apache.org/jira/browse/GEODE-8016
> [2]
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-examples