You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geode.apache.org by Mark Hanson <mh...@pivotal.io> on 2020/01/14 23:44:03 UTC
Release Manager for 1.12 (February 3rd)
Hello All,
It is that time again. It is time to cut a new release branch for 1.12 on February 3rd.
We need a volunteer! No experience required. Committer status would be helpful, but not required.
In the mean time, we should focus on ensuring the CI is stable and start planning to cut the branch.
I would recommend our handy wiki as a great place to see what you are in for should you choose to volunteer (please do)
https://cwiki.apache.org/confluence/display/GEODE/Releasing+Apache+Geode <https://cwiki.apache.org/confluence/display/GEODE/Releasing+Apache+Geode>
If we could have a volunteer by the end of the week, that would be amazing!
Thanks,
Mark
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Posted by Owen Nichols <on...@pivotal.io>.
The language “sufficient for a user to build and test the release” suggest to me that "./gradlew test” should work (i.e. don’t release *just* src). It would be a slippery slope to construe this guideline as a mandate to include any and all tools, scripts, etc that may ever have been used in conjunction with validating a release.
> On Jan 16, 2020, at 10:23 AM, Dan Smith <ds...@pivotal.io> wrote:
>
> We are supposed to be including all of the source necessary to test Geode
> in the source release [1] - I think that would include benchmarks as well.
>
> I don't really see any compelling reason *not* to include the benchmarks,
> let's go ahead and get them into our source release!
>
> [1]
> http://www.apache.org/legal/release-policy.html#what-must-every-release-contain
>
> On Wed, Jan 15, 2020 at 10:26 PM Owen Nichols <on...@pivotal.io> wrote:
>
>> +1 for no changes
>>
>> On Wed, Jan 15, 2020 at 5:57 PM Jacob Barrett <jb...@pivotal.io> wrote:
>>
>>> We can live in areas of gray that don’t require any changes. Nobody is
>>> asking for benchmarks so let’s not do work to add them. Nobody is
>>> complaining they CI is included so let’s not do work to remove them. Is
>> it
>>> ideal, meh...
>>>
>>>> On Jan 15, 2020, at 5:50 PM, Mark Hanson <mh...@pivotal.io> wrote:
>>>>
>>>> Just my two cents.
>>>>
>>>> I think that we should probably strip CI into a separate repo. The key
>>> indicator is that if something were wrong in the CI yaml, would I hold a
>>> release for that? I think no. So that suggests to me it is a separate
>>> thing. Same goes for benchmarks. If we were failing a benchmark I would
>> be
>>> concerned, but if the script were broken, would I hold the release? I
>> think
>>> no as well.
>>>>
>>>> I think that says that the CI code should also be a separate repo.
>>>>
>>>> Thanks,
>>>> Mark
>>>>
>>>>> On Jan 14, 2020, at 10:21 PM, Jacob Barrett <jb...@pivotal.io>
>>> wrote:
>>>>>
>>>>> Until someone outside of the geode ci community is asking for it I
>> just
>>> don’t see utility in going through the motions of making a release for
>> it.
>>>>>
>>>>>>> On Jan 14, 2020, at 10:13 PM, Owen Nichols <on...@pivotal.io>
>>> wrote:
>>>>>>
>>>>>> The source is already public, so on some level a source release is
>> no
>>> different from a git tag. Benchmarks has matured enough that I think it
>>> makes sense to at least start branching and tagging the geode-benchmarks
>>> repo to capture exactly what was used in that Geode release.
>>>>>>
>>>>>> Others in the dev and user community may find the benchmarks useful
>> in
>>> other ways than we use them. While our focus for CI is on tuning for
>>> repeatability, someone else might just want a load generator to break in
>> a
>>> new cluster or get some rough numbers. Some might want to get under the
>>> hood and tinker and tune, or contribute their own benchmarks, with the
>>> understanding that it’s not a turnkey or standalone product, but a tool
>>> that requires getting your hands dirty.
>>>>>>
>>>>>> Would a “1 page” readme with a few tips on “how to run on a laptop”
>> be
>>> enough to let other interested contributors help get geode-benchmarks to
>> a
>>> “better state”?
>>>>>>
>>>>>>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett <jb...@pivotal.io>
>>> wrote:
>>>>>>>
>>>>>>> I don’t think the benchmarks provide any material benefit to a user
>>> in their current state. They are heavily tuned for our CI process which
>>> relies on very beefy machines. Usage on other hardware will require more
>>> tuning. I don’t think it’s worth putting the source in the release until
>>> they are in a better state.
>>>>>>>
>>>>>>> -Jake
>>>>>>>
>>>>>>>
>>>>>>>>> On Jan 14, 2020, at 4:14 PM, Dan Smith <ds...@pivotal.io> wrote:
>>>>>>>>
>>>>>>>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <onichols@pivotal.io
>>>
>>> wrote:
>>>>>>>>
>>>>>>>>> I believe the desire is to include the source code for
>>> geode-benchmarks as
>>>>>>>>> part of the official geode release, much like how we include
>>> geode-examples.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Oh! I thought you meant running the benchmarks in the release
>>> pipeline - I
>>>>>>>> think last release we were running them but decided they were too
>>> flaky to
>>>>>>>> use.
>>>>>>>>
>>>>>>>> +1 to including the benchmark source in the source release.
>>>>>>>>
>>>>>>>> -Dan
>>>>>>
>>>>
>>>
>>
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Posted by Anilkumar Gingade <ag...@pivotal.io>.
+1 to include the performance benchmark code. It provides an
opportunity for community to use it and develop on it (a must needed when
Geode is termed as performant data product).
On Thu, Jan 16, 2020 at 6:35 PM Robert Houghton <rh...@pivotal.io>
wrote:
> Let's not vote until there is a call to vote, folks...
>
>
>
> On Thu, Jan 16, 2020, 18:31 Jacob Barrett <jb...@pivotal.io> wrote:
>
> > I would characterize my vote as 0. I really don’t care either way. Just
> > sharing I think they have no value in a release.
> >
> > > On Jan 16, 2020, at 6:08 PM, Owen Nichols <on...@pivotal.io> wrote:
> > >
> > > Geode PMC has 52 members. If this were a vote, it looks like the
> > results would have been:
> > > +1: 2 (Anthony, Dan)
> > > -1: 1 (Jake)
> > >
> > > If the next release manager were to go ahead and put geode-benchmarks
> in
> > the Geode 1.12.0 source release, at least 3 PMC members would need to be
> > willing to vote +1. So it sounds like we need a few more of the other 49
> > PMC members to weigh in on this discussion.
> > >
> > > To summarize so far:
> > >
> > > Proposal:
> > > - add a geode-benchmarks-n.n.n-src.tgz artifact to all Geode releases
> > going forward, starting with 1.12.0
> > >
> > > Arguments in favor:
> > > - why not?
> > > - it’s already public
> > > - we should default to including all things
> > > - it might be of interest to the user community
> > > - it might encourage contributions back to further improve it
> > > - it is required by CI, which is already included
> > > - Apache mandates that source releases must include test code too
> > >
> > > Arguments against:
> > > - doing nothing is less work
> > > - it will burden PMC members with additional work to validate and vote
> > on RCs
> > > - nobody outside the dev community has asked for it to be included
> > > - maybe it’s not ready
> > > - maybe it’s not documented well enough
> > > - it’s not needed to use Geode
> > > - Apache's legal separation between dev stuff and public release stuff
> > > - legal or license review may be not have been conducted yet
> > >
> > >
> > >>> On Jan 16, 2020, at 4:48 PM, Dan Smith <ds...@pivotal.io> wrote:
> > >>>
> > >>> If geode-benchmarks is included, that implies that an RC cannot be
> > >> approved until reviewers can successfully run the benchmark suite from
> > the
> > >> geode-benchmarks source distribution. Is that what we want?
> > >>
> > >> I think it would be sufficient to run the tests of the benchmarks, eg
> > >> ./gradlew test
> > >>
> > >>> Deploying CI pipelines and running Benchmarks seems like a prime
> > example
> > >> of things we’d be happy to help others in the community with on the
> dev
> > >> list — but not something we would expect questions about on the user
> > list.
> > >>
> > >> I think it would be valuable to share our benchmarks with the geode
> user
> > >> community. The benchmark framework itself (the harness module) is a
> > fairly
> > >> generic benchmarking framework than can be used to benchmark anything
> > that
> > >> can be spun up using java. The geode-benchmark module has geode
> > benchmarks
> > >> that could be used for testing specific hardware, for example.
> > >>
> > >> -Dan
> > >>
> > >>> On Thu, Jan 16, 2020 at 12:37 PM Owen Nichols <on...@pivotal.io>
> > wrote:
> > >>>
> > >>> When voting on RC candidates, PMC members "are required to download
> the
> > >>> signed source code package, compile it as provided, and test the
> > resulting
> > >>> executable on their own platform”.
> > >>>
> > >>> If geode-benchmarks is included, that implies that an RC cannot be
> > >>> approved until reviewers can successfully run the benchmark suite
> from
> > the
> > >>> geode-benchmarks source distribution. Is that what we want?
> > >>>
> > >>> Similarly, if CI is included, that seems to imply that an RC cannot
> be
> > >>> approved until reviewers can stand up their own pipeline from the
> > geode/ci
> > >>> source distribution. Is that what we want?
> > >>>
> > >>> So far there doesn’t seem to be consensus on what to include in a
> Geode
> > >>> source release, but let’s keep in mind that anything we add to the
> > release
> > >>> becomes an Act Of The Foundation and is held to a higher standard.
> > Apache
> > >>> makes a clear distinction between between development activity and
> > official
> > >>> releases to the public. Development activity is anything that should
> > stay
> > >>> within the dev list. Deploying CI pipelines and running Benchmarks
> > seems
> > >>> like a prime example of things we’d be happy to help others in the
> > >>> community with on the dev list — but not something we would expect
> > >>> questions about on the user list.
> > >>>
> > >>>> On Jan 16, 2020, at 10:23 AM, Dan Smith <ds...@pivotal.io> wrote:
> > >>>>
> > >>>> We are supposed to be including all of the source necessary to test
> > Geode
> > >>>> in the source release [1] - I think that would include benchmarks as
> > >>> well.
> > >>>>
> > >>>> I don't really see any compelling reason *not* to include the
> > benchmarks,
> > >>>> let's go ahead and get them into our source release!
> > >>>>
> > >>>> [1]
> > >>>>
> > >>>
> >
> http://www.apache.org/legal/release-policy.html#what-must-every-release-contain
> > >>>>
> > >>>> On Wed, Jan 15, 2020 at 10:26 PM Owen Nichols <on...@pivotal.io>
> > >>> wrote:
> > >>>>
> > >>>>> +1 for no changes
> > >>>>>
> > >>>>> On Wed, Jan 15, 2020 at 5:57 PM Jacob Barrett <jbarrett@pivotal.io
> >
> > >>> wrote:
> > >>>>>
> > >>>>>> We can live in areas of gray that don’t require any changes.
> Nobody
> > is
> > >>>>>> asking for benchmarks so let’s not do work to add them. Nobody is
> > >>>>>> complaining they CI is included so let’s not do work to remove
> > them. Is
> > >>>>> it
> > >>>>>> ideal, meh...
> > >>>>>>
> > >>>>>>> On Jan 15, 2020, at 5:50 PM, Mark Hanson <mh...@pivotal.io>
> > wrote:
> > >>>>>>>
> > >>>>>>> Just my two cents.
> > >>>>>>>
> > >>>>>>> I think that we should probably strip CI into a separate repo.
> The
> > key
> > >>>>>> indicator is that if something were wrong in the CI yaml, would I
> > hold
> > >>> a
> > >>>>>> release for that? I think no. So that suggests to me it is a
> > separate
> > >>>>>> thing. Same goes for benchmarks. If we were failing a benchmark I
> > would
> > >>>>> be
> > >>>>>> concerned, but if the script were broken, would I hold the
> release?
> > I
> > >>>>> think
> > >>>>>> no as well.
> > >>>>>>>
> > >>>>>>> I think that says that the CI code should also be a separate
> repo.
> > >>>>>>>
> > >>>>>>> Thanks,
> > >>>>>>> Mark
> > >>>>>>>
> > >>>>>>>> On Jan 14, 2020, at 10:21 PM, Jacob Barrett <
> jbarrett@pivotal.io>
> > >>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>> Until someone outside of the geode ci community is asking for
> it I
> > >>>>> just
> > >>>>>> don’t see utility in going through the motions of making a release
> > for
> > >>>>> it.
> > >>>>>>>>
> > >>>>>>>>>> On Jan 14, 2020, at 10:13 PM, Owen Nichols <
> onichols@pivotal.io
> > >
> > >>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>> The source is already public, so on some level a source
> release
> > is
> > >>>>> no
> > >>>>>> different from a git tag. Benchmarks has matured enough that I
> > think
> > >>> it
> > >>>>>> makes sense to at least start branching and tagging the
> > >>> geode-benchmarks
> > >>>>>> repo to capture exactly what was used in that Geode release.
> > >>>>>>>>>
> > >>>>>>>>> Others in the dev and user community may find the benchmarks
> > useful
> > >>>>> in
> > >>>>>> other ways than we use them. While our focus for CI is on tuning
> > for
> > >>>>>> repeatability, someone else might just want a load generator to
> > break
> > >>> in
> > >>>>> a
> > >>>>>> new cluster or get some rough numbers. Some might want to get
> under
> > >>> the
> > >>>>>> hood and tinker and tune, or contribute their own benchmarks, with
> > the
> > >>>>>> understanding that it’s not a turnkey or standalone product, but a
> > tool
> > >>>>>> that requires getting your hands dirty.
> > >>>>>>>>>
> > >>>>>>>>> Would a “1 page” readme with a few tips on “how to run on a
> > laptop”
> > >>>>> be
> > >>>>>> enough to let other interested contributors help get
> > geode-benchmarks
> > >>> to
> > >>>>> a
> > >>>>>> “better state”?
> > >>>>>>>>>
> > >>>>>>>>>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett <
> jbarrett@pivotal.io
> > >
> > >>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> I don’t think the benchmarks provide any material benefit to a
> > user
> > >>>>>> in their current state. They are heavily tuned for our CI process
> > which
> > >>>>>> relies on very beefy machines. Usage on other hardware will
> require
> > >>> more
> > >>>>>> tuning. I don’t think it’s worth putting the source in the release
> > >>> until
> > >>>>>> they are in a better state.
> > >>>>>>>>>>
> > >>>>>>>>>> -Jake
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>>> On Jan 14, 2020, at 4:14 PM, Dan Smith <ds...@pivotal.io>
> > >>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <
> > >>> onichols@pivotal.io
> > >>>>>>
> > >>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> I believe the desire is to include the source code for
> > >>>>>> geode-benchmarks as
> > >>>>>>>>>>>> part of the official geode release, much like how we include
> > >>>>>> geode-examples.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> Oh! I thought you meant running the benchmarks in the release
> > >>>>>> pipeline - I
> > >>>>>>>>>>> think last release we were running them but decided they were
> > too
> > >>>>>> flaky to
> > >>>>>>>>>>> use.
> > >>>>>>>>>>>
> > >>>>>>>>>>> +1 to including the benchmark source in the source release.
> > >>>>>>>>>>>
> > >>>>>>>>>>> -Dan
> > >>>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> > >>>
> > >
> >
>
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Posted by Robert Houghton <rh...@pivotal.io>.
Let's not vote until there is a call to vote, folks...
On Thu, Jan 16, 2020, 18:31 Jacob Barrett <jb...@pivotal.io> wrote:
> I would characterize my vote as 0. I really don’t care either way. Just
> sharing I think they have no value in a release.
>
> > On Jan 16, 2020, at 6:08 PM, Owen Nichols <on...@pivotal.io> wrote:
> >
> > Geode PMC has 52 members. If this were a vote, it looks like the
> results would have been:
> > +1: 2 (Anthony, Dan)
> > -1: 1 (Jake)
> >
> > If the next release manager were to go ahead and put geode-benchmarks in
> the Geode 1.12.0 source release, at least 3 PMC members would need to be
> willing to vote +1. So it sounds like we need a few more of the other 49
> PMC members to weigh in on this discussion.
> >
> > To summarize so far:
> >
> > Proposal:
> > - add a geode-benchmarks-n.n.n-src.tgz artifact to all Geode releases
> going forward, starting with 1.12.0
> >
> > Arguments in favor:
> > - why not?
> > - it’s already public
> > - we should default to including all things
> > - it might be of interest to the user community
> > - it might encourage contributions back to further improve it
> > - it is required by CI, which is already included
> > - Apache mandates that source releases must include test code too
> >
> > Arguments against:
> > - doing nothing is less work
> > - it will burden PMC members with additional work to validate and vote
> on RCs
> > - nobody outside the dev community has asked for it to be included
> > - maybe it’s not ready
> > - maybe it’s not documented well enough
> > - it’s not needed to use Geode
> > - Apache's legal separation between dev stuff and public release stuff
> > - legal or license review may be not have been conducted yet
> >
> >
> >>> On Jan 16, 2020, at 4:48 PM, Dan Smith <ds...@pivotal.io> wrote:
> >>>
> >>> If geode-benchmarks is included, that implies that an RC cannot be
> >> approved until reviewers can successfully run the benchmark suite from
> the
> >> geode-benchmarks source distribution. Is that what we want?
> >>
> >> I think it would be sufficient to run the tests of the benchmarks, eg
> >> ./gradlew test
> >>
> >>> Deploying CI pipelines and running Benchmarks seems like a prime
> example
> >> of things we’d be happy to help others in the community with on the dev
> >> list — but not something we would expect questions about on the user
> list.
> >>
> >> I think it would be valuable to share our benchmarks with the geode user
> >> community. The benchmark framework itself (the harness module) is a
> fairly
> >> generic benchmarking framework than can be used to benchmark anything
> that
> >> can be spun up using java. The geode-benchmark module has geode
> benchmarks
> >> that could be used for testing specific hardware, for example.
> >>
> >> -Dan
> >>
> >>> On Thu, Jan 16, 2020 at 12:37 PM Owen Nichols <on...@pivotal.io>
> wrote:
> >>>
> >>> When voting on RC candidates, PMC members "are required to download the
> >>> signed source code package, compile it as provided, and test the
> resulting
> >>> executable on their own platform”.
> >>>
> >>> If geode-benchmarks is included, that implies that an RC cannot be
> >>> approved until reviewers can successfully run the benchmark suite from
> the
> >>> geode-benchmarks source distribution. Is that what we want?
> >>>
> >>> Similarly, if CI is included, that seems to imply that an RC cannot be
> >>> approved until reviewers can stand up their own pipeline from the
> geode/ci
> >>> source distribution. Is that what we want?
> >>>
> >>> So far there doesn’t seem to be consensus on what to include in a Geode
> >>> source release, but let’s keep in mind that anything we add to the
> release
> >>> becomes an Act Of The Foundation and is held to a higher standard.
> Apache
> >>> makes a clear distinction between between development activity and
> official
> >>> releases to the public. Development activity is anything that should
> stay
> >>> within the dev list. Deploying CI pipelines and running Benchmarks
> seems
> >>> like a prime example of things we’d be happy to help others in the
> >>> community with on the dev list — but not something we would expect
> >>> questions about on the user list.
> >>>
> >>>> On Jan 16, 2020, at 10:23 AM, Dan Smith <ds...@pivotal.io> wrote:
> >>>>
> >>>> We are supposed to be including all of the source necessary to test
> Geode
> >>>> in the source release [1] - I think that would include benchmarks as
> >>> well.
> >>>>
> >>>> I don't really see any compelling reason *not* to include the
> benchmarks,
> >>>> let's go ahead and get them into our source release!
> >>>>
> >>>> [1]
> >>>>
> >>>
> http://www.apache.org/legal/release-policy.html#what-must-every-release-contain
> >>>>
> >>>> On Wed, Jan 15, 2020 at 10:26 PM Owen Nichols <on...@pivotal.io>
> >>> wrote:
> >>>>
> >>>>> +1 for no changes
> >>>>>
> >>>>> On Wed, Jan 15, 2020 at 5:57 PM Jacob Barrett <jb...@pivotal.io>
> >>> wrote:
> >>>>>
> >>>>>> We can live in areas of gray that don’t require any changes. Nobody
> is
> >>>>>> asking for benchmarks so let’s not do work to add them. Nobody is
> >>>>>> complaining they CI is included so let’s not do work to remove
> them. Is
> >>>>> it
> >>>>>> ideal, meh...
> >>>>>>
> >>>>>>> On Jan 15, 2020, at 5:50 PM, Mark Hanson <mh...@pivotal.io>
> wrote:
> >>>>>>>
> >>>>>>> Just my two cents.
> >>>>>>>
> >>>>>>> I think that we should probably strip CI into a separate repo. The
> key
> >>>>>> indicator is that if something were wrong in the CI yaml, would I
> hold
> >>> a
> >>>>>> release for that? I think no. So that suggests to me it is a
> separate
> >>>>>> thing. Same goes for benchmarks. If we were failing a benchmark I
> would
> >>>>> be
> >>>>>> concerned, but if the script were broken, would I hold the release?
> I
> >>>>> think
> >>>>>> no as well.
> >>>>>>>
> >>>>>>> I think that says that the CI code should also be a separate repo.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Mark
> >>>>>>>
> >>>>>>>> On Jan 14, 2020, at 10:21 PM, Jacob Barrett <jb...@pivotal.io>
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Until someone outside of the geode ci community is asking for it I
> >>>>> just
> >>>>>> don’t see utility in going through the motions of making a release
> for
> >>>>> it.
> >>>>>>>>
> >>>>>>>>>> On Jan 14, 2020, at 10:13 PM, Owen Nichols <onichols@pivotal.io
> >
> >>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> The source is already public, so on some level a source release
> is
> >>>>> no
> >>>>>> different from a git tag. Benchmarks has matured enough that I
> think
> >>> it
> >>>>>> makes sense to at least start branching and tagging the
> >>> geode-benchmarks
> >>>>>> repo to capture exactly what was used in that Geode release.
> >>>>>>>>>
> >>>>>>>>> Others in the dev and user community may find the benchmarks
> useful
> >>>>> in
> >>>>>> other ways than we use them. While our focus for CI is on tuning
> for
> >>>>>> repeatability, someone else might just want a load generator to
> break
> >>> in
> >>>>> a
> >>>>>> new cluster or get some rough numbers. Some might want to get under
> >>> the
> >>>>>> hood and tinker and tune, or contribute their own benchmarks, with
> the
> >>>>>> understanding that it’s not a turnkey or standalone product, but a
> tool
> >>>>>> that requires getting your hands dirty.
> >>>>>>>>>
> >>>>>>>>> Would a “1 page” readme with a few tips on “how to run on a
> laptop”
> >>>>> be
> >>>>>> enough to let other interested contributors help get
> geode-benchmarks
> >>> to
> >>>>> a
> >>>>>> “better state”?
> >>>>>>>>>
> >>>>>>>>>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett <jbarrett@pivotal.io
> >
> >>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> I don’t think the benchmarks provide any material benefit to a
> user
> >>>>>> in their current state. They are heavily tuned for our CI process
> which
> >>>>>> relies on very beefy machines. Usage on other hardware will require
> >>> more
> >>>>>> tuning. I don’t think it’s worth putting the source in the release
> >>> until
> >>>>>> they are in a better state.
> >>>>>>>>>>
> >>>>>>>>>> -Jake
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> On Jan 14, 2020, at 4:14 PM, Dan Smith <ds...@pivotal.io>
> >>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <
> >>> onichols@pivotal.io
> >>>>>>
> >>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> I believe the desire is to include the source code for
> >>>>>> geode-benchmarks as
> >>>>>>>>>>>> part of the official geode release, much like how we include
> >>>>>> geode-examples.
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Oh! I thought you meant running the benchmarks in the release
> >>>>>> pipeline - I
> >>>>>>>>>>> think last release we were running them but decided they were
> too
> >>>>>> flaky to
> >>>>>>>>>>> use.
> >>>>>>>>>>>
> >>>>>>>>>>> +1 to including the benchmark source in the source release.
> >>>>>>>>>>>
> >>>>>>>>>>> -Dan
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>
> >>>
> >
>
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Posted by Jacob Barrett <jb...@pivotal.io>.
I would characterize my vote as 0. I really don’t care either way. Just sharing I think they have no value in a release.
> On Jan 16, 2020, at 6:08 PM, Owen Nichols <on...@pivotal.io> wrote:
>
> Geode PMC has 52 members. If this were a vote, it looks like the results would have been:
> +1: 2 (Anthony, Dan)
> -1: 1 (Jake)
>
> If the next release manager were to go ahead and put geode-benchmarks in the Geode 1.12.0 source release, at least 3 PMC members would need to be willing to vote +1. So it sounds like we need a few more of the other 49 PMC members to weigh in on this discussion.
>
> To summarize so far:
>
> Proposal:
> - add a geode-benchmarks-n.n.n-src.tgz artifact to all Geode releases going forward, starting with 1.12.0
>
> Arguments in favor:
> - why not?
> - it’s already public
> - we should default to including all things
> - it might be of interest to the user community
> - it might encourage contributions back to further improve it
> - it is required by CI, which is already included
> - Apache mandates that source releases must include test code too
>
> Arguments against:
> - doing nothing is less work
> - it will burden PMC members with additional work to validate and vote on RCs
> - nobody outside the dev community has asked for it to be included
> - maybe it’s not ready
> - maybe it’s not documented well enough
> - it’s not needed to use Geode
> - Apache's legal separation between dev stuff and public release stuff
> - legal or license review may be not have been conducted yet
>
>
>>> On Jan 16, 2020, at 4:48 PM, Dan Smith <ds...@pivotal.io> wrote:
>>>
>>> If geode-benchmarks is included, that implies that an RC cannot be
>> approved until reviewers can successfully run the benchmark suite from the
>> geode-benchmarks source distribution. Is that what we want?
>>
>> I think it would be sufficient to run the tests of the benchmarks, eg
>> ./gradlew test
>>
>>> Deploying CI pipelines and running Benchmarks seems like a prime example
>> of things we’d be happy to help others in the community with on the dev
>> list — but not something we would expect questions about on the user list.
>>
>> I think it would be valuable to share our benchmarks with the geode user
>> community. The benchmark framework itself (the harness module) is a fairly
>> generic benchmarking framework than can be used to benchmark anything that
>> can be spun up using java. The geode-benchmark module has geode benchmarks
>> that could be used for testing specific hardware, for example.
>>
>> -Dan
>>
>>> On Thu, Jan 16, 2020 at 12:37 PM Owen Nichols <on...@pivotal.io> wrote:
>>>
>>> When voting on RC candidates, PMC members "are required to download the
>>> signed source code package, compile it as provided, and test the resulting
>>> executable on their own platform”.
>>>
>>> If geode-benchmarks is included, that implies that an RC cannot be
>>> approved until reviewers can successfully run the benchmark suite from the
>>> geode-benchmarks source distribution. Is that what we want?
>>>
>>> Similarly, if CI is included, that seems to imply that an RC cannot be
>>> approved until reviewers can stand up their own pipeline from the geode/ci
>>> source distribution. Is that what we want?
>>>
>>> So far there doesn’t seem to be consensus on what to include in a Geode
>>> source release, but let’s keep in mind that anything we add to the release
>>> becomes an Act Of The Foundation and is held to a higher standard. Apache
>>> makes a clear distinction between between development activity and official
>>> releases to the public. Development activity is anything that should stay
>>> within the dev list. Deploying CI pipelines and running Benchmarks seems
>>> like a prime example of things we’d be happy to help others in the
>>> community with on the dev list — but not something we would expect
>>> questions about on the user list.
>>>
>>>> On Jan 16, 2020, at 10:23 AM, Dan Smith <ds...@pivotal.io> wrote:
>>>>
>>>> We are supposed to be including all of the source necessary to test Geode
>>>> in the source release [1] - I think that would include benchmarks as
>>> well.
>>>>
>>>> I don't really see any compelling reason *not* to include the benchmarks,
>>>> let's go ahead and get them into our source release!
>>>>
>>>> [1]
>>>>
>>> http://www.apache.org/legal/release-policy.html#what-must-every-release-contain
>>>>
>>>> On Wed, Jan 15, 2020 at 10:26 PM Owen Nichols <on...@pivotal.io>
>>> wrote:
>>>>
>>>>> +1 for no changes
>>>>>
>>>>> On Wed, Jan 15, 2020 at 5:57 PM Jacob Barrett <jb...@pivotal.io>
>>> wrote:
>>>>>
>>>>>> We can live in areas of gray that don’t require any changes. Nobody is
>>>>>> asking for benchmarks so let’s not do work to add them. Nobody is
>>>>>> complaining they CI is included so let’s not do work to remove them. Is
>>>>> it
>>>>>> ideal, meh...
>>>>>>
>>>>>>> On Jan 15, 2020, at 5:50 PM, Mark Hanson <mh...@pivotal.io> wrote:
>>>>>>>
>>>>>>> Just my two cents.
>>>>>>>
>>>>>>> I think that we should probably strip CI into a separate repo. The key
>>>>>> indicator is that if something were wrong in the CI yaml, would I hold
>>> a
>>>>>> release for that? I think no. So that suggests to me it is a separate
>>>>>> thing. Same goes for benchmarks. If we were failing a benchmark I would
>>>>> be
>>>>>> concerned, but if the script were broken, would I hold the release? I
>>>>> think
>>>>>> no as well.
>>>>>>>
>>>>>>> I think that says that the CI code should also be a separate repo.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Mark
>>>>>>>
>>>>>>>> On Jan 14, 2020, at 10:21 PM, Jacob Barrett <jb...@pivotal.io>
>>>>>> wrote:
>>>>>>>>
>>>>>>>> Until someone outside of the geode ci community is asking for it I
>>>>> just
>>>>>> don’t see utility in going through the motions of making a release for
>>>>> it.
>>>>>>>>
>>>>>>>>>> On Jan 14, 2020, at 10:13 PM, Owen Nichols <on...@pivotal.io>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> The source is already public, so on some level a source release is
>>>>> no
>>>>>> different from a git tag. Benchmarks has matured enough that I think
>>> it
>>>>>> makes sense to at least start branching and tagging the
>>> geode-benchmarks
>>>>>> repo to capture exactly what was used in that Geode release.
>>>>>>>>>
>>>>>>>>> Others in the dev and user community may find the benchmarks useful
>>>>> in
>>>>>> other ways than we use them. While our focus for CI is on tuning for
>>>>>> repeatability, someone else might just want a load generator to break
>>> in
>>>>> a
>>>>>> new cluster or get some rough numbers. Some might want to get under
>>> the
>>>>>> hood and tinker and tune, or contribute their own benchmarks, with the
>>>>>> understanding that it’s not a turnkey or standalone product, but a tool
>>>>>> that requires getting your hands dirty.
>>>>>>>>>
>>>>>>>>> Would a “1 page” readme with a few tips on “how to run on a laptop”
>>>>> be
>>>>>> enough to let other interested contributors help get geode-benchmarks
>>> to
>>>>> a
>>>>>> “better state”?
>>>>>>>>>
>>>>>>>>>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett <jb...@pivotal.io>
>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> I don’t think the benchmarks provide any material benefit to a user
>>>>>> in their current state. They are heavily tuned for our CI process which
>>>>>> relies on very beefy machines. Usage on other hardware will require
>>> more
>>>>>> tuning. I don’t think it’s worth putting the source in the release
>>> until
>>>>>> they are in a better state.
>>>>>>>>>>
>>>>>>>>>> -Jake
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>> On Jan 14, 2020, at 4:14 PM, Dan Smith <ds...@pivotal.io>
>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <
>>> onichols@pivotal.io
>>>>>>
>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I believe the desire is to include the source code for
>>>>>> geode-benchmarks as
>>>>>>>>>>>> part of the official geode release, much like how we include
>>>>>> geode-examples.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Oh! I thought you meant running the benchmarks in the release
>>>>>> pipeline - I
>>>>>>>>>>> think last release we were running them but decided they were too
>>>>>> flaky to
>>>>>>>>>>> use.
>>>>>>>>>>>
>>>>>>>>>>> +1 to including the benchmark source in the source release.
>>>>>>>>>>>
>>>>>>>>>>> -Dan
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>
>>>
>
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Posted by Owen Nichols <on...@pivotal.io>.
Geode PMC has 52 members. If this were a vote, it looks like the results would have been:
+1: 2 (Anthony, Dan)
-1: 1 (Jake)
If the next release manager were to go ahead and put geode-benchmarks in the Geode 1.12.0 source release, at least 3 PMC members would need to be willing to vote +1. So it sounds like we need a few more of the other 49 PMC members to weigh in on this discussion.
To summarize so far:
Proposal:
- add a geode-benchmarks-n.n.n-src.tgz artifact to all Geode releases going forward, starting with 1.12.0
Arguments in favor:
- why not?
- it’s already public
- we should default to including all things
- it might be of interest to the user community
- it might encourage contributions back to further improve it
- it is required by CI, which is already included
- Apache mandates that source releases must include test code too
Arguments against:
- doing nothing is less work
- it will burden PMC members with additional work to validate and vote on RCs
- nobody outside the dev community has asked for it to be included
- maybe it’s not ready
- maybe it’s not documented well enough
- it’s not needed to use Geode
- Apache's legal separation between dev stuff and public release stuff
- legal or license review may be not have been conducted yet
> On Jan 16, 2020, at 4:48 PM, Dan Smith <ds...@pivotal.io> wrote:
>
>> If geode-benchmarks is included, that implies that an RC cannot be
> approved until reviewers can successfully run the benchmark suite from the
> geode-benchmarks source distribution. Is that what we want?
>
> I think it would be sufficient to run the tests of the benchmarks, eg
> ./gradlew test
>
>> Deploying CI pipelines and running Benchmarks seems like a prime example
> of things we’d be happy to help others in the community with on the dev
> list — but not something we would expect questions about on the user list.
>
> I think it would be valuable to share our benchmarks with the geode user
> community. The benchmark framework itself (the harness module) is a fairly
> generic benchmarking framework than can be used to benchmark anything that
> can be spun up using java. The geode-benchmark module has geode benchmarks
> that could be used for testing specific hardware, for example.
>
> -Dan
>
> On Thu, Jan 16, 2020 at 12:37 PM Owen Nichols <on...@pivotal.io> wrote:
>
>> When voting on RC candidates, PMC members "are required to download the
>> signed source code package, compile it as provided, and test the resulting
>> executable on their own platform”.
>>
>> If geode-benchmarks is included, that implies that an RC cannot be
>> approved until reviewers can successfully run the benchmark suite from the
>> geode-benchmarks source distribution. Is that what we want?
>>
>> Similarly, if CI is included, that seems to imply that an RC cannot be
>> approved until reviewers can stand up their own pipeline from the geode/ci
>> source distribution. Is that what we want?
>>
>> So far there doesn’t seem to be consensus on what to include in a Geode
>> source release, but let’s keep in mind that anything we add to the release
>> becomes an Act Of The Foundation and is held to a higher standard. Apache
>> makes a clear distinction between between development activity and official
>> releases to the public. Development activity is anything that should stay
>> within the dev list. Deploying CI pipelines and running Benchmarks seems
>> like a prime example of things we’d be happy to help others in the
>> community with on the dev list — but not something we would expect
>> questions about on the user list.
>>
>>> On Jan 16, 2020, at 10:23 AM, Dan Smith <ds...@pivotal.io> wrote:
>>>
>>> We are supposed to be including all of the source necessary to test Geode
>>> in the source release [1] - I think that would include benchmarks as
>> well.
>>>
>>> I don't really see any compelling reason *not* to include the benchmarks,
>>> let's go ahead and get them into our source release!
>>>
>>> [1]
>>>
>> http://www.apache.org/legal/release-policy.html#what-must-every-release-contain
>>>
>>> On Wed, Jan 15, 2020 at 10:26 PM Owen Nichols <on...@pivotal.io>
>> wrote:
>>>
>>>> +1 for no changes
>>>>
>>>> On Wed, Jan 15, 2020 at 5:57 PM Jacob Barrett <jb...@pivotal.io>
>> wrote:
>>>>
>>>>> We can live in areas of gray that don’t require any changes. Nobody is
>>>>> asking for benchmarks so let’s not do work to add them. Nobody is
>>>>> complaining they CI is included so let’s not do work to remove them. Is
>>>> it
>>>>> ideal, meh...
>>>>>
>>>>>> On Jan 15, 2020, at 5:50 PM, Mark Hanson <mh...@pivotal.io> wrote:
>>>>>>
>>>>>> Just my two cents.
>>>>>>
>>>>>> I think that we should probably strip CI into a separate repo. The key
>>>>> indicator is that if something were wrong in the CI yaml, would I hold
>> a
>>>>> release for that? I think no. So that suggests to me it is a separate
>>>>> thing. Same goes for benchmarks. If we were failing a benchmark I would
>>>> be
>>>>> concerned, but if the script were broken, would I hold the release? I
>>>> think
>>>>> no as well.
>>>>>>
>>>>>> I think that says that the CI code should also be a separate repo.
>>>>>>
>>>>>> Thanks,
>>>>>> Mark
>>>>>>
>>>>>>> On Jan 14, 2020, at 10:21 PM, Jacob Barrett <jb...@pivotal.io>
>>>>> wrote:
>>>>>>>
>>>>>>> Until someone outside of the geode ci community is asking for it I
>>>> just
>>>>> don’t see utility in going through the motions of making a release for
>>>> it.
>>>>>>>
>>>>>>>>> On Jan 14, 2020, at 10:13 PM, Owen Nichols <on...@pivotal.io>
>>>>> wrote:
>>>>>>>>
>>>>>>>> The source is already public, so on some level a source release is
>>>> no
>>>>> different from a git tag. Benchmarks has matured enough that I think
>> it
>>>>> makes sense to at least start branching and tagging the
>> geode-benchmarks
>>>>> repo to capture exactly what was used in that Geode release.
>>>>>>>>
>>>>>>>> Others in the dev and user community may find the benchmarks useful
>>>> in
>>>>> other ways than we use them. While our focus for CI is on tuning for
>>>>> repeatability, someone else might just want a load generator to break
>> in
>>>> a
>>>>> new cluster or get some rough numbers. Some might want to get under
>> the
>>>>> hood and tinker and tune, or contribute their own benchmarks, with the
>>>>> understanding that it’s not a turnkey or standalone product, but a tool
>>>>> that requires getting your hands dirty.
>>>>>>>>
>>>>>>>> Would a “1 page” readme with a few tips on “how to run on a laptop”
>>>> be
>>>>> enough to let other interested contributors help get geode-benchmarks
>> to
>>>> a
>>>>> “better state”?
>>>>>>>>
>>>>>>>>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett <jb...@pivotal.io>
>>>>> wrote:
>>>>>>>>>
>>>>>>>>> I don’t think the benchmarks provide any material benefit to a user
>>>>> in their current state. They are heavily tuned for our CI process which
>>>>> relies on very beefy machines. Usage on other hardware will require
>> more
>>>>> tuning. I don’t think it’s worth putting the source in the release
>> until
>>>>> they are in a better state.
>>>>>>>>>
>>>>>>>>> -Jake
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> On Jan 14, 2020, at 4:14 PM, Dan Smith <ds...@pivotal.io>
>> wrote:
>>>>>>>>>>
>>>>>>>>>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <
>> onichols@pivotal.io
>>>>>
>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> I believe the desire is to include the source code for
>>>>> geode-benchmarks as
>>>>>>>>>>> part of the official geode release, much like how we include
>>>>> geode-examples.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Oh! I thought you meant running the benchmarks in the release
>>>>> pipeline - I
>>>>>>>>>> think last release we were running them but decided they were too
>>>>> flaky to
>>>>>>>>>> use.
>>>>>>>>>>
>>>>>>>>>> +1 to including the benchmark source in the source release.
>>>>>>>>>>
>>>>>>>>>> -Dan
>>>>>>>>
>>>>>>
>>>>>
>>>>
>>
>>
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Posted by Dan Smith <ds...@pivotal.io>.
> If geode-benchmarks is included, that implies that an RC cannot be
approved until reviewers can successfully run the benchmark suite from the
geode-benchmarks source distribution. Is that what we want?
I think it would be sufficient to run the tests of the benchmarks, eg
./gradlew test
> Deploying CI pipelines and running Benchmarks seems like a prime example
of things we’d be happy to help others in the community with on the dev
list — but not something we would expect questions about on the user list.
I think it would be valuable to share our benchmarks with the geode user
community. The benchmark framework itself (the harness module) is a fairly
generic benchmarking framework than can be used to benchmark anything that
can be spun up using java. The geode-benchmark module has geode benchmarks
that could be used for testing specific hardware, for example.
-Dan
On Thu, Jan 16, 2020 at 12:37 PM Owen Nichols <on...@pivotal.io> wrote:
> When voting on RC candidates, PMC members "are required to download the
> signed source code package, compile it as provided, and test the resulting
> executable on their own platform”.
>
> If geode-benchmarks is included, that implies that an RC cannot be
> approved until reviewers can successfully run the benchmark suite from the
> geode-benchmarks source distribution. Is that what we want?
>
> Similarly, if CI is included, that seems to imply that an RC cannot be
> approved until reviewers can stand up their own pipeline from the geode/ci
> source distribution. Is that what we want?
>
> So far there doesn’t seem to be consensus on what to include in a Geode
> source release, but let’s keep in mind that anything we add to the release
> becomes an Act Of The Foundation and is held to a higher standard. Apache
> makes a clear distinction between between development activity and official
> releases to the public. Development activity is anything that should stay
> within the dev list. Deploying CI pipelines and running Benchmarks seems
> like a prime example of things we’d be happy to help others in the
> community with on the dev list — but not something we would expect
> questions about on the user list.
>
> > On Jan 16, 2020, at 10:23 AM, Dan Smith <ds...@pivotal.io> wrote:
> >
> > We are supposed to be including all of the source necessary to test Geode
> > in the source release [1] - I think that would include benchmarks as
> well.
> >
> > I don't really see any compelling reason *not* to include the benchmarks,
> > let's go ahead and get them into our source release!
> >
> > [1]
> >
> http://www.apache.org/legal/release-policy.html#what-must-every-release-contain
> >
> > On Wed, Jan 15, 2020 at 10:26 PM Owen Nichols <on...@pivotal.io>
> wrote:
> >
> >> +1 for no changes
> >>
> >> On Wed, Jan 15, 2020 at 5:57 PM Jacob Barrett <jb...@pivotal.io>
> wrote:
> >>
> >>> We can live in areas of gray that don’t require any changes. Nobody is
> >>> asking for benchmarks so let’s not do work to add them. Nobody is
> >>> complaining they CI is included so let’s not do work to remove them. Is
> >> it
> >>> ideal, meh...
> >>>
> >>>> On Jan 15, 2020, at 5:50 PM, Mark Hanson <mh...@pivotal.io> wrote:
> >>>>
> >>>> Just my two cents.
> >>>>
> >>>> I think that we should probably strip CI into a separate repo. The key
> >>> indicator is that if something were wrong in the CI yaml, would I hold
> a
> >>> release for that? I think no. So that suggests to me it is a separate
> >>> thing. Same goes for benchmarks. If we were failing a benchmark I would
> >> be
> >>> concerned, but if the script were broken, would I hold the release? I
> >> think
> >>> no as well.
> >>>>
> >>>> I think that says that the CI code should also be a separate repo.
> >>>>
> >>>> Thanks,
> >>>> Mark
> >>>>
> >>>>> On Jan 14, 2020, at 10:21 PM, Jacob Barrett <jb...@pivotal.io>
> >>> wrote:
> >>>>>
> >>>>> Until someone outside of the geode ci community is asking for it I
> >> just
> >>> don’t see utility in going through the motions of making a release for
> >> it.
> >>>>>
> >>>>>>> On Jan 14, 2020, at 10:13 PM, Owen Nichols <on...@pivotal.io>
> >>> wrote:
> >>>>>>
> >>>>>> The source is already public, so on some level a source release is
> >> no
> >>> different from a git tag. Benchmarks has matured enough that I think
> it
> >>> makes sense to at least start branching and tagging the
> geode-benchmarks
> >>> repo to capture exactly what was used in that Geode release.
> >>>>>>
> >>>>>> Others in the dev and user community may find the benchmarks useful
> >> in
> >>> other ways than we use them. While our focus for CI is on tuning for
> >>> repeatability, someone else might just want a load generator to break
> in
> >> a
> >>> new cluster or get some rough numbers. Some might want to get under
> the
> >>> hood and tinker and tune, or contribute their own benchmarks, with the
> >>> understanding that it’s not a turnkey or standalone product, but a tool
> >>> that requires getting your hands dirty.
> >>>>>>
> >>>>>> Would a “1 page” readme with a few tips on “how to run on a laptop”
> >> be
> >>> enough to let other interested contributors help get geode-benchmarks
> to
> >> a
> >>> “better state”?
> >>>>>>
> >>>>>>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett <jb...@pivotal.io>
> >>> wrote:
> >>>>>>>
> >>>>>>> I don’t think the benchmarks provide any material benefit to a user
> >>> in their current state. They are heavily tuned for our CI process which
> >>> relies on very beefy machines. Usage on other hardware will require
> more
> >>> tuning. I don’t think it’s worth putting the source in the release
> until
> >>> they are in a better state.
> >>>>>>>
> >>>>>>> -Jake
> >>>>>>>
> >>>>>>>
> >>>>>>>>> On Jan 14, 2020, at 4:14 PM, Dan Smith <ds...@pivotal.io>
> wrote:
> >>>>>>>>
> >>>>>>>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <
> onichols@pivotal.io
> >>>
> >>> wrote:
> >>>>>>>>
> >>>>>>>>> I believe the desire is to include the source code for
> >>> geode-benchmarks as
> >>>>>>>>> part of the official geode release, much like how we include
> >>> geode-examples.
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>> Oh! I thought you meant running the benchmarks in the release
> >>> pipeline - I
> >>>>>>>> think last release we were running them but decided they were too
> >>> flaky to
> >>>>>>>> use.
> >>>>>>>>
> >>>>>>>> +1 to including the benchmark source in the source release.
> >>>>>>>>
> >>>>>>>> -Dan
> >>>>>>
> >>>>
> >>>
> >>
>
>
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Posted by Owen Nichols <on...@pivotal.io>.
When voting on RC candidates, PMC members "are required to download the signed source code package, compile it as provided, and test the resulting executable on their own platform”.
If geode-benchmarks is included, that implies that an RC cannot be approved until reviewers can successfully run the benchmark suite from the geode-benchmarks source distribution. Is that what we want?
Similarly, if CI is included, that seems to imply that an RC cannot be approved until reviewers can stand up their own pipeline from the geode/ci source distribution. Is that what we want?
So far there doesn’t seem to be consensus on what to include in a Geode source release, but let’s keep in mind that anything we add to the release becomes an Act Of The Foundation and is held to a higher standard. Apache makes a clear distinction between between development activity and official releases to the public. Development activity is anything that should stay within the dev list. Deploying CI pipelines and running Benchmarks seems like a prime example of things we’d be happy to help others in the community with on the dev list — but not something we would expect questions about on the user list.
> On Jan 16, 2020, at 10:23 AM, Dan Smith <ds...@pivotal.io> wrote:
>
> We are supposed to be including all of the source necessary to test Geode
> in the source release [1] - I think that would include benchmarks as well.
>
> I don't really see any compelling reason *not* to include the benchmarks,
> let's go ahead and get them into our source release!
>
> [1]
> http://www.apache.org/legal/release-policy.html#what-must-every-release-contain
>
> On Wed, Jan 15, 2020 at 10:26 PM Owen Nichols <on...@pivotal.io> wrote:
>
>> +1 for no changes
>>
>> On Wed, Jan 15, 2020 at 5:57 PM Jacob Barrett <jb...@pivotal.io> wrote:
>>
>>> We can live in areas of gray that don’t require any changes. Nobody is
>>> asking for benchmarks so let’s not do work to add them. Nobody is
>>> complaining they CI is included so let’s not do work to remove them. Is
>> it
>>> ideal, meh...
>>>
>>>> On Jan 15, 2020, at 5:50 PM, Mark Hanson <mh...@pivotal.io> wrote:
>>>>
>>>> Just my two cents.
>>>>
>>>> I think that we should probably strip CI into a separate repo. The key
>>> indicator is that if something were wrong in the CI yaml, would I hold a
>>> release for that? I think no. So that suggests to me it is a separate
>>> thing. Same goes for benchmarks. If we were failing a benchmark I would
>> be
>>> concerned, but if the script were broken, would I hold the release? I
>> think
>>> no as well.
>>>>
>>>> I think that says that the CI code should also be a separate repo.
>>>>
>>>> Thanks,
>>>> Mark
>>>>
>>>>> On Jan 14, 2020, at 10:21 PM, Jacob Barrett <jb...@pivotal.io>
>>> wrote:
>>>>>
>>>>> Until someone outside of the geode ci community is asking for it I
>> just
>>> don’t see utility in going through the motions of making a release for
>> it.
>>>>>
>>>>>>> On Jan 14, 2020, at 10:13 PM, Owen Nichols <on...@pivotal.io>
>>> wrote:
>>>>>>
>>>>>> The source is already public, so on some level a source release is
>> no
>>> different from a git tag. Benchmarks has matured enough that I think it
>>> makes sense to at least start branching and tagging the geode-benchmarks
>>> repo to capture exactly what was used in that Geode release.
>>>>>>
>>>>>> Others in the dev and user community may find the benchmarks useful
>> in
>>> other ways than we use them. While our focus for CI is on tuning for
>>> repeatability, someone else might just want a load generator to break in
>> a
>>> new cluster or get some rough numbers. Some might want to get under the
>>> hood and tinker and tune, or contribute their own benchmarks, with the
>>> understanding that it’s not a turnkey or standalone product, but a tool
>>> that requires getting your hands dirty.
>>>>>>
>>>>>> Would a “1 page” readme with a few tips on “how to run on a laptop”
>> be
>>> enough to let other interested contributors help get geode-benchmarks to
>> a
>>> “better state”?
>>>>>>
>>>>>>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett <jb...@pivotal.io>
>>> wrote:
>>>>>>>
>>>>>>> I don’t think the benchmarks provide any material benefit to a user
>>> in their current state. They are heavily tuned for our CI process which
>>> relies on very beefy machines. Usage on other hardware will require more
>>> tuning. I don’t think it’s worth putting the source in the release until
>>> they are in a better state.
>>>>>>>
>>>>>>> -Jake
>>>>>>>
>>>>>>>
>>>>>>>>> On Jan 14, 2020, at 4:14 PM, Dan Smith <ds...@pivotal.io> wrote:
>>>>>>>>
>>>>>>>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <onichols@pivotal.io
>>>
>>> wrote:
>>>>>>>>
>>>>>>>>> I believe the desire is to include the source code for
>>> geode-benchmarks as
>>>>>>>>> part of the official geode release, much like how we include
>>> geode-examples.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Oh! I thought you meant running the benchmarks in the release
>>> pipeline - I
>>>>>>>> think last release we were running them but decided they were too
>>> flaky to
>>>>>>>> use.
>>>>>>>>
>>>>>>>> +1 to including the benchmark source in the source release.
>>>>>>>>
>>>>>>>> -Dan
>>>>>>
>>>>
>>>
>>
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Posted by Dan Smith <ds...@pivotal.io>.
We are supposed to be including all of the source necessary to test Geode
in the source release [1] - I think that would include benchmarks as well.
I don't really see any compelling reason *not* to include the benchmarks,
let's go ahead and get them into our source release!
[1]
http://www.apache.org/legal/release-policy.html#what-must-every-release-contain
On Wed, Jan 15, 2020 at 10:26 PM Owen Nichols <on...@pivotal.io> wrote:
> +1 for no changes
>
> On Wed, Jan 15, 2020 at 5:57 PM Jacob Barrett <jb...@pivotal.io> wrote:
>
> > We can live in areas of gray that don’t require any changes. Nobody is
> > asking for benchmarks so let’s not do work to add them. Nobody is
> > complaining they CI is included so let’s not do work to remove them. Is
> it
> > ideal, meh...
> >
> > > On Jan 15, 2020, at 5:50 PM, Mark Hanson <mh...@pivotal.io> wrote:
> > >
> > > Just my two cents.
> > >
> > > I think that we should probably strip CI into a separate repo. The key
> > indicator is that if something were wrong in the CI yaml, would I hold a
> > release for that? I think no. So that suggests to me it is a separate
> > thing. Same goes for benchmarks. If we were failing a benchmark I would
> be
> > concerned, but if the script were broken, would I hold the release? I
> think
> > no as well.
> > >
> > > I think that says that the CI code should also be a separate repo.
> > >
> > > Thanks,
> > > Mark
> > >
> > >> On Jan 14, 2020, at 10:21 PM, Jacob Barrett <jb...@pivotal.io>
> > wrote:
> > >>
> > >> Until someone outside of the geode ci community is asking for it I
> just
> > don’t see utility in going through the motions of making a release for
> it.
> > >>
> > >>>> On Jan 14, 2020, at 10:13 PM, Owen Nichols <on...@pivotal.io>
> > wrote:
> > >>>
> > >>> The source is already public, so on some level a source release is
> no
> > different from a git tag. Benchmarks has matured enough that I think it
> > makes sense to at least start branching and tagging the geode-benchmarks
> > repo to capture exactly what was used in that Geode release.
> > >>>
> > >>> Others in the dev and user community may find the benchmarks useful
> in
> > other ways than we use them. While our focus for CI is on tuning for
> > repeatability, someone else might just want a load generator to break in
> a
> > new cluster or get some rough numbers. Some might want to get under the
> > hood and tinker and tune, or contribute their own benchmarks, with the
> > understanding that it’s not a turnkey or standalone product, but a tool
> > that requires getting your hands dirty.
> > >>>
> > >>> Would a “1 page” readme with a few tips on “how to run on a laptop”
> be
> > enough to let other interested contributors help get geode-benchmarks to
> a
> > “better state”?
> > >>>
> > >>>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett <jb...@pivotal.io>
> > wrote:
> > >>>>
> > >>>> I don’t think the benchmarks provide any material benefit to a user
> > in their current state. They are heavily tuned for our CI process which
> > relies on very beefy machines. Usage on other hardware will require more
> > tuning. I don’t think it’s worth putting the source in the release until
> > they are in a better state.
> > >>>>
> > >>>> -Jake
> > >>>>
> > >>>>
> > >>>>>> On Jan 14, 2020, at 4:14 PM, Dan Smith <ds...@pivotal.io> wrote:
> > >>>>>
> > >>>>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <onichols@pivotal.io
> >
> > wrote:
> > >>>>>
> > >>>>>> I believe the desire is to include the source code for
> > geode-benchmarks as
> > >>>>>> part of the official geode release, much like how we include
> > geode-examples.
> > >>>>>>
> > >>>>>
> > >>>>> Oh! I thought you meant running the benchmarks in the release
> > pipeline - I
> > >>>>> think last release we were running them but decided they were too
> > flaky to
> > >>>>> use.
> > >>>>>
> > >>>>> +1 to including the benchmark source in the source release.
> > >>>>>
> > >>>>> -Dan
> > >>>
> > >
> >
>
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Posted by Owen Nichols <on...@pivotal.io>.
+1 for no changes
On Wed, Jan 15, 2020 at 5:57 PM Jacob Barrett <jb...@pivotal.io> wrote:
> We can live in areas of gray that don’t require any changes. Nobody is
> asking for benchmarks so let’s not do work to add them. Nobody is
> complaining they CI is included so let’s not do work to remove them. Is it
> ideal, meh...
>
> > On Jan 15, 2020, at 5:50 PM, Mark Hanson <mh...@pivotal.io> wrote:
> >
> > Just my two cents.
> >
> > I think that we should probably strip CI into a separate repo. The key
> indicator is that if something were wrong in the CI yaml, would I hold a
> release for that? I think no. So that suggests to me it is a separate
> thing. Same goes for benchmarks. If we were failing a benchmark I would be
> concerned, but if the script were broken, would I hold the release? I think
> no as well.
> >
> > I think that says that the CI code should also be a separate repo.
> >
> > Thanks,
> > Mark
> >
> >> On Jan 14, 2020, at 10:21 PM, Jacob Barrett <jb...@pivotal.io>
> wrote:
> >>
> >> Until someone outside of the geode ci community is asking for it I just
> don’t see utility in going through the motions of making a release for it.
> >>
> >>>> On Jan 14, 2020, at 10:13 PM, Owen Nichols <on...@pivotal.io>
> wrote:
> >>>
> >>> The source is already public, so on some level a source release is no
> different from a git tag. Benchmarks has matured enough that I think it
> makes sense to at least start branching and tagging the geode-benchmarks
> repo to capture exactly what was used in that Geode release.
> >>>
> >>> Others in the dev and user community may find the benchmarks useful in
> other ways than we use them. While our focus for CI is on tuning for
> repeatability, someone else might just want a load generator to break in a
> new cluster or get some rough numbers. Some might want to get under the
> hood and tinker and tune, or contribute their own benchmarks, with the
> understanding that it’s not a turnkey or standalone product, but a tool
> that requires getting your hands dirty.
> >>>
> >>> Would a “1 page” readme with a few tips on “how to run on a laptop” be
> enough to let other interested contributors help get geode-benchmarks to a
> “better state”?
> >>>
> >>>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett <jb...@pivotal.io>
> wrote:
> >>>>
> >>>> I don’t think the benchmarks provide any material benefit to a user
> in their current state. They are heavily tuned for our CI process which
> relies on very beefy machines. Usage on other hardware will require more
> tuning. I don’t think it’s worth putting the source in the release until
> they are in a better state.
> >>>>
> >>>> -Jake
> >>>>
> >>>>
> >>>>>> On Jan 14, 2020, at 4:14 PM, Dan Smith <ds...@pivotal.io> wrote:
> >>>>>
> >>>>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <on...@pivotal.io>
> wrote:
> >>>>>
> >>>>>> I believe the desire is to include the source code for
> geode-benchmarks as
> >>>>>> part of the official geode release, much like how we include
> geode-examples.
> >>>>>>
> >>>>>
> >>>>> Oh! I thought you meant running the benchmarks in the release
> pipeline - I
> >>>>> think last release we were running them but decided they were too
> flaky to
> >>>>> use.
> >>>>>
> >>>>> +1 to including the benchmark source in the source release.
> >>>>>
> >>>>> -Dan
> >>>
> >
>
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Posted by Jacob Barrett <jb...@pivotal.io>.
We can live in areas of gray that don’t require any changes. Nobody is asking for benchmarks so let’s not do work to add them. Nobody is complaining they CI is included so let’s not do work to remove them. Is it ideal, meh...
> On Jan 15, 2020, at 5:50 PM, Mark Hanson <mh...@pivotal.io> wrote:
>
> Just my two cents.
>
> I think that we should probably strip CI into a separate repo. The key indicator is that if something were wrong in the CI yaml, would I hold a release for that? I think no. So that suggests to me it is a separate thing. Same goes for benchmarks. If we were failing a benchmark I would be concerned, but if the script were broken, would I hold the release? I think no as well.
>
> I think that says that the CI code should also be a separate repo.
>
> Thanks,
> Mark
>
>> On Jan 14, 2020, at 10:21 PM, Jacob Barrett <jb...@pivotal.io> wrote:
>>
>> Until someone outside of the geode ci community is asking for it I just don’t see utility in going through the motions of making a release for it.
>>
>>>> On Jan 14, 2020, at 10:13 PM, Owen Nichols <on...@pivotal.io> wrote:
>>>
>>> The source is already public, so on some level a source release is no different from a git tag. Benchmarks has matured enough that I think it makes sense to at least start branching and tagging the geode-benchmarks repo to capture exactly what was used in that Geode release.
>>>
>>> Others in the dev and user community may find the benchmarks useful in other ways than we use them. While our focus for CI is on tuning for repeatability, someone else might just want a load generator to break in a new cluster or get some rough numbers. Some might want to get under the hood and tinker and tune, or contribute their own benchmarks, with the understanding that it’s not a turnkey or standalone product, but a tool that requires getting your hands dirty.
>>>
>>> Would a “1 page” readme with a few tips on “how to run on a laptop” be enough to let other interested contributors help get geode-benchmarks to a “better state”?
>>>
>>>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett <jb...@pivotal.io> wrote:
>>>>
>>>> I don’t think the benchmarks provide any material benefit to a user in their current state. They are heavily tuned for our CI process which relies on very beefy machines. Usage on other hardware will require more tuning. I don’t think it’s worth putting the source in the release until they are in a better state.
>>>>
>>>> -Jake
>>>>
>>>>
>>>>>> On Jan 14, 2020, at 4:14 PM, Dan Smith <ds...@pivotal.io> wrote:
>>>>>
>>>>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <on...@pivotal.io> wrote:
>>>>>
>>>>>> I believe the desire is to include the source code for geode-benchmarks as
>>>>>> part of the official geode release, much like how we include geode-examples.
>>>>>>
>>>>>
>>>>> Oh! I thought you meant running the benchmarks in the release pipeline - I
>>>>> think last release we were running them but decided they were too flaky to
>>>>> use.
>>>>>
>>>>> +1 to including the benchmark source in the source release.
>>>>>
>>>>> -Dan
>>>
>
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Posted by Mark Hanson <mh...@pivotal.io>.
Just my two cents.
I think that we should probably strip CI into a separate repo. The key indicator is that if something were wrong in the CI yaml, would I hold a release for that? I think no. So that suggests to me it is a separate thing. Same goes for benchmarks. If we were failing a benchmark I would be concerned, but if the script were broken, would I hold the release? I think no as well.
I think that says that the CI code should also be a separate repo.
Thanks,
Mark
> On Jan 14, 2020, at 10:21 PM, Jacob Barrett <jb...@pivotal.io> wrote:
>
> Until someone outside of the geode ci community is asking for it I just don’t see utility in going through the motions of making a release for it.
>
>> On Jan 14, 2020, at 10:13 PM, Owen Nichols <on...@pivotal.io> wrote:
>>
>> The source is already public, so on some level a source release is no different from a git tag. Benchmarks has matured enough that I think it makes sense to at least start branching and tagging the geode-benchmarks repo to capture exactly what was used in that Geode release.
>>
>> Others in the dev and user community may find the benchmarks useful in other ways than we use them. While our focus for CI is on tuning for repeatability, someone else might just want a load generator to break in a new cluster or get some rough numbers. Some might want to get under the hood and tinker and tune, or contribute their own benchmarks, with the understanding that it’s not a turnkey or standalone product, but a tool that requires getting your hands dirty.
>>
>> Would a “1 page” readme with a few tips on “how to run on a laptop” be enough to let other interested contributors help get geode-benchmarks to a “better state”?
>>
>>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett <jb...@pivotal.io> wrote:
>>>
>>> I don’t think the benchmarks provide any material benefit to a user in their current state. They are heavily tuned for our CI process which relies on very beefy machines. Usage on other hardware will require more tuning. I don’t think it’s worth putting the source in the release until they are in a better state.
>>>
>>> -Jake
>>>
>>>
>>>>> On Jan 14, 2020, at 4:14 PM, Dan Smith <ds...@pivotal.io> wrote:
>>>>
>>>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <on...@pivotal.io> wrote:
>>>>
>>>>> I believe the desire is to include the source code for geode-benchmarks as
>>>>> part of the official geode release, much like how we include geode-examples.
>>>>>
>>>>
>>>> Oh! I thought you meant running the benchmarks in the release pipeline - I
>>>> think last release we were running them but decided they were too flaky to
>>>> use.
>>>>
>>>> +1 to including the benchmark source in the source release.
>>>>
>>>> -Dan
>>
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Posted by Owen Nichols <on...@pivotal.io>.
I can see your argument that geode-benchmarks is strictly part of Geode CI for now, and CI is not “part of Geode” or generally useful to anyone outside the Geode CI community. If so, I think it would also be a good idea to exclude geode/ci from Geode source releases.
> On Jan 14, 2020, at 10:21 PM, Jacob Barrett <jb...@pivotal.io> wrote:
>
> Until someone outside of the geode ci community is asking for it I just don’t see utility in going through the motions of making a release for it.
>
>> On Jan 14, 2020, at 10:13 PM, Owen Nichols <on...@pivotal.io> wrote:
>>
>> The source is already public, so on some level a source release is no different from a git tag. Benchmarks has matured enough that I think it makes sense to at least start branching and tagging the geode-benchmarks repo to capture exactly what was used in that Geode release.
>>
>> Others in the dev and user community may find the benchmarks useful in other ways than we use them. While our focus for CI is on tuning for repeatability, someone else might just want a load generator to break in a new cluster or get some rough numbers. Some might want to get under the hood and tinker and tune, or contribute their own benchmarks, with the understanding that it’s not a turnkey or standalone product, but a tool that requires getting your hands dirty.
>>
>> Would a “1 page” readme with a few tips on “how to run on a laptop” be enough to let other interested contributors help get geode-benchmarks to a “better state”?
>>
>>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett <jb...@pivotal.io> wrote:
>>>
>>> I don’t think the benchmarks provide any material benefit to a user in their current state. They are heavily tuned for our CI process which relies on very beefy machines. Usage on other hardware will require more tuning. I don’t think it’s worth putting the source in the release until they are in a better state.
>>>
>>> -Jake
>>>
>>>
>>>>> On Jan 14, 2020, at 4:14 PM, Dan Smith <ds...@pivotal.io> wrote:
>>>>
>>>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <on...@pivotal.io> wrote:
>>>>
>>>>> I believe the desire is to include the source code for geode-benchmarks as
>>>>> part of the official geode release, much like how we include geode-examples.
>>>>>
>>>>
>>>> Oh! I thought you meant running the benchmarks in the release pipeline - I
>>>> think last release we were running them but decided they were too flaky to
>>>> use.
>>>>
>>>> +1 to including the benchmark source in the source release.
>>>>
>>>> -Dan
>>
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Posted by Jacob Barrett <jb...@pivotal.io>.
Until someone outside of the geode ci community is asking for it I just don’t see utility in going through the motions of making a release for it.
> On Jan 14, 2020, at 10:13 PM, Owen Nichols <on...@pivotal.io> wrote:
>
> The source is already public, so on some level a source release is no different from a git tag. Benchmarks has matured enough that I think it makes sense to at least start branching and tagging the geode-benchmarks repo to capture exactly what was used in that Geode release.
>
> Others in the dev and user community may find the benchmarks useful in other ways than we use them. While our focus for CI is on tuning for repeatability, someone else might just want a load generator to break in a new cluster or get some rough numbers. Some might want to get under the hood and tinker and tune, or contribute their own benchmarks, with the understanding that it’s not a turnkey or standalone product, but a tool that requires getting your hands dirty.
>
> Would a “1 page” readme with a few tips on “how to run on a laptop” be enough to let other interested contributors help get geode-benchmarks to a “better state”?
>
>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett <jb...@pivotal.io> wrote:
>>
>> I don’t think the benchmarks provide any material benefit to a user in their current state. They are heavily tuned for our CI process which relies on very beefy machines. Usage on other hardware will require more tuning. I don’t think it’s worth putting the source in the release until they are in a better state.
>>
>> -Jake
>>
>>
>>>> On Jan 14, 2020, at 4:14 PM, Dan Smith <ds...@pivotal.io> wrote:
>>>
>>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <on...@pivotal.io> wrote:
>>>
>>>> I believe the desire is to include the source code for geode-benchmarks as
>>>> part of the official geode release, much like how we include geode-examples.
>>>>
>>>
>>> Oh! I thought you meant running the benchmarks in the release pipeline - I
>>> think last release we were running them but decided they were too flaky to
>>> use.
>>>
>>> +1 to including the benchmark source in the source release.
>>>
>>> -Dan
>
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Posted by Owen Nichols <on...@pivotal.io>.
The source is already public, so on some level a source release is no different from a git tag. Benchmarks has matured enough that I think it makes sense to at least start branching and tagging the geode-benchmarks repo to capture exactly what was used in that Geode release.
Others in the dev and user community may find the benchmarks useful in other ways than we use them. While our focus for CI is on tuning for repeatability, someone else might just want a load generator to break in a new cluster or get some rough numbers. Some might want to get under the hood and tinker and tune, or contribute their own benchmarks, with the understanding that it’s not a turnkey or standalone product, but a tool that requires getting your hands dirty.
Would a “1 page” readme with a few tips on “how to run on a laptop” be enough to let other interested contributors help get geode-benchmarks to a “better state”?
> On Jan 14, 2020, at 9:38 PM, Jacob Barrett <jb...@pivotal.io> wrote:
>
> I don’t think the benchmarks provide any material benefit to a user in their current state. They are heavily tuned for our CI process which relies on very beefy machines. Usage on other hardware will require more tuning. I don’t think it’s worth putting the source in the release until they are in a better state.
>
> -Jake
>
>
>> On Jan 14, 2020, at 4:14 PM, Dan Smith <ds...@pivotal.io> wrote:
>>
>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <on...@pivotal.io> wrote:
>>
>>> I believe the desire is to include the source code for geode-benchmarks as
>>> part of the official geode release, much like how we include geode-examples.
>>>
>>
>> Oh! I thought you meant running the benchmarks in the release pipeline - I
>> think last release we were running them but decided they were too flaky to
>> use.
>>
>> +1 to including the benchmark source in the source release.
>>
>> -Dan
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Posted by Jacob Barrett <jb...@pivotal.io>.
I don’t think the benchmarks provide any material benefit to a user in their current state. They are heavily tuned for our CI process which relies on very beefy machines. Usage on other hardware will require more tuning. I don’t think it’s worth putting the source in the release until they are in a better state.
-Jake
> On Jan 14, 2020, at 4:14 PM, Dan Smith <ds...@pivotal.io> wrote:
>
> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <on...@pivotal.io> wrote:
>
>> I believe the desire is to include the source code for geode-benchmarks as
>> part of the official geode release, much like how we include geode-examples.
>>
>
> Oh! I thought you meant running the benchmarks in the release pipeline - I
> think last release we were running them but decided they were too flaky to
> use.
>
> +1 to including the benchmark source in the source release.
>
> -Dan
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Posted by Dan Smith <ds...@pivotal.io>.
On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <on...@pivotal.io> wrote:
> I believe the desire is to include the source code for geode-benchmarks as
> part of the official geode release, much like how we include geode-examples.
>
Oh! I thought you meant running the benchmarks in the release pipeline - I
think last release we were running them but decided they were too flaky to
use.
+1 to including the benchmark source in the source release.
-Dan
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Posted by Owen Nichols <on...@pivotal.io>.
I believe the desire is to include the source code for geode-benchmarks as part of the official geode release, much like how we include geode-examples.
> On Jan 14, 2020, at 4:07 PM, Dan Smith <ds...@pivotal.io> wrote:
>
> Looks like the benchmarks have passed for around 30 builds in a row. +1 to
> including them if they don't fail for spurious reasons between now and
> cutting the release.
>
> BTW - Do we have stats on the variance we see in the benchmark numbers? Are
> we getting close to failing occasionally? What is the criteria that will
> cause them to fail? Is this on the wiki somewhere?
>
> -Dan
>
> On Tue, Jan 14, 2020 at 3:58 PM Owen Nichols <on...@pivotal.io> wrote:
>
>> @Anthony has twice proposed[1][2] that we start including geode-benchmarks
>> in our release process. Is this worth considering for the upcoming 1.12.0
>> release?
>>
>> [1]
>> https://lists.apache.org/thread.html/c7bd84b6e6f5464ed674ed447fe8922097237932967b4ed1966e79d3%40%3Cdev.geode.apache.org%3E
>> [2]
>> https://lists.apache.org/thread.html/c9c73e726742c78a1ff9739379a51f7eeab33a05d99531a38b537670%40%3Cdev.geode.apache.org%3E
>>
>>> On Jan 14, 2020, at 3:44 PM, Mark Hanson <mh...@pivotal.io> wrote:
>>>
>>> It is that time again. It is time to cut a new release branch for 1.12
>> on February 3rd.
>>>
>>
>>
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Posted by Dan Smith <ds...@pivotal.io>.
Looks like the benchmarks have passed for around 30 builds in a row. +1 to
including them if they don't fail for spurious reasons between now and
cutting the release.
BTW - Do we have stats on the variance we see in the benchmark numbers? Are
we getting close to failing occasionally? What is the criteria that will
cause them to fail? Is this on the wiki somewhere?
-Dan
On Tue, Jan 14, 2020 at 3:58 PM Owen Nichols <on...@pivotal.io> wrote:
> @Anthony has twice proposed[1][2] that we start including geode-benchmarks
> in our release process. Is this worth considering for the upcoming 1.12.0
> release?
>
> [1]
> https://lists.apache.org/thread.html/c7bd84b6e6f5464ed674ed447fe8922097237932967b4ed1966e79d3%40%3Cdev.geode.apache.org%3E
> [2]
> https://lists.apache.org/thread.html/c9c73e726742c78a1ff9739379a51f7eeab33a05d99531a38b537670%40%3Cdev.geode.apache.org%3E
>
> > On Jan 14, 2020, at 3:44 PM, Mark Hanson <mh...@pivotal.io> wrote:
> >
> > It is that time again. It is time to cut a new release branch for 1.12
> on February 3rd.
> >
>
>
[DISCUSS] include geode-benchmarks in 1.12 release
Posted by Owen Nichols <on...@pivotal.io>.
@Anthony has twice proposed[1][2] that we start including geode-benchmarks in our release process. Is this worth considering for the upcoming 1.12.0 release?
[1] https://lists.apache.org/thread.html/c7bd84b6e6f5464ed674ed447fe8922097237932967b4ed1966e79d3%40%3Cdev.geode.apache.org%3E
[2] https://lists.apache.org/thread.html/c9c73e726742c78a1ff9739379a51f7eeab33a05d99531a38b537670%40%3Cdev.geode.apache.org%3E
> On Jan 14, 2020, at 3:44 PM, Mark Hanson <mh...@pivotal.io> wrote:
>
> It is that time again. It is time to cut a new release branch for 1.12 on February 3rd.
>