[SUMMARY] Proposal for Spark Release Strategy

[Patrick Wendell <pw...@gmail.com>]

Hey All,

Thanks for everyone who participated in this thread. I've distilled
feedback based on the discussion and wanted to summarize the
conclusions:

- People seem universally +1 on semantic versioning in general.

- People seem universally +1 on having a public merge windows for releases.

- People seem universally +1 on a policy of having associated JIRA's
with features.

- Everyone believes link-level compatiblity should be the goal. Some
people think we should outright promise it now. Others thing we should
either not promise it or promise it later.
--> Compromise: let's do one minor release 1.0->1.1 to convince
ourselves this is possible (some issues with Scala traits will make
this tricky). Then we can codify it in writing. I've created
SPARK-1069 [1] to clearly establish that this is the goal for 1.X
family of releases.

- Some people think we should add particular features before having 1.0.
--> Version 1.X indicates API stability rather than a feature set;
this was clarified.
--> That said, people still have several months to work on features if
they really want to get them in for this release.

I'm going to integrate this feedback and post a tentative version of
the release guidelines to the wiki.

With all this said, I would like to move the master version to
1.0.0-SNAPSHOT as the main concerns with this have been addressed and
clarified. This merely represents a tentative consensus and the
release is still subject to a formal vote amongst PMC members.

[1] https://spark-project.atlassian.net/browse/SPARK-1069

- Patrick

Re: [SUMMARY] Proposal for Spark Release Strategy

[Andy Konwinski <an...@gmail.com>]

Thanks for the summary Patrick. I'm glad that we discussed the options
before pulling the trigger on a version number update (my -1 had only been
about committing a major version update without thorough discussion).
IMO that's been addressed and given the discussion, I'm changing to a +1
for 1.0.0
On Feb 8, 2014 12:56 PM, "Patrick Wendell" <pw...@gmail.com> wrote:

> Hey All,
>
> Thanks for everyone who participated in this thread. I've distilled
> feedback based on the discussion and wanted to summarize the
> conclusions:
>
> - People seem universally +1 on semantic versioning in general.
>
> - People seem universally +1 on having a public merge windows for releases.
>
> - People seem universally +1 on a policy of having associated JIRA's
> with features.
>
> - Everyone believes link-level compatiblity should be the goal. Some
> people think we should outright promise it now. Others thing we should
> either not promise it or promise it later.
> --> Compromise: let's do one minor release 1.0->1.1 to convince
> ourselves this is possible (some issues with Scala traits will make
> this tricky). Then we can codify it in writing. I've created
> SPARK-1069 [1] to clearly establish that this is the goal for 1.X
> family of releases.
>
> - Some people think we should add particular features before having 1.0.
> --> Version 1.X indicates API stability rather than a feature set;
> this was clarified.
> --> That said, people still have several months to work on features if
> they really want to get them in for this release.
>
> I'm going to integrate this feedback and post a tentative version of
> the release guidelines to the wiki.
>
> With all this said, I would like to move the master version to
> 1.0.0-SNAPSHOT as the main concerns with this have been addressed and
> clarified. This merely represents a tentative consensus and the
> release is still subject to a formal vote amongst PMC members.
>
> [1] https://spark-project.atlassian.net/browse/SPARK-1069
>
> - Patrick
>

Re: [SUMMARY] Proposal for Spark Release Strategy

[Henry Saputra <he...@gmail.com>]

:)

Sure thing. I will create JIRA ticket for this.

Thx guys,

Henry

On Saturday, February 8, 2014, Patrick Wendell <pw...@gmail.com> wrote:

> :P - I'm pretty sure this can be done but it will require some work -
> we already use the github API in our merge script and we could hook
> something like that up with the jenkins tests. Henry maybe you could
> create a JIRA for this for Spark 1.0?
>
> - Patrick
>
> On Sat, Feb 8, 2014 at 3:20 PM, Mark Hamstra <mark@clearstorydata.com<javascript:;>>
> wrote:
> > I know that it can be done -- which is different from saying that I know
> how to set it up.
> >
> >
> >> On Feb 8, 2014, at 2:57 PM, Henry Saputra <henry.saputra@gmail.com<javascript:;>>
> wrote:
> >>
> >> Patrick, do you know if there is a way to check if a Github PR's
> >> subject/ title contains JIRA number and will raise warning by the
> >> Jenkins?
> >>
> >> - Henry
> >>
> >>> On Sat, Feb 8, 2014 at 12:56 PM, Patrick Wendell <pwendell@gmail.com<javascript:;>>
> wrote:
> >>> Hey All,
> >>>
> >>> Thanks for everyone who participated in this thread. I've distilled
> >>> feedback based on the discussion and wanted to summarize the
> >>> conclusions:
> >>>
> >>> - People seem universally +1 on semantic versioning in general.
> >>>
> >>> - People seem universally +1 on having a public merge windows for
> releases.
> >>>
> >>> - People seem universally +1 on a policy of having associated JIRA's
> >>> with features.
> >>>
> >>> - Everyone believes link-level compatiblity should be the goal. Some
> >>> people think we should outright promise it now. Others thing we should
> >>> either not promise it or promise it later.
> >>> --> Compromise: let's do one minor release 1.0->1.1 to convince
> >>> ourselves this is possible (some issues with Scala traits will make
> >>> this tricky). Then we can codify it in writing. I've created
> >>> SPARK-1069 [1] to clearly establish that this is the goal for 1.X
> >>> family of releases.
> >>>
> >>> - Some people think we should add particular features before having
> 1.0.
> >>> --> Version 1.X indicates API stability rather than a feature set;
> >>> this was clarified.
> >>> --> That said, people still have several months to work on features if
> >>> they really want to get them in for this release.
> >>>
> >>> I'm going to integrate this feedback and post a tentative version of
> >>> the release guidelines to the wiki.
> >>>
> >>> With all this said, I would like to move the master version to
> >>> 1.0.0-SNAPSHOT as the main concerns with this have been addressed and
> >>> clarified. This merely represents a tentative consensus and the
> >>> release is still subject to a formal vote amongst PMC members.
> >>>
> >>> [1] https://spark-project.atlassian.net/browse/SPARK-1069
> >>>
> >>> - Patrick
>

Re: [SUMMARY] Proposal for Spark Release Strategy

[Henry Saputra <he...@gmail.com>]

Ok, JIRA ticket filed [1] for this one.

- Henry

[1] https://spark-project.atlassian.net/browse/SPARK-1070

On Sat, Feb 8, 2014 at 3:39 PM, Patrick Wendell <pw...@gmail.com> wrote:
> :P - I'm pretty sure this can be done but it will require some work -
> we already use the github API in our merge script and we could hook
> something like that up with the jenkins tests. Henry maybe you could
> create a JIRA for this for Spark 1.0?
>
> - Patrick
>
> On Sat, Feb 8, 2014 at 3:20 PM, Mark Hamstra <ma...@clearstorydata.com> wrote:
>> I know that it can be done -- which is different from saying that I know how to set it up.
>>
>>
>>> On Feb 8, 2014, at 2:57 PM, Henry Saputra <he...@gmail.com> wrote:
>>>
>>> Patrick, do you know if there is a way to check if a Github PR's
>>> subject/ title contains JIRA number and will raise warning by the
>>> Jenkins?
>>>
>>> - Henry
>>>
>>>> On Sat, Feb 8, 2014 at 12:56 PM, Patrick Wendell <pw...@gmail.com> wrote:
>>>> Hey All,
>>>>
>>>> Thanks for everyone who participated in this thread. I've distilled
>>>> feedback based on the discussion and wanted to summarize the
>>>> conclusions:
>>>>
>>>> - People seem universally +1 on semantic versioning in general.
>>>>
>>>> - People seem universally +1 on having a public merge windows for releases.
>>>>
>>>> - People seem universally +1 on a policy of having associated JIRA's
>>>> with features.
>>>>
>>>> - Everyone believes link-level compatiblity should be the goal. Some
>>>> people think we should outright promise it now. Others thing we should
>>>> either not promise it or promise it later.
>>>> --> Compromise: let's do one minor release 1.0->1.1 to convince
>>>> ourselves this is possible (some issues with Scala traits will make
>>>> this tricky). Then we can codify it in writing. I've created
>>>> SPARK-1069 [1] to clearly establish that this is the goal for 1.X
>>>> family of releases.
>>>>
>>>> - Some people think we should add particular features before having 1.0.
>>>> --> Version 1.X indicates API stability rather than a feature set;
>>>> this was clarified.
>>>> --> That said, people still have several months to work on features if
>>>> they really want to get them in for this release.
>>>>
>>>> I'm going to integrate this feedback and post a tentative version of
>>>> the release guidelines to the wiki.
>>>>
>>>> With all this said, I would like to move the master version to
>>>> 1.0.0-SNAPSHOT as the main concerns with this have been addressed and
>>>> clarified. This merely represents a tentative consensus and the
>>>> release is still subject to a formal vote amongst PMC members.
>>>>
>>>> [1] https://spark-project.atlassian.net/browse/SPARK-1069
>>>>
>>>> - Patrick

Re: [SUMMARY] Proposal for Spark Release Strategy

[Patrick Wendell <pw...@gmail.com>]

:P - I'm pretty sure this can be done but it will require some work -
we already use the github API in our merge script and we could hook
something like that up with the jenkins tests. Henry maybe you could
create a JIRA for this for Spark 1.0?

- Patrick

On Sat, Feb 8, 2014 at 3:20 PM, Mark Hamstra <ma...@clearstorydata.com> wrote:
> I know that it can be done -- which is different from saying that I know how to set it up.
>
>
>> On Feb 8, 2014, at 2:57 PM, Henry Saputra <he...@gmail.com> wrote:
>>
>> Patrick, do you know if there is a way to check if a Github PR's
>> subject/ title contains JIRA number and will raise warning by the
>> Jenkins?
>>
>> - Henry
>>
>>> On Sat, Feb 8, 2014 at 12:56 PM, Patrick Wendell <pw...@gmail.com> wrote:
>>> Hey All,
>>>
>>> Thanks for everyone who participated in this thread. I've distilled
>>> feedback based on the discussion and wanted to summarize the
>>> conclusions:
>>>
>>> - People seem universally +1 on semantic versioning in general.
>>>
>>> - People seem universally +1 on having a public merge windows for releases.
>>>
>>> - People seem universally +1 on a policy of having associated JIRA's
>>> with features.
>>>
>>> - Everyone believes link-level compatiblity should be the goal. Some
>>> people think we should outright promise it now. Others thing we should
>>> either not promise it or promise it later.
>>> --> Compromise: let's do one minor release 1.0->1.1 to convince
>>> ourselves this is possible (some issues with Scala traits will make
>>> this tricky). Then we can codify it in writing. I've created
>>> SPARK-1069 [1] to clearly establish that this is the goal for 1.X
>>> family of releases.
>>>
>>> - Some people think we should add particular features before having 1.0.
>>> --> Version 1.X indicates API stability rather than a feature set;
>>> this was clarified.
>>> --> That said, people still have several months to work on features if
>>> they really want to get them in for this release.
>>>
>>> I'm going to integrate this feedback and post a tentative version of
>>> the release guidelines to the wiki.
>>>
>>> With all this said, I would like to move the master version to
>>> 1.0.0-SNAPSHOT as the main concerns with this have been addressed and
>>> clarified. This merely represents a tentative consensus and the
>>> release is still subject to a formal vote amongst PMC members.
>>>
>>> [1] https://spark-project.atlassian.net/browse/SPARK-1069
>>>
>>> - Patrick

Re: [SUMMARY] Proposal for Spark Release Strategy

[Mark Hamstra <ma...@clearstorydata.com>]

I know that it can be done -- which is different from saying that I know how to set it up.


> On Feb 8, 2014, at 2:57 PM, Henry Saputra <he...@gmail.com> wrote:
> 
> Patrick, do you know if there is a way to check if a Github PR's
> subject/ title contains JIRA number and will raise warning by the
> Jenkins?
> 
> - Henry
> 
>> On Sat, Feb 8, 2014 at 12:56 PM, Patrick Wendell <pw...@gmail.com> wrote:
>> Hey All,
>> 
>> Thanks for everyone who participated in this thread. I've distilled
>> feedback based on the discussion and wanted to summarize the
>> conclusions:
>> 
>> - People seem universally +1 on semantic versioning in general.
>> 
>> - People seem universally +1 on having a public merge windows for releases.
>> 
>> - People seem universally +1 on a policy of having associated JIRA's
>> with features.
>> 
>> - Everyone believes link-level compatiblity should be the goal. Some
>> people think we should outright promise it now. Others thing we should
>> either not promise it or promise it later.
>> --> Compromise: let's do one minor release 1.0->1.1 to convince
>> ourselves this is possible (some issues with Scala traits will make
>> this tricky). Then we can codify it in writing. I've created
>> SPARK-1069 [1] to clearly establish that this is the goal for 1.X
>> family of releases.
>> 
>> - Some people think we should add particular features before having 1.0.
>> --> Version 1.X indicates API stability rather than a feature set;
>> this was clarified.
>> --> That said, people still have several months to work on features if
>> they really want to get them in for this release.
>> 
>> I'm going to integrate this feedback and post a tentative version of
>> the release guidelines to the wiki.
>> 
>> With all this said, I would like to move the master version to
>> 1.0.0-SNAPSHOT as the main concerns with this have been addressed and
>> clarified. This merely represents a tentative consensus and the
>> release is still subject to a formal vote amongst PMC members.
>> 
>> [1] https://spark-project.atlassian.net/browse/SPARK-1069
>> 
>> - Patrick

Re: [SUMMARY] Proposal for Spark Release Strategy

[Henry Saputra <he...@gmail.com>]

Patrick, do you know if there is a way to check if a Github PR's
subject/ title contains JIRA number and will raise warning by the
Jenkins?

- Henry

On Sat, Feb 8, 2014 at 12:56 PM, Patrick Wendell <pw...@gmail.com> wrote:
> Hey All,
>
> Thanks for everyone who participated in this thread. I've distilled
> feedback based on the discussion and wanted to summarize the
> conclusions:
>
> - People seem universally +1 on semantic versioning in general.
>
> - People seem universally +1 on having a public merge windows for releases.
>
> - People seem universally +1 on a policy of having associated JIRA's
> with features.
>
> - Everyone believes link-level compatiblity should be the goal. Some
> people think we should outright promise it now. Others thing we should
> either not promise it or promise it later.
> --> Compromise: let's do one minor release 1.0->1.1 to convince
> ourselves this is possible (some issues with Scala traits will make
> this tricky). Then we can codify it in writing. I've created
> SPARK-1069 [1] to clearly establish that this is the goal for 1.X
> family of releases.
>
> - Some people think we should add particular features before having 1.0.
> --> Version 1.X indicates API stability rather than a feature set;
> this was clarified.
> --> That said, people still have several months to work on features if
> they really want to get them in for this release.
>
> I'm going to integrate this feedback and post a tentative version of
> the release guidelines to the wiki.
>
> With all this said, I would like to move the master version to
> 1.0.0-SNAPSHOT as the main concerns with this have been addressed and
> clarified. This merely represents a tentative consensus and the
> release is still subject to a formal vote amongst PMC members.
>
> [1] https://spark-project.atlassian.net/browse/SPARK-1069
>
> - Patrick