You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Matei Zaharia <ma...@gmail.com> on 2013/09/03 03:48:18 UTC
[Licensing check] Spark 0.8.0-incubating RC1
Hi everyone,
In preparation for the 0.8 release, I've put together a first release candidate mostly to help mentors or more experienced Apache people check whether the licensing, POMs, etc are right: http://www.cs.berkeley.edu/~matei/spark-0.8.0-incubating-RC1.tgz. I've tried to write the LICENSE, NOTICE, and POMs as required by Apache. Please take a look if you have a chance -- the ReadMe contains instructions on how to build it and launch the shell.
This first release candidate is still missing some doc pages, so it won't be the final release, but it should have in place all the packaging stuff we'd like for 0.8, and it should be able to run out of the box on Linux, Windows and Mac OS X. Feel free to also test it for functionality; we may merge in a couple more bug fixes but most of the functionality will be there.
Matei
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Matei Zaharia <ma...@gmail.com>.
Cool, thanks for trying it out!
Regarding the pull request model, let's see how it goes. I'd prefer not to have to support two different ways to send patches, but let's see what people ask for -- maybe existing Apache contributors will be more familiar with the JIRA way. I do want to document the model much better on our "how to contribute" page before our last release candidate of 0.8 though.
Matei
On Sep 3, 2013, at 8:48 PM, Tsuyoshi OZAWA <oz...@gmail.com> wrote:
> Hi Matei,
>
> I've confirmed some points on my environment(OS X 10.8.4 and HotSpotVM
> 1.6.0_37):
> 1. sbt/sbt assembly works correctly.
> 2. sbt/sbt package works correctly.
> 2. sbt/sbt test works correctly(its results are grean).
> 4. ./run-example org.apache.spark.examples.SparkPi local 1000
>
> Thanks,
> Tsuyoshi
>
> On Tue, Sep 3, 2013 at 1:32 PM, Henry Saputra <he...@gmail.com> wrote:
>> Hi Matei,
>>
>> Are we "locking" pull requests to github repo by tomorrow?
>> Meaning no more push to GitHub repo for Spark.
>>
>> From your email seems like there will be more potential pull requests for
>> github repo to be merged back to ASF Git repo.
>>
>> - Henry
>>
>> On Monday, September 2, 2013, Matei Zaharia wrote:
>>
>>> Hi everyone,
>>>
>>> In preparation for the 0.8 release, I've put together a first release
>>> candidate mostly to help mentors or more experienced Apache people check
>>> whether the licensing, POMs, etc are right:
>>> http://www.cs.berkeley.edu/~matei/spark-0.8.0-incubating-RC1.tgz. I've
>>> tried to write the LICENSE, NOTICE, and POMs as required by Apache. Please
>>> take a look if you have a chance -- the ReadMe contains instructions on how
>>> to build it and launch the shell.
>>>
>>> This first release candidate is still missing some doc pages, so it won't
>>> be the final release, but it should have in place all the packaging stuff
>>> we'd like for 0.8, and it should be able to run out of the box on Linux,
>>> Windows and Mac OS X. Feel free to also test it for functionality; we may
>>> merge in a couple more bug fixes but most of the functionality will be
>>> there.
>>>
>>> Matei
>
>
>
> --
> - Tsuyoshi
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Tsuyoshi OZAWA <oz...@gmail.com>.
Hi Matei,
I've confirmed some points on my environment(OS X 10.8.4 and HotSpotVM
1.6.0_37):
1. sbt/sbt assembly works correctly.
2. sbt/sbt package works correctly.
2. sbt/sbt test works correctly(its results are grean).
4. ./run-example org.apache.spark.examples.SparkPi local 1000
Thanks,
Tsuyoshi
On Tue, Sep 3, 2013 at 1:32 PM, Henry Saputra <he...@gmail.com> wrote:
> Hi Matei,
>
> Are we "locking" pull requests to github repo by tomorrow?
> Meaning no more push to GitHub repo for Spark.
>
> From your email seems like there will be more potential pull requests for
> github repo to be merged back to ASF Git repo.
>
> - Henry
>
> On Monday, September 2, 2013, Matei Zaharia wrote:
>
>> Hi everyone,
>>
>> In preparation for the 0.8 release, I've put together a first release
>> candidate mostly to help mentors or more experienced Apache people check
>> whether the licensing, POMs, etc are right:
>> http://www.cs.berkeley.edu/~matei/spark-0.8.0-incubating-RC1.tgz. I've
>> tried to write the LICENSE, NOTICE, and POMs as required by Apache. Please
>> take a look if you have a chance -- the ReadMe contains instructions on how
>> to build it and launch the shell.
>>
>> This first release candidate is still missing some doc pages, so it won't
>> be the final release, but it should have in place all the packaging stuff
>> we'd like for 0.8, and it should be able to run out of the box on Linux,
>> Windows and Mac OS X. Feel free to also test it for functionality; we may
>> merge in a couple more bug fixes but most of the functionality will be
>> there.
>>
>> Matei
--
- Tsuyoshi
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Henry Saputra <he...@gmail.com>.
Hi Matei,
Are we "locking" pull requests to github repo by tomorrow?
Meaning no more push to GitHub repo for Spark.
>From your email seems like there will be more potential pull requests for
github repo to be merged back to ASF Git repo.
- Henry
On Monday, September 2, 2013, Matei Zaharia wrote:
> Hi everyone,
>
> In preparation for the 0.8 release, I've put together a first release
> candidate mostly to help mentors or more experienced Apache people check
> whether the licensing, POMs, etc are right:
> http://www.cs.berkeley.edu/~matei/spark-0.8.0-incubating-RC1.tgz. I've
> tried to write the LICENSE, NOTICE, and POMs as required by Apache. Please
> take a look if you have a chance -- the ReadMe contains instructions on how
> to build it and launch the shell.
>
> This first release candidate is still missing some doc pages, so it won't
> be the final release, but it should have in place all the packaging stuff
> we'd like for 0.8, and it should be able to run out of the box on Linux,
> Windows and Mac OS X. Feel free to also test it for functionality; we may
> merge in a couple more bug fixes but most of the functionality will be
> there.
>
> Matei
Re: apache/incubator-spark repo ready to use?
Posted by Henry Saputra <he...@gmail.com>.
Ah yes, sorry I jump the gun there. You moved the website so fast I
forget you already migrate the how to contribute page :)
Mea culpa from me.
- Henry
On Wednesday, September 4, 2013, Matei Zaharia wrote:
> Henry, the problem is that we already have a website:
> http://spark.incubator.apache.org/docs/latest/contributing-to-spark.html.
> That's what most people read, so we need to update that.
>
> Matei
>
> On Sep 4, 2013, at 10:06 AM, Henry Saputra <henry.saputra@gmail.com<javascript:;>>
> wrote:
>
> > Lets stick it to the wiki for now. Some ASF projects also put their how
> to
> > contribute doc in wiki for easy update.
> > Once we get it stable and working well we could move it to website.
> >
> > - Henry
> >
> > On Wednesday, September 4, 2013, Matei Zaharia wrote:
> >
> >> Let me try to put together a doc on this. I wanted to wait for 0.8 to be
> >> out to simplify the process and immediately have a new "how to
> contribute"
> >> doc on the website, but I guess I can host this elsewhere if the release
> >> takes a while.
> >>
> >> Matei
> >>
> >> On Sep 4, 2013, at 8:37 AM, Roman Shaposhnik <rvs@apache.org<javascript:;>
> <javascript:;>>
> >> wrote:
> >>
> >>> On Tue, Sep 3, 2013 at 11:50 PM, Henry Saputra <
> henry.saputra@gmail.com <javascript:;><javascript:;>>
> >> wrote:
> >>>> Matei, I think it would be better to freeze the pull requests against
> >>>> mesos:spark GitHub repo and make announcement to start making pull
> >>>> request against apache/incubator-spark.
> >>>
> >>> +1. That would make a lot of sense.
> >>>
> >>> Thanks,
> >>> Roman.
> >>
> >>
>
>
Re: apache/incubator-spark repo ready to use?
Posted by Matei Zaharia <ma...@gmail.com>.
Henry, the problem is that we already have a website: http://spark.incubator.apache.org/docs/latest/contributing-to-spark.html. That's what most people read, so we need to update that.
Matei
On Sep 4, 2013, at 10:06 AM, Henry Saputra <he...@gmail.com> wrote:
> Lets stick it to the wiki for now. Some ASF projects also put their how to
> contribute doc in wiki for easy update.
> Once we get it stable and working well we could move it to website.
>
> - Henry
>
> On Wednesday, September 4, 2013, Matei Zaharia wrote:
>
>> Let me try to put together a doc on this. I wanted to wait for 0.8 to be
>> out to simplify the process and immediately have a new "how to contribute"
>> doc on the website, but I guess I can host this elsewhere if the release
>> takes a while.
>>
>> Matei
>>
>> On Sep 4, 2013, at 8:37 AM, Roman Shaposhnik <rvs@apache.org<javascript:;>>
>> wrote:
>>
>>> On Tue, Sep 3, 2013 at 11:50 PM, Henry Saputra <henry.saputra@gmail.com<javascript:;>>
>> wrote:
>>>> Matei, I think it would be better to freeze the pull requests against
>>>> mesos:spark GitHub repo and make announcement to start making pull
>>>> request against apache/incubator-spark.
>>>
>>> +1. That would make a lot of sense.
>>>
>>> Thanks,
>>> Roman.
>>
>>
Re: apache/incubator-spark repo ready to use?
Posted by Henry Saputra <he...@gmail.com>.
Lets stick it to the wiki for now. Some ASF projects also put their how to
contribute doc in wiki for easy update.
Once we get it stable and working well we could move it to website.
- Henry
On Wednesday, September 4, 2013, Matei Zaharia wrote:
> Let me try to put together a doc on this. I wanted to wait for 0.8 to be
> out to simplify the process and immediately have a new "how to contribute"
> doc on the website, but I guess I can host this elsewhere if the release
> takes a while.
>
> Matei
>
> On Sep 4, 2013, at 8:37 AM, Roman Shaposhnik <rvs@apache.org<javascript:;>>
> wrote:
>
> > On Tue, Sep 3, 2013 at 11:50 PM, Henry Saputra <henry.saputra@gmail.com<javascript:;>>
> wrote:
> >> Matei, I think it would be better to freeze the pull requests against
> >> mesos:spark GitHub repo and make announcement to start making pull
> >> request against apache/incubator-spark.
> >
> > +1. That would make a lot of sense.
> >
> > Thanks,
> > Roman.
>
>
Re: apache/incubator-spark repo ready to use?
Posted by Matei Zaharia <ma...@gmail.com>.
Let me try to put together a doc on this. I wanted to wait for 0.8 to be out to simplify the process and immediately have a new "how to contribute" doc on the website, but I guess I can host this elsewhere if the release takes a while.
Matei
On Sep 4, 2013, at 8:37 AM, Roman Shaposhnik <rv...@apache.org> wrote:
> On Tue, Sep 3, 2013 at 11:50 PM, Henry Saputra <he...@gmail.com> wrote:
>> Matei, I think it would be better to freeze the pull requests against
>> mesos:spark GitHub repo and make announcement to start making pull
>> request against apache/incubator-spark.
>
> +1. That would make a lot of sense.
>
> Thanks,
> Roman.
Re: apache/incubator-spark repo ready to use?
Posted by Roman Shaposhnik <rv...@apache.org>.
On Tue, Sep 3, 2013 at 11:50 PM, Henry Saputra <he...@gmail.com> wrote:
> Matei, I think it would be better to freeze the pull requests against
> mesos:spark GitHub repo and make announcement to start making pull
> request against apache/incubator-spark.
+1. That would make a lot of sense.
Thanks,
Roman.
Re: apache/incubator-spark repo ready to use?
Posted by Henry Saputra <he...@gmail.com>.
Matei, I think it would be better to freeze the pull requests against
mesos:spark GitHub repo and make announcement to start making pull
request against apache/incubator-spark.
We did the announcement for mailing lists move but I think we need to
do formal repo transition announcement too.
- Henry
On Tue, Sep 3, 2013 at 11:35 PM, Matei Zaharia <ma...@gmail.com> wrote:
>
> > "The plan"? Can we do that now? If I'm putting together a pull request
> > for post-0.8, is there any reason to wait to fork
> > apache/incubator-spark?
>
> Nope, go for it. It would be great if people started doing this now.
>
> Matei
>
Re: apache/incubator-spark repo ready to use?
Posted by Henry Saputra <he...@gmail.com>.
+1 =)
On Tue, Sep 3, 2013 at 11:35 PM, Matei Zaharia <ma...@gmail.com> wrote:
>> "The plan"? Can we do that now? If I'm putting together a pull request
>> for post-0.8, is there any reason to wait to fork
>> apache/incubator-spark?
>
> Nope, go for it. It would be great if people started doing this now.
>
> Matei
>
Re: apache/incubator-spark repo ready to use?
Posted by Matei Zaharia <ma...@gmail.com>.
> "The plan"? Can we do that now? If I'm putting together a pull request
> for post-0.8, is there any reason to wait to fork
> apache/incubator-spark?
Nope, go for it. It would be great if people started doing this now.
Matei
apache/incubator-spark repo ready to use?
Posted by Mike <sp...@good-with-numbers.com>.
Matei wrote:
> the plan is as follows:
>
> - Make pull request against the mirror
>
>> What is going to be the process for making pull requests? Can they
>> be made against the github mirror
>> (https://github.com/apache/incubator-spark), or must we use some
>> other way?
>>
>>> We'll probably use the GitHub repo for the last few changes in this
>>> release and then switch.
"The plan"? Can we do that now? If I'm putting together a pull request
for post-0.8, is there any reason to wait to fork
apache/incubator-spark?
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Matei Zaharia <ma...@gmail.com>.
Yup, use a) for pre-0.8 stuff.
Matei
On Sep 4, 2013, at 12:40 AM, Evan Chan <ev...@ooyala.com> wrote:
> Sorry just to be super clear but "the GitHub repo" (ie PRs for 0.8
> docs) refers to:
>
> a) github.com/mesos/spark
> b) github.com/apache/incubator-spark
>
> I'm assuming a) but don't want to be wrong.... thanks!
>
> On Tue, Sep 3, 2013 at 7:08 PM, Matei Zaharia <ma...@gmail.com> wrote:
>> Yes, please do the PRs against the GitHub repo for now.
>>
>> Matei
>>
>> On Sep 3, 2013, at 4:57 PM, Evan Chan <ev...@ooyala.com> wrote:
>>
>>> Sorry one more clarification.
>>>
>>> For doc pull requests for 0.8 release, should these be done against
>>> the existing mesos/spark repo, or against the mirror at
>>> apache/incubator-spark ?
>>> I'm hoping to clear up a couple things in the docs before the release this week.
>>>
>>> thanks,
>>> -Evan
>>>
>>>
>>> On Tue, Sep 3, 2013 at 2:25 PM, Mark Hamstra <ma...@clearstorydata.com> wrote:
>>>> Okay, so is there any way to get github's compare view to be happy with
>>>> differently-named repositories? What I want is to be able to compare and
>>>> generate a pull request between
>>>> github.com/apache/incubator-spark:masterand, e.g.,
>>>> github.com/markhamstra/spark:myBranch and not need to create a new
>>>> github.com/markhamstra/incubator-spark. It's bad enough that the
>>>> differently-named repos don't show up in the compare-view drop-down
>>>> choices, but I also haven't been able to find a hand-crafted URL that will
>>>> make this work.
>>>>
>>>>
>>>>
>>>> On Tue, Sep 3, 2013 at 1:38 PM, Matei Zaharia <ma...@gmail.com>wrote:
>>>>
>>>>> Yup, the plan is as follows:
>>>>>
>>>>> - Make pull request against the mirror
>>>>> - Code review on GitHub as usual
>>>>> - Whoever merges it will simply merge it into the main Apache repo; when
>>>>> this propagates, the PR will be marked as merged
>>>>>
>>>>> I found at least one other Apache project that did this:
>>>>> http://wiki.apache.org/cordova/ContributorWorkflow.
>>>>>
>>>>> Matei
>>>>>
>>>>> On Sep 3, 2013, at 10:39 AM, Mark Hamstra <ma...@clearstorydata.com> wrote:
>>>>>
>>>>>> What is going to be the process for making pull requests? Can they be
>>>>> made
>>>>>> against the github mirror (https://github.com/apache/incubator-spark),
>>>>> or
>>>>>> must we use some other way?
>>>>>>
>>>>>>
>>>>>> On Tue, Sep 3, 2013 at 10:28 AM, Matei Zaharia <matei.zaharia@gmail.com
>>>>>> wrote:
>>>>>>
>>>>>>> Hi guys,
>>>>>>>
>>>>>>>> So are you planning to release 0.8 from the master branch (which is at
>>>>>>>> a106ed8... now) or from branch-0.8?
>>>>>>>
>>>>>>> Right now the branches are the same in terms of content (though I might
>>>>>>> not have merged the latest changes into 0.8). If we add stuff into
>>>>> master
>>>>>>> that we won't want in 0.8 we'll break that.
>>>>>>>
>>>>>>>> My recommendation is that we start to use the Incubator release
>>>>>>> doc/guide:
>>>>>>>>
>>>>>>>> http://incubator.apache.org/guides/releasemanagement.html
>>>>>>>
>>>>>>> Cool, thanks for the pointer. I'll try to follow the steps there about
>>>>>>> signing.
>>>>>>>
>>>>>>>> Are we "locking" pull requests to github repo by tomorrow?
>>>>>>>> Meaning no more push to GitHub repo for Spark.
>>>>>>>>
>>>>>>>> From your email seems like there will be more potential pull requests
>>>>> for
>>>>>>>> github repo to be merged back to ASF Git repo.
>>>>>>>
>>>>>>> We'll probably use the GitHub repo for the last few changes in this
>>>>>>> release and then switch. The reason is that there's a bit of work to do
>>>>>>> pull requests against the Apache one.
>>>>>>>
>>>>>>> Matei
>>>>>
>>>>>
>>>
>>>
>>>
>>> --
>>> --
>>> Evan Chan
>>> Staff Engineer
>>> ev@ooyala.com |
>>
>
>
>
> --
> --
> Evan Chan
> Staff Engineer
> ev@ooyala.com |
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Evan Chan <ev...@ooyala.com>.
Sorry just to be super clear but "the GitHub repo" (ie PRs for 0.8
docs) refers to:
a) github.com/mesos/spark
b) github.com/apache/incubator-spark
I'm assuming a) but don't want to be wrong.... thanks!
On Tue, Sep 3, 2013 at 7:08 PM, Matei Zaharia <ma...@gmail.com> wrote:
> Yes, please do the PRs against the GitHub repo for now.
>
> Matei
>
> On Sep 3, 2013, at 4:57 PM, Evan Chan <ev...@ooyala.com> wrote:
>
>> Sorry one more clarification.
>>
>> For doc pull requests for 0.8 release, should these be done against
>> the existing mesos/spark repo, or against the mirror at
>> apache/incubator-spark ?
>> I'm hoping to clear up a couple things in the docs before the release this week.
>>
>> thanks,
>> -Evan
>>
>>
>> On Tue, Sep 3, 2013 at 2:25 PM, Mark Hamstra <ma...@clearstorydata.com> wrote:
>>> Okay, so is there any way to get github's compare view to be happy with
>>> differently-named repositories? What I want is to be able to compare and
>>> generate a pull request between
>>> github.com/apache/incubator-spark:masterand, e.g.,
>>> github.com/markhamstra/spark:myBranch and not need to create a new
>>> github.com/markhamstra/incubator-spark. It's bad enough that the
>>> differently-named repos don't show up in the compare-view drop-down
>>> choices, but I also haven't been able to find a hand-crafted URL that will
>>> make this work.
>>>
>>>
>>>
>>> On Tue, Sep 3, 2013 at 1:38 PM, Matei Zaharia <ma...@gmail.com>wrote:
>>>
>>>> Yup, the plan is as follows:
>>>>
>>>> - Make pull request against the mirror
>>>> - Code review on GitHub as usual
>>>> - Whoever merges it will simply merge it into the main Apache repo; when
>>>> this propagates, the PR will be marked as merged
>>>>
>>>> I found at least one other Apache project that did this:
>>>> http://wiki.apache.org/cordova/ContributorWorkflow.
>>>>
>>>> Matei
>>>>
>>>> On Sep 3, 2013, at 10:39 AM, Mark Hamstra <ma...@clearstorydata.com> wrote:
>>>>
>>>>> What is going to be the process for making pull requests? Can they be
>>>> made
>>>>> against the github mirror (https://github.com/apache/incubator-spark),
>>>> or
>>>>> must we use some other way?
>>>>>
>>>>>
>>>>> On Tue, Sep 3, 2013 at 10:28 AM, Matei Zaharia <matei.zaharia@gmail.com
>>>>> wrote:
>>>>>
>>>>>> Hi guys,
>>>>>>
>>>>>>> So are you planning to release 0.8 from the master branch (which is at
>>>>>>> a106ed8... now) or from branch-0.8?
>>>>>>
>>>>>> Right now the branches are the same in terms of content (though I might
>>>>>> not have merged the latest changes into 0.8). If we add stuff into
>>>> master
>>>>>> that we won't want in 0.8 we'll break that.
>>>>>>
>>>>>>> My recommendation is that we start to use the Incubator release
>>>>>> doc/guide:
>>>>>>>
>>>>>>> http://incubator.apache.org/guides/releasemanagement.html
>>>>>>
>>>>>> Cool, thanks for the pointer. I'll try to follow the steps there about
>>>>>> signing.
>>>>>>
>>>>>>> Are we "locking" pull requests to github repo by tomorrow?
>>>>>>> Meaning no more push to GitHub repo for Spark.
>>>>>>>
>>>>>>> From your email seems like there will be more potential pull requests
>>>> for
>>>>>>> github repo to be merged back to ASF Git repo.
>>>>>>
>>>>>> We'll probably use the GitHub repo for the last few changes in this
>>>>>> release and then switch. The reason is that there's a bit of work to do
>>>>>> pull requests against the Apache one.
>>>>>>
>>>>>> Matei
>>>>
>>>>
>>
>>
>>
>> --
>> --
>> Evan Chan
>> Staff Engineer
>> ev@ooyala.com |
>
--
--
Evan Chan
Staff Engineer
ev@ooyala.com |
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Matei Zaharia <ma...@gmail.com>.
Yes, please do the PRs against the GitHub repo for now.
Matei
On Sep 3, 2013, at 4:57 PM, Evan Chan <ev...@ooyala.com> wrote:
> Sorry one more clarification.
>
> For doc pull requests for 0.8 release, should these be done against
> the existing mesos/spark repo, or against the mirror at
> apache/incubator-spark ?
> I'm hoping to clear up a couple things in the docs before the release this week.
>
> thanks,
> -Evan
>
>
> On Tue, Sep 3, 2013 at 2:25 PM, Mark Hamstra <ma...@clearstorydata.com> wrote:
>> Okay, so is there any way to get github's compare view to be happy with
>> differently-named repositories? What I want is to be able to compare and
>> generate a pull request between
>> github.com/apache/incubator-spark:masterand, e.g.,
>> github.com/markhamstra/spark:myBranch and not need to create a new
>> github.com/markhamstra/incubator-spark. It's bad enough that the
>> differently-named repos don't show up in the compare-view drop-down
>> choices, but I also haven't been able to find a hand-crafted URL that will
>> make this work.
>>
>>
>>
>> On Tue, Sep 3, 2013 at 1:38 PM, Matei Zaharia <ma...@gmail.com>wrote:
>>
>>> Yup, the plan is as follows:
>>>
>>> - Make pull request against the mirror
>>> - Code review on GitHub as usual
>>> - Whoever merges it will simply merge it into the main Apache repo; when
>>> this propagates, the PR will be marked as merged
>>>
>>> I found at least one other Apache project that did this:
>>> http://wiki.apache.org/cordova/ContributorWorkflow.
>>>
>>> Matei
>>>
>>> On Sep 3, 2013, at 10:39 AM, Mark Hamstra <ma...@clearstorydata.com> wrote:
>>>
>>>> What is going to be the process for making pull requests? Can they be
>>> made
>>>> against the github mirror (https://github.com/apache/incubator-spark),
>>> or
>>>> must we use some other way?
>>>>
>>>>
>>>> On Tue, Sep 3, 2013 at 10:28 AM, Matei Zaharia <matei.zaharia@gmail.com
>>>> wrote:
>>>>
>>>>> Hi guys,
>>>>>
>>>>>> So are you planning to release 0.8 from the master branch (which is at
>>>>>> a106ed8... now) or from branch-0.8?
>>>>>
>>>>> Right now the branches are the same in terms of content (though I might
>>>>> not have merged the latest changes into 0.8). If we add stuff into
>>> master
>>>>> that we won't want in 0.8 we'll break that.
>>>>>
>>>>>> My recommendation is that we start to use the Incubator release
>>>>> doc/guide:
>>>>>>
>>>>>> http://incubator.apache.org/guides/releasemanagement.html
>>>>>
>>>>> Cool, thanks for the pointer. I'll try to follow the steps there about
>>>>> signing.
>>>>>
>>>>>> Are we "locking" pull requests to github repo by tomorrow?
>>>>>> Meaning no more push to GitHub repo for Spark.
>>>>>>
>>>>>> From your email seems like there will be more potential pull requests
>>> for
>>>>>> github repo to be merged back to ASF Git repo.
>>>>>
>>>>> We'll probably use the GitHub repo for the last few changes in this
>>>>> release and then switch. The reason is that there's a bit of work to do
>>>>> pull requests against the Apache one.
>>>>>
>>>>> Matei
>>>
>>>
>
>
>
> --
> --
> Evan Chan
> Staff Engineer
> ev@ooyala.com |
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Evan Chan <ev...@ooyala.com>.
Matei,
On Tue, Sep 3, 2013 at 7:09 PM, Matei Zaharia <ma...@gmail.com> wrote:
> BTW, what docs are you planning to write? Something on make-distribution.sh would be nice.
Yes, I was planning to write something on make-distribution.sh, and
reference it in the standalone and Mesos deploy guides.
I was also going to document public methods in SparkContext that have
not been documented before, such as getPersistentRdds,
getExecutorStatus etc. Some folks on my team don't realize that such
methods existed as they were not in the doc.
-Evan
>
> Matei
>
> On Sep 3, 2013, at 4:57 PM, Evan Chan <ev...@ooyala.com> wrote:
>
>> Sorry one more clarification.
>>
>> For doc pull requests for 0.8 release, should these be done against
>> the existing mesos/spark repo, or against the mirror at
>> apache/incubator-spark ?
>> I'm hoping to clear up a couple things in the docs before the release this week.
>>
>> thanks,
>> -Evan
>>
>>
>> On Tue, Sep 3, 2013 at 2:25 PM, Mark Hamstra <ma...@clearstorydata.com> wrote:
>>> Okay, so is there any way to get github's compare view to be happy with
>>> differently-named repositories? What I want is to be able to compare and
>>> generate a pull request between
>>> github.com/apache/incubator-spark:masterand, e.g.,
>>> github.com/markhamstra/spark:myBranch and not need to create a new
>>> github.com/markhamstra/incubator-spark. It's bad enough that the
>>> differently-named repos don't show up in the compare-view drop-down
>>> choices, but I also haven't been able to find a hand-crafted URL that will
>>> make this work.
>>>
>>>
>>>
>>> On Tue, Sep 3, 2013 at 1:38 PM, Matei Zaharia <ma...@gmail.com>wrote:
>>>
>>>> Yup, the plan is as follows:
>>>>
>>>> - Make pull request against the mirror
>>>> - Code review on GitHub as usual
>>>> - Whoever merges it will simply merge it into the main Apache repo; when
>>>> this propagates, the PR will be marked as merged
>>>>
>>>> I found at least one other Apache project that did this:
>>>> http://wiki.apache.org/cordova/ContributorWorkflow.
>>>>
>>>> Matei
>>>>
>>>> On Sep 3, 2013, at 10:39 AM, Mark Hamstra <ma...@clearstorydata.com> wrote:
>>>>
>>>>> What is going to be the process for making pull requests? Can they be
>>>> made
>>>>> against the github mirror (https://github.com/apache/incubator-spark),
>>>> or
>>>>> must we use some other way?
>>>>>
>>>>>
>>>>> On Tue, Sep 3, 2013 at 10:28 AM, Matei Zaharia <matei.zaharia@gmail.com
>>>>> wrote:
>>>>>
>>>>>> Hi guys,
>>>>>>
>>>>>>> So are you planning to release 0.8 from the master branch (which is at
>>>>>>> a106ed8... now) or from branch-0.8?
>>>>>>
>>>>>> Right now the branches are the same in terms of content (though I might
>>>>>> not have merged the latest changes into 0.8). If we add stuff into
>>>> master
>>>>>> that we won't want in 0.8 we'll break that.
>>>>>>
>>>>>>> My recommendation is that we start to use the Incubator release
>>>>>> doc/guide:
>>>>>>>
>>>>>>> http://incubator.apache.org/guides/releasemanagement.html
>>>>>>
>>>>>> Cool, thanks for the pointer. I'll try to follow the steps there about
>>>>>> signing.
>>>>>>
>>>>>>> Are we "locking" pull requests to github repo by tomorrow?
>>>>>>> Meaning no more push to GitHub repo for Spark.
>>>>>>>
>>>>>>> From your email seems like there will be more potential pull requests
>>>> for
>>>>>>> github repo to be merged back to ASF Git repo.
>>>>>>
>>>>>> We'll probably use the GitHub repo for the last few changes in this
>>>>>> release and then switch. The reason is that there's a bit of work to do
>>>>>> pull requests against the Apache one.
>>>>>>
>>>>>> Matei
>>>>
>>>>
>>
>>
>>
>> --
>> --
>> Evan Chan
>> Staff Engineer
>> ev@ooyala.com |
>
--
--
Evan Chan
Staff Engineer
ev@ooyala.com |
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Matei Zaharia <ma...@gmail.com>.
BTW, what docs are you planning to write? Something on make-distribution.sh would be nice.
Matei
On Sep 3, 2013, at 4:57 PM, Evan Chan <ev...@ooyala.com> wrote:
> Sorry one more clarification.
>
> For doc pull requests for 0.8 release, should these be done against
> the existing mesos/spark repo, or against the mirror at
> apache/incubator-spark ?
> I'm hoping to clear up a couple things in the docs before the release this week.
>
> thanks,
> -Evan
>
>
> On Tue, Sep 3, 2013 at 2:25 PM, Mark Hamstra <ma...@clearstorydata.com> wrote:
>> Okay, so is there any way to get github's compare view to be happy with
>> differently-named repositories? What I want is to be able to compare and
>> generate a pull request between
>> github.com/apache/incubator-spark:masterand, e.g.,
>> github.com/markhamstra/spark:myBranch and not need to create a new
>> github.com/markhamstra/incubator-spark. It's bad enough that the
>> differently-named repos don't show up in the compare-view drop-down
>> choices, but I also haven't been able to find a hand-crafted URL that will
>> make this work.
>>
>>
>>
>> On Tue, Sep 3, 2013 at 1:38 PM, Matei Zaharia <ma...@gmail.com>wrote:
>>
>>> Yup, the plan is as follows:
>>>
>>> - Make pull request against the mirror
>>> - Code review on GitHub as usual
>>> - Whoever merges it will simply merge it into the main Apache repo; when
>>> this propagates, the PR will be marked as merged
>>>
>>> I found at least one other Apache project that did this:
>>> http://wiki.apache.org/cordova/ContributorWorkflow.
>>>
>>> Matei
>>>
>>> On Sep 3, 2013, at 10:39 AM, Mark Hamstra <ma...@clearstorydata.com> wrote:
>>>
>>>> What is going to be the process for making pull requests? Can they be
>>> made
>>>> against the github mirror (https://github.com/apache/incubator-spark),
>>> or
>>>> must we use some other way?
>>>>
>>>>
>>>> On Tue, Sep 3, 2013 at 10:28 AM, Matei Zaharia <matei.zaharia@gmail.com
>>>> wrote:
>>>>
>>>>> Hi guys,
>>>>>
>>>>>> So are you planning to release 0.8 from the master branch (which is at
>>>>>> a106ed8... now) or from branch-0.8?
>>>>>
>>>>> Right now the branches are the same in terms of content (though I might
>>>>> not have merged the latest changes into 0.8). If we add stuff into
>>> master
>>>>> that we won't want in 0.8 we'll break that.
>>>>>
>>>>>> My recommendation is that we start to use the Incubator release
>>>>> doc/guide:
>>>>>>
>>>>>> http://incubator.apache.org/guides/releasemanagement.html
>>>>>
>>>>> Cool, thanks for the pointer. I'll try to follow the steps there about
>>>>> signing.
>>>>>
>>>>>> Are we "locking" pull requests to github repo by tomorrow?
>>>>>> Meaning no more push to GitHub repo for Spark.
>>>>>>
>>>>>> From your email seems like there will be more potential pull requests
>>> for
>>>>>> github repo to be merged back to ASF Git repo.
>>>>>
>>>>> We'll probably use the GitHub repo for the last few changes in this
>>>>> release and then switch. The reason is that there's a bit of work to do
>>>>> pull requests against the Apache one.
>>>>>
>>>>> Matei
>>>
>>>
>
>
>
> --
> --
> Evan Chan
> Staff Engineer
> ev@ooyala.com |
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Evan Chan <ev...@ooyala.com>.
Sorry one more clarification.
For doc pull requests for 0.8 release, should these be done against
the existing mesos/spark repo, or against the mirror at
apache/incubator-spark ?
I'm hoping to clear up a couple things in the docs before the release this week.
thanks,
-Evan
On Tue, Sep 3, 2013 at 2:25 PM, Mark Hamstra <ma...@clearstorydata.com> wrote:
> Okay, so is there any way to get github's compare view to be happy with
> differently-named repositories? What I want is to be able to compare and
> generate a pull request between
> github.com/apache/incubator-spark:masterand, e.g.,
> github.com/markhamstra/spark:myBranch and not need to create a new
> github.com/markhamstra/incubator-spark. It's bad enough that the
> differently-named repos don't show up in the compare-view drop-down
> choices, but I also haven't been able to find a hand-crafted URL that will
> make this work.
>
>
>
> On Tue, Sep 3, 2013 at 1:38 PM, Matei Zaharia <ma...@gmail.com>wrote:
>
>> Yup, the plan is as follows:
>>
>> - Make pull request against the mirror
>> - Code review on GitHub as usual
>> - Whoever merges it will simply merge it into the main Apache repo; when
>> this propagates, the PR will be marked as merged
>>
>> I found at least one other Apache project that did this:
>> http://wiki.apache.org/cordova/ContributorWorkflow.
>>
>> Matei
>>
>> On Sep 3, 2013, at 10:39 AM, Mark Hamstra <ma...@clearstorydata.com> wrote:
>>
>> > What is going to be the process for making pull requests? Can they be
>> made
>> > against the github mirror (https://github.com/apache/incubator-spark),
>> or
>> > must we use some other way?
>> >
>> >
>> > On Tue, Sep 3, 2013 at 10:28 AM, Matei Zaharia <matei.zaharia@gmail.com
>> >wrote:
>> >
>> >> Hi guys,
>> >>
>> >>> So are you planning to release 0.8 from the master branch (which is at
>> >>> a106ed8... now) or from branch-0.8?
>> >>
>> >> Right now the branches are the same in terms of content (though I might
>> >> not have merged the latest changes into 0.8). If we add stuff into
>> master
>> >> that we won't want in 0.8 we'll break that.
>> >>
>> >>> My recommendation is that we start to use the Incubator release
>> >> doc/guide:
>> >>>
>> >>> http://incubator.apache.org/guides/releasemanagement.html
>> >>
>> >> Cool, thanks for the pointer. I'll try to follow the steps there about
>> >> signing.
>> >>
>> >>> Are we "locking" pull requests to github repo by tomorrow?
>> >>> Meaning no more push to GitHub repo for Spark.
>> >>>
>> >>> From your email seems like there will be more potential pull requests
>> for
>> >>> github repo to be merged back to ASF Git repo.
>> >>
>> >> We'll probably use the GitHub repo for the last few changes in this
>> >> release and then switch. The reason is that there's a bit of work to do
>> >> pull requests against the Apache one.
>> >>
>> >> Matei
>>
>>
--
--
Evan Chan
Staff Engineer
ev@ooyala.com |
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Mark Hamstra <ma...@clearstorydata.com>.
Okay, so is there any way to get github's compare view to be happy with
differently-named repositories? What I want is to be able to compare and
generate a pull request between
github.com/apache/incubator-spark:masterand, e.g.,
github.com/markhamstra/spark:myBranch and not need to create a new
github.com/markhamstra/incubator-spark. It's bad enough that the
differently-named repos don't show up in the compare-view drop-down
choices, but I also haven't been able to find a hand-crafted URL that will
make this work.
On Tue, Sep 3, 2013 at 1:38 PM, Matei Zaharia <ma...@gmail.com>wrote:
> Yup, the plan is as follows:
>
> - Make pull request against the mirror
> - Code review on GitHub as usual
> - Whoever merges it will simply merge it into the main Apache repo; when
> this propagates, the PR will be marked as merged
>
> I found at least one other Apache project that did this:
> http://wiki.apache.org/cordova/ContributorWorkflow.
>
> Matei
>
> On Sep 3, 2013, at 10:39 AM, Mark Hamstra <ma...@clearstorydata.com> wrote:
>
> > What is going to be the process for making pull requests? Can they be
> made
> > against the github mirror (https://github.com/apache/incubator-spark),
> or
> > must we use some other way?
> >
> >
> > On Tue, Sep 3, 2013 at 10:28 AM, Matei Zaharia <matei.zaharia@gmail.com
> >wrote:
> >
> >> Hi guys,
> >>
> >>> So are you planning to release 0.8 from the master branch (which is at
> >>> a106ed8... now) or from branch-0.8?
> >>
> >> Right now the branches are the same in terms of content (though I might
> >> not have merged the latest changes into 0.8). If we add stuff into
> master
> >> that we won't want in 0.8 we'll break that.
> >>
> >>> My recommendation is that we start to use the Incubator release
> >> doc/guide:
> >>>
> >>> http://incubator.apache.org/guides/releasemanagement.html
> >>
> >> Cool, thanks for the pointer. I'll try to follow the steps there about
> >> signing.
> >>
> >>> Are we "locking" pull requests to github repo by tomorrow?
> >>> Meaning no more push to GitHub repo for Spark.
> >>>
> >>> From your email seems like there will be more potential pull requests
> for
> >>> github repo to be merged back to ASF Git repo.
> >>
> >> We'll probably use the GitHub repo for the last few changes in this
> >> release and then switch. The reason is that there's a bit of work to do
> >> pull requests against the Apache one.
> >>
> >> Matei
>
>
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Michael Joyce <ml...@gmail.com>.
Agreed. It would be nice for people to be able to contribute in a way that
doesn't force them through Github. I can't see that as being painful but it
would probably be nice for others to have the option at the very least. I
think the vast majority of people when given the two options will pick
Github so we wouldn't really lose out on much. Plus it would be a bummer to
miss out on contributions because someone is uncomfortable with Github or
pull request in general.
----------------
Mike
On Tue, Sep 3, 2013 at 8:18 PM, Henry Saputra <he...@gmail.com>wrote:
> Hey Reynold,
>
> It is just another way to contribute patches and can be used as transition
> flow for people than come from SVN or other centralized source control
> software.
>
> - Henry
>
>
> On Tue, Sep 3, 2013 at 8:08 PM, Reynold Xin <re...@gmail.com> wrote:
>
> > That seems substantially more overhead than generating github pull
> > requests. Is there any particular reason we want to do that?
> >
> >
> > On Wed, Sep 4, 2013 at 11:01 AM, Henry Saputra <henry.saputra@gmail.com
> > >wrote:
> >
> > > Thanks Guys.
> > >
> > > In other ASF projects I also allow people to attach the git diff to the
> > > JIRA itself (once we have one) and apply the patch and merge manually.
> > > I believe we could later configure ASF Jenkins to run when a patch is
> > > attached to JIRA (like in HBase and Hadoop).
> > >
> > > Do we want to also describe/ allow this alternative way to contribute
> > > patches?
> > >
> > >
> > > - Henry
> > >
> > >
> > > On Tue, Sep 3, 2013 at 7:08 PM, Matei Zaharia <matei.zaharia@gmail.com
> > > >wrote:
> > >
> > > > As far as I understood, we will have to manually merge those PRs into
> > the
> > > > Apache repo. However, GitHub will notice that they're "merged" as
> soon
> > as
> > > > it sees those commits in the repo, and will automatically close them.
> > At
> > > > least this is my experience merging other peoples' code (sometimes I
> > just
> > > > check out their branch from their repo and merge it manually).
> > > >
> > > > Matei
> > > >
> > > > On Sep 3, 2013, at 6:52 PM, Michael Joyce <jo...@apache.org> wrote:
> > > >
> > > > > Henry,
> > > > >
> > > > > I fairly certain that we'll have to manually resolve the pull
> > requests.
> > > > As
> > > > > far as I know, the Github mirror is simply a read-only mirror of
> the
> > > > > project's repository (be it svn or git). Hopefully someone will
> chime
> > > in
> > > > > and correct me if I'm wrong.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > -- Joyce
> > > > >
> > > > >
> > > > > On Tue, Sep 3, 2013 at 6:18 PM, Henry Saputra <
> > henry.saputra@gmail.com
> > > > >wrote:
> > > > >
> > > > >> So looks like we need to manually resolve the Github pull
> requests.
> > > > >>
> > > > >> Or, does github automatically know that a particular merge to ASF
> > git
> > > > repo
> > > > >> is associated to a GitHub pull request?
> > > > >>
> > > > >> - Henry
> > > > >>
> > > > >>
> > > > >> On Tue, Sep 3, 2013 at 1:38 PM, Matei Zaharia <
> > > matei.zaharia@gmail.com
> > > > >>> wrote:
> > > > >>
> > > > >>> Yup, the plan is as follows:
> > > > >>>
> > > > >>> - Make pull request against the mirror
> > > > >>> - Code review on GitHub as usual
> > > > >>> - Whoever merges it will simply merge it into the main Apache
> repo;
> > > > when
> > > > >>> this propagates, the PR will be marked as merged
> > > > >>>
> > > > >>> I found at least one other Apache project that did this:
> > > > >>> http://wiki.apache.org/cordova/ContributorWorkflow.
> > > > >>>
> > > > >>> Matei
> > > > >>>
> > > > >>> On Sep 3, 2013, at 10:39 AM, Mark Hamstra <
> mark@clearstorydata.com
> > >
> > > > >> wrote:
> > > > >>>
> > > > >>>> What is going to be the process for making pull requests? Can
> > they
> > > be
> > > > >>> made
> > > > >>>> against the github mirror (
> > > https://github.com/apache/incubator-spark
> > > > ),
> > > > >>> or
> > > > >>>> must we use some other way?
> > > > >>>>
> > > > >>>>
> > > > >>>> On Tue, Sep 3, 2013 at 10:28 AM, Matei Zaharia <
> > > > >> matei.zaharia@gmail.com
> > > > >>>> wrote:
> > > > >>>>
> > > > >>>>> Hi guys,
> > > > >>>>>
> > > > >>>>>> So are you planning to release 0.8 from the master branch
> (which
> > > is
> > > > >> at
> > > > >>>>>> a106ed8... now) or from branch-0.8?
> > > > >>>>>
> > > > >>>>> Right now the branches are the same in terms of content
> (though I
> > > > >> might
> > > > >>>>> not have merged the latest changes into 0.8). If we add stuff
> > into
> > > > >>> master
> > > > >>>>> that we won't want in 0.8 we'll break that.
> > > > >>>>>
> > > > >>>>>> My recommendation is that we start to use the Incubator
> release
> > > > >>>>> doc/guide:
> > > > >>>>>>
> > > > >>>>>> http://incubator.apache.org/guides/releasemanagement.html
> > > > >>>>>
> > > > >>>>> Cool, thanks for the pointer. I'll try to follow the steps
> there
> > > > about
> > > > >>>>> signing.
> > > > >>>>>
> > > > >>>>>> Are we "locking" pull requests to github repo by tomorrow?
> > > > >>>>>> Meaning no more push to GitHub repo for Spark.
> > > > >>>>>>
> > > > >>>>>> From your email seems like there will be more potential pull
> > > > requests
> > > > >>> for
> > > > >>>>>> github repo to be merged back to ASF Git repo.
> > > > >>>>>
> > > > >>>>> We'll probably use the GitHub repo for the last few changes in
> > this
> > > > >>>>> release and then switch. The reason is that there's a bit of
> work
> > > to
> > > > >> do
> > > > >>>>> pull requests against the Apache one.
> > > > >>>>>
> > > > >>>>> Matei
> > > > >>>
> > > > >>>
> > > > >>
> > > >
> > > >
> > >
> >
>
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Henry Saputra <he...@gmail.com>.
Hey Reynold,
It is just another way to contribute patches and can be used as transition
flow for people than come from SVN or other centralized source control
software.
- Henry
On Tue, Sep 3, 2013 at 8:08 PM, Reynold Xin <re...@gmail.com> wrote:
> That seems substantially more overhead than generating github pull
> requests. Is there any particular reason we want to do that?
>
>
> On Wed, Sep 4, 2013 at 11:01 AM, Henry Saputra <henry.saputra@gmail.com
> >wrote:
>
> > Thanks Guys.
> >
> > In other ASF projects I also allow people to attach the git diff to the
> > JIRA itself (once we have one) and apply the patch and merge manually.
> > I believe we could later configure ASF Jenkins to run when a patch is
> > attached to JIRA (like in HBase and Hadoop).
> >
> > Do we want to also describe/ allow this alternative way to contribute
> > patches?
> >
> >
> > - Henry
> >
> >
> > On Tue, Sep 3, 2013 at 7:08 PM, Matei Zaharia <matei.zaharia@gmail.com
> > >wrote:
> >
> > > As far as I understood, we will have to manually merge those PRs into
> the
> > > Apache repo. However, GitHub will notice that they're "merged" as soon
> as
> > > it sees those commits in the repo, and will automatically close them.
> At
> > > least this is my experience merging other peoples' code (sometimes I
> just
> > > check out their branch from their repo and merge it manually).
> > >
> > > Matei
> > >
> > > On Sep 3, 2013, at 6:52 PM, Michael Joyce <jo...@apache.org> wrote:
> > >
> > > > Henry,
> > > >
> > > > I fairly certain that we'll have to manually resolve the pull
> requests.
> > > As
> > > > far as I know, the Github mirror is simply a read-only mirror of the
> > > > project's repository (be it svn or git). Hopefully someone will chime
> > in
> > > > and correct me if I'm wrong.
> > > >
> > > >
> > > >
> > > >
> > > > -- Joyce
> > > >
> > > >
> > > > On Tue, Sep 3, 2013 at 6:18 PM, Henry Saputra <
> henry.saputra@gmail.com
> > > >wrote:
> > > >
> > > >> So looks like we need to manually resolve the Github pull requests.
> > > >>
> > > >> Or, does github automatically know that a particular merge to ASF
> git
> > > repo
> > > >> is associated to a GitHub pull request?
> > > >>
> > > >> - Henry
> > > >>
> > > >>
> > > >> On Tue, Sep 3, 2013 at 1:38 PM, Matei Zaharia <
> > matei.zaharia@gmail.com
> > > >>> wrote:
> > > >>
> > > >>> Yup, the plan is as follows:
> > > >>>
> > > >>> - Make pull request against the mirror
> > > >>> - Code review on GitHub as usual
> > > >>> - Whoever merges it will simply merge it into the main Apache repo;
> > > when
> > > >>> this propagates, the PR will be marked as merged
> > > >>>
> > > >>> I found at least one other Apache project that did this:
> > > >>> http://wiki.apache.org/cordova/ContributorWorkflow.
> > > >>>
> > > >>> Matei
> > > >>>
> > > >>> On Sep 3, 2013, at 10:39 AM, Mark Hamstra <mark@clearstorydata.com
> >
> > > >> wrote:
> > > >>>
> > > >>>> What is going to be the process for making pull requests? Can
> they
> > be
> > > >>> made
> > > >>>> against the github mirror (
> > https://github.com/apache/incubator-spark
> > > ),
> > > >>> or
> > > >>>> must we use some other way?
> > > >>>>
> > > >>>>
> > > >>>> On Tue, Sep 3, 2013 at 10:28 AM, Matei Zaharia <
> > > >> matei.zaharia@gmail.com
> > > >>>> wrote:
> > > >>>>
> > > >>>>> Hi guys,
> > > >>>>>
> > > >>>>>> So are you planning to release 0.8 from the master branch (which
> > is
> > > >> at
> > > >>>>>> a106ed8... now) or from branch-0.8?
> > > >>>>>
> > > >>>>> Right now the branches are the same in terms of content (though I
> > > >> might
> > > >>>>> not have merged the latest changes into 0.8). If we add stuff
> into
> > > >>> master
> > > >>>>> that we won't want in 0.8 we'll break that.
> > > >>>>>
> > > >>>>>> My recommendation is that we start to use the Incubator release
> > > >>>>> doc/guide:
> > > >>>>>>
> > > >>>>>> http://incubator.apache.org/guides/releasemanagement.html
> > > >>>>>
> > > >>>>> Cool, thanks for the pointer. I'll try to follow the steps there
> > > about
> > > >>>>> signing.
> > > >>>>>
> > > >>>>>> Are we "locking" pull requests to github repo by tomorrow?
> > > >>>>>> Meaning no more push to GitHub repo for Spark.
> > > >>>>>>
> > > >>>>>> From your email seems like there will be more potential pull
> > > requests
> > > >>> for
> > > >>>>>> github repo to be merged back to ASF Git repo.
> > > >>>>>
> > > >>>>> We'll probably use the GitHub repo for the last few changes in
> this
> > > >>>>> release and then switch. The reason is that there's a bit of work
> > to
> > > >> do
> > > >>>>> pull requests against the Apache one.
> > > >>>>>
> > > >>>>> Matei
> > > >>>
> > > >>>
> > > >>
> > >
> > >
> >
>
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Reynold Xin <re...@gmail.com>.
That seems substantially more overhead than generating github pull
requests. Is there any particular reason we want to do that?
On Wed, Sep 4, 2013 at 11:01 AM, Henry Saputra <he...@gmail.com>wrote:
> Thanks Guys.
>
> In other ASF projects I also allow people to attach the git diff to the
> JIRA itself (once we have one) and apply the patch and merge manually.
> I believe we could later configure ASF Jenkins to run when a patch is
> attached to JIRA (like in HBase and Hadoop).
>
> Do we want to also describe/ allow this alternative way to contribute
> patches?
>
>
> - Henry
>
>
> On Tue, Sep 3, 2013 at 7:08 PM, Matei Zaharia <matei.zaharia@gmail.com
> >wrote:
>
> > As far as I understood, we will have to manually merge those PRs into the
> > Apache repo. However, GitHub will notice that they're "merged" as soon as
> > it sees those commits in the repo, and will automatically close them. At
> > least this is my experience merging other peoples' code (sometimes I just
> > check out their branch from their repo and merge it manually).
> >
> > Matei
> >
> > On Sep 3, 2013, at 6:52 PM, Michael Joyce <jo...@apache.org> wrote:
> >
> > > Henry,
> > >
> > > I fairly certain that we'll have to manually resolve the pull requests.
> > As
> > > far as I know, the Github mirror is simply a read-only mirror of the
> > > project's repository (be it svn or git). Hopefully someone will chime
> in
> > > and correct me if I'm wrong.
> > >
> > >
> > >
> > >
> > > -- Joyce
> > >
> > >
> > > On Tue, Sep 3, 2013 at 6:18 PM, Henry Saputra <henry.saputra@gmail.com
> > >wrote:
> > >
> > >> So looks like we need to manually resolve the Github pull requests.
> > >>
> > >> Or, does github automatically know that a particular merge to ASF git
> > repo
> > >> is associated to a GitHub pull request?
> > >>
> > >> - Henry
> > >>
> > >>
> > >> On Tue, Sep 3, 2013 at 1:38 PM, Matei Zaharia <
> matei.zaharia@gmail.com
> > >>> wrote:
> > >>
> > >>> Yup, the plan is as follows:
> > >>>
> > >>> - Make pull request against the mirror
> > >>> - Code review on GitHub as usual
> > >>> - Whoever merges it will simply merge it into the main Apache repo;
> > when
> > >>> this propagates, the PR will be marked as merged
> > >>>
> > >>> I found at least one other Apache project that did this:
> > >>> http://wiki.apache.org/cordova/ContributorWorkflow.
> > >>>
> > >>> Matei
> > >>>
> > >>> On Sep 3, 2013, at 10:39 AM, Mark Hamstra <ma...@clearstorydata.com>
> > >> wrote:
> > >>>
> > >>>> What is going to be the process for making pull requests? Can they
> be
> > >>> made
> > >>>> against the github mirror (
> https://github.com/apache/incubator-spark
> > ),
> > >>> or
> > >>>> must we use some other way?
> > >>>>
> > >>>>
> > >>>> On Tue, Sep 3, 2013 at 10:28 AM, Matei Zaharia <
> > >> matei.zaharia@gmail.com
> > >>>> wrote:
> > >>>>
> > >>>>> Hi guys,
> > >>>>>
> > >>>>>> So are you planning to release 0.8 from the master branch (which
> is
> > >> at
> > >>>>>> a106ed8... now) or from branch-0.8?
> > >>>>>
> > >>>>> Right now the branches are the same in terms of content (though I
> > >> might
> > >>>>> not have merged the latest changes into 0.8). If we add stuff into
> > >>> master
> > >>>>> that we won't want in 0.8 we'll break that.
> > >>>>>
> > >>>>>> My recommendation is that we start to use the Incubator release
> > >>>>> doc/guide:
> > >>>>>>
> > >>>>>> http://incubator.apache.org/guides/releasemanagement.html
> > >>>>>
> > >>>>> Cool, thanks for the pointer. I'll try to follow the steps there
> > about
> > >>>>> signing.
> > >>>>>
> > >>>>>> Are we "locking" pull requests to github repo by tomorrow?
> > >>>>>> Meaning no more push to GitHub repo for Spark.
> > >>>>>>
> > >>>>>> From your email seems like there will be more potential pull
> > requests
> > >>> for
> > >>>>>> github repo to be merged back to ASF Git repo.
> > >>>>>
> > >>>>> We'll probably use the GitHub repo for the last few changes in this
> > >>>>> release and then switch. The reason is that there's a bit of work
> to
> > >> do
> > >>>>> pull requests against the Apache one.
> > >>>>>
> > >>>>> Matei
> > >>>
> > >>>
> > >>
> >
> >
>
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Henry Saputra <he...@gmail.com>.
Thanks Guys.
In other ASF projects I also allow people to attach the git diff to the
JIRA itself (once we have one) and apply the patch and merge manually.
I believe we could later configure ASF Jenkins to run when a patch is
attached to JIRA (like in HBase and Hadoop).
Do we want to also describe/ allow this alternative way to contribute
patches?
- Henry
On Tue, Sep 3, 2013 at 7:08 PM, Matei Zaharia <ma...@gmail.com>wrote:
> As far as I understood, we will have to manually merge those PRs into the
> Apache repo. However, GitHub will notice that they're "merged" as soon as
> it sees those commits in the repo, and will automatically close them. At
> least this is my experience merging other peoples' code (sometimes I just
> check out their branch from their repo and merge it manually).
>
> Matei
>
> On Sep 3, 2013, at 6:52 PM, Michael Joyce <jo...@apache.org> wrote:
>
> > Henry,
> >
> > I fairly certain that we'll have to manually resolve the pull requests.
> As
> > far as I know, the Github mirror is simply a read-only mirror of the
> > project's repository (be it svn or git). Hopefully someone will chime in
> > and correct me if I'm wrong.
> >
> >
> >
> >
> > -- Joyce
> >
> >
> > On Tue, Sep 3, 2013 at 6:18 PM, Henry Saputra <henry.saputra@gmail.com
> >wrote:
> >
> >> So looks like we need to manually resolve the Github pull requests.
> >>
> >> Or, does github automatically know that a particular merge to ASF git
> repo
> >> is associated to a GitHub pull request?
> >>
> >> - Henry
> >>
> >>
> >> On Tue, Sep 3, 2013 at 1:38 PM, Matei Zaharia <matei.zaharia@gmail.com
> >>> wrote:
> >>
> >>> Yup, the plan is as follows:
> >>>
> >>> - Make pull request against the mirror
> >>> - Code review on GitHub as usual
> >>> - Whoever merges it will simply merge it into the main Apache repo;
> when
> >>> this propagates, the PR will be marked as merged
> >>>
> >>> I found at least one other Apache project that did this:
> >>> http://wiki.apache.org/cordova/ContributorWorkflow.
> >>>
> >>> Matei
> >>>
> >>> On Sep 3, 2013, at 10:39 AM, Mark Hamstra <ma...@clearstorydata.com>
> >> wrote:
> >>>
> >>>> What is going to be the process for making pull requests? Can they be
> >>> made
> >>>> against the github mirror (https://github.com/apache/incubator-spark
> ),
> >>> or
> >>>> must we use some other way?
> >>>>
> >>>>
> >>>> On Tue, Sep 3, 2013 at 10:28 AM, Matei Zaharia <
> >> matei.zaharia@gmail.com
> >>>> wrote:
> >>>>
> >>>>> Hi guys,
> >>>>>
> >>>>>> So are you planning to release 0.8 from the master branch (which is
> >> at
> >>>>>> a106ed8... now) or from branch-0.8?
> >>>>>
> >>>>> Right now the branches are the same in terms of content (though I
> >> might
> >>>>> not have merged the latest changes into 0.8). If we add stuff into
> >>> master
> >>>>> that we won't want in 0.8 we'll break that.
> >>>>>
> >>>>>> My recommendation is that we start to use the Incubator release
> >>>>> doc/guide:
> >>>>>>
> >>>>>> http://incubator.apache.org/guides/releasemanagement.html
> >>>>>
> >>>>> Cool, thanks for the pointer. I'll try to follow the steps there
> about
> >>>>> signing.
> >>>>>
> >>>>>> Are we "locking" pull requests to github repo by tomorrow?
> >>>>>> Meaning no more push to GitHub repo for Spark.
> >>>>>>
> >>>>>> From your email seems like there will be more potential pull
> requests
> >>> for
> >>>>>> github repo to be merged back to ASF Git repo.
> >>>>>
> >>>>> We'll probably use the GitHub repo for the last few changes in this
> >>>>> release and then switch. The reason is that there's a bit of work to
> >> do
> >>>>> pull requests against the Apache one.
> >>>>>
> >>>>> Matei
> >>>
> >>>
> >>
>
>
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Matei Zaharia <ma...@gmail.com>.
As far as I understood, we will have to manually merge those PRs into the Apache repo. However, GitHub will notice that they're "merged" as soon as it sees those commits in the repo, and will automatically close them. At least this is my experience merging other peoples' code (sometimes I just check out their branch from their repo and merge it manually).
Matei
On Sep 3, 2013, at 6:52 PM, Michael Joyce <jo...@apache.org> wrote:
> Henry,
>
> I fairly certain that we'll have to manually resolve the pull requests. As
> far as I know, the Github mirror is simply a read-only mirror of the
> project's repository (be it svn or git). Hopefully someone will chime in
> and correct me if I'm wrong.
>
>
>
>
> -- Joyce
>
>
> On Tue, Sep 3, 2013 at 6:18 PM, Henry Saputra <he...@gmail.com>wrote:
>
>> So looks like we need to manually resolve the Github pull requests.
>>
>> Or, does github automatically know that a particular merge to ASF git repo
>> is associated to a GitHub pull request?
>>
>> - Henry
>>
>>
>> On Tue, Sep 3, 2013 at 1:38 PM, Matei Zaharia <matei.zaharia@gmail.com
>>> wrote:
>>
>>> Yup, the plan is as follows:
>>>
>>> - Make pull request against the mirror
>>> - Code review on GitHub as usual
>>> - Whoever merges it will simply merge it into the main Apache repo; when
>>> this propagates, the PR will be marked as merged
>>>
>>> I found at least one other Apache project that did this:
>>> http://wiki.apache.org/cordova/ContributorWorkflow.
>>>
>>> Matei
>>>
>>> On Sep 3, 2013, at 10:39 AM, Mark Hamstra <ma...@clearstorydata.com>
>> wrote:
>>>
>>>> What is going to be the process for making pull requests? Can they be
>>> made
>>>> against the github mirror (https://github.com/apache/incubator-spark),
>>> or
>>>> must we use some other way?
>>>>
>>>>
>>>> On Tue, Sep 3, 2013 at 10:28 AM, Matei Zaharia <
>> matei.zaharia@gmail.com
>>>> wrote:
>>>>
>>>>> Hi guys,
>>>>>
>>>>>> So are you planning to release 0.8 from the master branch (which is
>> at
>>>>>> a106ed8... now) or from branch-0.8?
>>>>>
>>>>> Right now the branches are the same in terms of content (though I
>> might
>>>>> not have merged the latest changes into 0.8). If we add stuff into
>>> master
>>>>> that we won't want in 0.8 we'll break that.
>>>>>
>>>>>> My recommendation is that we start to use the Incubator release
>>>>> doc/guide:
>>>>>>
>>>>>> http://incubator.apache.org/guides/releasemanagement.html
>>>>>
>>>>> Cool, thanks for the pointer. I'll try to follow the steps there about
>>>>> signing.
>>>>>
>>>>>> Are we "locking" pull requests to github repo by tomorrow?
>>>>>> Meaning no more push to GitHub repo for Spark.
>>>>>>
>>>>>> From your email seems like there will be more potential pull requests
>>> for
>>>>>> github repo to be merged back to ASF Git repo.
>>>>>
>>>>> We'll probably use the GitHub repo for the last few changes in this
>>>>> release and then switch. The reason is that there's a bit of work to
>> do
>>>>> pull requests against the Apache one.
>>>>>
>>>>> Matei
>>>
>>>
>>
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Michael Joyce <jo...@apache.org>.
Henry,
I fairly certain that we'll have to manually resolve the pull requests. As
far as I know, the Github mirror is simply a read-only mirror of the
project's repository (be it svn or git). Hopefully someone will chime in
and correct me if I'm wrong.
-- Joyce
On Tue, Sep 3, 2013 at 6:18 PM, Henry Saputra <he...@gmail.com>wrote:
> So looks like we need to manually resolve the Github pull requests.
>
> Or, does github automatically know that a particular merge to ASF git repo
> is associated to a GitHub pull request?
>
> - Henry
>
>
> On Tue, Sep 3, 2013 at 1:38 PM, Matei Zaharia <matei.zaharia@gmail.com
> >wrote:
>
> > Yup, the plan is as follows:
> >
> > - Make pull request against the mirror
> > - Code review on GitHub as usual
> > - Whoever merges it will simply merge it into the main Apache repo; when
> > this propagates, the PR will be marked as merged
> >
> > I found at least one other Apache project that did this:
> > http://wiki.apache.org/cordova/ContributorWorkflow.
> >
> > Matei
> >
> > On Sep 3, 2013, at 10:39 AM, Mark Hamstra <ma...@clearstorydata.com>
> wrote:
> >
> > > What is going to be the process for making pull requests? Can they be
> > made
> > > against the github mirror (https://github.com/apache/incubator-spark),
> > or
> > > must we use some other way?
> > >
> > >
> > > On Tue, Sep 3, 2013 at 10:28 AM, Matei Zaharia <
> matei.zaharia@gmail.com
> > >wrote:
> > >
> > >> Hi guys,
> > >>
> > >>> So are you planning to release 0.8 from the master branch (which is
> at
> > >>> a106ed8... now) or from branch-0.8?
> > >>
> > >> Right now the branches are the same in terms of content (though I
> might
> > >> not have merged the latest changes into 0.8). If we add stuff into
> > master
> > >> that we won't want in 0.8 we'll break that.
> > >>
> > >>> My recommendation is that we start to use the Incubator release
> > >> doc/guide:
> > >>>
> > >>> http://incubator.apache.org/guides/releasemanagement.html
> > >>
> > >> Cool, thanks for the pointer. I'll try to follow the steps there about
> > >> signing.
> > >>
> > >>> Are we "locking" pull requests to github repo by tomorrow?
> > >>> Meaning no more push to GitHub repo for Spark.
> > >>>
> > >>> From your email seems like there will be more potential pull requests
> > for
> > >>> github repo to be merged back to ASF Git repo.
> > >>
> > >> We'll probably use the GitHub repo for the last few changes in this
> > >> release and then switch. The reason is that there's a bit of work to
> do
> > >> pull requests against the Apache one.
> > >>
> > >> Matei
> >
> >
>
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by "Mattmann, Chris A (398J)" <ch...@jpl.nasa.gov>.
Guys there are Github hooks set up by Jukka Zitting and others
in ASF infra that will monitor the ASF mirror on Github, and then
bring pull requests back to list as emails with links to the patches
for inclusion.
Please contact infra folks if there are questions in getting it set
up and it may require an INFRA ticket.
HTH!
Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: Henry Saputra <he...@gmail.com>
Reply-To: "dev@spark.incubator.apache.org" <de...@spark.incubator.apache.org>
Date: Tuesday, September 3, 2013 6:18 PM
To: "dev@spark.incubator.apache.org" <de...@spark.incubator.apache.org>
Subject: Re: [Licensing check] Spark 0.8.0-incubating RC1
>So looks like we need to manually resolve the Github pull requests.
>
>Or, does github automatically know that a particular merge to ASF git repo
>is associated to a GitHub pull request?
>
>- Henry
>
>
>On Tue, Sep 3, 2013 at 1:38 PM, Matei Zaharia
><ma...@gmail.com>wrote:
>
>> Yup, the plan is as follows:
>>
>> - Make pull request against the mirror
>> - Code review on GitHub as usual
>> - Whoever merges it will simply merge it into the main Apache repo; when
>> this propagates, the PR will be marked as merged
>>
>> I found at least one other Apache project that did this:
>> http://wiki.apache.org/cordova/ContributorWorkflow.
>>
>> Matei
>>
>> On Sep 3, 2013, at 10:39 AM, Mark Hamstra <ma...@clearstorydata.com>
>>wrote:
>>
>> > What is going to be the process for making pull requests? Can they be
>> made
>> > against the github mirror (https://github.com/apache/incubator-spark),
>> or
>> > must we use some other way?
>> >
>> >
>> > On Tue, Sep 3, 2013 at 10:28 AM, Matei Zaharia
>><matei.zaharia@gmail.com
>> >wrote:
>> >
>> >> Hi guys,
>> >>
>> >>> So are you planning to release 0.8 from the master branch (which is
>>at
>> >>> a106ed8... now) or from branch-0.8?
>> >>
>> >> Right now the branches are the same in terms of content (though I
>>might
>> >> not have merged the latest changes into 0.8). If we add stuff into
>> master
>> >> that we won't want in 0.8 we'll break that.
>> >>
>> >>> My recommendation is that we start to use the Incubator release
>> >> doc/guide:
>> >>>
>> >>> http://incubator.apache.org/guides/releasemanagement.html
>> >>
>> >> Cool, thanks for the pointer. I'll try to follow the steps there
>>about
>> >> signing.
>> >>
>> >>> Are we "locking" pull requests to github repo by tomorrow?
>> >>> Meaning no more push to GitHub repo for Spark.
>> >>>
>> >>> From your email seems like there will be more potential pull
>>requests
>> for
>> >>> github repo to be merged back to ASF Git repo.
>> >>
>> >> We'll probably use the GitHub repo for the last few changes in this
>> >> release and then switch. The reason is that there's a bit of work to
>>do
>> >> pull requests against the Apache one.
>> >>
>> >> Matei
>>
>>
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Henry Saputra <he...@gmail.com>.
So looks like we need to manually resolve the Github pull requests.
Or, does github automatically know that a particular merge to ASF git repo
is associated to a GitHub pull request?
- Henry
On Tue, Sep 3, 2013 at 1:38 PM, Matei Zaharia <ma...@gmail.com>wrote:
> Yup, the plan is as follows:
>
> - Make pull request against the mirror
> - Code review on GitHub as usual
> - Whoever merges it will simply merge it into the main Apache repo; when
> this propagates, the PR will be marked as merged
>
> I found at least one other Apache project that did this:
> http://wiki.apache.org/cordova/ContributorWorkflow.
>
> Matei
>
> On Sep 3, 2013, at 10:39 AM, Mark Hamstra <ma...@clearstorydata.com> wrote:
>
> > What is going to be the process for making pull requests? Can they be
> made
> > against the github mirror (https://github.com/apache/incubator-spark),
> or
> > must we use some other way?
> >
> >
> > On Tue, Sep 3, 2013 at 10:28 AM, Matei Zaharia <matei.zaharia@gmail.com
> >wrote:
> >
> >> Hi guys,
> >>
> >>> So are you planning to release 0.8 from the master branch (which is at
> >>> a106ed8... now) or from branch-0.8?
> >>
> >> Right now the branches are the same in terms of content (though I might
> >> not have merged the latest changes into 0.8). If we add stuff into
> master
> >> that we won't want in 0.8 we'll break that.
> >>
> >>> My recommendation is that we start to use the Incubator release
> >> doc/guide:
> >>>
> >>> http://incubator.apache.org/guides/releasemanagement.html
> >>
> >> Cool, thanks for the pointer. I'll try to follow the steps there about
> >> signing.
> >>
> >>> Are we "locking" pull requests to github repo by tomorrow?
> >>> Meaning no more push to GitHub repo for Spark.
> >>>
> >>> From your email seems like there will be more potential pull requests
> for
> >>> github repo to be merged back to ASF Git repo.
> >>
> >> We'll probably use the GitHub repo for the last few changes in this
> >> release and then switch. The reason is that there's a bit of work to do
> >> pull requests against the Apache one.
> >>
> >> Matei
>
>
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Matei Zaharia <ma...@gmail.com>.
Yup, the plan is as follows:
- Make pull request against the mirror
- Code review on GitHub as usual
- Whoever merges it will simply merge it into the main Apache repo; when this propagates, the PR will be marked as merged
I found at least one other Apache project that did this: http://wiki.apache.org/cordova/ContributorWorkflow.
Matei
On Sep 3, 2013, at 10:39 AM, Mark Hamstra <ma...@clearstorydata.com> wrote:
> What is going to be the process for making pull requests? Can they be made
> against the github mirror (https://github.com/apache/incubator-spark), or
> must we use some other way?
>
>
> On Tue, Sep 3, 2013 at 10:28 AM, Matei Zaharia <ma...@gmail.com>wrote:
>
>> Hi guys,
>>
>>> So are you planning to release 0.8 from the master branch (which is at
>>> a106ed8... now) or from branch-0.8?
>>
>> Right now the branches are the same in terms of content (though I might
>> not have merged the latest changes into 0.8). If we add stuff into master
>> that we won't want in 0.8 we'll break that.
>>
>>> My recommendation is that we start to use the Incubator release
>> doc/guide:
>>>
>>> http://incubator.apache.org/guides/releasemanagement.html
>>
>> Cool, thanks for the pointer. I'll try to follow the steps there about
>> signing.
>>
>>> Are we "locking" pull requests to github repo by tomorrow?
>>> Meaning no more push to GitHub repo for Spark.
>>>
>>> From your email seems like there will be more potential pull requests for
>>> github repo to be merged back to ASF Git repo.
>>
>> We'll probably use the GitHub repo for the last few changes in this
>> release and then switch. The reason is that there's a bit of work to do
>> pull requests against the Apache one.
>>
>> Matei
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Mark Hamstra <ma...@clearstorydata.com>.
What is going to be the process for making pull requests? Can they be made
against the github mirror (https://github.com/apache/incubator-spark), or
must we use some other way?
On Tue, Sep 3, 2013 at 10:28 AM, Matei Zaharia <ma...@gmail.com>wrote:
> Hi guys,
>
> > So are you planning to release 0.8 from the master branch (which is at
> > a106ed8... now) or from branch-0.8?
>
> Right now the branches are the same in terms of content (though I might
> not have merged the latest changes into 0.8). If we add stuff into master
> that we won't want in 0.8 we'll break that.
>
> > My recommendation is that we start to use the Incubator release
> doc/guide:
> >
> > http://incubator.apache.org/guides/releasemanagement.html
>
> Cool, thanks for the pointer. I'll try to follow the steps there about
> signing.
>
> > Are we "locking" pull requests to github repo by tomorrow?
> > Meaning no more push to GitHub repo for Spark.
> >
> > From your email seems like there will be more potential pull requests for
> > github repo to be merged back to ASF Git repo.
>
> We'll probably use the GitHub repo for the last few changes in this
> release and then switch. The reason is that there's a bit of work to do
> pull requests against the Apache one.
>
> Matei
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Matei Zaharia <ma...@gmail.com>.
Hi guys,
> So are you planning to release 0.8 from the master branch (which is at
> a106ed8... now) or from branch-0.8?
Right now the branches are the same in terms of content (though I might not have merged the latest changes into 0.8). If we add stuff into master that we won't want in 0.8 we'll break that.
> My recommendation is that we start to use the Incubator release doc/guide:
>
> http://incubator.apache.org/guides/releasemanagement.html
Cool, thanks for the pointer. I'll try to follow the steps there about signing.
> Are we "locking" pull requests to github repo by tomorrow?
> Meaning no more push to GitHub repo for Spark.
>
> From your email seems like there will be more potential pull requests for
> github repo to be merged back to ASF Git repo.
We'll probably use the GitHub repo for the last few changes in this release and then switch. The reason is that there's a bit of work to do pull requests against the Apache one.
Matei
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Mark Hamstra <ma...@clearstorydata.com>.
So are you planning to release 0.8 from the master branch (which is at
a106ed8... now) or from branch-0.8?
On Mon, Sep 2, 2013 at 6:49 PM, Matei Zaharia <ma...@gmail.com>wrote:
> By the way, this tar file also corresponds to commit
> a106ed8b97e707b36818c11d1d7211fa28636178 in our Apache git repo.
>
> Matei
>
> On Sep 2, 2013, at 6:48 PM, Matei Zaharia <ma...@gmail.com> wrote:
>
> > Hi everyone,
> >
> > In preparation for the 0.8 release, I've put together a first release
> candidate mostly to help mentors or more experienced Apache people check
> whether the licensing, POMs, etc are right:
> http://www.cs.berkeley.edu/~matei/spark-0.8.0-incubating-RC1.tgz. I've
> tried to write the LICENSE, NOTICE, and POMs as required by Apache. Please
> take a look if you have a chance -- the ReadMe contains instructions on how
> to build it and launch the shell.
> >
> > This first release candidate is still missing some doc pages, so it
> won't be the final release, but it should have in place all the packaging
> stuff we'd like for 0.8, and it should be able to run out of the box on
> Linux, Windows and Mac OS X. Feel free to also test it for functionality;
> we may merge in a couple more bug fixes but most of the functionality will
> be there.
> >
> > Matei
>
>
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by Matei Zaharia <ma...@gmail.com>.
By the way, this tar file also corresponds to commit a106ed8b97e707b36818c11d1d7211fa28636178 in our Apache git repo.
Matei
On Sep 2, 2013, at 6:48 PM, Matei Zaharia <ma...@gmail.com> wrote:
> Hi everyone,
>
> In preparation for the 0.8 release, I've put together a first release candidate mostly to help mentors or more experienced Apache people check whether the licensing, POMs, etc are right: http://www.cs.berkeley.edu/~matei/spark-0.8.0-incubating-RC1.tgz. I've tried to write the LICENSE, NOTICE, and POMs as required by Apache. Please take a look if you have a chance -- the ReadMe contains instructions on how to build it and launch the shell.
>
> This first release candidate is still missing some doc pages, so it won't be the final release, but it should have in place all the packaging stuff we'd like for 0.8, and it should be able to run out of the box on Linux, Windows and Mac OS X. Feel free to also test it for functionality; we may merge in a couple more bug fixes but most of the functionality will be there.
>
> Matei
Re: [Licensing check] Spark 0.8.0-incubating RC1
Posted by "Mattmann, Chris A (398J)" <ch...@jpl.nasa.gov>.
Thanks Matei, I'll try and give it a look.
My recommendation is that we start to use the Incubator release doc/guide:
http://incubator.apache.org/guides/releasemanagement.html
Note that there is a new method for publishing release candidates and
releases
that uses SVNpubsub:
http://www.apache.org/dev/release.html#upload-ci
We should probably start using that. It will require a little back and
forth
with infra but will be worth it in the end. If no one files the issue
by this evening Eastern time, I'll file it.
Alternatively, at the least, the release candidate should be
housed on Apache servers, e.g., in your p.a.o/public_html/ directory, along
with *.md5 *.asc and *.sha files, and perhaps a CHANGES.txt/change log.
Also (and this is all in the guide above) your release signing key
should be uploaded to id.apache.org and then it will appear here:
http://people.apache.org/keys/group/spark.asc
HTH and happy to help out as needed.
Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: Matei Zaharia <ma...@gmail.com>
Reply-To: "dev@spark.incubator.apache.org" <de...@spark.incubator.apache.org>
Date: Monday, September 2, 2013 6:48 PM
To: "dev@spark.incubator.apache.org" <de...@spark.incubator.apache.org>
Subject: [Licensing check] Spark 0.8.0-incubating RC1
>Hi everyone,
>
>In preparation for the 0.8 release, I've put together a first release
>candidate mostly to help mentors or more experienced Apache people check
>whether the licensing, POMs, etc are right:
>http://www.cs.berkeley.edu/~matei/spark-0.8.0-incubating-RC1.tgz. I've
>tried to write the LICENSE, NOTICE, and POMs as required by Apache.
>Please take a look if you have a chance -- the ReadMe contains
>instructions on how to build it and launch the shell.
>
>This first release candidate is still missing some doc pages, so it won't
>be the final release, but it should have in place all the packaging stuff
>we'd like for 0.8, and it should be able to run out of the box on Linux,
>Windows and Mac OS X. Feel free to also test it for functionality; we may
>merge in a couple more bug fixes but most of the functionality will be
>there.
>
>Matei