You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Michael Armbrust <mi...@databricks.com> on 2017/11/09 23:54:20 UTC
Timeline for Spark 2.3
According to the timeline posted on the website, we are nearing branch cut
for Spark 2.3. I'd like to propose pushing this out towards mid to late
December for a couple of reasons and would like to hear what people think.
1. I've done release management during the Thanksgiving / Christmas time
before and in my experience, we don't actually get a lot of testing during
this time due to vacations and other commitments. I think beginning the RC
process in early January would give us the best coverage in the shortest
amount of time.
2. There are several large initiatives in progress that given a little more
time would leave us with a much more exciting 2.3 release. Specifically,
the work on the history server, Kubernetes and continuous processing.
3. Given the actual release date of Spark 2.2, I think we'll still get
Spark 2.3 out roughly 6 months after.
Thoughts?
Michael
Re: Timeline for Spark 2.3
Posted by Sameer Agarwal <sa...@databricks.com>.
Thanks everyone! I'll then go ahead and cut the branch early next week.
This will give us enough time to set up all the necessary jenkins builders
before the voting starts. We can then start the formal release process a
week after (8th January) once everybody gets back from vacation.
Regards,
Sameer
On Thu, Dec 21, 2017 at 8:48 PM, Kazuaki Ishizaki <IS...@jp.ibm.com>
wrote:
> +1 for cutting a branch earlier.
> In some Asian countries, 1st, 2nd, and 3rd January are off.
> https://www.timeanddate.com/holidays/
> How about 4th or 5th?
>
> Regards,
> Kazuaki Ishizaki
>
>
>
> From: Felix Cheung <fe...@hotmail.com>
> To: Michael Armbrust <mi...@databricks.com>, Holden Karau <
> holden@pigscanfly.ca>
> Cc: Sameer Agarwal <sa...@databricks.com>, Erik Erlandson <
> eerlands@redhat.com>, dev <de...@spark.apache.org>
> Date: 2017/12/21 04:48
> Subject: Re: Timeline for Spark 2.3
> ------------------------------
>
>
>
> +1
> I think the earlier we cut a branch the better.
>
> ------------------------------
>
> *From:* Michael Armbrust <mi...@databricks.com>
> *Sent:* Tuesday, December 19, 2017 4:41:44 PM
> *To:* Holden Karau
> *Cc:* Sameer Agarwal; Erik Erlandson; dev
> *Subject:* Re: Timeline for Spark 2.3
>
> Do people really need to be around for the branch cut (modulo the person
> cutting the branch)?
>
> 1st or 2nd doesn't really matter to me, but I am +1 kicking this off as
> soon as we enter the new year :)
>
> Michael
>
> On Tue, Dec 19, 2017 at 4:39 PM, Holden Karau <*holden@pigscanfly.ca*
> <ho...@pigscanfly.ca>> wrote:
> Sounds reasonable, although I'd choose the 2nd perhaps just since lots of
> folks are off on the 1st?
>
> On Tue, Dec 19, 2017 at 4:36 PM, Sameer Agarwal <*sameer@databricks.com*
> <sa...@databricks.com>> wrote:
> Let's aim for the 2.3 branch cut on 1st Jan and RC1 a week after that
> (i.e., week of 8th Jan)?
>
>
> On Fri, Dec 15, 2017 at 12:54 AM, Holden Karau <*holden@pigscanfly.ca*
> <ho...@pigscanfly.ca>> wrote:
> So personally I’d be in favour or pushing to early January, doing a
> release over the holidays is a little rough with herding all of people to
> vote.
>
> On Thu, Dec 14, 2017 at 11:49 PM Erik Erlandson <*eerlands@redhat.com*
> <ee...@redhat.com>> wrote:
> I wanted to check in on the state of the 2.3 freeze schedule. Original
> proposal was "late Dec", which is a bit open to interpretation.
>
> We are working to get some refactoring done on the integration testing for
> the Kubernetes back-end in preparation for testing upcoming release
> candidates, however holiday vacation time is about to begin taking its toll
> both on upstream reviewing and on the "downstream" spark-on-kube fork.
>
> If the freeze pushed into January, that would take some of the pressure
> off the kube back-end upstreaming. However, regardless, I was wondering if
> the dates could be clarified.
> Cheers,
> Erik
>
>
> On Mon, Nov 13, 2017 at 5:13 PM, *djiang@dataxu.com* <dj...@dataxu.com><
> *djiang@dataxu.com* <dj...@dataxu.com>> wrote:
> Hi,
>
> What is the process to request an issue/fix to be included in the next
> release? Is there a place to vote for features?
> I am interested in *https://issues.apache.org/jira/browse/SPARK-13127*
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_SPARK-2D13127&d=DwMF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=b70dG_9wpCdZSkBJahHYQ4IwKMdp2hQM29f-ZCGj9Pg&m=Pf-JJof66PoQGQUgW6qFA_cpH0Awhw47UCeiS_yKk4w&s=T5EQWQG8BY6A-LfNsjYaCuQPbXyTRFvv232dlVq73E0&e=>,
> to see
> if we can get Spark upgrade parquet to 1.9.0, which addresses the
> *https://issues.apache.org/jira/browse/PARQUET-686*
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_PARQUET-2D686&d=DwMF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=b70dG_9wpCdZSkBJahHYQ4IwKMdp2hQM29f-ZCGj9Pg&m=Pf-JJof66PoQGQUgW6qFA_cpH0Awhw47UCeiS_yKk4w&s=TcCl94jLWekyKQhsXmW3je6cZV-Ag2hirypA6AE25zA&e=>
> .
> Can we include the fix in Spark 2.3 release?
>
> Thanks,
>
> Dong
>
>
>
> --
> Sent from: *http://apache-spark-developers-list.1001551.n3.nabble.com/*
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__apache-2Dspark-2Ddevelopers-2Dlist.1001551.n3.nabble.com_&d=DwMF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=b70dG_9wpCdZSkBJahHYQ4IwKMdp2hQM29f-ZCGj9Pg&m=Pf-JJof66PoQGQUgW6qFA_cpH0Awhw47UCeiS_yKk4w&s=x7nQABsJc1-X1KVoEk74111pvRsCGFHNVwbQbNxe08U&e=>
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: *dev-unsubscribe@spark.apache.org*
> <de...@spark.apache.org>
>
>
> --
> Twitter: *https://twitter.com/holdenkarau*
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_holdenkarau&d=DwMF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=b70dG_9wpCdZSkBJahHYQ4IwKMdp2hQM29f-ZCGj9Pg&m=Pf-JJof66PoQGQUgW6qFA_cpH0Awhw47UCeiS_yKk4w&s=_VfEEjAQB6s9lnRFOsQndsIGeWpbEnGjcdbFLEi4ypU&e=>
>
>
>
> --
> Sameer Agarwal
> Software Engineer | Databricks Inc.
> *http://cs.berkeley.edu/~sameerag*
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__cs.berkeley.edu_-7Esameerag&d=DwMF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=b70dG_9wpCdZSkBJahHYQ4IwKMdp2hQM29f-ZCGj9Pg&m=Pf-JJof66PoQGQUgW6qFA_cpH0Awhw47UCeiS_yKk4w&s=MmFBEd5RECy9AdyKvBAGl1c7HqSyovjEOS_vrw-RsLk&e=>
>
>
>
> --
> Twitter: *https://twitter.com/holdenkarau*
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_holdenkarau&d=DwMF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=b70dG_9wpCdZSkBJahHYQ4IwKMdp2hQM29f-ZCGj9Pg&m=Pf-JJof66PoQGQUgW6qFA_cpH0Awhw47UCeiS_yKk4w&s=_VfEEjAQB6s9lnRFOsQndsIGeWpbEnGjcdbFLEi4ypU&e=>
>
>
>
--
Sameer Agarwal
Software Engineer | Databricks Inc.
http://cs.berkeley.edu/~sameerag
Re: Timeline for Spark 2.3
Posted by Kazuaki Ishizaki <IS...@jp.ibm.com>.
+1 for cutting a branch earlier.
In some Asian countries, 1st, 2nd, and 3rd January are off.
https://www.timeanddate.com/holidays/
How about 4th or 5th?
Regards,
Kazuaki Ishizaki
From: Felix Cheung <fe...@hotmail.com>
To: Michael Armbrust <mi...@databricks.com>, Holden Karau
<ho...@pigscanfly.ca>
Cc: Sameer Agarwal <sa...@databricks.com>, Erik Erlandson
<ee...@redhat.com>, dev <de...@spark.apache.org>
Date: 2017/12/21 04:48
Subject: Re: Timeline for Spark 2.3
+1
I think the earlier we cut a branch the better.
From: Michael Armbrust <mi...@databricks.com>
Sent: Tuesday, December 19, 2017 4:41:44 PM
To: Holden Karau
Cc: Sameer Agarwal; Erik Erlandson; dev
Subject: Re: Timeline for Spark 2.3
Do people really need to be around for the branch cut (modulo the person
cutting the branch)?
1st or 2nd doesn't really matter to me, but I am +1 kicking this off as
soon as we enter the new year :)
Michael
On Tue, Dec 19, 2017 at 4:39 PM, Holden Karau <ho...@pigscanfly.ca>
wrote:
Sounds reasonable, although I'd choose the 2nd perhaps just since lots of
folks are off on the 1st?
On Tue, Dec 19, 2017 at 4:36 PM, Sameer Agarwal <sa...@databricks.com>
wrote:
Let's aim for the 2.3 branch cut on 1st Jan and RC1 a week after that
(i.e., week of 8th Jan)?
On Fri, Dec 15, 2017 at 12:54 AM, Holden Karau <ho...@pigscanfly.ca>
wrote:
So personally I’d be in favour or pushing to early January, doing a
release over the holidays is a little rough with herding all of people to
vote.
On Thu, Dec 14, 2017 at 11:49 PM Erik Erlandson <ee...@redhat.com>
wrote:
I wanted to check in on the state of the 2.3 freeze schedule. Original
proposal was "late Dec", which is a bit open to interpretation.
We are working to get some refactoring done on the integration testing for
the Kubernetes back-end in preparation for testing upcoming release
candidates, however holiday vacation time is about to begin taking its
toll both on upstream reviewing and on the "downstream" spark-on-kube
fork.
If the freeze pushed into January, that would take some of the pressure
off the kube back-end upstreaming. However, regardless, I was wondering if
the dates could be clarified.
Cheers,
Erik
On Mon, Nov 13, 2017 at 5:13 PM, djiang@dataxu.com <dj...@dataxu.com>
wrote:
Hi,
What is the process to request an issue/fix to be included in the next
release? Is there a place to vote for features?
I am interested in https://issues.apache.org/jira/browse/SPARK-13127, to
see
if we can get Spark upgrade parquet to 1.9.0, which addresses the
https://issues.apache.org/jira/browse/PARQUET-686.
Can we include the fix in Spark 2.3 release?
Thanks,
Dong
--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
--
Twitter: https://twitter.com/holdenkarau
--
Sameer Agarwal
Software Engineer | Databricks Inc.
http://cs.berkeley.edu/~sameerag
--
Twitter: https://twitter.com/holdenkarau
Re: Timeline for Spark 2.3
Posted by Felix Cheung <fe...@hotmail.com>.
+1
I think the earlier we cut a branch the better.
________________________________
From: Michael Armbrust <mi...@databricks.com>
Sent: Tuesday, December 19, 2017 4:41:44 PM
To: Holden Karau
Cc: Sameer Agarwal; Erik Erlandson; dev
Subject: Re: Timeline for Spark 2.3
Do people really need to be around for the branch cut (modulo the person cutting the branch)?
1st or 2nd doesn't really matter to me, but I am +1 kicking this off as soon as we enter the new year :)
Michael
On Tue, Dec 19, 2017 at 4:39 PM, Holden Karau <ho...@pigscanfly.ca>> wrote:
Sounds reasonable, although I'd choose the 2nd perhaps just since lots of folks are off on the 1st?
On Tue, Dec 19, 2017 at 4:36 PM, Sameer Agarwal <sa...@databricks.com>> wrote:
Let's aim for the 2.3 branch cut on 1st Jan and RC1 a week after that (i.e., week of 8th Jan)?
On Fri, Dec 15, 2017 at 12:54 AM, Holden Karau <ho...@pigscanfly.ca>> wrote:
So personally I’d be in favour or pushing to early January, doing a release over the holidays is a little rough with herding all of people to vote.
On Thu, Dec 14, 2017 at 11:49 PM Erik Erlandson <ee...@redhat.com>> wrote:
I wanted to check in on the state of the 2.3 freeze schedule. Original proposal was "late Dec", which is a bit open to interpretation.
We are working to get some refactoring done on the integration testing for the Kubernetes back-end in preparation for testing upcoming release candidates, however holiday vacation time is about to begin taking its toll both on upstream reviewing and on the "downstream" spark-on-kube fork.
If the freeze pushed into January, that would take some of the pressure off the kube back-end upstreaming. However, regardless, I was wondering if the dates could be clarified.
Cheers,
Erik
On Mon, Nov 13, 2017 at 5:13 PM, djiang@dataxu.com<ma...@dataxu.com> <dj...@dataxu.com>> wrote:
Hi,
What is the process to request an issue/fix to be included in the next
release? Is there a place to vote for features?
I am interested in https://issues.apache.org/jira/browse/SPARK-13127, to see
if we can get Spark upgrade parquet to 1.9.0, which addresses the
https://issues.apache.org/jira/browse/PARQUET-686.
Can we include the fix in Spark 2.3 release?
Thanks,
Dong
--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org<ma...@spark.apache.org>
--
Twitter: https://twitter.com/holdenkarau
--
Sameer Agarwal
Software Engineer | Databricks Inc.
http://cs.berkeley.edu/~sameerag
--
Twitter: https://twitter.com/holdenkarau
Re: Timeline for Spark 2.3
Posted by Michael Armbrust <mi...@databricks.com>.
Do people really need to be around for the branch cut (modulo the person
cutting the branch)?
1st or 2nd doesn't really matter to me, but I am +1 kicking this off as
soon as we enter the new year :)
Michael
On Tue, Dec 19, 2017 at 4:39 PM, Holden Karau <ho...@pigscanfly.ca> wrote:
> Sounds reasonable, although I'd choose the 2nd perhaps just since lots of
> folks are off on the 1st?
>
> On Tue, Dec 19, 2017 at 4:36 PM, Sameer Agarwal <sa...@databricks.com>
> wrote:
>
>> Let's aim for the 2.3 branch cut on 1st Jan and RC1 a week after that
>> (i.e., week of 8th Jan)?
>>
>>
>> On Fri, Dec 15, 2017 at 12:54 AM, Holden Karau <ho...@pigscanfly.ca>
>> wrote:
>>
>>> So personally I’d be in favour or pushing to early January, doing a
>>> release over the holidays is a little rough with herding all of people to
>>> vote.
>>>
>>> On Thu, Dec 14, 2017 at 11:49 PM Erik Erlandson <ee...@redhat.com>
>>> wrote:
>>>
>>>> I wanted to check in on the state of the 2.3 freeze schedule. Original
>>>> proposal was "late Dec", which is a bit open to interpretation.
>>>>
>>>> We are working to get some refactoring done on the integration testing
>>>> for the Kubernetes back-end in preparation for testing upcoming release
>>>> candidates, however holiday vacation time is about to begin taking its toll
>>>> both on upstream reviewing and on the "downstream" spark-on-kube fork.
>>>>
>>>> If the freeze pushed into January, that would take some of the pressure
>>>> off the kube back-end upstreaming. However, regardless, I was wondering if
>>>> the dates could be clarified.
>>>> Cheers,
>>>> Erik
>>>>
>>>>
>>>> On Mon, Nov 13, 2017 at 5:13 PM, djiang@dataxu.com <dj...@dataxu.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> What is the process to request an issue/fix to be included in the next
>>>>> release? Is there a place to vote for features?
>>>>> I am interested in https://issues.apache.org/jira/browse/SPARK-13127,
>>>>> to see
>>>>> if we can get Spark upgrade parquet to 1.9.0, which addresses the
>>>>> https://issues.apache.org/jira/browse/PARQUET-686.
>>>>> Can we include the fix in Spark 2.3 release?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Dong
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>
>>>>>
>>>> --
>>> Twitter: https://twitter.com/holdenkarau
>>>
>>
>>
>>
>> --
>> Sameer Agarwal
>> Software Engineer | Databricks Inc.
>> http://cs.berkeley.edu/~sameerag
>>
>
>
>
> --
> Twitter: https://twitter.com/holdenkarau
>
Re: Timeline for Spark 2.3
Posted by Holden Karau <ho...@pigscanfly.ca>.
Sounds reasonable, although I'd choose the 2nd perhaps just since lots of
folks are off on the 1st?
On Tue, Dec 19, 2017 at 4:36 PM, Sameer Agarwal <sa...@databricks.com>
wrote:
> Let's aim for the 2.3 branch cut on 1st Jan and RC1 a week after that
> (i.e., week of 8th Jan)?
>
>
> On Fri, Dec 15, 2017 at 12:54 AM, Holden Karau <ho...@pigscanfly.ca>
> wrote:
>
>> So personally I’d be in favour or pushing to early January, doing a
>> release over the holidays is a little rough with herding all of people to
>> vote.
>>
>> On Thu, Dec 14, 2017 at 11:49 PM Erik Erlandson <ee...@redhat.com>
>> wrote:
>>
>>> I wanted to check in on the state of the 2.3 freeze schedule. Original
>>> proposal was "late Dec", which is a bit open to interpretation.
>>>
>>> We are working to get some refactoring done on the integration testing
>>> for the Kubernetes back-end in preparation for testing upcoming release
>>> candidates, however holiday vacation time is about to begin taking its toll
>>> both on upstream reviewing and on the "downstream" spark-on-kube fork.
>>>
>>> If the freeze pushed into January, that would take some of the pressure
>>> off the kube back-end upstreaming. However, regardless, I was wondering if
>>> the dates could be clarified.
>>> Cheers,
>>> Erik
>>>
>>>
>>> On Mon, Nov 13, 2017 at 5:13 PM, djiang@dataxu.com <dj...@dataxu.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> What is the process to request an issue/fix to be included in the next
>>>> release? Is there a place to vote for features?
>>>> I am interested in https://issues.apache.org/jira/browse/SPARK-13127,
>>>> to see
>>>> if we can get Spark upgrade parquet to 1.9.0, which addresses the
>>>> https://issues.apache.org/jira/browse/PARQUET-686.
>>>> Can we include the fix in Spark 2.3 release?
>>>>
>>>> Thanks,
>>>>
>>>> Dong
>>>>
>>>>
>>>>
>>>> --
>>>> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>
>>>>
>>> --
>> Twitter: https://twitter.com/holdenkarau
>>
>
>
>
> --
> Sameer Agarwal
> Software Engineer | Databricks Inc.
> http://cs.berkeley.edu/~sameerag
>
--
Twitter: https://twitter.com/holdenkarau
Re: Timeline for Spark 2.3
Posted by Sameer Agarwal <sa...@databricks.com>.
Let's aim for the 2.3 branch cut on 1st Jan and RC1 a week after that
(i.e., week of 8th Jan)?
On Fri, Dec 15, 2017 at 12:54 AM, Holden Karau <ho...@pigscanfly.ca> wrote:
> So personally I’d be in favour or pushing to early January, doing a
> release over the holidays is a little rough with herding all of people to
> vote.
>
> On Thu, Dec 14, 2017 at 11:49 PM Erik Erlandson <ee...@redhat.com>
> wrote:
>
>> I wanted to check in on the state of the 2.3 freeze schedule. Original
>> proposal was "late Dec", which is a bit open to interpretation.
>>
>> We are working to get some refactoring done on the integration testing
>> for the Kubernetes back-end in preparation for testing upcoming release
>> candidates, however holiday vacation time is about to begin taking its toll
>> both on upstream reviewing and on the "downstream" spark-on-kube fork.
>>
>> If the freeze pushed into January, that would take some of the pressure
>> off the kube back-end upstreaming. However, regardless, I was wondering if
>> the dates could be clarified.
>> Cheers,
>> Erik
>>
>>
>> On Mon, Nov 13, 2017 at 5:13 PM, djiang@dataxu.com <dj...@dataxu.com>
>> wrote:
>>
>>> Hi,
>>>
>>> What is the process to request an issue/fix to be included in the next
>>> release? Is there a place to vote for features?
>>> I am interested in https://issues.apache.org/jira/browse/SPARK-13127,
>>> to see
>>> if we can get Spark upgrade parquet to 1.9.0, which addresses the
>>> https://issues.apache.org/jira/browse/PARQUET-686.
>>> Can we include the fix in Spark 2.3 release?
>>>
>>> Thanks,
>>>
>>> Dong
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>
>>>
>> --
> Twitter: https://twitter.com/holdenkarau
>
--
Sameer Agarwal
Software Engineer | Databricks Inc.
http://cs.berkeley.edu/~sameerag
Re: Timeline for Spark 2.3
Posted by Holden Karau <ho...@pigscanfly.ca>.
So personally I’d be in favour or pushing to early January, doing a release
over the holidays is a little rough with herding all of people to vote.
On Thu, Dec 14, 2017 at 11:49 PM Erik Erlandson <ee...@redhat.com> wrote:
> I wanted to check in on the state of the 2.3 freeze schedule. Original
> proposal was "late Dec", which is a bit open to interpretation.
>
> We are working to get some refactoring done on the integration testing for
> the Kubernetes back-end in preparation for testing upcoming release
> candidates, however holiday vacation time is about to begin taking its toll
> both on upstream reviewing and on the "downstream" spark-on-kube fork.
>
> If the freeze pushed into January, that would take some of the pressure
> off the kube back-end upstreaming. However, regardless, I was wondering if
> the dates could be clarified.
> Cheers,
> Erik
>
>
> On Mon, Nov 13, 2017 at 5:13 PM, djiang@dataxu.com <dj...@dataxu.com>
> wrote:
>
>> Hi,
>>
>> What is the process to request an issue/fix to be included in the next
>> release? Is there a place to vote for features?
>> I am interested in https://issues.apache.org/jira/browse/SPARK-13127, to
>> see
>> if we can get Spark upgrade parquet to 1.9.0, which addresses the
>> https://issues.apache.org/jira/browse/PARQUET-686.
>> Can we include the fix in Spark 2.3 release?
>>
>> Thanks,
>>
>> Dong
>>
>>
>>
>> --
>> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>
>>
> --
Twitter: https://twitter.com/holdenkarau
Re: Timeline for Spark 2.3
Posted by Erik Erlandson <ee...@redhat.com>.
I wanted to check in on the state of the 2.3 freeze schedule. Original
proposal was "late Dec", which is a bit open to interpretation.
We are working to get some refactoring done on the integration testing for
the Kubernetes back-end in preparation for testing upcoming release
candidates, however holiday vacation time is about to begin taking its toll
both on upstream reviewing and on the "downstream" spark-on-kube fork.
If the freeze pushed into January, that would take some of the pressure off
the kube back-end upstreaming. However, regardless, I was wondering if the
dates could be clarified.
Cheers,
Erik
On Mon, Nov 13, 2017 at 5:13 PM, djiang@dataxu.com <dj...@dataxu.com>
wrote:
> Hi,
>
> What is the process to request an issue/fix to be included in the next
> release? Is there a place to vote for features?
> I am interested in https://issues.apache.org/jira/browse/SPARK-13127, to
> see
> if we can get Spark upgrade parquet to 1.9.0, which addresses the
> https://issues.apache.org/jira/browse/PARQUET-686.
> Can we include the fix in Spark 2.3 release?
>
> Thanks,
>
> Dong
>
>
>
> --
> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>
Re: Timeline for Spark 2.3
Posted by "djiang@dataxu.com" <dj...@dataxu.com>.
Hi,
What is the process to request an issue/fix to be included in the next
release? Is there a place to vote for features?
I am interested in https://issues.apache.org/jira/browse/SPARK-13127, to see
if we can get Spark upgrade parquet to 1.9.0, which addresses the
https://issues.apache.org/jira/browse/PARQUET-686.
Can we include the fix in Spark 2.3 release?
Thanks,
Dong
--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Timeline for Spark 2.3
Posted by Sameer Agarwal <sa...@databricks.com>.
Sounds good to me too. In addition to what has already been pointed out
about the Spark History Server and the Kubernetes support, this would also
give us enough time to further polish the new data source v2 API and the
vectorized UDF API to iron out any kinks.
I'd like to volunteer to serve as the release manager for Spark 2.3. In
terms of bandwidth, I will be available during this holiday season as I
have no vacation planed during the Dec-Jan timeframe. And I'm fairly
familiar with most of the major efforts targeted for the 2.3 release.
Thanks,
Sameer
On Fri, Nov 10, 2017 at 2:07 AM, Sean Owen <so...@cloudera.com> wrote:
> The original timeline was just +6 months from last planned release, so
> there was nothing too magic about it. That was pushed from +4 . The only
> risk here is that an extra month becomes 2, 3, and so users aren't getting
> the other 1000 fixes. But no particular problem with moving it back.
>
> On Thu, Nov 9, 2017, 5:54 PM Michael Armbrust <mi...@databricks.com>
> wrote:
>
>> According to the timeline posted on the website, we are nearing branch
>> cut for Spark 2.3. I'd like to propose pushing this out towards mid to
>> late December for a couple of reasons and would like to hear what people
>> think.
>>
>> 1. I've done release management during the Thanksgiving / Christmas time
>> before and in my experience, we don't actually get a lot of testing during
>> this time due to vacations and other commitments. I think beginning the RC
>> process in early January would give us the best coverage in the shortest
>> amount of time.
>> 2. There are several large initiatives in progress that given a little
>> more time would leave us with a much more exciting 2.3 release.
>> Specifically, the work on the history server, Kubernetes and continuous
>> processing
>> 3. Given the actual release date of Spark 2.2, I think we'll still get
>> Spark 2.3 out roughly 6 months after.
>>
>> Thoughts?
>>
>> Michael
>>
>
--
Sameer Agarwal
Software Engineer | Databricks Inc.
http://cs.berkeley.edu/~sameerag
Re: Timeline for Spark 2.3
Posted by Sean Owen <so...@cloudera.com>.
The original timeline was just +6 months from last planned release, so
there was nothing too magic about it. That was pushed from +4 . The only
risk here is that an extra month becomes 2, 3, and so users aren't getting
the other 1000 fixes. But no particular problem with moving it back.
On Thu, Nov 9, 2017, 5:54 PM Michael Armbrust <mi...@databricks.com>
wrote:
> According to the timeline posted on the website, we are nearing branch cut
> for Spark 2.3. I'd like to propose pushing this out towards mid to late
> December for a couple of reasons and would like to hear what people think.
>
> 1. I've done release management during the Thanksgiving / Christmas time
> before and in my experience, we don't actually get a lot of testing during
> this time due to vacations and other commitments. I think beginning the RC
> process in early January would give us the best coverage in the shortest
> amount of time.
> 2. There are several large initiatives in progress that given a little
> more time would leave us with a much more exciting 2.3 release.
> Specifically, the work on the history server, Kubernetes and continuous
> processing
> 3. Given the actual release date of Spark 2.2, I think we'll still get
> Spark 2.3 out roughly 6 months after.
>
> Thoughts?
>
> Michael
>
Re: Timeline for Spark 2.3
Posted by Marco Gaido <ma...@gmail.com>.
I would love too to have SPARK-18016. I think it would help a lot of users.
2017-11-10 5:58 GMT+01:00 Nick Pentreath <ni...@gmail.com>:
> +1 I think that’s practical
>
> On Fri, 10 Nov 2017 at 03:13, Erik Erlandson <ee...@redhat.com> wrote:
>
>> +1 on extending the deadline. It will significantly improve the logistics
>> for upstreaming the Kubernetes back-end. Also agreed, on the general
>> realities of reduced bandwidth over the Nov-Dec holiday season.
>> Erik
>>
>> On Thu, Nov 9, 2017 at 6:03 PM, Matei Zaharia <ma...@gmail.com>
>> wrote:
>>
>>> I’m also +1 on extending this to get Kubernetes and other features in.
>>>
>>> Matei
>>>
>>> > On Nov 9, 2017, at 4:04 PM, Anirudh Ramanathan
>>> <fo...@google.com.INVALID> wrote:
>>> >
>>> > This would help the community on the Kubernetes effort quite a bit -
>>> giving us additional time for reviews and testing for the 2.3 release.
>>> >
>>> > On Thu, Nov 9, 2017 at 3:56 PM, Justin Miller <
>>> justin.miller@protectwise.com> wrote:
>>> > That sounds fine to me. I’m hoping that this ticket can make it into
>>> Spark 2.3: https://issues.apache.org/jira/browse/SPARK-18016
>>> >
>>> > It’s causing some pretty considerable problems when we alter the
>>> columns to be nullable, but we are OK for now without that.
>>> >
>>> > Best,
>>> > Justin
>>> >
>>> >> On Nov 9, 2017, at 4:54 PM, Michael Armbrust <mi...@databricks.com>
>>> wrote:
>>> >>
>>> >> According to the timeline posted on the website, we are nearing
>>> branch cut for Spark 2.3. I'd like to propose pushing this out towards mid
>>> to late December for a couple of reasons and would like to hear what people
>>> think.
>>> >>
>>> >> 1. I've done release management during the Thanksgiving / Christmas
>>> time before and in my experience, we don't actually get a lot of testing
>>> during this time due to vacations and other commitments. I think beginning
>>> the RC process in early January would give us the best coverage in the
>>> shortest amount of time.
>>> >> 2. There are several large initiatives in progress that given a
>>> little more time would leave us with a much more exciting 2.3 release.
>>> Specifically, the work on the history server, Kubernetes and continuous
>>> processing.
>>> >> 3. Given the actual release date of Spark 2.2, I think we'll still
>>> get Spark 2.3 out roughly 6 months after.
>>> >>
>>> >> Thoughts?
>>> >>
>>> >> Michael
>>> >
>>> >
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>
>>>
>>
Re: Timeline for Spark 2.3
Posted by Nick Pentreath <ni...@gmail.com>.
+1 I think that’s practical
On Fri, 10 Nov 2017 at 03:13, Erik Erlandson <ee...@redhat.com> wrote:
> +1 on extending the deadline. It will significantly improve the logistics
> for upstreaming the Kubernetes back-end. Also agreed, on the general
> realities of reduced bandwidth over the Nov-Dec holiday season.
> Erik
>
> On Thu, Nov 9, 2017 at 6:03 PM, Matei Zaharia <ma...@gmail.com>
> wrote:
>
>> I’m also +1 on extending this to get Kubernetes and other features in.
>>
>> Matei
>>
>> > On Nov 9, 2017, at 4:04 PM, Anirudh Ramanathan
>> <fo...@google.com.INVALID> wrote:
>> >
>> > This would help the community on the Kubernetes effort quite a bit -
>> giving us additional time for reviews and testing for the 2.3 release.
>> >
>> > On Thu, Nov 9, 2017 at 3:56 PM, Justin Miller <
>> justin.miller@protectwise.com> wrote:
>> > That sounds fine to me. I’m hoping that this ticket can make it into
>> Spark 2.3: https://issues.apache.org/jira/browse/SPARK-18016
>> >
>> > It’s causing some pretty considerable problems when we alter the
>> columns to be nullable, but we are OK for now without that.
>> >
>> > Best,
>> > Justin
>> >
>> >> On Nov 9, 2017, at 4:54 PM, Michael Armbrust <mi...@databricks.com>
>> wrote:
>> >>
>> >> According to the timeline posted on the website, we are nearing branch
>> cut for Spark 2.3. I'd like to propose pushing this out towards mid to
>> late December for a couple of reasons and would like to hear what people
>> think.
>> >>
>> >> 1. I've done release management during the Thanksgiving / Christmas
>> time before and in my experience, we don't actually get a lot of testing
>> during this time due to vacations and other commitments. I think beginning
>> the RC process in early January would give us the best coverage in the
>> shortest amount of time.
>> >> 2. There are several large initiatives in progress that given a little
>> more time would leave us with a much more exciting 2.3 release.
>> Specifically, the work on the history server, Kubernetes and continuous
>> processing.
>> >> 3. Given the actual release date of Spark 2.2, I think we'll still get
>> Spark 2.3 out roughly 6 months after.
>> >>
>> >> Thoughts?
>> >>
>> >> Michael
>> >
>> >
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>
>>
>
Re: Timeline for Spark 2.3
Posted by Erik Erlandson <ee...@redhat.com>.
+1 on extending the deadline. It will significantly improve the logistics
for upstreaming the Kubernetes back-end. Also agreed, on the general
realities of reduced bandwidth over the Nov-Dec holiday season.
Erik
On Thu, Nov 9, 2017 at 6:03 PM, Matei Zaharia <ma...@gmail.com>
wrote:
> I’m also +1 on extending this to get Kubernetes and other features in.
>
> Matei
>
> > On Nov 9, 2017, at 4:04 PM, Anirudh Ramanathan <fo...@google.com.INVALID>
> wrote:
> >
> > This would help the community on the Kubernetes effort quite a bit -
> giving us additional time for reviews and testing for the 2.3 release.
> >
> > On Thu, Nov 9, 2017 at 3:56 PM, Justin Miller <
> justin.miller@protectwise.com> wrote:
> > That sounds fine to me. I’m hoping that this ticket can make it into
> Spark 2.3: https://issues.apache.org/jira/browse/SPARK-18016
> >
> > It’s causing some pretty considerable problems when we alter the columns
> to be nullable, but we are OK for now without that.
> >
> > Best,
> > Justin
> >
> >> On Nov 9, 2017, at 4:54 PM, Michael Armbrust <mi...@databricks.com>
> wrote:
> >>
> >> According to the timeline posted on the website, we are nearing branch
> cut for Spark 2.3. I'd like to propose pushing this out towards mid to
> late December for a couple of reasons and would like to hear what people
> think.
> >>
> >> 1. I've done release management during the Thanksgiving / Christmas
> time before and in my experience, we don't actually get a lot of testing
> during this time due to vacations and other commitments. I think beginning
> the RC process in early January would give us the best coverage in the
> shortest amount of time.
> >> 2. There are several large initiatives in progress that given a little
> more time would leave us with a much more exciting 2.3 release.
> Specifically, the work on the history server, Kubernetes and continuous
> processing.
> >> 3. Given the actual release date of Spark 2.2, I think we'll still get
> Spark 2.3 out roughly 6 months after.
> >>
> >> Thoughts?
> >>
> >> Michael
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>
Re: Timeline for Spark 2.3
Posted by Matei Zaharia <ma...@gmail.com>.
I’m also +1 on extending this to get Kubernetes and other features in.
Matei
> On Nov 9, 2017, at 4:04 PM, Anirudh Ramanathan <fo...@google.com.INVALID> wrote:
>
> This would help the community on the Kubernetes effort quite a bit - giving us additional time for reviews and testing for the 2.3 release.
>
> On Thu, Nov 9, 2017 at 3:56 PM, Justin Miller <ju...@protectwise.com> wrote:
> That sounds fine to me. I’m hoping that this ticket can make it into Spark 2.3: https://issues.apache.org/jira/browse/SPARK-18016
>
> It’s causing some pretty considerable problems when we alter the columns to be nullable, but we are OK for now without that.
>
> Best,
> Justin
>
>> On Nov 9, 2017, at 4:54 PM, Michael Armbrust <mi...@databricks.com> wrote:
>>
>> According to the timeline posted on the website, we are nearing branch cut for Spark 2.3. I'd like to propose pushing this out towards mid to late December for a couple of reasons and would like to hear what people think.
>>
>> 1. I've done release management during the Thanksgiving / Christmas time before and in my experience, we don't actually get a lot of testing during this time due to vacations and other commitments. I think beginning the RC process in early January would give us the best coverage in the shortest amount of time.
>> 2. There are several large initiatives in progress that given a little more time would leave us with a much more exciting 2.3 release. Specifically, the work on the history server, Kubernetes and continuous processing.
>> 3. Given the actual release date of Spark 2.2, I think we'll still get Spark 2.3 out roughly 6 months after.
>>
>> Thoughts?
>>
>> Michael
>
>
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Timeline for Spark 2.3
Posted by Anirudh Ramanathan <fo...@google.com.INVALID>.
This would help the community on the Kubernetes effort quite a bit - giving
us additional time for reviews and testing for the 2.3 release.
On Thu, Nov 9, 2017 at 3:56 PM, Justin Miller <justin.miller@protectwise.com
> wrote:
> That sounds fine to me. I’m hoping that this ticket can make it into Spark
> 2.3: https://issues.apache.org/jira/browse/SPARK-18016
>
> It’s causing some pretty considerable problems when we alter the columns
> to be nullable, but we are OK for now without that.
>
> Best,
> Justin
>
> On Nov 9, 2017, at 4:54 PM, Michael Armbrust <mi...@databricks.com>
> wrote:
>
> According to the timeline posted on the website, we are nearing branch cut
> for Spark 2.3. I'd like to propose pushing this out towards mid to late
> December for a couple of reasons and would like to hear what people think.
>
> 1. I've done release management during the Thanksgiving / Christmas time
> before and in my experience, we don't actually get a lot of testing during
> this time due to vacations and other commitments. I think beginning the RC
> process in early January would give us the best coverage in the shortest
> amount of time.
> 2. There are several large initiatives in progress that given a little
> more time would leave us with a much more exciting 2.3 release.
> Specifically, the work on the history server, Kubernetes and continuous
> processing.
> 3. Given the actual release date of Spark 2.2, I think we'll still get
> Spark 2.3 out roughly 6 months after.
>
> Thoughts?
>
> Michael
>
>
>
Re: Timeline for Spark 2.3
Posted by Justin Miller <ju...@protectwise.com>.
That sounds fine to me. I’m hoping that this ticket can make it into Spark 2.3: https://issues.apache.org/jira/browse/SPARK-18016 <https://issues.apache.org/jira/browse/SPARK-18016>
It’s causing some pretty considerable problems when we alter the columns to be nullable, but we are OK for now without that.
Best,
Justin
> On Nov 9, 2017, at 4:54 PM, Michael Armbrust <mi...@databricks.com> wrote:
>
> According to the timeline posted on the website, we are nearing branch cut for Spark 2.3. I'd like to propose pushing this out towards mid to late December for a couple of reasons and would like to hear what people think.
>
> 1. I've done release management during the Thanksgiving / Christmas time before and in my experience, we don't actually get a lot of testing during this time due to vacations and other commitments. I think beginning the RC process in early January would give us the best coverage in the shortest amount of time.
> 2. There are several large initiatives in progress that given a little more time would leave us with a much more exciting 2.3 release. Specifically, the work on the history server, Kubernetes and continuous processing.
> 3. Given the actual release date of Spark 2.2, I think we'll still get Spark 2.3 out roughly 6 months after.
>
> Thoughts?
>
> Michael