You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Tathagata Das <ta...@gmail.com> on 2014/04/09 23:54:03 UTC
Spark 0.9.1 released
Hi everyone,
We have just posted Spark 0.9.1, which is a maintenance release with
bug fixes, performance improvements, better stability with YARN and
improved parity of the Scala and Python API. We recommend all 0.9.0
users to upgrade to this stable release.
This is the first release since Spark graduated as a top level Apache
project. Contributions to this release came from 37 developers.
The full release notes are at:
http://spark.apache.org/releases/spark-release-0-9-1.html
You can download the release at:
http://spark.apache.org/downloads.html
Thanks all the developers who contributed to this release:
Aaron Davidson, Aaron Kimball, Andrew Ash, Andrew Or, Andrew Tulloch,
Bijay Bisht, Bouke van der Bijl, Bryn Keller, Chen Chao,
Christian Lundgren, Diana Carroll, Emtiaz Ahmed, Frank Dai,
Henry Saputra, jianghan, Josh Rosen, Jyotiska NK, Kay Ousterhout,
Kousuke Saruta, Mark Grover, Matei Zaharia, Nan Zhu, Nick Lanham,
Patrick Wendell, Prabin Banka, Prashant Sharma, Qiuzhuang,
Raymond Liu, Reynold Xin, Sandy Ryza, Sean Owen, Shixiong Zhu,
shiyun.wxm, Stevo Slavić, Tathagata Das, Tom Graves, Xiangrui Meng
TD
Re: Spark 0.9.1 released
Posted by Nicholas Chammas <ni...@gmail.com>.
Ah, looks good now. It took me a minute to realize that doing a hard
refresh on the docs page was missing the RDD class doc page...
And thanks for updating the release notes.
On Wed, Apr 9, 2014 at 7:21 PM, Tathagata Das
<ta...@gmail.com>wrote:
> Thanks Nick for pointing that out! I have updated the release notes<http://spark.apache.org/releases/spark-release-0-9-1.html>.
> But I see the new operations like repartition in the latest PySpark RDD
> docs <http://spark.apache.org/docs/latest/api/pyspark/index.html>. Maybe
> refresh the page couple of times?
>
> TD
>
>
> On Wed, Apr 9, 2014 at 3:58 PM, Nicholas Chammas <
> nicholas.chammas@gmail.com> wrote:
>
>> A very nice addition for us PySpark users in 0.9.1 is the addition of
>> RDD.repartition(), which is not mentioned in the release notes<http://spark.apache.org/releases/spark-release-0-9-1.html>
>> !
>>
>> This is super helpful for when you create an RDD from a gzipped file and
>> then need to explicitly shuffle the data around to parallelize operations
>> on it appropriately.
>>
>> Thanks people!
>>
>> FYI, docs/latest<http://spark.apache.org/docs/latest/api/pyspark/index.html>hasn't been updated yet to reflect the new additions to PySpark.
>>
>> Nick
>>
>>
>>
>> On Wed, Apr 9, 2014 at 6:07 PM, Matei Zaharia <ma...@gmail.com>wrote:
>>
>>> Thanks TD for managing this release, and thanks to everyone who
>>> contributed!
>>>
>>> Matei
>>>
>>> On Apr 9, 2014, at 2:59 PM, Tathagata Das <ta...@gmail.com>
>>> wrote:
>>>
>>> A small additional note: Please use the direct download links in the
>>> Spark Downloads <http://spark.apache.org/downloads.html> page. The
>>> Apache mirrors take a day or so to sync from the main repo, so may not work
>>> immediately.
>>>
>>> TD
>>>
>>>
>>> On Wed, Apr 9, 2014 at 2:54 PM, Tathagata Das <
>>> tathagata.das1565@gmail.com> wrote:
>>>
>>>> Hi everyone,
>>>>
>>>> We have just posted Spark 0.9.1, which is a maintenance release with
>>>> bug fixes, performance improvements, better stability with YARN and
>>>> improved parity of the Scala and Python API. We recommend all 0.9.0
>>>> users to upgrade to this stable release.
>>>>
>>>> This is the first release since Spark graduated as a top level Apache
>>>> project. Contributions to this release came from 37 developers.
>>>>
>>>> The full release notes are at:
>>>> http://spark.apache.org/releases/spark-release-0-9-1.html
>>>>
>>>> You can download the release at:
>>>> http://spark.apache.org/downloads.html
>>>>
>>>> Thanks all the developers who contributed to this release:
>>>> Aaron Davidson, Aaron Kimball, Andrew Ash, Andrew Or, Andrew Tulloch,
>>>> Bijay Bisht, Bouke van der Bijl, Bryn Keller, Chen Chao,
>>>> Christian Lundgren, Diana Carroll, Emtiaz Ahmed, Frank Dai,
>>>> Henry Saputra, jianghan, Josh Rosen, Jyotiska NK, Kay Ousterhout,
>>>> Kousuke Saruta, Mark Grover, Matei Zaharia, Nan Zhu, Nick Lanham,
>>>> Patrick Wendell, Prabin Banka, Prashant Sharma, Qiuzhuang,
>>>> Raymond Liu, Reynold Xin, Sandy Ryza, Sean Owen, Shixiong Zhu,
>>>> shiyun.wxm, Stevo Slavić, Tathagata Das, Tom Graves, Xiangrui Meng
>>>>
>>>> TD
>>>>
>>>
>>>
>>>
>>
>
Re: Spark 0.9.1 released
Posted by Tathagata Das <ta...@gmail.com>.
Thanks Nick for pointing that out! I have updated the release
notes<http://spark.apache.org/releases/spark-release-0-9-1.html>.
But I see the new operations like repartition in the latest PySpark
RDD docs<http://spark.apache.org/docs/latest/api/pyspark/index.html>.
Maybe refresh the page couple of times?
TD
On Wed, Apr 9, 2014 at 3:58 PM, Nicholas Chammas <nicholas.chammas@gmail.com
> wrote:
> A very nice addition for us PySpark users in 0.9.1 is the addition of
> RDD.repartition(), which is not mentioned in the release notes<http://spark.apache.org/releases/spark-release-0-9-1.html>
> !
>
> This is super helpful for when you create an RDD from a gzipped file and
> then need to explicitly shuffle the data around to parallelize operations
> on it appropriately.
>
> Thanks people!
>
> FYI, docs/latest<http://spark.apache.org/docs/latest/api/pyspark/index.html>hasn't been updated yet to reflect the new additions to PySpark.
>
> Nick
>
>
>
> On Wed, Apr 9, 2014 at 6:07 PM, Matei Zaharia <ma...@gmail.com>wrote:
>
>> Thanks TD for managing this release, and thanks to everyone who
>> contributed!
>>
>> Matei
>>
>> On Apr 9, 2014, at 2:59 PM, Tathagata Das <ta...@gmail.com>
>> wrote:
>>
>> A small additional note: Please use the direct download links in the
>> Spark Downloads <http://spark.apache.org/downloads.html> page. The
>> Apache mirrors take a day or so to sync from the main repo, so may not work
>> immediately.
>>
>> TD
>>
>>
>> On Wed, Apr 9, 2014 at 2:54 PM, Tathagata Das <
>> tathagata.das1565@gmail.com> wrote:
>>
>>> Hi everyone,
>>>
>>> We have just posted Spark 0.9.1, which is a maintenance release with
>>> bug fixes, performance improvements, better stability with YARN and
>>> improved parity of the Scala and Python API. We recommend all 0.9.0
>>> users to upgrade to this stable release.
>>>
>>> This is the first release since Spark graduated as a top level Apache
>>> project. Contributions to this release came from 37 developers.
>>>
>>> The full release notes are at:
>>> http://spark.apache.org/releases/spark-release-0-9-1.html
>>>
>>> You can download the release at:
>>> http://spark.apache.org/downloads.html
>>>
>>> Thanks all the developers who contributed to this release:
>>> Aaron Davidson, Aaron Kimball, Andrew Ash, Andrew Or, Andrew Tulloch,
>>> Bijay Bisht, Bouke van der Bijl, Bryn Keller, Chen Chao,
>>> Christian Lundgren, Diana Carroll, Emtiaz Ahmed, Frank Dai,
>>> Henry Saputra, jianghan, Josh Rosen, Jyotiska NK, Kay Ousterhout,
>>> Kousuke Saruta, Mark Grover, Matei Zaharia, Nan Zhu, Nick Lanham,
>>> Patrick Wendell, Prabin Banka, Prashant Sharma, Qiuzhuang,
>>> Raymond Liu, Reynold Xin, Sandy Ryza, Sean Owen, Shixiong Zhu,
>>> shiyun.wxm, Stevo Slavić, Tathagata Das, Tom Graves, Xiangrui Meng
>>>
>>> TD
>>>
>>
>>
>>
>
Re: Spark 0.9.1 released
Posted by Tathagata Das <ta...@gmail.com>.
Thanks Nick for pointing that out! I have updated the release
notes<http://spark.apache.org/releases/spark-release-0-9-1.html>.
But I see the new operations like repartition in the latest PySpark
RDD docs<http://spark.apache.org/docs/latest/api/pyspark/index.html>.
Maybe refresh the page couple of times?
TD
On Wed, Apr 9, 2014 at 3:58 PM, Nicholas Chammas <nicholas.chammas@gmail.com
> wrote:
> A very nice addition for us PySpark users in 0.9.1 is the addition of
> RDD.repartition(), which is not mentioned in the release notes<http://spark.apache.org/releases/spark-release-0-9-1.html>
> !
>
> This is super helpful for when you create an RDD from a gzipped file and
> then need to explicitly shuffle the data around to parallelize operations
> on it appropriately.
>
> Thanks people!
>
> FYI, docs/latest<http://spark.apache.org/docs/latest/api/pyspark/index.html>hasn't been updated yet to reflect the new additions to PySpark.
>
> Nick
>
>
>
> On Wed, Apr 9, 2014 at 6:07 PM, Matei Zaharia <ma...@gmail.com>wrote:
>
>> Thanks TD for managing this release, and thanks to everyone who
>> contributed!
>>
>> Matei
>>
>> On Apr 9, 2014, at 2:59 PM, Tathagata Das <ta...@gmail.com>
>> wrote:
>>
>> A small additional note: Please use the direct download links in the
>> Spark Downloads <http://spark.apache.org/downloads.html> page. The
>> Apache mirrors take a day or so to sync from the main repo, so may not work
>> immediately.
>>
>> TD
>>
>>
>> On Wed, Apr 9, 2014 at 2:54 PM, Tathagata Das <
>> tathagata.das1565@gmail.com> wrote:
>>
>>> Hi everyone,
>>>
>>> We have just posted Spark 0.9.1, which is a maintenance release with
>>> bug fixes, performance improvements, better stability with YARN and
>>> improved parity of the Scala and Python API. We recommend all 0.9.0
>>> users to upgrade to this stable release.
>>>
>>> This is the first release since Spark graduated as a top level Apache
>>> project. Contributions to this release came from 37 developers.
>>>
>>> The full release notes are at:
>>> http://spark.apache.org/releases/spark-release-0-9-1.html
>>>
>>> You can download the release at:
>>> http://spark.apache.org/downloads.html
>>>
>>> Thanks all the developers who contributed to this release:
>>> Aaron Davidson, Aaron Kimball, Andrew Ash, Andrew Or, Andrew Tulloch,
>>> Bijay Bisht, Bouke van der Bijl, Bryn Keller, Chen Chao,
>>> Christian Lundgren, Diana Carroll, Emtiaz Ahmed, Frank Dai,
>>> Henry Saputra, jianghan, Josh Rosen, Jyotiska NK, Kay Ousterhout,
>>> Kousuke Saruta, Mark Grover, Matei Zaharia, Nan Zhu, Nick Lanham,
>>> Patrick Wendell, Prabin Banka, Prashant Sharma, Qiuzhuang,
>>> Raymond Liu, Reynold Xin, Sandy Ryza, Sean Owen, Shixiong Zhu,
>>> shiyun.wxm, Stevo Slavić, Tathagata Das, Tom Graves, Xiangrui Meng
>>>
>>> TD
>>>
>>
>>
>>
>
Re: Spark 0.9.1 released
Posted by Nicholas Chammas <ni...@gmail.com>.
A very nice addition for us PySpark users in 0.9.1 is the addition of
RDD.repartition(), which is not mentioned in the release
notes<http://spark.apache.org/releases/spark-release-0-9-1.html>
!
This is super helpful for when you create an RDD from a gzipped file and
then need to explicitly shuffle the data around to parallelize operations
on it appropriately.
Thanks people!
FYI, docs/latest<http://spark.apache.org/docs/latest/api/pyspark/index.html>hasn't
been updated yet to reflect the new additions to PySpark.
Nick
On Wed, Apr 9, 2014 at 6:07 PM, Matei Zaharia <ma...@gmail.com>wrote:
> Thanks TD for managing this release, and thanks to everyone who
> contributed!
>
> Matei
>
> On Apr 9, 2014, at 2:59 PM, Tathagata Das <ta...@gmail.com>
> wrote:
>
> A small additional note: Please use the direct download links in the Spark
> Downloads <http://spark.apache.org/downloads.html> page. The Apache
> mirrors take a day or so to sync from the main repo, so may not work
> immediately.
>
> TD
>
>
> On Wed, Apr 9, 2014 at 2:54 PM, Tathagata Das <tathagata.das1565@gmail.com
> > wrote:
>
>> Hi everyone,
>>
>> We have just posted Spark 0.9.1, which is a maintenance release with
>> bug fixes, performance improvements, better stability with YARN and
>> improved parity of the Scala and Python API. We recommend all 0.9.0
>> users to upgrade to this stable release.
>>
>> This is the first release since Spark graduated as a top level Apache
>> project. Contributions to this release came from 37 developers.
>>
>> The full release notes are at:
>> http://spark.apache.org/releases/spark-release-0-9-1.html
>>
>> You can download the release at:
>> http://spark.apache.org/downloads.html
>>
>> Thanks all the developers who contributed to this release:
>> Aaron Davidson, Aaron Kimball, Andrew Ash, Andrew Or, Andrew Tulloch,
>> Bijay Bisht, Bouke van der Bijl, Bryn Keller, Chen Chao,
>> Christian Lundgren, Diana Carroll, Emtiaz Ahmed, Frank Dai,
>> Henry Saputra, jianghan, Josh Rosen, Jyotiska NK, Kay Ousterhout,
>> Kousuke Saruta, Mark Grover, Matei Zaharia, Nan Zhu, Nick Lanham,
>> Patrick Wendell, Prabin Banka, Prashant Sharma, Qiuzhuang,
>> Raymond Liu, Reynold Xin, Sandy Ryza, Sean Owen, Shixiong Zhu,
>> shiyun.wxm, Stevo Slavić, Tathagata Das, Tom Graves, Xiangrui Meng
>>
>> TD
>>
>
>
>
Re: Spark 0.9.1 released
Posted by Matei Zaharia <ma...@gmail.com>.
Thanks TD for managing this release, and thanks to everyone who contributed!
Matei
On Apr 9, 2014, at 2:59 PM, Tathagata Das <ta...@gmail.com> wrote:
> A small additional note: Please use the direct download links in the Spark Downloads page. The Apache mirrors take a day or so to sync from the main repo, so may not work immediately.
>
> TD
>
>
> On Wed, Apr 9, 2014 at 2:54 PM, Tathagata Das <ta...@gmail.com> wrote:
> Hi everyone,
>
> We have just posted Spark 0.9.1, which is a maintenance release with
> bug fixes, performance improvements, better stability with YARN and
> improved parity of the Scala and Python API. We recommend all 0.9.0
> users to upgrade to this stable release.
>
> This is the first release since Spark graduated as a top level Apache
> project. Contributions to this release came from 37 developers.
>
> The full release notes are at:
> http://spark.apache.org/releases/spark-release-0-9-1.html
>
> You can download the release at:
> http://spark.apache.org/downloads.html
>
> Thanks all the developers who contributed to this release:
> Aaron Davidson, Aaron Kimball, Andrew Ash, Andrew Or, Andrew Tulloch,
> Bijay Bisht, Bouke van der Bijl, Bryn Keller, Chen Chao,
> Christian Lundgren, Diana Carroll, Emtiaz Ahmed, Frank Dai,
> Henry Saputra, jianghan, Josh Rosen, Jyotiska NK, Kay Ousterhout,
> Kousuke Saruta, Mark Grover, Matei Zaharia, Nan Zhu, Nick Lanham,
> Patrick Wendell, Prabin Banka, Prashant Sharma, Qiuzhuang,
> Raymond Liu, Reynold Xin, Sandy Ryza, Sean Owen, Shixiong Zhu,
> shiyun.wxm, Stevo Slavić, Tathagata Das, Tom Graves, Xiangrui Meng
>
> TD
>
Re: Spark 0.9.1 released
Posted by Matei Zaharia <ma...@gmail.com>.
Thanks TD for managing this release, and thanks to everyone who contributed!
Matei
On Apr 9, 2014, at 2:59 PM, Tathagata Das <ta...@gmail.com> wrote:
> A small additional note: Please use the direct download links in the Spark Downloads page. The Apache mirrors take a day or so to sync from the main repo, so may not work immediately.
>
> TD
>
>
> On Wed, Apr 9, 2014 at 2:54 PM, Tathagata Das <ta...@gmail.com> wrote:
> Hi everyone,
>
> We have just posted Spark 0.9.1, which is a maintenance release with
> bug fixes, performance improvements, better stability with YARN and
> improved parity of the Scala and Python API. We recommend all 0.9.0
> users to upgrade to this stable release.
>
> This is the first release since Spark graduated as a top level Apache
> project. Contributions to this release came from 37 developers.
>
> The full release notes are at:
> http://spark.apache.org/releases/spark-release-0-9-1.html
>
> You can download the release at:
> http://spark.apache.org/downloads.html
>
> Thanks all the developers who contributed to this release:
> Aaron Davidson, Aaron Kimball, Andrew Ash, Andrew Or, Andrew Tulloch,
> Bijay Bisht, Bouke van der Bijl, Bryn Keller, Chen Chao,
> Christian Lundgren, Diana Carroll, Emtiaz Ahmed, Frank Dai,
> Henry Saputra, jianghan, Josh Rosen, Jyotiska NK, Kay Ousterhout,
> Kousuke Saruta, Mark Grover, Matei Zaharia, Nan Zhu, Nick Lanham,
> Patrick Wendell, Prabin Banka, Prashant Sharma, Qiuzhuang,
> Raymond Liu, Reynold Xin, Sandy Ryza, Sean Owen, Shixiong Zhu,
> shiyun.wxm, Stevo Slavić, Tathagata Das, Tom Graves, Xiangrui Meng
>
> TD
>
Re: Spark 0.9.1 released
Posted by Tathagata Das <ta...@gmail.com>.
A small additional note: Please use the direct download links in the Spark
Downloads <http://spark.apache.org/downloads.html> page. The Apache mirrors
take a day or so to sync from the main repo, so may not work immediately.
TD
On Wed, Apr 9, 2014 at 2:54 PM, Tathagata Das
<ta...@gmail.com>wrote:
> Hi everyone,
>
> We have just posted Spark 0.9.1, which is a maintenance release with
> bug fixes, performance improvements, better stability with YARN and
> improved parity of the Scala and Python API. We recommend all 0.9.0
> users to upgrade to this stable release.
>
> This is the first release since Spark graduated as a top level Apache
> project. Contributions to this release came from 37 developers.
>
> The full release notes are at:
> http://spark.apache.org/releases/spark-release-0-9-1.html
>
> You can download the release at:
> http://spark.apache.org/downloads.html
>
> Thanks all the developers who contributed to this release:
> Aaron Davidson, Aaron Kimball, Andrew Ash, Andrew Or, Andrew Tulloch,
> Bijay Bisht, Bouke van der Bijl, Bryn Keller, Chen Chao,
> Christian Lundgren, Diana Carroll, Emtiaz Ahmed, Frank Dai,
> Henry Saputra, jianghan, Josh Rosen, Jyotiska NK, Kay Ousterhout,
> Kousuke Saruta, Mark Grover, Matei Zaharia, Nan Zhu, Nick Lanham,
> Patrick Wendell, Prabin Banka, Prashant Sharma, Qiuzhuang,
> Raymond Liu, Reynold Xin, Sandy Ryza, Sean Owen, Shixiong Zhu,
> shiyun.wxm, Stevo Slavić, Tathagata Das, Tom Graves, Xiangrui Meng
>
> TD
>
Re: Spark 0.9.1 released
Posted by Tathagata Das <ta...@gmail.com>.
A small additional note: Please use the direct download links in the Spark
Downloads <http://spark.apache.org/downloads.html> page. The Apache mirrors
take a day or so to sync from the main repo, so may not work immediately.
TD
On Wed, Apr 9, 2014 at 2:54 PM, Tathagata Das
<ta...@gmail.com>wrote:
> Hi everyone,
>
> We have just posted Spark 0.9.1, which is a maintenance release with
> bug fixes, performance improvements, better stability with YARN and
> improved parity of the Scala and Python API. We recommend all 0.9.0
> users to upgrade to this stable release.
>
> This is the first release since Spark graduated as a top level Apache
> project. Contributions to this release came from 37 developers.
>
> The full release notes are at:
> http://spark.apache.org/releases/spark-release-0-9-1.html
>
> You can download the release at:
> http://spark.apache.org/downloads.html
>
> Thanks all the developers who contributed to this release:
> Aaron Davidson, Aaron Kimball, Andrew Ash, Andrew Or, Andrew Tulloch,
> Bijay Bisht, Bouke van der Bijl, Bryn Keller, Chen Chao,
> Christian Lundgren, Diana Carroll, Emtiaz Ahmed, Frank Dai,
> Henry Saputra, jianghan, Josh Rosen, Jyotiska NK, Kay Ousterhout,
> Kousuke Saruta, Mark Grover, Matei Zaharia, Nan Zhu, Nick Lanham,
> Patrick Wendell, Prabin Banka, Prashant Sharma, Qiuzhuang,
> Raymond Liu, Reynold Xin, Sandy Ryza, Sean Owen, Shixiong Zhu,
> shiyun.wxm, Stevo Slavić, Tathagata Das, Tom Graves, Xiangrui Meng
>
> TD
>