You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by "Deyhim, Parviz" <pa...@amazon.com> on 2014/02/14 17:53:53 UTC
Spark 0.8.1 on Amazon Elastic MapReduce
Spark community,
Wanted to let you know that the version of Spark and Shark on Amazon Elastic MapReduce has been updated to 0.8.1. This new version provides a much better experience in terms of stability and performance but also supports the following features:
* Integration with Amazon Cloudwatch
* Integration of Spark Streaming with Amazon Kinesis.
* Automatic log shipping to S3
For a complete detail of the features Spark on EMR provides, please see the following article: http://aws.amazon.com/articles/4926593393724923
And yes I'm working hard to push another update to support 0.9.0 :)
What would be great is to hear from the community on what other features you like to see on Spark on EMR. For example, how useful is autoscaling for Spark? Any other features you like to see?
Thanks,
Parviz Deyhim
Solutions Architect
Amazon Web Services<http://aws.amazon.com/>
E: parvizd@amazon.com
M: 408.315.2305
[Description: Description: Description: C:\Users\aiden\AppData\Local\Microsoft\Windows\Temporary Internet Files\Content.Word\aws.gif]<http://aws.amazon.com/>
Re: Spark 0.8.1 on Amazon Elastic MapReduce
Posted by Tathagata Das <ta...@gmail.com>.
That's interesting! I am curious to find out as well.
TD
On Fri, Feb 14, 2014 at 9:28 AM, Nick Pentreath <ni...@gmail.com>wrote:
> Thanks Parviz, this looks great and good to see it getting updated. Look
> forward to 0.9.0!
>
> A perhaps stupid question - where does the KinesisWordCount example live?
> Is that an Amazon example, since I don't see it under the streaming
> examples included in the Spark project. If it's a third party example is it
> possible to get the code?
>
> Thanks
> Nick
>
>
> On Fri, Feb 14, 2014 at 6:53 PM, Deyhim, Parviz <pa...@amazon.com>wrote:
>
>> Spark community,
>>
>> Wanted to let you know that the version of Spark and Shark on Amazon
>> Elastic MapReduce has been updated to 0.8.1. This new version provides a
>> much better experience in terms of stability and performance but also
>> supports the following features:
>>
>> - Integration with Amazon Cloudwatch
>> - Integration of Spark Streaming with Amazon Kinesis.
>> - Automatic log shipping to S3
>>
>> For a complete detail of the features Spark on EMR provides, please see
>> the following article: http://aws.amazon.com/articles/4926593393724923
>>
>> And yes I'm working hard to push another update to support 0.9.0 :)
>>
>> What would be great is to hear from the community on what other
>> features you like to see on Spark on EMR. For example, how useful is
>> autoscaling for Spark? Any other features you like to see?
>>
>> Thanks,
>>
>> *Parviz Deyhim*
>>
>> Solutions Architect
>>
>> *Amazon Web Services <http://aws.amazon.com/>*
>>
>> E: parvizd@amazon.com
>>
>> M: 408.315.2305
>>
>>
>>
>> [image: Description: Description: Description:
>> C:\Users\aiden\AppData\Local\Microsoft\Windows\Temporary Internet
>> Files\Content.Word\aws.gif] <http://aws.amazon.com/>
>>
>
>
Re: Spark 0.8.1 on Amazon Elastic MapReduce
Posted by Tathagata Das <ta...@gmail.com>.
That's interesting! I am curious to find out as well.
TD
On Fri, Feb 14, 2014 at 9:28 AM, Nick Pentreath <ni...@gmail.com>wrote:
> Thanks Parviz, this looks great and good to see it getting updated. Look
> forward to 0.9.0!
>
> A perhaps stupid question - where does the KinesisWordCount example live?
> Is that an Amazon example, since I don't see it under the streaming
> examples included in the Spark project. If it's a third party example is it
> possible to get the code?
>
> Thanks
> Nick
>
>
> On Fri, Feb 14, 2014 at 6:53 PM, Deyhim, Parviz <pa...@amazon.com>wrote:
>
>> Spark community,
>>
>> Wanted to let you know that the version of Spark and Shark on Amazon
>> Elastic MapReduce has been updated to 0.8.1. This new version provides a
>> much better experience in terms of stability and performance but also
>> supports the following features:
>>
>> - Integration with Amazon Cloudwatch
>> - Integration of Spark Streaming with Amazon Kinesis.
>> - Automatic log shipping to S3
>>
>> For a complete detail of the features Spark on EMR provides, please see
>> the following article: http://aws.amazon.com/articles/4926593393724923
>>
>> And yes I'm working hard to push another update to support 0.9.0 :)
>>
>> What would be great is to hear from the community on what other
>> features you like to see on Spark on EMR. For example, how useful is
>> autoscaling for Spark? Any other features you like to see?
>>
>> Thanks,
>>
>> *Parviz Deyhim*
>>
>> Solutions Architect
>>
>> *Amazon Web Services <http://aws.amazon.com/>*
>>
>> E: parvizd@amazon.com
>>
>> M: 408.315.2305
>>
>>
>>
>> [image: Description: Description: Description:
>> C:\Users\aiden\AppData\Local\Microsoft\Windows\Temporary Internet
>> Files\Content.Word\aws.gif] <http://aws.amazon.com/>
>>
>
>
Re: Spark 0.8.1 on Amazon Elastic MapReduce
Posted by Nick Pentreath <ni...@gmail.com>.
Thanks Parviz, this looks great and good to see it getting updated. Look
forward to 0.9.0!
A perhaps stupid question - where does the KinesisWordCount example live?
Is that an Amazon example, since I don't see it under the streaming
examples included in the Spark project. If it's a third party example is it
possible to get the code?
Thanks
Nick
On Fri, Feb 14, 2014 at 6:53 PM, Deyhim, Parviz <pa...@amazon.com> wrote:
> Spark community,
>
> Wanted to let you know that the version of Spark and Shark on Amazon
> Elastic MapReduce has been updated to 0.8.1. This new version provides a
> much better experience in terms of stability and performance but also
> supports the following features:
>
> - Integration with Amazon Cloudwatch
> - Integration of Spark Streaming with Amazon Kinesis.
> - Automatic log shipping to S3
>
> For a complete detail of the features Spark on EMR provides, please see
> the following article: http://aws.amazon.com/articles/4926593393724923
>
> And yes I'm working hard to push another update to support 0.9.0 :)
>
> What would be great is to hear from the community on what other features
> you like to see on Spark on EMR. For example, how useful is autoscaling for
> Spark? Any other features you like to see?
>
> Thanks,
>
> *Parviz Deyhim*
>
> Solutions Architect
>
> *Amazon Web Services <http://aws.amazon.com/>*
>
> E: parvizd@amazon.com
>
> M: 408.315.2305
>
>
>
> [image: Description: Description: Description:
> C:\Users\aiden\AppData\Local\Microsoft\Windows\Temporary Internet
> Files\Content.Word\aws.gif] <http://aws.amazon.com/>
>
Re: Spark 0.8.1 on Amazon Elastic MapReduce
Posted by Nick Pentreath <ni...@gmail.com>.
Thanks Parviz, this looks great and good to see it getting updated. Look
forward to 0.9.0!
A perhaps stupid question - where does the KinesisWordCount example live?
Is that an Amazon example, since I don't see it under the streaming
examples included in the Spark project. If it's a third party example is it
possible to get the code?
Thanks
Nick
On Fri, Feb 14, 2014 at 6:53 PM, Deyhim, Parviz <pa...@amazon.com> wrote:
> Spark community,
>
> Wanted to let you know that the version of Spark and Shark on Amazon
> Elastic MapReduce has been updated to 0.8.1. This new version provides a
> much better experience in terms of stability and performance but also
> supports the following features:
>
> - Integration with Amazon Cloudwatch
> - Integration of Spark Streaming with Amazon Kinesis.
> - Automatic log shipping to S3
>
> For a complete detail of the features Spark on EMR provides, please see
> the following article: http://aws.amazon.com/articles/4926593393724923
>
> And yes I'm working hard to push another update to support 0.9.0 :)
>
> What would be great is to hear from the community on what other features
> you like to see on Spark on EMR. For example, how useful is autoscaling for
> Spark? Any other features you like to see?
>
> Thanks,
>
> *Parviz Deyhim*
>
> Solutions Architect
>
> *Amazon Web Services <http://aws.amazon.com/>*
>
> E: parvizd@amazon.com
>
> M: 408.315.2305
>
>
>
> [image: Description: Description: Description:
> C:\Users\aiden\AppData\Local\Microsoft\Windows\Temporary Internet
> Files\Content.Word\aws.gif] <http://aws.amazon.com/>
>