You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Serega Sheypak <se...@gmail.com> on 2015/04/22 15:41:33 UTC
cassandra and spark from cloudera distirbution
Hi, are Cassandra and Spark from Cloudera compatible?
Where can I find these compatilibity notes?
Re: cassandra and spark from cloudera distirbution
Posted by Serega Sheypak <se...@gmail.com>.
we use vanilla spark right now.
Here is an ansible script to install spark:
https://github.com/seregasheypak/ansible-vagrant-dse-spark/blob/master/roles/spark_configuration/tasks/main.yml
and cassandra dse:
https://github.com/seregasheypak/ansible-vagrant-dse-spark/blob/master/roles/dse-install/tasks/main.yml
2015-04-22 20:11 GMT+02:00 Sebastian Estevez <sebastian.estevez@datastax.com
>:
> Oh sorry Jay mentioned DSE and you said we already use it.
>
> I think the answer to your question is Brian's response.
>
> These are the dse docs if you want to read about it:
>
>
> http://docs.datastax.com/en/datastax_enterprise/4.6/datastax_enterprise/spark/sparkTOC.html
> On Apr 22, 2015 2:05 PM, "Serega Sheypak" <se...@gmail.com>
> wrote:
>
>> What is "embedded" spark? Where can I read about it?
>> Right now we just install spark 1.2 built fro hadoop 2.4 and use it to
>> query data from cassandra.
>>
>> 2015-04-22 19:56 GMT+02:00 Sebastian Estevez <
>> sebastian.estevez@datastax.com>:
>>
>>> There is no supported way to replace the embedded spark that comes in
>>> DSE with something else.
>>>
>>> However you could probably read or write from/to DSE / Cassandra from a
>>> cloudera spark cluster using the open source DataStax connector. Are you
>>> looking for a particular feature that is not available in Spark 1.1?
>>> On Apr 22, 2015 1:50 PM, "Serega Sheypak" <se...@gmail.com>
>>> wrote:
>>>
>>>> We already use it. Would like to use Spark from cloudera distribution.
>>>> Should it work?
>>>>
>>>> 2015-04-22 19:43 GMT+02:00 Jay Ken <ja...@gmail.com>:
>>>>
>>>>> There is a Enerprise Edition from Datastax; where they have Spark and
>>>>> Cassandra Integration.
>>>>>
>>>>>
>>>>> http://www.datastax.com/what-we-offer/products-services/datastax-enterprise
>>>>>
>>>>> Thanks,
>>>>> Jay
>>>>>
>>>>> On Wed, Apr 22, 2015 at 6:41 AM, Serega Sheypak <
>>>>> serega.sheypak@gmail.com> wrote:
>>>>>
>>>>>> Hi, are Cassandra and Spark from Cloudera compatible?
>>>>>> Where can I find these compatilibity notes?
>>>>>>
>>>>>
>>>>>
>>>>
>>
Re: cassandra and spark from cloudera distirbution
Posted by Sebastian Estevez <se...@datastax.com>.
Oh sorry Jay mentioned DSE and you said we already use it.
I think the answer to your question is Brian's response.
These are the dse docs if you want to read about it:
http://docs.datastax.com/en/datastax_enterprise/4.6/datastax_enterprise/spark/sparkTOC.html
On Apr 22, 2015 2:05 PM, "Serega Sheypak" <se...@gmail.com> wrote:
> What is "embedded" spark? Where can I read about it?
> Right now we just install spark 1.2 built fro hadoop 2.4 and use it to
> query data from cassandra.
>
> 2015-04-22 19:56 GMT+02:00 Sebastian Estevez <
> sebastian.estevez@datastax.com>:
>
>> There is no supported way to replace the embedded spark that comes in DSE
>> with something else.
>>
>> However you could probably read or write from/to DSE / Cassandra from a
>> cloudera spark cluster using the open source DataStax connector. Are you
>> looking for a particular feature that is not available in Spark 1.1?
>> On Apr 22, 2015 1:50 PM, "Serega Sheypak" <se...@gmail.com>
>> wrote:
>>
>>> We already use it. Would like to use Spark from cloudera distribution.
>>> Should it work?
>>>
>>> 2015-04-22 19:43 GMT+02:00 Jay Ken <ja...@gmail.com>:
>>>
>>>> There is a Enerprise Edition from Datastax; where they have Spark and
>>>> Cassandra Integration.
>>>>
>>>>
>>>> http://www.datastax.com/what-we-offer/products-services/datastax-enterprise
>>>>
>>>> Thanks,
>>>> Jay
>>>>
>>>> On Wed, Apr 22, 2015 at 6:41 AM, Serega Sheypak <
>>>> serega.sheypak@gmail.com> wrote:
>>>>
>>>>> Hi, are Cassandra and Spark from Cloudera compatible?
>>>>> Where can I find these compatilibity notes?
>>>>>
>>>>
>>>>
>>>
>
Re: cassandra and spark from cloudera distirbution
Posted by Serega Sheypak <se...@gmail.com>.
What is "embedded" spark? Where can I read about it?
Right now we just install spark 1.2 built fro hadoop 2.4 and use it to
query data from cassandra.
2015-04-22 19:56 GMT+02:00 Sebastian Estevez <sebastian.estevez@datastax.com
>:
> There is no supported way to replace the embedded spark that comes in DSE
> with something else.
>
> However you could probably read or write from/to DSE / Cassandra from a
> cloudera spark cluster using the open source DataStax connector. Are you
> looking for a particular feature that is not available in Spark 1.1?
> On Apr 22, 2015 1:50 PM, "Serega Sheypak" <se...@gmail.com>
> wrote:
>
>> We already use it. Would like to use Spark from cloudera distribution.
>> Should it work?
>>
>> 2015-04-22 19:43 GMT+02:00 Jay Ken <ja...@gmail.com>:
>>
>>> There is a Enerprise Edition from Datastax; where they have Spark and
>>> Cassandra Integration.
>>>
>>>
>>> http://www.datastax.com/what-we-offer/products-services/datastax-enterprise
>>>
>>> Thanks,
>>> Jay
>>>
>>> On Wed, Apr 22, 2015 at 6:41 AM, Serega Sheypak <
>>> serega.sheypak@gmail.com> wrote:
>>>
>>>> Hi, are Cassandra and Spark from Cloudera compatible?
>>>> Where can I find these compatilibity notes?
>>>>
>>>
>>>
>>
Re: cassandra and spark from cloudera distirbution
Posted by Sebastian Estevez <se...@datastax.com>.
There is no supported way to replace the embedded spark that comes in DSE
with something else.
However you could probably read or write from/to DSE / Cassandra from a
cloudera spark cluster using the open source DataStax connector. Are you
looking for a particular feature that is not available in Spark 1.1?
On Apr 22, 2015 1:50 PM, "Serega Sheypak" <se...@gmail.com> wrote:
> We already use it. Would like to use Spark from cloudera distribution.
> Should it work?
>
> 2015-04-22 19:43 GMT+02:00 Jay Ken <ja...@gmail.com>:
>
>> There is a Enerprise Edition from Datastax; where they have Spark and
>> Cassandra Integration.
>>
>>
>> http://www.datastax.com/what-we-offer/products-services/datastax-enterprise
>>
>> Thanks,
>> Jay
>>
>> On Wed, Apr 22, 2015 at 6:41 AM, Serega Sheypak <serega.sheypak@gmail.com
>> > wrote:
>>
>>> Hi, are Cassandra and Spark from Cloudera compatible?
>>> Where can I find these compatilibity notes?
>>>
>>
>>
>
Re: cassandra and spark from cloudera distirbution
Posted by Brian O'Neill <bo...@alumni.brown.edu>.
Depends which veresion of Spark you are running on Cloudera.
Once you know that have a look at the compatibility chart here:
https://github.com/datastax/spark-cassandra-connector
-brian
---
Brian O'Neill
Chief Technology Officer
Health Market Science, a LexisNexis Company
215.588.6024 Mobile @boneill42 <http://www.twitter.com/boneill42>
This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material. If
you received this email in error and are not the intended recipient, or the
person responsible to deliver it to the intended recipient, please contact
the sender at the email above and delete this email and any attachments and
destroy any copies thereof. Any review, retransmission, dissemination,
copying or other use of, or taking any action in reliance upon, this
information by persons or entities other than the intended recipient is
strictly prohibited.
From: Serega Sheypak <se...@gmail.com>
Reply-To: <us...@cassandra.apache.org>
Date: Wednesday, April 22, 2015 at 1:48 PM
To: user <us...@cassandra.apache.org>
Subject: Re: cassandra and spark from cloudera distirbution
We already use it. Would like to use Spark from cloudera distribution.
Should it work?
2015-04-22 19:43 GMT+02:00 Jay Ken <ja...@gmail.com>:
> There is a Enerprise Edition from Datastax; where they have Spark and
> Cassandra Integration.
>
> http://www.datastax.com/what-we-offer/products-services/datastax-enterprise
>
> Thanks,
> Jay
>
> On Wed, Apr 22, 2015 at 6:41 AM, Serega Sheypak <se...@gmail.com>
> wrote:
>> Hi, are Cassandra and Spark from Cloudera compatible?
>> Where can I find these compatilibity notes?
>
Re: cassandra and spark from cloudera distirbution
Posted by Serega Sheypak <se...@gmail.com>.
We already use it. Would like to use Spark from cloudera distribution.
Should it work?
2015-04-22 19:43 GMT+02:00 Jay Ken <ja...@gmail.com>:
> There is a Enerprise Edition from Datastax; where they have Spark and
> Cassandra Integration.
>
> http://www.datastax.com/what-we-offer/products-services/datastax-enterprise
>
> Thanks,
> Jay
>
> On Wed, Apr 22, 2015 at 6:41 AM, Serega Sheypak <se...@gmail.com>
> wrote:
>
>> Hi, are Cassandra and Spark from Cloudera compatible?
>> Where can I find these compatilibity notes?
>>
>
>
Re: cassandra and spark from cloudera distirbution
Posted by Jay Ken <ja...@gmail.com>.
There is a Enerprise Edition from Datastax; where they have Spark and
Cassandra Integration.
http://www.datastax.com/what-we-offer/products-services/datastax-enterprise
Thanks,
Jay
On Wed, Apr 22, 2015 at 6:41 AM, Serega Sheypak <se...@gmail.com>
wrote:
> Hi, are Cassandra and Spark from Cloudera compatible?
> Where can I find these compatilibity notes?
>