You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Kevin Burton <bu...@spinn3r.com> on 2014/11/13 06:28:44 UTC
Can spark read and write to cassandra without HDFS?
We have all our data in Cassandra so I’d prefer to not have to bring up
Hadoop/HDFS as that’s just another thing that can break.
But I’m reading that spark requires a shared filesystem like HDFS or S3…
Can I use Tachyon or this or something simple for a shared filesystem?
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: Can spark read and write to cassandra without HDFS?
Posted by Kevin Burton <bu...@spinn3r.com>.
Yes. That’s what I was planning on using actually.
I was just curious whether intermediate data had to be kept in HDFS but
this answers my question. thanks.
On Wed, Nov 12, 2014 at 9:33 PM, Harold Nguyen <ha...@nexgate.com> wrote:
> Hi Kevin,
>
> Yes, Spark can read and write to Cassandra without Hadoop. Have you seen
> this:
>
> https://github.com/datastax/spark-cassandra-connector
>
> Harold
>
> On Wed, Nov 12, 2014 at 9:28 PM, Kevin Burton <bu...@spinn3r.com> wrote:
>
>> We have all our data in Cassandra so I’d prefer to not have to bring up
>> Hadoop/HDFS as that’s just another thing that can break.
>>
>> But I’m reading that spark requires a shared filesystem like HDFS or S3…
>> Can I use Tachyon or this or something simple for a shared filesystem?
>>
>>
>> --
>>
>> Founder/CEO Spinn3r.com
>> Location: *San Francisco, CA*
>> blog: http://burtonator.wordpress.com
>> … or check out my Google+ profile
>> <https://plus.google.com/102718274791889610666/posts>
>> <http://spinn3r.com>
>>
>>
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: Can spark read and write to cassandra without HDFS?
Posted by Harold Nguyen <ha...@nexgate.com>.
Hi Kevin,
Yes, Spark can read and write to Cassandra without Hadoop. Have you seen
this:
https://github.com/datastax/spark-cassandra-connector
Harold
On Wed, Nov 12, 2014 at 9:28 PM, Kevin Burton <bu...@spinn3r.com> wrote:
> We have all our data in Cassandra so I’d prefer to not have to bring up
> Hadoop/HDFS as that’s just another thing that can break.
>
> But I’m reading that spark requires a shared filesystem like HDFS or S3…
> Can I use Tachyon or this or something simple for a shared filesystem?
>
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>
>