You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Manoj Khangaonkar <kh...@gmail.com> on 2014/05/08 23:43:38 UTC

Cassandra & MapReduce/Storm/ etc

Hi,

Searching for Cassandra with MapReduce, I am finding that the search
results are really dated -- from version 0.7 & 2010/2011.

Is there a good blog/article that describes how using MapReduce on
Cassandra table ?

>From my naive understanding, Cassandra is all about partitioning. Querying
is based on partitionkey + clustered column(s).

Inputs to MapReduce is a sequence of Key,values. For Storm it is a stream
of tuples.

If a database table is input source for MapReduce or Storm, for me , this
is in the simple case, is translating to a full table scan of the input
table, which can timeout and is generally not a recommended access pattern
in Cassandra.

My initial reaction is that if I need to process data with MapReduce or
Storm, reading it from Cassandra might not be the optimal way. Storing the
output to Cassandra however does make sense.

If anyone had links to blogs or personal experience in this area, I would
appreciate if you can share it.

regards

Re: Cassandra & MapReduce/Storm/ etc

Posted by Shamim <sr...@yandex.ru>.
Hi,
  check out these following links:
1) http://frommyworkshop.blogspot.ru/search/label/Cassandra
2) http://frommyworkshop.blogspot.ru/2012/07/single-node-hadoop-cassandra-pig-setup.html
-- 
Best regards
  Shamim A.


11.05.2014, 22:17, "Manoj Khangaonkar" <kh...@gmail.com>:
> Hi,
>
> Searching for Cassandra with MapReduce, I am finding that the search results are really dated -- from version 0.7 & 2010/2011.
>
> Is there a good blog/article that describes how using MapReduce on Cassandra table ?
>
> From my naive understanding, Cassandra is all about partitioning. Querying is based on partitionkey + clustered column(s).
>
> Inputs to MapReduce is a sequence of Key,values. For Storm it is a stream of tuples.
>
> If a database table is input source for MapReduce or Storm, for me , this is in the simple case, is translating to a full table scan of the input table, which can timeout and is generally not a recommended access pattern in Cassandra.
>
> My initial reaction is that if I need to process data with MapReduce or Storm, reading it from Cassandra might not be the optimal way. Storing the output to Cassandra however does make sense.
>
> If anyone had links to blogs or personal experience in this area, I would appreciate if you can share it.
>
> regards

Re: Cassandra & MapReduce/Storm/ etc

Posted by Aaron Morton <aa...@thelastpickle.com>.
> Is there a good blog/article that describes how using MapReduce on Cassandra table ?
The best way to get into cassandra and hadoop is to play with Cassandra DSE. 

It’s free for development, costs for production, and is an easy way to learn about hadoop integration without having to worry about the installation process.

http://www.datastax.com/docs/datastax_enterprise3.1/solutions/about_hadoop

> If a database table is input source for MapReduce or Storm, for me , this is in the simple case, is translating to a full table scan of the input table, which can timeout and is generally not a recommended access pattern in Cassandra. 
The Hadoop integration is token aware, it splits the tasks to run local on the node. The tasks then scan over the token range local to the node. 

Hope that helps. 
A

-----------------
Aaron Morton
New Zealand
@aaronmorton

Co-Founder & Principal Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com

On 9/05/2014, at 9:43 am, Manoj Khangaonkar <kh...@gmail.com> wrote:

> Hi,
> 
> Searching for Cassandra with MapReduce, I am finding that the search results are really dated -- from version 0.7 & 2010/2011.
> 
> Is there a good blog/article that describes how using MapReduce on Cassandra table ?
> 
> From my naive understanding, Cassandra is all about partitioning. Querying is based on partitionkey + clustered column(s).
> 
> Inputs to MapReduce is a sequence of Key,values. For Storm it is a stream of tuples.
> 
> If a database table is input source for MapReduce or Storm, for me , this is in the simple case, is translating to a full table scan of the input table, which can timeout and is generally not a recommended access pattern in Cassandra. 
> 
> My initial reaction is that if I need to process data with MapReduce or Storm, reading it from Cassandra might not be the optimal way. Storing the output to Cassandra however does make sense.
> 
> If anyone had links to blogs or personal experience in this area, I would appreciate if you can share it.
> 
> regards
> 
> 
> 


Re: Cassandra & MapReduce/Storm/ etc

Posted by Jack Krupansky <ja...@basetechnology.com>.
Here’s a meetup talk on analytics using Cassandra, Storm, and Kafka:
http://www.slideshare.net/aih1013/building-largescale-analytics-platform-with-storm-kafka-and-cassandra-nyc-storm-user-group-meetup-21st-nov-2013

-- Jack Krupansky

From: Manoj Khangaonkar 
Sent: Thursday, May 8, 2014 5:43 PM
To: user@cassandra.apache.org 
Subject: Cassandra & MapReduce/Storm/ etc

Hi,


Searching for Cassandra with MapReduce, I am finding that the search results are really dated -- from version 0.7 & 2010/2011.


Is there a good blog/article that describes how using MapReduce on Cassandra table ?


>From my naive understanding, Cassandra is all about partitioning. Querying is based on partitionkey + clustered column(s).


Inputs to MapReduce is a sequence of Key,values. For Storm it is a stream of tuples.


If a database table is input source for MapReduce or Storm, for me , this is in the simple case, is translating to a full table scan of the input table, which can timeout and is generally not a recommended access pattern in Cassandra. 


My initial reaction is that if I need to process data with MapReduce or Storm, reading it from Cassandra might not be the optimal way. Storing the output to Cassandra however does make sense.


If anyone had links to blogs or personal experience in this area, I would appreciate if you can share it.


regards