You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by "Ayush V." <ay...@gmail.com> on 2012/12/11 17:46:11 UTC
Filter data on row key in Cassandra Hadoop's Random Partitioner
I'm working on Cassandra Hadoop intergration (MapReduce). We have used Random
Partioner to insert data to gain faster write. Now we have to read that data
from cassandra in MapReduce and perform some calculation on it.
>From the lots of data we have in cassandra we wan't to fetch data only for
particular ROW-KEYs but we are unable to do it due to RandomPartioner -
assertion is there in code.
Can anyone please guide me how should I filter data based on RowKey on
Cassandra level itself (I know data is distributed across regions using Hash
of the RowKey)?
Does using secondary indexes (still trying to understand how it works) will
solve my problem or is there some other way around?
I will be really appreciated if someone could answer my queries.
Thanks
AV
--
View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Filter-data-on-row-key-in-Cassandra-Hadoop-s-Random-Partitioner-tp7584212.html
Sent from the cassandra-user@incubator.apache.org mailing list archive at Nabble.com.
Re: Filter data on row key in Cassandra Hadoop's Random Partitioner
Posted by "Ayush V." <ay...@gmail.com>.
Thanks Hiller and Shamim.
Let me share more details. I want to use cassandra MR to calculate some
KPI's on the data which is stored in cassandra continuously. So here
fetching whole data from cassandra every time seems an overhead to me?
The rowkey I'm using is like "(timestamp/60000)_otherid"; this CF contains
reference of rowkeys of actual data stored in other CF. so to calculate KPI
I will work for a particular minute and fetch data from other CF, and
process it.
--
View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Filter-data-on-row-key-in-Cassandra-Hadoop-s-Random-Partitioner-tp7584212p7584263.html
Sent from the cassandra-user@incubator.apache.org mailing list archive at Nabble.com.
Re: Filter data on row key in Cassandra Hadoop's Random Partitioner
Posted by Шамим <sr...@yandex.ru>.
You can use Apache PIG to load data and filter it by row key, filter in pig is very fast.
Regards
Shamim
11.12.2012, 20:46, "Ayush V." <ay...@gmail.com>:
> I'm working on Cassandra Hadoop intergration (MapReduce). We have used Random
> Partioner to insert data to gain faster write. Now we have to read that data
> from cassandra in MapReduce and perform some calculation on it.
>
> From the lots of data we have in cassandra we wan't to fetch data only for
> particular ROW-KEYs but we are unable to do it due to RandomPartioner -
> assertion is there in code.
>
> Can anyone please guide me how should I filter data based on RowKey on
> Cassandra level itself (I know data is distributed across regions using Hash
> of the RowKey)?
>
> Does using secondary indexes (still trying to understand how it works) will
> solve my problem or is there some other way around?
>
> I will be really appreciated if someone could answer my queries.
>
> Thanks
> AV
>
> --
> View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Filter-data-on-row-key-in-Cassandra-Hadoop-s-Random-Partitioner-tp7584212.html
> Sent from the cassandra-user@incubator.apache.org mailing list archive at Nabble.com.
Re: Filter data on row key in Cassandra Hadoop's Random Partitioner
Posted by "Hiller, Dean" <De...@nrel.gov>.
You may want to look into CQL3 as I hear there may be a way to specify the
query so only those rows are map/reduced. I am not sure if that is out
yet or not but I remember someone from datastax telling me about it.
Dean
On 12/11/12 9:46 AM, "Ayush V." <ay...@gmail.com> wrote:
>I'm working on Cassandra Hadoop intergration (MapReduce). We have used
>Random
>Partioner to insert data to gain faster write. Now we have to read that
>data
>from cassandra in MapReduce and perform some calculation on it.
>
>From the lots of data we have in cassandra we wan't to fetch data only for
>particular ROW-KEYs but we are unable to do it due to RandomPartioner -
>assertion is there in code.
>
>Can anyone please guide me how should I filter data based on RowKey on
>Cassandra level itself (I know data is distributed across regions using
>Hash
>of the RowKey)?
>
>Does using secondary indexes (still trying to understand how it works)
>will
>solve my problem or is there some other way around?
>
>I will be really appreciated if someone could answer my queries.
>
>Thanks
>AV
>
>
>
>--
>View this message in context:
>http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Filter-da
>ta-on-row-key-in-Cassandra-Hadoop-s-Random-Partitioner-tp7584212.html
>Sent from the cassandra-user@incubator.apache.org mailing list archive at
>Nabble.com.