querying time series from hadoop

[mete <ef...@gmail.com>]

Hello  folks,

i have come up with a basic time series cql schema based on the articles
here:

http://www.datastax.com/dev/blog/advanced-time-series-with-cassandra

so simply put its something like:

rowkey, timestamp, col3, col4 etc...

where rowkey and timestamp are compound keys.

Where i am having issues is to efficiently query this data structure.

When i use cqlsh and query it is perfectly fine:

select * from table where rowkey='row key' and date > xxx and date <= yyy

but when i am using the java driver, the driver already uses row key for
token statements and i cannot execute the query above, therefore it does a
full scan of rows.

The issue that i am having is discussed here:

http://stackoverflow.com/questions/19189649/composite-key-in-cassandra-with-pig

i have gone through the relevant jira issues 6151 and 6311. This behaviour
is supposed to be fixed in 2.0.x but so far it is not there. So now i will
try to patch my cassandra 1.2.11 installation but i just wanted to ask you
guys first, if there is any other solution that does not involve a release.

i assume that this is somewhat a common use case, the articles i referred
seems to be old enough and unless i am missing something obvious i cannot
query this schema efficiently with the current version (1.2.x or 2.0.x)

Does anyone has a similar issue? Any pointers are welcome.

Regards
Mete

Re: querying time series from hadoop

[Aaron Morton <aa...@thelastpickle.com>]

> So now i will try to patch my cassandra 1.2.11 installation but i just wanted to ask you guys first, if there is any other solution that does not involve a release.
That patch in CASSANDRA-6311 is for 2.0 you cannot apply it to 1.2

> but when i am using the java driver, the driver already uses row key for token statements and i cannot execute the query above, therefore it does a full scan of rows.
The  ColumnFamilyRecordReader is designed to read lots of rows, not a single row. 

You should be able to use the java driver from a hadoop task though to read a single row. Can you provide some more info on what you are doing ? 

Cheers

-----------------
Aaron Morton
New Zealand
@aaronmorton

Co-Founder & Principal Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com

On 26/12/2013, at 9:56 pm, mete <ef...@gmail.com> wrote:

> Hello  folks, 
> 
> i have come up with a basic time series cql schema based on the articles here:
> 
> http://www.datastax.com/dev/blog/advanced-time-series-with-cassandra
> 
> so simply put its something like:
> 
> rowkey, timestamp, col3, col4 etc... 
> 
> where rowkey and timestamp are compound keys.
> 
> Where i am having issues is to efficiently query this data structure. 
> 
> When i use cqlsh and query it is perfectly fine:
> 
> select * from table where rowkey='row key' and date > xxx and date <= yyy
> 
> but when i am using the java driver, the driver already uses row key for token statements and i cannot execute the query above, therefore it does a full scan of rows.
> 
> The issue that i am having is discussed here:
> 
> http://stackoverflow.com/questions/19189649/composite-key-in-cassandra-with-pig
> 
> i have gone through the relevant jira issues 6151 and 6311. This behaviour is supposed to be fixed in 2.0.x but so far it is not there. So now i will try to patch my cassandra 1.2.11 installation but i just wanted to ask you guys first, if there is any other solution that does not involve a release.
> 
> i assume that this is somewhat a common use case, the articles i referred seems to be old enough and unless i am missing something obvious i cannot query this schema efficiently with the current version (1.2.x or 2.0.x)
> 
> Does anyone has a similar issue? Any pointers are welcome.
> 
> Regards
> Mete
> 
> 
> 
>