You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Roland Gude <ro...@yoochoose.com> on 2011/03/04 17:09:55 UTC
AW: cant seem to figure out secondary index definition
Hi again,
I am still suffering from this error (which severely limits testability for me right now). Doesn't anybody have an idea, why IndexSliceQueries work if the index is created with cassandra-cli and why it does not if the index is created with cassandra.yaml? Is there some way to find out, which indexes exist for which columns in a given cluster?
Any help would be greatly appreciated
-----Ursprüngliche Nachricht-----
Von: Roland Gude [mailto:roland.gude@yoochoose.com]
Gesendet: Montag, 21. Februar 2011 18:36
An: user@cassandra.apache.org
Betreff: Re: cant seem to figure out secondary index definition
Yes,
It has such a Clause. I am very certain that this is Not my Code because the very Same program Works against a Cluster of the Index is created with the cli and it does not, when the Index is configured with cassandra.yaml
My assumption is, that the Index Kreation with configured file is flawed (it dös Not Seem to use the Same Code As configuration Parameters are named differently) i suspekt it to create the Index for the wrong Column.
Greetings,
Roland.
Am 17.r.2011 um 21:46 schrieb "Nate McCall" <na...@datastax.com>:
> How are you constructing the IndexSlicesQuery? Does it have an equals
> clause with that UUID as the column name?
>
> On Thu, Feb 17, 2011 at 11:32 AM, Roland Gude <ro...@yoochoose.com> wrote:
>> Hi again,
>>
>>
>>
>> i am still having trouble with this.
>>
>> If I define the index using cli with these commands:
>>
>> create column family A with column_type='Standard' and
>> comparator='TimeUUIDType' and keys_cached=200000 and read_repair_chance=1.0
>> and rows_cached=0.0 and column_metadata=[{column_name:
>> 00000000-0000-1000-0000-000000000000, validation_class: UTF8Type,
>> index_name: MyIndex, index_type: KEYS}];
>>
>> create column family B with column_type='Standard' and
>> comparator='TimeUUIDType' and keys_cached=200000 and read_repair_chance=1.0
>> and rows_cached=0.0 and column_metadata=[{column_name:
>> 00000000-0000-1000-0000-000000000000, validation_class: UTF8Type,
>> index_name: MyIndex, index_type: KEYS}];
>>
>>
>>
>> I can do IndexedSliceQueries as expected
>>
>>
>>
>> In my unit tests where I use an embedded Cassandra instance configured via
>> yaml like this:
>>
>> - column_metadata: [{name: 00000000-0000-1000-0000-000000000000,
>> validator_class: UTF8Type, index_name: MyIndex, index_type: KEYS}]
>>
>> compare_with: TimeUUIDType
>>
>> gc_grace_seconds: 864000
>>
>> keys_cached: 0.0
>>
>> max_compaction_threshold: 32
>>
>> min_compaction_threshold: 4
>>
>> name: A
>>
>> read_repair_chance: 1.0
>>
>> rows_cached: 0.0
>>
>> - column_metadata: [{name: 00000000-0000-1000-0000-000000000000,
>> validator_class: UTF8Type, index_name: MyIndex, index_type: KEYS}]
>>
>> compare_with: TimeUUIDType
>>
>> gc_grace_seconds: 864000
>>
>> keys_cached: 0.0
>>
>> max_compaction_threshold: 32
>>
>> min_compaction_threshold: 4
>>
>> name: B
>>
>> read_repair_chance: 1.0
>>
>> rows_cached: 0.0
>>
>>
>>
>> I get these Exceptions:
>>
>>
>>
>> 18:23:55.973 [CassandraDataFetcher-queries] ERROR
>> c.y.s.c.i.event.CassandraDataFetcher - Query
>> me.prettyprint.cassandra.model.IndexedSlicesQuery@1bbd3e2 failed, stop
>> query.
>>
>> me.prettyprint.hector.api.exceptions.HInvalidRequestException:
>> InvalidRequestException(why:No indexed columns present in index clause with
>> operator EQ)
>>
>> at
>> me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:42)
>> ~[hector-core-0.7.0-26.jar:na]
>>
>> at
>> me.prettyprint.cassandra.service.KeyspaceServiceImpl$12.execute(KeyspaceServiceImpl.java:513)
>> ~[hector-core-0.7.0-26.jar:na]
>>
>> at
>> me.prettyprint.cassandra.service.KeyspaceServiceImpl$12.execute(KeyspaceServiceImpl.java:495)
>> ~[hector-core-0.7.0-26.jar:na]
>>
>> at
>> me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:101)
>> ~[hector-core-0.7.0-26.jar:na]
>>
>> at
>> me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:161)
>> ~[hector-core-0.7.0-26.jar:na]
>>
>> at
>> me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:129)
>> ~[hector-core-0.7.0-26.jar:na]
>>
>> at
>> me.prettyprint.cassandra.service.KeyspaceServiceImpl.getIndexedSlices(KeyspaceServiceImpl.java:517)
>> ~[hector-core-0.7.0-26.jar:na]
>>
>> at
>> me.prettyprint.cassandra.model.IndexedSlicesQuery$1.doInKeyspace(IndexedSlicesQuery.java:140)
>> ~[hector-core-0.7.0-26.jar:na]
>>
>> at
>> me.prettyprint.cassandra.model.IndexedSlicesQuery$1.doInKeyspace(IndexedSlicesQuery.java:131)
>> ~[hector-core-0.7.0-26.jar:na]
>>
>> at
>> me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20)
>> ~[hector-core-0.7.0-26.jar:na]
>>
>> at
>> me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85)
>> ~[hector-core-0.7.0-26.jar:na]
>>
>> at
>> me.prettyprint.cassandra.model.IndexedSlicesQuery.execute(IndexedSlicesQuery.java:130)
>> ~[hector-core-0.7.0-26.jar:na]
>>
>> at
>> com.yoochoose.services.cassandra.internal.event.CassandraDataFetcher$1.onMessage(CassandraDataFetcher.java:60)
>> [classes/:na]
>>
>> at
>> com.yoochoose.services.cassandra.internal.event.CassandraDataFetcher$1.onMessage(CassandraDataFetcher.java:47)
>> [classes/:na]
>>
>> at
>> org.jetlang.channels.ChannelSubscription$1.run(ChannelSubscription.java:31)
>> [jetlang-0.2.1.jar:na]
>>
>> at
>> org.jetlang.core.BatchExecutorImpl.execute(BatchExecutorImpl.java:11)
>> [jetlang-0.2.1.jar:na]
>>
>> at
>> org.jetlang.core.RunnableExecutorImpl.run(RunnableExecutorImpl.java:34)
>> [jetlang-0.2.1.jar:na]
>>
>> at org.jetlang.fibers.ThreadFiber.runThread(ThreadFiber.java:51)
>> [jetlang-0.2.1.jar:na]
>>
>> at org.jetlang.fibers.ThreadFiber.access$000(ThreadFiber.java:10)
>> [jetlang-0.2.1.jar:na]
>>
>> at org.jetlang.fibers.ThreadFiber$1.run(ThreadFiber.java:27)
>> [jetlang-0.2.1.jar:na]
>>
>> at java.lang.Thread.run(Thread.java:619) [na:1.6.0_17]
>>
>> Caused by: org.apache.cassandra.thrift.InvalidRequestException: null
>>
>> at
>> org.apache.cassandra.thrift.Cassandra$get_indexed_slices_result.read(Cassandra.java:13299)
>> ~[cassandra-all-0.7.0.jar:0.7.0]
>>
>> at
>> org.apache.cassandra.thrift.Cassandra$Client.recv_get_indexed_slices(Cassandra.java:780)
>> ~[cassandra-all-0.7.0.jar:0.7.0]
>>
>> at
>> org.apache.cassandra.thrift.Cassandra$Client.get_indexed_slices(Cassandra.java:752)
>> ~[cassandra-all-0.7.0.jar:0.7.0]
>>
>> at
>> me.prettyprint.cassandra.service.KeyspaceServiceImpl$12.execute(KeyspaceServiceImpl.java:501)
>> ~[hector-core-0.7.0-26.jar:na]
>>
>> ... 19 common frames omitted
>>
>>
>>
>>
>>
>>
>>
>> With the very same code and data.
>>
>> I assume that the column name I give in Cassandra.yaml is somehow not
>> inmterpreted as a TimedUUID or something.
>>
>>
>>
>> Any help would be greatly appreciated
>>
>>
>>
>> Greetings,
>>
>> roland
>>
>>
>>
>>
>>
>> Von: Michal Augustýn [mailto:augustyn.michal@gmail.com]
>> Gesendet: Dienstag, 15. Februar 2011 16:22
>>
>> An: user@cassandra.apache.org
>> Betreff: Re: cant seem to figure out secondary index definition
>>
>>
>>
>> Ah, ok. I checked that in source and the problem is that you wrote
>> "validation_class" but you should "validator_class".
>>
>>
>>
>> Augi
>>
>> 2011/2/15 Roland Gude <ro...@yoochoose.com>
>>
>> Yeah i know about that, but the definition i have is for a cluster that is
>> started/stopped from a unit test with hector embeddedServerHelper, which
>> takes definitions from the yaml.
>>
>> So i’d still like to define the index in the yaml file (it should very well
>> be possible I guess)
>>
>>
>>
>>
>>
>> Von: Michal Augustýn [mailto:augustyn.michal@gmail.com]
>> Gesendet: Dienstag, 15. Februar 2011 15:53
>> An: user@cassandra.apache.org
>> Betreff: Re: cant seem to figure out secondary index definition
>>
>>
>>
>> Hi,
>>
>>
>>
>> if you download Cassandra and look into "conf/cassandra.yaml" then you can
>> see this:
>>
>>
>>
>> "this keyspace definition is for demonstration purposes only. Cassandra will
>> not load these definitions during startup.
>> See http://wiki.apache.org/cassandra/FAQ#no_keyspaces for an explanation."
>>
>>
>>
>> So you should make all schema-related operation via Thrift/AVRO API, or you
>> can use Cassandra CLI.
>>
>>
>>
>> Augi
>>
>>
>>
>> 2011/2/15 Roland Gude <ro...@yoochoose.com>
>>
>> Hi,
>>
>>
>>
>> i am a little puzzled on creation of secondary indexes and the docs in that
>> area are still very sparse.
>>
>> What I am trying to do is – in a columnfamily with TimeUUID comparator, I
>> want the “special” timeuuid 00000000-0000-1000-0000-000000000000 to be
>> indexed. The value being some UTF8 string on which I want to perform
>> equality checks.
>>
>>
>>
>> What do I need to put in my cassandra.yaml file?
>>
>> Something like this?
>>
>>
>>
>> - column_metadata: [{name: 00000000-0000-1000-0000-000000000000,
>> validation_class: UTF8Type, index_name: MyIndex, index_type: KEYS}]
>>
>>
>>
>> This gives me that error:
>>
>>
>>
>> 15:05:12.492 [pool-1-thread-1] ERROR o.a.c.config.DatabaseDescriptor - Fatal
>> error: null; Can't construct a java object for
>> tag:yaml.org,2002:org.apache.cassandra.config.Config; exception=Cannot
>> create property=keyspaces for
>> JavaBean=org.apache.cassandra.config.Config@7eb6e2; Cannot create
>> property=column_families for
>> JavaBean=org.apache.cassandra.config.RawKeyspace@987a33; Cannot create
>> property=column_metadata for
>> JavaBean=org.apache.cassandra.config.RawColumnFamily@716cb7; Cannot create
>> property=validation_class for
>> JavaBean=org.apache.cassandra.config.RawColumnDefinition@e29820; Unable to
>> find property 'validation_class' on class:
>> org.apache.cassandra.config.RawColumnDefinition
>>
>> Bad configuration; unable to start server
>>
>>
>>
>>
>>
>> I am furthermor uncertain if the column name will be correctly used if given
>> like this. Should I put the byte representation of the uuid there?
>>
>>
>>
>> Greetings,
>>
>> roland
>>
>> --
>>
>> YOOCHOOSE GmbH
>>
>>
>>
>> Roland Gude
>>
>> Software Engineer
>>
>>
>>
>> Im Mediapark 8, 50670 Köln
>>
>>
>>
>> +49 221 4544151 (Tel)
>>
>> +49 221 4544159 (Fax)
>>
>> +49 171 7894057 (Mobil)
>>
>>
>>
>>
>>
>> Email: roland.gude@yoochoose.com
>>
>> WWW: www.yoochoose.com
>>
>>
>>
>> YOOCHOOSE GmbH
>>
>> Geschäftsführer: Dr. Uwe Alkemper, Michael Friedmann
>>
>> Handelsregister: Amtsgericht Köln HRB 65275
>>
>> Ust-Ident-Nr: DE 264 773 520
>>
>> Sitz der Gesellschaft: Köln
>>
>>
>>
>>
>>
>>
>
Re: cant seem to figure out secondary index definition
Posted by Roshan Dawrani <ro...@gmail.com>.
On Sun, Mar 6, 2011 at 4:54 PM, Roshan Dawrani <ro...@gmail.com>wrote:
>
>
> ----------------------------------------------------------------------------------------------------------------------
> keyspaces:
> - name: firstname
> validator_class: UTF8Type
> index_type: KEYS
> index_name: LastNameIdx
>
Please correct a typo and change the name for the 2nd column to lastname in
the sample. :-)
Re: cant seem to figure out secondary index definition
Posted by Jürgen Link <ju...@googlemail.com>.
Am 07.03.2011 02:10, schrieb Tyler Hobbs:
> You want to add an index on a CF with TimeUUIDType column names? I
> think you've probably mistaken the purpose of compare_with.
> If you haven't, I think you'll need to add the index programmatically
> in order to specify a non ascii/unicode column name for the index.
I'd like to pull up Roland's original post here. We have the following
situation:
our IndexedSliceQueries work perfectly against our cassandra cluster,
defining indexes via CLI:
create column family A with column_type='Standard' and
>> comparator='TimeUUIDType' and keys_cached=200000 and read_repair_chance=1.0
>> and rows_cached=0.0 and column_metadata=[{column_name:
>> 00000000-0000-1000-0000-000000000000, validation_class: UTF8Type,
>> index_name: MyIndex, index_type: KEYS}];
The only stuff we can't get to work is with embedded cassandra instance
for our unit tests.
We currently define our indexes _for the embedded instance_ with the
following yaml directives:
- column_metadata: [{name: 00000000-0000-1000-0000-000000000000,
validator_class: UTF8Type, index_name: MyIndex, index_type: KEYS}]
compare_with: TimeUUIDType
gc_grace_seconds: 864000
keys_cached: 0.0
max_compaction_threshold: 32
min_compaction_threshold: 4
name: A
read_repair_chance: 1.0
rows_cached: 0.0
So, the question remains: how can we transform our perfectly-working
index definition from
above to a reasonable yaml-definition for an embedded instance?
If yaml isn't suitable for this task, what would be the preferred way to
do it programmatically?
Jürgen
Re: cant seem to figure out secondary index definition
Posted by Tyler Hobbs <ty...@datastax.com>.
On Sun, Mar 6, 2011 at 4:49 PM, Jürgen Link <ju...@googlemail.com>wrote:
> Hi Roshan,
> thanks for your post. I quickly ran over it, and the only difference I can
> actually see is the compare_with type (we use TimeUUIDType).
>
> Any other suggestions, anyone?
>
You want to add an index on a CF with TimeUUIDType column names? I think
you've probably mistaken the purpose of compare_with.
If you haven't, I think you'll need to add the index programmatically in
order to specify a non ascii/unicode column name for the index.
--
Tyler Hobbs
Software Engineer, DataStax <http://datastax.com/>
Maintainer of the pycassa <http://github.com/pycassa/pycassa> Cassandra
Python client library
Re: cant seem to figure out secondary index definition
Posted by Jürgen Link <ju...@googlemail.com>.
Hi Roshan,
thanks for your post. I quickly ran over it, and the only difference I
can actually see is the compare_with type (we use TimeUUIDType).
Any other suggestions, anyone?
Am 06.03.2011 12:24, schrieb Roshan Dawrani:
> Hi,
>
> Sure. Here is a sample of how we define it at the end of cassandra.yaml.
>
> In the keyspace MyApp, it defines a column family MyUser, that has
> secondary indexes on 2 String columns - firstname, and lastname.
>
> Does it help?
>
> ----------------------------------------------------------------------------------------------------------------------
> keyspaces:
> - name: MyApp
> replica_placement_strategy:
> org.apache.cassandra.locator.SimpleStrategy
> replication_factor: 1
> column_families:
> - name: MyUser
> compare_with: org.apache.cassandra.db.marshal.BytesType
> column_metadata:
> - name: firstname
> validator_class: UTF8Type
> index_type: KEYS
> index_name: FirstNameIdx
> - name: firstname
> validator_class: UTF8Type
> index_type: KEYS
> index_name: LastNameIdx
> ----------------------------------------------------------------------------------------------------------------------
>
> Regards,
> Roshan
>
> On Sun, Mar 6, 2011 at 4:34 PM, Jürgen Link
> <juergen.link@googlemail.com <ma...@googlemail.com>> wrote:
>
> Hi Roshan,
> could you please post a small sample from your yaml file?
> As documentation of indexes is quite sparse, we're grateful for
> any working example.
>
> Cheers
> Jürgen
>
> Am 04.03.2011 19:27, schrieb Roshan Dawrani:
>> On Fri, Mar 4, 2011 at 11:52 PM, Jürgen Link
>> <juergen.link@googlemail.com
>> <ma...@googlemail.com>> wrote:
>>
>> Hi Jonathan,
>> as Roland is already out of office, I'd like to jump in.
>> Maybe this somehow got lost in the moddle of this thread,
>> indexing works fine in our "real" cassandra cluster.
>> For our test cases, we use an embedded cassandra instance,
>> which is configured via yaml.
>> In case indexes cannot be defined via yaml (for embedded
>> instances), is there a more preferred way to do so?
>>
>>
>> Sorry, I haven't followed the whole of this thread yet, but I
>> just noticed this mail and would like to add that we also use
>> embedded Cassandra instance for our dev / tests and out yaml has
>> a number of indexes and they work just fine. We are on Cassandar
>> 0.7.0.
>
>
>
Re: cant seem to figure out secondary index definition
Posted by Roshan Dawrani <ro...@gmail.com>.
Hi,
Sure. Here is a sample of how we define it at the end of cassandra.yaml.
In the keyspace MyApp, it defines a column family MyUser, that has secondary
indexes on 2 String columns - firstname, and lastname.
Does it help?
----------------------------------------------------------------------------------------------------------------------
keyspaces:
- name: MyApp
replica_placement_strategy:
org.apache.cassandra.locator.SimpleStrategy
replication_factor: 1
column_families:
- name: MyUser
compare_with: org.apache.cassandra.db.marshal.BytesType
column_metadata:
- name: firstname
validator_class: UTF8Type
index_type: KEYS
index_name: FirstNameIdx
- name: firstname
validator_class: UTF8Type
index_type: KEYS
index_name: LastNameIdx
----------------------------------------------------------------------------------------------------------------------
Regards,
Roshan
On Sun, Mar 6, 2011 at 4:34 PM, Jürgen Link <ju...@googlemail.com>wrote:
> Hi Roshan,
> could you please post a small sample from your yaml file?
> As documentation of indexes is quite sparse, we're grateful for any working
> example.
>
> Cheers
> Jürgen
>
> Am 04.03.2011 19:27, schrieb Roshan Dawrani:
>
> On Fri, Mar 4, 2011 at 11:52 PM, Jürgen Link <ju...@googlemail.com>wrote:
>
>> Hi Jonathan,
>> as Roland is already out of office, I'd like to jump in.
>> Maybe this somehow got lost in the moddle of this thread, indexing works
>> fine in our "real" cassandra cluster.
>> For our test cases, we use an embedded cassandra instance, which is
>> configured via yaml.
>> In case indexes cannot be defined via yaml (for embedded instances), is
>> there a more preferred way to do so?
>>
>
> Sorry, I haven't followed the whole of this thread yet, but I just
> noticed this mail and would like to add that we also use embedded Cassandra
> instance for our dev / tests and out yaml has a number of indexes and they
> work just fine. We are on Cassandar 0.7.0.
>
>
>
>
Re: cant seem to figure out secondary index definition
Posted by Jürgen Link <ju...@googlemail.com>.
Hi Roshan,
could you please post a small sample from your yaml file?
As documentation of indexes is quite sparse, we're grateful for any
working example.
Cheers
Jürgen
Am 04.03.2011 19:27, schrieb Roshan Dawrani:
> On Fri, Mar 4, 2011 at 11:52 PM, Jürgen Link
> <juergen.link@googlemail.com <ma...@googlemail.com>> wrote:
>
> Hi Jonathan,
> as Roland is already out of office, I'd like to jump in.
> Maybe this somehow got lost in the moddle of this thread, indexing
> works fine in our "real" cassandra cluster.
> For our test cases, we use an embedded cassandra instance, which
> is configured via yaml.
> In case indexes cannot be defined via yaml (for embedded
> instances), is there a more preferred way to do so?
>
>
> Sorry, I haven't followed the whole of this thread yet, but I just
> noticed this mail and would like to add that we also use embedded
> Cassandra instance for our dev / tests and out yaml has a number of
> indexes and they work just fine. We are on Cassandar 0.7.0.
Re: cant seem to figure out secondary index definition
Posted by Roshan Dawrani <ro...@gmail.com>.
On Fri, Mar 4, 2011 at 11:52 PM, Jürgen Link <ju...@googlemail.com>wrote:
> Hi Jonathan,
> as Roland is already out of office, I'd like to jump in.
> Maybe this somehow got lost in the moddle of this thread, indexing works
> fine in our "real" cassandra cluster.
> For our test cases, we use an embedded cassandra instance, which is
> configured via yaml.
> In case indexes cannot be defined via yaml (for embedded instances), is
> there a more preferred way to do so?
>
Sorry, I haven't followed the whole of this thread yet, but I just noticed
this mail and would like to add that we also use embedded Cassandra instance
for our dev / tests and out yaml has a number of indexes and they work just
fine. We are on Cassandar 0.7.0.
Re: cant seem to figure out secondary index definition
Posted by Jürgen Link <ju...@googlemail.com>.
Hi Jonathan,
as Roland is already out of office, I'd like to jump in.
Maybe this somehow got lost in the moddle of this thread, indexing works
fine in our "real" cassandra cluster.
For our test cases, we use an embedded cassandra instance, which is
configured via yaml.
In case indexes cannot be defined via yaml (for embedded instances), is
there a more preferred way to do so?
Cheers
Jürgen
Am 04.03.2011 19:15, schrieb Jonathan Ellis:
> On Fri, Mar 4, 2011 at 10:09 AM, Roland Gude<ro...@yoochoose.com> wrote:
>> Hi again,
>>
>> I am still suffering from this error (which severely limits testability for me right now). Doesn't anybody have an idea, why IndexSliceQueries work if the index is created with cassandra-cli and why it does not if the index is created with cassandra.yaml?
> the yaml import is only intended for upgrading from 0.6, which didn't
> have indexes. i wouldn't be surprised if indexes doesn't work there.
>
>> Is there some way to find out, which indexes exist for which columns in a given cluster?
> describe keyspace in the cli
>
Re: cant seem to figure out secondary index definition
Posted by Jonathan Ellis <jb...@gmail.com>.
On Fri, Mar 4, 2011 at 10:09 AM, Roland Gude <ro...@yoochoose.com> wrote:
> Hi again,
>
> I am still suffering from this error (which severely limits testability for me right now). Doesn't anybody have an idea, why IndexSliceQueries work if the index is created with cassandra-cli and why it does not if the index is created with cassandra.yaml?
the yaml import is only intended for upgrading from 0.6, which didn't
have indexes. i wouldn't be surprised if indexes doesn't work there.
> Is there some way to find out, which indexes exist for which columns in a given cluster?
describe keyspace in the cli
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com