You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Timmy Turner <ti...@gmail.com> on 2012/10/29 12:34:50 UTC
CQL3: Unknown property 'comparator'?
Does CQL3 not allow dynamic columns (column names) any more?
Re: CQL3: Unknown property 'comparator'?
Posted by Timmy Turner <ti...@gmail.com>.
> CQL 3 is not locked, the more feedback the better.
I'm pretty sure that my use case is not relevant enough to consider
special cases for it in Cassandra/CQL. In fact it's quite the other
way around - it's my job to determine how well my tool fits Cassandra
an other databases.
> Can you provide details on the use case?
Basically what I tried to do was some sort of transparent client-side
encryption, for those few that want to run their database in the
cloud, but are too paranoid to do so - i.e. hook the Cassandra (CQL)
library, parse the query myself (using a parser generated by
Cassandra's own grammar definition), extract the values the user said
he wanted encrypted (he did so in a configuration file), encrypt them,
replace the original values with the encrypted ones in the query
string, and hand it over to the actual CQL client library. If a result
set came back, then I decrypted it before ultimately handing it over
to the user.
I also had the option to encrypt column names with individual keys per
tuple, so they wouldn't result in the same ciphertext across tuples
(for those ultra-paranoid users). Other than severely limiting the
functionality of Cassandra (i.e. a user couldn't select specific
columns any more, if more than one tuple was to be queried from the
database; but queries for one specific tuple would still work without
any kind of limitation), this worked fine in CQL2. In CQL3 this,
obviously, isn't possible any more, if the users won't settle for
using composite keys instead of columns.
> AFAIK to be "schema free" you use the "row-column-value" model. While this
> looks like many rows it is a single store engine row.
I know. CQL3 is still a superset of CQL2's features, so in theory it
can still do everything CQL2 could - but the real question is how
Cassandra users will end up using it, and what kind of usage patterns
the current data model encourages.
2012/10/30 aaron morton <aa...@thelastpickle.com>:
> My use case relied on "mangling" the column names for various
>
> Can you provide details on the use case?
> CQL 3 is not locked, the more feedback the better.
>
> That's not a problem, but I still needed a definitive statement that
> "schema-free tables" (not column families obviously) are now
> impossible within CQL, that I can document in the project as the
> reason for the changes.
>
> AFAIK to be "schema free" you use the "row-column-value" model. While this
> looks like many rows it is a single store engine row.
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 30/10/2012, at 8:45 PM, Timmy Turner <ti...@gmail.com> wrote:
>
> Is your use case covered in the article above ?
>
>
> My use case relied on "mangling" the column names for various
> purposes, and CQL3 with its transposed columns does of course still
> allow for that, but it means rewriting the part of the application
> that dealt with CQL so it can handle the new syntax/data model.
>
> That's not a problem, but I still needed a definitive statement that
> "schema-free tables" (not column families obviously) are now
> impossible within CQL, that I can document in the project as the
> reason for the changes.
>
>
> Thanks!
>
> 2012/10/30 aaron morton <aa...@thelastpickle.com>:
>
> More background http://www.datastax.com/dev/blog/thrift-to-cql3
>
> So does this mean that in CQL 3 an explicit schema is absolutely
> mandatory?
>
> Not really, it sort of depends on your view.
>
> Lets say this is a "schema free" CF definition in CLI
>
> create column family clicks
> with key_validation_class = UTF8Type
> and comparator = DateType
> and default_validation_class = UTF8Type
>
> It could be used for wide rows with lots of columns, where the name is a
> date.
>
> As the article at the top says, this CQL 3 DDL is equivalent:
>
> CREATE TABLE clicks (
> key text,
> column1 timestamp,
> value text,
> PRIMARY KEY (key, column)
> ) WITH COMPACT STORAGE
>
> This creates a single row inside C*, column name is a date. The difference
> is CQL 3 pivots this one "storage engine" row into multiple CQL 3 rows. (See
> article)
>
> So far so good. Let's add some schema:
>
> CREATE TABLE clicks (
> user_id text,
> click_time timestamp,
> click_url text,
> PRIMARY KEY (user_id, click_time)
> ) WITH COMPACT STORAGE
>
> That's functionally the same but has some more schema in it. It tells CQL 3
> that the label to use for the name of a column is "click_time". Previously
> the label was "column1".
>
>
> It's now impossible (within CQL) to add new
> (non-primary-key) columns only for individual rows implicitly with
> DML-queries (insert/update)?.
>
> Is your use case covered in the article above ?
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 30/10/2012, at 2:31 AM, Timmy Turner <ti...@gmail.com> wrote:
>
> Thank you! That article helps clear up a lot of my confusion about the
> changes between CQL 2 and 3, since I was wondering how to
> access/manipulate CompositeType/DynamicCompositeType columns through
> CQL.
>
> So does this mean that in CQL 3 an explicit schema is absolutely
> mandatory? It's now impossible (within CQL) to add new
> (non-primary-key) columns only for individual rows implicitly with
> DML-queries (insert/update)?.
>
>
>
>
> 2012/10/29 Sylvain Lebresne <sy...@datastax.com>:
>
> CQL3 does absolutely allow dynamic column families, but does it
> differently from CQL2. See
> http://www.datastax.com/dev/blog/cql3-for-cassandra-experts.
>
> --
> Sylvain
>
> On Mon, Oct 29, 2012 at 12:34 PM, Timmy Turner <ti...@gmail.com> wrote:
>
> Does CQL3 not allow dynamic columns (column names) any more?
>
>
>
Re: CQL3: Unknown property 'comparator'?
Posted by aaron morton <aa...@thelastpickle.com>.
> My use case relied on "mangling" the column names for various
Can you provide details on the use case?
CQL 3 is not locked, the more feedback the better.
> That's not a problem, but I still needed a definitive statement that
> "schema-free tables" (not column families obviously) are now
> impossible within CQL, that I can document in the project as the
> reason for the changes.
AFAIK to be "schema free" you use the "row-column-value" model. While this looks like many rows it is a single store engine row.
Cheers
-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com
On 30/10/2012, at 8:45 PM, Timmy Turner <ti...@gmail.com> wrote:
>> Is your use case covered in the article above ?
>
> My use case relied on "mangling" the column names for various
> purposes, and CQL3 with its transposed columns does of course still
> allow for that, but it means rewriting the part of the application
> that dealt with CQL so it can handle the new syntax/data model.
>
> That's not a problem, but I still needed a definitive statement that
> "schema-free tables" (not column families obviously) are now
> impossible within CQL, that I can document in the project as the
> reason for the changes.
>
>
> Thanks!
>
> 2012/10/30 aaron morton <aa...@thelastpickle.com>:
>> More background http://www.datastax.com/dev/blog/thrift-to-cql3
>>
>> So does this mean that in CQL 3 an explicit schema is absolutely
>> mandatory?
>>
>> Not really, it sort of depends on your view.
>>
>> Lets say this is a "schema free" CF definition in CLI
>>
>> create column family clicks
>> with key_validation_class = UTF8Type
>> and comparator = DateType
>> and default_validation_class = UTF8Type
>>
>> It could be used for wide rows with lots of columns, where the name is a
>> date.
>>
>> As the article at the top says, this CQL 3 DDL is equivalent:
>>
>> CREATE TABLE clicks (
>> key text,
>> column1 timestamp,
>> value text,
>> PRIMARY KEY (key, column)
>> ) WITH COMPACT STORAGE
>>
>> This creates a single row inside C*, column name is a date. The difference
>> is CQL 3 pivots this one "storage engine" row into multiple CQL 3 rows. (See
>> article)
>>
>> So far so good. Let's add some schema:
>>
>> CREATE TABLE clicks (
>> user_id text,
>> click_time timestamp,
>> click_url text,
>> PRIMARY KEY (user_id, click_time)
>> ) WITH COMPACT STORAGE
>>
>> That's functionally the same but has some more schema in it. It tells CQL 3
>> that the label to use for the name of a column is "click_time". Previously
>> the label was "column1".
>>
>>
>> It's now impossible (within CQL) to add new
>> (non-primary-key) columns only for individual rows implicitly with
>> DML-queries (insert/update)?.
>>
>> Is your use case covered in the article above ?
>>
>> Cheers
>>
>> -----------------
>> Aaron Morton
>> Freelance Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>> On 30/10/2012, at 2:31 AM, Timmy Turner <ti...@gmail.com> wrote:
>>
>> Thank you! That article helps clear up a lot of my confusion about the
>> changes between CQL 2 and 3, since I was wondering how to
>> access/manipulate CompositeType/DynamicCompositeType columns through
>> CQL.
>>
>> So does this mean that in CQL 3 an explicit schema is absolutely
>> mandatory? It's now impossible (within CQL) to add new
>> (non-primary-key) columns only for individual rows implicitly with
>> DML-queries (insert/update)?.
>>
>>
>>
>>
>> 2012/10/29 Sylvain Lebresne <sy...@datastax.com>:
>>
>> CQL3 does absolutely allow dynamic column families, but does it
>> differently from CQL2. See
>> http://www.datastax.com/dev/blog/cql3-for-cassandra-experts.
>>
>> --
>> Sylvain
>>
>> On Mon, Oct 29, 2012 at 12:34 PM, Timmy Turner <ti...@gmail.com> wrote:
>>
>> Does CQL3 not allow dynamic columns (column names) any more?
>>
>>
Re: CQL3: Unknown property 'comparator'?
Posted by Timmy Turner <ti...@gmail.com>.
> Is your use case covered in the article above ?
My use case relied on "mangling" the column names for various
purposes, and CQL3 with its transposed columns does of course still
allow for that, but it means rewriting the part of the application
that dealt with CQL so it can handle the new syntax/data model.
That's not a problem, but I still needed a definitive statement that
"schema-free tables" (not column families obviously) are now
impossible within CQL, that I can document in the project as the
reason for the changes.
Thanks!
2012/10/30 aaron morton <aa...@thelastpickle.com>:
> More background http://www.datastax.com/dev/blog/thrift-to-cql3
>
> So does this mean that in CQL 3 an explicit schema is absolutely
> mandatory?
>
> Not really, it sort of depends on your view.
>
> Lets say this is a "schema free" CF definition in CLI
>
> create column family clicks
> with key_validation_class = UTF8Type
> and comparator = DateType
> and default_validation_class = UTF8Type
>
> It could be used for wide rows with lots of columns, where the name is a
> date.
>
> As the article at the top says, this CQL 3 DDL is equivalent:
>
> CREATE TABLE clicks (
> key text,
> column1 timestamp,
> value text,
> PRIMARY KEY (key, column)
> ) WITH COMPACT STORAGE
>
> This creates a single row inside C*, column name is a date. The difference
> is CQL 3 pivots this one "storage engine" row into multiple CQL 3 rows. (See
> article)
>
> So far so good. Let's add some schema:
>
> CREATE TABLE clicks (
> user_id text,
> click_time timestamp,
> click_url text,
> PRIMARY KEY (user_id, click_time)
> ) WITH COMPACT STORAGE
>
> That's functionally the same but has some more schema in it. It tells CQL 3
> that the label to use for the name of a column is "click_time". Previously
> the label was "column1".
>
>
> It's now impossible (within CQL) to add new
> (non-primary-key) columns only for individual rows implicitly with
> DML-queries (insert/update)?.
>
> Is your use case covered in the article above ?
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 30/10/2012, at 2:31 AM, Timmy Turner <ti...@gmail.com> wrote:
>
> Thank you! That article helps clear up a lot of my confusion about the
> changes between CQL 2 and 3, since I was wondering how to
> access/manipulate CompositeType/DynamicCompositeType columns through
> CQL.
>
> So does this mean that in CQL 3 an explicit schema is absolutely
> mandatory? It's now impossible (within CQL) to add new
> (non-primary-key) columns only for individual rows implicitly with
> DML-queries (insert/update)?.
>
>
>
>
> 2012/10/29 Sylvain Lebresne <sy...@datastax.com>:
>
> CQL3 does absolutely allow dynamic column families, but does it
> differently from CQL2. See
> http://www.datastax.com/dev/blog/cql3-for-cassandra-experts.
>
> --
> Sylvain
>
> On Mon, Oct 29, 2012 at 12:34 PM, Timmy Turner <ti...@gmail.com> wrote:
>
> Does CQL3 not allow dynamic columns (column names) any more?
>
>
Re: CQL3: Unknown property 'comparator'?
Posted by aaron morton <aa...@thelastpickle.com>.
More background http://www.datastax.com/dev/blog/thrift-to-cql3
> So does this mean that in CQL 3 an explicit schema is absolutely
> mandatory?
Not really, it sort of depends on your view.
Lets say this is a "schema free" CF definition in CLI
create column family clicks
with key_validation_class = UTF8Type
and comparator = DateType
and default_validation_class = UTF8Type
It could be used for wide rows with lots of columns, where the name is a date.
As the article at the top says, this CQL 3 DDL is equivalent:
CREATE TABLE clicks (
key text,
column1 timestamp,
value text,
PRIMARY KEY (key, column)
) WITH COMPACT STORAGE
This creates a single row inside C*, column name is a date. The difference is CQL 3 pivots this one "storage engine" row into multiple CQL 3 rows. (See article)
So far so good. Let's add some schema:
CREATE TABLE clicks (
user_id text,
click_time timestamp,
click_url text,
PRIMARY KEY (user_id, click_time)
) WITH COMPACT STORAGE
That's functionally the same but has some more schema in it. It tells CQL 3 that the label to use for the name of a column is "click_time". Previously the label was "column1".
> It's now impossible (within CQL) to add new
> (non-primary-key) columns only for individual rows implicitly with
> DML-queries (insert/update)?.
Is your use case covered in the article above ?
Cheers
-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com
On 30/10/2012, at 2:31 AM, Timmy Turner <ti...@gmail.com> wrote:
> Thank you! That article helps clear up a lot of my confusion about the
> changes between CQL 2 and 3, since I was wondering how to
> access/manipulate CompositeType/DynamicCompositeType columns through
> CQL.
>
> So does this mean that in CQL 3 an explicit schema is absolutely
> mandatory? It's now impossible (within CQL) to add new
> (non-primary-key) columns only for individual rows implicitly with
> DML-queries (insert/update)?.
>
>
>
>
> 2012/10/29 Sylvain Lebresne <sy...@datastax.com>:
>> CQL3 does absolutely allow dynamic column families, but does it
>> differently from CQL2. See
>> http://www.datastax.com/dev/blog/cql3-for-cassandra-experts.
>>
>> --
>> Sylvain
>>
>> On Mon, Oct 29, 2012 at 12:34 PM, Timmy Turner <ti...@gmail.com> wrote:
>>> Does CQL3 not allow dynamic columns (column names) any more?
Re: CQL3: Unknown property 'comparator'?
Posted by Timmy Turner <ti...@gmail.com>.
Thank you! That article helps clear up a lot of my confusion about the
changes between CQL 2 and 3, since I was wondering how to
access/manipulate CompositeType/DynamicCompositeType columns through
CQL.
So does this mean that in CQL 3 an explicit schema is absolutely
mandatory? It's now impossible (within CQL) to add new
(non-primary-key) columns only for individual rows implicitly with
DML-queries (insert/update)?.
2012/10/29 Sylvain Lebresne <sy...@datastax.com>:
> CQL3 does absolutely allow dynamic column families, but does it
> differently from CQL2. See
> http://www.datastax.com/dev/blog/cql3-for-cassandra-experts.
>
> --
> Sylvain
>
> On Mon, Oct 29, 2012 at 12:34 PM, Timmy Turner <ti...@gmail.com> wrote:
>> Does CQL3 not allow dynamic columns (column names) any more?
Re: CQL3: Unknown property 'comparator'?
Posted by Sylvain Lebresne <sy...@datastax.com>.
CQL3 does absolutely allow dynamic column families, but does it
differently from CQL2. See
http://www.datastax.com/dev/blog/cql3-for-cassandra-experts.
--
Sylvain
On Mon, Oct 29, 2012 at 12:34 PM, Timmy Turner <ti...@gmail.com> wrote:
> Does CQL3 not allow dynamic columns (column names) any more?