[jira] Issue Comment Edited: (CASSANDRA-1472) Add bitmap secondary indexes

["Gary Dusbabek (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-1472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983248#comment-12983248 ] 

Gary Dusbabek edited comment on CASSANDRA-1472 at 2/12/11 1:50 PM:
-------------------------------------------------------------------

bq. would it be reasonable to commit this patch with Avro's file format, and then to discuss replacing the file format in a separate issue
Not a good idea, imo. 

bq. writing our own file format is an optimization
I don't think it is an optimization at all.  It's a way of protecting us from ourselves.  The only way to do that and still use avro is to create our own readers according to the spec, no?

bq.  This decision needs to be made for technical reasons and not grounded in NIH.
The reasons are technical.  The fact is that the way we use avro is constantly breaking on us (I'm referring to migrations).  Some of this is probably self-inflicted.  We don't have much experience yet with what happens when records change slightly or radically over time and how avro copes with that.  The experience we do have leads me to believe that unless we are extremely cautious, we'll end up in a hole.  See CASSANDRA-2001 for an example.  Two seemly innocuous changes have both managed to break migration deserialization.

      was (Author: gdusbabek):
    bq. would it be reasonable to commit this patch with Avro's file format, and then to discuss replacing the file format in a separate issue
Not a good idea, imo. 

bq. writing our own file format is an optimization
I don't think it is an optimization at all.  It's a way of protecting us from ourselves.  The only way to do that and still use avro is to create our own readers according to the spec, no?

bq.  This decision needs to be made for technical reasons and not grounded in NIH.
Poppycock.  The reasons are technical.  The fact is that the way we use avro is constantly breaking on us (I'm referring to migrations).  Some of this is probably self-inflicted, but either way--we should avoid chopping down the forest until we can operate the chainsaw without cutting off our fingers.  We don't have much experience yet with what happens when records change slightly or radically over time and how avro copes with that.  The experience we do have leads me to believe that unless we are extremely cautious, we'll end up in a hole.  See CASSANDRA-2001 for an example.  Two seemly innocuous changes have both managed to break migration deserialization.
  
> Add bitmap secondary indexes
> ----------------------------
>
>                 Key: CASSANDRA-1472
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1472
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Stu Hood
>            Assignee: Stu Hood
>             Fix For: 0.7.2
>
>         Attachments: 0.7-1472-v5.tgz, 0.7-1472-v6.tgz, 0001-CASSANDRA-1472-rebased-to-0.7-branch.txt, 0019-Rename-bugfixes-and-fileclose.txt, 1472-v3.tgz, 1472-v4.tgz, 1472-v5.tgz, anatomy.png, v4-bench-c32.txt
>
>
> Bitmap indexes are a very efficient structure for dealing with immutable data. We can take advantage of the fact that SSTables are immutable by attaching them directly to SSTables as a new component (supported by CASSANDRA-1471).

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira