You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Yang <te...@gmail.com> on 2011/07/01 00:44:15 UTC
Re: custom reconciling columns?
ok, I kind of found the magic bullet , but you can only use it to shoot your
enemy close really close range :)
for read path, the thrift API already limits the output to a list of
columns, so it does not make sense to use maps in the internal operations.
plus the return CF on the read path is not going to be modified/shared by
any other threads, so synchronization is not necessary. so
the solution is to modify ColumnFamilyStore so that getTopLevelColumns takes
a returnCF param, instead of always constructing it inside with
ColumnFamily.create().
so only read path behavior is changed.
in read path, we pass in a FastColumnFamily implementation, which uses an
ArrayList internally to store sorted columns, and do binary search to insert
, and merge to addAll(column).
I tried out this, it's about 50% faster on rows with 3000 cols.
Jonathan: do you think this is a viable approach? the only disadvantage is a
slight change to getTopLevelColumns so we have 2 flavors of this method
Thanks
Yang
On Wed, Jun 29, 2011 at 5:51 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> On Tue, Jun 28, 2011 at 10:06 PM, Yang <te...@gmail.com> wrote:
> > I'm trying to see whether there are some easy magic bullets for a drop-in
> > replacement for concurrentSkipListMap...
>
> I'm highly interested if you find one. :)
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com
>
Re: custom reconciling columns?
Posted by Yang <te...@gmail.com>.
https://issues.apache.org/jira/browse/CASSANDRA-2843
thanks
Yang
On Fri, Jul 1, 2011 at 12:09 AM, Sylvain Lebresne <sy...@datastax.com>wrote:
> I think it's an interesting solution. And we can probably avoid the two
> getTopLevelColumns flavors with at bit a refactor. Let's open a ticket
> however,
> because this is starting to be off-topic for the user mailing list.
>
> --
> Sylvain
>
> On Fri, Jul 1, 2011 at 12:44 AM, Yang <te...@gmail.com> wrote:
> > ok, I kind of found the magic bullet , but you can only use it to shoot
> your
> > enemy close really close range :)
> >
> > for read path, the thrift API already limits the output to a list of
> > columns, so it does not make sense to use maps in the internal
> operations.
> > plus the return CF on the read path is not going to be modified/shared by
> > any other threads, so synchronization is not necessary. so
> > the solution is to modify ColumnFamilyStore so that getTopLevelColumns
> takes
> > a returnCF param, instead of always constructing it inside with
> > ColumnFamily.create().
> > so only read path behavior is changed.
> > in read path, we pass in a FastColumnFamily implementation, which uses an
> > ArrayList internally to store sorted columns, and do binary search to
> insert
> > , and merge to addAll(column).
> > I tried out this, it's about 50% faster on rows with 3000 cols.
> >
> > Jonathan: do you think this is a viable approach? the only disadvantage
> is a
> > slight change to getTopLevelColumns so we have 2 flavors of this method
> > Thanks
> > Yang
> >
> > On Wed, Jun 29, 2011 at 5:51 PM, Jonathan Ellis <jb...@gmail.com>
> wrote:
> >>
> >> On Tue, Jun 28, 2011 at 10:06 PM, Yang <te...@gmail.com> wrote:
> >> > I'm trying to see whether there are some easy magic bullets for a
> >> > drop-in
> >> > replacement for concurrentSkipListMap...
> >>
> >> I'm highly interested if you find one. :)
> >>
> >> --
> >> Jonathan Ellis
> >> Project Chair, Apache Cassandra
> >> co-founder of DataStax, the source for professional Cassandra support
> >> http://www.datastax.com
> >
> >
>
Re: custom reconciling columns?
Posted by Sylvain Lebresne <sy...@datastax.com>.
I think it's an interesting solution. And we can probably avoid the two
getTopLevelColumns flavors with at bit a refactor. Let's open a ticket however,
because this is starting to be off-topic for the user mailing list.
--
Sylvain
On Fri, Jul 1, 2011 at 12:44 AM, Yang <te...@gmail.com> wrote:
> ok, I kind of found the magic bullet , but you can only use it to shoot your
> enemy close really close range :)
>
> for read path, the thrift API already limits the output to a list of
> columns, so it does not make sense to use maps in the internal operations.
> plus the return CF on the read path is not going to be modified/shared by
> any other threads, so synchronization is not necessary. so
> the solution is to modify ColumnFamilyStore so that getTopLevelColumns takes
> a returnCF param, instead of always constructing it inside with
> ColumnFamily.create().
> so only read path behavior is changed.
> in read path, we pass in a FastColumnFamily implementation, which uses an
> ArrayList internally to store sorted columns, and do binary search to insert
> , and merge to addAll(column).
> I tried out this, it's about 50% faster on rows with 3000 cols.
>
> Jonathan: do you think this is a viable approach? the only disadvantage is a
> slight change to getTopLevelColumns so we have 2 flavors of this method
> Thanks
> Yang
>
> On Wed, Jun 29, 2011 at 5:51 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>>
>> On Tue, Jun 28, 2011 at 10:06 PM, Yang <te...@gmail.com> wrote:
>> > I'm trying to see whether there are some easy magic bullets for a
>> > drop-in
>> > replacement for concurrentSkipListMap...
>>
>> I'm highly interested if you find one. :)
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com
>
>