You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jonathan Ellis (JIRA)" <ji...@apache.org> on 2009/09/15 20:24:57 UTC
[jira] Created: (CASSANDRA-446) Use DecoratedKey objects in
Memtable, SSTableReader/Writer objects
Use DecoratedKey objects in Memtable, SSTableReader/Writer objects
------------------------------------------------------------------
Key: CASSANDRA-446
URL: https://issues.apache.org/jira/browse/CASSANDRA-446
Project: Cassandra
Issue Type: Improvement
Reporter: Jonathan Ellis
See CASSANDRA-420 for introduction of DecoratedKey.
IMO we should move Memtable and SSTR objects to using DK objects, too. This will help Memtable flushes similar to how this helps BMt, and using DK objects in SSTR will avoid a ton of potential confusion about whether a String is a key or a decorated key.
As a side benefit, moving the decoration operation out of memtable flush and into the insert path will help make better use of multicore systems, since we can have multiple write threads but only one core can flush at a time. (See CASSANDRA-445.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-446) Use DecoratedKey objects in
Memtable, SSTableReader/Writer objects
Posted by "Johan Oskarsson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Johan Oskarsson updated CASSANDRA-446:
--------------------------------------
Attachment: CASSANDRA-446.patch
> Use DecoratedKey objects in Memtable, SSTableReader/Writer objects
> ------------------------------------------------------------------
>
> Key: CASSANDRA-446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-446
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: CASSANDRA-446.patch
>
>
> See CASSANDRA-420 for introduction of DecoratedKey.
> IMO we should move Memtable and SSTR objects to using DK objects, too. This will help Memtable flushes similar to how this helps BMt, and using DK objects in SSTR will avoid a ton of potential confusion about whether a String is a key or a decorated key.
> As a side benefit, moving the decoration operation out of memtable flush and into the insert path will help make better use of multicore systems, since we can have multiple write threads but only one core can flush at a time. (See CASSANDRA-445.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-446) Use DecoratedKey objects in
Memtable, SSTableReader/Writer objects
Posted by "Johan Oskarsson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12761555#action_12761555 ]
Johan Oskarsson commented on CASSANDRA-446:
-------------------------------------------
+1. The changes look good to me.
I assume the overhead of decorating the keys in the comparator in the StorageProxy won't be too bad if it's just a few entries being compared.
> Use DecoratedKey objects in Memtable, SSTableReader/Writer objects
> ------------------------------------------------------------------
>
> Key: CASSANDRA-446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-446
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: CASSANDRA-446.patch, CASSANDRA-446.patch, CASSANDRA-446.patch
>
>
> See CASSANDRA-420 for introduction of DecoratedKey.
> IMO we should move Memtable and SSTR objects to using DK objects, too. This will help Memtable flushes similar to how this helps BMt, and using DK objects in SSTR will avoid a ton of potential confusion about whether a String is a key or a decorated key.
> As a side benefit, moving the decoration operation out of memtable flush and into the insert path will help make better use of multicore systems, since we can have multiple write threads but only one core can flush at a time. (See CASSANDRA-445.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-446) Use DecoratedKey objects in
Memtable, SSTableReader/Writer objects
Posted by "Johan Oskarsson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Johan Oskarsson updated CASSANDRA-446:
--------------------------------------
Attachment: CASSANDRA-446.patch
Updated to work with trunk post CASSANDRA-467. That bug is also solved by this patch but in a different way.
> Use DecoratedKey objects in Memtable, SSTableReader/Writer objects
> ------------------------------------------------------------------
>
> Key: CASSANDRA-446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-446
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: CASSANDRA-446.patch, CASSANDRA-446.patch
>
>
> See CASSANDRA-420 for introduction of DecoratedKey.
> IMO we should move Memtable and SSTR objects to using DK objects, too. This will help Memtable flushes similar to how this helps BMt, and using DK objects in SSTR will avoid a ton of potential confusion about whether a String is a key or a decorated key.
> As a side benefit, moving the decoration operation out of memtable flush and into the insert path will help make better use of multicore systems, since we can have multiple write threads but only one core can flush at a time. (See CASSANDRA-445.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-446) Use DecoratedKey objects in
Memtable, SSTableReader/Writer objects
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-446:
-------------------------------------
Attachment: CASSANDRA-446.patch
Some modifications:
- was using a decoration comparator on the strings returned by getKeyRange (which are undecorated). fixed. this was the only place using the old String DK comparators, so removed those too.
- made code use partitioner.convertToDiskFormat explicitly instead of relying on DK.toString; the two did not return the same values but needed to (e.g. bloom filter related code was relying on this). making toString return something human-readable instead removes the temptation of treating them as interchangeable.
- made OPP DK (null, key) again; there's no reason (?) to generate unnecessary garbage
> Use DecoratedKey objects in Memtable, SSTableReader/Writer objects
> ------------------------------------------------------------------
>
> Key: CASSANDRA-446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-446
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: CASSANDRA-446.patch, CASSANDRA-446.patch, CASSANDRA-446.patch
>
>
> See CASSANDRA-420 for introduction of DecoratedKey.
> IMO we should move Memtable and SSTR objects to using DK objects, too. This will help Memtable flushes similar to how this helps BMt, and using DK objects in SSTR will avoid a ton of potential confusion about whether a String is a key or a decorated key.
> As a side benefit, moving the decoration operation out of memtable flush and into the insert path will help make better use of multicore systems, since we can have multiple write threads but only one core can flush at a time. (See CASSANDRA-445.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-446) Use DecoratedKey objects in
Memtable, SSTableReader/Writer objects
Posted by "Chris Goffinet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12759372#action_12759372 ]
Chris Goffinet commented on CASSANDRA-446:
------------------------------------------
Any progress on this? I could take it over but if your almost done no need :)
> Use DecoratedKey objects in Memtable, SSTableReader/Writer objects
> ------------------------------------------------------------------
>
> Key: CASSANDRA-446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-446
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
>
> See CASSANDRA-420 for introduction of DecoratedKey.
> IMO we should move Memtable and SSTR objects to using DK objects, too. This will help Memtable flushes similar to how this helps BMt, and using DK objects in SSTR will avoid a ton of potential confusion about whether a String is a key or a decorated key.
> As a side benefit, moving the decoration operation out of memtable flush and into the insert path will help make better use of multicore systems, since we can have multiple write threads but only one core can flush at a time. (See CASSANDRA-445.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-446) Use DecoratedKey objects in
Memtable, SSTableReader/Writer objects
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-446:
-------------------------------------
Component/s: Core
Fix Version/s: 0.5
> Use DecoratedKey objects in Memtable, SSTableReader/Writer objects
> ------------------------------------------------------------------
>
> Key: CASSANDRA-446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-446
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Fix For: 0.5
>
>
> See CASSANDRA-420 for introduction of DecoratedKey.
> IMO we should move Memtable and SSTR objects to using DK objects, too. This will help Memtable flushes similar to how this helps BMt, and using DK objects in SSTR will avoid a ton of potential confusion about whether a String is a key or a decorated key.
> As a side benefit, moving the decoration operation out of memtable flush and into the insert path will help make better use of multicore systems, since we can have multiple write threads but only one core can flush at a time. (See CASSANDRA-445.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-446) Use DecoratedKey objects in
Memtable, SSTableReader/Writer objects
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755622#action_12755622 ]
Jonathan Ellis commented on CASSANDRA-446:
------------------------------------------
Note that while using DK objects means we could easily serialize the BigIntegers (in particular) more efficiently than as Strings I'd like to avoid messing with the on-disk format for 0.5 if we can at all help it.
> Use DecoratedKey objects in Memtable, SSTableReader/Writer objects
> ------------------------------------------------------------------
>
> Key: CASSANDRA-446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-446
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Jonathan Ellis
>
> See CASSANDRA-420 for introduction of DecoratedKey.
> IMO we should move Memtable and SSTR objects to using DK objects, too. This will help Memtable flushes similar to how this helps BMt, and using DK objects in SSTR will avoid a ton of potential confusion about whether a String is a key or a decorated key.
> As a side benefit, moving the decoration operation out of memtable flush and into the insert path will help make better use of multicore systems, since we can have multiple write threads but only one core can flush at a time. (See CASSANDRA-445.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-446) Use DecoratedKey objects in
Memtable, SSTableReader/Writer objects
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12761867#action_12761867 ]
Hudson commented on CASSANDRA-446:
----------------------------------
Integrated in Cassandra #216 (See [http://hudson.zones.apache.org/hudson/job/Cassandra/216/])
Use DecoratedKey objects in Memtable, SSTableReader/Writer objects.
patch by johano; reviewed by jbellis for
> Use DecoratedKey objects in Memtable, SSTableReader/Writer objects
> ------------------------------------------------------------------
>
> Key: CASSANDRA-446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-446
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Johan Oskarsson
> Fix For: 0.5
>
> Attachments: CASSANDRA-446.patch, CASSANDRA-446.patch, CASSANDRA-446.patch
>
>
> See CASSANDRA-420 for introduction of DecoratedKey.
> IMO we should move Memtable and SSTR objects to using DK objects, too. This will help Memtable flushes similar to how this helps BMt, and using DK objects in SSTR will avoid a ton of potential confusion about whether a String is a key or a decorated key.
> As a side benefit, moving the decoration operation out of memtable flush and into the insert path will help make better use of multicore systems, since we can have multiple write threads but only one core can flush at a time. (See CASSANDRA-445.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.