You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jonathan Ellis (JIRA)" <ji...@apache.org> on 2009/05/22 17:26:45 UTC
[jira] Created: (CASSANDRA-196) Doing a descending range still
returns columns in ascending order
Doing a descending range still returns columns in ascending order
-----------------------------------------------------------------
Key: CASSANDRA-196
URL: https://issues.apache.org/jira/browse/CASSANDRA-196
Project: Cassandra
Issue Type: Bug
Reporter: Jonathan Ellis
If I do
result = table.getSliceFrom(row, "Standard1:col5", false, 2);
cf = result.getColumnFamily("Standard1");
I expect to get back columns in the order 5, 4, 3 (at the thrift level, it's turned into a list) but instead I get 3, 4, 5 because using a CF as the return vehicle re-sorts them by the standard comparator.
The simplest solution is to allow user-defined column ordering as in CASSANDRA-185 and always return columns in that order (i.e., remove the ascending bool). This also allows us to make the columngroup fetching more efficient in the best case (deserializing one column at a time instead of a group at a time).
OTOH using one index to allow fetching items relatively efficiently in either directly is cool. But my gut says it's relatively uncommon to want to access in both directions at once, and even more uncommon to not be able to do a reverse() on the client side (because of data volume, for instance). Forcing a separate CF for this special case of a special case might be worth the tradeoff.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-196) Doing a descending range still
returns columns in ascending order
Posted by "Jun Rao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jun Rao updated CASSANDRA-196:
------------------------------
Attachment: issue196.patchv4
Attache patch v4. Changed to collections.reverse and added a sc test. rebased to trunk.
> Doing a descending range still returns columns in ascending order
> -----------------------------------------------------------------
>
> Key: CASSANDRA-196
> URL: https://issues.apache.org/jira/browse/CASSANDRA-196
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jun Rao
> Fix For: 0.4
>
> Attachments: 196-systest.patch, issue196.patchv3, issue196.patchv4, issue_196.patchv2
>
>
> If I do
> result = table.getSliceFrom(row, "Standard1:col5", false, 2);
> cf = result.getColumnFamily("Standard1");
> I expect to get back columns in the order 5, 4, 3 (at the thrift level, it's turned into a list) but instead I get 3, 4, 5 because using a CF as the return vehicle re-sorts them by the standard comparator.
> The simplest solution is to allow user-defined column ordering as in CASSANDRA-185 and always return columns in that order (i.e., remove the ascending bool). This also allows us to make the columngroup fetching more efficient in the best case (deserializing one column at a time instead of a group at a time).
> OTOH using one index to allow fetching items relatively efficiently in either directly is cool. But my gut says it's relatively uncommon to want to access in both directions at once, and even more uncommon to not be able to do a reverse() on the client side (because of data volume, for instance). Forcing a separate CF for this special case of a special case might be worth the tradeoff.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-196) Doing a descending range still
returns columns in ascending order
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12733616#action_12733616 ]
Jonathan Ellis commented on CASSANDRA-196:
------------------------------------------
Thanks! Can you rebase to trunk? (Yep, I did try. :)
> Doing a descending range still returns columns in ascending order
> -----------------------------------------------------------------
>
> Key: CASSANDRA-196
> URL: https://issues.apache.org/jira/browse/CASSANDRA-196
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jun Rao
> Fix For: 0.4
>
> Attachments: 196-systest.patch, issue_196.patchv2
>
>
> If I do
> result = table.getSliceFrom(row, "Standard1:col5", false, 2);
> cf = result.getColumnFamily("Standard1");
> I expect to get back columns in the order 5, 4, 3 (at the thrift level, it's turned into a list) but instead I get 3, 4, 5 because using a CF as the return vehicle re-sorts them by the standard comparator.
> The simplest solution is to allow user-defined column ordering as in CASSANDRA-185 and always return columns in that order (i.e., remove the ascending bool). This also allows us to make the columngroup fetching more efficient in the best case (deserializing one column at a time instead of a group at a time).
> OTOH using one index to allow fetching items relatively efficiently in either directly is cool. But my gut says it's relatively uncommon to want to access in both directions at once, and even more uncommon to not be able to do a reverse() on the client side (because of data volume, for instance). Forcing a separate CF for this special case of a special case might be worth the tradeoff.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-196) Doing a descending range still
returns columns in ascending order
Posted by "Jun Rao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jun Rao updated CASSANDRA-196:
------------------------------
Attachment: issue196.patchv3
attache patchv3. rebased to trunk.
> Doing a descending range still returns columns in ascending order
> -----------------------------------------------------------------
>
> Key: CASSANDRA-196
> URL: https://issues.apache.org/jira/browse/CASSANDRA-196
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jun Rao
> Fix For: 0.4
>
> Attachments: 196-systest.patch, issue196.patchv3, issue_196.patchv2
>
>
> If I do
> result = table.getSliceFrom(row, "Standard1:col5", false, 2);
> cf = result.getColumnFamily("Standard1");
> I expect to get back columns in the order 5, 4, 3 (at the thrift level, it's turned into a list) but instead I get 3, 4, 5 because using a CF as the return vehicle re-sorts them by the standard comparator.
> The simplest solution is to allow user-defined column ordering as in CASSANDRA-185 and always return columns in that order (i.e., remove the ascending bool). This also allows us to make the columngroup fetching more efficient in the best case (deserializing one column at a time instead of a group at a time).
> OTOH using one index to allow fetching items relatively efficiently in either directly is cool. But my gut says it's relatively uncommon to want to access in both directions at once, and even more uncommon to not be able to do a reverse() on the client side (because of data volume, for instance). Forcing a separate CF for this special case of a special case might be worth the tradeoff.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CASSANDRA-196) Doing a descending range still
returns columns in ascending order
Posted by "Jun Rao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jun Rao reassigned CASSANDRA-196:
---------------------------------
Assignee: Jun Rao
> Doing a descending range still returns columns in ascending order
> -----------------------------------------------------------------
>
> Key: CASSANDRA-196
> URL: https://issues.apache.org/jira/browse/CASSANDRA-196
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jun Rao
> Fix For: 0.4
>
> Attachments: 196-systest.patch
>
>
> If I do
> result = table.getSliceFrom(row, "Standard1:col5", false, 2);
> cf = result.getColumnFamily("Standard1");
> I expect to get back columns in the order 5, 4, 3 (at the thrift level, it's turned into a list) but instead I get 3, 4, 5 because using a CF as the return vehicle re-sorts them by the standard comparator.
> The simplest solution is to allow user-defined column ordering as in CASSANDRA-185 and always return columns in that order (i.e., remove the ascending bool). This also allows us to make the columngroup fetching more efficient in the best case (deserializing one column at a time instead of a group at a time).
> OTOH using one index to allow fetching items relatively efficiently in either directly is cool. But my gut says it's relatively uncommon to want to access in both directions at once, and even more uncommon to not be able to do a reverse() on the client side (because of data volume, for instance). Forcing a separate CF for this special case of a special case might be worth the tradeoff.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-196) Doing a descending range still
returns columns in ascending order
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712160#action_12712160 ]
Jonathan Ellis commented on CASSANDRA-196:
------------------------------------------
Yes, we can add a bandaid there, but "iterate in descending order, to add to a sortedset in ascending order, to reverse before sending back to the client" isn't exactly awesome. In general exposing an API that looks to the public like it is low-cost but is really relatively high-cost is a bad idea.
(Yes, we should always do buffered reads, which is why I specifically said we want to _deserialize_ one at a time" :)
> Doing a descending range still returns columns in ascending order
> -----------------------------------------------------------------
>
> Key: CASSANDRA-196
> URL: https://issues.apache.org/jira/browse/CASSANDRA-196
> Project: Cassandra
> Issue Type: Bug
> Reporter: Jonathan Ellis
>
> If I do
> result = table.getSliceFrom(row, "Standard1:col5", false, 2);
> cf = result.getColumnFamily("Standard1");
> I expect to get back columns in the order 5, 4, 3 (at the thrift level, it's turned into a list) but instead I get 3, 4, 5 because using a CF as the return vehicle re-sorts them by the standard comparator.
> The simplest solution is to allow user-defined column ordering as in CASSANDRA-185 and always return columns in that order (i.e., remove the ascending bool). This also allows us to make the columngroup fetching more efficient in the best case (deserializing one column at a time instead of a group at a time).
> OTOH using one index to allow fetching items relatively efficiently in either directly is cool. But my gut says it's relatively uncommon to want to access in both directions at once, and even more uncommon to not be able to do a reverse() on the client side (because of data volume, for instance). Forcing a separate CF for this special case of a special case might be worth the tradeoff.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-196) Doing a descending range still
returns columns in ascending order
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12733864#action_12733864 ]
Jonathan Ellis commented on CASSANDRA-196:
------------------------------------------
add(0) is a bad operation to do on a ArrayList, we should probably either use a deque or just do Collections.reverse at the end of the adds.
other than that it looks fine to me.
can you add a test slicing super subcolumns too?
> Doing a descending range still returns columns in ascending order
> -----------------------------------------------------------------
>
> Key: CASSANDRA-196
> URL: https://issues.apache.org/jira/browse/CASSANDRA-196
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jun Rao
> Fix For: 0.4
>
> Attachments: 196-systest.patch, issue196.patchv3, issue_196.patchv2
>
>
> If I do
> result = table.getSliceFrom(row, "Standard1:col5", false, 2);
> cf = result.getColumnFamily("Standard1");
> I expect to get back columns in the order 5, 4, 3 (at the thrift level, it's turned into a list) but instead I get 3, 4, 5 because using a CF as the return vehicle re-sorts them by the standard comparator.
> The simplest solution is to allow user-defined column ordering as in CASSANDRA-185 and always return columns in that order (i.e., remove the ascending bool). This also allows us to make the columngroup fetching more efficient in the best case (deserializing one column at a time instead of a group at a time).
> OTOH using one index to allow fetching items relatively efficiently in either directly is cool. But my gut says it's relatively uncommon to want to access in both directions at once, and even more uncommon to not be able to do a reverse() on the client side (because of data volume, for instance). Forcing a separate CF for this special case of a special case might be worth the tradeoff.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-196) Doing a descending range still
returns columns in ascending order
Posted by "Jun Rao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jun Rao updated CASSANDRA-196:
------------------------------
Attachment: issue_196.patchv2
Attache a patch, including a fix + the system test that Sandeep added.
> Doing a descending range still returns columns in ascending order
> -----------------------------------------------------------------
>
> Key: CASSANDRA-196
> URL: https://issues.apache.org/jira/browse/CASSANDRA-196
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jun Rao
> Fix For: 0.4
>
> Attachments: 196-systest.patch, issue_196.patchv2
>
>
> If I do
> result = table.getSliceFrom(row, "Standard1:col5", false, 2);
> cf = result.getColumnFamily("Standard1");
> I expect to get back columns in the order 5, 4, 3 (at the thrift level, it's turned into a list) but instead I get 3, 4, 5 because using a CF as the return vehicle re-sorts them by the standard comparator.
> The simplest solution is to allow user-defined column ordering as in CASSANDRA-185 and always return columns in that order (i.e., remove the ascending bool). This also allows us to make the columngroup fetching more efficient in the best case (deserializing one column at a time instead of a group at a time).
> OTOH using one index to allow fetching items relatively efficiently in either directly is cool. But my gut says it's relatively uncommon to want to access in both directions at once, and even more uncommon to not be able to do a reverse() on the client side (because of data volume, for instance). Forcing a separate CF for this special case of a special case might be worth the tradeoff.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-196) Doing a descending range still
returns columns in ascending order
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-196:
-------------------------------------
Fix Version/s: 0.4
> Doing a descending range still returns columns in ascending order
> -----------------------------------------------------------------
>
> Key: CASSANDRA-196
> URL: https://issues.apache.org/jira/browse/CASSANDRA-196
> Project: Cassandra
> Issue Type: Bug
> Reporter: Jonathan Ellis
> Fix For: 0.4
>
>
> If I do
> result = table.getSliceFrom(row, "Standard1:col5", false, 2);
> cf = result.getColumnFamily("Standard1");
> I expect to get back columns in the order 5, 4, 3 (at the thrift level, it's turned into a list) but instead I get 3, 4, 5 because using a CF as the return vehicle re-sorts them by the standard comparator.
> The simplest solution is to allow user-defined column ordering as in CASSANDRA-185 and always return columns in that order (i.e., remove the ascending bool). This also allows us to make the columngroup fetching more efficient in the best case (deserializing one column at a time instead of a group at a time).
> OTOH using one index to allow fetching items relatively efficiently in either directly is cool. But my gut says it's relatively uncommon to want to access in both directions at once, and even more uncommon to not be able to do a reverse() on the client side (because of data volume, for instance). Forcing a separate CF for this special case of a special case might be worth the tradeoff.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-196) Doing a descending range still
returns columns in ascending order
Posted by "Jun Rao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712147#action_12712147 ]
Jun Rao commented on CASSANDRA-196:
-----------------------------------
To fix the ordering of the returned result, we can reverse the returned list just before returning the thrift call, if the order is descending.
In terms of I/O, it is always better to fetch a group of columns from SSTable (instead of a column at a time). We can think about deserielizing a column at a time from the buffer. Doing this for descending order is bit difficult, but may still be possible.
> Doing a descending range still returns columns in ascending order
> -----------------------------------------------------------------
>
> Key: CASSANDRA-196
> URL: https://issues.apache.org/jira/browse/CASSANDRA-196
> Project: Cassandra
> Issue Type: Bug
> Reporter: Jonathan Ellis
>
> If I do
> result = table.getSliceFrom(row, "Standard1:col5", false, 2);
> cf = result.getColumnFamily("Standard1");
> I expect to get back columns in the order 5, 4, 3 (at the thrift level, it's turned into a list) but instead I get 3, 4, 5 because using a CF as the return vehicle re-sorts them by the standard comparator.
> The simplest solution is to allow user-defined column ordering as in CASSANDRA-185 and always return columns in that order (i.e., remove the ascending bool). This also allows us to make the columngroup fetching more efficient in the best case (deserializing one column at a time instead of a group at a time).
> OTOH using one index to allow fetching items relatively efficiently in either directly is cool. But my gut says it's relatively uncommon to want to access in both directions at once, and even more uncommon to not be able to do a reverse() on the client side (because of data volume, for instance). Forcing a separate CF for this special case of a special case might be worth the tradeoff.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-196) Doing a descending range still
returns columns in ascending order
Posted by "Sandeep Tata (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sandeep Tata updated CASSANDRA-196:
-----------------------------------
Attachment: 196-systest.patch
System test for descending slices.
> Doing a descending range still returns columns in ascending order
> -----------------------------------------------------------------
>
> Key: CASSANDRA-196
> URL: https://issues.apache.org/jira/browse/CASSANDRA-196
> Project: Cassandra
> Issue Type: Bug
> Reporter: Jonathan Ellis
> Fix For: 0.4
>
> Attachments: 196-systest.patch
>
>
> If I do
> result = table.getSliceFrom(row, "Standard1:col5", false, 2);
> cf = result.getColumnFamily("Standard1");
> I expect to get back columns in the order 5, 4, 3 (at the thrift level, it's turned into a list) but instead I get 3, 4, 5 because using a CF as the return vehicle re-sorts them by the standard comparator.
> The simplest solution is to allow user-defined column ordering as in CASSANDRA-185 and always return columns in that order (i.e., remove the ascending bool). This also allows us to make the columngroup fetching more efficient in the best case (deserializing one column at a time instead of a group at a time).
> OTOH using one index to allow fetching items relatively efficiently in either directly is cool. But my gut says it's relatively uncommon to want to access in both directions at once, and even more uncommon to not be able to do a reverse() on the client side (because of data volume, for instance). Forcing a separate CF for this special case of a special case might be worth the tradeoff.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.