You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Hernan Badenes (JIRA)" <ji...@apache.org> on 2010/01/12 22:11:54 UTC
[jira] Created: (CASSANDRA-691) get_key_range limit parameter has a
smaller range than i32
get_key_range limit parameter has a smaller range than i32
----------------------------------------------------------
Key: CASSANDRA-691
URL: https://issues.apache.org/jira/browse/CASSANDRA-691
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.9
Environment: linux x86
Reporter: Hernan Badenes
Priority: Minor
get_key_range fails with an exception (TApplicationException("Internal error processing get_key_range")) if the maxResults parameter is any value greater than 134217728 = 0x8000000 . Note that the i32 parameter allows a larger maximum value -- e.g. from Java one one try to pass Integer.MAX_VALUE to obtain all the keys.
I experience the problem throught thrift, both from java and python. The same code was working on 0.4.2, while this is happening on code just checked out from trunk. I don't think this is a thrift problem, since other calls (e.g. get_slice with a SlicePredicate(SliceRange)) allow MAX_INT to be passed.
Perhaps the solution is just to document the maximum allowed value in the API.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-691) get_key_range limit parameter has a
smaller range than i32
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-691:
-------------------------------------
Affects Version/s: (was: 0.9)
0.5
> get_key_range limit parameter has a smaller range than i32
> ----------------------------------------------------------
>
> Key: CASSANDRA-691
> URL: https://issues.apache.org/jira/browse/CASSANDRA-691
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: linux x86
> Reporter: Hernan Badenes
> Priority: Minor
>
> get_key_range fails with an exception (TApplicationException("Internal error processing get_key_range")) if the maxResults parameter is any value greater than 134217728 = 0x8000000 . Note that the i32 parameter allows a larger maximum value -- e.g. from Java one one try to pass Integer.MAX_VALUE to obtain all the keys.
> I experience the problem throught thrift, both from java and python. The same code was working on 0.4.2, while this is happening on code just checked out from trunk. I don't think this is a thrift problem, since other calls (e.g. get_slice with a SlicePredicate(SliceRange)) allow MAX_INT to be passed.
> Perhaps the solution is just to document the maximum allowed value in the API.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-691) get_key_range limit parameter has
a smaller range than i32
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799396#action_12799396 ]
Jonathan Ellis commented on CASSANDRA-691:
------------------------------------------
What is the internal error that gets logged on the server?
134217728 is a lot of keys.
> get_key_range limit parameter has a smaller range than i32
> ----------------------------------------------------------
>
> Key: CASSANDRA-691
> URL: https://issues.apache.org/jira/browse/CASSANDRA-691
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.9
> Environment: linux x86
> Reporter: Hernan Badenes
> Priority: Minor
>
> get_key_range fails with an exception (TApplicationException("Internal error processing get_key_range")) if the maxResults parameter is any value greater than 134217728 = 0x8000000 . Note that the i32 parameter allows a larger maximum value -- e.g. from Java one one try to pass Integer.MAX_VALUE to obtain all the keys.
> I experience the problem throught thrift, both from java and python. The same code was working on 0.4.2, while this is happening on code just checked out from trunk. I don't think this is a thrift problem, since other calls (e.g. get_slice with a SlicePredicate(SliceRange)) allow MAX_INT to be passed.
> Perhaps the solution is just to document the maximum allowed value in the API.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-691) get_key_range limit parameter has
a smaller range than i32
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799797#action_12799797 ]
Jonathan Ellis commented on CASSANDRA-691:
------------------------------------------
allowing you to OOM your server by requesting more rows than you have memory for is more of a tradeoff than a feature.
> get_key_range limit parameter has a smaller range than i32
> ----------------------------------------------------------
>
> Key: CASSANDRA-691
> URL: https://issues.apache.org/jira/browse/CASSANDRA-691
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: linux x86
> Reporter: Hernan Badenes
> Priority: Minor
>
> get_key_range fails with an exception (TApplicationException("Internal error processing get_key_range")) if the maxResults parameter is any value greater than 134217728 = 0x8000000 . Note that the i32 parameter allows a larger maximum value -- e.g. from Java one one try to pass Integer.MAX_VALUE to obtain all the keys.
> I experience the problem throught thrift, both from java and python. The same code was working on 0.4.2, while this is happening on code just checked out from trunk. I don't think this is a thrift problem, since other calls (e.g. get_slice with a SlicePredicate(SliceRange)) allow MAX_INT to be passed.
> Perhaps the solution is just to document the maximum allowed value in the API.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-691) get_key_range limit parameter has
a smaller range than i32
Posted by "Hernan Badenes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799794#action_12799794 ]
Hernan Badenes commented on CASSANDRA-691:
------------------------------------------
oops - ok, I will review my get_key_range calls and make sure I do not request too many at once. Is this then an intended "feature"?
> get_key_range limit parameter has a smaller range than i32
> ----------------------------------------------------------
>
> Key: CASSANDRA-691
> URL: https://issues.apache.org/jira/browse/CASSANDRA-691
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: linux x86
> Reporter: Hernan Badenes
> Priority: Minor
>
> get_key_range fails with an exception (TApplicationException("Internal error processing get_key_range")) if the maxResults parameter is any value greater than 134217728 = 0x8000000 . Note that the i32 parameter allows a larger maximum value -- e.g. from Java one one try to pass Integer.MAX_VALUE to obtain all the keys.
> I experience the problem throught thrift, both from java and python. The same code was working on 0.4.2, while this is happening on code just checked out from trunk. I don't think this is a thrift problem, since other calls (e.g. get_slice with a SlicePredicate(SliceRange)) allow MAX_INT to be passed.
> Perhaps the solution is just to document the maximum allowed value in the API.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CASSANDRA-691) get_key_range limit parameter has
a smaller range than i32
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis resolved CASSANDRA-691.
--------------------------------------
Resolution: Won't Fix
Yes, to fix a bug in 0.4, the 0.5 code creates a Set of size max_results to handle duplicate removal. Don't pass a max_results larger than you have enough memory to handle this with (or you'd OOM anyway if you actually got anywhere near that many results...)
> get_key_range limit parameter has a smaller range than i32
> ----------------------------------------------------------
>
> Key: CASSANDRA-691
> URL: https://issues.apache.org/jira/browse/CASSANDRA-691
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: linux x86
> Reporter: Hernan Badenes
> Priority: Minor
>
> get_key_range fails with an exception (TApplicationException("Internal error processing get_key_range")) if the maxResults parameter is any value greater than 134217728 = 0x8000000 . Note that the i32 parameter allows a larger maximum value -- e.g. from Java one one try to pass Integer.MAX_VALUE to obtain all the keys.
> I experience the problem throught thrift, both from java and python. The same code was working on 0.4.2, while this is happening on code just checked out from trunk. I don't think this is a thrift problem, since other calls (e.g. get_slice with a SlicePredicate(SliceRange)) allow MAX_INT to be passed.
> Perhaps the solution is just to document the maximum allowed value in the API.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-691) get_key_range limit parameter has
a smaller range than i32
Posted by "Hernan Badenes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799780#action_12799780 ]
Hernan Badenes commented on CASSANDRA-691:
------------------------------------------
$ ./Cassandra-remote -h myhost:9160 get_key_range KeySpace CF a z 134217728 1
(... all my keys ...)
$ ./Cassandra-remote -h myhost:9160 get_key_range KeySpace CF a z 134217729 1
...
: Internal error processing get_key_range
And in cassandra logs:
2010-01-13 13:11:16,039 ERROR Cassandra$Processor$get_key_range.process Internal error processing get_key_range
java.lang.OutOfMemoryError: Java heap space
I agree 134217728 is an absurd number of keys -- my point is that it would be better to know which is the limit for a request. Sometimes you just want to request "all keys", regardless the number in the database (I do, at least!).
The OOM error above looks strange -- is it there any allocation based on maxResults?
> get_key_range limit parameter has a smaller range than i32
> ----------------------------------------------------------
>
> Key: CASSANDRA-691
> URL: https://issues.apache.org/jira/browse/CASSANDRA-691
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.9
> Environment: linux x86
> Reporter: Hernan Badenes
> Priority: Minor
>
> get_key_range fails with an exception (TApplicationException("Internal error processing get_key_range")) if the maxResults parameter is any value greater than 134217728 = 0x8000000 . Note that the i32 parameter allows a larger maximum value -- e.g. from Java one one try to pass Integer.MAX_VALUE to obtain all the keys.
> I experience the problem throught thrift, both from java and python. The same code was working on 0.4.2, while this is happening on code just checked out from trunk. I don't think this is a thrift problem, since other calls (e.g. get_slice with a SlicePredicate(SliceRange)) allow MAX_INT to be passed.
> Perhaps the solution is just to document the maximum allowed value in the API.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.