You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "bjc (JIRA)" <ji...@apache.org> on 2010/02/08 23:33:27 UTC
[jira] Created: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
in a cluster, get_range_slice() does not return all the keys it should
----------------------------------------------------------------------
Key: CASSANDRA-781
URL: https://issues.apache.org/jira/browse/CASSANDRA-781
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.5
Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
Reporter: bjc
Fix For: 0.5, 0.6
get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
I posted a program to reproduce the behavior:
http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
Here are the other emails describing the issue:
http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: 0002-fix-slices-over-non-trivial-wrapped-ranges.txt
0001-fix-timeout-bug.txt
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 0001-fix-timeout-bug.txt, 0002-fix-slices-over-non-trivial-wrapped-ranges.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: 781.txt
Reproduced on trunk. Can you try the attached patch there? (We can backport to 0.5 afterwards.)
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831468#action_12831468 ]
Jonathan Ellis commented on CASSANDRA-781:
------------------------------------------
Sylvain: absolutely right, patch 01 has a fix for this.
Jack: updated patch 02 with more logging at INFO, maybe that will help. I can't reproduce even w/ your tokens and several runs of 100 keys. I am using a slightly simpler test, though:
import uuid
ks = "Keyspace1"
cf = "Super1"
path = ColumnPath(cf, "foo", "is")
value = "cool"
# insert, record keys in `keys` set
keys = set()
for i in xrange(100):
key = uuid.uuid4().hex
client.insert(ks, key, path, value, 0, ConsistencyLevel.ONE)
keys.add(key)
# remove keys found from set
parent = ColumnParent(column_family=cf)
slice_range = SliceRange(start="key", finish="key")
predicate = SlicePredicate(slice_range=slice_range)
result = client.get_range_slice(ks, parent, predicate, "", "", 1000, ConsistencyLevel.ONE)
for row in result:
keys.discard(row.key)
# if there are any left over, there is a bug
assert not keys, list(sorted(keys))
... this will of course only work until you insert more than 1000 keys. (maybe your original test has a similar limitation, i don't remember.)
(Are you still testing w/ RF of 2? If so maybe patch 01 will help you too.)
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 0001-fix-timeout-bug.txt, 0002-fix-slices-over-non-trivial-wrapped-ranges.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: Bounds.java
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 0001-fix-timeout-bug.txt, 0002-fix-slices-over-non-trivial-wrapped-ranges.txt, Bounds.java
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: (was: AbstractBounds.java)
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Hernan Badenes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831443#action_12831443 ]
Hernan Badenes commented on CASSANDRA-781:
------------------------------------------
I am hitting a similar problem on 3-node cluster I run. I do receive 5 rows in the get_range_slice query (from "" to ""), instead of an empty result set as described here (there exist more than just those 5 returned).
I am running 0.5.0 and could be able to test a patch for that version.
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "bjc (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831682#action_12831682 ]
bjc commented on CASSANDRA-781:
-------------------------------
Makes sense! Can you try one more time? I still can't apply the patch properly.. Or, maybe I'm doing something wrong? Here is the transcript.
$ svn checkout -r908163 https://svn.apache.org/repos/asf/incubator/cassandra/trunk cassandra
...
A cassandra/README.txt
U cassandra
Checked out revision 908163.
$ cd cassandra
$ wget https://issues.apache.org/jira/secure/attachment/12435345/0001-fix-timeout-bug.txt
--2010-02-09 21:40:51-- https://issues.apache.org/jira/secure/attachment/124353
45/0001-fix-timeout-bug.txt
Resolving issues.apache.org... 140.211.11.140
Connecting to issues.apache.org|140.211.11.140|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1873 (1.8K) [text/plain]
Saving to: `0001-fix-timeout-bug.txt'
100%[======================================>] 1,873 --.-K/s in 0s
2010-02-09 21:40:52 (47.0 MB/s) - `0001-fix-timeout-bug.txt' saved [1873/1873]
$ patch -p1 <0001-fix-timeout-bug.txt
patching file src/java/org/apache/cassandra/service/StorageProxy.java
$ wget https://issues.apache.org/jira/secure/attachment/12435346/0002-fix-slices-over-non-trivial-wrapped-ranges.txt
--2010-02-09 21:41:13-- https://issues.apache.org/jira/secure/attachment/124353
46/0002-fix-slices-over-non-trivial-wrapped-ranges.txt
Resolving issues.apache.org... 140.211.11.140
Connecting to issues.apache.org|140.211.11.140|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 14776 (14K) [text/plain]
Saving to: `0002-fix-slices-over-non-trivial-wrapped-ranges.txt'
100%[======================================>] 14,776 74.8K/s in 0.2s
2010-02-09 21:41:14 (74.8 KB/s) - `0002-fix-slices-over-non-trivial-wrapped-ranges.txt' saved [14776/14776]
$ patch -p1 <0002-fix-slices-over-non-trivial-wrapped-ranges.txt
patching file src/java/org/apache/cassandra/db/ColumnFamilyStore.java
patching file src/java/org/apache/cassandra/dht/AbstractBounds.java
Hunk #1 FAILED at 29.
1 out of 1 hunk FAILED -- saving rejects to file src/java/org/apache/cassandra/dht/AbstractBounds.java.rej
patching file src/java/org/apache/cassandra/dht/Bounds.java
Hunk #1 FAILED at 1.
Hunk #2 FAILED at 11.
2 out of 2 hunks FAILED -- saving rejects to file src/java/org/apache/cassandra/dht/Bounds.java.rej
patching file src/java/org/apache/cassandra/dht/Range.java
patching file src/java/org/apache/cassandra/service/StorageProxy.java
patching file src/java/org/apache/cassandra/service/StorageService.java
patching file test/unit/org/apache/cassandra/dht/BoundsTest.java
patching file test/unit/org/apache/cassandra/dht/RangeTest.java
$
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 0001-fix-timeout-bug.txt, 0002-fix-slices-over-non-trivial-wrapped-ranges.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834042#action_12834042 ]
Jonathan Ellis commented on CASSANDRA-781:
------------------------------------------
right, for get_range_slice we changed the contract from get_key_range -- it can't tell the difference between "this row has other data, but not data in the columns you requested" and "this row has been deleted entirely" w/o a relatively expensive query, so we decided to just return the slice as-is.
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12832828#action_12832828 ]
Jonathan Ellis commented on CASSANDRA-781:
------------------------------------------
If you're using RP instead of OPP you will see that.
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: (was: 781.txt)
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "bjc (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831752#action_12831752 ]
bjc commented on CASSANDRA-781:
-------------------------------
Ok, got the patch applied properly and things look better! The simple test passes. Awesome!! :) However, the more complicated test uses a "start" offset after the first get_range_slice(), and that still causes an exception. From the log:
INFO 23:15:52,425 scanning node range (20123910036548544936247138992367052936,67283373037552029587203789575295250400]
ERROR 23:15:52,425 Internal error processing get_range_slice
java.lang.AssertionError: [124451343962032323897724984972289130546,67283373037552029587203789575295250400]
at org.apache.cassandra.dht.Bounds.<init>(Bounds.java:26)
at org.apache.cassandra.dht.Bounds.getRangeOrBounds(Bounds.java:74)
at org.apache.cassandra.dht.Bounds.restrictTo(Bounds.java:59)
at org.apache.cassandra.service.StorageProxy.getRangeSlice(StorageProxy.java:559)
at org.apache.cassandra.thrift.CassandraServer.get_range_slice(CassandraServer.java:560)
at org.apache.cassandra.thrift.Cassandra$Processor$get_range_slice.process(Cassandra.java:1189)
at org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:984)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:253)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
If you have trouble duplicating this one I can give you more information, but I did not have difficulty getting it to happen. It happened every time I tried. For example, you can get it to happen just by modifying one line of the simple test:
result = client.get_range_slice(ks, parent, predicate, "b37e14bb37304e0096e2e77a8fc88a5b", "", 1000, ConsistencyLevel.ONE)
Of course, if you don't scan starting from "" then the simple test doesn't make sense, because you might specifically exclude keys you are looking for by starting from the string I put in.
However, the more complicated test I posted earlier makes sense and exercises the start and end ranges of get_range_slice(). So..can we go back to the complicated test and specifically make that one work? It's also nice because you should be able to run it over and over, since it removes the keys at the end. With the simple test you have to manually flush the data and restart the servers each time.
Thanks so much for fixing this! I am getting more familiar with the java so soon I might be able to fix some bugs like this.
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 0001-fix-timeout-bug.txt, 0002-fix-slices-over-non-trivial-wrapped-ranges.txt, AbstractBounds.java, Bounds.java
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Hernan Badenes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836616#action_12836616 ]
Hernan Badenes commented on CASSANDRA-781:
------------------------------------------
Great, thank you. The testcases for the current fix seem thorough, but if it helps I can do the testing with my own data as soon as a new patch is available.
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "bjc (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12832399#action_12832399 ]
bjc commented on CASSANDRA-781:
-------------------------------
Almost there I think. I found that the keys were not being returned in sorted order as they were before, so my trick of taking the last key in a limited range, and using that to start the next limited range did not work. So, I modified the test to sort the keys, then take the last one. When I did this I found another bug: when there are fewer keys in the specified range, duplicates are returned. Also, when I played around with the start and end for the range the server starting giving AssertionErrors:
ERROR 06:25:41,032 Internal error processing get_range_slice
java.lang.AssertionError: [125358492461525499902293558181143752059,1244525150549
22950280650433865080672503]
at org.apache.cassandra.dht.Bounds.<init>(Bounds.java:26)
at org.apache.cassandra.dht.Bounds.<init>(Bounds.java:18)
at org.apache.cassandra.thrift.CassandraServer.get_range_slice(Cassandra
Server.java:558)
at org.apache.cassandra.thrift.Cassandra$Processor$get_range_slice.proce
ss(Cassandra.java:1189)
at org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.jav
a:984)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadP
oolServer.java:253)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec
utor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:908)
at java.lang.Thread.run(Thread.java:619)
The modified test (below) gets stuck in an infinite loop. If you run it in ipython, you can control-c to get back to the interpretor, then type "result" to look at it's contents. It is a sorted list of keys. Look at the last key.. Here is the transcript showing the last two keys from result:
'ff8bfa30777f455695bf934ac7cfedac',
'ffb701ea740646b9955f0e339f8e3ee2']
In [70]: result2 = client.get_range_slice(ks, cparent, p, start, "", seg, cl)
In [71]: len(result2)
Out[71]: 1000
In [72]: result3 = client.get_range_slice(ks, cparent, p, start, "", seg, cl)
In [73]: len(result3)
Out[73]: 1000
In [74]: start
Out[74]: 'ffb701ea740646b9955f0e339f8e3ee2'
In [75]: result4 = client.get_range_slice(ks, cparent, p, start, start, seg, cl)
In [76]: len(result4)
Out[76]: 1000
In [77]: result5 = client.get_range_slice(ks, cparent, p, start, start, seg, cl)
In [78]: len(result5)
Out[78]: 1
In [79]:
That can't be right. Here is the latest test..
import sys
import time
import uuid
from thrift import Thrift
from thrift.transport import TTransport
from thrift.transport import TSocket
from thrift.protocol.TBinaryProtocol import TBinaryProtocolAccelerated
import sys
sys.path.insert(0,'/usr/local/cassandra/interface/thrift/gen-py')
from cassandra import Cassandra
from cassandra.ttypes import *
num_keys = 10000
socket = TSocket.TSocket("10.212.87.165", 9160)
transport = TTransport.TBufferedTransport(socket)
protocol = TBinaryProtocol.TBinaryProtocolAccelerated(transport)
client = Cassandra.Client(protocol)
ks = "Keyspace1"
cf = "Super1"
cl = ConsistencyLevel.ONE
d = {}
transport.open()
if 1:
## insert keys using the raw thrift interface
cpath = ColumnPath(cf, "foo", "is")
value = "cool"
for i in xrange(num_keys):
ts = time.time()
key = uuid.uuid4().hex
client.insert(ks, key, cpath, value, ts, cl)
d[key] = 1
else:
## insert keys using pycassa!
import pycassa
client = pycassa.connect(["10.212.87.165:9160"])
cf_test = pycassa.ColumnFamily(client, ks, cf, super=True)
for i in xrange(num_keys):
key = uuid.uuid4().hex
cf_test.insert(key, { 'params' : { 'is' : 'cool' }})
d[key] = 1
cparent = ColumnParent(column_family=cf)
slice_range = SliceRange(start="key", finish="key")
p = SlicePredicate(slice_range=slice_range)
done = False
seg = 1000
start = ""
## do a scan using either get_key_range() (deprecated) or get_range_slice()
## for every key returned that is in the dictionary, mark it as found
while not done:
print "start", start
result = client.get_range_slice(ks, cparent, p, start, "", seg, cl)
def getkey(x):
return x.key
result = map(getkey, result)
result.sort()
for r in result:
if d.has_key(r):
d[r] = 0
if len(result) < seg: done = True
else: start = result[seg-1]
cpath = ColumnPath(column_family=cf, super_column='foo')
## get, remove all the keys
## print all the keys that were not marked 0
for k in d:
result = client.get(ks, k, cpath, cl)
#print result
if d[k] == 1:
print k, "not marked 0"
#else:
# print k, "was marked 0!"
ts = time.time()
client.remove(ks, k, cpath, ts, cl)
BTW, this time around my nodetool worked perfectly! When I first brought the two nodes up, they selected keys that were too close, and one node ended up with all the load. So I ran loadbalance, and it worked great! That was really awesome. The only thing I noticed was a single key that should have been found returned a NotFoundException. I'll keep an eye on this one, too. Best,
Jack
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: (was: Bounds.java)
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: (was: 781.txt)
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834053#action_12834053 ]
Jonathan Ellis commented on CASSANDRA-781:
------------------------------------------
fix committed to trunk. backport to 0.5 pending.
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Hernan Badenes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12837242#action_12837242 ]
Hernan Badenes commented on CASSANDRA-781:
------------------------------------------
+1 to 781-backport.txt, it solved the problem in my cluster. Thank you!
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 781-backport.txt, 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "bjc (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833454#action_12833454 ]
bjc commented on CASSANDRA-781:
-------------------------------
Some of the patches didn't apply, some problem as before? I checked out freshly just now. Can you post the individual files again? Thanks.. Or..maybe this is the problem: why does your most recently attached patch have a Wednesday time stamp on it? Here is what I get at the top:
commit 630c33353647f062134d66afa3b487d95abe03fe
Author: Jonathan Ellis <jo...@rackspace.com>
Date: Wed Feb 10 18:04:08 2010 -0600
fix range queries
Shouldn't that be Friday? Here's the transcript:
$ svn checkout https://svn.apache.org/repos/asf/incubator/cassan
dra/trunk cassandra
A cassandra/test
A cassandra/test/unit
...
$ cd cassandra
$ wget https://issues.apache.org/jira/secure/attachment/12435762
/781.txt
--2010-02-13 20:57:49-- https://issues.apache.org/jira/secure/attachment/124357
62/781.txt
Resolving issues.apache.org... 140.211.11.140
Connecting to issues.apache.org|140.211.11.140|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 47140 (46K) [text/plain]
Saving to: `781.txt'
100%[======================================>] 47,140 120K/s in 0.4s
2010-02-13 20:57:50 (120 KB/s) - `781.txt' saved [47140/47140]
$ patch -p1 <781.txt
patching file src/java/org/apache/cassandra/db/ColumnFamilyStore.java
patching file src/java/org/apache/cassandra/db/RangeSliceReply.java
patching file src/java/org/apache/cassandra/dht/AbstractBounds.java
Hunk #1 FAILED at 1.
Hunk #2 FAILED at 25.
2 out of 2 hunks FAILED -- saving rejects to file src/java/org/apache/cassandra/
dht/AbstractBounds.java.rej
patching file src/java/org/apache/cassandra/dht/Bounds.java
Hunk #1 FAILED at 1.
Hunk #2 FAILED at 8.
2 out of 2 hunks FAILED -- saving rejects to file src/java/org/apache/cassandra/
dht/Bounds.java.rej
patching file src/java/org/apache/cassandra/dht/Range.java
patching file src/java/org/apache/cassandra/service/RangeSliceResponseResolver.j
ava
patching file src/java/org/apache/cassandra/service/StorageProxy.java
patching file src/java/org/apache/cassandra/service/StorageService.java
patching file src/java/org/apache/cassandra/thrift/CassandraServer.java
patching file src/java/org/apache/cassandra/thrift/ThriftValidation.java
patching file test/system/test_server.py
patching file test/unit/org/apache/cassandra/dht/BoundsTest.java
patching file test/unit/org/apache/cassandra/dht/RangeIntersectionTest.java
patching file test/unit/org/apache/cassandra/dht/RangeTest.java
$
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis resolved CASSANDRA-781.
--------------------------------------
Resolution: Fixed
committed to 0.5, thanks for testing.
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 781-backport.txt, 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: (was: 781.txt)
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Hernan Badenes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836603#action_12836603 ]
Hernan Badenes commented on CASSANDRA-781:
------------------------------------------
Is this fix going to be backported to 0.5? (any ETA?)
Thanks!
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: 781.txt
re-attached w/ line endings fixed.
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: AbstractBounds.java
weird, happens for me too. lame!
attached Bounds and AbstractBounds as they should look, post-patch. just do what you showed in the transcript, then overwrite the local copies with these.
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 0001-fix-timeout-bug.txt, 0002-fix-slices-over-non-trivial-wrapped-ranges.txt, AbstractBounds.java, Bounds.java
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: (was: 781.txt)
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: (was: 781.txt)
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "bjc (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12832830#action_12832830 ]
bjc commented on CASSANDRA-781:
-------------------------------
Ahh!! Right you are, I was using RP instead of OPP. Ok, but now here is another problem: if I insert 10 keys and then ask for them back, it works. However, if I insert 10 more and do a range scan with start="", I don't get the lowest key:
In [17]: run test_bug_simple3.py
insert aa3cf33059d64dac8aef4a250bc5ea9c
insert a7dbda2925eb4b439c89cd71d56b5113
insert f28e92d5e5554857940c9d3386bf4121
insert 2b8ec460e7d346cbaf3dcb00e1aaaf91
insert 7792c98f0c3948299c622c73b906df66
insert 37a8bfdb69b642ba8e96d33b060f789d
insert 38c18f5d3d2c46cbb4e44b603a8acdbd
insert bef8104ea9184abaa3f0788ef7b2e0db
insert 934fe04d30cc4a96b1f1a9e7930316b8
insert 1d3413e88af946349f148c4fafeb6bf7
result 1d3413e88af946349f148c4fafeb6bf7
result 2b8ec460e7d346cbaf3dcb00e1aaaf91
result 37a8bfdb69b642ba8e96d33b060f789d
result 38c18f5d3d2c46cbb4e44b603a8acdbd
result 7792c98f0c3948299c622c73b906df66
result a7dbda2925eb4b439c89cd71d56b5113
result aa3cf33059d64dac8aef4a250bc5ea9c
result bef8104ea9184abaa3f0788ef7b2e0db
result f28e92d5e5554857940c9d3386bf4121
start f28e92d5e5554857940c9d3386bf4121
result f28e92d5e5554857940c9d3386bf4121
In [18]: run test_bug_simple3.py
insert 4eb0300540ec4b4083fbaf33741fc4a5
insert 12b43ba967314b369faff7e59902d6c2
insert 5b4b729676bc4ea2816620c3b6dff080
insert cf2fda1b11d843f1ae7949dbbb7d179d
insert c9d0cf4a1e9a48caa143afd2b0268f70
insert 9a044cff59b940d5bfbeffd58b01ee8e
insert d2ee042f0b0b4f7ea86e6e2c0dfdcfdd
insert d239aee577684c27afea2fe7e3361bdf
insert 706b20976f974de49bda61d55b9c2a63
insert 36177455bc3b4469b7e6f51897c9f3ba
result a7dbda2925eb4b439c89cd71d56b5113
result aa3cf33059d64dac8aef4a250bc5ea9c
result bef8104ea9184abaa3f0788ef7b2e0db
result c9d0cf4a1e9a48caa143afd2b0268f70
result cf2fda1b11d843f1ae7949dbbb7d179d
start cf2fda1b11d843f1ae7949dbbb7d179d
result cf2fda1b11d843f1ae7949dbbb7d179d
result d239aee577684c27afea2fe7e3361bdf
result d2ee042f0b0b4f7ea86e6e2c0dfdcfdd
result f28e92d5e5554857940c9d3386bf4121
In [19]:
See what I mean? In the first run I inserted "1d3413e88af946349f148c4fafeb6bf7" but the second range scan I get "a7dbda2925eb4b439c89cd71d56b5113" back first, even when I set start="". Could this somehow be my fault too?
Test follows:
import uuid
from thrift import Thrift
from thrift.transport import TTransport
from thrift.transport import TSocket
from thrift.protocol.TBinaryProtocol import TBinaryProtocolAccelerated
import sys
sys.path.insert(0,'/usr/local/cassandra/interface/thrift/gen-py')
from cassandra import Cassandra
from cassandra.ttypes import *
socket = TSocket.TSocket("10.212.87.165", 9160)
transport = TTransport.TBufferedTransport(socket)
protocol = TBinaryProtocol.TBinaryProtocolAccelerated(transport)
client = Cassandra.Client(protocol)
transport.open()
ks = "Keyspace1"
cf = "Super1"
path = ColumnPath(cf, "foo", "is")
value = "cool"
for i in xrange(10):
key = uuid.uuid4().hex
print "insert", key
client.insert(ks, key, path, value, 0, ConsistencyLevel.ONE)
parent = ColumnParent(column_family=cf)
slice_range = SliceRange(start="key", finish="key")
predicate = SlicePredicate(slice_range=slice_range)
result = client.get_range_slice(ks, parent, predicate, "", "", 5, ConsistencyLevel.ONE)
for row in result:
print "result", row.key
start = result[-1].key
print "start", start
result = client.get_range_slice(ks, parent, predicate, start, "", 10, ConsistencyLevel.ONE)
for row in result:
print "result", row.key
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831451#action_12831451 ]
Sylvain Lebresne commented on CASSANDRA-781:
--------------------------------------------
I add the same problem, range_slice on 2 nodes was missing results.
I updated to trunk and applied 781.txt. It fixes the missing results but
introduce a timeout exception (I don't believe it is the patch fault though).
The timeout happens with consistencyLevel.ONE (but I suspect it could happen
with QUORUM too). Looking a bit to the details, it seems that
RangeSliceResponseResolver wait for a response from every live natural
endpoints (in isDataPresent()). But in StorageProxy, the rangeSlice message is
only sent to 'responseCount' endpoints (which will be 1 for
consistencyLevel.ONE). Hence the timeout.
I'm not sure what would be the best way to deal with that though.
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "bjc (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834016#action_12834016 ]
bjc commented on CASSANDRA-781:
-------------------------------
Victory!! I think this patch works. :)
One last possible issue: if I remove keys, then do a get_range_slice(), they still show up. A get() on a removed key will return a "not found" exception. Should get_range_slice() be aware of the removal? I'm guessing this is an issue about not properly processing the "tombstone".
Due to the complexity of get_range_slice(), maybe it's not worth processing the tombstone?
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836612#action_12836612 ]
Jonathan Ellis commented on CASSANDRA-781:
------------------------------------------
Yes, that's at the top of my list. (I've been busy with PyCon.)
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Component/s: Core
Fix Version/s: (was: 0.6)
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: 0002-fix-slices-over-non-trivial-wrapped-ranges.txt
0001-fix-timeout-bug.txt
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 0001-fix-timeout-bug.txt, 0002-fix-slices-over-non-trivial-wrapped-ranges.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "bjc (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12832827#action_12832827 ]
bjc commented on CASSANDRA-781:
-------------------------------
I don't think my test is buggy. I realize that the range is start inclusive, and it does pass a key that exists as start, but sets "done = True" if the range scan returns less keys than requested. Since it passes "" as the end/finish, this should return less keys than requested when you get to the end, provided you ask for more than one key (which I do).
I think the last remaining problem is with the sorting! I bet that is why using "" for my finish string doesn't work. Here's the problem I see now (transcript followed by test):
I put 10 random keys in, ask for them back. They aren't sorted, so I sort them and take the highest. I use that as start, and "" as finish. This should give me one key back, but instead I get 10. Could it be that my columnfamily definition is different than yours? Here's mine:
<ColumnFamily ColumnType="Super"
CompareWith="UTF8Type"
CompareSubcolumnsWith="UTF8Type"
Name="Super1"
RowsCached="1000"
KeysCachedFraction="0"
Comment="A column family with supercolumns, whose column and subcolumn names are UTF8 strings"/>
In [17]: run test_bug_simple2.py
result1 before sorting
af37b718213b4219897ea1564ebc8900
f196ad5537294840b2de0a636202dbd2
7578ba38b66d4708a38663717e020959
b5266af926a647c3a1a4d2f62dfe952c
d729d5181bac42a48ac3e49d9700047e
4d58b6fbea214d0c9c7a9f288feba2d8
41df7aee7d674a75a4943d89153f9bde
4e121f95459e4f67a6cd3c06b2d078e7
99b2b03675a8413f94e60e3d1bbded8c
66121c5c863f4c1f804a46b8c2136fe9
result1 after sorting
41df7aee7d674a75a4943d89153f9bde
4d58b6fbea214d0c9c7a9f288feba2d8
4e121f95459e4f67a6cd3c06b2d078e7
66121c5c863f4c1f804a46b8c2136fe9
7578ba38b66d4708a38663717e020959
99b2b03675a8413f94e60e3d1bbded8c
af37b718213b4219897ea1564ebc8900
b5266af926a647c3a1a4d2f62dfe952c
d729d5181bac42a48ac3e49d9700047e
f196ad5537294840b2de0a636202dbd2
start f196ad5537294840b2de0a636202dbd2
result2
f196ad5537294840b2de0a636202dbd2
7578ba38b66d4708a38663717e020959
b5266af926a647c3a1a4d2f62dfe952c
d729d5181bac42a48ac3e49d9700047e
4d58b6fbea214d0c9c7a9f288feba2d8
41df7aee7d674a75a4943d89153f9bde
4e121f95459e4f67a6cd3c06b2d078e7
99b2b03675a8413f94e60e3d1bbded8c
66121c5c863f4c1f804a46b8c2136fe9
b8b290a864464271ad30df1bbab2f2b7
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
/k/jack/bridge/test_bug_simple2.py in <module>()
54 for r in result2: print r.key
55
---> 56 assert len(result2) == 1
57
58
AssertionError:
WARNING: Failure executing file: <test_bug_simple2.py>
In [18]:
import uuid
from thrift import Thrift
from thrift.transport import TTransport
from thrift.transport import TSocket
from thrift.protocol.TBinaryProtocol import TBinaryProtocolAccelerated
import sys
sys.path.insert(0,'/usr/local/cassandra/interface/thrift/gen-py')
from cassandra import Cassandra
from cassandra.ttypes import *
socket = TSocket.TSocket("10.212.87.165", 9160)
transport = TTransport.TBufferedTransport(socket)
protocol = TBinaryProtocol.TBinaryProtocolAccelerated(transport)
client = Cassandra.Client(protocol)
transport.open()
ks = "Keyspace1"
cf = "Super1"
path = ColumnPath(cf, "foo", "is")
value = "cool"
for i in xrange(100):
key = uuid.uuid4().hex
client.insert(ks, key, path, value, 0, ConsistencyLevel.ONE)
parent = ColumnParent(column_family=cf)
slice_range = SliceRange(start="key", finish="key")
predicate = SlicePredicate(slice_range=slice_range)
result1 = client.get_range_slice(ks, parent, predicate, "", "", 10, ConsistencyLevel.ONE)
print "result1 before sorting"
for r in result1: print r.key
def getkey(x): return x.key
print "result1 after sorting"
result1 = map(getkey, result1)
result1.sort()
for r in result1: print r
start = result1[-1]
print "start", start
result2 = client.get_range_slice(ks, parent, predicate, start, "", 10, ConsistencyLevel.ONE)
print "result2"
for r in result2: print r.key
assert len(result2) == 1
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: (was: 0001-fix-timeout-bug.txt)
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis reassigned CASSANDRA-781:
----------------------------------------
Assignee: Jonathan Ellis
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "bjc (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831346#action_12831346 ]
bjc commented on CASSANDRA-781:
-------------------------------
Before applying 781.txt to trunk I was getting an exception. The exception is now gone! Awesome. However, I still have the range scanning problem. Here are a few example runs of the test:
$ python test_bug.py get_range_slice
ebbde791748641be951802d64d48c62d not marked 0
9bfa7ad8abce48a9a45daccfa3772f29 not marked 0
$ python test_bug.py get_range_slice
c554dc532bfb462b950990b6824f11c1 not marked 0
e6c4a0100508451ea3a1d13088877dd9 not marked 0
$ python test_bug.py get_range_slice
$ python test_bug.py get_range_slice
$ python test_bug.py get_range_slice
27fc88c8e7ab489d96f4c749cc86aca1 not marked 0
$ python test_bug.py get_range_slice
b522cca4bd6f4282a507525598139f95 not marked 0
$ python test_bug.py get_range_slice
d045b3edabc949fea30242722a11587a not marked 0
$
Sigh, I just realized that under trunk my nodetool doesn't work. However, I got the tokens from the log:
INFO 08:10:50,338 Saved Token not found. Using 68054825649105441942293089893012253843
INFO 08:10:53,293 Saved Token not found. Using 44181284974408316254372647768836513112
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: 781.txt
new patch attached.
> ERROR 06:25:41,032 Internal error processing get_range_slice
added InvalidRequestException when start > end, which fixes this.
> The modified test (below) gets stuck in an infinite loop
Your test is buggy. :)
range_slice (like key_range) is start-INCLUSIVE, so if you pass a key that exists as start, you will always get at least one result, the start one.
> the keys were not being returned in sorted order
This is working fine for me. Not sure what you were seeing.
> when there are fewer keys in the specified range, duplicates are returned
Sounds like another illustration of start-inclusiveness.
If you still see problems, can you narrow it down to a specific set of keys, rather than relying on randomness to maybe reproduce it once in a while? That would help a lot. Thanks!
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "bjc (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833477#action_12833477 ]
bjc commented on CASSANDRA-781:
-------------------------------
Thanks for fixing the patch, but I'm sorry...it still doesn't work right.. :( I found a couple of weird things. First, sometimes when I ask for 10, it gives more. Second, if I pass start="" it still doesn't start at the beginning.
However, this doesn't happen every time I start fresh. Maybe it's depedent on the tokens. Here are the tokens for the case I show below:
INFO 23:38:21,989 Saved Token not found. Using 0njMRYmU9KiXE80d
INFO 23:38:20,112 Saved Token not found. Using I8LW8J6h9UCuz0dC
The first token belongs to the node I am attaching the client to.
This functionality seems surprisingly complicated. Maybe it would help to write down pseudo-code for the way it should work? I have to admit that I cannot piece it together by reading the comments in your patch.
Here is a transcript and test:
First run, looks ok:
ip-10-212-87-165$ python test_bug_simple3.py
insert 10b470f49a7c46bd938d784ca4096b63
insert 47f70aacb3e94e10acaf8e86edac7169
insert 9a3b7d3b921345bebc4f2bedc1db7c01
insert c1c9dab59abd4f4ca33ee79f71a179e9
insert cff81b145faf4648ac8ae001973c6c75
insert c752d33e5d344312908e5008e6cdae3e
insert 6e9e32e8b89845bb935d993a9c8bcb13
insert c286bf2711bc45c1ab033561112c2313
insert 2dad487ddfa94c81b52c8b4d35d3cb5c
insert 9c62c7dafdb94dfdbdf52b527bdd2b24
result 10b470f49a7c46bd938d784ca4096b63
result 2dad487ddfa94c81b52c8b4d35d3cb5c
result 47f70aacb3e94e10acaf8e86edac7169
result 6e9e32e8b89845bb935d993a9c8bcb13
result 9a3b7d3b921345bebc4f2bedc1db7c01
result 9c62c7dafdb94dfdbdf52b527bdd2b24
result c1c9dab59abd4f4ca33ee79f71a179e9
result c286bf2711bc45c1ab033561112c2313
result c752d33e5d344312908e5008e6cdae3e
result cff81b145faf4648ac8ae001973c6c75
total_keys 10
Second run, get 18 keys when I asked for 10:
ip-10-212-87-165$ python test_bug_simple3.py
insert f873d662dccf46c28080a01286e09ed8
insert 903776c2f45740389aa52675bf47c7ec
insert 0e80401a9052405a898d11e5ae874a13
insert 398d51ba174b4c9db8c25ca6cd2c9454
insert 50f1cd47dd284ee9b9573b4dfce39134
insert 20fa43d2365b4dfab9b05a93992315d0
insert e009d5b76e8840b784fe6b9b649ae1df
insert 63497f9d63c74b99a681fa2fc52751ac
insert 824bbcf997de48a99cad174e9e1f1eec
insert 01c5a6506f4247068660c20338a03bb3
result 01c5a6506f4247068660c20338a03bb3
result 0e80401a9052405a898d11e5ae874a13
result 10b470f49a7c46bd938d784ca4096b63
result 20fa43d2365b4dfab9b05a93992315d0
result 2dad487ddfa94c81b52c8b4d35d3cb5c
result 398d51ba174b4c9db8c25ca6cd2c9454
result 47f70aacb3e94e10acaf8e86edac7169
result 50f1cd47dd284ee9b9573b4dfce39134
result 63497f9d63c74b99a681fa2fc52751ac
result 6e9e32e8b89845bb935d993a9c8bcb13
result 824bbcf997de48a99cad174e9e1f1eec
result 903776c2f45740389aa52675bf47c7ec
result c1c9dab59abd4f4ca33ee79f71a179e9
result c286bf2711bc45c1ab033561112c2313
result c752d33e5d344312908e5008e6cdae3e
result cff81b145faf4648ac8ae001973c6c75
result e009d5b76e8840b784fe6b9b649ae1df
result f873d662dccf46c28080a01286e09ed8
total_keys 18
Third run, start at "ca.." even though I pass start="" and all the previous keys remain:
ip-10-212-87-165$ python test_bug_simple3.py
insert ca97d7efb63448f8a62d6f7f73044236
insert 91363a713b714af88ac2191caeea5351
insert 7b6756d0ab8e450b826b1abc7210d524
insert e6c6765497af4078b93e1a1470bd3194
insert e3457f26754c4e7cb7ef606f98e7bb78
insert 99643eb237ea4ca8b50cac4bb4d58edd
insert ec3e1f81359b4ae08cfed73899934a93
insert ae2b990ceb044bf194a879059f823ecf
insert 2c1494f0ad3d48d2bf4feb33f40cf38e
insert 0cb1c2e906b64fee89f7729052e0810e
result ae2b990ceb044bf194a879059f823ecf
result c1c9dab59abd4f4ca33ee79f71a179e9
result c286bf2711bc45c1ab033561112c2313
result c752d33e5d344312908e5008e6cdae3e
result ca97d7efb63448f8a62d6f7f73044236
result cff81b145faf4648ac8ae001973c6c75
result e009d5b76e8840b784fe6b9b649ae1df
result e3457f26754c4e7cb7ef606f98e7bb78
result e6c6765497af4078b93e1a1470bd3194
result ec3e1f81359b4ae08cfed73899934a93
total_keys 10
ip-10-212-87-165$
Here is another run (different tokens):
INFO 00:04:17,105 Saved Token not found. Using Iw1khrAgM5sd6WnX
INFO 00:04:15,795 Saved Token not found. Using IgLbq912n2xEP99G
In this case I don't see the problem where I get back more keys than I asked for, but I don't get the 10 lowest keys in the second request. 10 are returned, but they are not ordered consistently with what I know is in the db.
First run, notice key "893.." is inserted:
ip-10-212-87-165$ python test_bug_simple3.py
insert 7be5d87bc45843cfaffd36fd654aee53
insert 8ef4727d83474570aa2111bee3929a5f
insert 9a9a91b6b662430092db0209d63a5c9e
insert e45afe1f0e364012acd0dead5b75ea13
insert 10171c87634842aea4f16d46d611c435
insert 10b6f92ac6a447088a82c4ec13056f1e
insert c1acbde9ae454ea2819322975322206b
insert 89352cf117dd4cb9ab935cbb5f230ba0
insert 0b5d924f04174459969594d6293b9aca
insert e2471db4d8f445f2b0c36f3b2a5bb650
result 0b5d924f04174459969594d6293b9aca
result 10171c87634842aea4f16d46d611c435
result 10b6f92ac6a447088a82c4ec13056f1e
result 7be5d87bc45843cfaffd36fd654aee53
result 89352cf117dd4cb9ab935cbb5f230ba0
result 8ef4727d83474570aa2111bee3929a5f
result 9a9a91b6b662430092db0209d63a5c9e
result c1acbde9ae454ea2819322975322206b
result e2471db4d8f445f2b0c36f3b2a5bb650
result e45afe1f0e364012acd0dead5b75ea13
total_keys 10
Second run, notice the results start with "0.." but "893.." is not returned (though "b2.." is, and other higher keys):
ip-10-212-87-165$ python test_bug_simple3.py
insert 85d282dfa03a466eb51d03f4eb5dacd5
insert a61e7757eaed4ef79fc7bf35f47843f7
insert 9b9b6e3f22994827b0dddcc16105ff7d
insert 880b1644636845d8b1c92faf1f6d8484
insert 5d1d7d7b26ec4540a89c027bccc17e06
insert 4df05d38950f44b29df604c165e1148f
insert 0c9f818aa28a47fb832b6a0929b94280
insert 7e7b829c136046a88b120a4a373d9a6b
insert b2d497713f9342de85bb31b5c0e69af6
insert 80e250253f1942aaab2e9b49880918c0
result 0b5d924f04174459969594d6293b9aca
result 0c9f818aa28a47fb832b6a0929b94280
result 10171c87634842aea4f16d46d611c435
result 10b6f92ac6a447088a82c4ec13056f1e
result 4df05d38950f44b29df604c165e1148f
result a61e7757eaed4ef79fc7bf35f47843f7
result b2d497713f9342de85bb31b5c0e69af6
result c1acbde9ae454ea2819322975322206b
result e2471db4d8f445f2b0c36f3b2a5bb650
result e45afe1f0e364012acd0dead5b75ea13
total_keys 10
ip-10-212-87-165$
I found another issue with "nodetool ring" which might be related to the patch:
$ sudo bin/nodetool -h localhost ring
Exception in thread "main" java.lang.reflect.UndeclaredThrowableException
at $Proxy0.getRangeToEndPointMap(Unknown Source)
at org.apache.cassandra.tools.NodeProbe.getRangeToEndPointMap(NodeProbe.java:151)
at org.apache.cassandra.tools.NodeCmd.printRing(NodeCmd.java:74)
at org.apache.cassandra.tools.NodeCmd.main(NodeCmd.java:403)
Caused by: java.rmi.UnmarshalException: error unmarshalling return; nested exception is:
java.io.WriteAbortedException: writing aborted; java.io.NotSerializableException: org.apache.cassandra.dht.OrderPreservingPartitioner
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:173)
at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)
at javax.management.remote.rmi.RMIConnectionImpl_Stub.invoke(Unknown Source)
at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.invoke(RMIConnector.java:993)
at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:288)
... 4 more
Caused by: java.io.WriteAbortedException: writing aborted; java.io.NotSerializableException: org.apache.cassandra.dht.OrderPreservingPartitioner
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1333)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
at java.util.HashMap.readObject(HashMap.java:1029)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1849)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
at sun.rmi.server.UnicastRef.unmarshalValue(UnicastRef.java:306)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:155)
... 8 more
Caused by: java.io.NotSerializableException: org.apache.cassandra.dht.OrderPreservingPartitioner
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1156)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1474)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326)
at java.util.HashMap.writeObject(HashMap.java:1000)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:945)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1461)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326)
at sun.rmi.server.UnicastRef.marshalValue(UnicastRef.java:274)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:315)
at sun.rmi.transport.Transport$1.run(Transport.java:159)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
$
Here is the test I ran above:
import uuid
from thrift import Thrift
from thrift.transport import TTransport
from thrift.transport import TSocket
from thrift.protocol.TBinaryProtocol import TBinaryProtocolAccelerated
import sys
sys.path.insert(0,'/usr/local/cassandra/interface/thrift/gen-py')
from cassandra import Cassandra
from cassandra.ttypes import *
socket = TSocket.TSocket("10.212.87.165", 9160)
transport = TTransport.TBufferedTransport(socket)
protocol = TBinaryProtocol.TBinaryProtocolAccelerated(transport)
client = Cassandra.Client(protocol)
transport.open()
ks = "Keyspace1"
cf = "Super1"
path = ColumnPath(cf, "foo", "is")
value = "cool"
for i in xrange(10):
key = uuid.uuid4().hex
print "insert", key
client.insert(ks, key, path, value, 0, ConsistencyLevel.ONE)
print
parent = ColumnParent(column_family=cf)
slice_range = SliceRange(start="key", finish="key")
predicate = SlicePredicate(slice_range=slice_range)
total_keys = 0
result = client.get_range_slice(ks, parent, predicate, "", "", 10, ConsistencyLevel.ONE)
for row in result:
total_keys += 1
print "result", row.key
print "total_keys", total_keys
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: (was: 0002-fix-slices-over-non-trivial-wrapped-ranges.txt)
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 0001-fix-timeout-bug.txt, 0002-fix-slices-over-non-trivial-wrapped-ranges.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: 781.txt
Yes, it's more complicated than it looks. :)
Attached version fixes regression w/ result set size, and also start key when it falls into a wrapped node range.
There's also a ton of debug logging if you turn that on in log4j, btw.
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: 781.txt
Ah, yes, definitely reintroduced a bug in picking the range to start scanning in. Fix attached.
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834886#action_12834886 ]
Hudson commented on CASSANDRA-781:
----------------------------------
Integrated in Cassandra #357 (See [http://hudson.zones.apache.org/hudson/job/Cassandra/357/])
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: 781.txt
committed patch 1, the timeout fix.
here is a new patch that should fix the remaining range issues.
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831677#action_12831677 ]
Jonathan Ellis commented on CASSANDRA-781:
------------------------------------------
Sorry, those patches that didn't apply were probably important. :)
I've rebased against r908163, can you revert and try again?
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 0001-fix-timeout-bug.txt, 0002-fix-slices-over-non-trivial-wrapped-ranges.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: 781-backport.txt
attached backport of fix to 0.5.
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 781-backport.txt, 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: (was: 0002-fix-slices-over-non-trivial-wrapped-ranges.txt)
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 781.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-781:
-------------------------------------
Attachment: (was: 0001-fix-timeout-bug.txt)
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 0001-fix-timeout-bug.txt, 0002-fix-slices-over-non-trivial-wrapped-ranges.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-781) in a cluster, get_range_slice()
does not return all the keys it should
Posted by "bjc (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831668#action_12831668 ]
bjc commented on CASSANDRA-781:
-------------------------------
Ok, I checked out a fresh copy of trunk and applied both new patches (0001 and 0002). There were some errors for 0002, but upon looking at the code it seems parts of the patch have already been committed to the SVN repo, so what I ended up with is the right thing.
$ patch -p1 <0001-fix-timeout-bug.txt
patching file src/java/org/apache/cassandra/service/StorageProxy.java
$ patch -p1 <0002-fix-slices-over-non-trivial-wrapped-ranges.txt
patching file src/java/org/apache/cassandra/db/ColumnFamilyStore.java
Hunk #1 succeeded at 43 (offset 1 line).
Hunk #2 succeeded at 1075 (offset 2 lines).
patching file src/java/org/apache/cassandra/dht/AbstractBounds.java
Hunk #1 FAILED at 29.
1 out of 1 hunk FAILED -- saving rejects to file src/java/org/apache/cassandra/dht/AbstractBounds.java.rej
patching file src/java/org/apache/cassandra/dht/Bounds.java
Hunk #1 FAILED at 1.
Hunk #2 FAILED at 11.
2 out of 2 hunks FAILED -- saving rejects to file src/java/org/apache/cassandra/dht/Bounds.java.rej
patching file src/java/org/apache/cassandra/dht/Range.java
patching file src/java/org/apache/cassandra/service/StorageProxy.java
patching file src/java/org/apache/cassandra/service/StorageService.java
Reversed (or previously applied) patch detected! Assume -R? [n]
Apply anyway? [n]
Skipping patch.
3 out of 3 hunks ignored -- saving rejects to file src/java/org/apache/cassandra/service/StorageService.java.rej
patching file test/unit/org/apache/cassandra/dht/BoundsTest.java
patching file test/unit/org/apache/cassandra/dht/RangeTest.java
$
Copied my storage-conf.xml into place, which has only two changes: Seeds defined and binding addresses changed to null strings. Thus, my RF is 1 now, not 2 as it was before.
I now use your simpler test. Here is the test with all the import statements:
import uuid
from thrift import Thrift
from thrift.transport import TTransport
from thrift.transport import TSocket
from thrift.protocol.TBinaryProtocol import TBinaryProtocolAccelerated
from cassandra import Cassandra
from cassandra.ttypes import *
num_keys = 10
socket = TSocket.TSocket("10.212.87.165", 9160)
transport = TTransport.TBufferedTransport(socket)
protocol = TBinaryProtocol.TBinaryProtocolAccelerated(transport)
client = Cassandra.Client(protocol)
transport.open()
ks = "Keyspace1"
cf = "Super1"
path = ColumnPath(cf, "foo", "is")
value = "cool"
# insert, record keys in `keys` set
keys = set()
for i in xrange(100):
key = uuid.uuid4().hex
client.insert(ks, key, path, value, 0, ConsistencyLevel.ONE)
keys.add(key)
# remove keys found from set
parent = ColumnParent(column_family=cf)
slice_range = SliceRange(start="key", finish="key")
predicate = SlicePredicate(slice_range=slice_range)
result = client.get_range_slice(ks, parent, predicate, "", "", 1000, ConsistencyLevel.ONE)
for row in result:
keys.discard(row.key)
# if there are any left over, there is a bug
assert not keys, list(sorted(keys))
I cleared out the data/commitlog dirs and launched both nodes. The tokens:
INFO 21:02:08,768 Saved Token not found. Using 136351045523563703929320485474511375137
INFO 21:02:09,509 Saved Token not found. Using 20118706661854036583649958139769313744
Now.. run the test!
$ python test_bug_simple.py
Traceback (most recent call last):
File "test_bug_simple.py", line 36, in <module>
result = client.get_range_slice(ks, parent, predicate, "", "", 1000, ConsistencyLevel.ONE)
File "/usr/local/python//lib/python2.6/site-packages/cassandra/Cassandra.py", line 486, in get_range_slice
File "/usr/local/python//lib/python2.6/site-packages/cassandra/Cassandra.py", line 508, in recv_get_range_slice
thrift.Thrift.TApplicationException: Internal error processing get_range_slice
The log from the node I am querying:
INFO 21:02:08,768 Saved Token not found. Using 13635104552356370392932048547451
1375137
INFO 21:02:08,933 Starting up server gossip
INFO 21:02:09,118 Cassandra starting up...
INFO 21:02:10,686 Node /10.212.230.176 is now part of the cluster
INFO 21:02:11,750 InetAddress /10.212.230.176 is now UP
INFO 21:05:43,927 scanning node range (136351045523563703929320485474511375137,
20118706661854036583649958139769313744]
ERROR 21:05:43,927 Internal error processing get_range_slice
java.lang.AssertionError
at org.apache.cassandra.dht.Bounds.<init>(Bounds.java:16)
at org.apache.cassandra.dht.Bounds.restrictTo(Bounds.java:34)
at org.apache.cassandra.service.StorageProxy.getRangeSlice(StorageProxy.
java:559)
at org.apache.cassandra.thrift.CassandraServer.get_range_slice(Cassandra
Server.java:560)
at org.apache.cassandra.thrift.Cassandra$Processor$get_range_slice.proce
ss(Cassandra.java:1189)
at org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.jav
a:984)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadP
oolServer.java:253)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec
utor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:908)
at java.lang.Thread.run(Thread.java:619)
> in a cluster, get_range_slice() does not return all the keys it should
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Environment: Debian 5 lenny on EC2, Gentoo linux, Windows XP
> Reporter: bjc
> Assignee: Jonathan Ellis
> Fix For: 0.5, 0.6
>
> Attachments: 0001-fix-timeout-bug.txt, 0002-fix-slices-over-non-trivial-wrapped-ranges.txt
>
>
> get_range_slice() does not return the same set of keys as get_key_range() in 0.5.0 final.
> I posted a program to reproduce the behavior:
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01474.html
> Apparently, you must have more than one node to get the behavior. Also, it may depend on the locations of the nodes on the ring.. I.e., if you don't generate enough keys randomly, then by chance they could all fall on the same host and you might not see the behavior, although I was able to get it to happen using only 2 nodes and 10 keys.
> Here are the other emails describing the issue:
> http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg02423.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.