You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Eric van Orsouw <er...@eventis.nl> on 2010/11/10 13:08:17 UTC
iteration does not yield all data with consistency ONE
Hello,
We have a cluster of 4 nodes (0.6.6) and use the random partitioner and a replication of 2.
When I insert a number of rows I can always retrieve them by their explicit id (get_range_slices("<key>","", 1).
Playing with consistency levels and temporarily shutting down a Cassandra node all yields the expected result.
However when I use get_range_slices("","", n) to iterate over all rows, I sometimes don't get anything (depending on the node).
I then reduced the problem to inserting just a single row.
Specifically, the 'iteration' only seems to succeed when I issue the request to the node that contains the first copy.
I Discovered that when I iterate using a consistency level of Quorum/All the iteration always succeeds and I properly get the one row.
So a solution would be to always use consistency level One/All but that has a performance penalty.
Can anyone explain why iterating using get_range_slices("","",n) does not always function with consistency level One on all nodes?
Thanks,
Eric
P.S. To rule out any discussion on whether or not to use iteration in the first place, we only plan to use it for backup and periodic cleanup cycles.
Re: iteration does not yield all data with consistency ONE
Posted by Jonathan Ellis <jb...@gmail.com>.
Great. Thanks for helping us track that down!
On Mon, Nov 15, 2010 at 6:45 AM, Eric van Orsouw
<er...@eventis.nl> wrote:
> Jonathan, I have downloaded 0.6.8 and the iteration problem is fixed.
> Thanks,
> Eric
>
> -----Original Message-----
> From: Jonathan Ellis [mailto:jbellis@gmail.com]
> Sent: donderdag 11 november 2010 23:05
> To: user
> Subject: Re: iteration does not yield all data with consistency ONE
>
> Can you try with the latest 0.6 svn branch?
> https://svn.apache.org/repos/asf/cassandra/branches/cassandra-0.6 It
> may be caused by https://issues.apache.org/jira/browse/CASSANDRA-1722
> which is fixed there.
>
> On Thu, Nov 11, 2010 at 4:56 AM, Eric van Orsouw
> <er...@eventis.nl> wrote:
>> I have tried it with RF=1 and 2 nodes and the result was the same.
>> In this case I am not able to query anything, regardless the consistency level.
>>
>> I have conducted some more tests with 4 nodes and RF=2 and before each test I also completely cleared all datafiles (data/commit/saved_cache).
>> Each time I do this the nodes get different locations on the ring and the noticed behavior is different (but similar) each time.
>> I have added the details for 2 concrete scenario;
>>
>> ----
>> 1) clear all 3 data directories for all nodes
>> 2) startup all nodes; the ring is;
>>
>> # bin/nodetool -h 172.16.21.12 ring
>> Address Status Load Range Ring
>> 81296677393575658129031223607666487650
>> 172.16.21.12 Up 489 bytes 61644956546475371096781179489550878842 |<--|
>> 172.16.21.14 Up 489 bytes 63811055530446251457866768401307252614 | |
>> 172.16.21.13 Up 489 bytes 71622328744148061175473750202165524046 | |
>> 172.16.21.11 Up 489 bytes 81296677393575658129031223607666487650 |-->|
>>
>> 3) I use the following ColumnFamily <ColumnFamily Name="Customers" ColumnType="Super" CompareWith="AsciiType"/>
>> And add the following rows (key and values utf8 strings)
>> Customer['1'].simple.id='1'
>> Customer['2'].simple.id='2'
>> Customer['3'].simple.id='3'
>> Customer['4'].simple.id='4'
>>
>> 4) Iterating over the columns now yield;
>> Iterating Nodes .11 thru.13 ==> all rows, regardless of consistency level
>> Iterating Node.14 ==> only row '3' and '4' with consistency level ONE, all rows with consistency level All
>>
>> ----
>> Repating all process steps again yields the following ring and iteration results
>> # bin/nodetool -h 172.16.21.11 ring
>> Address Status Load Range Ring
>> 157208721428268855331814529932509412075
>> 172.16.21.11 Up 489 bytes 18662665007498667259581716616851966032 |<--|
>> 172.16.21.13 Up 489 bytes 125997322478065775200568923383951796531 | |
>> 172.16.21.12 Up 489 bytes 148125235552363592916130178190645177045 | |
>> 172.16.21.14 Up 489 bytes 157208721428268855331814529932509412075 |-->|
>>
>> Iterating Nodes .11 .12 .14 ==> no data with consistency level One, all rows with consistency level All
>> Iterating Node .13 ==> only row '3' and '4' with consistency level ONE, all rows with consistency level All
>>
>> I appears to me there is an inconsistency somewhere, I hope this info helps.
>>
>> Eric
>>
>>
>> -----Original Message-----
>> From: Jonathan Ellis [mailto:jbellis@gmail.com]
>> Sent: woensdag 10 november 2010 17:48
>> To: user
>> Subject: Re: iteration does not yield all data with consistency ONE
>>
>> Interesting. Does it simplify further to RF=1 and 2 nodes?
>>
>> On Wed, Nov 10, 2010 at 8:58 AM, Eric van Orsouw
>> <er...@eventis.nl> wrote:
>>> No, all nodes were up and running while the single key was inserted.
>>> The insert however was with consistency One. I assume however that the replicas are still written in this case.
>>> It is btw also very reproducible.
>>>
>>> -----Original Message-----
>>> From: Jonathan Ellis [mailto:jbellis@gmail.com]
>>> Sent: woensdag 10 november 2010 15:44
>>> To: user
>>> Subject: Re: iteration does not yield all data with consistency ONE
>>>
>>> Was the node that should have the other replica of this row down when
>>> it was inserted?
>>>
>>> On Wed, Nov 10, 2010 at 6:08 AM, Eric van Orsouw
>>> <er...@eventis.nl> wrote:
>>>>
>>>> Hello,
>>>>
>>>>
>>>>
>>>> We have a cluster of 4 nodes (0.6.6) and use the random partitioner and a replication of 2.
>>>>
>>>> When I insert a number of rows I can always retrieve them by their explicit id (get_range_slices("<key>","", 1).
>>>>
>>>> Playing with consistency levels and temporarily shutting down a Cassandra node all yields the expected result.
>>>>
>>>>
>>>>
>>>> However when I use get_range_slices("","", n) to iterate over all rows, I sometimes don't get anything (depending on the node).
>>>>
>>>>
>>>>
>>>> I then reduced the problem to inserting just a single row.
>>>>
>>>> Specifically, the 'iteration' only seems to succeed when I issue the request to the node that contains the first copy.
>>>>
>>>> I Discovered that when I iterate using a consistency level of Quorum/All the iteration always succeeds and I properly get the one row.
>>>>
>>>>
>>>>
>>>> So a solution would be to always use consistency level One/All but that has a performance penalty.
>>>>
>>>>
>>>>
>>>> Can anyone explain why iterating using get_range_slices("","",n) does not always function with consistency level One on all nodes?
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Eric
>>>>
>>>>
>>>>
>>>> P.S. To rule out any discussion on whether or not to use iteration in the first place, we only plan to use it for backup and periodic cleanup cycles.
>>>
>>>
>>> --
>>> Jonathan Ellis
>>> Project Chair, Apache Cassandra
>>> co-founder of Riptano, the source for professional Cassandra support
>>> http://riptano.com
>>>
>>
>>
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of Riptano, the source for professional Cassandra support
>> http://riptano.com
>>
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of Riptano, the source for professional Cassandra support
> http://riptano.com
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com
RE: iteration does not yield all data with consistency ONE
Posted by Eric van Orsouw <er...@eventis.nl>.
Jonathan, I have downloaded 0.6.8 and the iteration problem is fixed.
Thanks,
Eric
-----Original Message-----
From: Jonathan Ellis [mailto:jbellis@gmail.com]
Sent: donderdag 11 november 2010 23:05
To: user
Subject: Re: iteration does not yield all data with consistency ONE
Can you try with the latest 0.6 svn branch?
https://svn.apache.org/repos/asf/cassandra/branches/cassandra-0.6 It
may be caused by https://issues.apache.org/jira/browse/CASSANDRA-1722
which is fixed there.
On Thu, Nov 11, 2010 at 4:56 AM, Eric van Orsouw
<er...@eventis.nl> wrote:
> I have tried it with RF=1 and 2 nodes and the result was the same.
> In this case I am not able to query anything, regardless the consistency level.
>
> I have conducted some more tests with 4 nodes and RF=2 and before each test I also completely cleared all datafiles (data/commit/saved_cache).
> Each time I do this the nodes get different locations on the ring and the noticed behavior is different (but similar) each time.
> I have added the details for 2 concrete scenario;
>
> ----
> 1) clear all 3 data directories for all nodes
> 2) startup all nodes; the ring is;
>
> # bin/nodetool -h 172.16.21.12 ring
> Address Status Load Range Ring
> 81296677393575658129031223607666487650
> 172.16.21.12 Up 489 bytes 61644956546475371096781179489550878842 |<--|
> 172.16.21.14 Up 489 bytes 63811055530446251457866768401307252614 | |
> 172.16.21.13 Up 489 bytes 71622328744148061175473750202165524046 | |
> 172.16.21.11 Up 489 bytes 81296677393575658129031223607666487650 |-->|
>
> 3) I use the following ColumnFamily <ColumnFamily Name="Customers" ColumnType="Super" CompareWith="AsciiType"/>
> And add the following rows (key and values utf8 strings)
> Customer['1'].simple.id='1'
> Customer['2'].simple.id='2'
> Customer['3'].simple.id='3'
> Customer['4'].simple.id='4'
>
> 4) Iterating over the columns now yield;
> Iterating Nodes .11 thru.13 ==> all rows, regardless of consistency level
> Iterating Node.14 ==> only row '3' and '4' with consistency level ONE, all rows with consistency level All
>
> ----
> Repating all process steps again yields the following ring and iteration results
> # bin/nodetool -h 172.16.21.11 ring
> Address Status Load Range Ring
> 157208721428268855331814529932509412075
> 172.16.21.11 Up 489 bytes 18662665007498667259581716616851966032 |<--|
> 172.16.21.13 Up 489 bytes 125997322478065775200568923383951796531 | |
> 172.16.21.12 Up 489 bytes 148125235552363592916130178190645177045 | |
> 172.16.21.14 Up 489 bytes 157208721428268855331814529932509412075 |-->|
>
> Iterating Nodes .11 .12 .14 ==> no data with consistency level One, all rows with consistency level All
> Iterating Node .13 ==> only row '3' and '4' with consistency level ONE, all rows with consistency level All
>
> I appears to me there is an inconsistency somewhere, I hope this info helps.
>
> Eric
>
>
> -----Original Message-----
> From: Jonathan Ellis [mailto:jbellis@gmail.com]
> Sent: woensdag 10 november 2010 17:48
> To: user
> Subject: Re: iteration does not yield all data with consistency ONE
>
> Interesting. Does it simplify further to RF=1 and 2 nodes?
>
> On Wed, Nov 10, 2010 at 8:58 AM, Eric van Orsouw
> <er...@eventis.nl> wrote:
>> No, all nodes were up and running while the single key was inserted.
>> The insert however was with consistency One. I assume however that the replicas are still written in this case.
>> It is btw also very reproducible.
>>
>> -----Original Message-----
>> From: Jonathan Ellis [mailto:jbellis@gmail.com]
>> Sent: woensdag 10 november 2010 15:44
>> To: user
>> Subject: Re: iteration does not yield all data with consistency ONE
>>
>> Was the node that should have the other replica of this row down when
>> it was inserted?
>>
>> On Wed, Nov 10, 2010 at 6:08 AM, Eric van Orsouw
>> <er...@eventis.nl> wrote:
>>>
>>> Hello,
>>>
>>>
>>>
>>> We have a cluster of 4 nodes (0.6.6) and use the random partitioner and a replication of 2.
>>>
>>> When I insert a number of rows I can always retrieve them by their explicit id (get_range_slices("<key>","", 1).
>>>
>>> Playing with consistency levels and temporarily shutting down a Cassandra node all yields the expected result.
>>>
>>>
>>>
>>> However when I use get_range_slices("","", n) to iterate over all rows, I sometimes don't get anything (depending on the node).
>>>
>>>
>>>
>>> I then reduced the problem to inserting just a single row.
>>>
>>> Specifically, the 'iteration' only seems to succeed when I issue the request to the node that contains the first copy.
>>>
>>> I Discovered that when I iterate using a consistency level of Quorum/All the iteration always succeeds and I properly get the one row.
>>>
>>>
>>>
>>> So a solution would be to always use consistency level One/All but that has a performance penalty.
>>>
>>>
>>>
>>> Can anyone explain why iterating using get_range_slices("","",n) does not always function with consistency level One on all nodes?
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Eric
>>>
>>>
>>>
>>> P.S. To rule out any discussion on whether or not to use iteration in the first place, we only plan to use it for backup and periodic cleanup cycles.
>>
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of Riptano, the source for professional Cassandra support
>> http://riptano.com
>>
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of Riptano, the source for professional Cassandra support
> http://riptano.com
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com
Re: iteration does not yield all data with consistency ONE
Posted by Jonathan Ellis <jb...@gmail.com>.
Can you try with the latest 0.6 svn branch?
https://svn.apache.org/repos/asf/cassandra/branches/cassandra-0.6 It
may be caused by https://issues.apache.org/jira/browse/CASSANDRA-1722
which is fixed there.
On Thu, Nov 11, 2010 at 4:56 AM, Eric van Orsouw
<er...@eventis.nl> wrote:
> I have tried it with RF=1 and 2 nodes and the result was the same.
> In this case I am not able to query anything, regardless the consistency level.
>
> I have conducted some more tests with 4 nodes and RF=2 and before each test I also completely cleared all datafiles (data/commit/saved_cache).
> Each time I do this the nodes get different locations on the ring and the noticed behavior is different (but similar) each time.
> I have added the details for 2 concrete scenario;
>
> ----
> 1) clear all 3 data directories for all nodes
> 2) startup all nodes; the ring is;
>
> # bin/nodetool -h 172.16.21.12 ring
> Address Status Load Range Ring
> 81296677393575658129031223607666487650
> 172.16.21.12 Up 489 bytes 61644956546475371096781179489550878842 |<--|
> 172.16.21.14 Up 489 bytes 63811055530446251457866768401307252614 | |
> 172.16.21.13 Up 489 bytes 71622328744148061175473750202165524046 | |
> 172.16.21.11 Up 489 bytes 81296677393575658129031223607666487650 |-->|
>
> 3) I use the following ColumnFamily <ColumnFamily Name="Customers" ColumnType="Super" CompareWith="AsciiType"/>
> And add the following rows (key and values utf8 strings)
> Customer['1'].simple.id='1'
> Customer['2'].simple.id='2'
> Customer['3'].simple.id='3'
> Customer['4'].simple.id='4'
>
> 4) Iterating over the columns now yield;
> Iterating Nodes .11 thru.13 ==> all rows, regardless of consistency level
> Iterating Node.14 ==> only row '3' and '4' with consistency level ONE, all rows with consistency level All
>
> ----
> Repating all process steps again yields the following ring and iteration results
> # bin/nodetool -h 172.16.21.11 ring
> Address Status Load Range Ring
> 157208721428268855331814529932509412075
> 172.16.21.11 Up 489 bytes 18662665007498667259581716616851966032 |<--|
> 172.16.21.13 Up 489 bytes 125997322478065775200568923383951796531 | |
> 172.16.21.12 Up 489 bytes 148125235552363592916130178190645177045 | |
> 172.16.21.14 Up 489 bytes 157208721428268855331814529932509412075 |-->|
>
> Iterating Nodes .11 .12 .14 ==> no data with consistency level One, all rows with consistency level All
> Iterating Node .13 ==> only row '3' and '4' with consistency level ONE, all rows with consistency level All
>
> I appears to me there is an inconsistency somewhere, I hope this info helps.
>
> Eric
>
>
> -----Original Message-----
> From: Jonathan Ellis [mailto:jbellis@gmail.com]
> Sent: woensdag 10 november 2010 17:48
> To: user
> Subject: Re: iteration does not yield all data with consistency ONE
>
> Interesting. Does it simplify further to RF=1 and 2 nodes?
>
> On Wed, Nov 10, 2010 at 8:58 AM, Eric van Orsouw
> <er...@eventis.nl> wrote:
>> No, all nodes were up and running while the single key was inserted.
>> The insert however was with consistency One. I assume however that the replicas are still written in this case.
>> It is btw also very reproducible.
>>
>> -----Original Message-----
>> From: Jonathan Ellis [mailto:jbellis@gmail.com]
>> Sent: woensdag 10 november 2010 15:44
>> To: user
>> Subject: Re: iteration does not yield all data with consistency ONE
>>
>> Was the node that should have the other replica of this row down when
>> it was inserted?
>>
>> On Wed, Nov 10, 2010 at 6:08 AM, Eric van Orsouw
>> <er...@eventis.nl> wrote:
>>>
>>> Hello,
>>>
>>>
>>>
>>> We have a cluster of 4 nodes (0.6.6) and use the random partitioner and a replication of 2.
>>>
>>> When I insert a number of rows I can always retrieve them by their explicit id (get_range_slices("<key>","", 1).
>>>
>>> Playing with consistency levels and temporarily shutting down a Cassandra node all yields the expected result.
>>>
>>>
>>>
>>> However when I use get_range_slices("","", n) to iterate over all rows, I sometimes don't get anything (depending on the node).
>>>
>>>
>>>
>>> I then reduced the problem to inserting just a single row.
>>>
>>> Specifically, the 'iteration' only seems to succeed when I issue the request to the node that contains the first copy.
>>>
>>> I Discovered that when I iterate using a consistency level of Quorum/All the iteration always succeeds and I properly get the one row.
>>>
>>>
>>>
>>> So a solution would be to always use consistency level One/All but that has a performance penalty.
>>>
>>>
>>>
>>> Can anyone explain why iterating using get_range_slices("","",n) does not always function with consistency level One on all nodes?
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Eric
>>>
>>>
>>>
>>> P.S. To rule out any discussion on whether or not to use iteration in the first place, we only plan to use it for backup and periodic cleanup cycles.
>>
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of Riptano, the source for professional Cassandra support
>> http://riptano.com
>>
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of Riptano, the source for professional Cassandra support
> http://riptano.com
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com
Re: iteration does not yield all data with consistency ONE
Posted by Brandon Williams <dr...@gmail.com>.
On Thu, Nov 11, 2010 at 4:56 AM, Eric van Orsouw <eric.van.orsouw@eventis.nl
> wrote:
> I appears to me there is an inconsistency somewhere, I hope this info
> helps.
get_range_slices does not perform read repair, so inconsistencies at cl.ONE
won't be resolved. Invoke nodetool repair everywhere and try again.
-Brandon
RE: iteration does not yield all data with consistency ONE
Posted by Eric van Orsouw <er...@eventis.nl>.
I have tried it with RF=1 and 2 nodes and the result was the same.
In this case I am not able to query anything, regardless the consistency level.
I have conducted some more tests with 4 nodes and RF=2 and before each test I also completely cleared all datafiles (data/commit/saved_cache).
Each time I do this the nodes get different locations on the ring and the noticed behavior is different (but similar) each time.
I have added the details for 2 concrete scenario;
----
1) clear all 3 data directories for all nodes
2) startup all nodes; the ring is;
# bin/nodetool -h 172.16.21.12 ring
Address Status Load Range Ring
81296677393575658129031223607666487650
172.16.21.12 Up 489 bytes 61644956546475371096781179489550878842 |<--|
172.16.21.14 Up 489 bytes 63811055530446251457866768401307252614 | |
172.16.21.13 Up 489 bytes 71622328744148061175473750202165524046 | |
172.16.21.11 Up 489 bytes 81296677393575658129031223607666487650 |-->|
3) I use the following ColumnFamily <ColumnFamily Name="Customers" ColumnType="Super" CompareWith="AsciiType"/>
And add the following rows (key and values utf8 strings)
Customer['1'].simple.id='1'
Customer['2'].simple.id='2'
Customer['3'].simple.id='3'
Customer['4'].simple.id='4'
4) Iterating over the columns now yield;
Iterating Nodes .11 thru.13 ==> all rows, regardless of consistency level
Iterating Node.14 ==> only row '3' and '4' with consistency level ONE, all rows with consistency level All
----
Repating all process steps again yields the following ring and iteration results
# bin/nodetool -h 172.16.21.11 ring
Address Status Load Range Ring
157208721428268855331814529932509412075
172.16.21.11 Up 489 bytes 18662665007498667259581716616851966032 |<--|
172.16.21.13 Up 489 bytes 125997322478065775200568923383951796531 | |
172.16.21.12 Up 489 bytes 148125235552363592916130178190645177045 | |
172.16.21.14 Up 489 bytes 157208721428268855331814529932509412075 |-->|
Iterating Nodes .11 .12 .14 ==> no data with consistency level One, all rows with consistency level All
Iterating Node .13 ==> only row '3' and '4' with consistency level ONE, all rows with consistency level All
I appears to me there is an inconsistency somewhere, I hope this info helps.
Eric
-----Original Message-----
From: Jonathan Ellis [mailto:jbellis@gmail.com]
Sent: woensdag 10 november 2010 17:48
To: user
Subject: Re: iteration does not yield all data with consistency ONE
Interesting. Does it simplify further to RF=1 and 2 nodes?
On Wed, Nov 10, 2010 at 8:58 AM, Eric van Orsouw
<er...@eventis.nl> wrote:
> No, all nodes were up and running while the single key was inserted.
> The insert however was with consistency One. I assume however that the replicas are still written in this case.
> It is btw also very reproducible.
>
> -----Original Message-----
> From: Jonathan Ellis [mailto:jbellis@gmail.com]
> Sent: woensdag 10 november 2010 15:44
> To: user
> Subject: Re: iteration does not yield all data with consistency ONE
>
> Was the node that should have the other replica of this row down when
> it was inserted?
>
> On Wed, Nov 10, 2010 at 6:08 AM, Eric van Orsouw
> <er...@eventis.nl> wrote:
>>
>> Hello,
>>
>>
>>
>> We have a cluster of 4 nodes (0.6.6) and use the random partitioner and a replication of 2.
>>
>> When I insert a number of rows I can always retrieve them by their explicit id (get_range_slices("<key>","", 1).
>>
>> Playing with consistency levels and temporarily shutting down a Cassandra node all yields the expected result.
>>
>>
>>
>> However when I use get_range_slices("","", n) to iterate over all rows, I sometimes don't get anything (depending on the node).
>>
>>
>>
>> I then reduced the problem to inserting just a single row.
>>
>> Specifically, the 'iteration' only seems to succeed when I issue the request to the node that contains the first copy.
>>
>> I Discovered that when I iterate using a consistency level of Quorum/All the iteration always succeeds and I properly get the one row.
>>
>>
>>
>> So a solution would be to always use consistency level One/All but that has a performance penalty.
>>
>>
>>
>> Can anyone explain why iterating using get_range_slices("","",n) does not always function with consistency level One on all nodes?
>>
>>
>>
>> Thanks,
>>
>> Eric
>>
>>
>>
>> P.S. To rule out any discussion on whether or not to use iteration in the first place, we only plan to use it for backup and periodic cleanup cycles.
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of Riptano, the source for professional Cassandra support
> http://riptano.com
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com
Re: iteration does not yield all data with consistency ONE
Posted by Jonathan Ellis <jb...@gmail.com>.
Interesting. Does it simplify further to RF=1 and 2 nodes?
On Wed, Nov 10, 2010 at 8:58 AM, Eric van Orsouw
<er...@eventis.nl> wrote:
> No, all nodes were up and running while the single key was inserted.
> The insert however was with consistency One. I assume however that the replicas are still written in this case.
> It is btw also very reproducible.
>
> -----Original Message-----
> From: Jonathan Ellis [mailto:jbellis@gmail.com]
> Sent: woensdag 10 november 2010 15:44
> To: user
> Subject: Re: iteration does not yield all data with consistency ONE
>
> Was the node that should have the other replica of this row down when
> it was inserted?
>
> On Wed, Nov 10, 2010 at 6:08 AM, Eric van Orsouw
> <er...@eventis.nl> wrote:
>>
>> Hello,
>>
>>
>>
>> We have a cluster of 4 nodes (0.6.6) and use the random partitioner and a replication of 2.
>>
>> When I insert a number of rows I can always retrieve them by their explicit id (get_range_slices("<key>","", 1).
>>
>> Playing with consistency levels and temporarily shutting down a Cassandra node all yields the expected result.
>>
>>
>>
>> However when I use get_range_slices("","", n) to iterate over all rows, I sometimes don't get anything (depending on the node).
>>
>>
>>
>> I then reduced the problem to inserting just a single row.
>>
>> Specifically, the 'iteration' only seems to succeed when I issue the request to the node that contains the first copy.
>>
>> I Discovered that when I iterate using a consistency level of Quorum/All the iteration always succeeds and I properly get the one row.
>>
>>
>>
>> So a solution would be to always use consistency level One/All but that has a performance penalty.
>>
>>
>>
>> Can anyone explain why iterating using get_range_slices("","",n) does not always function with consistency level One on all nodes?
>>
>>
>>
>> Thanks,
>>
>> Eric
>>
>>
>>
>> P.S. To rule out any discussion on whether or not to use iteration in the first place, we only plan to use it for backup and periodic cleanup cycles.
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of Riptano, the source for professional Cassandra support
> http://riptano.com
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com
RE: iteration does not yield all data with consistency ONE
Posted by Eric van Orsouw <er...@eventis.nl>.
No, all nodes were up and running while the single key was inserted.
The insert however was with consistency One. I assume however that the replicas are still written in this case.
It is btw also very reproducible.
-----Original Message-----
From: Jonathan Ellis [mailto:jbellis@gmail.com]
Sent: woensdag 10 november 2010 15:44
To: user
Subject: Re: iteration does not yield all data with consistency ONE
Was the node that should have the other replica of this row down when
it was inserted?
On Wed, Nov 10, 2010 at 6:08 AM, Eric van Orsouw
<er...@eventis.nl> wrote:
>
> Hello,
>
>
>
> We have a cluster of 4 nodes (0.6.6) and use the random partitioner and a replication of 2.
>
> When I insert a number of rows I can always retrieve them by their explicit id (get_range_slices("<key>","", 1).
>
> Playing with consistency levels and temporarily shutting down a Cassandra node all yields the expected result.
>
>
>
> However when I use get_range_slices("","", n) to iterate over all rows, I sometimes don't get anything (depending on the node).
>
>
>
> I then reduced the problem to inserting just a single row.
>
> Specifically, the 'iteration' only seems to succeed when I issue the request to the node that contains the first copy.
>
> I Discovered that when I iterate using a consistency level of Quorum/All the iteration always succeeds and I properly get the one row.
>
>
>
> So a solution would be to always use consistency level One/All but that has a performance penalty.
>
>
>
> Can anyone explain why iterating using get_range_slices("","",n) does not always function with consistency level One on all nodes?
>
>
>
> Thanks,
>
> Eric
>
>
>
> P.S. To rule out any discussion on whether or not to use iteration in the first place, we only plan to use it for backup and periodic cleanup cycles.
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com
Re: iteration does not yield all data with consistency ONE
Posted by Jonathan Ellis <jb...@gmail.com>.
Was the node that should have the other replica of this row down when
it was inserted?
On Wed, Nov 10, 2010 at 6:08 AM, Eric van Orsouw
<er...@eventis.nl> wrote:
>
> Hello,
>
>
>
> We have a cluster of 4 nodes (0.6.6) and use the random partitioner and a replication of 2.
>
> When I insert a number of rows I can always retrieve them by their explicit id (get_range_slices(“<key>”,””, 1).
>
> Playing with consistency levels and temporarily shutting down a Cassandra node all yields the expected result.
>
>
>
> However when I use get_range_slices(“”,””, n) to iterate over all rows, I sometimes don’t get anything (depending on the node).
>
>
>
> I then reduced the problem to inserting just a single row.
>
> Specifically, the ‘iteration’ only seems to succeed when I issue the request to the node that contains the first copy.
>
> I Discovered that when I iterate using a consistency level of Quorum/All the iteration always succeeds and I properly get the one row.
>
>
>
> So a solution would be to always use consistency level One/All but that has a performance penalty.
>
>
>
> Can anyone explain why iterating using get_range_slices(“”,””,n) does not always function with consistency level One on all nodes?
>
>
>
> Thanks,
>
> Eric
>
>
>
> P.S. To rule out any discussion on whether or not to use iteration in the first place, we only plan to use it for backup and periodic cleanup cycles.
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com