You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Cathy Daw (JIRA)" <ji...@apache.org> on 2011/07/26 22:58:09 UTC
[jira] [Created] (CASSANDRA-2950) Data from truncated Counter CF
reappears after server restart
Data from truncated Counter CF reappears after server restart
-------------------------------------------------------------
Key: CASSANDRA-2950
URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
Project: Cassandra
Issue Type: Bug
Reporter: Cathy Daw
* Configure 3 node cluster
* Ensure the java stress tool creates Keyspace1 with RF=3
{code}
// Run Stress Tool to generate 10 keys, 1 column
stress --operation=COUNTER_ADD --family-type=Standard --num-keys=1000 --num-different-keys=10 --columns=1 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
// Verify 10 keys in CLI
use Keyspace1;
list Counter1;
//TRUNCATE CF in CLI
use Keyspace1;
truncate counter1;
list counter1;
// Run stress tool and verify creation of 1 key with 1 column valued @ 1000
stress --operation=COUNTER_ADD --family-type=Standard --num-keys=1000 --num-different-keys=1 --columns=1 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
// Run stress tool and verify update of existing key -- Final result is 2 columns valued at 1500, 500.
stress --operation=COUNTER_ADD --family-type=Standard --num-keys=500 --num-different-keys=1 --columns=2 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
// Run stress tool and verify update of existing key -- Final result is 3 columns valued at 2100, 1100, 600.
stress --operation=COUNTER_ADD --family-type=Standard --num-keys=600 --num-different-keys=1 --columns=3 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
{code}
*Data while all three nodes are up*
{code}
[default@Keyspace1] list Counter1;
Using default limit of 100
-------------------
RowKey: 30
=> (counter=4330, value=2100)
=> (counter=4331, value=1100)
=> (counter=4332, value=600)
{code}
* Shutdown nodes 1,2,3
* Startup nodes 1,2,3
* Verify in CLI: 11 keys. I am expecting only 1.
*Data after bouncing nodes*
{code}
[default@Keyspace1] list Counter1;
Using default limit of 100
-------------------
RowKey: 3036
=> (counter=4330, value=500597)
-------------------
RowKey: 3038
=> (counter=4330, value=500591)
-------------------
RowKey: 3039
=> (counter=4330, value=500609)
-------------------
RowKey: 3033
=> (counter=4330, value=500607)
-------------------
RowKey: 3037
=> (counter=4330, value=500601)
-------------------
RowKey: 30
=> (counter=4330, value=2708611)
=> (counter=4331, value=606482)
=> (counter=4332, value=180798)
-------------------
RowKey: 3030
=> (counter=4330, value=500616)
-------------------
RowKey: 3032
=> (counter=4330, value=500596)
-------------------
RowKey: 3031
=> (counter=4330, value=500613)
-------------------
RowKey: 3035
=> (counter=4330, value=500624)
-------------------
RowKey: 3034
=> (counter=4330, value=500618)
11 Rows Returned.
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2950) Data from truncated Counter CF
reappears after server restart
Posted by "Cathy Daw (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13071475#comment-13071475 ]
Cathy Daw commented on CASSANDRA-2950:
--------------------------------------
This is a general issue with all CF's. updating bug.
> Data from truncated Counter CF reappears after server restart
> -------------------------------------------------------------
>
> Key: CASSANDRA-2950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
> Project: Cassandra
> Issue Type: Bug
> Reporter: Cathy Daw
> Assignee: Sylvain Lebresne
>
> * Configure 3 node cluster
> * Ensure the java stress tool creates Keyspace1 with RF=3
> {code}
> // Run Stress Tool to generate 10 keys, 1 column
> stress --operation=COUNTER_ADD --family-type=Standard --num-keys=1000 --num-different-keys=10 --columns=1 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
> // Verify 10 keys in CLI
> use Keyspace1;
> list Counter1;
> //TRUNCATE CF in CLI
> use Keyspace1;
> truncate counter1;
> list counter1;
> // Run stress tool and verify creation of 1 key with 1 column valued @ 1000
> stress --operation=COUNTER_ADD --family-type=Standard --num-keys=1000 --num-different-keys=1 --columns=1 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
> // Run stress tool and verify update of existing key -- Final result is 2 columns valued at 1500, 500.
> stress --operation=COUNTER_ADD --family-type=Standard --num-keys=500 --num-different-keys=1 --columns=2 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
> // Run stress tool and verify update of existing key -- Final result is 3 columns valued at 2100, 1100, 600.
> stress --operation=COUNTER_ADD --family-type=Standard --num-keys=600 --num-different-keys=1 --columns=3 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
> {code}
> *Data while all three nodes are up*
> {code}
> [default@Keyspace1] list Counter1;
> Using default limit of 100
> -------------------
> RowKey: 30
> => (counter=4330, value=2100)
> => (counter=4331, value=1100)
> => (counter=4332, value=600)
> {code}
> * Shutdown nodes 1,2,3
> * Startup nodes 1,2,3
> * Verify in CLI: 11 keys. I am expecting only 1.
> *Data after bouncing nodes*
> {code}
> [default@Keyspace1] list Counter1;
> Using default limit of 100
> -------------------
> RowKey: 3036
> => (counter=4330, value=500597)
> -------------------
> RowKey: 3038
> => (counter=4330, value=500591)
> -------------------
> RowKey: 3039
> => (counter=4330, value=500609)
> -------------------
> RowKey: 3033
> => (counter=4330, value=500607)
> -------------------
> RowKey: 3037
> => (counter=4330, value=500601)
> -------------------
> RowKey: 30
> => (counter=4330, value=2708611)
> => (counter=4331, value=606482)
> => (counter=4332, value=180798)
> -------------------
> RowKey: 3030
> => (counter=4330, value=500616)
> -------------------
> RowKey: 3032
> => (counter=4330, value=500596)
> -------------------
> RowKey: 3031
> => (counter=4330, value=500613)
> -------------------
> RowKey: 3035
> => (counter=4330, value=500624)
> -------------------
> RowKey: 3034
> => (counter=4330, value=500618)
> 11 Rows Returned.
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2950) Data from truncated CF reappears
after server restart
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-2950:
--------------------------------------
Attachment: 2950-v2.txt
v2:
{noformat}
// Bonus bonus: simply forceFlush of all the CF is not enough, because if
// for a given column family the memtable is clean, forceFlush will return
// immediately, even though there could be a memtable being flush at the same
// time. So to guarantee that all segments can be cleaned out, we need
// "waitForActiveFlushes" after the new segment has been created.
{noformat}
> Data from truncated CF reappears after server restart
> -----------------------------------------------------
>
> Key: CASSANDRA-2950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Cathy Daw
> Assignee: Jonathan Ellis
> Fix For: 0.8.5
>
> Attachments: 2950-v2.txt, 2950.txt
>
>
> * Configure 3 node cluster
> * Ensure the java stress tool creates Keyspace1 with RF=3
> {code}
> // Run Stress Tool to generate 10 keys, 1 column
> stress --operation=INSERT -t 2 --num-keys=50 --columns=20 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 50 keys in CLI
> use Keyspace1;
> list Standard1;
> // TRUNCATE CF in CLI
> use Keyspace1;
> truncate counter1;
> list counter1;
> // Run stress tool and verify creation of 1 key with 10 columns
> stress --operation=INSERT -t 2 --num-keys=1 --columns=10 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 1 key in CLI
> use Keyspace1;
> list Standard1;
> // Restart all three nodes
> // You will see 51 keys in CLI
> use Keyspace1;
> list Standard1;
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2950) Data from truncated Counter CF
reappears after server restart
Posted by "Cathy Daw (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13071469#comment-13071469 ]
Cathy Daw commented on CASSANDRA-2950:
--------------------------------------
Confirmed this only occurs with Counters. There is a limitation on deleting counters, that they may appear if they haven't been applied to all nodes and compacted away. Not sure if this falls under the same limitation, since in a traditional RDBMS, the semantics for truncate are different from delete.
> Data from truncated Counter CF reappears after server restart
> -------------------------------------------------------------
>
> Key: CASSANDRA-2950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
> Project: Cassandra
> Issue Type: Bug
> Reporter: Cathy Daw
>
> * Configure 3 node cluster
> * Ensure the java stress tool creates Keyspace1 with RF=3
> {code}
> // Run Stress Tool to generate 10 keys, 1 column
> stress --operation=COUNTER_ADD --family-type=Standard --num-keys=1000 --num-different-keys=10 --columns=1 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
> // Verify 10 keys in CLI
> use Keyspace1;
> list Counter1;
> //TRUNCATE CF in CLI
> use Keyspace1;
> truncate counter1;
> list counter1;
> // Run stress tool and verify creation of 1 key with 1 column valued @ 1000
> stress --operation=COUNTER_ADD --family-type=Standard --num-keys=1000 --num-different-keys=1 --columns=1 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
> // Run stress tool and verify update of existing key -- Final result is 2 columns valued at 1500, 500.
> stress --operation=COUNTER_ADD --family-type=Standard --num-keys=500 --num-different-keys=1 --columns=2 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
> // Run stress tool and verify update of existing key -- Final result is 3 columns valued at 2100, 1100, 600.
> stress --operation=COUNTER_ADD --family-type=Standard --num-keys=600 --num-different-keys=1 --columns=3 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
> {code}
> *Data while all three nodes are up*
> {code}
> [default@Keyspace1] list Counter1;
> Using default limit of 100
> -------------------
> RowKey: 30
> => (counter=4330, value=2100)
> => (counter=4331, value=1100)
> => (counter=4332, value=600)
> {code}
> * Shutdown nodes 1,2,3
> * Startup nodes 1,2,3
> * Verify in CLI: 11 keys. I am expecting only 1.
> *Data after bouncing nodes*
> {code}
> [default@Keyspace1] list Counter1;
> Using default limit of 100
> -------------------
> RowKey: 3036
> => (counter=4330, value=500597)
> -------------------
> RowKey: 3038
> => (counter=4330, value=500591)
> -------------------
> RowKey: 3039
> => (counter=4330, value=500609)
> -------------------
> RowKey: 3033
> => (counter=4330, value=500607)
> -------------------
> RowKey: 3037
> => (counter=4330, value=500601)
> -------------------
> RowKey: 30
> => (counter=4330, value=2708611)
> => (counter=4331, value=606482)
> => (counter=4332, value=180798)
> -------------------
> RowKey: 3030
> => (counter=4330, value=500616)
> -------------------
> RowKey: 3032
> => (counter=4330, value=500596)
> -------------------
> RowKey: 3031
> => (counter=4330, value=500613)
> -------------------
> RowKey: 3035
> => (counter=4330, value=500624)
> -------------------
> RowKey: 3034
> => (counter=4330, value=500618)
> 11 Rows Returned.
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2950) Data from truncated CF
reappears after server restart
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13083482#comment-13083482 ]
Hudson commented on CASSANDRA-2950:
-----------------------------------
Integrated in Cassandra-0.8 #272 (See [https://builds.apache.org/job/Cassandra-0.8/272/])
make sure truncate clears out the commitlog
patch by jbellis; reviewed by slebresne for CASSANDRA-2950
jbellis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1156763
Files :
* /cassandra/branches/cassandra-0.8/CHANGES.txt
* /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/SystemTable.java
* /cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/db/RecoveryManagerTruncateTest.java
* /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
* /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
> Data from truncated CF reappears after server restart
> -----------------------------------------------------
>
> Key: CASSANDRA-2950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Cathy Daw
> Assignee: Jonathan Ellis
> Fix For: 0.8.5
>
> Attachments: 2950-v2.txt, 2950-v3_0.8.patch, 2950.txt
>
>
> * Configure 3 node cluster
> * Ensure the java stress tool creates Keyspace1 with RF=3
> {code}
> // Run Stress Tool to generate 10 keys, 1 column
> stress --operation=INSERT -t 2 --num-keys=50 --columns=20 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 50 keys in CLI
> use Keyspace1;
> list Standard1;
> // TRUNCATE CF in CLI
> use Keyspace1;
> truncate counter1;
> list counter1;
> // Run stress tool and verify creation of 1 key with 10 columns
> stress --operation=INSERT -t 2 --num-keys=1 --columns=10 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 1 key in CLI
> use Keyspace1;
> list Standard1;
> // Restart all three nodes
> // You will see 51 keys in CLI
> use Keyspace1;
> list Standard1;
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2950) Data from truncated CF reappears
after server restart
Posted by "Cathy Daw (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cathy Daw updated CASSANDRA-2950:
---------------------------------
Description:
* Configure 3 node cluster
* Ensure the java stress tool creates Keyspace1 with RF=3
{code}
// Run Stress Tool to generate 10 keys, 1 column
stress --operation=INSERT -t 2 --num-keys=50 --columns=20 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
// Verify 50 keys in CLI
use Keyspace1;
list Standard1;
// TRUNCATE CF in CLI
use Keyspace1;
truncate counter1;
list counter1;
// Run stress tool and verify creation of 1 key with 10 columns
stress --operation=INSERT -t 2 --num-keys=1 --columns=10 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
// Verify 1 key in CLI
use Keyspace1;
list Standard1;
// Restart all three nodes
// You will see 51 keys in CLI
use Keyspace1;
list Standard1;
{code}
was:
* Configure 3 node cluster
* Ensure the java stress tool creates Keyspace1 with RF=3
{code}
// Run Stress Tool to generate 10 keys, 1 column
stress --operation=COUNTER_ADD --family-type=Standard --num-keys=1000 --num-different-keys=10 --columns=1 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
// Verify 10 keys in CLI
use Keyspace1;
list Counter1;
//TRUNCATE CF in CLI
use Keyspace1;
truncate counter1;
list counter1;
// Run stress tool and verify creation of 1 key with 1 column valued @ 1000
stress --operation=COUNTER_ADD --family-type=Standard --num-keys=1000 --num-different-keys=1 --columns=1 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
// Run stress tool and verify update of existing key -- Final result is 2 columns valued at 1500, 500.
stress --operation=COUNTER_ADD --family-type=Standard --num-keys=500 --num-different-keys=1 --columns=2 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
// Run stress tool and verify update of existing key -- Final result is 3 columns valued at 2100, 1100, 600.
stress --operation=COUNTER_ADD --family-type=Standard --num-keys=600 --num-different-keys=1 --columns=3 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
{code}
*Data while all three nodes are up*
{code}
[default@Keyspace1] list Counter1;
Using default limit of 100
-------------------
RowKey: 30
=> (counter=4330, value=2100)
=> (counter=4331, value=1100)
=> (counter=4332, value=600)
{code}
* Shutdown nodes 1,2,3
* Startup nodes 1,2,3
* Verify in CLI: 11 keys. I am expecting only 1.
*Data after bouncing nodes*
{code}
[default@Keyspace1] list Counter1;
Using default limit of 100
-------------------
RowKey: 3036
=> (counter=4330, value=500597)
-------------------
RowKey: 3038
=> (counter=4330, value=500591)
-------------------
RowKey: 3039
=> (counter=4330, value=500609)
-------------------
RowKey: 3033
=> (counter=4330, value=500607)
-------------------
RowKey: 3037
=> (counter=4330, value=500601)
-------------------
RowKey: 30
=> (counter=4330, value=2708611)
=> (counter=4331, value=606482)
=> (counter=4332, value=180798)
-------------------
RowKey: 3030
=> (counter=4330, value=500616)
-------------------
RowKey: 3032
=> (counter=4330, value=500596)
-------------------
RowKey: 3031
=> (counter=4330, value=500613)
-------------------
RowKey: 3035
=> (counter=4330, value=500624)
-------------------
RowKey: 3034
=> (counter=4330, value=500618)
11 Rows Returned.
{code}
Summary: Data from truncated CF reappears after server restart (was: Data from truncated Counter CF reappears after server restart)
> Data from truncated CF reappears after server restart
> -----------------------------------------------------
>
> Key: CASSANDRA-2950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
> Project: Cassandra
> Issue Type: Bug
> Reporter: Cathy Daw
> Assignee: Sylvain Lebresne
>
> * Configure 3 node cluster
> * Ensure the java stress tool creates Keyspace1 with RF=3
> {code}
> // Run Stress Tool to generate 10 keys, 1 column
> stress --operation=INSERT -t 2 --num-keys=50 --columns=20 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 50 keys in CLI
> use Keyspace1;
> list Standard1;
> // TRUNCATE CF in CLI
> use Keyspace1;
> truncate counter1;
> list counter1;
> // Run stress tool and verify creation of 1 key with 10 columns
> stress --operation=INSERT -t 2 --num-keys=1 --columns=10 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 1 key in CLI
> use Keyspace1;
> list Standard1;
> // Restart all three nodes
> // You will see 51 keys in CLI
> use Keyspace1;
> list Standard1;
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2950) Data from truncated Counter CF
reappears after server restart
Posted by "Cathy Daw (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cathy Daw updated CASSANDRA-2950:
---------------------------------
Comment: was deleted
(was: Confirmed this only occurs with Counters. There is a limitation on deleting counters, that they may appear if they haven't been applied to all nodes and compacted away. Not sure if this falls under the same limitation, since in a traditional RDBMS, the semantics for truncate are different from delete.)
> Data from truncated Counter CF reappears after server restart
> -------------------------------------------------------------
>
> Key: CASSANDRA-2950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
> Project: Cassandra
> Issue Type: Bug
> Reporter: Cathy Daw
> Assignee: Sylvain Lebresne
>
> * Configure 3 node cluster
> * Ensure the java stress tool creates Keyspace1 with RF=3
> {code}
> // Run Stress Tool to generate 10 keys, 1 column
> stress --operation=COUNTER_ADD --family-type=Standard --num-keys=1000 --num-different-keys=10 --columns=1 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
> // Verify 10 keys in CLI
> use Keyspace1;
> list Counter1;
> //TRUNCATE CF in CLI
> use Keyspace1;
> truncate counter1;
> list counter1;
> // Run stress tool and verify creation of 1 key with 1 column valued @ 1000
> stress --operation=COUNTER_ADD --family-type=Standard --num-keys=1000 --num-different-keys=1 --columns=1 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
> // Run stress tool and verify update of existing key -- Final result is 2 columns valued at 1500, 500.
> stress --operation=COUNTER_ADD --family-type=Standard --num-keys=500 --num-different-keys=1 --columns=2 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
> // Run stress tool and verify update of existing key -- Final result is 3 columns valued at 2100, 1100, 600.
> stress --operation=COUNTER_ADD --family-type=Standard --num-keys=600 --num-different-keys=1 --columns=3 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
> {code}
> *Data while all three nodes are up*
> {code}
> [default@Keyspace1] list Counter1;
> Using default limit of 100
> -------------------
> RowKey: 30
> => (counter=4330, value=2100)
> => (counter=4331, value=1100)
> => (counter=4332, value=600)
> {code}
> * Shutdown nodes 1,2,3
> * Startup nodes 1,2,3
> * Verify in CLI: 11 keys. I am expecting only 1.
> *Data after bouncing nodes*
> {code}
> [default@Keyspace1] list Counter1;
> Using default limit of 100
> -------------------
> RowKey: 3036
> => (counter=4330, value=500597)
> -------------------
> RowKey: 3038
> => (counter=4330, value=500591)
> -------------------
> RowKey: 3039
> => (counter=4330, value=500609)
> -------------------
> RowKey: 3033
> => (counter=4330, value=500607)
> -------------------
> RowKey: 3037
> => (counter=4330, value=500601)
> -------------------
> RowKey: 30
> => (counter=4330, value=2708611)
> => (counter=4331, value=606482)
> => (counter=4332, value=180798)
> -------------------
> RowKey: 3030
> => (counter=4330, value=500616)
> -------------------
> RowKey: 3032
> => (counter=4330, value=500596)
> -------------------
> RowKey: 3031
> => (counter=4330, value=500613)
> -------------------
> RowKey: 3035
> => (counter=4330, value=500624)
> -------------------
> RowKey: 3034
> => (counter=4330, value=500618)
> 11 Rows Returned.
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2950) Data from truncated CF
reappears after server restart
Posted by "Cathy Daw (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13071479#comment-13071479 ]
Cathy Daw commented on CASSANDRA-2950:
--------------------------------------
The other permutation of this bug looked like, assuming write with CL.Q:
* Insert 50 (3 nodes up)
* truncate CF (3 nodes up)
* Insert 1 (3 nodes up)
* Bring node3 down
* Delete 1 (2 nodes up)
* Bring up node3 and run repair
* Take down node1 and node2.
* Query node3 with CL.ONE: list Standard1; --- 30 rows returned
Not sure, but this looked suspicious in my logs:
{code}
INFO 01:19:45,616 Streaming to /50.57.114.45
INFO 01:19:45,689 Finished streaming session 698609583499991 from /50.57.107.176
INFO 01:19:45,690 Finished streaming session 698609609994154 from /50.57.114.45
INFO 01:19:46,501 Finished streaming repair with /50.57.114.45 for (0,56713727820156410577229101238628035242]: 0 oustanding to complete session
INFO 01:19:46,531 Compacted to /var/lib/cassandra/data/Keyspace1/Standard1-tmp-g-106-Data.db. 16,646,523 to 16,646,352 (~99% of original) bytes for 30 keys. Time: 1,509ms.
INFO 01:19:46,930 Finished streaming repair with /50.57.107.176 for (113427455640312821154458202477256070484,0]: 1 oustanding to complete session
INFO 01:19:47,619 Finished streaming repair with /50.57.114.45 for (113427455640312821154458202477256070484,0]: 0 oustanding to complete session
INFO 01:19:48,232 Finished streaming repair with /50.57.107.176 for (56713727820156410577229101238628035242,113427455640312821154458202477256070484]: 1 oustanding to complete session
INFO 01:19:48,856 Finished streaming repair with /50.57.114.45 for (56713727820156410577229101238628035242,113427455640312821154458202477256070484]: 0 oustanding to complete session
{code}
> Data from truncated CF reappears after server restart
> -----------------------------------------------------
>
> Key: CASSANDRA-2950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
> Project: Cassandra
> Issue Type: Bug
> Reporter: Cathy Daw
> Assignee: Sylvain Lebresne
>
> * Configure 3 node cluster
> * Ensure the java stress tool creates Keyspace1 with RF=3
> {code}
> // Run Stress Tool to generate 10 keys, 1 column
> stress --operation=INSERT -t 2 --num-keys=50 --columns=20 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 50 keys in CLI
> use Keyspace1;
> list Standard1;
> // TRUNCATE CF in CLI
> use Keyspace1;
> truncate counter1;
> list counter1;
> // Run stress tool and verify creation of 1 key with 10 columns
> stress --operation=INSERT -t 2 --num-keys=1 --columns=10 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 1 key in CLI
> use Keyspace1;
> list Standard1;
> // Restart all three nodes
> // You will see 51 keys in CLI
> use Keyspace1;
> list Standard1;
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2950) Data from truncated CF reappears
after server restart
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-2950:
----------------------------------------
Attachment: 2950-v3_0.8.patch
Attaching a v3 that is rebased against 0.8. I've also slightly change the logic in Truncate to submit all the flushes and then call waitForActiveFlushes, as this is slightly simpler and should work equally well as far as I can tell.
Apart from that, this lgtm.
> Data from truncated CF reappears after server restart
> -----------------------------------------------------
>
> Key: CASSANDRA-2950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Cathy Daw
> Assignee: Jonathan Ellis
> Fix For: 0.8.5
>
> Attachments: 2950-v2.txt, 2950-v3_0.8.patch, 2950.txt
>
>
> * Configure 3 node cluster
> * Ensure the java stress tool creates Keyspace1 with RF=3
> {code}
> // Run Stress Tool to generate 10 keys, 1 column
> stress --operation=INSERT -t 2 --num-keys=50 --columns=20 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 50 keys in CLI
> use Keyspace1;
> list Standard1;
> // TRUNCATE CF in CLI
> use Keyspace1;
> truncate counter1;
> list counter1;
> // Run stress tool and verify creation of 1 key with 10 columns
> stress --operation=INSERT -t 2 --num-keys=1 --columns=10 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 1 key in CLI
> use Keyspace1;
> list Standard1;
> // Restart all three nodes
> // You will see 51 keys in CLI
> use Keyspace1;
> list Standard1;
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2950) Data from truncated CF
reappears after server restart
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081969#comment-13081969 ]
Brandon Williams commented on CASSANDRA-2950:
---------------------------------------------
bq. but we record CL "context" at time of flush in the sstable it makes, and we on replay we ignore any mutations from before that position.
I think there's something wrong with that, then:
{noformat}
INFO 21:25:15,274 Replaying /var/lib/cassandra/commitlog/CommitLog-1312924388053.log
DEBUG 21:25:15,290 Replaying /var/lib/cassandra/commitlog/CommitLog-1312924388053.log starting at 0
DEBUG 21:25:15,291 Reading mutation at 0
DEBUG 21:25:15,295 replaying mutation for system.4c: {ColumnFamily(LocationInfo [47656e65726174696f6e:false:4@1312924388140000,])}
DEBUG 21:25:15,321 Reading mutation at 89
DEBUG 21:25:15,322 replaying mutation for system.426f6f747374726170: {ColumnFamily(LocationInfo [42:false:1@1312924388203,])}
DEBUG 21:25:15,322 Reading mutation at 174
DEBUG 21:25:15,322 replaying mutation for system.4c: {ColumnFamily(LocationInfo [546f6b656e:false:16@1312924388204,])}
DEBUG 21:25:15,322 Reading mutation at 270
DEBUG 21:25:15,324 replaying mutation for Keyspace1.3030: {ColumnFamily(Standard1 [C0:false:34@1312924813259,C1:false:34@1312924813260,C2:false:34@1312924813260,C3:false:34@1312924813260,C4:false:34@1312924813260,])}
{noformat}
The last entry there is the first of many errant mutations.
> Data from truncated CF reappears after server restart
> -----------------------------------------------------
>
> Key: CASSANDRA-2950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
> Project: Cassandra
> Issue Type: Bug
> Reporter: Cathy Daw
> Assignee: Brandon Williams
>
> * Configure 3 node cluster
> * Ensure the java stress tool creates Keyspace1 with RF=3
> {code}
> // Run Stress Tool to generate 10 keys, 1 column
> stress --operation=INSERT -t 2 --num-keys=50 --columns=20 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 50 keys in CLI
> use Keyspace1;
> list Standard1;
> // TRUNCATE CF in CLI
> use Keyspace1;
> truncate counter1;
> list counter1;
> // Run stress tool and verify creation of 1 key with 10 columns
> stress --operation=INSERT -t 2 --num-keys=1 --columns=10 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 1 key in CLI
> use Keyspace1;
> list Standard1;
> // Restart all three nodes
> // You will see 51 keys in CLI
> use Keyspace1;
> list Standard1;
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2950) Data from truncated CF reappears
after server restart
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis reassigned CASSANDRA-2950:
-----------------------------------------
Assignee: Brandon Williams (was: Sylvain Lebresne)
> Data from truncated CF reappears after server restart
> -----------------------------------------------------
>
> Key: CASSANDRA-2950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
> Project: Cassandra
> Issue Type: Bug
> Reporter: Cathy Daw
> Assignee: Brandon Williams
>
> * Configure 3 node cluster
> * Ensure the java stress tool creates Keyspace1 with RF=3
> {code}
> // Run Stress Tool to generate 10 keys, 1 column
> stress --operation=INSERT -t 2 --num-keys=50 --columns=20 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 50 keys in CLI
> use Keyspace1;
> list Standard1;
> // TRUNCATE CF in CLI
> use Keyspace1;
> truncate counter1;
> list counter1;
> // Run stress tool and verify creation of 1 key with 10 columns
> stress --operation=INSERT -t 2 --num-keys=1 --columns=10 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 1 key in CLI
> use Keyspace1;
> list Standard1;
> // Restart all three nodes
> // You will see 51 keys in CLI
> use Keyspace1;
> list Standard1;
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2950) Data from truncated CF
reappears after server restart
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082430#comment-13082430 ]
Sylvain Lebresne commented on CASSANDRA-2950:
---------------------------------------------
I think the forceFlush of all the CF is not safe, because if for a given column family the memtable is clean, forceFlush will return immediately, even though there could be a memtable being flush at the same time (or pending flush). So we cannot be sure all the old segment are clean after the waitFutures (I know, it took me some time to figure out some problem with repair for this very reason when the repairs were not properly synchronized).
What we would need is to add to the future we wait on the futures of all the flush being processed at that time. Sounds annoying though.
> Data from truncated CF reappears after server restart
> -----------------------------------------------------
>
> Key: CASSANDRA-2950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Cathy Daw
> Assignee: Jonathan Ellis
> Fix For: 0.8.4
>
> Attachments: 2950.txt
>
>
> * Configure 3 node cluster
> * Ensure the java stress tool creates Keyspace1 with RF=3
> {code}
> // Run Stress Tool to generate 10 keys, 1 column
> stress --operation=INSERT -t 2 --num-keys=50 --columns=20 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 50 keys in CLI
> use Keyspace1;
> list Standard1;
> // TRUNCATE CF in CLI
> use Keyspace1;
> truncate counter1;
> list counter1;
> // Run stress tool and verify creation of 1 key with 10 columns
> stress --operation=INSERT -t 2 --num-keys=1 --columns=10 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 1 key in CLI
> use Keyspace1;
> list Standard1;
> // Restart all three nodes
> // You will see 51 keys in CLI
> use Keyspace1;
> list Standard1;
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2950) Data from truncated CF
reappears after server restart
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081941#comment-13081941 ]
Brandon Williams commented on CASSANDRA-2950:
---------------------------------------------
Currently, truncate does:
* force a flush
* record the time
* delete any sstables older than the time
This isn't quite enough if the machine crashes shortly afterward, however, since there can be mutations present in the commitlog that were previously truncated and are now resurrected by CL replay.
One thing we could do is record the truncate time for the CF in the system ks and then ignore mutations older than that, however this would require time synchronization between the client and the server to be accurate.
> Data from truncated CF reappears after server restart
> -----------------------------------------------------
>
> Key: CASSANDRA-2950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
> Project: Cassandra
> Issue Type: Bug
> Reporter: Cathy Daw
> Assignee: Brandon Williams
>
> * Configure 3 node cluster
> * Ensure the java stress tool creates Keyspace1 with RF=3
> {code}
> // Run Stress Tool to generate 10 keys, 1 column
> stress --operation=INSERT -t 2 --num-keys=50 --columns=20 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 50 keys in CLI
> use Keyspace1;
> list Standard1;
> // TRUNCATE CF in CLI
> use Keyspace1;
> truncate counter1;
> list counter1;
> // Run stress tool and verify creation of 1 key with 10 columns
> stress --operation=INSERT -t 2 --num-keys=1 --columns=10 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 1 key in CLI
> use Keyspace1;
> list Standard1;
> // Restart all three nodes
> // You will see 51 keys in CLI
> use Keyspace1;
> list Standard1;
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2950) Data from truncated Counter CF
reappears after server restart
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis reassigned CASSANDRA-2950:
-----------------------------------------
Assignee: Sylvain Lebresne
> Data from truncated Counter CF reappears after server restart
> -------------------------------------------------------------
>
> Key: CASSANDRA-2950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
> Project: Cassandra
> Issue Type: Bug
> Reporter: Cathy Daw
> Assignee: Sylvain Lebresne
>
> * Configure 3 node cluster
> * Ensure the java stress tool creates Keyspace1 with RF=3
> {code}
> // Run Stress Tool to generate 10 keys, 1 column
> stress --operation=COUNTER_ADD --family-type=Standard --num-keys=1000 --num-different-keys=10 --columns=1 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
> // Verify 10 keys in CLI
> use Keyspace1;
> list Counter1;
> //TRUNCATE CF in CLI
> use Keyspace1;
> truncate counter1;
> list counter1;
> // Run stress tool and verify creation of 1 key with 1 column valued @ 1000
> stress --operation=COUNTER_ADD --family-type=Standard --num-keys=1000 --num-different-keys=1 --columns=1 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
> // Run stress tool and verify update of existing key -- Final result is 2 columns valued at 1500, 500.
> stress --operation=COUNTER_ADD --family-type=Standard --num-keys=500 --num-different-keys=1 --columns=2 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
> // Run stress tool and verify update of existing key -- Final result is 3 columns valued at 2100, 1100, 600.
> stress --operation=COUNTER_ADD --family-type=Standard --num-keys=600 --num-different-keys=1 --columns=3 --consistency-level=QUORUM --average-size-values --replication-factor=3 --nodes=node1,node1
> {code}
> *Data while all three nodes are up*
> {code}
> [default@Keyspace1] list Counter1;
> Using default limit of 100
> -------------------
> RowKey: 30
> => (counter=4330, value=2100)
> => (counter=4331, value=1100)
> => (counter=4332, value=600)
> {code}
> * Shutdown nodes 1,2,3
> * Startup nodes 1,2,3
> * Verify in CLI: 11 keys. I am expecting only 1.
> *Data after bouncing nodes*
> {code}
> [default@Keyspace1] list Counter1;
> Using default limit of 100
> -------------------
> RowKey: 3036
> => (counter=4330, value=500597)
> -------------------
> RowKey: 3038
> => (counter=4330, value=500591)
> -------------------
> RowKey: 3039
> => (counter=4330, value=500609)
> -------------------
> RowKey: 3033
> => (counter=4330, value=500607)
> -------------------
> RowKey: 3037
> => (counter=4330, value=500601)
> -------------------
> RowKey: 30
> => (counter=4330, value=2708611)
> => (counter=4331, value=606482)
> => (counter=4332, value=180798)
> -------------------
> RowKey: 3030
> => (counter=4330, value=500616)
> -------------------
> RowKey: 3032
> => (counter=4330, value=500596)
> -------------------
> RowKey: 3031
> => (counter=4330, value=500613)
> -------------------
> RowKey: 3035
> => (counter=4330, value=500624)
> -------------------
> RowKey: 3034
> => (counter=4330, value=500618)
> 11 Rows Returned.
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2950) Data from truncated CF reappears
after server restart
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis reassigned CASSANDRA-2950:
-----------------------------------------
Assignee: Jonathan Ellis (was: Brandon Williams)
> Data from truncated CF reappears after server restart
> -----------------------------------------------------
>
> Key: CASSANDRA-2950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
> Project: Cassandra
> Issue Type: Bug
> Reporter: Cathy Daw
> Assignee: Jonathan Ellis
>
> * Configure 3 node cluster
> * Ensure the java stress tool creates Keyspace1 with RF=3
> {code}
> // Run Stress Tool to generate 10 keys, 1 column
> stress --operation=INSERT -t 2 --num-keys=50 --columns=20 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 50 keys in CLI
> use Keyspace1;
> list Standard1;
> // TRUNCATE CF in CLI
> use Keyspace1;
> truncate counter1;
> list counter1;
> // Run stress tool and verify creation of 1 key with 10 columns
> stress --operation=INSERT -t 2 --num-keys=1 --columns=10 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 1 key in CLI
> use Keyspace1;
> list Standard1;
> // Restart all three nodes
> // You will see 51 keys in CLI
> use Keyspace1;
> list Standard1;
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2950) Data from truncated CF
reappears after server restart
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081966#comment-13081966 ]
Jonathan Ellis commented on CASSANDRA-2950:
-------------------------------------------
but we record CL "context" at time of flush in the sstable it makes, and we on replay we ignore any mutations from before that position.
checked and we do wait for flush to complete in truncate.
> Data from truncated CF reappears after server restart
> -----------------------------------------------------
>
> Key: CASSANDRA-2950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
> Project: Cassandra
> Issue Type: Bug
> Reporter: Cathy Daw
> Assignee: Brandon Williams
>
> * Configure 3 node cluster
> * Ensure the java stress tool creates Keyspace1 with RF=3
> {code}
> // Run Stress Tool to generate 10 keys, 1 column
> stress --operation=INSERT -t 2 --num-keys=50 --columns=20 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 50 keys in CLI
> use Keyspace1;
> list Standard1;
> // TRUNCATE CF in CLI
> use Keyspace1;
> truncate counter1;
> list counter1;
> // Run stress tool and verify creation of 1 key with 10 columns
> stress --operation=INSERT -t 2 --num-keys=1 --columns=10 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 1 key in CLI
> use Keyspace1;
> list Standard1;
> // Restart all three nodes
> // You will see 51 keys in CLI
> use Keyspace1;
> list Standard1;
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2950) Data from truncated CF reappears
after server restart
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-2950:
--------------------------------------
Attachment: 2950.txt
Ah, CASSANDRA-2419 keeps on giving...
bq. but we record CL "context" at time of flush in the sstable it makes, and we on replay we ignore any mutations from before that position.
The obvious problem with this is that the point of truncate is to blow away such sstables... Patch attached. Comment explains the core fix:
{noformat}
// Bonus complication: since we store replay position in sstable metadata,
// truncating those sstables means we will replay any CL segments from the
// beginning if we restart before they are discarded for normal reasons
// post-truncate. So we need to (a) force a new segment so the currently
// active one can be discarded, and (b) flush *all* CFs so that unflushed
// data in others don't keep any pre-truncate CL segments alive.
{noformat}
Patch also fixes the bug in ReplayManagerTruncateTest that made it miss this.
> Data from truncated CF reappears after server restart
> -----------------------------------------------------
>
> Key: CASSANDRA-2950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Cathy Daw
> Assignee: Jonathan Ellis
> Fix For: 0.8.4
>
> Attachments: 2950.txt
>
>
> * Configure 3 node cluster
> * Ensure the java stress tool creates Keyspace1 with RF=3
> {code}
> // Run Stress Tool to generate 10 keys, 1 column
> stress --operation=INSERT -t 2 --num-keys=50 --columns=20 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 50 keys in CLI
> use Keyspace1;
> list Standard1;
> // TRUNCATE CF in CLI
> use Keyspace1;
> truncate counter1;
> list counter1;
> // Run stress tool and verify creation of 1 key with 10 columns
> stress --operation=INSERT -t 2 --num-keys=1 --columns=10 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 1 key in CLI
> use Keyspace1;
> list Standard1;
> // Restart all three nodes
> // You will see 51 keys in CLI
> use Keyspace1;
> list Standard1;
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2950) Data from truncated CF
reappears after server restart
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082431#comment-13082431 ]
Brandon Williams commented on CASSANDRA-2950:
---------------------------------------------
+1, though this patch is against trunk, not 0.8. Also mistakenly bumps the log4j level to debug.
> Data from truncated CF reappears after server restart
> -----------------------------------------------------
>
> Key: CASSANDRA-2950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Cathy Daw
> Assignee: Jonathan Ellis
> Fix For: 0.8.4
>
> Attachments: 2950.txt
>
>
> * Configure 3 node cluster
> * Ensure the java stress tool creates Keyspace1 with RF=3
> {code}
> // Run Stress Tool to generate 10 keys, 1 column
> stress --operation=INSERT -t 2 --num-keys=50 --columns=20 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 50 keys in CLI
> use Keyspace1;
> list Standard1;
> // TRUNCATE CF in CLI
> use Keyspace1;
> truncate counter1;
> list counter1;
> // Run stress tool and verify creation of 1 key with 10 columns
> stress --operation=INSERT -t 2 --num-keys=1 --columns=10 --consistency-level=QUORUM --average-size-values --replication-factor=3 --create-index=KEYS --nodes=cathy1,cathy2
> // Verify 1 key in CLI
> use Keyspace1;
> list Standard1;
> // Restart all three nodes
> // You will see 51 keys in CLI
> use Keyspace1;
> list Standard1;
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira