[jira] [Commented] (CASSANDRA-5922) Delete doesn't delete data.

["Sylvain Lebresne (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13747687#comment-13747687 ] 

Sylvain Lebresne commented on CASSANDRA-5922:
---------------------------------------------

A simple script that helps reproducing would be good, and if it is in the form of a dtest (https://github.com/riptano/cassandra-dtest/blob/master/cql_tests.py) that would be even more awesome as this would eliminate the possibility that it's a bug in astyanax-jpa.
                
> Delete doesn't delete data.
> ---------------------------
>
>                 Key: CASSANDRA-5922
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5922
>             Project: Cassandra
>          Issue Type: Bug
>         Environment: 4-node cluster w/ Cassandra v1.2.8
> Oracle JDK 1.6.0_45
> Netflix Astyanax 1.56.42
> Quorum read and write consistency level
>            Reporter: Chris Eineke
>
> In a nutcase, I'm running several test cases against my Cassandra JPA implementation (astyanax-jpa, see https://github.com/ceineke/astyanax-jpa) and sometimes(!) batched deletes seem not to delete all data. 
> Here's the sequence of prepared CQL3 statements that is causing the issue to appear:
> TRUNCATE compositeentity;
> (delete all records so we have a clean slate)
> INSERT INTO compositeentity (compositekeypartone, compositekeyparttwo, compositekeypartthree, astring, auuid) VALUES (?, ?, ?, ?, ?);
> INSERT INTO compositeentity (compositekeypartone, compositekeyparttwo, compositekeypartthree, astring, auuid) VALUES (?, ?, ?, ?, ?);
> (insert two unique rows into the table)
> SELECT compositekeypartone, compositekeyparttwo, compositekeypartthree FROM compositeentity WHERE compositekeypartone = ? AND compositekeyparttwo = ? AND compositekeypartthree = ?;
> SELECT compositekeypartone, compositekeyparttwo, compositekeypartthree FROM compositeentity WHERE compositekeypartone = ? AND compositekeyparttwo = ? AND compositekeypartthree = ?;
> (load both rows from the table to validate their existence)
> SELECT COUNT(1) FROM compositeentity;
> (counts rows to validate the number of records in the table)
> BEGIN BATCH  DELETE  FROM compositeentity WHERE compositekeypartone = ? AND compositekeyparttwo = ? AND compositekeypartthree = ?; DELETE  FROM compositeentity WHERE compositekeypartone = ? AND compositekeyparttwo = ? AND compositekeypartthree = ?; APPLY BATCH;
> (uses a logged batch to delete the two rows from the table)
> SELECT compositekeypartone, compositekeyparttwo, compositekeypartthree FROM compositeentity WHERE compositekeypartone = ? AND compositekeyparttwo = ? AND compositekeypartthree = ?;
> SELECT compositekeypartone, compositekeyparttwo, compositekeypartthree FROM compositeentity WHERE compositekeypartone = ? AND compositekeyparttwo = ? AND compositekeypartthree = ?;
> (tries loads the rows from the table to check that they don't exist anymore)
> After the delete, Cassandra has deleted only the first row so that the second SELECT here actually returns data. So far, this behaviour occurs randomly.
> This happens even if there's a long sleep (1s, 10s) between the batch delete and the selects. It is always the second row that isn't deleted, never the first.
> Thinking it might be timing issue (based on http://ria101.wordpress.com/2011/02/08/cassandra-the-importance-of-system-clocks-avoiding-oom-and-how-to-escape-oom-meltdown/), I've set up NTP to keep the clocks synchronized across all nodes (one node acts as the master which syncs to time.nrc.ca and {0,1,2,3}.ca.pool.ntp.org, whereas the remaining ones sync against the master. This hasn't reduced the number of times this behaviour crops up.
> (I am executing all statements with QUORUM level consistency.)
> I'm open to suggestions as to why this occurs and how I can fix it, if this can be fixed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira