You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Oliver Lockwood (JIRA)" <ji...@apache.org> on 2016/03/02 10:30:18 UTC
[jira] [Comment Edited] (CASSANDRA-11288) Schema agreement appears
to be false positive following a DROP TABLE command
[ https://issues.apache.org/jira/browse/CASSANDRA-11288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15175324#comment-15175324 ]
Oliver Lockwood edited comment on CASSANDRA-11288 at 3/2/16 9:30 AM:
---------------------------------------------------------------------
I'm going to bump the severity to Major since I've got fairly reliable reproduction of this issue and it's manifested in 4-node environments as well as simpler 2-node ones.
See the log output below for a fuller example. What's interesting is that the "agreed" schema AFTER the DROP TABLE command has been issued is exactly the same version ({c264a58b-a3a4-39f8-84ae-c0960b7e8b4c}) as that at the end of the previous DDL change. So it looks like the nodes agree that nothing has changed yet, even though the EVENT DROPPED TABLE has come through.
{noformat}
2016-03-02 09:11:13 DEBUG SchemaChanger:61 - Executing cql statement ALTER TABLE table_8 WITH compaction = { 'class' : 'LeveledCompactionStrategy' };
2016-03-02 09:11:13 DEBUG Cluster:2256 - Received event EVENT UPDATED TABLE testkeyspace.table_8, scheduling delivery
2016-03-02 09:11:13 DEBUG Cluster:2199 - Refreshing schema for testkeyspace.table_8 (TABLE)
2016-03-02 09:11:13 DEBUG ControlConnection:676 - Checking for schema agreement: versions are [d138c607-135d-3081-99ec-350661791061, c264a58b-a3a4-39f8-84ae-c0960b7e8b4c]
2016-03-02 09:11:14 DEBUG ControlConnection:676 - Checking for schema agreement: versions are [c264a58b-a3a4-39f8-84ae-c0960b7e8b4c]
2016-03-02 09:11:15 DEBUG ControlConnection:294 - [Control connection] Refreshing schema for testkeyspace.table_8 (TABLE)
2016-03-02 09:11:15 DEBUG SchemaChanger:61 - Executing cql statement DROP TABLE table_1;
2016-03-02 09:11:15 DEBUG Cluster:2256 - Received event EVENT DROPPED TABLE testkeyspace.table_1, scheduling delivery
2016-03-02 09:11:15 DEBUG ControlConnection:676 - Checking for schema agreement: versions are [c264a58b-a3a4-39f8-84ae-c0960b7e8b4c]
2016-03-02 09:11:15 DEBUG SchemaChanger:61 - Executing cql statement CREATE TABLE table_1 (party_id text primary key, profile text) WITH compaction = { 'class' : 'LeveledCompactionStrategy' };
Exception in thread "main" com.datastax.driver.core.exceptions.AlreadyExistsException: Table testkeyspace.table_1 already exists
at com.datastax.driver.core.exceptions.AlreadyExistsException.copy(AlreadyExistsException.java:111)
at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:214)
at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52)
{noformat}
was (Author: oliver.lockwood):
I'm going to bump the severity to Major since I've got fairly reliable reproduction of this issue and it's manifested in 4-node environments as well as simpler 2-node ones.
See the log output below for a fuller example. What's interesting is that the "agreed" schema AFTER the DROP TABLE command has been issued is exactly the same version ({c264a58b-a3a4-39f8-84ae-c0960b7e8b4c}) as that at the end of the previous DDL change. So it looks like the nodes agree that nothing has changed yet, even though the EVENT DROPPED TABLE has come through.
{quote}
2016-03-02 09:11:13 DEBUG SchemaChanger:61 - Executing cql statement ALTER TABLE table_8 WITH compaction = { 'class' : 'LeveledCompactionStrategy' };
2016-03-02 09:11:13 DEBUG Cluster:2256 - Received event EVENT UPDATED TABLE testkeyspace.table_8, scheduling delivery
2016-03-02 09:11:13 DEBUG Cluster:2199 - Refreshing schema for testkeyspace.table_8 (TABLE)
2016-03-02 09:11:13 DEBUG ControlConnection:676 - Checking for schema agreement: versions are [d138c607-135d-3081-99ec-350661791061, c264a58b-a3a4-39f8-84ae-c0960b7e8b4c]
2016-03-02 09:11:14 DEBUG ControlConnection:676 - Checking for schema agreement: versions are [c264a58b-a3a4-39f8-84ae-c0960b7e8b4c]
2016-03-02 09:11:15 DEBUG ControlConnection:294 - [Control connection] Refreshing schema for testkeyspace.table_8 (TABLE)
2016-03-02 09:11:15 DEBUG SchemaChanger:61 - Executing cql statement DROP TABLE table_1;
2016-03-02 09:11:15 DEBUG Cluster:2256 - Received event EVENT DROPPED TABLE testkeyspace.table_1, scheduling delivery
2016-03-02 09:11:15 DEBUG ControlConnection:676 - Checking for schema agreement: versions are [c264a58b-a3a4-39f8-84ae-c0960b7e8b4c]
2016-03-02 09:11:15 DEBUG SchemaChanger:61 - Executing cql statement CREATE TABLE table_1 (party_id text primary key, profile text) WITH compaction = { 'class' : 'LeveledCompactionStrategy' };
Exception in thread "main" com.datastax.driver.core.exceptions.AlreadyExistsException: Table testkeyspace.table_1 already exists
at com.datastax.driver.core.exceptions.AlreadyExistsException.copy(AlreadyExistsException.java:111)
at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:214)
at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52){quote}
> Schema agreement appears to be false positive following a DROP TABLE command
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-11288
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11288
> Project: Cassandra
> Issue Type: Bug
> Environment: Cassandra 2.0.14.439 (DSE 4.6.7)
> 2 nodes OR 4 nodes
> Connecting with Datastax Java driver 2.1.8
> Reporter: Oliver Lockwood
> Priority: Minor
>
> As part of a schema migration operation, our application is calling the following operations on the Java driver consecutively:
> {quote}
> session.execute("DROP TABLE table_name");
> session.execute("CREATE TABLE table_name (...)");
> {quote}
> The second of these sometimes fails with a {{DriverException}} whose message is "Table keyspace.table_name already exists".
> In the schema migration operation, there's 4 of these drop/create pairings and, although it's random which exact one fails, we've never managed to get further than the third operation in approximately 10 attempts - so there's a reasonably high proportion of failure.
> I don't believe this is a driver issue because the driver is checking for schema agreement (as per https://github.com/datastax/java-driver/blob/2.1/driver-core/src/main/java/com/datastax/driver/core/ControlConnection.java#L701) and we are seeing a log message to that effect.
> {quote}
> c.d.d.c.ControlConnection - [] [] [] [] [] [] [] [] Checking for schema agreement: versions are [02bce936-fddd-3bef-bb54-124d31bede57]
> {quote}
> This log message appears in between our own logs which say "Executing statement DROP TABLE..." and "Executing statement CREATE TABLE...", so we can be reasonably sure this log message refers to the DROP operation being viewed as "in agreement".
> Could this be a bug in the Cassandra server erroneously reporting that the schemas are in agreement across the 2 nodes when, in fact, they are not?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)