[jira] [Commented] (CASSANDRA-5025) Schema push/pull race

["Daniel van 't Oever (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-5025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15804212#comment-15804212 ] 

Daniel van 't Oever commented on CASSANDRA-5025:
------------------------------------------------

I experience similar problems in a one node cluster. When I start 3 applications at the same time, they all try to migrate the cassandra schema (but will wait for each other using a locking table). However, they will check for this lock table using a CREATE TABLE IF NOT EXISTS

Cassandra driver 2.1.10.1

{noformat}
2017-01-03 09:57:22,372 · WARN · cluster2-nio-worker-1 · com.datastax.driver.core.RequestHandler · /127.0.0.1:9042 replied with server error (java.lang.RuntimeException: java.util.concurrent.ExecutionException: org.apache.cassandra.exceptions.ConfigurationException: Column family ID mismatch (found a1c87c40-d192-11e6-a126-1d2c09c16740; expected a1c56f00-d192-11e6-a126-1d2c09c16740)), defuncting connection. ·  ·  ·  ·
2017-01-03 09:57:22,394 · ERROR · main · com.contrastsecurity.cassandra.migration.action.Migrate · Migration of keyspace ces2 to version 0.1.0.1 failed! Please restore backups and roll back database and code! ·  ·  ·  ·
2017-01-03 09:57:24,652 · ERROR · main · nl.mypackage.CassandraMigrationService · Error during migration ·  ·  ·  ·
com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /127.0.0.1:9042 (com.datastax.driver.core.exceptions.DriverException: Timeout while trying to acquire available connection (you may want to increase the driver number of per-host connections)))
        at com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:84)
        at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
        at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:217)
       at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:54)
        at com.contrastsecurity.cassandra.migration.dao.SchemaVersionDAO.tablesExist(SchemaVersionDAO.java:88)
{noformat}

What is the recommended way to perform schema migrations in a Cassandra cluster?

> Schema push/pull race
> ---------------------
>
>                 Key: CASSANDRA-5025
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5025
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.1.0
>            Reporter: Jonathan Ellis
>            Assignee: Jonathan Ellis
>            Priority: Minor
>             Fix For: 1.1.8
>
>         Attachments: 5025-v2.txt, 5025-v3.txt, 5025-v4.txt, 5025-v5.txt, 5025.txt
>
>
> When a schema change is made, the coordinator pushes the delta to the other nodes in the cluster.  This is more efficient than sending the entire schema.  But the coordinator also announces the new schema version, so the other nodes' reception of the new version races with processing the delta, and usually seeing the new schema wins.  So the other nodes also issue a pull to the coordinator for the entire schema.
> Thus, schema changes tend to become O(n) in the number of KS and CF present.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)