You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Bartłomiej Romański (JIRA)" <ji...@apache.org> on 2012/09/10 13:51:07 UTC
[jira] [Created] (CASSANDRA-4639) Incorrect counter values
Bartłomiej Romański created CASSANDRA-4639:
----------------------------------------------
Summary: Incorrect counter values
Key: CASSANDRA-4639
URL: https://issues.apache.org/jira/browse/CASSANDRA-4639
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.1.2
Environment: We've got a production 21 node cluster with 3 virtual data centers. Keyspace that contains counter column families has a replication 3 on DC1 and 1 on DC3. DC1 is using SSD drives, DC3 spinning hard drives. We are using Ubuntu Server as an OS. Machines have 24GB of RAM.
Reporter: Bartłomiej Romański
Priority: Critical
Since yesterday almost all counters are incorrect. Usually about 4-5 times higher than expected. In logs we've got this message:
ERROR [MutationStage:15] 2012-09-10 13:47:13,280 CounterContext.java (line 381) invalid counter shard detected; (6de8e940-dd23-11e1-0000-5233df6faaff, 7, 242) and (6de8e940-dd23-11e1-0000-5233df6faaff, 7, 392) differ only in count; will pick highest to self-heal; this indicates a bug or corruption generated a bad counter shard
every couple of seconds.
This cluster was running without any serious problems for at least 2 months.
Any ideas?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4639) Incorrect counter values
Posted by "Omid Aladini (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453065#comment-13453065 ]
Omid Aladini commented on CASSANDRA-4639:
-----------------------------------------
This looks the same as CASSANDRA-4417 which is currently in progress.
> Incorrect counter values
> ------------------------
>
> Key: CASSANDRA-4639
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4639
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.1.2
> Environment: We've got a production 21 node cluster with 3 virtual data centers. Keyspace that contains counter column families has a replication 3 on DC1 and 1 on DC3. DC1 is using SSD drives, DC3 spinning hard drives. We are using Ubuntu Server as an OS. Machines have 24GB of RAM.
> Reporter: Bartłomiej Romański
> Priority: Critical
>
> Since yesterday almost all counters are incorrect. Usually about 4-5 times higher than expected. In logs we've got this message:
> ERROR [MutationStage:15] 2012-09-10 13:47:13,280 CounterContext.java (line 381) invalid counter shard detected; (6de8e940-dd23-11e1-0000-5233df6faaff, 7, 242) and (6de8e940-dd23-11e1-0000-5233df6faaff, 7, 392) differ only in count; will pick highest to self-heal; this indicates a bug or corruption generated a bad counter shard
> every couple of seconds.
> This cluster was running without any serious problems for at least 2 months.
> Any ideas?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-4639) Incorrect counter values
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne resolved CASSANDRA-4639.
-----------------------------------------
Resolution: Duplicate
Actually, while the logged error is likely due to CASSANDRA-4417, it's quite possible that the 'counters are about 4-5 times higher than they should be' is due to CASSANDRA-4436. In any case, it's tracked elsewhere and I would suggest an upgrade (to avoid CASSANDRA-4436).
> Incorrect counter values
> ------------------------
>
> Key: CASSANDRA-4639
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4639
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.1.2
> Environment: We've got a production 21 node cluster with 3 virtual data centers. Keyspace that contains counter column families has a replication 3 on DC1 and 1 on DC3. DC1 is using SSD drives, DC3 spinning hard drives. We are using Ubuntu Server as an OS. Machines have 24GB of RAM.
> Reporter: Bartłomiej Romański
> Priority: Critical
>
> Since yesterday almost all counters are incorrect. Usually about 4-5 times higher than expected. In logs we've got this message:
> ERROR [MutationStage:15] 2012-09-10 13:47:13,280 CounterContext.java (line 381) invalid counter shard detected; (6de8e940-dd23-11e1-0000-5233df6faaff, 7, 242) and (6de8e940-dd23-11e1-0000-5233df6faaff, 7, 392) differ only in count; will pick highest to self-heal; this indicates a bug or corruption generated a bad counter shard
> every couple of seconds.
> This cluster was running without any serious problems for at least 2 months.
> Any ideas?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira