You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Romain Hardouin (JIRA)" <ji...@apache.org> on 2018/09/04 16:45:00 UTC
[jira] [Commented] (CASSANDRA-14647) Reading cardinality from
Statistics.db failed
[ https://issues.apache.org/jira/browse/CASSANDRA-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16603303#comment-16603303 ]
Romain Hardouin commented on CASSANDRA-14647:
---------------------------------------------
After removing {{EstimatedPartitionCount}} from Datadog's cassandra.yaml on one node, I don't see any warning for 4 days.
[~nezdali] can you double check that your change was effective (e.g. monitoring service restarted, etc.)?
> Reading cardinality from Statistics.db failed
> ---------------------------------------------
>
> Key: CASSANDRA-14647
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14647
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Environment: Clients are doing only writes with Local One, cluster consist of 3 regions with RF3.
> Storage is configured wth jbod/XFS on 10 x 1Tb disks
> IOPS limit for each disk 500 (total 5000 iops)
> Bandwith for each disk 60mb/s (600 total)
> OS is Debian linux.
> Reporter: Vitali Djatsuk
> Priority: Major
> Fix For: 3.0.x
>
> Attachments: cassandra_compaction_pending_tasks_7days.png
>
>
> There is some issue with sstable metadata which is visible in system.log, the messages says:
> {noformat}
> WARN [Thread-6] 2018-07-25 07:12:47,928 SSTableReader.java:249 - Reading cardinality from Statistics.db failed for /opt/data/disk5/data/keyspace/table/mc-big-Data.db.{noformat}
> Although there is no such file.
> The message has appeared after i've changed the compaction strategy from SizeTiered to Leveled. Compaction strategy has been changed region by region (total 3 regions) and it has coincided with the double client write traffic increase.
> I have tried to run nodetool scrub to rebuilt the sstable, but that does not fix the issue.
> So very hard to define the steps to reproduce, probably it will be:
> # run stress tool with write traffic
> # under load change compaction strategy from SireTiered to Leveled for the bunch of hosts
> # add more write traffic
> Reading the code it is said that if this metadata is broken, then "estimating the keys will be done using index summary".
> [https://github.com/apache/cassandra/blob/cassandra-3.0.17/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java#L247]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org