You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Paulo Motta (JIRA)" <ji...@apache.org> on 2016/05/20 17:34:12 UTC
[jira] [Comment Edited] (CASSANDRA-11831) Ability to disable
purgeable tombstone check via startup flag
[ https://issues.apache.org/jira/browse/CASSANDRA-11831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15293784#comment-15293784 ]
Paulo Motta edited comment on CASSANDRA-11831 at 5/20/16 5:33 PM:
------------------------------------------------------------------
LGTM, just two minor nits:
* On the unit tests, it seems you're setting {{cassandra.never_purge_tombstones}} globally via system properties, won't this propagate to other tests? Perhaps it's better to set the property directly during these tests on {{CompactionController}} ? Maybe that's why some testing are failing on [trunk|http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-11831-trunk-testall/lastCompletedBuild/testReport/] ?
* To avoid doing useless tombstone compactions that won't drop tombstones, should we maybe also disable tombstone compactions on {{AbstractCompactionStrategy}} if {{cassandra.never_purge_tombstones=true}} ?
edit: this is just a review of {{cassandra.never_purge_tombstones}} flag, we should probably move discussion of a more long term solution to the problem of improving compaction throughput with many sstables to another ticket.
was (Author: pauloricardomg):
LGTM, just two minor nits:
* On the unit tests, it seems you're setting {{cassandra.never_purge_tombstones}} globally via system properties, won't this propagate to other tests? Perhaps it's better to set the property directly during these tests on {{CompactionController}} ? Maybe that's why some testing are failing on [trunk|http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-11831-trunk-testall/lastCompletedBuild/testReport/] ?
* To avoid doing useless tombstone compactions that won't drop tombstones, should we maybe also disable tombstone compactions on {{AbstractCompactionStrategy}} if {{cassandra.never_purge_tombstones=true}} ?
> Ability to disable purgeable tombstone check via startup flag
> -------------------------------------------------------------
>
> Key: CASSANDRA-11831
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11831
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Ryan Svihla
> Assignee: Marcus Eriksson
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> On Cassandra 2.1.14 hen a node gets way behind and has 10s of thousand sstables it appears a lot of the CPU time is spent doing checks like this on a call to getMaxPurgeableTimestamp
> org.apache.cassandra.utils.Murmur3BloomFilter.hash(java.nio.ByteBuffer, int, int, long, long[]) @bci=13, line=57 (Compiled frame; information may be imprecise)
> - org.apache.cassandra.utils.BloomFilter.indexes(java.nio.ByteBuffer) @bci=22, line=82 (Compiled frame)
> - org.apache.cassandra.utils.BloomFilter.isPresent(java.nio.ByteBuffer) @bci=2, line=107 (Compiled frame)
> - org.apache.cassandra.db.compaction.CompactionController.maxPurgeableTimestamp(org.apache.cassandra.db.DecoratedKey) @bci=89, line=186 (Compiled frame)
> - org.apache.cassandra.db.compaction.LazilyCompactedRow.getMaxPurgeableTimestamp() @bci=21, line=99 (Compiled frame)
> - org.apache.cassandra.db.compaction.LazilyCompactedRow.access$300(org.apache.cassandra.db.compaction.LazilyCompactedRow) @bci=1, line=49 (Compiled frame)
> - org.apache.cassandra.db.compaction.LazilyCompactedRow$Reducer.getReduced() @bci=241, line=296 (Compiled frame)
> - org.apache.cassandra.db.compaction.LazilyCompactedRow$Reducer.getReduced() @bci=1, line=206 (Compiled frame)
> - org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext() @bci=44, line=206 (Compiled frame)
> - com.google.common.collect.AbstractIterator.tryToComputeNext() @bci=9, line=143 (Compiled frame)
> - com.google.common.collect.AbstractIterator.hasNext() @bci=61, line=138 (Compiled frame)
> - com.google.common.collect.Iterators$7.computeNext() @bci=4, line=645 (Compiled frame)
> - com.google.common.collect.AbstractIterator.tryToComputeNext() @bci=9, line=143 (Compiled frame)
> - com.google.common.collect.AbstractIterator.hasNext() @bci=61, line=138 (Compiled frame)
> - org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(java.util.Iterator) @bci=1, line=166 (Compiled frame)
> - org.apache.cassandra.db.compaction.LazilyCompactedRow.write(long, org.apache.cassandra.io.util.DataOutputPlus) @bci=52, line=121 (Compiled frame)
> - org.apache.cassandra.io.sstable.SSTableWriter.append(org.apache.cassandra.db.compaction.AbstractCompactedRow) @bci=18, line=193 (Compiled frame)
> - org.apache.cassandra.io.sstable.SSTableRewriter.append(org.apache.cassandra.db.compaction.AbstractCompactedRow) @bci=13, line=127 (Compiled frame)
> - org.apache.cassandra.db.compaction.CompactionTask.runMayThrow() @bci=666, line=197 (Compiled frame)
> - org.apache.cassandra.utils.WrappedRunnable.run() @bci=1, line=28 (Compiled frame)
> - org.apache.cassandra.db.compaction.CompactionTask.executeInternal(org.apache.cassandra.db.compaction.CompactionManager$CompactionExecutorStatsCollector) @bci=6, line=73 (Compiled frame)
> - org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(org.apache.cassandra.db.compaction.CompactionManager$CompactionExecutorStatsCollector) @bci=2, line=59 (Compiled frame)
> - org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run() @bci=125, line=264 (Compiled frame)
> - java.util.concurrent.Executors$RunnableAdapter.call() @bci=4, line=511 (Compiled frame)
> - java.util.concurrent.FutureTask.run() @bci=42, line=266 (Compiled frame)
> - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=95, line=1142 (Compiled frame)
> - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Compiled frame)
> - java.lang.Thread.run() @bci=11, line=745 (Compiled frame)
> If we could at least on startup pass a flag like -DskipTombstonePurgeCheck so we could in these particularly bad cases just avoid the calculation and merge tables until we have less to worry about then restart the node with that flag missing once we're down to a more manageable amount of sstables.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)