You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Marcus Eriksson (JIRA)" <ji...@apache.org> on 2019/04/23 08:23:00 UTC
[jira] [Updated] (CASSANDRA-15084) failed compactions in specific
cluster version 3.11.2
[ https://issues.apache.org/jira/browse/CASSANDRA-15084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marcus Eriksson updated CASSANDRA-15084:
----------------------------------------
Resolution: Duplicate
Status: Resolved (was: Triage Needed)
Could you try on 3.11.4? Looks like this is fixed by CASSANDRA-14649
> failed compactions in specific cluster version 3.11.2
> -----------------------------------------------------
>
> Key: CASSANDRA-15084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15084
> Project: Cassandra
> Issue Type: Bug
> Reporter: Tomer Yuval
> Priority: Normal
> Fix For: 3.11.x
>
>
> We are using Cassandra version 3.11.2
> In our biggest cluster, in the biggest table compactions keep failing.
> The amount of data in every node is around 900GB.
> After we run the compaction, we monitor the process using nodetool compactionstats and can see the progress continue until it reached 99.99% of the progress.
> In this point the compaction fail with the error below
> In addition, we increased index_summary_capacity_in_mb few times until the size of 4GB, but with no help.
>
> ERROR [CompactionExecutor:26976] 2019-04-11 13:55:24,917 CassandraDaemon.java:228 - Exception in thread Thread[CompactionExecutor:26976,1,main]
> java.lang.IllegalArgumentException: null
> at java.nio.Buffer.position(Buffer.java:244) ~[na:1.8.0_144]
> at org.apache.cassandra.io.util.SafeMemoryWriter.reallocate(SafeMemoryWriter.java:59) ~[apache-cassandra-3.11.2.jar:3.11.2]
> at org.apache.cassandra.io.util.SafeMemoryWriter.setCapacity(SafeMemoryWriter.java:68) ~[apache-cassandra-3.11.2.jar:3.11.2]
> at org.apache.cassandra.io.sstable.IndexSummaryBuilder.prepareToCommit(IndexSummaryBuilder.java:250) ~[apache-cassandra-3.11.2.jar:3.11.2]
> at org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.doPrepare(BigTableWriter.java:524) ~[apache-cassandra-3.11.2.jar:3.11.
> 2]
> at org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173) ~[apache-cassandra-3.11.2.
> jar:3.11.2]
> at org.apache.cassandra.io.sstable.format.big.BigTableWriter$TransactionalProxy.doPrepare(BigTableWriter.java:364) ~[apache-cassandra-3.11.2.ja
> r:3.11.2]
> at org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173) ~[apache-cassandra-3.11.2.
> jar:3.11.2]
> at org.apache.cassandra.io.sstable.format.SSTableWriter.prepareToCommit(SSTableWriter.java:281) ~[apache-cassandra-3.11.2.jar:3.11.2]
> at org.apache.cassandra.io.sstable.SSTableRewriter.doPrepare(SSTableRewriter.java:379) ~[apache-cassandra-3.11.2.jar:3.11.2]
> at org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173) ~[apache-cassandra-3.11.2.
> jar:3.11.2]
> at [org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.doPrepare(CompactionAwareWriter.java:112|http://org.apache.cassandra.db.compaction.writers.compactionawarewriter.doprepare%28compactionawarewriter.java:112/]) ~[apache-cassandra-3.11.2.jar:3.1
> 1.2]
> at org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173) ~[apache-cassandra-3.11.2.
> jar:3.11.2]
> at org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish(Transactional.java:184) ~[apache-cassandra-3.11.2.jar:3.11.
> 2]
> at [org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.finish(CompactionAwareWriter.java:122|http://org.apache.cassandra.db.compaction.writers.compactionawarewriter.finish%28compactionawarewriter.java:122/]) ~[apache-cassandra-3.11.2.jar:3.11.2
> ]
> at [org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:220|http://org.apache.cassandra.db.compaction.compactiontask.runmaythrow%28compactiontask.java:220/]) ~[apache-cassandra-3.11.2.jar:3.11.2]
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-3.11.2.jar:3.11.2]
> at [org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:85|http://org.apache.cassandra.db.compaction.compactiontask.executeinternal%28compactiontask.java:85/]) ~[apache-cassandra-3.11.2.jar:3.11.2]
> at [org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61|http://org.apache.cassandra.db.compaction.abstractcompactiontask.execute%28abstractcompactiontask.java:61/]) ~[apache-cassandra-3.11.2.jar:3.11.2]
> at [org.apache.cassandra.db.compaction.CompactionManager$10.runMayThrow(CompactionManager.java:746|http://org.apache.cassandra.db.compaction.compactionmanager%2410.runmaythrow%28compactionmanager.java:746/]) ~[apache-cassandra-3.11.2.jar:3.11.2]
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-3.11.2.jar:3.11.2]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_144]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_144]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_144]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_144]
> at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81) [apache-cassandra-3.11.2.jar:3.11.2]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org