You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Joshua McKenzie (JIRA)" <ji...@apache.org> on 2017/04/06 20:22:41 UTC
[jira] [Updated] (CASSANDRA-13011) heap exhaustion when cleaning
table with wide partitions and a secondary index attached to it
[ https://issues.apache.org/jira/browse/CASSANDRA-13011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joshua McKenzie updated CASSANDRA-13011:
----------------------------------------
Reviewer: Andrés de la Peña
> heap exhaustion when cleaning table with wide partitions and a secondary index attached to it
> ---------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-13011
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13011
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Reporter: Milan Majercik
> Fix For: 2.2.x
>
>
> We have a table with rather wide partitions and a secondary index attached to it. When tried to clean unused data on a node after expansion of our cluster via issuing {{nodetool cleanup}} command we observed a heap exhaustion issue. The culprit appears to be in method {{org.apache.cassandra.db.compaction.CompactionManager.CleanupStrategy.Full.cleanup}} as it tries to remove related secondary index entries. The method first populates a list will all cells belonging to the given partition...
> {code:java}
> while (row.hasNext())
> {
> OnDiskAtom column = row.next();
> if (column instanceof Cell && cfs.indexManager.indexes((Cell) column))
> {
> if (indexedColumnsInRow == null)
> indexedColumnsInRow = new ArrayList<>();
> indexedColumnsInRow.add((Cell) column);
> }
> }
> {code}
> ... and then submits it to the index manager for removal.
> {code:java}
> // acquire memtable lock here because secondary index deletion may cause a race. See CASSANDRA-3712
> try (OpOrder.Group opGroup = cfs.keyspace.writeOrder.start())
> {
> cfs.indexManager.deleteFromIndexes(row.getKey(), indexedColumnsInRow, opGroup);
> }
> {code}
> After imposing a limit on array size and implementing some sort of pagination the cleanup worked fine.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)