You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Milan Majercik (JIRA)" <ji...@apache.org> on 2016/12/07 07:43:59 UTC
[jira] [Updated] (CASSANDRA-13011) heap exhaustion when cleaning
table with wide partitions and a secondary index attached to it
[ https://issues.apache.org/jira/browse/CASSANDRA-13011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Milan Majercik updated CASSANDRA-13011:
---------------------------------------
Description:
We have a table with rather wide partitions and a secondary index attached to it. When tried to clean unused data on a node after expansion of our cluster via issuing {{nodetool cleanup}} command we observed a heap exhaustion issue. The culprit appears to be in method {{org.apache.cassandra.db.compaction.CompactionManager.CleanupStrategy.Full.cleanup}} as it tries to remove related secondary index entries. The method first populates a list will all cells belonging to the given partition...
{code:java}
while (row.hasNext())
{
OnDiskAtom column = row.next();
if (column instanceof Cell && cfs.indexManager.indexes((Cell) column))
{
if (indexedColumnsInRow == null)
indexedColumnsInRow = new ArrayList<>();
indexedColumnsInRow.add((Cell) column);
}
}
{code}
... and then submits it to the index manager for removal.
{code:java}
// acquire memtable lock here because secondary index deletion may cause a race. See CASSANDRA-3712
try (OpOrder.Group opGroup = cfs.keyspace.writeOrder.start())
{
cfs.indexManager.deleteFromIndexes(row.getKey(), indexedColumnsInRow, opGroup);
}
{code}
After imposing a limit on array size and implementing some sort of pagination the cleanup worked fine.
> heap exhaustion when cleaning table with wide partitions and a secondary index attached to it
> ---------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-13011
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13011
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Reporter: Milan Majercik
>
> We have a table with rather wide partitions and a secondary index attached to it. When tried to clean unused data on a node after expansion of our cluster via issuing {{nodetool cleanup}} command we observed a heap exhaustion issue. The culprit appears to be in method {{org.apache.cassandra.db.compaction.CompactionManager.CleanupStrategy.Full.cleanup}} as it tries to remove related secondary index entries. The method first populates a list will all cells belonging to the given partition...
> {code:java}
> while (row.hasNext())
> {
> OnDiskAtom column = row.next();
> if (column instanceof Cell && cfs.indexManager.indexes((Cell) column))
> {
> if (indexedColumnsInRow == null)
> indexedColumnsInRow = new ArrayList<>();
> indexedColumnsInRow.add((Cell) column);
> }
> }
> {code}
> ... and then submits it to the index manager for removal.
> {code:java}
> // acquire memtable lock here because secondary index deletion may cause a race. See CASSANDRA-3712
> try (OpOrder.Group opGroup = cfs.keyspace.writeOrder.start())
> {
> cfs.indexManager.deleteFromIndexes(row.getKey(), indexedColumnsInRow, opGroup);
> }
> {code}
> After imposing a limit on array size and implementing some sort of pagination the cleanup worked fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)