[jira] [Comment Edited] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope

["Sotirios Delimanolis (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15726613#comment-15726613 ] 

Sotirios Delimanolis edited comment on CASSANDRA-12979 at 12/6/16 8:29 PM:
---------------------------------------------------------------------------

+1

We hit this issue recently. A huge set of sstables couldn't get compacted. We've been running a version of this patch (in 2.2 and just some added logging) in production for a couple of days and it unblocks these compactions.

I suggest you open a separate ticket for the {{RuntimeException}}, though. Nothing is currently set up to handle it right now. It doesn't even get logged. I assume that's why this issue wasn't identified sooner.


was (Author: s_delima):
+1

We hit this issue recently. A huge set of sstables couldn't get compacted. We've been running a version of this patch (just more logging) in production for a couple of days and it unblocks these compactions.

I suggest you open a separate ticket for the {{RuntimeException}}, though. Nothing is currently set up to handle it right now. It doesn't even get logged. I assume that's why this issue wasn't identified sooner.

> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
> -----------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-12979
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Jon Haddad
>            Assignee: Jon Haddad
>             Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
>
> If a compaction occurs that looks like it'll take up more space than remaining disk available, the compaction manager attempts to reduce the scope of the compaction by calling {{reduceScopeForLimitedSpace()}} repeatedly.  
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated from the original call to {{hasAvailableDiskSpace}}, so the comparisons that are done will always be against the size of the original compaction, rather than the reduced scope one.
> Full method below:
> {code}
>     protected void checkAvailableDiskSpace(long estimatedSSTables, long expectedWriteSize)
>     {
>         if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType == OperationType.COMPACTION)
>         {
>             logger.info("Compaction space check is disabled");
>             return;
>         }
>         while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables, expectedWriteSize))
>         {
>             if (!reduceScopeForLimitedSpace())
>                 throw new RuntimeException(String.format("Not enough space for compaction, estimated sstables = %d, expected write size = %d", estimatedSSTables, expectedWriteSize));
>       
>         }
>     }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and {{expectedWriteSize}} after each iteration of {{reduceScopeForLimitedSpace}}.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)