[jira] [Commented] (KUDU-678) An empty delta block is orphaned when flushing or compacting with no edits

["Todd Lipcon (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KUDU-678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281689#comment-15281689 ] 

Todd Lipcon commented on KUDU-678:
----------------------------------

This might be a more serious issue for long-term running clusters. I just did some analysis on a data disk from a cluster that's been running for quite some time. The data drive has 354,887 blcks on it. Of those, the most common sizes are:

{code}
 number of blocks    block size
 -------------------------------
 305946 94
  37259 3286630
   1273 41402
   1212 41401
    739 41400
    296 41399
     89 41398
     37 813187
{code}

In terms of number of bytes, we're only talking 27MB (149MB if you pad out to 512-byte sectors), so not a big deal. But, these blocks take up block manager metadata and also make deletion of data directories really slow. It also seems that there are many block containers which are fully made up of these 94-byte blocks.

> An empty delta block is orphaned when flushing or compacting with no edits
> --------------------------------------------------------------------------
>
>                 Key: KUDU-678
>                 URL: https://issues.apache.org/jira/browse/KUDU-678
>             Project: Kudu
>          Issue Type: Bug
>          Components: tablet
>    Affects Versions: Private Beta
>            Reporter: Todd Lipcon
>            Priority: Minor
>
> Currently when the DRS writer writes an empty undo file, it doesn't add it to the RowSetMetadata. But, the block itself is still closed and left around in the BlockManager, causing a tiny data leak. These files are going to be small, so it's not a big deal, but we ought to fix it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)