[jira] Updated: (HBASE-1025) Reconstruction log playback has no bounds on memory used

["stack (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

stack updated HBASE-1025:
-------------------------

    Attachment: 1025.txt

This might not be too bad.  How replay of edits is done was changed a while back by Clint (IIRC).  He inserts straight into memstore where before we made a special instance of a memstore to put recovered edits into and we flushed that.  The flusher is running at time of log replay so it should trigger if memstores exceed configured sizes.

While in here, I changed the way we do replay of edits so its done once at the region level rather than once per column family.  It makes things cleaner and easier to think about.

This patch does not have the tests needed:

1. Verification that indeed the flusher will run if exceed memstore configured flush size
2. Verfication that edits are recovered particularly in multi family case

There is already an issue to test that deletes are recovered properly.  Can verify deletes over in that issue.

> Reconstruction log playback has no bounds on memory used
> --------------------------------------------------------
>
>                 Key: HBASE-1025
>                 URL: https://issues.apache.org/jira/browse/HBASE-1025
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>            Assignee: Kannan Muthukkaruppan
>             Fix For: 0.21.0
>
>         Attachments: 1025.txt
>
>
> Makes a TreeMap and just keeps adding edits without regard for size of edits applied; could cause OOME (I've not seen a definitive case though have seen an OOME around time of a reconstructionlog replay -- perhaps this the straw that broke the fleas antlers?)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.