[jira] [Updated] (HBASE-24859) Improve the storage cost for HBase map reduce table splits

["Sandeep Pal (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-24859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sandeep Pal updated HBASE-24859:
--------------------------------
    Summary: Improve the storage cost for HBase map reduce table splits  (was: Remove the empty regions from the hbase mapreduce splits)

> Improve the storage cost for HBase map reduce table splits
> ----------------------------------------------------------
>
>                 Key: HBASE-24859
>                 URL: https://issues.apache.org/jira/browse/HBASE-24859
>             Project: HBase
>          Issue Type: Improvement
>          Components: mapreduce
>            Reporter: Sandeep Pal
>            Assignee: Sandeep Pal
>            Priority: Major
>         Attachments: Screen Shot 2020-08-26 at 8.44.34 AM.png, hbase-24859.png
>
>
> It has been observed that when the table has too many regions, MR jobs consume more memory in the client. This is because we keep the region level information in memory and the memory heavy object is TableSplit because of the Scan object as a part of it.
> We can optimize the memory consumption by not loading the region level information if the region is empty based on the configuration.
> The default configuration can lead to all TableSplits in memory (no change from the current), but the configuration can enable the map-reduce job to ignore the empty regions. The configuration can be a part of MR job based. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)