[jira] [Updated] (PHOENIX-6081) Improvements to snapshot based MR input format

["Viraj Jasani (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PHOENIX-6081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Viraj Jasani updated PHOENIX-6081:
----------------------------------
    Fix Version/s:     (was: 4.16.1)
                   4.16.2

> Improvements to snapshot based MR input format
> ----------------------------------------------
>
>                 Key: PHOENIX-6081
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-6081
>             Project: Phoenix
>          Issue Type: Improvement
>          Components: core
>    Affects Versions: 5.0.0, 4.14.3
>            Reporter: Bharath Vissapragada
>            Priority: Major
>             Fix For: 4.17.0, 4.16.2
>
>
> Recently we switched an MR application from scanning live tables to scanning snapshots (PHOENIX-3744). We ran into a severe performance issue, which turned out to a correctness issue due to over-lapping scan splits generation. After some debugging we figured that it has been fixed via PHOENIX-4997. Even with that fix there are quite a few things we could improve about the snapshot based input format. Listing them here, perhaps we can break them into subtasks as needed.
> - Do not restore the snapshot per map task. Currently we restore the snapshot once per map task into a temp directory. For large tables on big clusters, this creates a storm of NN RPCs. We can do this once per job and let all the map tasks operate on the same restored snapshot. HBase already did this via HBASE-18806, we can do something similar.
> - Disable [cacheBlocks|[https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Scan.html#setCacheBlocks-boolean-]] on scans generated by input format. In our experiments block cache took a lot of memory for MR jobs. For full table scans this isn't of much use and can save a lot of memory.
> - Short circuit live-table codepaths when snapshots are enabled. Currently some codepaths make live table HBase RPCs to get a bunch of data. For example
> {noformat}
> private List<InputSplit> generateSplits(final QueryPlan qplan, Configuration config) throws IOException {
>     // We must call this in order to initialize the scans and splits from the query plan
>   ....
> // Get the RegionSizeCalculator
> try(org.apache.hadoop.hbase.client.Connection connection =
>             HBaseFactoryProvider.getHConnectionFactory().createConnection(config)) {
> RegionLocator regionLocator = connection.getRegionLocator(TableName.valueOf(tableName));
> RegionSizeCalculator sizeCalculator = new RegionSizeCalculator(regionLocator, connection
>         .getAdmin()); {noformat}
> This defeats the purpose of using snapshots. Refactor the code in a way that the snapshot based codepaths do minimal HBase RPCs and rely solely on snapshot manifest. Even better, improve locality of task scheduling based on snapshot's hfile block locations.
> - Disable indexes for query plan for scanning over snapshots. If there is an index based access path, getScans() can potentially return index based splits which is not what we want for snapshots.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)