You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Ted Yu (JIRA)" <ji...@apache.org> on 2016/03/08 20:29:40 UTC
[jira] [Commented] (HBASE-15428) Port auto-balancing to
MultiTableInputFormatBase
[ https://issues.apache.org/jira/browse/HBASE-15428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15185576#comment-15185576 ]
Ted Yu commented on HBASE-15428:
--------------------------------
Since there is no direct inheritance relationship between MultiTableInputFormatBase and TableInputFormatBase, a new class is needed.
> Port auto-balancing to MultiTableInputFormatBase
> ------------------------------------------------
>
> Key: HBASE-15428
> URL: https://issues.apache.org/jira/browse/HBASE-15428
> Project: HBase
> Issue Type: Improvement
> Components: mapreduce
> Reporter: Nathan Schile
> Assignee: Nathan Schile
>
> Apache Crunch currently uses [MultiTableInputFormatBase|https://github.com/apache/crunch/blob/apache-crunch-0.13.0/crunch-hbase/src/main/java/org/apache/crunch/io/hbase/HBaseSourceTarget.java#L88] as the default format for reading HBase data. I would like to use the functionality provided by [HBASE-12590|https://issues.apache.org/jira/browse/HBASE-12590] "A solution for data skew in HBase-Mapreduce Job", however it is only available in TableInputFormatBase. This JIRA is to port the changes from TableInputFormatBase into MultiTableInputFormatBase with respect toa [HBASE-12590|https://issues.apache.org/jira/browse/HBASE-12590].
> I would use to use the [TableInputFormatBase#calculateRebalancedSplits |https://github.com/apache/hbase/blob/rel/1.2.0/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableInputFormatBase.java#L381] and [TableInputFormatBase#getSplitKey|https://github.com/apache/hbase/blob/rel/1.2.0/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableInputFormatBase.java#L454] methods from TableInputFormatBase. Is it ok to use those methods directly from MultiTableInputFormatBase, or should I move them to a new class? I can submit a patch once I get direction on the above question.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)