You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Andrew Purtell (JIRA)" <ji...@apache.org> on 2015/01/14 19:23:35 UTC
[jira] [Commented] (HBASE-12596) bulkload needs to follow locality
[ https://issues.apache.org/jira/browse/HBASE-12596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14277374#comment-14277374 ]
Andrew Purtell commented on HBASE-12596:
----------------------------------------
If this can be made optional but defaulting to "true" it seems like a nice enhancement. Should have a way to turn this off so we can avoid a future patch to undo if it becomes necessary for some reason. Can the job submitter unset "hbase.mapreduce.hfileoutputformat.output.table.name"? I don't think that's possible with the current patch. Also, please provide a patch against the "master" branch.
> bulkload needs to follow locality
> ---------------------------------
>
> Key: HBASE-12596
> URL: https://issues.apache.org/jira/browse/HBASE-12596
> Project: HBase
> Issue Type: Improvement
> Components: HFile, regionserver
> Affects Versions: 0.98.8
> Environment: hadoop-2.3.0, hbase-0.98.8, jdk1.7
> Reporter: Victor Xu
> Attachments: HBASE-12596.patch
>
>
> Normally, we have 2 steps to perform a bulkload: 1. use a job to write HFiles to be loaded; 2. Move these HFiles to the right hdfs directory. However, the locality could be loss during the first step. Why not just write the HFiles directly into the right place? We can do this easily because StoreFile.WriterBuilder has the "withFavoredNodes" method, and we just need to call it in HFileOutputFormat's getNewWriter().
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)