[jira] [Issue Comment Deleted] (HBASE-12596) bulkload needs to follow locality

["Andrew Purtell (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-12596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Purtell updated HBASE-12596:
-----------------------------------
    Comment: was deleted

(was: bq. So if it is in 0.98, we probably want this in 1.0, 1.1, and especially 1.2 as well.

You can ask Enis, Nick, and Sean. I've done this before and have found Enis does not accept changes for 1.0 that do not fit into the bug fix bucket. I'd presume that the case here but I'm not speaking for him. 

As 0.98 RM I can accept changes that add features on our minor releases (0.98.x) but not on our patch releases (0.98.x.y) as long as they do not break wire compatibility. that's the old policy set in discussions ahead of the 0.98.0 release. Consensus and user needs evolve though. Java binary compat became a pain point for downstreamers and Phoenix so now I also check that as part of the RC process. 

Consensus can evolve again, to a new policy that disallows anything but but fixes to 0.98. If so we should as a courtesy have a discussion with 0.98 users on the implications. Already the code divergence between 0.98 and branch-1 is significant enough to make all but trivial changes an effort to backport. If we change the 0.98 accept criteria to follow what Enis is doing with 1.0, for example, this shuts down all enhancements and that divergence will get a lot bigger. At that point I would advocate that 0.98 is EOLed. I'd retire as its RM. At some point those two actions should happen anyway. )

> bulkload needs to follow locality
> ---------------------------------
>
>                 Key: HBASE-12596
>                 URL: https://issues.apache.org/jira/browse/HBASE-12596
>             Project: HBase
>          Issue Type: Improvement
>          Components: HFile, regionserver
>    Affects Versions: 0.98.8
>         Environment: hadoop-2.3.0, hbase-0.98.8, jdk1.7
>            Reporter: Victor Xu
>            Assignee: Victor Xu
>             Fix For: 2.0.0, 0.98.14, 1.3.0
>
>         Attachments: HBASE-12596-0.98-v1.patch, HBASE-12596-0.98-v2.patch, HBASE-12596-0.98-v3.patch, HBASE-12596-0.98-v4.patch, HBASE-12596-0.98-v5.patch, HBASE-12596-0.98-v6.patch, HBASE-12596-branch-1-v1.patch, HBASE-12596-branch-1-v2.patch, HBASE-12596-master-v1.patch, HBASE-12596-master-v2.patch, HBASE-12596-master-v3.patch, HBASE-12596-master-v4.patch, HBASE-12596-master-v5.patch, HBASE-12596-master-v6.patch, HBASE-12596.patch
>
>
> Normally, we have 2 steps to perform a bulkload: 1. use a job to write HFiles to be loaded; 2. Move these HFiles to the right hdfs directory. However, the locality could be loss during the first step. Why not just write the HFiles directly into the right place? We can do this easily because StoreFile.WriterBuilder has the "withFavoredNodes" method, and we just need to call it in HFileOutputFormat's getNewWriter().
> This feature is enabled by default, and we could use 'hbase.bulkload.locality.sensitive.enabled=false' to disable it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)