[jira] [Resolved] (HBASE-7246) Assignment#nodeChildrenChanged calls listChildrenAndWatchForNewChildren, overloading master & zookeper needlessly

["Andrew Kyle Purtell (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-7246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Kyle Purtell resolved HBASE-7246.
----------------------------------------
    Resolution: Incomplete

> Assignment#nodeChildrenChanged calls listChildrenAndWatchForNewChildren, overloading master & zookeper needlessly
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-7246
>                 URL: https://issues.apache.org/jira/browse/HBASE-7246
>             Project: HBase
>          Issue Type: Improvement
>          Components: master, Region Assignment
>    Affects Versions: 0.95.2
>            Reporter: Nicolas Liochon
>            Priority: Major
>
> Assignment#nodeChildrenChanged listens to change event on hbase/unassigned, then calls listChildrenAndWatchForNewChildren. As its name says, this function gets the whole list of unnasigned regions from ZK.
> The internal comments says it's only for split creation. I tried to remove it, it seems to work.
> The point is:
> - ZK does not allow us to know upfront if it's a split or not
> - 50% of the events will be deletions, we don't care about them but we can't know in advance
> - for large assignment, we will have a lot of children (hence events). Common practise in ZK is to limit the number of subnodes (see the threads http://www.mail-archive.com/zookeeper-user@hadoop.apache.org/msg01954.html)
> Monitoring the master, it was spending 10% of it's time on this during assignment. It seems it was not on the critical path (yet...), so removing it did not change the overall performances on a test scenario. In real life however, this typically adds workload to ZK, and ZK workload is precious on large failures... Same as master actually.
> If it's used only for split, we should have a znode specific for split and not a global znode.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)