Re: Too-aggressive FULL_AUTO rebalancing? (maybe fixed @ master)

[Michael Craig <mc...@box.com>]

Ok I tracked it down: https://github.com/apache/helix/pull/58

Could use a careful review from someone who knows more about helix! :)

On Mon, Oct 31, 2016 at 11:42 AM, Michael Craig <mc...@box.com> wrote:

> I tried the newer helix-0.6.6 tag from GitHub and it did not resolve this.
> I'll try to investigate further today.
>
> On Mon, Oct 24, 2016 at 10:33 AM, Michael Craig <mc...@box.com> wrote:
>
>> Awesome! Thanks again for your help Kishore and Lei
>>
>> On Mon, Oct 24, 2016 at 10:16 AM, Lei Xia <lx...@linkedin.com> wrote:
>>
>>>   Okey, let me port the fix into 0.6.x.
>>>
>>>
>>> Thanks
>>> Lei
>>>
>>> On Mon, Oct 24, 2016 at 10:12 AM, kishore g <g....@gmail.com> wrote:
>>>
>>>> Thanks. Lei, can we apply this to 0.6.x branch before cutting the
>>>> release.
>>>>
>>>> On Mon, Oct 24, 2016 at 10:01 AM, Michael Craig <mc...@box.com> wrote:
>>>>
>>>>> Found it: https://github.com/apache/helix/commit/dc9f129b67f8cacdf
>>>>> 0cd22288f166b56fc5654a0
>>>>>
>>>>> This commit was not ported to the 0.6.x line. Here is the original
>>>>> JIRA issue: https://issues.apache.org/jira/browse/HELIX-543
>>>>>
>>>>> On Fri, Oct 21, 2016 at 5:08 PM, Michael Craig <mc...@box.com> wrote:
>>>>>
>>>>>> I'm not sure. The diff between 0.6.6 and master is enormous :(
>>>>>> https://github.com/apache/helix/compare/helix-0.6.6...master
>>>>>>
>>>>>> On Fri, Oct 21, 2016 at 4:30 PM, kishore g <g....@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Will take a look at it. Do you know what's the difference between
>>>>>>> master and 0.6.6 tag. We can pull that change into 0.6.6
>>>>>>>
>>>>>>> On Oct 21, 2016 4:10 PM, "Michael Craig" <mc...@box.com> wrote:
>>>>>>>
>>>>>>>> Ok. I tried the helix-0.6.6 tag from GH and found the issue is
>>>>>>>> still present:
>>>>>>>>
>>>>>>>> https://gist.github.com/mkscrg/628ab964995c0be914d44654d26ae
>>>>>>>> 561/5af298a63c6796d4f087bc345179ae1fd5aabc33
>>>>>>>>
>>>>>>>> On Fri, Oct 21, 2016 at 3:22 PM, kishore g <g....@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Yes it should be fixed in 0.6.6. Lei is working on the release.
>>>>>>>>>
>>>>>>>>> On Oct 21, 2016 1:52 PM, "Michael Craig" <mc...@box.com> wrote:
>>>>>>>>>
>>>>>>>>>> (This came up in a prior thread—moving it out to clarify it from
>>>>>>>>>> that other question.)
>>>>>>>>>>
>>>>>>>>>> With helix-0.6.5, FULL_AUTO rebalancing seems too aggressive when
>>>>>>>>>> nodes reconnect to the cluster. For example, with 2 nodes + 1 resource (1
>>>>>>>>>> replica, 1 partition) + OnlineOffline: https://gist.gi
>>>>>>>>>> thub.com/mkscrg/628ab964995c0be914d44654d26ae561/99348c870e9
>>>>>>>>>> f028048c1d1cfdd15976325f293f9
>>>>>>>>>>
>>>>>>>>>> However, this seems to be fixed at the current master branch on
>>>>>>>>>> GitHub: https://gist.github.com/mkscrg/628ab964995c0be914d44
>>>>>>>>>> 654d26ae561/ec26a64a74b50c8c125ccd1f9bde1d8aa848a0b5
>>>>>>>>>>
>>>>>>>>>> Will this fix be released in an 0.6.x version?
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> *Lei Xia *Senior Software Engineer
>>> Data Infra/Nuage & Helix
>>> LinkedIn
>>>
>>> lxia@linkedin.com
>>> www.linkedin.com/in/lxia1
>>>
>>
>>
>