Integration with Helix for 0.6.0

[kishore g <g....@gmail.com>]

Hi,

I have the changes in S4-110 branch. Definitely it needs more testing and
some tasks like supporting udp.

 We have the following options for 0.6.0 release.

   1. Integrate completely with Helix. Here again we have some options
   1. Maintain backward compatibility, have both existing commands work
      along with new instructions. Doable but more work.
      2. Break the compatibility, provide the new set of instructions. Easy
      but existing users might not like it.
   2. Integrate but dont announce the new features(ok to have bugs and less
   features), have a config flag to enable new set of features.
   3. Dont do any integration. Post pone to next release.

Not sure when we want to make the 0.6.0 release. I prefer doing 2 now and
then think of going either 1.1 or 1.2 route in the next release.

Thoughts ?

Thanks,
Kishore G

Re: Integration with Helix for 0.6.0

[kishore g <g....@gmail.com>]

Sounds good, lets do 1.1. Your comments make sense. I will create sub tasks
for the S4-110.

On Thu, Jan 10, 2013 at 8:50 AM, Matthieu Morel <mm...@apache.org> wrote:

> the

Re: Integration with Helix for 0.6.0

[Matthieu Morel <mm...@apache.org>]

Hi Kishore,

I am in favor of 1.1:

- as you point out the integration with Helix still needs more testing
- thus in my opinion, we should not replace the existing cluster management system with Helix.
- but we want to be able to use Helix's features!

I had a look at the changes and it seems we could quite easily remove the hardcoded paths forcing to use Helix by providing optional paths and by providing some implementations through DI.

This way we keep the current system, that works well, and provide the new approach, as an option with more features


I added some comments about the patch for S4 in the relevant ticket: https://issues.apache.org/jira/browse/S4-110


What would be great is to have the complete implementation of the remote stream handling, since we absolutely need that, including for basic test scenarios.


Regards,

Matthieu




On Jan 10, 2013, at 02:15 , kishore g wrote:

> Hi,
> 
> I have the changes in S4-110 branch. Definitely it needs more testing and
> some tasks like supporting udp.
> 
> We have the following options for 0.6.0 release.
> 
>   1. Integrate completely with Helix. Here again we have some options
>   1. Maintain backward compatibility, have both existing commands work
>      along with new instructions. Doable but more work.
>      2. Break the compatibility, provide the new set of instructions. Easy
>      but existing users might not like it.
>   2. Integrate but dont announce the new features(ok to have bugs and less
>   features), have a config flag to enable new set of features.
>   3. Dont do any integration. Post pone to next release.
> 
> Not sure when we want to make the 0.6.0 release. I prefer doing 2 now and
> then think of going either 1.1 or 1.2 route in the next release.
> 
> Thoughts ?
> 
> Thanks,
> Kishore G