You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by Jeff Zhang <zj...@gmail.com> on 2017/01/20 10:14:15 UTC
[Discuss] Move some interpreters out of zeppelin project
As we talk in another thread [1] about moving some interpreters out of
zeppelin project. I open this thread to discuss it in more details. I'd
like to raise 4 questions for this.
1. Do we need to do this
2. If the answer is yes, which interpreters should be moved out
3. How do we integrate these interpreters into zeppelin
4. How does zeppelin work with these third party interpreters
I will first give my inputs on this.
*1. Do we need to do this ?*
Personally, I strongly +1 on this. Several reasons:
- Keep the zeppelin project much smaller
- Each interpreter's improvements won't be blocked by the release of
zeppelin. Interpreters can has its own release cycle as long as
zeppelin-interpreter doesn't break the compatibility.
- Zeppelin developer don't have the knowledge of all interpreters.
Sometimes it is very difficult for zeppelin committers to review a new
interpreter that he doesn't know.
2. Which interpreters should be moved out ?
We can discuss it in another thread about the min package.
3. How do we integrate these interpreters into zeppelin
Currently, user can install third party interpreter by running script (
http://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/manual/interpreterinstallation.html#3rd-party-interpreters),
but this is not convienient, and it is hard to let every user to be aware
of this feature. So I think we should do that in zeppelin UI. We should
allow user to install/uninstall/upgrade/downgrade third party interpreters
in the interpreter page.
4. How does zeppelin work with these third party interpreters
Besides the interface zeppelin expose to the third party interpreter to be
install/uninstall/upgrade/downgrade, it is third party interpreter's own
responsibility to develop and make new release.
Please help comment on these 4 questions and feel free to add any things
that I miss.
[1]
https://lists.apache.org/thread.html/69f606409790d7ba11422e8c6df941a75c5dfae0aca63eccf2f840bf@%3Cusers.zeppelin.apache.org%3E
Re: [Discuss] Move some interpreters out of zeppelin project
Posted by moon soo Lee <mo...@apache.org>.
Thanks Jeff for staring the thread.
Here's my thoughts
1. Do we need to do this
yes.
2. If the answer is yes, which interpreters should be moved out
If Zeppelin community has no problem maintaining certain interpreter, then
no reason to remove contribution from community.
However, if Zeppelin community can not maintain well (e.g. not catching up
target system version update, bug report is not taken care, etc), then we
can consider move out non-maintainable code from community.
3. How do we integrate these interpreters into zeppelin
Helium package description [1] already reserved package type 'INTERPRETER'
for it. And i hope 'helium' becomes a place
finding/installing/uninstalling/upgrading all pluggable modules in
Zeppelin. I can make pullrequest quickly to support INTERPRETER
installation through helium gui menu.
4. How does zeppelin work with these third party interpreters
In the point of view of encouraging 3rd party interpreter,
after 3) is done, Zeppelin-netinst package will display community managed
interpreters and 3rd party interpreters together in helium menu.
And their installation procedure will be exactly the same. (click 'enable'
button and click 'ok' on confirm dialog).
So, user will not see any difference between using community managed
interpreter and using 3rd party interpreter.
And this encourage develop more 3rd party interpreters than community
managed interpreters, i think.
Thanks,
moon
[1]
https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumPackage.java#L40
On Fri, Jan 20, 2017 at 6:39 AM Jongyoul Lee <jo...@gmail.com> wrote:
> Hi Jeff,
>
> Thanks for starting this issue.
>
> It increases flexibility of improving interpreters itself but it can also
> decreases stability of interpreters. I'm worried about this side-effect. As
> you mentioned, it's hard for me to review new interpreter that I didn't use
> but it couldn't be a reason why we divide some code from Zeppelin. We have
> to make more ppl as committers to review various interpreters. Thus I don't
> want some interpreters out of Zeppelin.
>
> But I, totally, agree about #3, #4. If we deploy minimum package of
> Zeppelin, we have to provide GUI for install/uninstall. If it's done,
> bin-all-pkg is meaningless and bin-min-pkg is enough.
>
> On Fri, Jan 20, 2017 at 7:14 PM, Jeff Zhang <zj...@gmail.com> wrote:
>
> > As we talk in another thread [1] about moving some interpreters out of
> > zeppelin project. I open this thread to discuss it in more details. I'd
> > like to raise 4 questions for this.
> >
> > 1. Do we need to do this
> > 2. If the answer is yes, which interpreters should be moved out
> > 3. How do we integrate these interpreters into zeppelin
> > 4. How does zeppelin work with these third party interpreters
> >
> > I will first give my inputs on this.
> >
> > *1. Do we need to do this ?*
> > Personally, I strongly +1 on this. Several reasons:
> >
> > - Keep the zeppelin project much smaller
> > - Each interpreter's improvements won't be blocked by the release of
> > zeppelin. Interpreters can has its own release cycle as long as
> > zeppelin-interpreter doesn't break the compatibility.
> > - Zeppelin developer don't have the knowledge of all interpreters.
> > Sometimes it is very difficult for zeppelin committers to review a new
> > interpreter that he doesn't know.
> >
> >
> > 2. Which interpreters should be moved out ?
> > We can discuss it in another thread about the min package.
> >
> > 3. How do we integrate these interpreters into zeppelin
> > Currently, user can install third party interpreter by running script (
> > http://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/manual/
> > interpreterinstallation.html#3rd-party-interpreters), but this is not
> > convienient, and it is hard to let every user to be aware of this
> feature.
> > So I think we should do that in zeppelin UI. We should allow user to
> > install/uninstall/upgrade/downgrade third party interpreters in the
> > interpreter page.
> >
> > 4. How does zeppelin work with these third party interpreters
> > Besides the interface zeppelin expose to the third party interpreter to
> be
> > install/uninstall/upgrade/downgrade, it is third party interpreter's own
> > responsibility to develop and make new release.
> >
> > Please help comment on these 4 questions and feel free to add any things
> > that I miss.
> >
> >
> > [1] https://lists.apache.org/thread.html/69f606409790d7ba11422e8c6df941
> > a75c5dfae0aca63eccf2f840bf@%3Cusers.zeppelin.apache.org%3E
> >
>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>
Re: [Discuss] Move some interpreters out of zeppelin project
Posted by moon soo Lee <mo...@apache.org>.
Thanks Jeff for staring the thread.
Here's my thoughts
1. Do we need to do this
yes.
2. If the answer is yes, which interpreters should be moved out
If Zeppelin community has no problem maintaining certain interpreter, then
no reason to remove contribution from community.
However, if Zeppelin community can not maintain well (e.g. not catching up
target system version update, bug report is not taken care, etc), then we
can consider move out non-maintainable code from community.
3. How do we integrate these interpreters into zeppelin
Helium package description [1] already reserved package type 'INTERPRETER'
for it. And i hope 'helium' becomes a place
finding/installing/uninstalling/upgrading all pluggable modules in
Zeppelin. I can make pullrequest quickly to support INTERPRETER
installation through helium gui menu.
4. How does zeppelin work with these third party interpreters
In the point of view of encouraging 3rd party interpreter,
after 3) is done, Zeppelin-netinst package will display community managed
interpreters and 3rd party interpreters together in helium menu.
And their installation procedure will be exactly the same. (click 'enable'
button and click 'ok' on confirm dialog).
So, user will not see any difference between using community managed
interpreter and using 3rd party interpreter.
And this encourage develop more 3rd party interpreters than community
managed interpreters, i think.
Thanks,
moon
[1]
https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumPackage.java#L40
On Fri, Jan 20, 2017 at 6:39 AM Jongyoul Lee <jo...@gmail.com> wrote:
> Hi Jeff,
>
> Thanks for starting this issue.
>
> It increases flexibility of improving interpreters itself but it can also
> decreases stability of interpreters. I'm worried about this side-effect. As
> you mentioned, it's hard for me to review new interpreter that I didn't use
> but it couldn't be a reason why we divide some code from Zeppelin. We have
> to make more ppl as committers to review various interpreters. Thus I don't
> want some interpreters out of Zeppelin.
>
> But I, totally, agree about #3, #4. If we deploy minimum package of
> Zeppelin, we have to provide GUI for install/uninstall. If it's done,
> bin-all-pkg is meaningless and bin-min-pkg is enough.
>
> On Fri, Jan 20, 2017 at 7:14 PM, Jeff Zhang <zj...@gmail.com> wrote:
>
> > As we talk in another thread [1] about moving some interpreters out of
> > zeppelin project. I open this thread to discuss it in more details. I'd
> > like to raise 4 questions for this.
> >
> > 1. Do we need to do this
> > 2. If the answer is yes, which interpreters should be moved out
> > 3. How do we integrate these interpreters into zeppelin
> > 4. How does zeppelin work with these third party interpreters
> >
> > I will first give my inputs on this.
> >
> > *1. Do we need to do this ?*
> > Personally, I strongly +1 on this. Several reasons:
> >
> > - Keep the zeppelin project much smaller
> > - Each interpreter's improvements won't be blocked by the release of
> > zeppelin. Interpreters can has its own release cycle as long as
> > zeppelin-interpreter doesn't break the compatibility.
> > - Zeppelin developer don't have the knowledge of all interpreters.
> > Sometimes it is very difficult for zeppelin committers to review a new
> > interpreter that he doesn't know.
> >
> >
> > 2. Which interpreters should be moved out ?
> > We can discuss it in another thread about the min package.
> >
> > 3. How do we integrate these interpreters into zeppelin
> > Currently, user can install third party interpreter by running script (
> > http://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/manual/
> > interpreterinstallation.html#3rd-party-interpreters), but this is not
> > convienient, and it is hard to let every user to be aware of this
> feature.
> > So I think we should do that in zeppelin UI. We should allow user to
> > install/uninstall/upgrade/downgrade third party interpreters in the
> > interpreter page.
> >
> > 4. How does zeppelin work with these third party interpreters
> > Besides the interface zeppelin expose to the third party interpreter to
> be
> > install/uninstall/upgrade/downgrade, it is third party interpreter's own
> > responsibility to develop and make new release.
> >
> > Please help comment on these 4 questions and feel free to add any things
> > that I miss.
> >
> >
> > [1] https://lists.apache.org/thread.html/69f606409790d7ba11422e8c6df941
> > a75c5dfae0aca63eccf2f840bf@%3Cusers.zeppelin.apache.org%3E
> >
>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>
Re: [Discuss] Move some interpreters out of zeppelin project
Posted by Jongyoul Lee <jo...@gmail.com>.
Hi Jeff,
Thanks for starting this issue.
It increases flexibility of improving interpreters itself but it can also
decreases stability of interpreters. I'm worried about this side-effect. As
you mentioned, it's hard for me to review new interpreter that I didn't use
but it couldn't be a reason why we divide some code from Zeppelin. We have
to make more ppl as committers to review various interpreters. Thus I don't
want some interpreters out of Zeppelin.
But I, totally, agree about #3, #4. If we deploy minimum package of
Zeppelin, we have to provide GUI for install/uninstall. If it's done,
bin-all-pkg is meaningless and bin-min-pkg is enough.
On Fri, Jan 20, 2017 at 7:14 PM, Jeff Zhang <zj...@gmail.com> wrote:
> As we talk in another thread [1] about moving some interpreters out of
> zeppelin project. I open this thread to discuss it in more details. I'd
> like to raise 4 questions for this.
>
> 1. Do we need to do this
> 2. If the answer is yes, which interpreters should be moved out
> 3. How do we integrate these interpreters into zeppelin
> 4. How does zeppelin work with these third party interpreters
>
> I will first give my inputs on this.
>
> *1. Do we need to do this ?*
> Personally, I strongly +1 on this. Several reasons:
>
> - Keep the zeppelin project much smaller
> - Each interpreter's improvements won't be blocked by the release of
> zeppelin. Interpreters can has its own release cycle as long as
> zeppelin-interpreter doesn't break the compatibility.
> - Zeppelin developer don't have the knowledge of all interpreters.
> Sometimes it is very difficult for zeppelin committers to review a new
> interpreter that he doesn't know.
>
>
> 2. Which interpreters should be moved out ?
> We can discuss it in another thread about the min package.
>
> 3. How do we integrate these interpreters into zeppelin
> Currently, user can install third party interpreter by running script (
> http://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/manual/
> interpreterinstallation.html#3rd-party-interpreters), but this is not
> convienient, and it is hard to let every user to be aware of this feature.
> So I think we should do that in zeppelin UI. We should allow user to
> install/uninstall/upgrade/downgrade third party interpreters in the
> interpreter page.
>
> 4. How does zeppelin work with these third party interpreters
> Besides the interface zeppelin expose to the third party interpreter to be
> install/uninstall/upgrade/downgrade, it is third party interpreter's own
> responsibility to develop and make new release.
>
> Please help comment on these 4 questions and feel free to add any things
> that I miss.
>
>
> [1] https://lists.apache.org/thread.html/69f606409790d7ba11422e8c6df941
> a75c5dfae0aca63eccf2f840bf@%3Cusers.zeppelin.apache.org%3E
>
--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net
Re: [Discuss] Move some interpreters out of zeppelin project
Posted by Jongyoul Lee <jo...@gmail.com>.
Hi Jeff,
Thanks for starting this issue.
It increases flexibility of improving interpreters itself but it can also
decreases stability of interpreters. I'm worried about this side-effect. As
you mentioned, it's hard for me to review new interpreter that I didn't use
but it couldn't be a reason why we divide some code from Zeppelin. We have
to make more ppl as committers to review various interpreters. Thus I don't
want some interpreters out of Zeppelin.
But I, totally, agree about #3, #4. If we deploy minimum package of
Zeppelin, we have to provide GUI for install/uninstall. If it's done,
bin-all-pkg is meaningless and bin-min-pkg is enough.
On Fri, Jan 20, 2017 at 7:14 PM, Jeff Zhang <zj...@gmail.com> wrote:
> As we talk in another thread [1] about moving some interpreters out of
> zeppelin project. I open this thread to discuss it in more details. I'd
> like to raise 4 questions for this.
>
> 1. Do we need to do this
> 2. If the answer is yes, which interpreters should be moved out
> 3. How do we integrate these interpreters into zeppelin
> 4. How does zeppelin work with these third party interpreters
>
> I will first give my inputs on this.
>
> *1. Do we need to do this ?*
> Personally, I strongly +1 on this. Several reasons:
>
> - Keep the zeppelin project much smaller
> - Each interpreter's improvements won't be blocked by the release of
> zeppelin. Interpreters can has its own release cycle as long as
> zeppelin-interpreter doesn't break the compatibility.
> - Zeppelin developer don't have the knowledge of all interpreters.
> Sometimes it is very difficult for zeppelin committers to review a new
> interpreter that he doesn't know.
>
>
> 2. Which interpreters should be moved out ?
> We can discuss it in another thread about the min package.
>
> 3. How do we integrate these interpreters into zeppelin
> Currently, user can install third party interpreter by running script (
> http://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/manual/
> interpreterinstallation.html#3rd-party-interpreters), but this is not
> convienient, and it is hard to let every user to be aware of this feature.
> So I think we should do that in zeppelin UI. We should allow user to
> install/uninstall/upgrade/downgrade third party interpreters in the
> interpreter page.
>
> 4. How does zeppelin work with these third party interpreters
> Besides the interface zeppelin expose to the third party interpreter to be
> install/uninstall/upgrade/downgrade, it is third party interpreter's own
> responsibility to develop and make new release.
>
> Please help comment on these 4 questions and feel free to add any things
> that I miss.
>
>
> [1] https://lists.apache.org/thread.html/69f606409790d7ba11422e8c6df941
> a75c5dfae0aca63eccf2f840bf@%3Cusers.zeppelin.apache.org%3E
>
--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net