You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by Ruslan Dautkhanov <da...@gmail.com> on 2017/04/03 04:04:44 UTC
Re: Roadmap for 0.8.0
That's exciting to see plans for 0.8.0 on the horizon.
Here's my top list for 0.8 :
- https://issues.apache.org/jira/browse/ZEPPELIN-2197 "Interpreter Idle
timeout"
This is a most-wanted feature by our Zeppelin admins. It was mentioned at
least once on this email chain.
- https://issues.apache.org/jira/browse/ZEPPELIN-1967 "Passing Z variables
to Shell Interpreter"
We had several of our users asking about this functionality. %sh and some
other interpreters can't be
parametrized by global variables. ZEPPELIN-1967 is one way of how this can
be solved.
- https://issues.apache.org/jira/browse/ZEPPELIN-1660 "Home directory
references (i.e. ~/zeppelin/) in zeppelin-env.sh don't work as expected"
Less of a critical compared to the above two, but it could complement the
multi-tenancy feature very well.
Best regards,
Ruslan Dautkhanov
On Wed, Mar 22, 2017 at 11:29 AM, Felix Cheung <fe...@hotmail.com>
wrote:
> +1 with latest/stable.
>
>
>
>
> ------------------------------
> *From:* moon soo Lee <mo...@apache.org>
> *Sent:* Tuesday, March 21, 2017 8:41:58 AM
> *To:* users@zeppelin.apache.org
> *Cc:* dev@zeppelin.apache.org
>
> *Subject:* Re: Roadmap for 0.8.0
>
> And if i suggest simplest way for us to set quality expectation to user,
> which will be labeling release in download page.
>
> Currently releases are divided into 2 categories in download page. 'Latest
> release' and 'Old releases'. I think we can treat 'Latest' as unstable and
> add one more category 'Stable release'.
>
> For example, once 0.8.0 is released,
>
> Latest release : 0.8.0
> Stable release : 0.7.1
> Old release : 0.6.2, 0.6.1 ....
>
> Once we feel confident about the stability of latest release, we can just
> change label from latest to stable in the download page. (and previous
> stable goes to old releases)
> We can even include formal vote for moving release from 'latest' to
> 'stable' in our release process, if it is necessary.
>
> Thanks,
> moon
>
> On Tue, Mar 21, 2017 at 6:59 AM moon soo Lee <mo...@apache.org> wrote:
>
>> Yes, having longer RC period will help.
>>
>> But if i recall 0.7.0 release, although 21 people participated verifying
>> through 4 RC for 15days, it wasn't enough to catch all critical problems
>> during the release process. After the release, we've got much more number
>> of bug reports, in next few days.
>>
>> Basically, verifying RC is limited to people who subscribe mailing list +
>> willing to contribute time to verify RC, which is much smaller number of
>> people who download release from download page. So having longer RC period
>> will definitely help and i think we should do, but I think it's still not
>> enough to make sure the quality, considering past history.
>>
>> AFAIK, releasing 0.8.0-preview, calling it unstable is up to the project.
>> ASF release process defines how to release source code, but it does not
>> really restrict what kind of 'version' the project should have releases.
>> For example, spark released spark-2.0.0-preview[1] before spark-2.0.0.
>>
>> Thanks,
>> moon
>>
>> [1] http://spark.apache.org/news/spark-2.0.0-preview.html
>>
>>
>> On Mon, Mar 20, 2017 at 11:31 PM Jongyoul Lee <jo...@gmail.com> wrote:
>>
>> I agree that it will help prolong RC period and use it actually. And also
>> we need code freeze for the new features and spend time to stabilize RC.
>>
>> On Tue, Mar 21, 2017 at 1:25 PM, Felix Cheung <fe...@hotmail.com>
>> wrote:
>>
>> +1 on quality and stabilization.
>>
>> I'm not sure if releasing as preview or calling it unstable fits with the
>> ASF release process though.
>>
>> Other projects have code freeze, RC (and longer RC iteration time) etc. -
>> do we think those will help improve quality when the release is finally cut?
>>
>>
>> _____________________________
>> From: Jianfeng (Jeff) Zhang <jz...@hortonworks.com>
>> Sent: Monday, March 20, 2017 6:13 PM
>> Subject: Re: Roadmap for 0.8.0
>> To: <us...@zeppelin.apache.org>, dev <de...@zeppelin.apache.org>
>>
>>
>>
>> Strongly +1 for adding system test for different interpreter modes and
>> focus on bug fixing than new features. I do heard from some users complain
>> about the bugs of zeppelin major release. A stabilized release is very
>> necessary for community.
>>
>>
>>
>>
>> Best Regard,
>> Jeff Zhang
>>
>>
>> From: moon soo Lee <moon@apache.org<mailto:moon@apache.org
>> <mo...@apache.org>>>
>> Reply-To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
>> <us...@zeppelin.apache.org>>" <users@zeppelin.apache.org<mai
>> lto:users@zeppelin.apache.org <us...@zeppelin.apache.org>>>
>> Date: Tuesday, March 21, 2017 at 4:10 AM
>> To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
>> <us...@zeppelin.apache.org>>" <users@zeppelin.apache.org<mai
>> lto:users@zeppelin.apache.org <us...@zeppelin.apache.org>>>, dev <
>> dev@zeppelin.apache.org<mailto:dev@zeppelin.apache.org
>> <de...@zeppelin.apache.org>>>
>>
>> Subject: Re: Roadmap for 0.8.0
>>
>> Great to see discussion for 0.8.0.
>> List of features for 0.8.0 looks really good.
>>
>> Interpreter factory refactoring
>> Interpreter layer supports various behavior depends on combination of
>> PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for
>> each combination as a first step.
>> Otherwise, any pullrequest will silently break one of behavior at any
>> time no matter we refactor or not. And fixing and testing this behavior is
>> so hard.
>> Once we have complete test cases, not only guarantee the behavior but
>> also make refactoring much easier.
>>
>>
>> 0.8.0 release
>> I'd like to suggest improvements on how we release a new version.
>>
>> In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3
>> months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months)
>>
>> I think the same thing will happen again with 0.8.0, while we're going to
>> make lots of changes and add many new features.
>> After we released 0.8.0, while 'Stabilizing' the new release, user who
>> tried the new release may get wrong impression of the quality. Which is
>> very bad and we already repeated the mistake in 0.6.0 and 0.7.0.
>>
>> So from 0.8.0 release, I'd suggest we improve way we release new version
>> to give user proper expectation. I think there're several ways of doing it.
>>
>> 1. Release 0.8.0-preview officially and then release 0.8.0.
>> 2. Release 0.8.0 with 'beta' or 'unstable' label. And keep 0.7.x as a
>> 'stable' release in the download page. Once 0.8.x release becomes stable
>> enough make 0.8.x release as a 'stable' and move 0.7.x to 'old' releases.
>>
>>
>> After 0.8.0,
>> Since Zeppelin projects starts, project went through some major
>> milestone, like
>>
>> - project gets first users and first contributor
>> - project went into Apache Incubator
>> - project became TLP.
>>
>> And I think it's time to think about hitting another major milestone.
>>
>> Considering features we already have, features we're planning on 0.8,
>> wide adoption of Zeppelin in the industry, I think it's time to focus on
>> make project more mature and make a 1.0 release. Which i think big
>> milestone for the project.
>>
>> After 0.8.0 release, I suggest we more focus on bug fixes, stability
>> improvement, optimizing user experience than adding new features. And with
>> subsequent minor release, 0.8.1, 0.8.2 ... moment we feel confident about
>> the quality, release it as a 1.0.0 instead of 0.8.x.
>>
>> Once we have 1.0.0 released, then I think we can make larger,
>> experimental changes on 2.0.0 branch aggressively, while we keep
>> maintaining 1.0.x branch.
>>
>>
>> Thanks,
>> moon
>>
>> On Mon, Mar 20, 2017 at 8:55 AM Felix Cheung <felixcheung_m@hotmail.com<
>> mailto:felixcheung_m@hotmail.com <fe...@hotmail.com>>> wrote:
>> There are several pending visualization improvements/PRs that would be
>> very good to get them in as well.
>>
>>
>> ________________________________
>> From: Jongyoul Lee <jongyoul@gmail.com<mailto:jongyoul@gmail.com
>> <jo...@gmail.com>>>
>> Sent: Sunday, March 19, 2017 9:03:24 PM
>> To: dev; users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
>> <us...@zeppelin.apache.org>>
>> Subject: Roadmap for 0.8.0
>>
>> Hi dev & users,
>>
>> Recently, community submits very new features for Apache Zeppelin. I
>> think it's very positive signals to improve Apache Zeppelin and its
>> community. But in another aspect, we should focus on what the next release
>> includes. I think we need to summarize and prioritize them. Here is what I
>> know:
>>
>> * Cluster management
>> * Admin feature
>> * Replace some context to separate users
>> * Helium online
>>
>> Feel free to talk if you want to add more things. I think we need to
>> choose which features will be included in 0.8.0, too.
>>
>> Regards,
>> Jongyoul Lee
>>
>> --
>> 이종열, Jongyoul Lee, 李宗烈
>> http://madeng.net
>>
>>
>>
>>
>>
>> --
>> 이종열, Jongyoul Lee, 李宗烈
>> http://madeng.net
>>
>>
Re: Roadmap for 0.8.0
Posted by Ruslan Dautkhanov <da...@gmail.com>.
Makes sense.
Thanks Jeff!
On Wed, Apr 5, 2017 at 11:18 PM Jeff Zhang <zj...@gmail.com> wrote:
>
> HI Ruslan,
>
> ZEPPELIN-1595 is trying to make ZeppelinContext extensible and available
> to other interpreters (Currently ZeppelinContext is only available in spark
> interpreter) . As you said you want to make {var1} as z.get(var1), that
> means you want to use ZeppelinContext in shell interpreter which require
> ZEPPELIN-1595.
>
>
>
> Ruslan Dautkhanov <da...@gmail.com>于2017年4月6日周四 上午5:46写道：
>
> Hi Jeff,
>
> I looked at the PR for ZEPPELIN-1595
> <https://issues.apache.org/jira/browse/ZEPPELIN-1595>
>
> It does not look it covers %sh interpreter.
>
> %sh and %sql interpters are somewhat unique as they don't have access to
> Zeppelin API (please correct me if I'm wrong)
>
> So what https://issues.apache.org/jira/browse/ZEPPELIN-1967 is
> suggesting, is to introduce syntax
> that is used in Jupyter notebooks, i.e. {*var1*} will be implied as
> z.get('var1'), for example:
>
> %sh
> /path/to/script --param8={*var1*} --param9={*var2*}
>
> where var1 and var2 would be implied to be fetched as z.get('var1')
> and z.get('var2') respectively.
>
>
> Or similarly for %sql :
>
> %sql
> create table dwh.table_{*year*} stores as parquet
> as
> select * from spark_df1 where year = {*year*}
>
> We miss a lot global variables for %sql and %sh so that a Zeppelin note
> can be used as a single parametrized
> orchestration for a whole workflow.
>
>
> Thank you,
> Ruslan Dautkhanov
>
> On Wed, Apr 5, 2017 at 12:01 AM, Jeff Zhang <zj...@gmail.com> wrote:
>
>
> Hi Ruslan,
>
> Regarding 'make zeppelinContext available in shell interpreter', you may
> want to check https://issues.apache.org/jira/browse/ZEPPELIN-1595
>
>
> Ruslan Dautkhanov <da...@gmail.com>于2017年4月3日周一 下午12:05写道：
>
> That's exciting to see plans for 0.8.0 on the horizon.
>
> Here's my top list for 0.8 :
>
> - https://issues.apache.org/jira/browse/ZEPPELIN-2197 "Interpreter Idle
> timeout"
> This is a most-wanted feature by our Zeppelin admins. It was mentioned at
> least once on this email chain.
>
> - https://issues.apache.org/jira/browse/ZEPPELIN-1967 "Passing Z
> variables to Shell Interpreter"
> We had several of our users asking about this functionality. %sh and some
> other interpreters can't be
> parametrized by global variables. ZEPPELIN-1967 is one way of how this
> can be solved.
>
> - https://issues.apache.org/jira/browse/ZEPPELIN-1660 "Home directory
> references (i.e. ~/zeppelin/) in zeppelin-env.sh don't work as expected"
> Less of a critical compared to the above two, but it could complement
> the multi-tenancy feature very well.
>
>
> Best regards,
> Ruslan Dautkhanov
>
> On Wed, Mar 22, 2017 at 11:29 AM, Felix Cheung <fe...@hotmail.com>
> wrote:
>
> +1 with latest/stable.
>
>
>
>
> ------------------------------
> *From:* moon soo Lee <mo...@apache.org>
> *Sent:* Tuesday, March 21, 2017 8:41:58 AM
> *To:* users@zeppelin.apache.org
> *Cc:* dev@zeppelin.apache.org
>
> *Subject:* Re: Roadmap for 0.8.0
>
>
> And if i suggest simplest way for us to set quality expectation to user,
> which will be labeling release in download page.
>
> Currently releases are divided into 2 categories in download page. 'Latest
> release' and 'Old releases'. I think we can treat 'Latest' as unstable and
> add one more category 'Stable release'.
>
> For example, once 0.8.0 is released,
>
> Latest release : 0.8.0
> Stable release : 0.7.1
> Old release : 0.6.2, 0.6.1 ....
>
> Once we feel confident about the stability of latest release, we can just
> change label from latest to stable in the download page. (and previous
> stable goes to old releases)
> We can even include formal vote for moving release from 'latest' to
> 'stable' in our release process, if it is necessary.
>
> Thanks,
> moon
>
> On Tue, Mar 21, 2017 at 6:59 AM moon soo Lee <mo...@apache.org> wrote:
>
> Yes, having longer RC period will help.
>
> But if i recall 0.7.0 release, although 21 people participated verifying
> through 4 RC for 15days, it wasn't enough to catch all critical problems
> during the release process. After the release, we've got much more number
> of bug reports, in next few days.
>
> Basically, verifying RC is limited to people who subscribe mailing list +
> willing to contribute time to verify RC, which is much smaller number of
> people who download release from download page. So having longer RC period
> will definitely help and i think we should do, but I think it's still not
> enough to make sure the quality, considering past history.
>
> AFAIK, releasing 0.8.0-preview, calling it unstable is up to the project.
> ASF release process defines how to release source code, but it does not
> really restrict what kind of 'version' the project should have releases.
> For example, spark released spark-2.0.0-preview[1] before spark-2.0.0.
>
> Thanks,
> moon
>
> [1] http://spark.apache.org/news/spark-2.0.0-preview.html
>
> On Mon, Mar 20, 2017 at 11:31 PM Jongyoul Lee <jo...@gmail.com> wrote:
>
> I agree that it will help prolong RC period and use it actually. And also
> we need code freeze for the new features and spend time to stabilize RC.
>
> On Tue, Mar 21, 2017 at 1:25 PM, Felix Cheung <fe...@hotmail.com>
> wrote:
>
> +1 on quality and stabilization.
>
> I'm not sure if releasing as preview or calling it unstable fits with the
> ASF release process though.
>
> Other projects have code freeze, RC (and longer RC iteration time) etc. -
> do we think those will help improve quality when the release is finally cut?
>
>
> _____________________________
> From: Jianfeng (Jeff) Zhang <jz...@hortonworks.com>
> Sent: Monday, March 20, 2017 6:13 PM
> Subject: Re: Roadmap for 0.8.0
>
> To: <us...@zeppelin.apache.org>, dev <de...@zeppelin.apache.org>
>
>
>
>
>
> Strongly +1 for adding system test for different interpreter modes and
> focus on bug fixing than new features. I do heard from some users complain
> about the bugs of zeppelin major release. A stabilized release is very
> necessary for community.
>
>
>
>
> Best Regard,
> Jeff Zhang
>
>
> From: moon soo Lee <moon@apache.org<mailto:moon@apache.org
> <mo...@apache.org>>>
>
> Reply-To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
> <us...@zeppelin.apache.org>>" <users@zeppelin.apache.org<
> mailto:users@zeppelin.apache.org <us...@zeppelin.apache.org>>>
>
>
> Date: Tuesday, March 21, 2017 at 4:10 AM
>
> To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
> <us...@zeppelin.apache.org>>" <users@zeppelin.apache.org<
> mailto:users@zeppelin.apache.org <us...@zeppelin.apache.org>>>, dev <
> dev@zeppelin.apache.org<mailto:dev@zeppelin.apache.org
> <de...@zeppelin.apache.org>>>
>
>
> Subject: Re: Roadmap for 0.8.0
>
> Great to see discussion for 0.8.0.
> List of features for 0.8.0 looks really good.
>
> Interpreter factory refactoring
> Interpreter layer supports various behavior depends on combination of
> PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for
> each combination as a first step.
> Otherwise, any pullrequest will silently break one of behavior at any time
> no matter we refactor or not. And fixing and testing this behavior is so
> hard.
> Once we have complete test cases, not only guarantee the behavior but also
> make refactoring much easier.
>
>
> 0.8.0 release
> I'd like to suggest improvements on how we release a new version.
>
> In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3
> months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months)
>
> I think the same thing will happen again with 0.8.0, while we're going to
> make lots of changes and add many new features.
> After we released 0.8.0, while 'Stabilizing' the new release, user who
> tried the new release may get wrong impression of the quality. Which is
> very bad and we already repeated the mistake in 0.6.0 and 0.7.0.
>
> So from 0.8.0 release, I'd suggest we improve way we release new version
> to give user proper expectation. I think there're several ways of doing it.
>
> 1. Release 0.8.0-preview officially and then release 0.8.0.
> 2. Release 0.8.0 with 'beta' or 'unstable' label. And keep 0.7.x as a
> 'stable' release in the download page. Once 0.8.x release becomes stable
> enough make 0.8.x release as a 'stable' and move 0.7.x to 'old' releases.
>
>
> After 0.8.0,
> Since Zeppelin projects starts, project went through some major milestone,
> like
>
> - project gets first users and first contributor
> - project went into Apache Incubator
> - project became TLP.
>
> And I think it's time to think about hitting another major milestone.
>
> Considering features we already have, features we're planning on 0.8, wide
> adoption of Zeppelin in the industry, I think it's time to focus on make
> project more mature and make a 1.0 release. Which i think big milestone for
> the project.
>
> After 0.8.0 release, I suggest we more focus on bug fixes, stability
> improvement, optimizing user experience than adding new features. And with
> subsequent minor release, 0.8.1, 0.8.2 ... moment we feel confident about
> the quality, release it as a 1.0.0 instead of 0.8.x.
>
> Once we have 1.0.0 released, then I think we can make larger, experimental
> changes on 2.0.0 branch aggressively, while we keep maintaining 1.0.x
> branch.
>
>
> Thanks,
> moon
>
> On Mon, Mar 20, 2017 at 8:55 AM Felix Cheung <felixcheung_m@hotmail.com<
> mailto:felixcheung_m@hotmail.com <fe...@hotmail.com>>> wrote:
> There are several pending visualization improvements/PRs that would be
> very good to get them in as well.
>
>
> ________________________________
> From: Jongyoul Lee <jongyoul@gmail.com<mailto:jongyoul@gmail.com
> <jo...@gmail.com>>>
> Sent: Sunday, March 19, 2017 9:03:24 PM
> To: dev; users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
> <us...@zeppelin.apache.org>>
> Subject: Roadmap for 0.8.0
>
> Hi dev & users,
>
> Recently, community submits very new features for Apache Zeppelin. I think
> it's very positive signals to improve Apache Zeppelin and its community.
> But in another aspect, we should focus on what the next release includes. I
> think we need to summarize and prioritize them. Here is what I know:
>
> * Cluster management
> * Admin feature
> * Replace some context to separate users
> * Helium online
>
> Feel free to talk if you want to add more things. I think we need to
> choose which features will be included in 0.8.0, too.
>
> Regards,
> Jongyoul Lee
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>
>
>
Re: Roadmap for 0.8.0
Posted by Jeff Zhang <zj...@gmail.com>.
HI Ruslan,
ZEPPELIN-1595 is trying to make ZeppelinContext extensible and available to
other interpreters (Currently ZeppelinContext is only available in spark
interpreter) . As you said you want to make {var1} as z.get(var1), that
means you want to use ZeppelinContext in shell interpreter which require
ZEPPELIN-1595.
Ruslan Dautkhanov <da...@gmail.com>于2017年4月6日周四 上午5:46写道：
> Hi Jeff,
>
> I looked at the PR for ZEPPELIN-1595
> <https://issues.apache.org/jira/browse/ZEPPELIN-1595>
>
> It does not look it covers %sh interpreter.
>
> %sh and %sql interpters are somewhat unique as they don't have access to
> Zeppelin API (please correct me if I'm wrong)
>
> So what https://issues.apache.org/jira/browse/ZEPPELIN-1967 is
> suggesting, is to introduce syntax
> that is used in Jupyter notebooks, i.e. {*var1*} will be implied as
> z.get('var1'), for example:
>
> %sh
> /path/to/script --param8={*var1*} --param9={*var2*}
>
> where var1 and var2 would be implied to be fetched as z.get('var1')
> and z.get('var2') respectively.
>
>
> Or similarly for %sql :
>
> %sql
> create table dwh.table_{*year*} stores as parquet
> as
> select * from spark_df1 where year = {*year*}
>
> We miss a lot global variables for %sql and %sh so that a Zeppelin note
> can be used as a single parametrized
> orchestration for a whole workflow.
>
>
> Thank you,
> Ruslan Dautkhanov
>
> On Wed, Apr 5, 2017 at 12:01 AM, Jeff Zhang <zj...@gmail.com> wrote:
>
>
> Hi Ruslan,
>
> Regarding 'make zeppelinContext available in shell interpreter', you may
> want to check https://issues.apache.org/jira/browse/ZEPPELIN-1595
>
>
> Ruslan Dautkhanov <da...@gmail.com>于2017年4月3日周一 下午12:05写道：
>
> That's exciting to see plans for 0.8.0 on the horizon.
>
> Here's my top list for 0.8 :
>
> - https://issues.apache.org/jira/browse/ZEPPELIN-2197 "Interpreter Idle
> timeout"
> This is a most-wanted feature by our Zeppelin admins. It was mentioned at
> least once on this email chain.
>
> - https://issues.apache.org/jira/browse/ZEPPELIN-1967 "Passing Z
> variables to Shell Interpreter"
> We had several of our users asking about this functionality. %sh and some
> other interpreters can't be
> parametrized by global variables. ZEPPELIN-1967 is one way of how this
> can be solved.
>
> - https://issues.apache.org/jira/browse/ZEPPELIN-1660 "Home directory
> references (i.e. ~/zeppelin/) in zeppelin-env.sh don't work as expected"
> Less of a critical compared to the above two, but it could complement
> the multi-tenancy feature very well.
>
>
> Best regards,
> Ruslan Dautkhanov
>
> On Wed, Mar 22, 2017 at 11:29 AM, Felix Cheung <fe...@hotmail.com>
> wrote:
>
> +1 with latest/stable.
>
>
>
>
> ------------------------------
> *From:* moon soo Lee <mo...@apache.org>
> *Sent:* Tuesday, March 21, 2017 8:41:58 AM
> *To:* users@zeppelin.apache.org
> *Cc:* dev@zeppelin.apache.org
>
> *Subject:* Re: Roadmap for 0.8.0
>
>
> And if i suggest simplest way for us to set quality expectation to user,
> which will be labeling release in download page.
>
> Currently releases are divided into 2 categories in download page. 'Latest
> release' and 'Old releases'. I think we can treat 'Latest' as unstable and
> add one more category 'Stable release'.
>
> For example, once 0.8.0 is released,
>
> Latest release : 0.8.0
> Stable release : 0.7.1
> Old release : 0.6.2, 0.6.1 ....
>
> Once we feel confident about the stability of latest release, we can just
> change label from latest to stable in the download page. (and previous
> stable goes to old releases)
> We can even include formal vote for moving release from 'latest' to
> 'stable' in our release process, if it is necessary.
>
> Thanks,
> moon
>
> On Tue, Mar 21, 2017 at 6:59 AM moon soo Lee <mo...@apache.org> wrote:
>
> Yes, having longer RC period will help.
>
> But if i recall 0.7.0 release, although 21 people participated verifying
> through 4 RC for 15days, it wasn't enough to catch all critical problems
> during the release process. After the release, we've got much more number
> of bug reports, in next few days.
>
> Basically, verifying RC is limited to people who subscribe mailing list +
> willing to contribute time to verify RC, which is much smaller number of
> people who download release from download page. So having longer RC period
> will definitely help and i think we should do, but I think it's still not
> enough to make sure the quality, considering past history.
>
> AFAIK, releasing 0.8.0-preview, calling it unstable is up to the project.
> ASF release process defines how to release source code, but it does not
> really restrict what kind of 'version' the project should have releases.
> For example, spark released spark-2.0.0-preview[1] before spark-2.0.0.
>
> Thanks,
> moon
>
> [1] http://spark.apache.org/news/spark-2.0.0-preview.html
>
> On Mon, Mar 20, 2017 at 11:31 PM Jongyoul Lee <jo...@gmail.com> wrote:
>
> I agree that it will help prolong RC period and use it actually. And also
> we need code freeze for the new features and spend time to stabilize RC.
>
> On Tue, Mar 21, 2017 at 1:25 PM, Felix Cheung <fe...@hotmail.com>
> wrote:
>
> +1 on quality and stabilization.
>
> I'm not sure if releasing as preview or calling it unstable fits with the
> ASF release process though.
>
> Other projects have code freeze, RC (and longer RC iteration time) etc. -
> do we think those will help improve quality when the release is finally cut?
>
>
> _____________________________
> From: Jianfeng (Jeff) Zhang <jz...@hortonworks.com>
> Sent: Monday, March 20, 2017 6:13 PM
> Subject: Re: Roadmap for 0.8.0
>
> To: <us...@zeppelin.apache.org>, dev <de...@zeppelin.apache.org>
>
>
>
>
>
> Strongly +1 for adding system test for different interpreter modes and
> focus on bug fixing than new features. I do heard from some users complain
> about the bugs of zeppelin major release. A stabilized release is very
> necessary for community.
>
>
>
>
> Best Regard,
> Jeff Zhang
>
>
> From: moon soo Lee <moon@apache.org<mailto:moon@apache.org
> <mo...@apache.org>>>
>
> Reply-To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
> <us...@zeppelin.apache.org>>" <users@zeppelin.apache.org<
> mailto:users@zeppelin.apache.org <us...@zeppelin.apache.org>>>
>
>
> Date: Tuesday, March 21, 2017 at 4:10 AM
>
> To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
> <us...@zeppelin.apache.org>>" <users@zeppelin.apache.org<
> mailto:users@zeppelin.apache.org <us...@zeppelin.apache.org>>>, dev <
> dev@zeppelin.apache.org<mailto:dev@zeppelin.apache.org
> <de...@zeppelin.apache.org>>>
>
>
> Subject: Re: Roadmap for 0.8.0
>
> Great to see discussion for 0.8.0.
> List of features for 0.8.0 looks really good.
>
> Interpreter factory refactoring
> Interpreter layer supports various behavior depends on combination of
> PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for
> each combination as a first step.
> Otherwise, any pullrequest will silently break one of behavior at any time
> no matter we refactor or not. And fixing and testing this behavior is so
> hard.
> Once we have complete test cases, not only guarantee the behavior but also
> make refactoring much easier.
>
>
> 0.8.0 release
> I'd like to suggest improvements on how we release a new version.
>
> In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3
> months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months)
>
> I think the same thing will happen again with 0.8.0, while we're going to
> make lots of changes and add many new features.
> After we released 0.8.0, while 'Stabilizing' the new release, user who
> tried the new release may get wrong impression of the quality. Which is
> very bad and we already repeated the mistake in 0.6.0 and 0.7.0.
>
> So from 0.8.0 release, I'd suggest we improve way we release new version
> to give user proper expectation. I think there're several ways of doing it.
>
> 1. Release 0.8.0-preview officially and then release 0.8.0.
> 2. Release 0.8.0 with 'beta' or 'unstable' label. And keep 0.7.x as a
> 'stable' release in the download page. Once 0.8.x release becomes stable
> enough make 0.8.x release as a 'stable' and move 0.7.x to 'old' releases.
>
>
> After 0.8.0,
> Since Zeppelin projects starts, project went through some major milestone,
> like
>
> - project gets first users and first contributor
> - project went into Apache Incubator
> - project became TLP.
>
> And I think it's time to think about hitting another major milestone.
>
> Considering features we already have, features we're planning on 0.8, wide
> adoption of Zeppelin in the industry, I think it's time to focus on make
> project more mature and make a 1.0 release. Which i think big milestone for
> the project.
>
> After 0.8.0 release, I suggest we more focus on bug fixes, stability
> improvement, optimizing user experience than adding new features. And with
> subsequent minor release, 0.8.1, 0.8.2 ... moment we feel confident about
> the quality, release it as a 1.0.0 instead of 0.8.x.
>
> Once we have 1.0.0 released, then I think we can make larger, experimental
> changes on 2.0.0 branch aggressively, while we keep maintaining 1.0.x
> branch.
>
>
> Thanks,
> moon
>
> On Mon, Mar 20, 2017 at 8:55 AM Felix Cheung <felixcheung_m@hotmail.com<
> mailto:felixcheung_m@hotmail.com <fe...@hotmail.com>>> wrote:
> There are several pending visualization improvements/PRs that would be
> very good to get them in as well.
>
>
> ________________________________
> From: Jongyoul Lee <jongyoul@gmail.com<mailto:jongyoul@gmail.com
> <jo...@gmail.com>>>
> Sent: Sunday, March 19, 2017 9:03:24 PM
> To: dev; users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
> <us...@zeppelin.apache.org>>
> Subject: Roadmap for 0.8.0
>
> Hi dev & users,
>
> Recently, community submits very new features for Apache Zeppelin. I think
> it's very positive signals to improve Apache Zeppelin and its community.
> But in another aspect, we should focus on what the next release includes. I
> think we need to summarize and prioritize them. Here is what I know:
>
> * Cluster management
> * Admin feature
> * Replace some context to separate users
> * Helium online
>
> Feel free to talk if you want to add more things. I think we need to
> choose which features will be included in 0.8.0, too.
>
> Regards,
> Jongyoul Lee
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>
>
>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>
>
>
Re: Roadmap for 0.8.0
Posted by Ruslan Dautkhanov <da...@gmail.com>.
Hi Jeff,
I looked at the PR for ZEPPELIN-1595
<https://issues.apache.org/jira/browse/ZEPPELIN-1595>
It does not look it covers %sh interpreter.
%sh and %sql interpters are somewhat unique as they don't have access to
Zeppelin API (please correct me if I'm wrong)
So what https://issues.apache.org/jira/browse/ZEPPELIN-1967 is suggesting,
is to introduce syntax
that is used in Jupyter notebooks, i.e. {*var1*} will be implied as
z.get('var1'), for example:
%sh
/path/to/script --param8={*var1*} --param9={*var2*}
where var1 and var2 would be implied to be fetched as z.get('var1')
and z.get('var2') respectively.
Or similarly for %sql :
%sql
create table dwh.table_{*year*} stores as parquet
as
select * from spark_df1 where year = {*year*}
We miss a lot global variables for %sql and %sh so that a Zeppelin note can
be used as a single parametrized
orchestration for a whole workflow.
Thank you,
Ruslan Dautkhanov
On Wed, Apr 5, 2017 at 12:01 AM, Jeff Zhang <zj...@gmail.com> wrote:
>
> Hi Ruslan,
>
> Regarding 'make zeppelinContext available in shell interpreter', you may
> want to check https://issues.apache.org/jira/browse/ZEPPELIN-1595
>
>
> Ruslan Dautkhanov <da...@gmail.com>于2017年4月3日周一 下午12:05写道：
>
>> That's exciting to see plans for 0.8.0 on the horizon.
>>
>> Here's my top list for 0.8 :
>>
>> - https://issues.apache.org/jira/browse/ZEPPELIN-2197 "Interpreter Idle
>> timeout"
>> This is a most-wanted feature by our Zeppelin admins. It was mentioned
>> at least once on this email chain.
>>
>> - https://issues.apache.org/jira/browse/ZEPPELIN-1967 "Passing Z
>> variables to Shell Interpreter"
>> We had several of our users asking about this functionality. %sh and
>> some other interpreters can't be
>> parametrized by global variables. ZEPPELIN-1967 is one way of how this
>> can be solved.
>>
>> - https://issues.apache.org/jira/browse/ZEPPELIN-1660 "Home directory
>> references (i.e. ~/zeppelin/) in zeppelin-env.sh don't work as expected"
>> Less of a critical compared to the above two, but it could complement
>> the multi-tenancy feature very well.
>>
>>
>> Best regards,
>> Ruslan Dautkhanov
>>
>> On Wed, Mar 22, 2017 at 11:29 AM, Felix Cheung <felixcheung_m@hotmail.com
>> > wrote:
>>
>> +1 with latest/stable.
>>
>>
>>
>>
>> ------------------------------
>> *From:* moon soo Lee <mo...@apache.org>
>> *Sent:* Tuesday, March 21, 2017 8:41:58 AM
>> *To:* users@zeppelin.apache.org
>> *Cc:* dev@zeppelin.apache.org
>>
>> *Subject:* Re: Roadmap for 0.8.0
>>
>>
>> And if i suggest simplest way for us to set quality expectation to user,
>> which will be labeling release in download page.
>>
>> Currently releases are divided into 2 categories in download page.
>> 'Latest release' and 'Old releases'. I think we can treat 'Latest' as
>> unstable and add one more category 'Stable release'.
>>
>> For example, once 0.8.0 is released,
>>
>> Latest release : 0.8.0
>> Stable release : 0.7.1
>> Old release : 0.6.2, 0.6.1 ....
>>
>> Once we feel confident about the stability of latest release, we can just
>> change label from latest to stable in the download page. (and previous
>> stable goes to old releases)
>> We can even include formal vote for moving release from 'latest' to
>> 'stable' in our release process, if it is necessary.
>>
>> Thanks,
>> moon
>>
>> On Tue, Mar 21, 2017 at 6:59 AM moon soo Lee <mo...@apache.org> wrote:
>>
>> Yes, having longer RC period will help.
>>
>> But if i recall 0.7.0 release, although 21 people participated verifying
>> through 4 RC for 15days, it wasn't enough to catch all critical problems
>> during the release process. After the release, we've got much more number
>> of bug reports, in next few days.
>>
>> Basically, verifying RC is limited to people who subscribe mailing list +
>> willing to contribute time to verify RC, which is much smaller number of
>> people who download release from download page. So having longer RC period
>> will definitely help and i think we should do, but I think it's still not
>> enough to make sure the quality, considering past history.
>>
>> AFAIK, releasing 0.8.0-preview, calling it unstable is up to the project.
>> ASF release process defines how to release source code, but it does not
>> really restrict what kind of 'version' the project should have releases.
>> For example, spark released spark-2.0.0-preview[1] before spark-2.0.0.
>>
>> Thanks,
>> moon
>>
>> [1] http://spark.apache.org/news/spark-2.0.0-preview.html
>>
>> On Mon, Mar 20, 2017 at 11:31 PM Jongyoul Lee <jo...@gmail.com> wrote:
>>
>> I agree that it will help prolong RC period and use it actually. And also
>> we need code freeze for the new features and spend time to stabilize RC.
>>
>> On Tue, Mar 21, 2017 at 1:25 PM, Felix Cheung <fe...@hotmail.com>
>> wrote:
>>
>> +1 on quality and stabilization.
>>
>> I'm not sure if releasing as preview or calling it unstable fits with the
>> ASF release process though.
>>
>> Other projects have code freeze, RC (and longer RC iteration time) etc. -
>> do we think those will help improve quality when the release is finally cut?
>>
>>
>> _____________________________
>> From: Jianfeng (Jeff) Zhang <jz...@hortonworks.com>
>> Sent: Monday, March 20, 2017 6:13 PM
>> Subject: Re: Roadmap for 0.8.0
>>
>> To: <us...@zeppelin.apache.org>, dev <de...@zeppelin.apache.org>
>>
>>
>>
>>
>>
>> Strongly +1 for adding system test for different interpreter modes and
>> focus on bug fixing than new features. I do heard from some users complain
>> about the bugs of zeppelin major release. A stabilized release is very
>> necessary for community.
>>
>>
>>
>>
>> Best Regard,
>> Jeff Zhang
>>
>>
>> From: moon soo Lee <moon@apache.org<mailto:moon@apache.org
>> <mo...@apache.org>>>
>>
>> Reply-To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
>> <us...@zeppelin.apache.org>>" <users@zeppelin.apache.org<mai
>> lto:users@zeppelin.apache.org <us...@zeppelin.apache.org>>>
>>
>>
>> Date: Tuesday, March 21, 2017 at 4:10 AM
>>
>> To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
>> <us...@zeppelin.apache.org>>" <users@zeppelin.apache.org<mai
>> lto:users@zeppelin.apache.org <us...@zeppelin.apache.org>>>, dev <
>> dev@zeppelin.apache.org<mailto:dev@zeppelin.apache.org
>> <de...@zeppelin.apache.org>>>
>>
>>
>> Subject: Re: Roadmap for 0.8.0
>>
>> Great to see discussion for 0.8.0.
>> List of features for 0.8.0 looks really good.
>>
>> Interpreter factory refactoring
>> Interpreter layer supports various behavior depends on combination of
>> PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for
>> each combination as a first step.
>> Otherwise, any pullrequest will silently break one of behavior at any
>> time no matter we refactor or not. And fixing and testing this behavior is
>> so hard.
>> Once we have complete test cases, not only guarantee the behavior but
>> also make refactoring much easier.
>>
>>
>> 0.8.0 release
>> I'd like to suggest improvements on how we release a new version.
>>
>> In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3
>> months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months)
>>
>> I think the same thing will happen again with 0.8.0, while we're going to
>> make lots of changes and add many new features.
>> After we released 0.8.0, while 'Stabilizing' the new release, user who
>> tried the new release may get wrong impression of the quality. Which is
>> very bad and we already repeated the mistake in 0.6.0 and 0.7.0.
>>
>> So from 0.8.0 release, I'd suggest we improve way we release new version
>> to give user proper expectation. I think there're several ways of doing it.
>>
>> 1. Release 0.8.0-preview officially and then release 0.8.0.
>> 2. Release 0.8.0 with 'beta' or 'unstable' label. And keep 0.7.x as a
>> 'stable' release in the download page. Once 0.8.x release becomes stable
>> enough make 0.8.x release as a 'stable' and move 0.7.x to 'old' releases.
>>
>>
>> After 0.8.0,
>> Since Zeppelin projects starts, project went through some major
>> milestone, like
>>
>> - project gets first users and first contributor
>> - project went into Apache Incubator
>> - project became TLP.
>>
>> And I think it's time to think about hitting another major milestone.
>>
>> Considering features we already have, features we're planning on 0.8,
>> wide adoption of Zeppelin in the industry, I think it's time to focus on
>> make project more mature and make a 1.0 release. Which i think big
>> milestone for the project.
>>
>> After 0.8.0 release, I suggest we more focus on bug fixes, stability
>> improvement, optimizing user experience than adding new features. And with
>> subsequent minor release, 0.8.1, 0.8.2 ... moment we feel confident about
>> the quality, release it as a 1.0.0 instead of 0.8.x.
>>
>> Once we have 1.0.0 released, then I think we can make larger,
>> experimental changes on 2.0.0 branch aggressively, while we keep
>> maintaining 1.0.x branch.
>>
>>
>> Thanks,
>> moon
>>
>> On Mon, Mar 20, 2017 at 8:55 AM Felix Cheung <felixcheung_m@hotmail.com<
>> mailto:felixcheung_m@hotmail.com <fe...@hotmail.com>>> wrote:
>> There are several pending visualization improvements/PRs that would be
>> very good to get them in as well.
>>
>>
>> ________________________________
>> From: Jongyoul Lee <jongyoul@gmail.com<mailto:jongyoul@gmail.com
>> <jo...@gmail.com>>>
>> Sent: Sunday, March 19, 2017 9:03:24 PM
>> To: dev; users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
>> <us...@zeppelin.apache.org>>
>> Subject: Roadmap for 0.8.0
>>
>> Hi dev & users,
>>
>> Recently, community submits very new features for Apache Zeppelin. I
>> think it's very positive signals to improve Apache Zeppelin and its
>> community. But in another aspect, we should focus on what the next release
>> includes. I think we need to summarize and prioritize them. Here is what I
>> know:
>>
>> * Cluster management
>> * Admin feature
>> * Replace some context to separate users
>> * Helium online
>>
>> Feel free to talk if you want to add more things. I think we need to
>> choose which features will be included in 0.8.0, too.
>>
>> Regards,
>> Jongyoul Lee
>>
>> --
>> 이종열, Jongyoul Lee, 李宗烈
>> http://madeng.net
>>
>>
>>
>>
>>
>> --
>> 이종열, Jongyoul Lee, 李宗烈
>> http://madeng.net
>>
>>
Re: Roadmap for 0.8.0
Posted by Ruslan Dautkhanov <da...@gmail.com>.
Hi Jeff,
I looked at the PR for ZEPPELIN-1595
<https://issues.apache.org/jira/browse/ZEPPELIN-1595>
It does not look it covers %sh interpreter.
%sh and %sql interpters are somewhat unique as they don't have access to
Zeppelin API (please correct me if I'm wrong)
So what https://issues.apache.org/jira/browse/ZEPPELIN-1967 is suggesting,
is to introduce syntax
that is used in Jupyter notebooks, i.e. {*var1*} will be implied as
z.get('var1'), for example:
%sh
/path/to/script --param8={*var1*} --param9={*var2*}
where var1 and var2 would be implied to be fetched as z.get('var1')
and z.get('var2') respectively.
Or similarly for %sql :
%sql
create table dwh.table_{*year*} stores as parquet
as
select * from spark_df1 where year = {*year*}
We miss a lot global variables for %sql and %sh so that a Zeppelin note can
be used as a single parametrized
orchestration for a whole workflow.
Thank you,
Ruslan Dautkhanov
On Wed, Apr 5, 2017 at 12:01 AM, Jeff Zhang <zj...@gmail.com> wrote:
>
> Hi Ruslan,
>
> Regarding 'make zeppelinContext available in shell interpreter', you may
> want to check https://issues.apache.org/jira/browse/ZEPPELIN-1595
>
>
> Ruslan Dautkhanov <da...@gmail.com>于2017年4月3日周一 下午12:05写道：
>
>> That's exciting to see plans for 0.8.0 on the horizon.
>>
>> Here's my top list for 0.8 :
>>
>> - https://issues.apache.org/jira/browse/ZEPPELIN-2197 "Interpreter Idle
>> timeout"
>> This is a most-wanted feature by our Zeppelin admins. It was mentioned
>> at least once on this email chain.
>>
>> - https://issues.apache.org/jira/browse/ZEPPELIN-1967 "Passing Z
>> variables to Shell Interpreter"
>> We had several of our users asking about this functionality. %sh and
>> some other interpreters can't be
>> parametrized by global variables. ZEPPELIN-1967 is one way of how this
>> can be solved.
>>
>> - https://issues.apache.org/jira/browse/ZEPPELIN-1660 "Home directory
>> references (i.e. ~/zeppelin/) in zeppelin-env.sh don't work as expected"
>> Less of a critical compared to the above two, but it could complement
>> the multi-tenancy feature very well.
>>
>>
>> Best regards,
>> Ruslan Dautkhanov
>>
>> On Wed, Mar 22, 2017 at 11:29 AM, Felix Cheung <felixcheung_m@hotmail.com
>> > wrote:
>>
>> +1 with latest/stable.
>>
>>
>>
>>
>> ------------------------------
>> *From:* moon soo Lee <mo...@apache.org>
>> *Sent:* Tuesday, March 21, 2017 8:41:58 AM
>> *To:* users@zeppelin.apache.org
>> *Cc:* dev@zeppelin.apache.org
>>
>> *Subject:* Re: Roadmap for 0.8.0
>>
>>
>> And if i suggest simplest way for us to set quality expectation to user,
>> which will be labeling release in download page.
>>
>> Currently releases are divided into 2 categories in download page.
>> 'Latest release' and 'Old releases'. I think we can treat 'Latest' as
>> unstable and add one more category 'Stable release'.
>>
>> For example, once 0.8.0 is released,
>>
>> Latest release : 0.8.0
>> Stable release : 0.7.1
>> Old release : 0.6.2, 0.6.1 ....
>>
>> Once we feel confident about the stability of latest release, we can just
>> change label from latest to stable in the download page. (and previous
>> stable goes to old releases)
>> We can even include formal vote for moving release from 'latest' to
>> 'stable' in our release process, if it is necessary.
>>
>> Thanks,
>> moon
>>
>> On Tue, Mar 21, 2017 at 6:59 AM moon soo Lee <mo...@apache.org> wrote:
>>
>> Yes, having longer RC period will help.
>>
>> But if i recall 0.7.0 release, although 21 people participated verifying
>> through 4 RC for 15days, it wasn't enough to catch all critical problems
>> during the release process. After the release, we've got much more number
>> of bug reports, in next few days.
>>
>> Basically, verifying RC is limited to people who subscribe mailing list +
>> willing to contribute time to verify RC, which is much smaller number of
>> people who download release from download page. So having longer RC period
>> will definitely help and i think we should do, but I think it's still not
>> enough to make sure the quality, considering past history.
>>
>> AFAIK, releasing 0.8.0-preview, calling it unstable is up to the project.
>> ASF release process defines how to release source code, but it does not
>> really restrict what kind of 'version' the project should have releases.
>> For example, spark released spark-2.0.0-preview[1] before spark-2.0.0.
>>
>> Thanks,
>> moon
>>
>> [1] http://spark.apache.org/news/spark-2.0.0-preview.html
>>
>> On Mon, Mar 20, 2017 at 11:31 PM Jongyoul Lee <jo...@gmail.com> wrote:
>>
>> I agree that it will help prolong RC period and use it actually. And also
>> we need code freeze for the new features and spend time to stabilize RC.
>>
>> On Tue, Mar 21, 2017 at 1:25 PM, Felix Cheung <fe...@hotmail.com>
>> wrote:
>>
>> +1 on quality and stabilization.
>>
>> I'm not sure if releasing as preview or calling it unstable fits with the
>> ASF release process though.
>>
>> Other projects have code freeze, RC (and longer RC iteration time) etc. -
>> do we think those will help improve quality when the release is finally cut?
>>
>>
>> _____________________________
>> From: Jianfeng (Jeff) Zhang <jz...@hortonworks.com>
>> Sent: Monday, March 20, 2017 6:13 PM
>> Subject: Re: Roadmap for 0.8.0
>>
>> To: <us...@zeppelin.apache.org>, dev <de...@zeppelin.apache.org>
>>
>>
>>
>>
>>
>> Strongly +1 for adding system test for different interpreter modes and
>> focus on bug fixing than new features. I do heard from some users complain
>> about the bugs of zeppelin major release. A stabilized release is very
>> necessary for community.
>>
>>
>>
>>
>> Best Regard,
>> Jeff Zhang
>>
>>
>> From: moon soo Lee <moon@apache.org<mailto:moon@apache.org
>> <mo...@apache.org>>>
>>
>> Reply-To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
>> <us...@zeppelin.apache.org>>" <users@zeppelin.apache.org<mai
>> lto:users@zeppelin.apache.org <us...@zeppelin.apache.org>>>
>>
>>
>> Date: Tuesday, March 21, 2017 at 4:10 AM
>>
>> To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
>> <us...@zeppelin.apache.org>>" <users@zeppelin.apache.org<mai
>> lto:users@zeppelin.apache.org <us...@zeppelin.apache.org>>>, dev <
>> dev@zeppelin.apache.org<mailto:dev@zeppelin.apache.org
>> <de...@zeppelin.apache.org>>>
>>
>>
>> Subject: Re: Roadmap for 0.8.0
>>
>> Great to see discussion for 0.8.0.
>> List of features for 0.8.0 looks really good.
>>
>> Interpreter factory refactoring
>> Interpreter layer supports various behavior depends on combination of
>> PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for
>> each combination as a first step.
>> Otherwise, any pullrequest will silently break one of behavior at any
>> time no matter we refactor or not. And fixing and testing this behavior is
>> so hard.
>> Once we have complete test cases, not only guarantee the behavior but
>> also make refactoring much easier.
>>
>>
>> 0.8.0 release
>> I'd like to suggest improvements on how we release a new version.
>>
>> In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3
>> months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months)
>>
>> I think the same thing will happen again with 0.8.0, while we're going to
>> make lots of changes and add many new features.
>> After we released 0.8.0, while 'Stabilizing' the new release, user who
>> tried the new release may get wrong impression of the quality. Which is
>> very bad and we already repeated the mistake in 0.6.0 and 0.7.0.
>>
>> So from 0.8.0 release, I'd suggest we improve way we release new version
>> to give user proper expectation. I think there're several ways of doing it.
>>
>> 1. Release 0.8.0-preview officially and then release 0.8.0.
>> 2. Release 0.8.0 with 'beta' or 'unstable' label. And keep 0.7.x as a
>> 'stable' release in the download page. Once 0.8.x release becomes stable
>> enough make 0.8.x release as a 'stable' and move 0.7.x to 'old' releases.
>>
>>
>> After 0.8.0,
>> Since Zeppelin projects starts, project went through some major
>> milestone, like
>>
>> - project gets first users and first contributor
>> - project went into Apache Incubator
>> - project became TLP.
>>
>> And I think it's time to think about hitting another major milestone.
>>
>> Considering features we already have, features we're planning on 0.8,
>> wide adoption of Zeppelin in the industry, I think it's time to focus on
>> make project more mature and make a 1.0 release. Which i think big
>> milestone for the project.
>>
>> After 0.8.0 release, I suggest we more focus on bug fixes, stability
>> improvement, optimizing user experience than adding new features. And with
>> subsequent minor release, 0.8.1, 0.8.2 ... moment we feel confident about
>> the quality, release it as a 1.0.0 instead of 0.8.x.
>>
>> Once we have 1.0.0 released, then I think we can make larger,
>> experimental changes on 2.0.0 branch aggressively, while we keep
>> maintaining 1.0.x branch.
>>
>>
>> Thanks,
>> moon
>>
>> On Mon, Mar 20, 2017 at 8:55 AM Felix Cheung <felixcheung_m@hotmail.com<
>> mailto:felixcheung_m@hotmail.com <fe...@hotmail.com>>> wrote:
>> There are several pending visualization improvements/PRs that would be
>> very good to get them in as well.
>>
>>
>> ________________________________
>> From: Jongyoul Lee <jongyoul@gmail.com<mailto:jongyoul@gmail.com
>> <jo...@gmail.com>>>
>> Sent: Sunday, March 19, 2017 9:03:24 PM
>> To: dev; users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
>> <us...@zeppelin.apache.org>>
>> Subject: Roadmap for 0.8.0
>>
>> Hi dev & users,
>>
>> Recently, community submits very new features for Apache Zeppelin. I
>> think it's very positive signals to improve Apache Zeppelin and its
>> community. But in another aspect, we should focus on what the next release
>> includes. I think we need to summarize and prioritize them. Here is what I
>> know:
>>
>> * Cluster management
>> * Admin feature
>> * Replace some context to separate users
>> * Helium online
>>
>> Feel free to talk if you want to add more things. I think we need to
>> choose which features will be included in 0.8.0, too.
>>
>> Regards,
>> Jongyoul Lee
>>
>> --
>> 이종열, Jongyoul Lee, 李宗烈
>> http://madeng.net
>>
>>
>>
>>
>>
>> --
>> 이종열, Jongyoul Lee, 李宗烈
>> http://madeng.net
>>
>>
Re: Roadmap for 0.8.0
Posted by Jeff Zhang <zj...@gmail.com>.
Hi Ruslan,
Regarding 'make zeppelinContext available in shell interpreter', you may
want to check https://issues.apache.org/jira/browse/ZEPPELIN-1595
Ruslan Dautkhanov <da...@gmail.com>于2017年4月3日周一 下午12:05写道：
> That's exciting to see plans for 0.8.0 on the horizon.
>
> Here's my top list for 0.8 :
>
> - https://issues.apache.org/jira/browse/ZEPPELIN-2197 "Interpreter Idle
> timeout"
> This is a most-wanted feature by our Zeppelin admins. It was mentioned at
> least once on this email chain.
>
> - https://issues.apache.org/jira/browse/ZEPPELIN-1967 "Passing Z
> variables to Shell Interpreter"
> We had several of our users asking about this functionality. %sh and some
> other interpreters can't be
> parametrized by global variables. ZEPPELIN-1967 is one way of how this
> can be solved.
>
> - https://issues.apache.org/jira/browse/ZEPPELIN-1660 "Home directory
> references (i.e. ~/zeppelin/) in zeppelin-env.sh don't work as expected"
> Less of a critical compared to the above two, but it could complement
> the multi-tenancy feature very well.
>
>
> Best regards,
> Ruslan Dautkhanov
>
> On Wed, Mar 22, 2017 at 11:29 AM, Felix Cheung <fe...@hotmail.com>
> wrote:
>
> +1 with latest/stable.
>
>
>
>
> ------------------------------
> *From:* moon soo Lee <mo...@apache.org>
> *Sent:* Tuesday, March 21, 2017 8:41:58 AM
> *To:* users@zeppelin.apache.org
> *Cc:* dev@zeppelin.apache.org
>
> *Subject:* Re: Roadmap for 0.8.0
>
>
> And if i suggest simplest way for us to set quality expectation to user,
> which will be labeling release in download page.
>
> Currently releases are divided into 2 categories in download page. 'Latest
> release' and 'Old releases'. I think we can treat 'Latest' as unstable and
> add one more category 'Stable release'.
>
> For example, once 0.8.0 is released,
>
> Latest release : 0.8.0
> Stable release : 0.7.1
> Old release : 0.6.2, 0.6.1 ....
>
> Once we feel confident about the stability of latest release, we can just
> change label from latest to stable in the download page. (and previous
> stable goes to old releases)
> We can even include formal vote for moving release from 'latest' to
> 'stable' in our release process, if it is necessary.
>
> Thanks,
> moon
>
> On Tue, Mar 21, 2017 at 6:59 AM moon soo Lee <mo...@apache.org> wrote:
>
> Yes, having longer RC period will help.
>
> But if i recall 0.7.0 release, although 21 people participated verifying
> through 4 RC for 15days, it wasn't enough to catch all critical problems
> during the release process. After the release, we've got much more number
> of bug reports, in next few days.
>
> Basically, verifying RC is limited to people who subscribe mailing list +
> willing to contribute time to verify RC, which is much smaller number of
> people who download release from download page. So having longer RC period
> will definitely help and i think we should do, but I think it's still not
> enough to make sure the quality, considering past history.
>
> AFAIK, releasing 0.8.0-preview, calling it unstable is up to the project.
> ASF release process defines how to release source code, but it does not
> really restrict what kind of 'version' the project should have releases.
> For example, spark released spark-2.0.0-preview[1] before spark-2.0.0.
>
> Thanks,
> moon
>
> [1] http://spark.apache.org/news/spark-2.0.0-preview.html
>
> On Mon, Mar 20, 2017 at 11:31 PM Jongyoul Lee <jo...@gmail.com> wrote:
>
> I agree that it will help prolong RC period and use it actually. And also
> we need code freeze for the new features and spend time to stabilize RC.
>
> On Tue, Mar 21, 2017 at 1:25 PM, Felix Cheung <fe...@hotmail.com>
> wrote:
>
> +1 on quality and stabilization.
>
> I'm not sure if releasing as preview or calling it unstable fits with the
> ASF release process though.
>
> Other projects have code freeze, RC (and longer RC iteration time) etc. -
> do we think those will help improve quality when the release is finally cut?
>
>
> _____________________________
> From: Jianfeng (Jeff) Zhang <jz...@hortonworks.com>
> Sent: Monday, March 20, 2017 6:13 PM
> Subject: Re: Roadmap for 0.8.0
>
> To: <us...@zeppelin.apache.org>, dev <de...@zeppelin.apache.org>
>
>
>
>
>
> Strongly +1 for adding system test for different interpreter modes and
> focus on bug fixing than new features. I do heard from some users complain
> about the bugs of zeppelin major release. A stabilized release is very
> necessary for community.
>
>
>
>
> Best Regard,
> Jeff Zhang
>
>
> From: moon soo Lee <moon@apache.org<mailto:moon@apache.org
> <mo...@apache.org>>>
>
> Reply-To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
> <us...@zeppelin.apache.org>>" <users@zeppelin.apache.org<
> mailto:users@zeppelin.apache.org <us...@zeppelin.apache.org>>>
>
>
> Date: Tuesday, March 21, 2017 at 4:10 AM
>
> To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
> <us...@zeppelin.apache.org>>" <users@zeppelin.apache.org<
> mailto:users@zeppelin.apache.org <us...@zeppelin.apache.org>>>, dev <
> dev@zeppelin.apache.org<mailto:dev@zeppelin.apache.org
> <de...@zeppelin.apache.org>>>
>
>
> Subject: Re: Roadmap for 0.8.0
>
> Great to see discussion for 0.8.0.
> List of features for 0.8.0 looks really good.
>
> Interpreter factory refactoring
> Interpreter layer supports various behavior depends on combination of
> PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for
> each combination as a first step.
> Otherwise, any pullrequest will silently break one of behavior at any time
> no matter we refactor or not. And fixing and testing this behavior is so
> hard.
> Once we have complete test cases, not only guarantee the behavior but also
> make refactoring much easier.
>
>
> 0.8.0 release
> I'd like to suggest improvements on how we release a new version.
>
> In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3
> months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months)
>
> I think the same thing will happen again with 0.8.0, while we're going to
> make lots of changes and add many new features.
> After we released 0.8.0, while 'Stabilizing' the new release, user who
> tried the new release may get wrong impression of the quality. Which is
> very bad and we already repeated the mistake in 0.6.0 and 0.7.0.
>
> So from 0.8.0 release, I'd suggest we improve way we release new version
> to give user proper expectation. I think there're several ways of doing it.
>
> 1. Release 0.8.0-preview officially and then release 0.8.0.
> 2. Release 0.8.0 with 'beta' or 'unstable' label. And keep 0.7.x as a
> 'stable' release in the download page. Once 0.8.x release becomes stable
> enough make 0.8.x release as a 'stable' and move 0.7.x to 'old' releases.
>
>
> After 0.8.0,
> Since Zeppelin projects starts, project went through some major milestone,
> like
>
> - project gets first users and first contributor
> - project went into Apache Incubator
> - project became TLP.
>
> And I think it's time to think about hitting another major milestone.
>
> Considering features we already have, features we're planning on 0.8, wide
> adoption of Zeppelin in the industry, I think it's time to focus on make
> project more mature and make a 1.0 release. Which i think big milestone for
> the project.
>
> After 0.8.0 release, I suggest we more focus on bug fixes, stability
> improvement, optimizing user experience than adding new features. And with
> subsequent minor release, 0.8.1, 0.8.2 ... moment we feel confident about
> the quality, release it as a 1.0.0 instead of 0.8.x.
>
> Once we have 1.0.0 released, then I think we can make larger, experimental
> changes on 2.0.0 branch aggressively, while we keep maintaining 1.0.x
> branch.
>
>
> Thanks,
> moon
>
> On Mon, Mar 20, 2017 at 8:55 AM Felix Cheung <felixcheung_m@hotmail.com<
> mailto:felixcheung_m@hotmail.com <fe...@hotmail.com>>> wrote:
> There are several pending visualization improvements/PRs that would be
> very good to get them in as well.
>
>
> ________________________________
> From: Jongyoul Lee <jongyoul@gmail.com<mailto:jongyoul@gmail.com
> <jo...@gmail.com>>>
> Sent: Sunday, March 19, 2017 9:03:24 PM
> To: dev; users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
> <us...@zeppelin.apache.org>>
> Subject: Roadmap for 0.8.0
>
> Hi dev & users,
>
> Recently, community submits very new features for Apache Zeppelin. I think
> it's very positive signals to improve Apache Zeppelin and its community.
> But in another aspect, we should focus on what the next release includes. I
> think we need to summarize and prioritize them. Here is what I know:
>
> * Cluster management
> * Admin feature
> * Replace some context to separate users
> * Helium online
>
> Feel free to talk if you want to add more things. I think we need to
> choose which features will be included in 0.8.0, too.
>
> Regards,
> Jongyoul Lee
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>
>
>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>
>
Re: Roadmap for 0.8.0
Posted by Jeff Zhang <zj...@gmail.com>.
Hi Ruslan,
Regarding 'make zeppelinContext available in shell interpreter', you may
want to check https://issues.apache.org/jira/browse/ZEPPELIN-1595
Ruslan Dautkhanov <da...@gmail.com>于2017年4月3日周一 下午12:05写道：
> That's exciting to see plans for 0.8.0 on the horizon.
>
> Here's my top list for 0.8 :
>
> - https://issues.apache.org/jira/browse/ZEPPELIN-2197 "Interpreter Idle
> timeout"
> This is a most-wanted feature by our Zeppelin admins. It was mentioned at
> least once on this email chain.
>
> - https://issues.apache.org/jira/browse/ZEPPELIN-1967 "Passing Z
> variables to Shell Interpreter"
> We had several of our users asking about this functionality. %sh and some
> other interpreters can't be
> parametrized by global variables. ZEPPELIN-1967 is one way of how this
> can be solved.
>
> - https://issues.apache.org/jira/browse/ZEPPELIN-1660 "Home directory
> references (i.e. ~/zeppelin/) in zeppelin-env.sh don't work as expected"
> Less of a critical compared to the above two, but it could complement
> the multi-tenancy feature very well.
>
>
> Best regards,
> Ruslan Dautkhanov
>
> On Wed, Mar 22, 2017 at 11:29 AM, Felix Cheung <fe...@hotmail.com>
> wrote:
>
> +1 with latest/stable.
>
>
>
>
> ------------------------------
> *From:* moon soo Lee <mo...@apache.org>
> *Sent:* Tuesday, March 21, 2017 8:41:58 AM
> *To:* users@zeppelin.apache.org
> *Cc:* dev@zeppelin.apache.org
>
> *Subject:* Re: Roadmap for 0.8.0
>
>
> And if i suggest simplest way for us to set quality expectation to user,
> which will be labeling release in download page.
>
> Currently releases are divided into 2 categories in download page. 'Latest
> release' and 'Old releases'. I think we can treat 'Latest' as unstable and
> add one more category 'Stable release'.
>
> For example, once 0.8.0 is released,
>
> Latest release : 0.8.0
> Stable release : 0.7.1
> Old release : 0.6.2, 0.6.1 ....
>
> Once we feel confident about the stability of latest release, we can just
> change label from latest to stable in the download page. (and previous
> stable goes to old releases)
> We can even include formal vote for moving release from 'latest' to
> 'stable' in our release process, if it is necessary.
>
> Thanks,
> moon
>
> On Tue, Mar 21, 2017 at 6:59 AM moon soo Lee <mo...@apache.org> wrote:
>
> Yes, having longer RC period will help.
>
> But if i recall 0.7.0 release, although 21 people participated verifying
> through 4 RC for 15days, it wasn't enough to catch all critical problems
> during the release process. After the release, we've got much more number
> of bug reports, in next few days.
>
> Basically, verifying RC is limited to people who subscribe mailing list +
> willing to contribute time to verify RC, which is much smaller number of
> people who download release from download page. So having longer RC period
> will definitely help and i think we should do, but I think it's still not
> enough to make sure the quality, considering past history.
>
> AFAIK, releasing 0.8.0-preview, calling it unstable is up to the project.
> ASF release process defines how to release source code, but it does not
> really restrict what kind of 'version' the project should have releases.
> For example, spark released spark-2.0.0-preview[1] before spark-2.0.0.
>
> Thanks,
> moon
>
> [1] http://spark.apache.org/news/spark-2.0.0-preview.html
>
> On Mon, Mar 20, 2017 at 11:31 PM Jongyoul Lee <jo...@gmail.com> wrote:
>
> I agree that it will help prolong RC period and use it actually. And also
> we need code freeze for the new features and spend time to stabilize RC.
>
> On Tue, Mar 21, 2017 at 1:25 PM, Felix Cheung <fe...@hotmail.com>
> wrote:
>
> +1 on quality and stabilization.
>
> I'm not sure if releasing as preview or calling it unstable fits with the
> ASF release process though.
>
> Other projects have code freeze, RC (and longer RC iteration time) etc. -
> do we think those will help improve quality when the release is finally cut?
>
>
> _____________________________
> From: Jianfeng (Jeff) Zhang <jz...@hortonworks.com>
> Sent: Monday, March 20, 2017 6:13 PM
> Subject: Re: Roadmap for 0.8.0
>
> To: <us...@zeppelin.apache.org>, dev <de...@zeppelin.apache.org>
>
>
>
>
>
> Strongly +1 for adding system test for different interpreter modes and
> focus on bug fixing than new features. I do heard from some users complain
> about the bugs of zeppelin major release. A stabilized release is very
> necessary for community.
>
>
>
>
> Best Regard,
> Jeff Zhang
>
>
> From: moon soo Lee <moon@apache.org<mailto:moon@apache.org
> <mo...@apache.org>>>
>
> Reply-To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
> <us...@zeppelin.apache.org>>" <users@zeppelin.apache.org<
> mailto:users@zeppelin.apache.org <us...@zeppelin.apache.org>>>
>
>
> Date: Tuesday, March 21, 2017 at 4:10 AM
>
> To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
> <us...@zeppelin.apache.org>>" <users@zeppelin.apache.org<
> mailto:users@zeppelin.apache.org <us...@zeppelin.apache.org>>>, dev <
> dev@zeppelin.apache.org<mailto:dev@zeppelin.apache.org
> <de...@zeppelin.apache.org>>>
>
>
> Subject: Re: Roadmap for 0.8.0
>
> Great to see discussion for 0.8.0.
> List of features for 0.8.0 looks really good.
>
> Interpreter factory refactoring
> Interpreter layer supports various behavior depends on combination of
> PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for
> each combination as a first step.
> Otherwise, any pullrequest will silently break one of behavior at any time
> no matter we refactor or not. And fixing and testing this behavior is so
> hard.
> Once we have complete test cases, not only guarantee the behavior but also
> make refactoring much easier.
>
>
> 0.8.0 release
> I'd like to suggest improvements on how we release a new version.
>
> In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3
> months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months)
>
> I think the same thing will happen again with 0.8.0, while we're going to
> make lots of changes and add many new features.
> After we released 0.8.0, while 'Stabilizing' the new release, user who
> tried the new release may get wrong impression of the quality. Which is
> very bad and we already repeated the mistake in 0.6.0 and 0.7.0.
>
> So from 0.8.0 release, I'd suggest we improve way we release new version
> to give user proper expectation. I think there're several ways of doing it.
>
> 1. Release 0.8.0-preview officially and then release 0.8.0.
> 2. Release 0.8.0 with 'beta' or 'unstable' label. And keep 0.7.x as a
> 'stable' release in the download page. Once 0.8.x release becomes stable
> enough make 0.8.x release as a 'stable' and move 0.7.x to 'old' releases.
>
>
> After 0.8.0,
> Since Zeppelin projects starts, project went through some major milestone,
> like
>
> - project gets first users and first contributor
> - project went into Apache Incubator
> - project became TLP.
>
> And I think it's time to think about hitting another major milestone.
>
> Considering features we already have, features we're planning on 0.8, wide
> adoption of Zeppelin in the industry, I think it's time to focus on make
> project more mature and make a 1.0 release. Which i think big milestone for
> the project.
>
> After 0.8.0 release, I suggest we more focus on bug fixes, stability
> improvement, optimizing user experience than adding new features. And with
> subsequent minor release, 0.8.1, 0.8.2 ... moment we feel confident about
> the quality, release it as a 1.0.0 instead of 0.8.x.
>
> Once we have 1.0.0 released, then I think we can make larger, experimental
> changes on 2.0.0 branch aggressively, while we keep maintaining 1.0.x
> branch.
>
>
> Thanks,
> moon
>
> On Mon, Mar 20, 2017 at 8:55 AM Felix Cheung <felixcheung_m@hotmail.com<
> mailto:felixcheung_m@hotmail.com <fe...@hotmail.com>>> wrote:
> There are several pending visualization improvements/PRs that would be
> very good to get them in as well.
>
>
> ________________________________
> From: Jongyoul Lee <jongyoul@gmail.com<mailto:jongyoul@gmail.com
> <jo...@gmail.com>>>
> Sent: Sunday, March 19, 2017 9:03:24 PM
> To: dev; users@zeppelin.apache.org<mailto:users@zeppelin.apache.org
> <us...@zeppelin.apache.org>>
> Subject: Roadmap for 0.8.0
>
> Hi dev & users,
>
> Recently, community submits very new features for Apache Zeppelin. I think
> it's very positive signals to improve Apache Zeppelin and its community.
> But in another aspect, we should focus on what the next release includes. I
> think we need to summarize and prioritize them. Here is what I know:
>
> * Cluster management
> * Admin feature
> * Replace some context to separate users
> * Helium online
>
> Feel free to talk if you want to add more things. I think we need to
> choose which features will be included in 0.8.0, too.
>
> Regards,
> Jongyoul Lee
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>
>
>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>
>