You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by moon soo Lee <mo...@apache.org> on 2018/03/07 18:21:40 UTC
[DISCUSS] Roadmap 0.9 and future
Hi forks,
There were an offline meeting yesterday at PaloAlto with contributors and
users. We've shared idea about current state of project and future project
roadmap and wishlists (meeting note [1]). It was really inspiring and
exciting time. Let me try summarize, move this discussion to online.
There were many ideas related to Interpreter. Especially, there were
consensus that Spark support is one of biggest strength of Zeppelin and
need to make further improvement to keep the strengths.
- Spark
- Immediate support of new spark release
- Ramp up support of current Spark feature (e.g. Display job progress
correctly)
- Spark streaming support
- Handling Livy timeout
- Other interpreters
- Better Hive support (e.g. configuration)
- Latest version PrestoDB support (pass property correctly)
- Run interpreter in containerized environment
- Let individual user upload custom library from user's machine directly
- Interpreter documentation is not detail enough
And people in the meeting excited about ConInterpreter ZEPPELIN-3085 [2] in
upcoming release, regarding dynamic/inline configuration of interpreter.
And there were ideas on other areas, too. like
- Separate Admin role and user role
- Sidebar with plugin widget
- Better integration with emerging framework like Tensorflow/MXNet/Ray
- Sharing data
- Schedule notebook from external scheduler
Regarding scheduling notebook, Luciano shared his project NotebookTools[3]
and it made people really excited.
Also, there were inspiring discussions about the community/project. Current
status and how can we make community/project more healthy. And here's some
ideas around the topic
- Need more frequent release
- More attention to code review to speed up
- Publishing roadmap beforehand to help contribution
- 'Newbie', 'low hanging fruit' tag helps contribution
- Enterprise friendly is another biggest strength of Zeppelin (in
addition to Spark support) need to keep improve.
I probably missed many idea shared yesterday. Please feel free to
add/correct the summary. Hope more people in the mailinglist join and
develop the idea together. And I think this discussion can leads community
shape 0.9 and future version of Zeppelin, and update and publish future
roadmap[4].
Best,
moon
Special thanks to ZEPL <https://www.zepl.com> for the swag and dinner.
[1]
https://docs.google.com/document/d/18Wc3pEFx3qm9XoME_V_B9k_LlAd1PLyKQQEveR1on1o/edit?usp=sharing
[2] https://issues.apache.org/jira/browse/ZEPPELIN-3085
[3]
https://github.com/SparkTC/notebook-exporter/tree/master/notebook-exporter
[4] https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap
RE: [DISCUSS] Roadmap 0.9 and future
Posted by "Partridge, Lucas (GE Aviation)" <Lu...@ge.com>.
Yes, thanks for sharing this roadmap, moon!
I don’t know what the “Sidebar with plugin widget” item was referring to, but navigation within large notebooks could definitely be improved. E.g., add a clickable outline view in a sidebar, like Jupyter does, which can be used to jump straight to a section. Maybe add collapsible sections too, which could encompass multiple paragraphs. Many users, including me, currently waste lots of time scrolling up and down within notebooks at the moment.
But this is a minor gripe as I think Zeppelin is a great tool and is growing well!
Thanks, Lucas.
From: moon soo Lee [mailto:moon@apache.org]
Sent: 07 March 2018 18:22
To: users@zeppelin.apache.org
Subject: EXT: [DISCUSS] Roadmap 0.9 and future
Hi forks,
There were an offline meeting yesterday at PaloAlto with contributors and users. We've shared idea about current state of project and future project roadmap and wishlists (meeting note [1]). It was really inspiring and exciting time. Let me try summarize, move this discussion to online.
There were many ideas related to Interpreter. Especially, there were consensus that Spark support is one of biggest strength of Zeppelin and need to make further improvement to keep the strengths.
* Spark
* Immediate support of new spark release
* Ramp up support of current Spark feature (e.g. Display job progress correctly)
* Spark streaming support
* Handling Livy timeout
* Other interpreters
* Better Hive support (e.g. configuration)
* Latest version PrestoDB support (pass property correctly)
* Run interpreter in containerized environment
* Let individual user upload custom library from user's machine directly
* Interpreter documentation is not detail enough
And people in the meeting excited about ConInterpreter ZEPPELIN-3085 [2] in upcoming release, regarding dynamic/inline configuration of interpreter.
And there were ideas on other areas, too. like
* Separate Admin role and user role
* Sidebar with plugin widget
* Better integration with emerging framework like Tensorflow/MXNet/Ray
* Sharing data
* Schedule notebook from external scheduler
Regarding scheduling notebook, Luciano shared his project NotebookTools[3] and it made people really excited.
Also, there were inspiring discussions about the community/project. Current status and how can we make community/project more healthy. And here's some ideas around the topic
* Need more frequent release
* More attention to code review to speed up
* Publishing roadmap beforehand to help contribution
* 'Newbie', 'low hanging fruit' tag helps contribution
* Enterprise friendly is another biggest strength of Zeppelin (in addition to Spark support) need to keep improve.
I probably missed many idea shared yesterday. Please feel free to add/correct the summary. Hope more people in the mailinglist join and develop the idea together. And I think this discussion can leads community shape 0.9 and future version of Zeppelin, and update and publish future roadmap[4].
Best,
moon
Special thanks to ZEPL<https://www.zepl.com> for the swag and dinner.
[1] https://docs.google.com/document/d/18Wc3pEFx3qm9XoME_V_B9k_LlAd1PLyKQQEveR1on1o/edit?usp=sharing
[2] https://issues.apache.org/jira/browse/ZEPPELIN-3085
[3] https://github.com/SparkTC/notebook-exporter/tree/master/notebook-exporter
[4] https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap
RE: [DISCUSS] Roadmap 0.9 and future
Posted by Belousov Maksim Eduardovich <m....@tinkoff.ru>.
Hello Moon and users!
Thanks a lot for sharing.
I propose to abandon the roadmap.
The idea of no roadmap in open source projects is from the book "Social Architecture". The author is Pieter Hintjen. As it's written: in 2007 he founded the successful and broad-ranging ZeroMQ community. There is the chapter "How ZeroMQ Lost Its Road Map" in [1].
Furthermore the current item list is more like "wish list".
I believe one can rename the wiki-page to "Community wish list"
[1] https://hintjens.gitbooks.io/social-architecture/content/chapter5.html
Regards,
Maksim Belousov
From: Jongyoul Lee [mailto:jongyoul@gmail.com]
Sent: Thursday, March 08, 2018 3:10 AM
To: users <us...@zeppelin.apache.org>
Subject: Re: [DISCUSS] Roadmap 0.9 and future
Thanks for the sharing.
As an enterprise user, it would help a lot to support Spark more concretely. And containerization became a default option to operate a framework.
For provide immediate release of Spark, I suggest separate release cycle between zeppelin-* module and interpreters. This would help contributor see more frequent releases of interpreters they are using.
Thanks,
JL
On Thu, Mar 8, 2018 at 4:08 AM, moon soo Lee <mo...@apache.org>> wrote:
I think there were couple of discussions about 0.8.0 release in the community. We have umbrella issue that tracks 0.8.0 release [1]. And i see not many blockers are left. Jeff volunteered the release manager for 0.8.0 and I think he can give better idea of 0.8.0 release estimate.
Thanks,
moon
[1] https://issues.apache.org/jira/browse/ZEPPELIN-2385
On Wed, Mar 7, 2018 at 10:49 AM Paul Brenner <pb...@placeiq.com>> wrote:
[https://share.polymail.io/v2/z/a/NWFhMDMwOTE2OTMz/BRe7pNgtAmQwzQfWdRurKMu_L3CexD7fbzihUQ9bmqEuXHBjiuR2B50qLJ6JeUnOOIHtNGf9IAWHv9b8xv6g5da0m57iEDKSQZ1e3rLZr6y8VKN7JHEMrJDVbOGhHSngSnovAYT-1vZDQE_5ziuawEs6EUk=.png]
Thanks for sharing the results of the meeting!
Regarding the point "Need more frequent release”, was there any discussion around when 0.8.0 will be officially released? I remember a message a few months ago suggesting that it might be soon.
[https://marketing.placeiq.net/images/placeiq.png]<http://www.placeiq.com/>
Paul Brenner
[https://marketing.placeiq.net/images/twitter1.png]<https://twitter.com/placeiq>
[https://marketing.placeiq.net/images/facebook.png]<https://www.facebook.com/PlaceIQ>
[https://marketing.placeiq.net/images/linkedin.png]<https://www.linkedin.com/company/placeiq>
DATA SCIENTIST
(217) 390-3033<tel:(217)%20390-3033>
[PlaceIQ:CES 2018]<http://pages.placeiq.com/2017-Integrated-Marketing-Whitepaper_LP_Download.html>
On Wed, Mar 07, 2018 at 1:21 PM moon soo Lee <moon soo Lee <mailto:moon+soo+Lee+%3Cmoon@apache.org%3E> > wrote:
Hi forks,
There were an offline meeting yesterday at PaloAlto with contributors and users. We've shared idea about current state of project and future project roadmap and wishlists (meeting note [1]). It was really inspiring and exciting time. Let me try summarize, move this discussion to online.
There were many ideas related to Interpreter. Especially, there were consensus that Spark support is one of biggest strength of Zeppelin and need to make further improvement to keep the strengths.
* Spark
* Immediate support of new spark release
* Ramp up support of current Spark feature (e.g. Display job progress correctly)
* Spark streaming support
* Handling Livy timeout
* Other interpreters
* Better Hive support (e.g. configuration)
* Latest version PrestoDB support (pass property correctly)
* Run interpreter in containerized environment
* Let individual user upload custom library from user's machine directly
* Interpreter documentation is not detail enough
And people in the meeting excited about ConInterpreter ZEPPELIN-3085 [2] in upcoming release, regarding dynamic/inline configuration of interpreter.
And there were ideas on other areas, too. like
* Separate Admin role and user role
* Sidebar with plugin widget
* Better integration with emerging framework like Tensorflow/MXNet/Ray
* Sharing data
* Schedule notebook from external scheduler
Regarding scheduling notebook, Luciano shared his project NotebookTools[3] and it made people really excited.
Also, there were inspiring discussions about the community/project. Current status and how can we make community/project more healthy. And here's some ideas around the topic
* Need more frequent release
* More attention to code review to speed up
* Publishing roadmap beforehand to help contribution
* 'Newbie', 'low hanging fruit' tag helps contribution
* Enterprise friendly is another biggest strength of Zeppelin (in addition to Spark support) need to keep improve.
I probably missed many idea shared yesterday. Please feel free to add/correct the summary. Hope more people in the mailinglist join and develop the idea together. And I think this discussion can leads community shape 0.9 and future version of Zeppelin, and update and publish future roadmap[4].
Best,
moon
Special thanks to ZEPL<https://www.zepl.com> for the swag and dinner.
[1] https://docs.google.com/document/d/18Wc3pEFx3qm9XoME_V_B9k_LlAd1PLyKQQEveR1on1o/edit?usp=sharing
[2] https://issues.apache.org/jira/browse/ZEPPELIN-3085
[3] https://github.com/SparkTC/notebook-exporter/tree/master/notebook-exporter
[4] https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap
--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net
Re: [DISCUSS] Roadmap 0.9 and future
Posted by Jongyoul Lee <jo...@gmail.com>.
Thanks for the sharing.
As an enterprise user, it would help a lot to support Spark more
concretely. And containerization became a default option to operate a
framework.
For provide immediate release of Spark, I suggest separate release cycle
between zeppelin-* module and interpreters. This would help contributor see
more frequent releases of interpreters they are using.
Thanks,
JL
On Thu, Mar 8, 2018 at 4:08 AM, moon soo Lee <mo...@apache.org> wrote:
> I think there were couple of discussions about 0.8.0 release in the
> community. We have umbrella issue that tracks 0.8.0 release [1]. And i see
> not many blockers are left. Jeff volunteered the release manager for 0.8.0
> and I think he can give better idea of 0.8.0 release estimate.
>
> Thanks,
> moon
>
> [1] https://issues.apache.org/jira/browse/ZEPPELIN-2385
>
>
> On Wed, Mar 7, 2018 at 10:49 AM Paul Brenner <pb...@placeiq.com> wrote:
>
>> Thanks for sharing the results of the meeting!
>>
>> Regarding the point "Need more frequent release”, was there any
>> discussion around when 0.8.0 will be officially released? I remember a
>> message a few months ago suggesting that it might be soon.
>>
>> <http://www.placeiq.com/> <http://www.placeiq.com/>
>> <http://www.placeiq.com/> *Paul Brenner* <https://twitter.com/placeiq>
>> <https://twitter.com/placeiq> <https://twitter.com/placeiq>
>> <https://www.facebook.com/PlaceIQ> <https://www.facebook.com/PlaceIQ>
>> <https://www.linkedin.com/company/placeiq>
>> <https://www.linkedin.com/company/placeiq>
>> DATA SCIENTIST
>> *(217) 390-3033 <(217)%20390-3033> *
>>
>> <http://www.placeiq.com/2015/05/26/placeiq-named-winner-of-prestigious-2015-oracle-data-cloud-activate-award/>
>> <http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
>> <http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
>> <http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
>> <http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
>> <http://placeiq.com/2016/03/08/measuring-addressable-tv-campaigns-is-now-possible/>
>> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
>> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
>> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
>> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
>> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
>> <http://pages.placeiq.com/Location-Data-Accuracy-Whitepaper-Download.html?utm_source=Signature&utm_medium=Email&utm_campaign=AccuracyWP>
>> <http://placeiq.com/2016/08/03/placeiq-bolsters-location-intelligence-platform-with-mastercard-insights/>
>> <http://placeiq.com/2016/10/26/the-making-of-a-location-data-industry-milestone/>
>> <http://placeiq.com/2016/12/07/placeiq-introduces-landmark-a-groundbreaking-offering-that-delivers-access-to-the-highest-quality-location-data-for-insights-that-fuel-limitless-business-decisions/>
>> <http://placeiq.com/2016/12/07/placeiq-introduces-landmark-a-groundbreaking-offering-that-delivers-access-to-the-highest-quality-location-data-for-insights-that-fuel-limitless-business-decisions/>
>> <https://www.placeiq.com/2017/05/placeiqs-landmark-powers-location-based-insight-innovation-for-ansible-gstv-havas-media-the-media-kitchen-and-more/>
>> <http://pages.placeiq.com/CES2018_MeetingRequest.html>[image:
>> PlaceIQ:CES 2018]
>> <http://pages.placeiq.com/2017-Integrated-Marketing-Whitepaper_LP_Download.html>
>>
>> On Wed, Mar 07, 2018 at 1:21 PM moon soo Lee <moon soo Lee
>> <moon+soo+Lee+%3Cmoon@apache.org%3E>> wrote:
>>
>>> Hi forks,
>>>
>>> There were an offline meeting yesterday at PaloAlto with contributors
>>> and users. We've shared idea about current state of project and future
>>> project roadmap and wishlists (meeting note [1]). It was really inspiring
>>> and exciting time. Let me try summarize, move this discussion to online.
>>>
>>> There were many ideas related to Interpreter. Especially, there were
>>> consensus that Spark support is one of biggest strength of Zeppelin and
>>> need to make further improvement to keep the strengths.
>>>
>>> - Spark
>>> - Immediate support of new spark release
>>> - Ramp up support of current Spark feature (e.g. Display job
>>> progress correctly)
>>> - Spark streaming support
>>> - Handling Livy timeout
>>> - Other interpreters
>>> - Better Hive support (e.g. configuration)
>>> - Latest version PrestoDB support (pass property correctly)
>>> - Run interpreter in containerized environment
>>> - Let individual user upload custom library from user's machine
>>> directly
>>> - Interpreter documentation is not detail enough
>>>
>>> And people in the meeting excited about ConInterpreter ZEPPELIN-3085 [2]
>>> in upcoming release, regarding dynamic/inline configuration of interpreter.
>>>
>>> And there were ideas on other areas, too. like
>>>
>>> - Separate Admin role and user role
>>> - Sidebar with plugin widget
>>> - Better integration with emerging framework like
>>> Tensorflow/MXNet/Ray
>>> - Sharing data
>>> - Schedule notebook from external scheduler
>>>
>>> Regarding scheduling notebook, Luciano shared his project
>>> NotebookTools[3] and it made people really excited.
>>>
>>> Also, there were inspiring discussions about the community/project.
>>> Current status and how can we make community/project more healthy. And
>>> here's some ideas around the topic
>>>
>>> - Need more frequent release
>>> - More attention to code review to speed up
>>> - Publishing roadmap beforehand to help contribution
>>> - 'Newbie', 'low hanging fruit' tag helps contribution
>>> - Enterprise friendly is another biggest strength of Zeppelin (in
>>> addition to Spark support) need to keep improve.
>>>
>>>
>>> I probably missed many idea shared yesterday. Please feel free to
>>> add/correct the summary. Hope more people in the mailinglist join and
>>> develop the idea together. And I think this discussion can leads community
>>> shape 0.9 and future version of Zeppelin, and update and publish future
>>> roadmap[4].
>>>
>>> Best,
>>> moon
>>>
>>> Special thanks to ZEPL <https://www.zepl.com> for the swag and dinner.
>>>
>>> [1] https://docs.google.com/document/d/18Wc3pEFx3qm9XoME_
>>> V_B9k_LlAd1PLyKQQEveR1on1o/edit?usp=sharing
>>> [2] https://issues.apache.org/jira/browse/ZEPPELIN-3085
>>> [3] https://github.com/SparkTC/notebook-exporter/
>>> tree/master/notebook-exporter
>>> [4] https://cwiki.apache.org/confluence/display/ZEPPELIN/
>>> Zeppelin+Roadmap
>>>
>>>
>>>
>>>
>>
--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net
Re: [DISCUSS] Roadmap 0.9 and future
Posted by moon soo Lee <mo...@apache.org>.
I think there were couple of discussions about 0.8.0 release in the
community. We have umbrella issue that tracks 0.8.0 release [1]. And i see
not many blockers are left. Jeff volunteered the release manager for 0.8.0
and I think he can give better idea of 0.8.0 release estimate.
Thanks,
moon
[1] https://issues.apache.org/jira/browse/ZEPPELIN-2385
On Wed, Mar 7, 2018 at 10:49 AM Paul Brenner <pb...@placeiq.com> wrote:
> Thanks for sharing the results of the meeting!
>
> Regarding the point "Need more frequent release”, was there any discussion
> around when 0.8.0 will be officially released? I remember a message a few
> months ago suggesting that it might be soon.
>
> <http://www.placeiq.com/> <http://www.placeiq.com/>
> <http://www.placeiq.com/> *Paul Brenner* <https://twitter.com/placeiq>
> <https://twitter.com/placeiq> <https://twitter.com/placeiq>
> <https://www.facebook.com/PlaceIQ> <https://www.facebook.com/PlaceIQ>
> <https://www.linkedin.com/company/placeiq>
> <https://www.linkedin.com/company/placeiq>
> DATA SCIENTIST
> *(217) 390-3033 <(217)%20390-3033> *
>
> <http://www.placeiq.com/2015/05/26/placeiq-named-winner-of-prestigious-2015-oracle-data-cloud-activate-award/>
> <http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
> <http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
> <http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
> <http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/>
> <http://placeiq.com/2016/03/08/measuring-addressable-tv-campaigns-is-now-possible/>
> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
> <http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/>
> <http://pages.placeiq.com/Location-Data-Accuracy-Whitepaper-Download.html?utm_source=Signature&utm_medium=Email&utm_campaign=AccuracyWP>
> <http://placeiq.com/2016/08/03/placeiq-bolsters-location-intelligence-platform-with-mastercard-insights/>
> <http://placeiq.com/2016/10/26/the-making-of-a-location-data-industry-milestone/>
> <http://placeiq.com/2016/12/07/placeiq-introduces-landmark-a-groundbreaking-offering-that-delivers-access-to-the-highest-quality-location-data-for-insights-that-fuel-limitless-business-decisions/>
> <http://placeiq.com/2016/12/07/placeiq-introduces-landmark-a-groundbreaking-offering-that-delivers-access-to-the-highest-quality-location-data-for-insights-that-fuel-limitless-business-decisions/>
> <https://www.placeiq.com/2017/05/placeiqs-landmark-powers-location-based-insight-innovation-for-ansible-gstv-havas-media-the-media-kitchen-and-more/>
> <http://pages.placeiq.com/CES2018_MeetingRequest.html>[image: PlaceIQ:CES
> 2018]
> <http://pages.placeiq.com/2017-Integrated-Marketing-Whitepaper_LP_Download.html>
>
> On Wed, Mar 07, 2018 at 1:21 PM moon soo Lee <moon soo Lee
> <moon+soo+Lee+%3Cmoon@apache.org%3E>> wrote:
>
>> Hi forks,
>>
>> There were an offline meeting yesterday at PaloAlto with contributors and
>> users. We've shared idea about current state of project and future project
>> roadmap and wishlists (meeting note [1]). It was really inspiring and
>> exciting time. Let me try summarize, move this discussion to online.
>>
>> There were many ideas related to Interpreter. Especially, there were
>> consensus that Spark support is one of biggest strength of Zeppelin and
>> need to make further improvement to keep the strengths.
>>
>> - Spark
>> - Immediate support of new spark release
>> - Ramp up support of current Spark feature (e.g. Display job
>> progress correctly)
>> - Spark streaming support
>> - Handling Livy timeout
>> - Other interpreters
>> - Better Hive support (e.g. configuration)
>> - Latest version PrestoDB support (pass property correctly)
>> - Run interpreter in containerized environment
>> - Let individual user upload custom library from user's machine
>> directly
>> - Interpreter documentation is not detail enough
>>
>> And people in the meeting excited about ConInterpreter ZEPPELIN-3085 [2]
>> in upcoming release, regarding dynamic/inline configuration of interpreter.
>>
>> And there were ideas on other areas, too. like
>>
>> - Separate Admin role and user role
>> - Sidebar with plugin widget
>> - Better integration with emerging framework like Tensorflow/MXNet/Ray
>> - Sharing data
>> - Schedule notebook from external scheduler
>>
>> Regarding scheduling notebook, Luciano shared his project
>> NotebookTools[3] and it made people really excited.
>>
>> Also, there were inspiring discussions about the community/project.
>> Current status and how can we make community/project more healthy. And
>> here's some ideas around the topic
>>
>> - Need more frequent release
>> - More attention to code review to speed up
>> - Publishing roadmap beforehand to help contribution
>> - 'Newbie', 'low hanging fruit' tag helps contribution
>> - Enterprise friendly is another biggest strength of Zeppelin (in
>> addition to Spark support) need to keep improve.
>>
>>
>> I probably missed many idea shared yesterday. Please feel free to
>> add/correct the summary. Hope more people in the mailinglist join and
>> develop the idea together. And I think this discussion can leads community
>> shape 0.9 and future version of Zeppelin, and update and publish future
>> roadmap[4].
>>
>> Best,
>> moon
>>
>> Special thanks to ZEPL <https://www.zepl.com> for the swag and dinner.
>>
>> [1]
>> https://docs.google.com/document/d/18Wc3pEFx3qm9XoME_V_B9k_LlAd1PLyKQQEveR1on1o/edit?usp=sharing
>> [2] https://issues.apache.org/jira/browse/ZEPPELIN-3085
>> [3]
>> https://github.com/SparkTC/notebook-exporter/tree/master/notebook-exporter
>> [4] https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap
>>
>>
>>
>>
>
[DISCUSS] Roadmap 0.9 and future
Posted by Paul Brenner <pb...@placeiq.com>.
Thanks for sharing the results of the meeting!
Regarding the point "Need more frequent release”, was there any discussion around when 0.8.0 will be officially released? I remember a message a few months ago suggesting that it might be soon.
( http://www.placeiq.com/ ) ( http://www.placeiq.com/ ) ( http://www.placeiq.com/ ) *Paul Brenner* ( https://twitter.com/placeiq ) ( https://twitter.com/placeiq ) ( https://twitter.com/placeiq ) ( https://www.facebook.com/PlaceIQ ) ( https://www.facebook.com/PlaceIQ ) ( https://www.linkedin.com/company/placeiq ) ( https://www.linkedin.com/company/placeiq ) DATA SCIENTIST (217) 390-3033
( http://www.placeiq.com/2015/05/26/placeiq-named-winner-of-prestigious-2015-oracle-data-cloud-activate-award/ ) ( http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/ ) ( http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/ ) ( http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/ ) ( http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/ ) ( http://placeiq.com/2016/03/08/measuring-addressable-tv-campaigns-is-now-possible/ ) ( http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/ ) ( http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/ ) ( http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/ ) ( http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/ ) ( http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/ ) ( http://pages.placeiq.com/Location-Data-Accuracy-Whitepaper-Download.html?utm_source=Signature&utm_medium=Email&utm_campaign=AccuracyWP ) ( http://placeiq.com/2016/08/03/placeiq-bolsters-location-intelligence-platform-with-mastercard-insights/ ) ( http://placeiq.com/2016/10/26/the-making-of-a-location-data-industry-milestone/ ) ( http://placeiq.com/2016/12/07/placeiq-introduces-landmark-a-groundbreaking-offering-that-delivers-access-to-the-highest-quality-location-data-for-insights-that-fuel-limitless-business-decisions/ ) ( http://placeiq.com/2016/12/07/placeiq-introduces-landmark-a-groundbreaking-offering-that-delivers-access-to-the-highest-quality-location-data-for-insights-that-fuel-limitless-business-decisions/ ) ( https://www.placeiq.com/2017/05/placeiqs-landmark-powers-location-based-insight-innovation-for-ansible-gstv-havas-media-the-media-kitchen-and-more/ ) ( http://pages.placeiq.com/CES2018_MeetingRequest.html ) PlaceIQ:CES 2018 ( http://pages.placeiq.com/2017-Integrated-Marketing-Whitepaper_LP_Download.html )
On Wed, Mar 07, 2018 at 1:21 PM moon soo Lee < moon soo Lee ( moon soo Lee <mo...@apache.org> ) > wrote:
>
> Hi forks,
>
>
> There were an offline meeting yesterday at PaloAlto with contributors and
> users. We've shared idea about current state of project and future project
> roadmap and wishlists (meeting note [1]). It was really inspiring and
> exciting time. Let me try summarize, move this discussion to online.
>
>
> There were many ideas related to Interpreter. Especially, there were
> consensus that Spark support is one of biggest strength of Zeppelin and
> need to make further improvement to keep the strengths.
>
>
> * Spark
>
>
>
> * Immediate support of new spark release
>
> * Ramp up support of current Spark feature (e.g. Display job progress
> correctly)
>
> * Spark streaming support
>
> * Handling Livy timeout
>
>
>
> * Other interpreters
>
>
>
> * Better Hive support (e.g. configuration)
>
> * Latest version PrestoDB support (pass property correctly)
>
>
>
> * Run interpreter in containerized environment
>
> * Let individual user upload custom library from user's machine directly
>
> * Interpreter documentation is not detail enough
>
>
>
>
> And people in the meeting excited about ConInterpreter ZEPPELIN-3085 [2]
> in upcoming release, regarding dynamic/inline configuration of
> interpreter.
>
>
> And there were ideas on other areas, too. like
>
>
> * Separate Admin role and user role
> * Sidebar with plugin widget
> * Better integration with emerging framework like Tensorflow/MXNet/Ray
> * Sharing data
> * Schedule notebook from external scheduler
>
>
> Regarding scheduling notebook, Luciano shared his project NotebookTools[3]
> and it made people really excited.
>
>
> Also, there were inspiring discussions about the community/project.
> Current status and how can we make community/project more healthy. And
> here's some ideas around the topic
>
>
> * Need more frequent release
> * More attention to code review to speed up
> * Publishing roadmap beforehand to help contribution
> * 'Newbie', 'low hanging fruit' tag helps contribution
> * Enterprise friendly is another biggest strength of Zeppelin (in addition
> to Spark support) need to keep improve.
>
>
>
>
> I probably missed many idea shared yesterday. Please feel free to
> add/correct the summary. Hope more people in the mailinglist join and
> develop the idea together. And I think this discussion can leads community
> shape 0.9 and future version of Zeppelin, and update and publish future
> roadmap[4].
>
>
> Best,
> moon
>
>
> Special thanks to ZEPL ( https://www.zepl.com ) for the swag and dinner.
>
>
> [1] https://docs.google.com/document/d/18Wc3pEFx3qm9XoME_V_B9k_LlAd1PLyKQQEveR1on1o/edit?usp=sharing
>
> [2] https://issues.apache.org/jira/browse/ZEPPELIN-3085
> [3] https://github.com/SparkTC/notebook-exporter/tree/master/notebook-exporter
>
> [4] https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap
>
>
>
>
>
>
>
Re: [DISCUSS] Roadmap 0.9 and future(Internet mail)
Posted by "lewinma(lewinma)" <le...@tencent.com>.
I start a zeppelin daemon on a bare metal node, and want to build a connection to the flink cluster started on K8s.
After setting job manager node ip and port. I just got the error below, could any one help me about that? Thanks a lot.
text: org.apache.flink.api.scala.DataSet[String] = org.apache.flink.api.scala.DataSet@9c8f964 counts: org.apache.flink.api.scala.AggregateDataSet[(String, Int)] = org.apache.flink.api.scala.AggregateDataSet@41d66e72 org.apache.flink.client.program.ProgramInvocationException: The program execution failed: Communication with JobManager failed: Lost connection to the JobManager. at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:409) at org.apache.flink.client.program.StandaloneClusterClient.submitJob(StandaloneClusterClient.java:95) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:382) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:369) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:344) at org.apache.flink.client.RemoteExecutor.executePlanWithJars(RemoteExecutor.java:211) at org.apache.flink.client.RemoteExecutor.executePlan(RemoteExecutor.java:188) at org.apache.flink.api.java.RemoteEnvironment.execute(RemoteEnvironment.java:172) at org.apache.flink.api.java.ExecutionEnvironment.execute(ExecutionEnvironment.java:896) at org.apache.flink.api.scala.ExecutionEnvironment.execute(ExecutionEnvironment.scala:637) at org.apache.flink.api.scala.DataSet.collect(DataSet.scala:547) ... 36 elided Caused by: org.apache.flink.runtime.client.JobExecutionException: Communication with JobManager failed: Lost connection to the JobManager. at org.apache.flink.runtime.client.JobClient.submitJobAndWait(JobClient.java:137) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:405) ... 46 more Caused by: org.apache.flink.runtime.client.JobClientActorConnectionTimeoutException: Lost connection to the JobManager. at org.apache.flink.runtime.client.JobClientActor.handleMessage(JobClientActor.java:252) at org.apache.flink.runtime.akka.FlinkUntypedActor.handleLeaderSessionID(FlinkUntypedActor.java:90) at org.apache.flink.runtime.akka.FlinkUntypedActor.onReceive(FlinkUntypedActor.java:70) at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167) at akka.actor.Actor$class.aroundReceive(Actor.scala:465) at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:97) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516) at akka.actor.ActorCell.invoke(ActorCell.scala:487) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254) at akka.dispatch.Mailbox.run(Mailbox.scala:221) at akka.dispatch.Mailbox.exec(Mailbox.scala:231) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.pollAndExecAll(ForkJoinPool.java:1253) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1346) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
On Sep 18, 2019, at 11:39 AM, Jeff Zhang <zj...@gmail.com>> wrote:
I think it is already supported. The flink interpreter will still run out of k8s, but the flink cluster runs in k8s, then as long as configure flink interpreter properly, you can connect the flink cluster in k8s
lewinma(lewinma) <le...@tencent.com>> 于2019年9月18日周三 上午10:38写道：
Do we have a plan of supporting Flink running on K8s?
Best Regards~
Lewinma
On Sep 17, 2019, at 9:32 PM, Jeff Zhang <zj...@gmail.com>> wrote:
Maybe in next Month (Sep)
mahmoud.khaled <ma...@incorta.com>> 于2019年9月17日周二 下午7:21写道：
Hi,
What's the planned date for 0.9 Release?
Thanks in advance.
--
Sent from: http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/
--
Best Regards
Jeff Zhang
--
Best Regards
Jeff Zhang
Re: [DISCUSS] Roadmap 0.9 and future(Internet mail)
Posted by "lewinma(lewinma)" <le...@tencent.com>.
I updated the version of Zeppelin in my environment and post a JIRA tickect as below:
https://issues.apache.org/jira/browse/ZEPPELIN-4332#
Any help would be very grateful. Thank you.
On Sep 20, 2019, at 10:49 AM, Jeff Zhang <zj...@gmail.com>> wrote:
yes, that's right, please attach the logs if this doesn't work for you.
lewinma(lewinma) <le...@tencent.com>> 于2019年9月18日周三 上午11:44写道：
Sounds great, are there any doc about that?
If only config the Flink job manager’s ip and port is enough.
Thanks a lot.
On Sep 18, 2019, at 11:39 AM, Jeff Zhang <zj...@gmail.com>> wrote:
I think it is already supported. The flink interpreter will still run out of k8s, but the flink cluster runs in k8s, then as long as configure flink interpreter properly, you can connect the flink cluster in k8s
lewinma(lewinma) <le...@tencent.com>> 于2019年9月18日周三 上午10:38写道：
Do we have a plan of supporting Flink running on K8s?
Best Regards~
Lewinma
On Sep 17, 2019, at 9:32 PM, Jeff Zhang <zj...@gmail.com>> wrote:
Maybe in next Month (Sep)
mahmoud.khaled <ma...@incorta.com>> 于2019年9月17日周二 下午7:21写道：
Hi,
What's the planned date for 0.9 Release?
Thanks in advance.
--
Sent from: http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/
--
Best Regards
Jeff Zhang
--
Best Regards
Jeff Zhang
--
Best Regards
Jeff Zhang
Re: [DISCUSS] Roadmap 0.9 and future(Internet mail)
Posted by Jeff Zhang <zj...@gmail.com>.
yes, that's right, please attach the logs if this doesn't work for you.
lewinma(lewinma) <le...@tencent.com> 于2019年9月18日周三 上午11:44写道：
> Sounds great, are there any doc about that?
> If only config the Flink job manager’s ip and port is enough.
> Thanks a lot.
>
> On Sep 18, 2019, at 11:39 AM, Jeff Zhang <zj...@gmail.com> wrote:
>
> I think it is already supported. The flink interpreter will still run out
> of k8s, but the flink cluster runs in k8s, then as long as configure flink
> interpreter properly, you can connect the flink cluster in k8s
>
> lewinma(lewinma) <le...@tencent.com> 于2019年9月18日周三 上午10:38写道：
>
>> Do we have a plan of supporting Flink running on K8s?
>>
>> Best Regards~
>> Lewinma
>>
>> On Sep 17, 2019, at 9:32 PM, Jeff Zhang <zj...@gmail.com> wrote:
>>
>> Maybe in next Month (Sep)
>>
>>
>> mahmoud.khaled <ma...@incorta.com> 于2019年9月17日周二 下午7:21写道：
>>
>>> Hi,
>>>
>>> What's the planned date for 0.9 Release?
>>>
>>> Thanks in advance.
>>>
>>>
>>>
>>> --
>>> Sent from:
>>> http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/
>>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>>
>>
>
> --
> Best Regards
>
> Jeff Zhang
>
>
>
--
Best Regards
Jeff Zhang
Re: [DISCUSS] Roadmap 0.9 and future(Internet mail)
Posted by "lewinma(lewinma)" <le...@tencent.com>.
Sounds great, are there any doc about that?
If only config the Flink job manager’s ip and port is enough.
Thanks a lot.
On Sep 18, 2019, at 11:39 AM, Jeff Zhang <zj...@gmail.com>> wrote:
I think it is already supported. The flink interpreter will still run out of k8s, but the flink cluster runs in k8s, then as long as configure flink interpreter properly, you can connect the flink cluster in k8s
lewinma(lewinma) <le...@tencent.com>> 于2019年9月18日周三 上午10:38写道：
Do we have a plan of supporting Flink running on K8s?
Best Regards~
Lewinma
On Sep 17, 2019, at 9:32 PM, Jeff Zhang <zj...@gmail.com>> wrote:
Maybe in next Month (Sep)
mahmoud.khaled <ma...@incorta.com>> 于2019年9月17日周二 下午7:21写道：
Hi,
What's the planned date for 0.9 Release?
Thanks in advance.
--
Sent from: http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/
--
Best Regards
Jeff Zhang
--
Best Regards
Jeff Zhang
Re: [DISCUSS] Roadmap 0.9 and future(Internet mail)
Posted by Jeff Zhang <zj...@gmail.com>.
I think it is already supported. The flink interpreter will still run out
of k8s, but the flink cluster runs in k8s, then as long as configure flink
interpreter properly, you can connect the flink cluster in k8s
lewinma(lewinma) <le...@tencent.com> 于2019年9月18日周三 上午10:38写道：
> Do we have a plan of supporting Flink running on K8s?
>
> Best Regards~
> Lewinma
>
> On Sep 17, 2019, at 9:32 PM, Jeff Zhang <zj...@gmail.com> wrote:
>
> Maybe in next Month (Sep)
>
>
> mahmoud.khaled <ma...@incorta.com> 于2019年9月17日周二 下午7:21写道：
>
>> Hi,
>>
>> What's the planned date for 0.9 Release?
>>
>> Thanks in advance.
>>
>>
>>
>> --
>> Sent from:
>> http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/
>>
>
>
> --
> Best Regards
>
> Jeff Zhang
>
>
>
--
Best Regards
Jeff Zhang
Re: [DISCUSS] Roadmap 0.9 and future(Internet mail)
Posted by "lewinma(lewinma)" <le...@tencent.com>.
Do we have a plan of supporting Flink running on K8s?
Best Regards~
Lewinma
On Sep 17, 2019, at 9:32 PM, Jeff Zhang <zj...@gmail.com>> wrote:
Maybe in next Month (Sep)
mahmoud.khaled <ma...@incorta.com>> 于2019年9月17日周二 下午7:21写道：
Hi,
What's the planned date for 0.9 Release?
Thanks in advance.
--
Sent from: http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/
--
Best Regards
Jeff Zhang
Re: [DISCUSS] Roadmap 0.9 and future
Posted by Jeff Zhang <zj...@gmail.com>.
Maybe in next Month (Sep)
mahmoud.khaled <ma...@incorta.com> 于2019年9月17日周二 下午7:21写道：
> Hi,
>
> What's the planned date for 0.9 Release?
>
> Thanks in advance.
>
>
>
> --
> Sent from:
> http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/
>
--
Best Regards
Jeff Zhang
Re: [DISCUSS] Roadmap 0.9 and future
Posted by "mahmoud.khaled" <ma...@incorta.com>.
Hi,
What's the planned date for 0.9 Release?
Thanks in advance.
--
Sent from: http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/
Re: [DISCUSS] Roadmap 0.9 and future
Posted by Ruslan Dautkhanov <da...@gmail.com>.
Thanks for sharing this moon !
Those are great ideas.
--
Ruslan Dautkhanov
On Wed, Mar 7, 2018 at 11:21 AM, moon soo Lee <mo...@apache.org> wrote:
> Hi forks,
>
> There were an offline meeting yesterday at PaloAlto with contributors and
> users. We've shared idea about current state of project and future project
> roadmap and wishlists (meeting note [1]). It was really inspiring and
> exciting time. Let me try summarize, move this discussion to online.
>
> There were many ideas related to Interpreter. Especially, there were
> consensus that Spark support is one of biggest strength of Zeppelin and
> need to make further improvement to keep the strengths.
>
> - Spark
> - Immediate support of new spark release
> - Ramp up support of current Spark feature (e.g. Display job
> progress correctly)
> - Spark streaming support
> - Handling Livy timeout
> - Other interpreters
> - Better Hive support (e.g. configuration)
> - Latest version PrestoDB support (pass property correctly)
> - Run interpreter in containerized environment
> - Let individual user upload custom library from user's machine
> directly
> - Interpreter documentation is not detail enough
>
> And people in the meeting excited about ConInterpreter ZEPPELIN-3085 [2]
> in upcoming release, regarding dynamic/inline configuration of interpreter.
>
> And there were ideas on other areas, too. like
>
> - Separate Admin role and user role
> - Sidebar with plugin widget
> - Better integration with emerging framework like Tensorflow/MXNet/Ray
> - Sharing data
> - Schedule notebook from external scheduler
>
> Regarding scheduling notebook, Luciano shared his project NotebookTools[3]
> and it made people really excited.
>
> Also, there were inspiring discussions about the community/project.
> Current status and how can we make community/project more healthy. And
> here's some ideas around the topic
>
> - Need more frequent release
> - More attention to code review to speed up
> - Publishing roadmap beforehand to help contribution
> - 'Newbie', 'low hanging fruit' tag helps contribution
> - Enterprise friendly is another biggest strength of Zeppelin (in
> addition to Spark support) need to keep improve.
>
>
> I probably missed many idea shared yesterday. Please feel free to
> add/correct the summary. Hope more people in the mailinglist join and
> develop the idea together. And I think this discussion can leads community
> shape 0.9 and future version of Zeppelin, and update and publish future
> roadmap[4].
>
> Best,
> moon
>
> Special thanks to ZEPL <https://www.zepl.com> for the swag and dinner.
>
> [1] https://docs.google.com/document/d/18Wc3pEFx3qm9XoME_
> V_B9k_LlAd1PLyKQQEveR1on1o/edit?usp=sharing
> [2] https://issues.apache.org/jira/browse/ZEPPELIN-3085
> [3] https://github.com/SparkTC/notebook-exporter/
> tree/master/notebook-exporter
> [4] https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap
>
>
>
>