You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by Girish Reddy <gi...@springml.com> on 2016/03/09 16:43:49 UTC

Stop execution if a paragraph errors

Hello:

It seems the default behavior in Zeppelin is to continue execution of
paragraph even if one of the paragraph errors out.  Is there a way to
configure it to stop execution as soon as a paragraph errors out?  Or is
this something that needs to be handled in the paragraph code?

Thanks,
-Girish

RE: Stop execution if a paragraph errors

Posted by "Guha, Ayan (AU - Melbourne)" <ay...@deloitte.com.au>.
Hi

This is my first mail to this group. I am writing this in order to get a fair understanding of how zeppelin can be integrated with Spark.

Our use case is to load few tables from a DB to Spark, run some transformation. Once done, we want to expose data through Zeppelin for analytics. I have few question around that to sound off any gross architectural flaws.

Questions:

1. How Zeppelin connects to Spark? Thrift JDBC? How is it different than JDBC server?

2. What is the scope of Spark application when it is used from Zeppelin? For example, if I have few subsequent actions in zeppelin like map,filter,reduceByKey, filter,collect. I assume this will translate to an application and get submitted to Spark. However, If I want to use reuse some part of the data (for example) after first map transformation in earlier application. Can I do it? Or will it be another application and another spark submit?

 In our use case data will already be loaded in RDDs. So how Zeppelin can access it? Is it even possible?

3. How can I control access on specific rdds to specific users in Zeppelin (assuming we have implemented some way of login mechanism in Zeppelin and we have a mapping between Zeppelin users and their LDAP accounts). Is it even possible?

appreciate any help/pointers/guidance.





-- 



This e-mail and any attachments to it are confidential. You must not use, 
disclose or act on the e-mail if you are not the intended recipient. If you 
have 

received this e-mail in error, please let us know by contacting the sender 
and deleting the original e-mail.
Liability limited by a scheme approved under Professional Standards 
Legislation. 
Deloitte refers to one or more of 

Deloitte Touche Tohmatsu Limited, a UK private company limited by 
guarantee, and its network of member firms, each of which is a legally 
separate and 

independent entity. Please see www.deloitte.com.au/about 
<http://www.deloitte.com/au/about> for a detailed 

description of the legal structure of Deloitte Touche Tohmatsu Limited and 
its member firms.



Re: Stop execution if a paragraph errors

Posted by moon soo Lee <mo...@apache.org>.
Hi Skanda,

Thanks for sharing great idea. I think you idea is really make sense. :-)

Thanks,
moon

On Wed, Mar 9, 2016 at 11:04 AM Skanda <sk...@gmail.com> wrote:

> Hi Moon & Girish
>
> The note has a runAll method that is being used by the cron scheduler.
> This method iterates the paragraphs and submits each of them for execution
> in a single go. This can be made to run sequentially by checking the
> paragraph status before triggering the next paragraph. We can have a config
> in paragraph settings in ui say "enable skip on failure" which can be
> checked to see whether next  paragraph should be triggered based on the
> status of the current paragraph. Also the notebook run can be modified to
> use this runAll method. Your thoughts pls.
>
> Regards
>
>
> Skanda
> On 09-Mar-2016 11:38 PM, "moon soo Lee" <mo...@apache.org> wrote:
>
>> Hi Girish,
>>
>> Currently, I think there're no simple way. Please feel free to file an
>> issue on JIRA.
>>
>> Technically, RunNote function iterates runParagraph in front-end side
>> [1].  To make execution stop on paragraph error,
>>
>> a) Need to iterate it on backend side by calling [2], otherwise closing
>> browser window may stop notebook run.
>> b) [2] need to be modified to not use scheduler from individual
>> interpreter but use another FIFO scheduler that run all job sequentially.
>> c) and maybe it's good to an option to choose, current behavior and this
>> new behavior in front-end side.
>>
>> Thanks,
>> moon
>>
>> [1]
>> https://github.com/apache/incubator-zeppelin/blob/master/zeppelin-web/src/app/notebook/notebook.controller.js#L172
>> [2]
>> https://github.com/apache/incubator-zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Note.java#L397
>>
>>
>> On Wed, Mar 9, 2016 at 7:44 AM Girish Reddy <gi...@springml.com> wrote:
>>
>>> Hello:
>>>
>>> It seems the default behavior in Zeppelin is to continue execution of
>>> paragraph even if one of the paragraph errors out.  Is there a way to
>>> configure it to stop execution as soon as a paragraph errors out?  Or is
>>> this something that needs to be handled in the paragraph code?
>>>
>>> Thanks,
>>> -Girish
>>>
>>>
>>>

Re: Stop execution if a paragraph errors

Posted by Skanda <sk...@gmail.com>.
Hi Moon & Girish

The note has a runAll method that is being used by the cron scheduler. This
method iterates the paragraphs and submits each of them for execution in a
single go. This can be made to run sequentially by checking the paragraph
status before triggering the next paragraph. We can have a config in
paragraph settings in ui say "enable skip on failure" which can be checked
to see whether next  paragraph should be triggered based on the status of
the current paragraph. Also the notebook run can be modified to use this
runAll method. Your thoughts pls.

Regards
Skanda
On 09-Mar-2016 11:38 PM, "moon soo Lee" <mo...@apache.org> wrote:

> Hi Girish,
>
> Currently, I think there're no simple way. Please feel free to file an
> issue on JIRA.
>
> Technically, RunNote function iterates runParagraph in front-end side
> [1].  To make execution stop on paragraph error,
>
> a) Need to iterate it on backend side by calling [2], otherwise closing
> browser window may stop notebook run.
> b) [2] need to be modified to not use scheduler from individual
> interpreter but use another FIFO scheduler that run all job sequentially.
> c) and maybe it's good to an option to choose, current behavior and this
> new behavior in front-end side.
>
> Thanks,
> moon
>
> [1]
> https://github.com/apache/incubator-zeppelin/blob/master/zeppelin-web/src/app/notebook/notebook.controller.js#L172
> [2]
> https://github.com/apache/incubator-zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Note.java#L397
>
>
> On Wed, Mar 9, 2016 at 7:44 AM Girish Reddy <gi...@springml.com> wrote:
>
>> Hello:
>>
>> It seems the default behavior in Zeppelin is to continue execution of
>> paragraph even if one of the paragraph errors out.  Is there a way to
>> configure it to stop execution as soon as a paragraph errors out?  Or is
>> this something that needs to be handled in the paragraph code?
>>
>> Thanks,
>> -Girish
>>
>>
>>

Re: Stop execution if a paragraph errors

Posted by moon soo Lee <mo...@apache.org>.
Hi Girish,

Currently, I think there're no simple way. Please feel free to file an
issue on JIRA.

Technically, RunNote function iterates runParagraph in front-end side [1].
To make execution stop on paragraph error,

a) Need to iterate it on backend side by calling [2], otherwise closing
browser window may stop notebook run.
b) [2] need to be modified to not use scheduler from individual interpreter
but use another FIFO scheduler that run all job sequentially.
c) and maybe it's good to an option to choose, current behavior and this
new behavior in front-end side.

Thanks,
moon

[1]
https://github.com/apache/incubator-zeppelin/blob/master/zeppelin-web/src/app/notebook/notebook.controller.js#L172
[2]
https://github.com/apache/incubator-zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Note.java#L397


On Wed, Mar 9, 2016 at 7:44 AM Girish Reddy <gi...@springml.com> wrote:

> Hello:
>
> It seems the default behavior in Zeppelin is to continue execution of
> paragraph even if one of the paragraph errors out.  Is there a way to
> configure it to stop execution as soon as a paragraph errors out?  Or is
> this something that needs to be handled in the paragraph code?
>
> Thanks,
> -Girish
>
>
>