[jira] [Commented] (OOZIE-3712) Oozie Verbose Option at Workflow Level which sets Java Verbose to all JVMs involved in the actions

["Dénes Bodó (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OOZIE-3712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655364#comment-17655364 ] 

Dénes Bodó commented on OOZIE-3712:
-----------------------------------

Hey [~prabhujoseph] 

Thanks for mentioning this issue. I agree that now turning on the debug level logging in Oozie actions is not like turning on a switch.

You mentioned that {{oozie.workflow.enable.verbose=true}} would be a job.properties configuration which means it will be applied to all the action in the workflow. I saw cases where a workflow contained dozens of actions and maybe the user does not want to turn on verbose class loading and such for all these actions but to only some of them. May I suggest to have this property fine tuned for system level, for workflow level and for action level too? So
 * if we set it in oozie-site.xml then all the action will be in debug. 
 * if we set it in job.properties or workflow.xml global configuration then all the action in the given workflow will be in debug.
 * if we set it in a workflow action's configuration then only that action will be in debug.

Also what do you suggest as a "verbose" option, what debug level should be turned on? Verbose class loading? Debug/Trace level logging for the given action? Should we allow the user to somehow configure in an easy way (switches) what verbose options they want?

 

Anyway, I like the idea. Do you plan working on it?

 

Thanks,

Denes

> Oozie Verbose Option at Workflow Level which sets Java Verbose to all JVMs involved in the actions
> --------------------------------------------------------------------------------------------------
>
>                 Key: OOZIE-3712
>                 URL: https://issues.apache.org/jira/browse/OOZIE-3712
>             Project: Oozie
>          Issue Type: Improvement
>            Reporter: Prabhu Joseph
>            Assignee: Prabhu Joseph
>            Priority: Major
>
> The most common issue in Oozie is NoSuhMethodError. And the best way to debug is to set the JVM option -verbose to the right JVM which has failed. The challenge is to identify which JVM failed and what is the OPTS option through which -verbose can be set. In MapReduce Action, MRAppMaster, MapTask and ReduceTask each have different way of passing OPTS. In Spark Action, Spark ApplicationMaster, Driver and Executors have different way of passing OPTS.
> This Jira intends to expose one config say oozie.workflow.enable.verbose=true in job.properties to user and internally Oozie sets -verbose to all the JVMs involved in the Actions (SparkAction, SqoopAction or MapReduceAction). This will be very useful for debugging the most common issue NoSuhMethodError.
>  
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)