[jira] [Comment Edited] (OOZIE-3712) Oozie Verbose Option at Workflow Level which sets Java Verbose to all JVMs involved in the actions

["Prabhu Joseph (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OOZIE-3712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655423#comment-17655423 ] 

Prabhu Joseph edited comment on OOZIE-3712 at 1/6/23 12:30 PM:
---------------------------------------------------------------

Thanks [~dionusos]  for reviewing this.
{quote} May I suggest to have this property fine tuned for system level, for workflow level and for action level too? 
{quote}
Yes this gives more control to user. Will use this way.
{quote}Also what do you suggest as a "verbose" option, what debug level should be turned on? Verbose class loading? Debug/Trace level logging for the given action? Should we allow the user to somehow configure in an easy way (switches) what verbose options they want?
{quote}
I had only Java verbose class loading option (-verbose:class) in mind, as that is useful to find jar conflicts leading to NoSuchMethodErrors. But we could use it to configure any common JVM flags which is needed across all JVMs part of a specific Action or all Actions.

Examples:

A. Enable Java Verbose Class Loading
{code:java}
oozie.workflow.action.common.jvmflags=-verbose:class
{code}
B. Enable log4j debug option
{code:java}
oozie.workflow.action.common.jvmflags=-Dlog4j.debug
{code}
C. Print GC Details
{code:java}
oozie.workflow.action.common.jvmflags=-XX:+PrintGCDetails and -XX:+PrintGCTimeStamps
{code}
 

 bq. Anyway, I like the idea. Do you plan working on it?

Yes, my teammate has started working on it. Will assign it to her once JIRA Account is created.

 

 


was (Author: prabhu joseph):
Thanks [~dionusos]  for reviewing this.
{quote} May I suggest to have this property fine tuned for system level, for workflow level and for action level too? 
{quote}
Yes this gives more control to user. Will use this way.
{quote}Also what do you suggest as a "verbose" option, what debug level should be turned on? Verbose class loading? Debug/Trace level logging for the given action? Should we allow the user to somehow configure in an easy way (switches) what verbose options they want?
{quote}
I had only Java verbose class loading option (-verbose:class) in mind, as that is useful to find jar conflicts leading to NoSuchMethodErrors. But we could use it configure any common JVM flags with default maybe -verbose:class and which will be set across all JVMs part of specific Action or all Actions.

Examples:

A. Enable Java Verbose Class Loading
{code:java}
oozie.workflow.action.common.jvmflags=-verbose:class
{code}
B. Enable log4j debug option
{code:java}
oozie.workflow.action.common.jvmflags=-Dlog4j.debug
{code}
C. Print GC Details
{code:java}
oozie.workflow.action.common.jvmflags=-XX:+PrintGCDetails and -XX:+PrintGCTimeStamps
{code}
 

 bq. Anyway, I like the idea. Do you plan working on it?

Yes, my teammate has started working on it. Will assign it to her once JIRA Account is created.

 

 

> Oozie Verbose Option at Workflow Level which sets Java Verbose to all JVMs involved in the actions
> --------------------------------------------------------------------------------------------------
>
>                 Key: OOZIE-3712
>                 URL: https://issues.apache.org/jira/browse/OOZIE-3712
>             Project: Oozie
>          Issue Type: Improvement
>            Reporter: Prabhu Joseph
>            Assignee: Prabhu Joseph
>            Priority: Major
>
> The most common issue in Oozie is NoSuhMethodError. And the best way to debug is to set the JVM option -verbose to the right JVM which has failed. The challenge is to identify which JVM failed and what is the OPTS option through which -verbose can be set. In MapReduce Action, MRAppMaster, MapTask and ReduceTask each have different way of passing OPTS. In Spark Action, Spark ApplicationMaster, Driver and Executors have different way of passing OPTS.
> This Jira intends to expose one config say oozie.workflow.enable.verbose=true in job.properties to user and internally Oozie sets -verbose to all the JVMs involved in the Actions (SparkAction, SqoopAction or MapReduceAction). This will be very useful for debugging the most common issue NoSuhMethodError.
>  
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)