You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Daniel Zhang <ja...@hotmail.com> on 2019/04/01 22:29:13 UTC

Oozie 5.0 coordinator issue

Hi, I am facing a strange issue on Oozie 5.0.

I have an Ozzie application that works fine in the workflow mode.
Here is the only 2 examples line of my oozie properties file:


oozie.wf.application.path=${appBaseDir}/workflow/workflow.xml
#oozie.coord.application.path=${appBaseDir}/workflow/coordinator.xml

And here is the content of the coordinator.xml file:

<coordinator-app name="GoogleAnalytics" start="2018-09-06T23:25Z"
                 end="2100-12-31T23:00Z" frequency="${coord:days(1)}" timezone="UTC"
                 xmlns="uri:oozie:coordinator:0.4">
    <action>
        <workflow>
            <app-path>${appBaseDir}/workflow/workflow.xml</app-path>
            <configuration>
                <property>
                    <name>nameNode</name>
                    <value>${nameNode}</value>
                </property>
                <property>
                    <name>jobTracker</name>
                    <value>${jobTracker}</value>
                </property>
            </configuration>
        </workflow>
    </action>
</coordinator-app>

Now, this whole thing works on AWS EMR with Oozie 4.3. But with the new version of AWS EMR comes with Oozie 5.0.0, I am facing the following strange error.

1) If I leave the properties file as above to running in the workflow mode by "oozie job -run -config googleanalytics_aws_qa.properties file, everything works under oozie 5.0
2) If I comment out the "wf" line above and uncomment out the "coord" line, I got the following error:

[hadoop@ip-10-0-60-152 oozieJobs]$ oozie job -run -config googleanalytics_aws_qa.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/oozie/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/oozie/lib/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Error: E0307 : E0307: Runtime error [App directory [null] does not exist and app definition cannot be created because of missing config value [oozie.jobs.api.generated.xml]]

And here is the stack trace in the oozie.log:
2019-04-01 22:03:55,806  WARN V1JobsServlet:523 - SERVER[ip-10-0-60-152.ec2.internal] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] URL[POST http://ip-10-0-60-152.ec2.internal:11000/oozie/v2/jobs?action=start&user=hadoop] error[E0307], E0307: Runtime error [App directory [null] does not exist and app definition cannot be created because of missing config value [oozie.jobs.api.generated.xml]]
org.apache.oozie.servlet.XServletException: E0307: Runtime error [App directory [null] does not exist and app definition cannot be created because of missing config value [oozie.jobs.api.generated.xml]]
        at org.apache.oozie.servlet.V1JobsServlet.checkAndWriteApplicationXMLToHDFS(V1JobsServlet.java:175)
        at org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:111)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
        at org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:305)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
        at org.apache.oozie.servlet.AuthFilter$2.doFilter(AuthFilter.java:171)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:644)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592)
        at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:176)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
        at org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:86)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
        at org.eclipse.jetty.server.Server.handle(Server.java:534)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
        at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)

I checked again and again, and I have no idea what this "[App directory [null]" coming from.

  *   I keep using the old XSD as workflow 0.3 and coordinator:0.5 under oozie 5.0, that should be supported right?
  *   I didn't see any issue of the application path in the properties file, and as it works under "workflow" mode, so there should be no issue.
  *   Since this error ONLY happens in the coordinator mode, and the only issue COULD come from the coordinator.xml. As I pasted above, it is so simple, and I really not sure where the issue comes from.
  *   I even use the XSD from https://oozie.apache.org/docs/5.0.0/CoordinatorFunctionalSpec.html#Oozie_Coordinator_Schema_0.4 to validate my coordinator.xml, and it is valid.
  *   So my questions are:
     *   Where is this App directory [null] coming from?
     *   The root reason for failure is due to AppDirectory is NULL, or definition cannot be created because of missing config value. If later, what kind of config value is missing we are talking about here?
     *   If it is due to AppDirectory is NULL, what could cause it ONLY happens in Coord mode, but work fine in Workflow mode?


Thanks

Yong

Re: Oozie 5.0 coordinator issue

Posted by Daniel Zhang <ja...@hotmail.com>.
Yes, you are right. The issue happened on EMR 5.22 for us, which comes with Oozie 5.1.0

I will test a lower version of EMR with Oozie 5.0.0. From the JIRA, it looks like this issue only happens since Oozie 5.1.0

For us, we are currently using EMR 5.12.2 with Oozie 4.3.0, and everything works fine.

But we need to upgrade for newer Spark version.

Yong

________________________________
From: Jerry Chabot <je...@microfocus.com>
Sent: Monday, April 1, 2019 6:54 PM
To: user@oozie.apache.org
Subject: RE: Oozie 5.0 coordinator issue

What EMR version are you using? Can you confirm you are using Oozie 5.0? I ask because I encountered a similar problem when we went from Oozie 5.0 to 5.1 while upgrading to EMR 5.22. I believe oozie.jobs.api.generated.xml  was added in Oozie 5.1. See https://issues.apache.org/jira/browse/OOZIE-3458, https://issues.apache.org/jira/browse/OOZIE-3457 and  related  bugs for details. An important point is the code reports "null" when that isn't the real problem.

-Jerry

-----Original Message-----
From: Daniel Zhang [mailto:java8964@hotmail.com]
Sent: Monday, April 1, 2019 6:29 PM
To: user@oozie.apache.org
Subject: Oozie 5.0 coordinator issue

Hi, I am facing a strange issue on Oozie 5.0.

I have an Ozzie application that works fine in the workflow mode.
Here is the only 2 examples line of my oozie properties file:


oozie.wf.application.path=${appBaseDir}/workflow/workflow.xml
#oozie.coord.application.path=${appBaseDir}/workflow/coordinator.xml

And here is the content of the coordinator.xml file:

<coordinator-app name="GoogleAnalytics" start="2018-09-06T23:25Z"
                 end="2100-12-31T23:00Z" frequency="${coord:days(1)}" timezone="UTC"
                 xmlns="uri:oozie:coordinator:0.4">
    <action>
        <workflow>
            <app-path>${appBaseDir}/workflow/workflow.xml</app-path>
            <configuration>
                <property>
                    <name>nameNode</name>
                    <value>${nameNode}</value>
                </property>
                <property>
                    <name>jobTracker</name>
                    <value>${jobTracker}</value>
                </property>
            </configuration>
        </workflow>
    </action>
</coordinator-app>

Now, this whole thing works on AWS EMR with Oozie 4.3. But with the new version of AWS EMR comes with Oozie 5.0.0, I am facing the following strange error.

1) If I leave the properties file as above to running in the workflow mode by "oozie job -run -config googleanalytics_aws_qa.properties file, everything works under oozie 5.0
2) If I comment out the "wf" line above and uncomment out the "coord" line, I got the following error:

[hadoop@ip-10-0-60-152 oozieJobs]$ oozie job -run -config googleanalytics_aws_qa.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/oozie/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/oozie/lib/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Error: E0307 : E0307: Runtime error [App directory [null] does not exist and app definition cannot be created because of missing config value [oozie.jobs.api.generated.xml]]

And here is the stack trace in the oozie.log:
2019-04-01 22:03:55,806  WARN V1JobsServlet:523 - SERVER[ip-10-0-60-152.ec2.internal] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] URL[POST http://ip-10-0-60-152.ec2.internal:11000/oozie/v2/jobs?action=start&user=hadoop] error[E0307], E0307: Runtime error [App directory [null] does not exist and app definition cannot be created because of missing config value [oozie.jobs.api.generated.xml]]
org.apache.oozie.servlet.XServletException: E0307: Runtime error [App directory [null] does not exist and app definition cannot be created because of missing config value [oozie.jobs.api.generated.xml]]
        at org.apache.oozie.servlet.V1JobsServlet.checkAndWriteApplicationXMLToHDFS(V1JobsServlet.java:175)
        at org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:111)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
        at org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:305)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
        at org.apache.oozie.servlet.AuthFilter$2.doFilter(AuthFilter.java:171)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:644)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592)
        at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:176)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
        at org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:86)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
        at org.eclipse.jetty.server.Server.handle(Server.java:534)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
        at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)

I checked again and again, and I have no idea what this "[App directory [null]" coming from.

  *   I keep using the old XSD as workflow 0.3 and coordinator:0.5 under oozie 5.0, that should be supported right?
  *   I didn't see any issue of the application path in the properties file, and as it works under "workflow" mode, so there should be no issue.
  *   Since this error ONLY happens in the coordinator mode, and the only issue COULD come from the coordinator.xml. As I pasted above, it is so simple, and I really not sure where the issue comes from.
  *   I even use the XSD from https://oozie.apache.org/docs/5.0.0/CoordinatorFunctionalSpec.html#Oozie_Coordinator_Schema_0.4 to validate my coordinator.xml, and it is valid.
  *   So my questions are:
     *   Where is this App directory [null] coming from?
     *   The root reason for failure is due to AppDirectory is NULL, or definition cannot be created because of missing config value. If later, what kind of config value is missing we are talking about here?
     *   If it is due to AppDirectory is NULL, what could cause it ONLY happens in Coord mode, but work fine in Workflow mode?


Thanks

Yong

RE: Oozie 5.0 coordinator issue

Posted by Jerry Chabot <je...@microfocus.com>.
What EMR version are you using? Can you confirm you are using Oozie 5.0? I ask because I encountered a similar problem when we went from Oozie 5.0 to 5.1 while upgrading to EMR 5.22. I believe oozie.jobs.api.generated.xml  was added in Oozie 5.1. See https://issues.apache.org/jira/browse/OOZIE-3458, https://issues.apache.org/jira/browse/OOZIE-3457 and  related  bugs for details. An important point is the code reports "null" when that isn't the real problem.

-Jerry

-----Original Message-----
From: Daniel Zhang [mailto:java8964@hotmail.com] 
Sent: Monday, April 1, 2019 6:29 PM
To: user@oozie.apache.org
Subject: Oozie 5.0 coordinator issue

Hi, I am facing a strange issue on Oozie 5.0.

I have an Ozzie application that works fine in the workflow mode.
Here is the only 2 examples line of my oozie properties file:


oozie.wf.application.path=${appBaseDir}/workflow/workflow.xml
#oozie.coord.application.path=${appBaseDir}/workflow/coordinator.xml

And here is the content of the coordinator.xml file:

<coordinator-app name="GoogleAnalytics" start="2018-09-06T23:25Z"
                 end="2100-12-31T23:00Z" frequency="${coord:days(1)}" timezone="UTC"
                 xmlns="uri:oozie:coordinator:0.4">
    <action>
        <workflow>
            <app-path>${appBaseDir}/workflow/workflow.xml</app-path>
            <configuration>
                <property>
                    <name>nameNode</name>
                    <value>${nameNode}</value>
                </property>
                <property>
                    <name>jobTracker</name>
                    <value>${jobTracker}</value>
                </property>
            </configuration>
        </workflow>
    </action>
</coordinator-app>

Now, this whole thing works on AWS EMR with Oozie 4.3. But with the new version of AWS EMR comes with Oozie 5.0.0, I am facing the following strange error.

1) If I leave the properties file as above to running in the workflow mode by "oozie job -run -config googleanalytics_aws_qa.properties file, everything works under oozie 5.0
2) If I comment out the "wf" line above and uncomment out the "coord" line, I got the following error:

[hadoop@ip-10-0-60-152 oozieJobs]$ oozie job -run -config googleanalytics_aws_qa.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/oozie/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/oozie/lib/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Error: E0307 : E0307: Runtime error [App directory [null] does not exist and app definition cannot be created because of missing config value [oozie.jobs.api.generated.xml]]

And here is the stack trace in the oozie.log:
2019-04-01 22:03:55,806  WARN V1JobsServlet:523 - SERVER[ip-10-0-60-152.ec2.internal] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] URL[POST http://ip-10-0-60-152.ec2.internal:11000/oozie/v2/jobs?action=start&user=hadoop] error[E0307], E0307: Runtime error [App directory [null] does not exist and app definition cannot be created because of missing config value [oozie.jobs.api.generated.xml]]
org.apache.oozie.servlet.XServletException: E0307: Runtime error [App directory [null] does not exist and app definition cannot be created because of missing config value [oozie.jobs.api.generated.xml]]
        at org.apache.oozie.servlet.V1JobsServlet.checkAndWriteApplicationXMLToHDFS(V1JobsServlet.java:175)
        at org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:111)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
        at org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:305)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
        at org.apache.oozie.servlet.AuthFilter$2.doFilter(AuthFilter.java:171)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:644)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592)
        at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:176)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
        at org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:86)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
        at org.eclipse.jetty.server.Server.handle(Server.java:534)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
        at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)

I checked again and again, and I have no idea what this "[App directory [null]" coming from.

  *   I keep using the old XSD as workflow 0.3 and coordinator:0.5 under oozie 5.0, that should be supported right?
  *   I didn't see any issue of the application path in the properties file, and as it works under "workflow" mode, so there should be no issue.
  *   Since this error ONLY happens in the coordinator mode, and the only issue COULD come from the coordinator.xml. As I pasted above, it is so simple, and I really not sure where the issue comes from.
  *   I even use the XSD from https://oozie.apache.org/docs/5.0.0/CoordinatorFunctionalSpec.html#Oozie_Coordinator_Schema_0.4 to validate my coordinator.xml, and it is valid.
  *   So my questions are:
     *   Where is this App directory [null] coming from?
     *   The root reason for failure is due to AppDirectory is NULL, or definition cannot be created because of missing config value. If later, what kind of config value is missing we are talking about here?
     *   If it is due to AppDirectory is NULL, what could cause it ONLY happens in Coord mode, but work fine in Workflow mode?


Thanks

Yong