You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@falcon.apache.org by Josh Clum <jo...@gmail.com> on 2014/08/13 18:54:43 UTC
Process NPE
Hi, I'm getting the following NPE when i launch with "falcon entity -type
process -schedule -name raw-cc-bp-ratio-lcms-comp-process". At the bottom
is my process which has no inputs.
If I look at
https://github.com/apache/incubator-falcon/blob/branch-0.5/common/src/main/java/org/apache/falcon/entity/ProcessHelper.java
it seems that this should never happen because process.getInputs() ==
null should
be true and should avoid the NPE. Any thoughts?
2014-08-13 16:49:05,152 ERROR -
[6221571@qtp-1696932027-3:vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
a38b5528-cae0-4b07-a8fc-482661e467a9] ~ Unable to schedule workflow
(AbstractSchedulableEntityManager:60)
java.lang.NullPointerException
at
org.apache.falcon.entity.ProcessHelper.getStorageType(ProcessHelper.java:57)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.isTableStorageType(OozieProcessWorkflowBuilder.java:646)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.shouldSetupHiveConfiguration(OozieProcessWorkflowBuilder.java:641)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createWorkflow(OozieProcessWorkflowBuilder.java:606)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createDefaultCoordinator(OozieProcessWorkflowBuilder.java:291)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.getCoordinators(OozieProcessWorkflowBuilder.java:206)
at
org.apache.falcon.workflow.OozieWorkflowBuilder.map(OozieWorkflowBuilder.java:142)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.newWorkflowSchedule(OozieProcessWorkflowBuilder.java:111)
at
org.apache.falcon.workflow.engine.OozieWorkflowEngine.schedule(OozieWorkflowEngine.java:119)
at
org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:68)
at
org.apache.falcon.resource.AbstractSchedulableEntityManager.schedule(AbstractSchedulableEntityManager.java:57)
at
org.apache.falcon.resource.SchedulableEntityManager.schedule(SchedulableEntityManager.java:85)
at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.falcon.resource.channel.IPCChannel.invoke(IPCChannel.java:48)
at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$7.doExecute(SchedulableEntityManagerProxy.java:324)
at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$EntityProxy.execute(SchedulableEntityManagerProxy.java:410)
at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule_aroundBody10(SchedulableEntityManagerProxy.java:326)
at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$AjcClosure11.run(SchedulableEntityManagerProxy.java:1)
at
org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
at
org.apache.falcon.aspect.AbstractFalconAspect.logAround(AbstractFalconAspect.java:50)
at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule(SchedulableEntityManagerProxy.java:310)
at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
at
org.apache.falcon.security.BasicAuthFilter$2.doFilter(BasicAuthFilter.java:183)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:392)
at
org.apache.falcon.security.BasicAuthFilter.doFilter(BasicAuthFilter.java:221)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
2014-08-13 16:49:05,154 ERROR -
[6221571@qtp-1696932027-3:vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
a38b5528-cae0-4b07-a8fc-482661e467a9] ~ Failure reason
(FalconWebException:40)
java.lang.NullPointerException
at
org.apache.falcon.entity.ProcessHelper.getStorageType(ProcessHelper.java:57)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.isTableStorageType(OozieProcessWorkflowBuilder.java:646)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.shouldSetupHiveConfiguration(OozieProcessWorkflowBuilder.java:641)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createWorkflow(OozieProcessWorkflowBuilder.java:606)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createDefaultCoordinator(OozieProcessWorkflowBuilder.java:291)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.getCoordinators(OozieProcessWorkflowBuilder.java:206)
at
org.apache.falcon.workflow.OozieWorkflowBuilder.map(OozieWorkflowBuilder.java:142)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.newWorkflowSchedule(OozieProcessWorkflowBuilder.java:111)
at
org.apache.falcon.workflow.engine.OozieWorkflowEngine.schedule(OozieWorkflowEngine.java:119)
at
org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:68)
at
org.apache.falcon.resource.AbstractSchedulableEntityManager.schedule(AbstractSchedulableEntityManager.java:57)
at
org.apache.falcon.resource.SchedulableEntityManager.schedule(SchedulableEntityManager.java:85)
at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.falcon.resource.channel.IPCChannel.invoke(IPCChannel.java:48)
at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$7.doExecute(SchedulableEntityManagerProxy.java:324)
at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$EntityProxy.execute(SchedulableEntityManagerProxy.java:410)
at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule_aroundBody10(SchedulableEntityManagerProxy.java:326)
at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$AjcClosure11.run(SchedulableEntityManagerProxy.java:1)
at
org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
at
org.apache.falcon.aspect.AbstractFalconAspect.logAround(AbstractFalconAspect.java:50)
at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule(SchedulableEntityManagerProxy.java:310)
at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
at
org.apache.falcon.security.BasicAuthFilter$2.doFilter(BasicAuthFilter.java:183)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:392)
at
org.apache.falcon.security.BasicAuthFilter.doFilter(BasicAuthFilter.java:221)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
2014-08-13 16:49:05,154 ERROR -
[6221571@qtp-1696932027-3:vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
a38b5528-cae0-4b07-a8fc-482661e467a9] ~ Action failed: Bad Request
Error:null
(FalconWebException:66)
2014-08-13 16:49:05,155 INFO -
[6221571@qtp-1696932027-3:vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
a38b5528-cae0-4b07-a8fc-482661e467a9] ~ {Action:schedule,
Dimensions:{entityType=process, colo=*,
entityName=raw-cc-bp-ratio-lcms-comp-process}, Status: FAILED,
Time-taken:161608565 ns} (METRIC:32)
My process looks like this:
<process name="raw-cc-bp-ratio-lcms-comp-process"
xmlns="uri:falcon:process:0.1">
<tags>type=raw</tags>
<clusters>
<cluster name="minicluster">
<validity start="2014-08-13T00:00Z" end="2016-01-01T00:00Z"/>
</cluster>
</clusters>
<parallel>1</parallel>
<order>FIFO</order>
<timeout>hours(2)</timeout>
<frequency>days(1)</frequency>
<outputs>
<output name="output" feed="raw-cc-bp-ratio-lcms-comp-feed"
instance="now(0,0)" />
</outputs>
<properties>
...
</properties>
<workflow name="sqoop_wf" version="2.0.0" engine="oozie"
path="/apps/huron/ingest/standard_ingest/sqoop_wf.xml" />
<retry policy="periodic" delay="minutes(10)" attempts="3" />
</process>
Re: Process NPE
Posted by Seetharam Venkatesh <ve...@innerzeal.com>.
James, feel free to create a jira and provide a patch. Thanks!
On Wed, Aug 13, 2014 at 10:52 AM, James Grant <ja...@queeg.org> wrote:
> I'm having the same issue and was just about to send a mail when you
> posted.
>
> I've had a look at the code and the problem seems to be when the process
> has no inputs. The check in getStorageType used to short circuit if there
> were no inputs but that changed as part of the changes in FALCON-443 to
> only short circuit if the process has no inputs and no outputs.
>
> Below is a patch that will prevent the exception (currently untested).
>
> diff --git
> a/common/src/main/java/org/apache/falcon/entity/ProcessHelper.java
> b/common/src/main/java/org/apache/falcon/entity/ProcessHelper.java
> index 59361e8..33f0aad 100644
> --- a/common/src/main/java/org/apache/falcon/entity/ProcessHelper.java
> +++ b/common/src/main/java/org/apache/falcon/entity/ProcessHelper.java
> @@ -59,11 +59,13 @@ public final class ProcessHelper {
> return storageType;
> }
>
> - for (Input input : process.getInputs().getInputs()) {
> - Feed feed = EntityUtil.getEntity(EntityType.FEED,
> input.getFeed());
> - storageType = FeedHelper.getStorageType(feed, cluster);
> - if (Storage.TYPE.TABLE == storageType) {
> - break;
> + if (process.getInputs() != null) {
> + for (Input input : process.getInputs().getInputs()) {
> + Feed feed = EntityUtil.getEntity(EntityType.FEED,
> input.getFeed());
> + storageType = FeedHelper.getStorageType(feed, cluster);
> + if (Storage.TYPE.TABLE == storageType) {
> + break;
> + }
> }
> }
>
> James
>
>
> On 13 August 2014 17:54, Josh Clum <jo...@gmail.com> wrote:
>
> > Hi, I'm getting the following NPE when i launch with "falcon entity -type
> > process -schedule -name raw-cc-bp-ratio-lcms-comp-process". At the bottom
> > is my process which has no inputs.
> >
> > If I look at
> >
> >
> https://github.com/apache/incubator-falcon/blob/branch-0.5/common/src/main/java/org/apache/falcon/entity/ProcessHelper.java
> > it seems that this should never happen because process.getInputs() ==
> > null should
> > be true and should avoid the NPE. Any thoughts?
> >
> > 2014-08-13 16:49:05,152 ERROR -
> > [6221571@qtp-1696932027-3
> >
> :vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
> > a38b5528-cae0-4b07-a8fc-482661e467a9] ~ Unable to schedule workflow
> > (AbstractSchedulableEntityManager:60)
> > java.lang.NullPointerException
> > at
> >
> >
> org.apache.falcon.entity.ProcessHelper.getStorageType(ProcessHelper.java:57)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.isTableStorageType(OozieProcessWorkflowBuilder.java:646)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.shouldSetupHiveConfiguration(OozieProcessWorkflowBuilder.java:641)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createWorkflow(OozieProcessWorkflowBuilder.java:606)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createDefaultCoordinator(OozieProcessWorkflowBuilder.java:291)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.getCoordinators(OozieProcessWorkflowBuilder.java:206)
> > at
> >
> >
> org.apache.falcon.workflow.OozieWorkflowBuilder.map(OozieWorkflowBuilder.java:142)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.newWorkflowSchedule(OozieProcessWorkflowBuilder.java:111)
> > at
> >
> >
> org.apache.falcon.workflow.engine.OozieWorkflowEngine.schedule(OozieWorkflowEngine.java:119)
> > at
> >
> >
> org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:68)
> > at
> >
> >
> org.apache.falcon.resource.AbstractSchedulableEntityManager.schedule(AbstractSchedulableEntityManager.java:57)
> > at
> >
> >
> org.apache.falcon.resource.SchedulableEntityManager.schedule(SchedulableEntityManager.java:85)
> > at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at
> org.apache.falcon.resource.channel.IPCChannel.invoke(IPCChannel.java:48)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$7.doExecute(SchedulableEntityManagerProxy.java:324)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$EntityProxy.execute(SchedulableEntityManagerProxy.java:410)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule_aroundBody10(SchedulableEntityManagerProxy.java:326)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$AjcClosure11.run(SchedulableEntityManagerProxy.java:1)
> > at
> > org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
> > at
> >
> >
> org.apache.falcon.aspect.AbstractFalconAspect.logAround(AbstractFalconAspect.java:50)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule(SchedulableEntityManagerProxy.java:310)
> > at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at
> >
> >
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> > at
> >
> >
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> > at
> >
> >
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> > at
> >
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
> > at
> >
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
> > at
> >
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
> > at
> >
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
> > at
> >
> >
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
> > at
> >
> >
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
> > at
> >
> >
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> > at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> > at
> >
> >
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> > at
> >
> >
> org.apache.falcon.security.BasicAuthFilter$2.doFilter(BasicAuthFilter.java:183)
> > at
> >
> >
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:392)
> > at
> >
> >
> org.apache.falcon.security.BasicAuthFilter.doFilter(BasicAuthFilter.java:221)
> > at
> >
> >
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> > at
> > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> > at
> >
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> > at
> > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> > at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> > at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> > at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> > at org.mortbay.jetty.Server.handle(Server.java:326)
> > at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> > at
> >
> >
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> > at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> > at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> > at
> >
> >
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
> > at
> >
> >
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> > 2014-08-13 16:49:05,154 ERROR -
> > [6221571@qtp-1696932027-3
> >
> :vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
> > a38b5528-cae0-4b07-a8fc-482661e467a9] ~ Failure reason
> > (FalconWebException:40)
> > java.lang.NullPointerException
> > at
> >
> >
> org.apache.falcon.entity.ProcessHelper.getStorageType(ProcessHelper.java:57)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.isTableStorageType(OozieProcessWorkflowBuilder.java:646)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.shouldSetupHiveConfiguration(OozieProcessWorkflowBuilder.java:641)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createWorkflow(OozieProcessWorkflowBuilder.java:606)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createDefaultCoordinator(OozieProcessWorkflowBuilder.java:291)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.getCoordinators(OozieProcessWorkflowBuilder.java:206)
> > at
> >
> >
> org.apache.falcon.workflow.OozieWorkflowBuilder.map(OozieWorkflowBuilder.java:142)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.newWorkflowSchedule(OozieProcessWorkflowBuilder.java:111)
> > at
> >
> >
> org.apache.falcon.workflow.engine.OozieWorkflowEngine.schedule(OozieWorkflowEngine.java:119)
> > at
> >
> >
> org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:68)
> > at
> >
> >
> org.apache.falcon.resource.AbstractSchedulableEntityManager.schedule(AbstractSchedulableEntityManager.java:57)
> > at
> >
> >
> org.apache.falcon.resource.SchedulableEntityManager.schedule(SchedulableEntityManager.java:85)
> > at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at
> org.apache.falcon.resource.channel.IPCChannel.invoke(IPCChannel.java:48)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$7.doExecute(SchedulableEntityManagerProxy.java:324)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$EntityProxy.execute(SchedulableEntityManagerProxy.java:410)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule_aroundBody10(SchedulableEntityManagerProxy.java:326)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$AjcClosure11.run(SchedulableEntityManagerProxy.java:1)
> > at
> > org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
> > at
> >
> >
> org.apache.falcon.aspect.AbstractFalconAspect.logAround(AbstractFalconAspect.java:50)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule(SchedulableEntityManagerProxy.java:310)
> > at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at
> >
> >
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> > at
> >
> >
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> > at
> >
> >
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> > at
> >
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
> > at
> >
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
> > at
> >
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
> > at
> >
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
> > at
> >
> >
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
> > at
> >
> >
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
> > at
> >
> >
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> > at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> > at
> >
> >
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> > at
> >
> >
> org.apache.falcon.security.BasicAuthFilter$2.doFilter(BasicAuthFilter.java:183)
> > at
> >
> >
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:392)
> > at
> >
> >
> org.apache.falcon.security.BasicAuthFilter.doFilter(BasicAuthFilter.java:221)
> > at
> >
> >
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> > at
> > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> > at
> >
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> > at
> > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> > at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> > at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> > at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> > at org.mortbay.jetty.Server.handle(Server.java:326)
> > at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> > at
> >
> >
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> > at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> > at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> > at
> >
> >
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
> > at
> >
> >
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> > 2014-08-13 16:49:05,154 ERROR -
> > [6221571@qtp-1696932027-3
> >
> :vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
> > a38b5528-cae0-4b07-a8fc-482661e467a9] ~ Action failed: Bad Request
> > Error:null
> > (FalconWebException:66)
> > 2014-08-13 16:49:05,155 INFO -
> > [6221571@qtp-1696932027-3
> >
> :vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
> > a38b5528-cae0-4b07-a8fc-482661e467a9] ~ {Action:schedule,
> > Dimensions:{entityType=process, colo=*,
> > entityName=raw-cc-bp-ratio-lcms-comp-process}, Status: FAILED,
> > Time-taken:161608565 ns} (METRIC:32)
> >
> > My process looks like this:
> >
> > <process name="raw-cc-bp-ratio-lcms-comp-process"
> > xmlns="uri:falcon:process:0.1">
> >
> > <tags>type=raw</tags>
> >
> > <clusters>
> > <cluster name="minicluster">
> > <validity start="2014-08-13T00:00Z" end="2016-01-01T00:00Z"/>
> > </cluster>
> > </clusters>
> >
> > <parallel>1</parallel>
> > <order>FIFO</order>
> > <timeout>hours(2)</timeout>
> > <frequency>days(1)</frequency>
> >
> > <outputs>
> > <output name="output" feed="raw-cc-bp-ratio-lcms-comp-feed"
> > instance="now(0,0)" />
> > </outputs>
> >
> > <properties>
> > ...
> > </properties>
> >
> > <workflow name="sqoop_wf" version="2.0.0" engine="oozie"
> > path="/apps/huron/ingest/standard_ingest/sqoop_wf.xml" />
> >
> > <retry policy="periodic" delay="minutes(10)" attempts="3" />
> >
> > </process>
> >
>
--
Regards,
Venkatesh
“Perfection (in design) is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.”
- Antoine de Saint-Exupéry
Re: Process NPE
Posted by Balu Vellanki <ba...@hortonworks.com>.
A JIRA https://issues.apache.org/jira/browse/FALCON-598 was opened by David
Kjerrumgaard for this issue. I am working on a patch now.
Thanks
Balu Vellanki
On Wed, Aug 13, 2014 at 10:52 AM, James Grant <ja...@queeg.org> wrote:
> I'm having the same issue and was just about to send a mail when you
> posted.
>
> I've had a look at the code and the problem seems to be when the process
> has no inputs. The check in getStorageType used to short circuit if there
> were no inputs but that changed as part of the changes in FALCON-443 to
> only short circuit if the process has no inputs and no outputs.
>
> Below is a patch that will prevent the exception (currently untested).
>
> diff --git
> a/common/src/main/java/org/apache/falcon/entity/ProcessHelper.java
> b/common/src/main/java/org/apache/falcon/entity/ProcessHelper.java
> index 59361e8..33f0aad 100644
> --- a/common/src/main/java/org/apache/falcon/entity/ProcessHelper.java
> +++ b/common/src/main/java/org/apache/falcon/entity/ProcessHelper.java
> @@ -59,11 +59,13 @@ public final class ProcessHelper {
> return storageType;
> }
>
> - for (Input input : process.getInputs().getInputs()) {
> - Feed feed = EntityUtil.getEntity(EntityType.FEED,
> input.getFeed());
> - storageType = FeedHelper.getStorageType(feed, cluster);
> - if (Storage.TYPE.TABLE == storageType) {
> - break;
> + if (process.getInputs() != null) {
> + for (Input input : process.getInputs().getInputs()) {
> + Feed feed = EntityUtil.getEntity(EntityType.FEED,
> input.getFeed());
> + storageType = FeedHelper.getStorageType(feed, cluster);
> + if (Storage.TYPE.TABLE == storageType) {
> + break;
> + }
> }
> }
>
> James
>
>
> On 13 August 2014 17:54, Josh Clum <jo...@gmail.com> wrote:
>
> > Hi, I'm getting the following NPE when i launch with "falcon entity -type
> > process -schedule -name raw-cc-bp-ratio-lcms-comp-process". At the bottom
> > is my process which has no inputs.
> >
> > If I look at
> >
> >
> https://github.com/apache/incubator-falcon/blob/branch-0.5/common/src/main/java/org/apache/falcon/entity/ProcessHelper.java
> > it seems that this should never happen because process.getInputs() ==
> > null should
> > be true and should avoid the NPE. Any thoughts?
> >
> > 2014-08-13 16:49:05,152 ERROR -
> > [6221571@qtp-1696932027-3
> >
> :vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
> > a38b5528-cae0-4b07-a8fc-482661e467a9] ~ Unable to schedule workflow
> > (AbstractSchedulableEntityManager:60)
> > java.lang.NullPointerException
> > at
> >
> >
> org.apache.falcon.entity.ProcessHelper.getStorageType(ProcessHelper.java:57)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.isTableStorageType(OozieProcessWorkflowBuilder.java:646)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.shouldSetupHiveConfiguration(OozieProcessWorkflowBuilder.java:641)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createWorkflow(OozieProcessWorkflowBuilder.java:606)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createDefaultCoordinator(OozieProcessWorkflowBuilder.java:291)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.getCoordinators(OozieProcessWorkflowBuilder.java:206)
> > at
> >
> >
> org.apache.falcon.workflow.OozieWorkflowBuilder.map(OozieWorkflowBuilder.java:142)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.newWorkflowSchedule(OozieProcessWorkflowBuilder.java:111)
> > at
> >
> >
> org.apache.falcon.workflow.engine.OozieWorkflowEngine.schedule(OozieWorkflowEngine.java:119)
> > at
> >
> >
> org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:68)
> > at
> >
> >
> org.apache.falcon.resource.AbstractSchedulableEntityManager.schedule(AbstractSchedulableEntityManager.java:57)
> > at
> >
> >
> org.apache.falcon.resource.SchedulableEntityManager.schedule(SchedulableEntityManager.java:85)
> > at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at
> org.apache.falcon.resource.channel.IPCChannel.invoke(IPCChannel.java:48)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$7.doExecute(SchedulableEntityManagerProxy.java:324)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$EntityProxy.execute(SchedulableEntityManagerProxy.java:410)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule_aroundBody10(SchedulableEntityManagerProxy.java:326)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$AjcClosure11.run(SchedulableEntityManagerProxy.java:1)
> > at
> > org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
> > at
> >
> >
> org.apache.falcon.aspect.AbstractFalconAspect.logAround(AbstractFalconAspect.java:50)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule(SchedulableEntityManagerProxy.java:310)
> > at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at
> >
> >
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> > at
> >
> >
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> > at
> >
> >
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> > at
> >
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
> > at
> >
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
> > at
> >
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
> > at
> >
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
> > at
> >
> >
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
> > at
> >
> >
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
> > at
> >
> >
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> > at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> > at
> >
> >
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> > at
> >
> >
> org.apache.falcon.security.BasicAuthFilter$2.doFilter(BasicAuthFilter.java:183)
> > at
> >
> >
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:392)
> > at
> >
> >
> org.apache.falcon.security.BasicAuthFilter.doFilter(BasicAuthFilter.java:221)
> > at
> >
> >
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> > at
> > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> > at
> >
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> > at
> > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> > at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> > at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> > at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> > at org.mortbay.jetty.Server.handle(Server.java:326)
> > at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> > at
> >
> >
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> > at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> > at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> > at
> >
> >
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
> > at
> >
> >
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> > 2014-08-13 16:49:05,154 ERROR -
> > [6221571@qtp-1696932027-3
> >
> :vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
> > a38b5528-cae0-4b07-a8fc-482661e467a9] ~ Failure reason
> > (FalconWebException:40)
> > java.lang.NullPointerException
> > at
> >
> >
> org.apache.falcon.entity.ProcessHelper.getStorageType(ProcessHelper.java:57)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.isTableStorageType(OozieProcessWorkflowBuilder.java:646)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.shouldSetupHiveConfiguration(OozieProcessWorkflowBuilder.java:641)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createWorkflow(OozieProcessWorkflowBuilder.java:606)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createDefaultCoordinator(OozieProcessWorkflowBuilder.java:291)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.getCoordinators(OozieProcessWorkflowBuilder.java:206)
> > at
> >
> >
> org.apache.falcon.workflow.OozieWorkflowBuilder.map(OozieWorkflowBuilder.java:142)
> > at
> >
> >
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.newWorkflowSchedule(OozieProcessWorkflowBuilder.java:111)
> > at
> >
> >
> org.apache.falcon.workflow.engine.OozieWorkflowEngine.schedule(OozieWorkflowEngine.java:119)
> > at
> >
> >
> org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:68)
> > at
> >
> >
> org.apache.falcon.resource.AbstractSchedulableEntityManager.schedule(AbstractSchedulableEntityManager.java:57)
> > at
> >
> >
> org.apache.falcon.resource.SchedulableEntityManager.schedule(SchedulableEntityManager.java:85)
> > at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at
> org.apache.falcon.resource.channel.IPCChannel.invoke(IPCChannel.java:48)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$7.doExecute(SchedulableEntityManagerProxy.java:324)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$EntityProxy.execute(SchedulableEntityManagerProxy.java:410)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule_aroundBody10(SchedulableEntityManagerProxy.java:326)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$AjcClosure11.run(SchedulableEntityManagerProxy.java:1)
> > at
> > org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
> > at
> >
> >
> org.apache.falcon.aspect.AbstractFalconAspect.logAround(AbstractFalconAspect.java:50)
> > at
> >
> >
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule(SchedulableEntityManagerProxy.java:310)
> > at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at
> >
> >
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> > at
> >
> >
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> > at
> >
> >
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> > at
> >
> >
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> > at
> >
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
> > at
> >
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
> > at
> >
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
> > at
> >
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
> > at
> >
> >
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
> > at
> >
> >
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
> > at
> >
> >
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> > at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> > at
> >
> >
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> > at
> >
> >
> org.apache.falcon.security.BasicAuthFilter$2.doFilter(BasicAuthFilter.java:183)
> > at
> >
> >
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:392)
> > at
> >
> >
> org.apache.falcon.security.BasicAuthFilter.doFilter(BasicAuthFilter.java:221)
> > at
> >
> >
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> > at
> > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> > at
> >
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> > at
> > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> > at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> > at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> > at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> > at org.mortbay.jetty.Server.handle(Server.java:326)
> > at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> > at
> >
> >
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> > at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> > at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> > at
> >
> >
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
> > at
> >
> >
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> > 2014-08-13 16:49:05,154 ERROR -
> > [6221571@qtp-1696932027-3
> >
> :vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
> > a38b5528-cae0-4b07-a8fc-482661e467a9] ~ Action failed: Bad Request
> > Error:null
> > (FalconWebException:66)
> > 2014-08-13 16:49:05,155 INFO -
> > [6221571@qtp-1696932027-3
> >
> :vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
> > a38b5528-cae0-4b07-a8fc-482661e467a9] ~ {Action:schedule,
> > Dimensions:{entityType=process, colo=*,
> > entityName=raw-cc-bp-ratio-lcms-comp-process}, Status: FAILED,
> > Time-taken:161608565 ns} (METRIC:32)
> >
> > My process looks like this:
> >
> > <process name="raw-cc-bp-ratio-lcms-comp-process"
> > xmlns="uri:falcon:process:0.1">
> >
> > <tags>type=raw</tags>
> >
> > <clusters>
> > <cluster name="minicluster">
> > <validity start="2014-08-13T00:00Z" end="2016-01-01T00:00Z"/>
> > </cluster>
> > </clusters>
> >
> > <parallel>1</parallel>
> > <order>FIFO</order>
> > <timeout>hours(2)</timeout>
> > <frequency>days(1)</frequency>
> >
> > <outputs>
> > <output name="output" feed="raw-cc-bp-ratio-lcms-comp-feed"
> > instance="now(0,0)" />
> > </outputs>
> >
> > <properties>
> > ...
> > </properties>
> >
> > <workflow name="sqoop_wf" version="2.0.0" engine="oozie"
> > path="/apps/huron/ingest/standard_ingest/sqoop_wf.xml" />
> >
> > <retry policy="periodic" delay="minutes(10)" attempts="3" />
> >
> > </process>
> >
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Process NPE
Posted by James Grant <ja...@queeg.org>.
I'm having the same issue and was just about to send a mail when you posted.
I've had a look at the code and the problem seems to be when the process
has no inputs. The check in getStorageType used to short circuit if there
were no inputs but that changed as part of the changes in FALCON-443 to
only short circuit if the process has no inputs and no outputs.
Below is a patch that will prevent the exception (currently untested).
diff --git
a/common/src/main/java/org/apache/falcon/entity/ProcessHelper.java
b/common/src/main/java/org/apache/falcon/entity/ProcessHelper.java
index 59361e8..33f0aad 100644
--- a/common/src/main/java/org/apache/falcon/entity/ProcessHelper.java
+++ b/common/src/main/java/org/apache/falcon/entity/ProcessHelper.java
@@ -59,11 +59,13 @@ public final class ProcessHelper {
return storageType;
}
- for (Input input : process.getInputs().getInputs()) {
- Feed feed = EntityUtil.getEntity(EntityType.FEED,
input.getFeed());
- storageType = FeedHelper.getStorageType(feed, cluster);
- if (Storage.TYPE.TABLE == storageType) {
- break;
+ if (process.getInputs() != null) {
+ for (Input input : process.getInputs().getInputs()) {
+ Feed feed = EntityUtil.getEntity(EntityType.FEED,
input.getFeed());
+ storageType = FeedHelper.getStorageType(feed, cluster);
+ if (Storage.TYPE.TABLE == storageType) {
+ break;
+ }
}
}
James
On 13 August 2014 17:54, Josh Clum <jo...@gmail.com> wrote:
> Hi, I'm getting the following NPE when i launch with "falcon entity -type
> process -schedule -name raw-cc-bp-ratio-lcms-comp-process". At the bottom
> is my process which has no inputs.
>
> If I look at
>
> https://github.com/apache/incubator-falcon/blob/branch-0.5/common/src/main/java/org/apache/falcon/entity/ProcessHelper.java
> it seems that this should never happen because process.getInputs() ==
> null should
> be true and should avoid the NPE. Any thoughts?
>
> 2014-08-13 16:49:05,152 ERROR -
> [6221571@qtp-1696932027-3
> :vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
> a38b5528-cae0-4b07-a8fc-482661e467a9] ~ Unable to schedule workflow
> (AbstractSchedulableEntityManager:60)
> java.lang.NullPointerException
> at
>
> org.apache.falcon.entity.ProcessHelper.getStorageType(ProcessHelper.java:57)
> at
>
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.isTableStorageType(OozieProcessWorkflowBuilder.java:646)
> at
>
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.shouldSetupHiveConfiguration(OozieProcessWorkflowBuilder.java:641)
> at
>
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createWorkflow(OozieProcessWorkflowBuilder.java:606)
> at
>
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createDefaultCoordinator(OozieProcessWorkflowBuilder.java:291)
> at
>
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.getCoordinators(OozieProcessWorkflowBuilder.java:206)
> at
>
> org.apache.falcon.workflow.OozieWorkflowBuilder.map(OozieWorkflowBuilder.java:142)
> at
>
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.newWorkflowSchedule(OozieProcessWorkflowBuilder.java:111)
> at
>
> org.apache.falcon.workflow.engine.OozieWorkflowEngine.schedule(OozieWorkflowEngine.java:119)
> at
>
> org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:68)
> at
>
> org.apache.falcon.resource.AbstractSchedulableEntityManager.schedule(AbstractSchedulableEntityManager.java:57)
> at
>
> org.apache.falcon.resource.SchedulableEntityManager.schedule(SchedulableEntityManager.java:85)
> at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.falcon.resource.channel.IPCChannel.invoke(IPCChannel.java:48)
> at
>
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$7.doExecute(SchedulableEntityManagerProxy.java:324)
> at
>
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$EntityProxy.execute(SchedulableEntityManagerProxy.java:410)
> at
>
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule_aroundBody10(SchedulableEntityManagerProxy.java:326)
> at
>
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$AjcClosure11.run(SchedulableEntityManagerProxy.java:1)
> at
> org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
> at
>
> org.apache.falcon.aspect.AbstractFalconAspect.logAround(AbstractFalconAspect.java:50)
> at
>
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule(SchedulableEntityManagerProxy.java:310)
> at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
>
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
>
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
>
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
>
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> at
>
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
>
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
>
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
>
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
>
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
> at
>
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
> at
>
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
> at
>
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
> at
>
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
> at
>
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
> at
>
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at
>
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> at
>
> org.apache.falcon.security.BasicAuthFilter$2.doFilter(BasicAuthFilter.java:183)
> at
>
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:392)
> at
>
> org.apache.falcon.security.BasicAuthFilter.doFilter(BasicAuthFilter.java:221)
> at
>
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
>
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
>
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
> at
>
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> 2014-08-13 16:49:05,154 ERROR -
> [6221571@qtp-1696932027-3
> :vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
> a38b5528-cae0-4b07-a8fc-482661e467a9] ~ Failure reason
> (FalconWebException:40)
> java.lang.NullPointerException
> at
>
> org.apache.falcon.entity.ProcessHelper.getStorageType(ProcessHelper.java:57)
> at
>
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.isTableStorageType(OozieProcessWorkflowBuilder.java:646)
> at
>
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.shouldSetupHiveConfiguration(OozieProcessWorkflowBuilder.java:641)
> at
>
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createWorkflow(OozieProcessWorkflowBuilder.java:606)
> at
>
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createDefaultCoordinator(OozieProcessWorkflowBuilder.java:291)
> at
>
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.getCoordinators(OozieProcessWorkflowBuilder.java:206)
> at
>
> org.apache.falcon.workflow.OozieWorkflowBuilder.map(OozieWorkflowBuilder.java:142)
> at
>
> org.apache.falcon.workflow.OozieProcessWorkflowBuilder.newWorkflowSchedule(OozieProcessWorkflowBuilder.java:111)
> at
>
> org.apache.falcon.workflow.engine.OozieWorkflowEngine.schedule(OozieWorkflowEngine.java:119)
> at
>
> org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:68)
> at
>
> org.apache.falcon.resource.AbstractSchedulableEntityManager.schedule(AbstractSchedulableEntityManager.java:57)
> at
>
> org.apache.falcon.resource.SchedulableEntityManager.schedule(SchedulableEntityManager.java:85)
> at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.falcon.resource.channel.IPCChannel.invoke(IPCChannel.java:48)
> at
>
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$7.doExecute(SchedulableEntityManagerProxy.java:324)
> at
>
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$EntityProxy.execute(SchedulableEntityManagerProxy.java:410)
> at
>
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule_aroundBody10(SchedulableEntityManagerProxy.java:326)
> at
>
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$AjcClosure11.run(SchedulableEntityManagerProxy.java:1)
> at
> org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
> at
>
> org.apache.falcon.aspect.AbstractFalconAspect.logAround(AbstractFalconAspect.java:50)
> at
>
> org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule(SchedulableEntityManagerProxy.java:310)
> at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
>
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
>
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
>
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
>
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> at
>
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
>
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
>
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
>
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
>
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
> at
>
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
> at
>
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
> at
>
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
> at
>
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
> at
>
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
> at
>
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at
>
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> at
>
> org.apache.falcon.security.BasicAuthFilter$2.doFilter(BasicAuthFilter.java:183)
> at
>
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:392)
> at
>
> org.apache.falcon.security.BasicAuthFilter.doFilter(BasicAuthFilter.java:221)
> at
>
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
>
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
>
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
> at
>
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> 2014-08-13 16:49:05,154 ERROR -
> [6221571@qtp-1696932027-3
> :vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
> a38b5528-cae0-4b07-a8fc-482661e467a9] ~ Action failed: Bad Request
> Error:null
> (FalconWebException:66)
> 2014-08-13 16:49:05,155 INFO -
> [6221571@qtp-1696932027-3
> :vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
> a38b5528-cae0-4b07-a8fc-482661e467a9] ~ {Action:schedule,
> Dimensions:{entityType=process, colo=*,
> entityName=raw-cc-bp-ratio-lcms-comp-process}, Status: FAILED,
> Time-taken:161608565 ns} (METRIC:32)
>
> My process looks like this:
>
> <process name="raw-cc-bp-ratio-lcms-comp-process"
> xmlns="uri:falcon:process:0.1">
>
> <tags>type=raw</tags>
>
> <clusters>
> <cluster name="minicluster">
> <validity start="2014-08-13T00:00Z" end="2016-01-01T00:00Z"/>
> </cluster>
> </clusters>
>
> <parallel>1</parallel>
> <order>FIFO</order>
> <timeout>hours(2)</timeout>
> <frequency>days(1)</frequency>
>
> <outputs>
> <output name="output" feed="raw-cc-bp-ratio-lcms-comp-feed"
> instance="now(0,0)" />
> </outputs>
>
> <properties>
> ...
> </properties>
>
> <workflow name="sqoop_wf" version="2.0.0" engine="oozie"
> path="/apps/huron/ingest/standard_ingest/sqoop_wf.xml" />
>
> <retry policy="periodic" delay="minutes(10)" attempts="3" />
>
> </process>
>