You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@falcon.apache.org by Gaurav Agarwal <gz...@gmail.com> on 2014/11/27 03:20:07 UTC
Need help for the problem that i faced
Hello I am running apache falcon and oozie with shell script to run the
workflow.
getting this error
*ell-node] Launcher ERROR, reason: Main class
[org.apache.oozie.action.hadoop.ShellMain], exit code [1]*
*2014-11-26 17:24:06,645 INFO ActionEndXCommand:543 -
SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
JOB[0000020-141126162609456-oozie-oozi-W]
ACTION[0000020-141126162609456-oozie-oozi-W@shell-node] ERROR is considered
as FAILED for SLA*
*2014-11-26 17:24:06,702 INFO ActionStartXCommand:543 -
SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
JOB[0000020-141126162609456-oozie-oozi-W]
ACTION[0000020-141126162609456-oozie-oozi-W@fail] Start action
[0000020-141126162609456-oozie-*
*Workflow*
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf">
<start to="shell-node"/>
<action name="shell-node">
<shell xmlns="uri:oozie:shell-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<exec>ingest.sh</exec>
<argument>${feedInstancePaths}</argument>
<file>${wf:appPath()}/ingest.sh#ingest.sh</file>
<!-- <file>/tmp/ingest.sh#ingest.sh</file> -->
<!-- <capture-output/> -->
</shell>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Shell action failed, error
message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>
This is the shell script
#!/bin/bash
java -cp "/media/sf_ToSandbox/BigData-0.0.1-SNAPSHOT.jar:/usr/hdp/2.2.0.0-913/sqoop/lib/*:/usr/hdp/2.2.0.0-913/hadoop/lib/*"
com.bigdata.falcon.StaticDataProvider
These are the feed and process and cluster information
<feed description="Raw customer email feed" name="FINALrawEmailFeed"
xmlns="uri:falcon:feed:0.1">
<tags>externalSystem=USWestEmailServers,classification=secure</tags>
<groups>churnAnalysisDataPipeline</groups>
<frequency>minutes(3)</frequency>
<late-arrival cut-off="hours(4)"/>
<clusters>
<cluster name="SprimaryCluster" type="source">
<validity start="2014-11-21T00:00Z" end="2016-03-31T00:00Z"/>
<retention limit="days(90)" action="delete"/>
</cluster>
</clusters>
<locations>
<location type="data"
path="/user/gaurav123/falcon/demo/primary/input/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
<location type="stats" path="/none"/>
<location type="meta" path="/none"/>
</locations>
<ACL owner="ambari-qa" group="users" permission="0777"/>
<schema location="/none" provider="none"/>
</feed>
*Process*
*-----------*
<?xml version="1.0" encoding="UTF-8"?>
<!--
Hourly process, ingests raw emails from all US west email servers
-->
<process name="IngestProcess" xmlns="uri:falcon:process:0.1">
<tags>pipeline=churnAnalysisDataPipeline,owner=ETLGroup,externalSystem=USWestEmailServers</tags>
<clusters>
<cluster name="SprimaryCluster">
<validity start="2014-11-24T09:35Z" end="2014-11-25T10:00Z"/>
</cluster>
</clusters>
<parallel>1</parallel>
<order>FIFO</order>
<frequency>minutes(3)</frequency>
<outputs>
<output name="output" feed="FINALrawEmailFeed" instance="now(0,0)"
/>
</outputs>
<workflow name="IngestWorkflow" version="2.0.0"
engine="oozie" path="/user/gaurav123/falcon/demo/apps/ingest/fs" />
<retry policy="periodic" delay="minutes(15)" attempts="3" />
</process>
Can you please let me know what is the issue , I have tried since long two
days? but unable to run the workflow with falcon process
Also got this exception
2014-11-26 17:47:42,548 ERROR CompletedActionXCommand:540 -
SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[] APP[-]
JOB[0000099-141126162609456-oozie-oozi-W]
ACTION[0000099-141126162609456-oozie-oozi-W@user-action] XException,
org.apache.oozie.command.CommandException: E0800: Action it is not
running its in [PREP] state, action
[0000099-141126162609456-oozie-oozi-W@user-action]
Re: Need help for the problem that i faced
Posted by Gaurav Agarwal <gz...@gmail.com>.
thanks issue is resolved.
On 11/27/14, Shwetha GS <sh...@inmobi.com> wrote:
> Try the java action in oozie:
> http://oozie.apache.org/docs/4.0.1/WorkflowFunctionalSpec.html#a3.2.7_Java_Action
>
> On Thu, Nov 27, 2014 at 11:49 AM, Gaurav Agarwal <gz...@gmail.com>
> wrote:
>
>> Shwetha
>>
>> Apologise to trouble everyone but i am new to this falcon. Do i need to
>> check the logs in JobBrowser?
>>
>> there is stderror
>>
>> Log Length: 188
>> Error: Could not find or load main class com.jpmc.cib.is.bigdata.
>> falcon.SemsStaticDataProvider
>> Failing Oozie Launcher, Main class
>> [org.apache.oozie.action.hadoop.ShellMain],
>> exit code [1]
>>
>>
>>
>>
>>
>> On Thu, Nov 27, 2014 at 11:43 AM, Shwetha GS <sh...@inmobi.com>
>> wrote:
>>
>> > oozie launches a hadoop job which executes the shell script. Check the
>> > launcher job logs
>> >
>> > On Thu, Nov 27, 2014 at 11:39 AM, Gaurav Agarwal <gz...@gmail.com>
>> > wrote:
>> >
>> > > Srikanth
>> > >
>> > > What do we mean by that in oozie context?
>> > >
>> > > Thanks
>> > >
>> > > On Thu, Nov 27, 2014 at 11:36 AM, Srikanth Sundarrajan <
>> > > sriksun@hotmail.com>
>> > > wrote:
>> > >
>> > > > Looks like the script ingest.sh is returning non-zero exit code.
>> > > >
>> > > > Regards
>> > > > Srikanth Sundarrajan
>> > > >
>> > > > > Date: Thu, 27 Nov 2014 11:31:30 +0530
>> > > > > Subject: Re: Need help for the problem that i faced
>> > > > > From: gzagarwal@gmail.com
>> > > > > To: dev@falcon.incubator.apache.org
>> > > > >
>> > > > > These are the log
>> > > > >
>> > > > > 2014-11-27 05:37:02,726 INFO ActionStartXCommand:543 - SERVER[
>> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
>> > > APP[shell-wf]
>> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
>> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:] Start action
>> > > > > [0000129-141126162609456-oozie-oozi-W@:start:] with user-retry
>> > state :
>> > > > > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
>> > > > > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
>> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
>> > > APP[shell-wf]
>> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
>> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
>> > > > > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action
>> > status=DONE
>> > > > > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
>> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
>> > > APP[shell-wf]
>> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
>> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
>> > > > > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action
>> > > > > updated
>> > in
>> > > > DB!
>> > > > > 2014-11-27 05:37:03,151 INFO ActionStartXCommand:543 - SERVER[
>> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
>> > > APP[shell-wf]
>> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
>> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Start
>> action
>> > > > > [0000129-141126162609456-oozie-oozi-W@shell-node] with user-retry
>> > > state
>> > > > :
>> > > > > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
>> > > > > 2014-11-27 05:37:05,377 INFO ShellActionExecutor:543 - SERVER[
>> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
>> > > APP[shell-wf]
>> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
>> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] checking
>> > > action,
>> > > > > hadoop job ID [job_1417019189193_0117] status [RUNNING]
>> > > > > 2014-11-27 05:37:05,378 INFO ActionStartXCommand:543 - SERVER[
>> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
>> > > APP[shell-wf]
>> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
>> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
>> > > > > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action
>> > > > > status=RUNNING
>> > > > > 2014-11-27 05:37:05,379 INFO ActionStartXCommand:543 - SERVER[
>> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
>> > > APP[shell-wf]
>> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
>> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
>> > > > > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action
>> > updated
>> > > in
>> > > > > DB!
>> > > > > 2014-11-27 05:37:49,260 INFO CallbackServlet:543 - SERVER[
>> > > > > sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-]
>> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
>> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] callback
>> for
>> > > > action
>> > > > > [0000129-141126162609456-oozie-oozi-W@shell-node]
>> > > > > 2014-11-27 05:37:50,753 INFO ShellActionExecutor:543 - SERVER[
>> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
>> > > APP[shell-wf]
>> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
>> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] action
>> > > > completed,
>> > > > > external ID [job_1417019189193_0117]
>> > > > > 2014-11-27 05:37:50,761 WARN ShellActionExecutor:546 - SERVER[
>> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
>> > > APP[shell-wf]
>> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
>> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Launcher
>> > > ERROR,
>> > > > > reason: Main class [org.apache.oozie.action.hadoop.ShellMain],
>> > > > > exit
>> > > code
>> > > > > [1]
>> > > > > 2014-11-27 05:37:50,865 INFO ActionEndXCommand:543 - SERVER[
>> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
>> > > APP[shell-wf]
>> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
>> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] ERROR is
>> > > > considered
>> > > > > as FAILED for SLA
>> > > > > 2014-11-27 05:37:50,941 INFO ActionStartXCommand:543 - SERVER[
>> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
>> > > APP[shell-wf]
>> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
>> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@fail] Start action
>> > > > > [0000129-141126162609456-oozie-oozi-W@fail] with user-retry state
>> :
>> > > > > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
>> > > > > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
>> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
>> > > APP[shell-wf]
>> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
>> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@fail]
>> > > > > [***0000129-141126162609456-oozie-oozi-W@fail***]Action
>> status=DONE
>> > > > > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
>> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
>> > > APP[shell-wf]
>> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
>> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@fail]
>> > > > > [***0000129-141126162609456-oozie-oozi-W@fail***]Action updated
>> > > > > in
>> > DB!
>> > > > >
>> > > > > On Thu, Nov 27, 2014 at 10:25 AM, Shwetha GS <
>> shwetha.gs@inmobi.com>
>> > > > wrote:
>> > > > >
>> > > > > > <exec>ingest.sh</exec>
>> > > > > > Try using absolute path.
>> > > > > >
>> > > > > > None of the logs imply any error. Can you attach all the oozie
>> logs
>> > > > for the
>> > > > > > workflow?
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > On Thu, Nov 27, 2014 at 7:50 AM, Gaurav Agarwal <
>> > gzagarwal@gmail.com
>> > > >
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hello I am running apache falcon and oozie with shell script
>> > > > > > > to
>> > run
>> > > > the
>> > > > > > > workflow.
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > getting this error
>> > > > > > >
>> > > > > > > *ell-node] Launcher ERROR, reason: Main class
>> > > > > > > [org.apache.oozie.action.hadoop.ShellMain], exit code [1]*
>> > > > > > > *2014-11-26 17:24:06,645 INFO ActionEndXCommand:543 -
>> > > > > > > SERVER[sandbox.hortonworks.com
>> > > > > > > <http://sandbox.hortonworks.com
>> >]
>> > > > > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
>> > > > > > > JOB[0000020-141126162609456-oozie-oozi-W]
>> > > > > > > ACTION[0000020-141126162609456-oozie-oozi-W@shell-node] ERROR
>> is
>> > > > > > > considered
>> > > > > > > as FAILED for SLA*
>> > > > > > > *2014-11-26 17:24:06,702 INFO ActionStartXCommand:543 -
>> > > > > > > SERVER[sandbox.hortonworks.com
>> > > > > > > <http://sandbox.hortonworks.com
>> >]
>> > > > > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
>> > > > > > > JOB[0000020-141126162609456-oozie-oozi-W]
>> > > > > > > ACTION[0000020-141126162609456-oozie-oozi-W@fail] Start
>> > > > > > > action
>> > > > > > > [0000020-141126162609456-oozie-*
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > *Workflow*
>> > > > > > >
>> > > > > > > <!--
>> > > > > > > Licensed to the Apache Software Foundation (ASF) under one
>> > > > > > > or more contributor license agreements. See the NOTICE
>> > > > > > > file
>> > > > > > > distributed with this work for additional information
>> > > > > > > regarding copyright ownership. The ASF licenses this file
>> > > > > > > to you under the Apache License, Version 2.0 (the
>> > > > > > > "License"); you may not use this file except in compliance
>> > > > > > > with the License. You may obtain a copy of the License at
>> > > > > > >
>> > > > > > > http://www.apache.org/licenses/LICENSE-2.0
>> > > > > > >
>> > > > > > > Unless required by applicable law or agreed to in writing,
>> > > software
>> > > > > > > distributed under the License is distributed on an "AS IS"
>> > BASIS,
>> > > > > > > WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
>> > > > > > > express
>> or
>> > > > > > implied.
>> > > > > > > See the License for the specific language governing
>> permissions
>> > > and
>> > > > > > > limitations under the License.
>> > > > > > > -->
>> > > > > > > <workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf">
>> > > > > > > <start to="shell-node"/>
>> > > > > > > <action name="shell-node">
>> > > > > > > <shell xmlns="uri:oozie:shell-action:0.2">
>> > > > > > > <job-tracker>${jobTracker}</job-tracker>
>> > > > > > > <name-node>${nameNode}</name-node>
>> > > > > > > <configuration>
>> > > > > > > <property>
>> > > > > > > <name>mapred.job.queue.name</name>
>> > > > > > > <value>${queueName}</value>
>> > > > > > > </property>
>> > > > > > > </configuration>
>> > > > > > > <exec>ingest.sh</exec>
>> > > > > > > <argument>${feedInstancePaths}</argument>
>> > > > > > > <file>${wf:appPath()}/ingest.sh#ingest.sh</file>
>> > > > > > > <!-- <file>/tmp/ingest.sh#ingest.sh</file> -->
>> > > > > > > <!-- <capture-output/> -->
>> > > > > > > </shell>
>> > > > > > > <ok to="end"/>
>> > > > > > > <error to="fail"/>
>> > > > > > > </action>
>> > > > > > > <kill name="fail">
>> > > > > > > <message>Shell action failed, error
>> > > > > > > message[${wf:errorMessage(wf:lastErrorNode())}]</message>
>> > > > > > > </kill>
>> > > > > > > <end name="end"/>
>> > > > > > > </workflow-app>
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > This is the shell script
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > #!/bin/bash
>> > > > > > >
>> > > > > > >
>> > > > > > > java -cp
>> > > > > > >
>> > > > > >
>> > > >
>> > >
>> >
>> "/media/sf_ToSandbox/BigData-0.0.1-SNAPSHOT.jar:/usr/hdp/2.2.0.0-913/sqoop/lib/*:/usr/hdp/2.2.0.0-913/hadoop/lib/*"
>> > > > > > > com.bigdata.falcon.StaticDataProvider
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > These are the feed and process and cluster information
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > <feed description="Raw customer email feed"
>> > > name="FINALrawEmailFeed"
>> > > > > > > xmlns="uri:falcon:feed:0.1">
>> > > > > > >
>> > > > > > >
>> > > >
>> > > > <tags>externalSystem=USWestEmailServers,classification=secure</tags>
>> > > > > > > <groups>churnAnalysisDataPipeline</groups>
>> > > > > > > <frequency>minutes(3)</frequency>
>> > > > > > > <late-arrival cut-off="hours(4)"/>
>> > > > > > > <clusters>
>> > > > > > > <cluster name="SprimaryCluster" type="source">
>> > > > > > > <validity start="2014-11-21T00:00Z"
>> > > > end="2016-03-31T00:00Z"/>
>> > > > > > > <retention limit="days(90)" action="delete"/>
>> > > > > > > </cluster>
>> > > > > > > </clusters>
>> > > > > > >
>> > > > > > > <locations>
>> > > > > > > <location type="data"
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > >
>> > >
>> >
>> path="/user/gaurav123/falcon/demo/primary/input/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
>> > > > > > > <location type="stats" path="/none"/>
>> > > > > > > <location type="meta" path="/none"/>
>> > > > > > > </locations>
>> > > > > > >
>> > > > > > > <ACL owner="ambari-qa" group="users" permission="0777"/>
>> > > > > > > <schema location="/none" provider="none"/>
>> > > > > > > </feed>
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > *Process*
>> > > > > > >
>> > > > > > > *-----------*
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > <?xml version="1.0" encoding="UTF-8"?>
>> > > > > > > <!--
>> > > > > > > Hourly process, ingests raw emails from all US west email
>> > servers
>> > > > > > > -->
>> > > > > > > <process name="IngestProcess" xmlns="uri:falcon:process:0.1">
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > >
>> > >
>> >
>> <tags>pipeline=churnAnalysisDataPipeline,owner=ETLGroup,externalSystem=USWestEmailServers</tags>
>> > > > > > >
>> > > > > > >
>> > > > > > > <clusters>
>> > > > > > > <cluster name="SprimaryCluster">
>> > > > > > > <validity start="2014-11-24T09:35Z"
>> > > > end="2014-11-25T10:00Z"/>
>> > > > > > > </cluster>
>> > > > > > > </clusters>
>> > > > > > >
>> > > > > > > <parallel>1</parallel>
>> > > > > > > <order>FIFO</order>
>> > > > > > > <frequency>minutes(3)</frequency>
>> > > > > > >
>> > > > > > > <outputs>
>> > > > > > > <output name="output" feed="FINALrawEmailFeed"
>> > > > > > instance="now(0,0)"
>> > > > > > > />
>> > > > > > > </outputs>
>> > > > > > >
>> > > > > > > <workflow name="IngestWorkflow" version="2.0.0"
>> > > > > > > engine="oozie"
>> > > path="/user/gaurav123/falcon/demo/apps/ingest/fs"
>> > > > />
>> > > > > > >
>> > > > > > > <retry policy="periodic" delay="minutes(15)" attempts="3"
>> />
>> > > > > > >
>> > > > > > > </process>
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > Can you please let me know what is the issue , I have tried
>> since
>> > > > long
>> > > > > > two
>> > > > > > > days? but unable to run the workflow with falcon process
>> > > > > > >
>> > > > > > > Also got this exception
>> > > > > > >
>> > > > > > > 2014-11-26 17:47:42,548 ERROR CompletedActionXCommand:540 -
>> > > > > > > SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[]
>> APP[-]
>> > > > > > > JOB[0000099-141126162609456-oozie-oozi-W]
>> > > > > > > ACTION[0000099-141126162609456-oozie-oozi-W@user-action]
>> > > XException,
>> > > > > > > org.apache.oozie.command.CommandException: E0800: Action it
>> > > > > > > is
>> > not
>> > > > > > > running its in [PREP] state, action
>> > > > > > > [0000099-141126162609456-oozie-oozi-W@user-action]
>> > > > > > >
>> > > > > >
>> > > > > > --
>> > > > > > _____________________________________________________________
>> > > > > > The information contained in this communication is intended
>> solely
>> > > for
>> > > > the
>> > > > > > use of the individual or entity to whom it is addressed and
>> others
>> > > > > > authorized to receive it. It may contain confidential or
>> > > > > > legally
>> > > > privileged
>> > > > > > information. If you are not the intended recipient you are
>> > > > > > hereby
>> > > > notified
>> > > > > > that any disclosure, copying, distribution or taking any action
>> in
>> > > > reliance
>> > > > > > on the contents of this information is strictly prohibited and
>> may
>> > be
>> > > > > > unlawful. If you have received this communication in error,
>> please
>> > > > notify
>> > > > > > us immediately by responding to this email and then delete it
>> from
>> > > your
>> > > > > > system. The firm is neither liable for the proper and complete
>> > > > transmission
>> > > > > > of the information contained in this communication nor for any
>> > delay
>> > > > in its
>> > > > > > receipt.
>> > > > > >
>> > > >
>> > > >
>> > >
>> >
>> > --
>> > _____________________________________________________________
>> > The information contained in this communication is intended solely for
>> the
>> > use of the individual or entity to whom it is addressed and others
>> > authorized to receive it. It may contain confidential or legally
>> privileged
>> > information. If you are not the intended recipient you are hereby
>> notified
>> > that any disclosure, copying, distribution or taking any action in
>> reliance
>> > on the contents of this information is strictly prohibited and may be
>> > unlawful. If you have received this communication in error, please
>> > notify
>> > us immediately by responding to this email and then delete it from your
>> > system. The firm is neither liable for the proper and complete
>> transmission
>> > of the information contained in this communication nor for any delay in
>> its
>> > receipt.
>> >
>>
>
> --
> _____________________________________________________________
> The information contained in this communication is intended solely for the
> use of the individual or entity to whom it is addressed and others
> authorized to receive it. It may contain confidential or legally privileged
>
> information. If you are not the intended recipient you are hereby notified
> that any disclosure, copying, distribution or taking any action in reliance
>
> on the contents of this information is strictly prohibited and may be
> unlawful. If you have received this communication in error, please notify
> us immediately by responding to this email and then delete it from your
> system. The firm is neither liable for the proper and complete transmission
>
> of the information contained in this communication nor for any delay in its
>
> receipt.
>
Re: Need help for the problem that i faced
Posted by Shwetha GS <sh...@inmobi.com>.
Try the java action in oozie:
http://oozie.apache.org/docs/4.0.1/WorkflowFunctionalSpec.html#a3.2.7_Java_Action
On Thu, Nov 27, 2014 at 11:49 AM, Gaurav Agarwal <gz...@gmail.com>
wrote:
> Shwetha
>
> Apologise to trouble everyone but i am new to this falcon. Do i need to
> check the logs in JobBrowser?
>
> there is stderror
>
> Log Length: 188
> Error: Could not find or load main class com.jpmc.cib.is.bigdata.
> falcon.SemsStaticDataProvider
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.ShellMain],
> exit code [1]
>
>
>
>
>
> On Thu, Nov 27, 2014 at 11:43 AM, Shwetha GS <sh...@inmobi.com>
> wrote:
>
> > oozie launches a hadoop job which executes the shell script. Check the
> > launcher job logs
> >
> > On Thu, Nov 27, 2014 at 11:39 AM, Gaurav Agarwal <gz...@gmail.com>
> > wrote:
> >
> > > Srikanth
> > >
> > > What do we mean by that in oozie context?
> > >
> > > Thanks
> > >
> > > On Thu, Nov 27, 2014 at 11:36 AM, Srikanth Sundarrajan <
> > > sriksun@hotmail.com>
> > > wrote:
> > >
> > > > Looks like the script ingest.sh is returning non-zero exit code.
> > > >
> > > > Regards
> > > > Srikanth Sundarrajan
> > > >
> > > > > Date: Thu, 27 Nov 2014 11:31:30 +0530
> > > > > Subject: Re: Need help for the problem that i faced
> > > > > From: gzagarwal@gmail.com
> > > > > To: dev@falcon.incubator.apache.org
> > > > >
> > > > > These are the log
> > > > >
> > > > > 2014-11-27 05:37:02,726 INFO ActionStartXCommand:543 - SERVER[
> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > > APP[shell-wf]
> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:] Start action
> > > > > [0000129-141126162609456-oozie-oozi-W@:start:] with user-retry
> > state :
> > > > > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> > > > > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > > APP[shell-wf]
> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
> > > > > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action
> > status=DONE
> > > > > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > > APP[shell-wf]
> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
> > > > > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action updated
> > in
> > > > DB!
> > > > > 2014-11-27 05:37:03,151 INFO ActionStartXCommand:543 - SERVER[
> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > > APP[shell-wf]
> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Start
> action
> > > > > [0000129-141126162609456-oozie-oozi-W@shell-node] with user-retry
> > > state
> > > > :
> > > > > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> > > > > 2014-11-27 05:37:05,377 INFO ShellActionExecutor:543 - SERVER[
> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > > APP[shell-wf]
> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] checking
> > > action,
> > > > > hadoop job ID [job_1417019189193_0117] status [RUNNING]
> > > > > 2014-11-27 05:37:05,378 INFO ActionStartXCommand:543 - SERVER[
> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > > APP[shell-wf]
> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
> > > > > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action
> > > > > status=RUNNING
> > > > > 2014-11-27 05:37:05,379 INFO ActionStartXCommand:543 - SERVER[
> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > > APP[shell-wf]
> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
> > > > > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action
> > updated
> > > in
> > > > > DB!
> > > > > 2014-11-27 05:37:49,260 INFO CallbackServlet:543 - SERVER[
> > > > > sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-]
> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] callback
> for
> > > > action
> > > > > [0000129-141126162609456-oozie-oozi-W@shell-node]
> > > > > 2014-11-27 05:37:50,753 INFO ShellActionExecutor:543 - SERVER[
> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > > APP[shell-wf]
> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] action
> > > > completed,
> > > > > external ID [job_1417019189193_0117]
> > > > > 2014-11-27 05:37:50,761 WARN ShellActionExecutor:546 - SERVER[
> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > > APP[shell-wf]
> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Launcher
> > > ERROR,
> > > > > reason: Main class [org.apache.oozie.action.hadoop.ShellMain], exit
> > > code
> > > > > [1]
> > > > > 2014-11-27 05:37:50,865 INFO ActionEndXCommand:543 - SERVER[
> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > > APP[shell-wf]
> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] ERROR is
> > > > considered
> > > > > as FAILED for SLA
> > > > > 2014-11-27 05:37:50,941 INFO ActionStartXCommand:543 - SERVER[
> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > > APP[shell-wf]
> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@fail] Start action
> > > > > [0000129-141126162609456-oozie-oozi-W@fail] with user-retry state
> :
> > > > > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> > > > > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > > APP[shell-wf]
> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@fail]
> > > > > [***0000129-141126162609456-oozie-oozi-W@fail***]Action
> status=DONE
> > > > > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > > APP[shell-wf]
> > > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@fail]
> > > > > [***0000129-141126162609456-oozie-oozi-W@fail***]Action updated in
> > DB!
> > > > >
> > > > > On Thu, Nov 27, 2014 at 10:25 AM, Shwetha GS <
> shwetha.gs@inmobi.com>
> > > > wrote:
> > > > >
> > > > > > <exec>ingest.sh</exec>
> > > > > > Try using absolute path.
> > > > > >
> > > > > > None of the logs imply any error. Can you attach all the oozie
> logs
> > > > for the
> > > > > > workflow?
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Nov 27, 2014 at 7:50 AM, Gaurav Agarwal <
> > gzagarwal@gmail.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hello I am running apache falcon and oozie with shell script to
> > run
> > > > the
> > > > > > > workflow.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > getting this error
> > > > > > >
> > > > > > > *ell-node] Launcher ERROR, reason: Main class
> > > > > > > [org.apache.oozie.action.hadoop.ShellMain], exit code [1]*
> > > > > > > *2014-11-26 17:24:06,645 INFO ActionEndXCommand:543 -
> > > > > > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com
> >]
> > > > > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > > > > > > JOB[0000020-141126162609456-oozie-oozi-W]
> > > > > > > ACTION[0000020-141126162609456-oozie-oozi-W@shell-node] ERROR
> is
> > > > > > > considered
> > > > > > > as FAILED for SLA*
> > > > > > > *2014-11-26 17:24:06,702 INFO ActionStartXCommand:543 -
> > > > > > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com
> >]
> > > > > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > > > > > > JOB[0000020-141126162609456-oozie-oozi-W]
> > > > > > > ACTION[0000020-141126162609456-oozie-oozi-W@fail] Start action
> > > > > > > [0000020-141126162609456-oozie-*
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > *Workflow*
> > > > > > >
> > > > > > > <!--
> > > > > > > Licensed to the Apache Software Foundation (ASF) under one
> > > > > > > or more contributor license agreements. See the NOTICE file
> > > > > > > distributed with this work for additional information
> > > > > > > regarding copyright ownership. The ASF licenses this file
> > > > > > > to you under the Apache License, Version 2.0 (the
> > > > > > > "License"); you may not use this file except in compliance
> > > > > > > with the License. You may obtain a copy of the License at
> > > > > > >
> > > > > > > http://www.apache.org/licenses/LICENSE-2.0
> > > > > > >
> > > > > > > Unless required by applicable law or agreed to in writing,
> > > software
> > > > > > > distributed under the License is distributed on an "AS IS"
> > BASIS,
> > > > > > > WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
> or
> > > > > > implied.
> > > > > > > See the License for the specific language governing
> permissions
> > > and
> > > > > > > limitations under the License.
> > > > > > > -->
> > > > > > > <workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf">
> > > > > > > <start to="shell-node"/>
> > > > > > > <action name="shell-node">
> > > > > > > <shell xmlns="uri:oozie:shell-action:0.2">
> > > > > > > <job-tracker>${jobTracker}</job-tracker>
> > > > > > > <name-node>${nameNode}</name-node>
> > > > > > > <configuration>
> > > > > > > <property>
> > > > > > > <name>mapred.job.queue.name</name>
> > > > > > > <value>${queueName}</value>
> > > > > > > </property>
> > > > > > > </configuration>
> > > > > > > <exec>ingest.sh</exec>
> > > > > > > <argument>${feedInstancePaths}</argument>
> > > > > > > <file>${wf:appPath()}/ingest.sh#ingest.sh</file>
> > > > > > > <!-- <file>/tmp/ingest.sh#ingest.sh</file> -->
> > > > > > > <!-- <capture-output/> -->
> > > > > > > </shell>
> > > > > > > <ok to="end"/>
> > > > > > > <error to="fail"/>
> > > > > > > </action>
> > > > > > > <kill name="fail">
> > > > > > > <message>Shell action failed, error
> > > > > > > message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> > > > > > > </kill>
> > > > > > > <end name="end"/>
> > > > > > > </workflow-app>
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > This is the shell script
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > #!/bin/bash
> > > > > > >
> > > > > > >
> > > > > > > java -cp
> > > > > > >
> > > > > >
> > > >
> > >
> >
> "/media/sf_ToSandbox/BigData-0.0.1-SNAPSHOT.jar:/usr/hdp/2.2.0.0-913/sqoop/lib/*:/usr/hdp/2.2.0.0-913/hadoop/lib/*"
> > > > > > > com.bigdata.falcon.StaticDataProvider
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > These are the feed and process and cluster information
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > <feed description="Raw customer email feed"
> > > name="FINALrawEmailFeed"
> > > > > > > xmlns="uri:falcon:feed:0.1">
> > > > > > >
> > > > > > >
> > > > <tags>externalSystem=USWestEmailServers,classification=secure</tags>
> > > > > > > <groups>churnAnalysisDataPipeline</groups>
> > > > > > > <frequency>minutes(3)</frequency>
> > > > > > > <late-arrival cut-off="hours(4)"/>
> > > > > > > <clusters>
> > > > > > > <cluster name="SprimaryCluster" type="source">
> > > > > > > <validity start="2014-11-21T00:00Z"
> > > > end="2016-03-31T00:00Z"/>
> > > > > > > <retention limit="days(90)" action="delete"/>
> > > > > > > </cluster>
> > > > > > > </clusters>
> > > > > > >
> > > > > > > <locations>
> > > > > > > <location type="data"
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> >
> path="/user/gaurav123/falcon/demo/primary/input/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
> > > > > > > <location type="stats" path="/none"/>
> > > > > > > <location type="meta" path="/none"/>
> > > > > > > </locations>
> > > > > > >
> > > > > > > <ACL owner="ambari-qa" group="users" permission="0777"/>
> > > > > > > <schema location="/none" provider="none"/>
> > > > > > > </feed>
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > *Process*
> > > > > > >
> > > > > > > *-----------*
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > <?xml version="1.0" encoding="UTF-8"?>
> > > > > > > <!--
> > > > > > > Hourly process, ingests raw emails from all US west email
> > servers
> > > > > > > -->
> > > > > > > <process name="IngestProcess" xmlns="uri:falcon:process:0.1">
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> >
> <tags>pipeline=churnAnalysisDataPipeline,owner=ETLGroup,externalSystem=USWestEmailServers</tags>
> > > > > > >
> > > > > > >
> > > > > > > <clusters>
> > > > > > > <cluster name="SprimaryCluster">
> > > > > > > <validity start="2014-11-24T09:35Z"
> > > > end="2014-11-25T10:00Z"/>
> > > > > > > </cluster>
> > > > > > > </clusters>
> > > > > > >
> > > > > > > <parallel>1</parallel>
> > > > > > > <order>FIFO</order>
> > > > > > > <frequency>minutes(3)</frequency>
> > > > > > >
> > > > > > > <outputs>
> > > > > > > <output name="output" feed="FINALrawEmailFeed"
> > > > > > instance="now(0,0)"
> > > > > > > />
> > > > > > > </outputs>
> > > > > > >
> > > > > > > <workflow name="IngestWorkflow" version="2.0.0"
> > > > > > > engine="oozie"
> > > path="/user/gaurav123/falcon/demo/apps/ingest/fs"
> > > > />
> > > > > > >
> > > > > > > <retry policy="periodic" delay="minutes(15)" attempts="3"
> />
> > > > > > >
> > > > > > > </process>
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Can you please let me know what is the issue , I have tried
> since
> > > > long
> > > > > > two
> > > > > > > days? but unable to run the workflow with falcon process
> > > > > > >
> > > > > > > Also got this exception
> > > > > > >
> > > > > > > 2014-11-26 17:47:42,548 ERROR CompletedActionXCommand:540 -
> > > > > > > SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[]
> APP[-]
> > > > > > > JOB[0000099-141126162609456-oozie-oozi-W]
> > > > > > > ACTION[0000099-141126162609456-oozie-oozi-W@user-action]
> > > XException,
> > > > > > > org.apache.oozie.command.CommandException: E0800: Action it is
> > not
> > > > > > > running its in [PREP] state, action
> > > > > > > [0000099-141126162609456-oozie-oozi-W@user-action]
> > > > > > >
> > > > > >
> > > > > > --
> > > > > > _____________________________________________________________
> > > > > > The information contained in this communication is intended
> solely
> > > for
> > > > the
> > > > > > use of the individual or entity to whom it is addressed and
> others
> > > > > > authorized to receive it. It may contain confidential or legally
> > > > privileged
> > > > > > information. If you are not the intended recipient you are hereby
> > > > notified
> > > > > > that any disclosure, copying, distribution or taking any action
> in
> > > > reliance
> > > > > > on the contents of this information is strictly prohibited and
> may
> > be
> > > > > > unlawful. If you have received this communication in error,
> please
> > > > notify
> > > > > > us immediately by responding to this email and then delete it
> from
> > > your
> > > > > > system. The firm is neither liable for the proper and complete
> > > > transmission
> > > > > > of the information contained in this communication nor for any
> > delay
> > > > in its
> > > > > > receipt.
> > > > > >
> > > >
> > > >
> > >
> >
> > --
> > _____________________________________________________________
> > The information contained in this communication is intended solely for
> the
> > use of the individual or entity to whom it is addressed and others
> > authorized to receive it. It may contain confidential or legally
> privileged
> > information. If you are not the intended recipient you are hereby
> notified
> > that any disclosure, copying, distribution or taking any action in
> reliance
> > on the contents of this information is strictly prohibited and may be
> > unlawful. If you have received this communication in error, please notify
> > us immediately by responding to this email and then delete it from your
> > system. The firm is neither liable for the proper and complete
> transmission
> > of the information contained in this communication nor for any delay in
> its
> > receipt.
> >
>
--
_____________________________________________________________
The information contained in this communication is intended solely for the
use of the individual or entity to whom it is addressed and others
authorized to receive it. It may contain confidential or legally privileged
information. If you are not the intended recipient you are hereby notified
that any disclosure, copying, distribution or taking any action in reliance
on the contents of this information is strictly prohibited and may be
unlawful. If you have received this communication in error, please notify
us immediately by responding to this email and then delete it from your
system. The firm is neither liable for the proper and complete transmission
of the information contained in this communication nor for any delay in its
receipt.
Re: Need help for the problem that i faced
Posted by Gaurav Agarwal <gz...@gmail.com>.
Shwetha
Apologise to trouble everyone but i am new to this falcon. Do i need to
check the logs in JobBrowser?
there is stderror
Log Length: 188
Error: Could not find or load main class com.jpmc.cib.is.bigdata.
falcon.SemsStaticDataProvider
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.ShellMain],
exit code [1]
On Thu, Nov 27, 2014 at 11:43 AM, Shwetha GS <sh...@inmobi.com> wrote:
> oozie launches a hadoop job which executes the shell script. Check the
> launcher job logs
>
> On Thu, Nov 27, 2014 at 11:39 AM, Gaurav Agarwal <gz...@gmail.com>
> wrote:
>
> > Srikanth
> >
> > What do we mean by that in oozie context?
> >
> > Thanks
> >
> > On Thu, Nov 27, 2014 at 11:36 AM, Srikanth Sundarrajan <
> > sriksun@hotmail.com>
> > wrote:
> >
> > > Looks like the script ingest.sh is returning non-zero exit code.
> > >
> > > Regards
> > > Srikanth Sundarrajan
> > >
> > > > Date: Thu, 27 Nov 2014 11:31:30 +0530
> > > > Subject: Re: Need help for the problem that i faced
> > > > From: gzagarwal@gmail.com
> > > > To: dev@falcon.incubator.apache.org
> > > >
> > > > These are the log
> > > >
> > > > 2014-11-27 05:37:02,726 INFO ActionStartXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:] Start action
> > > > [0000129-141126162609456-oozie-oozi-W@:start:] with user-retry
> state :
> > > > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> > > > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
> > > > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action
> status=DONE
> > > > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
> > > > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action updated
> in
> > > DB!
> > > > 2014-11-27 05:37:03,151 INFO ActionStartXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Start action
> > > > [0000129-141126162609456-oozie-oozi-W@shell-node] with user-retry
> > state
> > > :
> > > > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> > > > 2014-11-27 05:37:05,377 INFO ShellActionExecutor:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] checking
> > action,
> > > > hadoop job ID [job_1417019189193_0117] status [RUNNING]
> > > > 2014-11-27 05:37:05,378 INFO ActionStartXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
> > > > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action
> > > > status=RUNNING
> > > > 2014-11-27 05:37:05,379 INFO ActionStartXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
> > > > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action
> updated
> > in
> > > > DB!
> > > > 2014-11-27 05:37:49,260 INFO CallbackServlet:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] callback for
> > > action
> > > > [0000129-141126162609456-oozie-oozi-W@shell-node]
> > > > 2014-11-27 05:37:50,753 INFO ShellActionExecutor:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] action
> > > completed,
> > > > external ID [job_1417019189193_0117]
> > > > 2014-11-27 05:37:50,761 WARN ShellActionExecutor:546 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Launcher
> > ERROR,
> > > > reason: Main class [org.apache.oozie.action.hadoop.ShellMain], exit
> > code
> > > > [1]
> > > > 2014-11-27 05:37:50,865 INFO ActionEndXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] ERROR is
> > > considered
> > > > as FAILED for SLA
> > > > 2014-11-27 05:37:50,941 INFO ActionStartXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@fail] Start action
> > > > [0000129-141126162609456-oozie-oozi-W@fail] with user-retry state :
> > > > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> > > > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@fail]
> > > > [***0000129-141126162609456-oozie-oozi-W@fail***]Action status=DONE
> > > > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@fail]
> > > > [***0000129-141126162609456-oozie-oozi-W@fail***]Action updated in
> DB!
> > > >
> > > > On Thu, Nov 27, 2014 at 10:25 AM, Shwetha GS <sh...@inmobi.com>
> > > wrote:
> > > >
> > > > > <exec>ingest.sh</exec>
> > > > > Try using absolute path.
> > > > >
> > > > > None of the logs imply any error. Can you attach all the oozie logs
> > > for the
> > > > > workflow?
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Nov 27, 2014 at 7:50 AM, Gaurav Agarwal <
> gzagarwal@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Hello I am running apache falcon and oozie with shell script to
> run
> > > the
> > > > > > workflow.
> > > > > >
> > > > > >
> > > > > >
> > > > > > getting this error
> > > > > >
> > > > > > *ell-node] Launcher ERROR, reason: Main class
> > > > > > [org.apache.oozie.action.hadoop.ShellMain], exit code [1]*
> > > > > > *2014-11-26 17:24:06,645 INFO ActionEndXCommand:543 -
> > > > > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
> > > > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > > > > > JOB[0000020-141126162609456-oozie-oozi-W]
> > > > > > ACTION[0000020-141126162609456-oozie-oozi-W@shell-node] ERROR is
> > > > > > considered
> > > > > > as FAILED for SLA*
> > > > > > *2014-11-26 17:24:06,702 INFO ActionStartXCommand:543 -
> > > > > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
> > > > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > > > > > JOB[0000020-141126162609456-oozie-oozi-W]
> > > > > > ACTION[0000020-141126162609456-oozie-oozi-W@fail] Start action
> > > > > > [0000020-141126162609456-oozie-*
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > *Workflow*
> > > > > >
> > > > > > <!--
> > > > > > Licensed to the Apache Software Foundation (ASF) under one
> > > > > > or more contributor license agreements. See the NOTICE file
> > > > > > distributed with this work for additional information
> > > > > > regarding copyright ownership. The ASF licenses this file
> > > > > > to you under the Apache License, Version 2.0 (the
> > > > > > "License"); you may not use this file except in compliance
> > > > > > with the License. You may obtain a copy of the License at
> > > > > >
> > > > > > http://www.apache.org/licenses/LICENSE-2.0
> > > > > >
> > > > > > Unless required by applicable law or agreed to in writing,
> > software
> > > > > > distributed under the License is distributed on an "AS IS"
> BASIS,
> > > > > > WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> > > > > implied.
> > > > > > See the License for the specific language governing permissions
> > and
> > > > > > limitations under the License.
> > > > > > -->
> > > > > > <workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf">
> > > > > > <start to="shell-node"/>
> > > > > > <action name="shell-node">
> > > > > > <shell xmlns="uri:oozie:shell-action:0.2">
> > > > > > <job-tracker>${jobTracker}</job-tracker>
> > > > > > <name-node>${nameNode}</name-node>
> > > > > > <configuration>
> > > > > > <property>
> > > > > > <name>mapred.job.queue.name</name>
> > > > > > <value>${queueName}</value>
> > > > > > </property>
> > > > > > </configuration>
> > > > > > <exec>ingest.sh</exec>
> > > > > > <argument>${feedInstancePaths}</argument>
> > > > > > <file>${wf:appPath()}/ingest.sh#ingest.sh</file>
> > > > > > <!-- <file>/tmp/ingest.sh#ingest.sh</file> -->
> > > > > > <!-- <capture-output/> -->
> > > > > > </shell>
> > > > > > <ok to="end"/>
> > > > > > <error to="fail"/>
> > > > > > </action>
> > > > > > <kill name="fail">
> > > > > > <message>Shell action failed, error
> > > > > > message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> > > > > > </kill>
> > > > > > <end name="end"/>
> > > > > > </workflow-app>
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > This is the shell script
> > > > > >
> > > > > >
> > > > > >
> > > > > > #!/bin/bash
> > > > > >
> > > > > >
> > > > > > java -cp
> > > > > >
> > > > >
> > >
> >
> "/media/sf_ToSandbox/BigData-0.0.1-SNAPSHOT.jar:/usr/hdp/2.2.0.0-913/sqoop/lib/*:/usr/hdp/2.2.0.0-913/hadoop/lib/*"
> > > > > > com.bigdata.falcon.StaticDataProvider
> > > > > >
> > > > > >
> > > > > >
> > > > > > These are the feed and process and cluster information
> > > > > >
> > > > > >
> > > > > >
> > > > > > <feed description="Raw customer email feed"
> > name="FINALrawEmailFeed"
> > > > > > xmlns="uri:falcon:feed:0.1">
> > > > > >
> > > > > >
> > > <tags>externalSystem=USWestEmailServers,classification=secure</tags>
> > > > > > <groups>churnAnalysisDataPipeline</groups>
> > > > > > <frequency>minutes(3)</frequency>
> > > > > > <late-arrival cut-off="hours(4)"/>
> > > > > > <clusters>
> > > > > > <cluster name="SprimaryCluster" type="source">
> > > > > > <validity start="2014-11-21T00:00Z"
> > > end="2016-03-31T00:00Z"/>
> > > > > > <retention limit="days(90)" action="delete"/>
> > > > > > </cluster>
> > > > > > </clusters>
> > > > > >
> > > > > > <locations>
> > > > > > <location type="data"
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > >
> >
> path="/user/gaurav123/falcon/demo/primary/input/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
> > > > > > <location type="stats" path="/none"/>
> > > > > > <location type="meta" path="/none"/>
> > > > > > </locations>
> > > > > >
> > > > > > <ACL owner="ambari-qa" group="users" permission="0777"/>
> > > > > > <schema location="/none" provider="none"/>
> > > > > > </feed>
> > > > > >
> > > > > >
> > > > > >
> > > > > > *Process*
> > > > > >
> > > > > > *-----------*
> > > > > >
> > > > > >
> > > > > >
> > > > > > <?xml version="1.0" encoding="UTF-8"?>
> > > > > > <!--
> > > > > > Hourly process, ingests raw emails from all US west email
> servers
> > > > > > -->
> > > > > > <process name="IngestProcess" xmlns="uri:falcon:process:0.1">
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > >
> >
> <tags>pipeline=churnAnalysisDataPipeline,owner=ETLGroup,externalSystem=USWestEmailServers</tags>
> > > > > >
> > > > > >
> > > > > > <clusters>
> > > > > > <cluster name="SprimaryCluster">
> > > > > > <validity start="2014-11-24T09:35Z"
> > > end="2014-11-25T10:00Z"/>
> > > > > > </cluster>
> > > > > > </clusters>
> > > > > >
> > > > > > <parallel>1</parallel>
> > > > > > <order>FIFO</order>
> > > > > > <frequency>minutes(3)</frequency>
> > > > > >
> > > > > > <outputs>
> > > > > > <output name="output" feed="FINALrawEmailFeed"
> > > > > instance="now(0,0)"
> > > > > > />
> > > > > > </outputs>
> > > > > >
> > > > > > <workflow name="IngestWorkflow" version="2.0.0"
> > > > > > engine="oozie"
> > path="/user/gaurav123/falcon/demo/apps/ingest/fs"
> > > />
> > > > > >
> > > > > > <retry policy="periodic" delay="minutes(15)" attempts="3" />
> > > > > >
> > > > > > </process>
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Can you please let me know what is the issue , I have tried since
> > > long
> > > > > two
> > > > > > days? but unable to run the workflow with falcon process
> > > > > >
> > > > > > Also got this exception
> > > > > >
> > > > > > 2014-11-26 17:47:42,548 ERROR CompletedActionXCommand:540 -
> > > > > > SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[] APP[-]
> > > > > > JOB[0000099-141126162609456-oozie-oozi-W]
> > > > > > ACTION[0000099-141126162609456-oozie-oozi-W@user-action]
> > XException,
> > > > > > org.apache.oozie.command.CommandException: E0800: Action it is
> not
> > > > > > running its in [PREP] state, action
> > > > > > [0000099-141126162609456-oozie-oozi-W@user-action]
> > > > > >
> > > > >
> > > > > --
> > > > > _____________________________________________________________
> > > > > The information contained in this communication is intended solely
> > for
> > > the
> > > > > use of the individual or entity to whom it is addressed and others
> > > > > authorized to receive it. It may contain confidential or legally
> > > privileged
> > > > > information. If you are not the intended recipient you are hereby
> > > notified
> > > > > that any disclosure, copying, distribution or taking any action in
> > > reliance
> > > > > on the contents of this information is strictly prohibited and may
> be
> > > > > unlawful. If you have received this communication in error, please
> > > notify
> > > > > us immediately by responding to this email and then delete it from
> > your
> > > > > system. The firm is neither liable for the proper and complete
> > > transmission
> > > > > of the information contained in this communication nor for any
> delay
> > > in its
> > > > > receipt.
> > > > >
> > >
> > >
> >
>
> --
> _____________________________________________________________
> The information contained in this communication is intended solely for the
> use of the individual or entity to whom it is addressed and others
> authorized to receive it. It may contain confidential or legally privileged
> information. If you are not the intended recipient you are hereby notified
> that any disclosure, copying, distribution or taking any action in reliance
> on the contents of this information is strictly prohibited and may be
> unlawful. If you have received this communication in error, please notify
> us immediately by responding to this email and then delete it from your
> system. The firm is neither liable for the proper and complete transmission
> of the information contained in this communication nor for any delay in its
> receipt.
>
Re: Need help for the problem that i faced
Posted by Shwetha GS <sh...@inmobi.com>.
oozie launches a hadoop job which executes the shell script. Check the
launcher job logs
On Thu, Nov 27, 2014 at 11:39 AM, Gaurav Agarwal <gz...@gmail.com>
wrote:
> Srikanth
>
> What do we mean by that in oozie context?
>
> Thanks
>
> On Thu, Nov 27, 2014 at 11:36 AM, Srikanth Sundarrajan <
> sriksun@hotmail.com>
> wrote:
>
> > Looks like the script ingest.sh is returning non-zero exit code.
> >
> > Regards
> > Srikanth Sundarrajan
> >
> > > Date: Thu, 27 Nov 2014 11:31:30 +0530
> > > Subject: Re: Need help for the problem that i faced
> > > From: gzagarwal@gmail.com
> > > To: dev@falcon.incubator.apache.org
> > >
> > > These are the log
> > >
> > > 2014-11-27 05:37:02,726 INFO ActionStartXCommand:543 - SERVER[
> > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:] Start action
> > > [0000129-141126162609456-oozie-oozi-W@:start:] with user-retry state :
> > > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> > > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
> > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
> > > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action status=DONE
> > > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
> > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
> > > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action updated in
> > DB!
> > > 2014-11-27 05:37:03,151 INFO ActionStartXCommand:543 - SERVER[
> > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Start action
> > > [0000129-141126162609456-oozie-oozi-W@shell-node] with user-retry
> state
> > :
> > > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> > > 2014-11-27 05:37:05,377 INFO ShellActionExecutor:543 - SERVER[
> > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] checking
> action,
> > > hadoop job ID [job_1417019189193_0117] status [RUNNING]
> > > 2014-11-27 05:37:05,378 INFO ActionStartXCommand:543 - SERVER[
> > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
> > > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action
> > > status=RUNNING
> > > 2014-11-27 05:37:05,379 INFO ActionStartXCommand:543 - SERVER[
> > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
> > > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action updated
> in
> > > DB!
> > > 2014-11-27 05:37:49,260 INFO CallbackServlet:543 - SERVER[
> > > sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-]
> > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] callback for
> > action
> > > [0000129-141126162609456-oozie-oozi-W@shell-node]
> > > 2014-11-27 05:37:50,753 INFO ShellActionExecutor:543 - SERVER[
> > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] action
> > completed,
> > > external ID [job_1417019189193_0117]
> > > 2014-11-27 05:37:50,761 WARN ShellActionExecutor:546 - SERVER[
> > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Launcher
> ERROR,
> > > reason: Main class [org.apache.oozie.action.hadoop.ShellMain], exit
> code
> > > [1]
> > > 2014-11-27 05:37:50,865 INFO ActionEndXCommand:543 - SERVER[
> > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] ERROR is
> > considered
> > > as FAILED for SLA
> > > 2014-11-27 05:37:50,941 INFO ActionStartXCommand:543 - SERVER[
> > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > ACTION[0000129-141126162609456-oozie-oozi-W@fail] Start action
> > > [0000129-141126162609456-oozie-oozi-W@fail] with user-retry state :
> > > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> > > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
> > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > ACTION[0000129-141126162609456-oozie-oozi-W@fail]
> > > [***0000129-141126162609456-oozie-oozi-W@fail***]Action status=DONE
> > > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
> > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > ACTION[0000129-141126162609456-oozie-oozi-W@fail]
> > > [***0000129-141126162609456-oozie-oozi-W@fail***]Action updated in DB!
> > >
> > > On Thu, Nov 27, 2014 at 10:25 AM, Shwetha GS <sh...@inmobi.com>
> > wrote:
> > >
> > > > <exec>ingest.sh</exec>
> > > > Try using absolute path.
> > > >
> > > > None of the logs imply any error. Can you attach all the oozie logs
> > for the
> > > > workflow?
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, Nov 27, 2014 at 7:50 AM, Gaurav Agarwal <gzagarwal@gmail.com
> >
> > > > wrote:
> > > >
> > > > > Hello I am running apache falcon and oozie with shell script to run
> > the
> > > > > workflow.
> > > > >
> > > > >
> > > > >
> > > > > getting this error
> > > > >
> > > > > *ell-node] Launcher ERROR, reason: Main class
> > > > > [org.apache.oozie.action.hadoop.ShellMain], exit code [1]*
> > > > > *2014-11-26 17:24:06,645 INFO ActionEndXCommand:543 -
> > > > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
> > > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > > > > JOB[0000020-141126162609456-oozie-oozi-W]
> > > > > ACTION[0000020-141126162609456-oozie-oozi-W@shell-node] ERROR is
> > > > > considered
> > > > > as FAILED for SLA*
> > > > > *2014-11-26 17:24:06,702 INFO ActionStartXCommand:543 -
> > > > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
> > > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > > > > JOB[0000020-141126162609456-oozie-oozi-W]
> > > > > ACTION[0000020-141126162609456-oozie-oozi-W@fail] Start action
> > > > > [0000020-141126162609456-oozie-*
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > *Workflow*
> > > > >
> > > > > <!--
> > > > > Licensed to the Apache Software Foundation (ASF) under one
> > > > > or more contributor license agreements. See the NOTICE file
> > > > > distributed with this work for additional information
> > > > > regarding copyright ownership. The ASF licenses this file
> > > > > to you under the Apache License, Version 2.0 (the
> > > > > "License"); you may not use this file except in compliance
> > > > > with the License. You may obtain a copy of the License at
> > > > >
> > > > > http://www.apache.org/licenses/LICENSE-2.0
> > > > >
> > > > > Unless required by applicable law or agreed to in writing,
> software
> > > > > distributed under the License is distributed on an "AS IS" BASIS,
> > > > > WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> > > > implied.
> > > > > See the License for the specific language governing permissions
> and
> > > > > limitations under the License.
> > > > > -->
> > > > > <workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf">
> > > > > <start to="shell-node"/>
> > > > > <action name="shell-node">
> > > > > <shell xmlns="uri:oozie:shell-action:0.2">
> > > > > <job-tracker>${jobTracker}</job-tracker>
> > > > > <name-node>${nameNode}</name-node>
> > > > > <configuration>
> > > > > <property>
> > > > > <name>mapred.job.queue.name</name>
> > > > > <value>${queueName}</value>
> > > > > </property>
> > > > > </configuration>
> > > > > <exec>ingest.sh</exec>
> > > > > <argument>${feedInstancePaths}</argument>
> > > > > <file>${wf:appPath()}/ingest.sh#ingest.sh</file>
> > > > > <!-- <file>/tmp/ingest.sh#ingest.sh</file> -->
> > > > > <!-- <capture-output/> -->
> > > > > </shell>
> > > > > <ok to="end"/>
> > > > > <error to="fail"/>
> > > > > </action>
> > > > > <kill name="fail">
> > > > > <message>Shell action failed, error
> > > > > message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> > > > > </kill>
> > > > > <end name="end"/>
> > > > > </workflow-app>
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > This is the shell script
> > > > >
> > > > >
> > > > >
> > > > > #!/bin/bash
> > > > >
> > > > >
> > > > > java -cp
> > > > >
> > > >
> >
> "/media/sf_ToSandbox/BigData-0.0.1-SNAPSHOT.jar:/usr/hdp/2.2.0.0-913/sqoop/lib/*:/usr/hdp/2.2.0.0-913/hadoop/lib/*"
> > > > > com.bigdata.falcon.StaticDataProvider
> > > > >
> > > > >
> > > > >
> > > > > These are the feed and process and cluster information
> > > > >
> > > > >
> > > > >
> > > > > <feed description="Raw customer email feed"
> name="FINALrawEmailFeed"
> > > > > xmlns="uri:falcon:feed:0.1">
> > > > >
> > > > >
> > <tags>externalSystem=USWestEmailServers,classification=secure</tags>
> > > > > <groups>churnAnalysisDataPipeline</groups>
> > > > > <frequency>minutes(3)</frequency>
> > > > > <late-arrival cut-off="hours(4)"/>
> > > > > <clusters>
> > > > > <cluster name="SprimaryCluster" type="source">
> > > > > <validity start="2014-11-21T00:00Z"
> > end="2016-03-31T00:00Z"/>
> > > > > <retention limit="days(90)" action="delete"/>
> > > > > </cluster>
> > > > > </clusters>
> > > > >
> > > > > <locations>
> > > > > <location type="data"
> > > > >
> > > > >
> > > > >
> > > >
> >
> path="/user/gaurav123/falcon/demo/primary/input/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
> > > > > <location type="stats" path="/none"/>
> > > > > <location type="meta" path="/none"/>
> > > > > </locations>
> > > > >
> > > > > <ACL owner="ambari-qa" group="users" permission="0777"/>
> > > > > <schema location="/none" provider="none"/>
> > > > > </feed>
> > > > >
> > > > >
> > > > >
> > > > > *Process*
> > > > >
> > > > > *-----------*
> > > > >
> > > > >
> > > > >
> > > > > <?xml version="1.0" encoding="UTF-8"?>
> > > > > <!--
> > > > > Hourly process, ingests raw emails from all US west email servers
> > > > > -->
> > > > > <process name="IngestProcess" xmlns="uri:falcon:process:0.1">
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> >
> <tags>pipeline=churnAnalysisDataPipeline,owner=ETLGroup,externalSystem=USWestEmailServers</tags>
> > > > >
> > > > >
> > > > > <clusters>
> > > > > <cluster name="SprimaryCluster">
> > > > > <validity start="2014-11-24T09:35Z"
> > end="2014-11-25T10:00Z"/>
> > > > > </cluster>
> > > > > </clusters>
> > > > >
> > > > > <parallel>1</parallel>
> > > > > <order>FIFO</order>
> > > > > <frequency>minutes(3)</frequency>
> > > > >
> > > > > <outputs>
> > > > > <output name="output" feed="FINALrawEmailFeed"
> > > > instance="now(0,0)"
> > > > > />
> > > > > </outputs>
> > > > >
> > > > > <workflow name="IngestWorkflow" version="2.0.0"
> > > > > engine="oozie"
> path="/user/gaurav123/falcon/demo/apps/ingest/fs"
> > />
> > > > >
> > > > > <retry policy="periodic" delay="minutes(15)" attempts="3" />
> > > > >
> > > > > </process>
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Can you please let me know what is the issue , I have tried since
> > long
> > > > two
> > > > > days? but unable to run the workflow with falcon process
> > > > >
> > > > > Also got this exception
> > > > >
> > > > > 2014-11-26 17:47:42,548 ERROR CompletedActionXCommand:540 -
> > > > > SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[] APP[-]
> > > > > JOB[0000099-141126162609456-oozie-oozi-W]
> > > > > ACTION[0000099-141126162609456-oozie-oozi-W@user-action]
> XException,
> > > > > org.apache.oozie.command.CommandException: E0800: Action it is not
> > > > > running its in [PREP] state, action
> > > > > [0000099-141126162609456-oozie-oozi-W@user-action]
> > > > >
> > > >
> > > > --
> > > > _____________________________________________________________
> > > > The information contained in this communication is intended solely
> for
> > the
> > > > use of the individual or entity to whom it is addressed and others
> > > > authorized to receive it. It may contain confidential or legally
> > privileged
> > > > information. If you are not the intended recipient you are hereby
> > notified
> > > > that any disclosure, copying, distribution or taking any action in
> > reliance
> > > > on the contents of this information is strictly prohibited and may be
> > > > unlawful. If you have received this communication in error, please
> > notify
> > > > us immediately by responding to this email and then delete it from
> your
> > > > system. The firm is neither liable for the proper and complete
> > transmission
> > > > of the information contained in this communication nor for any delay
> > in its
> > > > receipt.
> > > >
> >
> >
>
--
_____________________________________________________________
The information contained in this communication is intended solely for the
use of the individual or entity to whom it is addressed and others
authorized to receive it. It may contain confidential or legally privileged
information. If you are not the intended recipient you are hereby notified
that any disclosure, copying, distribution or taking any action in reliance
on the contents of this information is strictly prohibited and may be
unlawful. If you have received this communication in error, please notify
us immediately by responding to this email and then delete it from your
system. The firm is neither liable for the proper and complete transmission
of the information contained in this communication nor for any delay in its
receipt.
Re: Need help for the problem that i faced
Posted by Shwetha GS <sh...@inmobi.com>.
2 ways of doing:
1. Falcon process exposes workflow lib(just like workflow path) which
should contain the path to the directory that contains the jar
example:
<workflow name="test" version="1.0.0" engine="oozie"
path="/user/guest/myapp" lib="/user/guest/mylib"/>
2. If the workflow is in <path>/workflow.xml, the jar should be in
<path>/lib/ directory
On Thu, Nov 27, 2014 at 11:42 AM, Gaurav Agarwal <gz...@gmail.com>
wrote:
> Apart from this I have two more questions?
>
> 1. How to invoke java class from a workflow ,where to keep the jar in which
> the class present?I kept the jar at the same location where the workflow is
> present still got the same error. ClassNotfound exception
>
> Thanks
>
> On Thu, Nov 27, 2014 at 11:39 AM, Gaurav Agarwal <gz...@gmail.com>
> wrote:
>
> > Srikanth
> >
> > What do we mean by that in oozie context?
> >
> > Thanks
> >
> > On Thu, Nov 27, 2014 at 11:36 AM, Srikanth Sundarrajan <
> > sriksun@hotmail.com> wrote:
> >
> >> Looks like the script ingest.sh is returning non-zero exit code.
> >>
> >> Regards
> >> Srikanth Sundarrajan
> >>
> >> > Date: Thu, 27 Nov 2014 11:31:30 +0530
> >> > Subject: Re: Need help for the problem that i faced
> >> > From: gzagarwal@gmail.com
> >> > To: dev@falcon.incubator.apache.org
> >> >
> >> > These are the log
> >> >
> >> > 2014-11-27 05:37:02,726 INFO ActionStartXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@:start:] Start action
> >> > [0000129-141126162609456-oozie-oozi-W@:start:] with user-retry state
> :
> >> > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> >> > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
> >> > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action
> status=DONE
> >> > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
> >> > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action updated in
> >> DB!
> >> > 2014-11-27 05:37:03,151 INFO ActionStartXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Start action
> >> > [0000129-141126162609456-oozie-oozi-W@shell-node] with user-retry
> >> state :
> >> > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> >> > 2014-11-27 05:37:05,377 INFO ShellActionExecutor:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] checking
> >> action,
> >> > hadoop job ID [job_1417019189193_0117] status [RUNNING]
> >> > 2014-11-27 05:37:05,378 INFO ActionStartXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
> >> > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action
> >> > status=RUNNING
> >> > 2014-11-27 05:37:05,379 INFO ActionStartXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
> >> > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action updated
> >> in
> >> > DB!
> >> > 2014-11-27 05:37:49,260 INFO CallbackServlet:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] callback for
> >> action
> >> > [0000129-141126162609456-oozie-oozi-W@shell-node]
> >> > 2014-11-27 05:37:50,753 INFO ShellActionExecutor:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] action
> >> completed,
> >> > external ID [job_1417019189193_0117]
> >> > 2014-11-27 05:37:50,761 WARN ShellActionExecutor:546 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Launcher
> ERROR,
> >> > reason: Main class [org.apache.oozie.action.hadoop.ShellMain], exit
> code
> >> > [1]
> >> > 2014-11-27 05:37:50,865 INFO ActionEndXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] ERROR is
> >> considered
> >> > as FAILED for SLA
> >> > 2014-11-27 05:37:50,941 INFO ActionStartXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@fail] Start action
> >> > [0000129-141126162609456-oozie-oozi-W@fail] with user-retry state :
> >> > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> >> > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@fail]
> >> > [***0000129-141126162609456-oozie-oozi-W@fail***]Action status=DONE
> >> > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@fail]
> >> > [***0000129-141126162609456-oozie-oozi-W@fail***]Action updated in
> DB!
> >> >
> >> > On Thu, Nov 27, 2014 at 10:25 AM, Shwetha GS <sh...@inmobi.com>
> >> wrote:
> >> >
> >> > > <exec>ingest.sh</exec>
> >> > > Try using absolute path.
> >> > >
> >> > > None of the logs imply any error. Can you attach all the oozie logs
> >> for the
> >> > > workflow?
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > On Thu, Nov 27, 2014 at 7:50 AM, Gaurav Agarwal <
> gzagarwal@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > Hello I am running apache falcon and oozie with shell script to
> run
> >> the
> >> > > > workflow.
> >> > > >
> >> > > >
> >> > > >
> >> > > > getting this error
> >> > > >
> >> > > > *ell-node] Launcher ERROR, reason: Main class
> >> > > > [org.apache.oozie.action.hadoop.ShellMain], exit code [1]*
> >> > > > *2014-11-26 17:24:06,645 INFO ActionEndXCommand:543 -
> >> > > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
> >> > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> >> > > > JOB[0000020-141126162609456-oozie-oozi-W]
> >> > > > ACTION[0000020-141126162609456-oozie-oozi-W@shell-node] ERROR is
> >> > > > considered
> >> > > > as FAILED for SLA*
> >> > > > *2014-11-26 17:24:06,702 INFO ActionStartXCommand:543 -
> >> > > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
> >> > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> >> > > > JOB[0000020-141126162609456-oozie-oozi-W]
> >> > > > ACTION[0000020-141126162609456-oozie-oozi-W@fail] Start action
> >> > > > [0000020-141126162609456-oozie-*
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > *Workflow*
> >> > > >
> >> > > > <!--
> >> > > > Licensed to the Apache Software Foundation (ASF) under one
> >> > > > or more contributor license agreements. See the NOTICE file
> >> > > > distributed with this work for additional information
> >> > > > regarding copyright ownership. The ASF licenses this file
> >> > > > to you under the Apache License, Version 2.0 (the
> >> > > > "License"); you may not use this file except in compliance
> >> > > > with the License. You may obtain a copy of the License at
> >> > > >
> >> > > > http://www.apache.org/licenses/LICENSE-2.0
> >> > > >
> >> > > > Unless required by applicable law or agreed to in writing,
> >> software
> >> > > > distributed under the License is distributed on an "AS IS"
> BASIS,
> >> > > > WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> >> > > implied.
> >> > > > See the License for the specific language governing permissions
> >> and
> >> > > > limitations under the License.
> >> > > > -->
> >> > > > <workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf">
> >> > > > <start to="shell-node"/>
> >> > > > <action name="shell-node">
> >> > > > <shell xmlns="uri:oozie:shell-action:0.2">
> >> > > > <job-tracker>${jobTracker}</job-tracker>
> >> > > > <name-node>${nameNode}</name-node>
> >> > > > <configuration>
> >> > > > <property>
> >> > > > <name>mapred.job.queue.name</name>
> >> > > > <value>${queueName}</value>
> >> > > > </property>
> >> > > > </configuration>
> >> > > > <exec>ingest.sh</exec>
> >> > > > <argument>${feedInstancePaths}</argument>
> >> > > > <file>${wf:appPath()}/ingest.sh#ingest.sh</file>
> >> > > > <!-- <file>/tmp/ingest.sh#ingest.sh</file> -->
> >> > > > <!-- <capture-output/> -->
> >> > > > </shell>
> >> > > > <ok to="end"/>
> >> > > > <error to="fail"/>
> >> > > > </action>
> >> > > > <kill name="fail">
> >> > > > <message>Shell action failed, error
> >> > > > message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> >> > > > </kill>
> >> > > > <end name="end"/>
> >> > > > </workflow-app>
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > This is the shell script
> >> > > >
> >> > > >
> >> > > >
> >> > > > #!/bin/bash
> >> > > >
> >> > > >
> >> > > > java -cp
> >> > > >
> >> > >
> >>
> "/media/sf_ToSandbox/BigData-0.0.1-SNAPSHOT.jar:/usr/hdp/2.2.0.0-913/sqoop/lib/*:/usr/hdp/2.2.0.0-913/hadoop/lib/*"
> >> > > > com.bigdata.falcon.StaticDataProvider
> >> > > >
> >> > > >
> >> > > >
> >> > > > These are the feed and process and cluster information
> >> > > >
> >> > > >
> >> > > >
> >> > > > <feed description="Raw customer email feed"
> name="FINALrawEmailFeed"
> >> > > > xmlns="uri:falcon:feed:0.1">
> >> > > >
> >> > > >
> >> <tags>externalSystem=USWestEmailServers,classification=secure</tags>
> >> > > > <groups>churnAnalysisDataPipeline</groups>
> >> > > > <frequency>minutes(3)</frequency>
> >> > > > <late-arrival cut-off="hours(4)"/>
> >> > > > <clusters>
> >> > > > <cluster name="SprimaryCluster" type="source">
> >> > > > <validity start="2014-11-21T00:00Z"
> >> end="2016-03-31T00:00Z"/>
> >> > > > <retention limit="days(90)" action="delete"/>
> >> > > > </cluster>
> >> > > > </clusters>
> >> > > >
> >> > > > <locations>
> >> > > > <location type="data"
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> path="/user/gaurav123/falcon/demo/primary/input/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
> >> > > > <location type="stats" path="/none"/>
> >> > > > <location type="meta" path="/none"/>
> >> > > > </locations>
> >> > > >
> >> > > > <ACL owner="ambari-qa" group="users" permission="0777"/>
> >> > > > <schema location="/none" provider="none"/>
> >> > > > </feed>
> >> > > >
> >> > > >
> >> > > >
> >> > > > *Process*
> >> > > >
> >> > > > *-----------*
> >> > > >
> >> > > >
> >> > > >
> >> > > > <?xml version="1.0" encoding="UTF-8"?>
> >> > > > <!--
> >> > > > Hourly process, ingests raw emails from all US west email servers
> >> > > > -->
> >> > > > <process name="IngestProcess" xmlns="uri:falcon:process:0.1">
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> <tags>pipeline=churnAnalysisDataPipeline,owner=ETLGroup,externalSystem=USWestEmailServers</tags>
> >> > > >
> >> > > >
> >> > > > <clusters>
> >> > > > <cluster name="SprimaryCluster">
> >> > > > <validity start="2014-11-24T09:35Z"
> >> end="2014-11-25T10:00Z"/>
> >> > > > </cluster>
> >> > > > </clusters>
> >> > > >
> >> > > > <parallel>1</parallel>
> >> > > > <order>FIFO</order>
> >> > > > <frequency>minutes(3)</frequency>
> >> > > >
> >> > > > <outputs>
> >> > > > <output name="output" feed="FINALrawEmailFeed"
> >> > > instance="now(0,0)"
> >> > > > />
> >> > > > </outputs>
> >> > > >
> >> > > > <workflow name="IngestWorkflow" version="2.0.0"
> >> > > > engine="oozie"
> >> path="/user/gaurav123/falcon/demo/apps/ingest/fs" />
> >> > > >
> >> > > > <retry policy="periodic" delay="minutes(15)" attempts="3" />
> >> > > >
> >> > > > </process>
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > Can you please let me know what is the issue , I have tried since
> >> long
> >> > > two
> >> > > > days? but unable to run the workflow with falcon process
> >> > > >
> >> > > > Also got this exception
> >> > > >
> >> > > > 2014-11-26 17:47:42,548 ERROR CompletedActionXCommand:540 -
> >> > > > SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[] APP[-]
> >> > > > JOB[0000099-141126162609456-oozie-oozi-W]
> >> > > > ACTION[0000099-141126162609456-oozie-oozi-W@user-action]
> >> XException,
> >> > > > org.apache.oozie.command.CommandException: E0800: Action it is not
> >> > > > running its in [PREP] state, action
> >> > > > [0000099-141126162609456-oozie-oozi-W@user-action]
> >> > > >
> >> > >
> >> > > --
> >> > > _____________________________________________________________
> >> > > The information contained in this communication is intended solely
> >> for the
> >> > > use of the individual or entity to whom it is addressed and others
> >> > > authorized to receive it. It may contain confidential or legally
> >> privileged
> >> > > information. If you are not the intended recipient you are hereby
> >> notified
> >> > > that any disclosure, copying, distribution or taking any action in
> >> reliance
> >> > > on the contents of this information is strictly prohibited and may
> be
> >> > > unlawful. If you have received this communication in error, please
> >> notify
> >> > > us immediately by responding to this email and then delete it from
> >> your
> >> > > system. The firm is neither liable for the proper and complete
> >> transmission
> >> > > of the information contained in this communication nor for any delay
> >> in its
> >> > > receipt.
> >> > >
> >>
> >>
> >
> >
>
--
_____________________________________________________________
The information contained in this communication is intended solely for the
use of the individual or entity to whom it is addressed and others
authorized to receive it. It may contain confidential or legally privileged
information. If you are not the intended recipient you are hereby notified
that any disclosure, copying, distribution or taking any action in reliance
on the contents of this information is strictly prohibited and may be
unlawful. If you have received this communication in error, please notify
us immediately by responding to this email and then delete it from your
system. The firm is neither liable for the proper and complete transmission
of the information contained in this communication nor for any delay in its
receipt.
Re: Need help for the problem that i faced
Posted by Gaurav Agarwal <gz...@gmail.com>.
Apart from this I have two more questions?
1. How to invoke java class from a workflow ,where to keep the jar in which
the class present?I kept the jar at the same location where the workflow is
present still got the same error. ClassNotfound exception
Thanks
On Thu, Nov 27, 2014 at 11:39 AM, Gaurav Agarwal <gz...@gmail.com>
wrote:
> Srikanth
>
> What do we mean by that in oozie context?
>
> Thanks
>
> On Thu, Nov 27, 2014 at 11:36 AM, Srikanth Sundarrajan <
> sriksun@hotmail.com> wrote:
>
>> Looks like the script ingest.sh is returning non-zero exit code.
>>
>> Regards
>> Srikanth Sundarrajan
>>
>> > Date: Thu, 27 Nov 2014 11:31:30 +0530
>> > Subject: Re: Need help for the problem that i faced
>> > From: gzagarwal@gmail.com
>> > To: dev@falcon.incubator.apache.org
>> >
>> > These are the log
>> >
>> > 2014-11-27 05:37:02,726 INFO ActionStartXCommand:543 - SERVER[
>> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
>> > JOB[0000129-141126162609456-oozie-oozi-W]
>> > ACTION[0000129-141126162609456-oozie-oozi-W@:start:] Start action
>> > [0000129-141126162609456-oozie-oozi-W@:start:] with user-retry state :
>> > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
>> > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
>> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
>> > JOB[0000129-141126162609456-oozie-oozi-W]
>> > ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
>> > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action status=DONE
>> > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
>> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
>> > JOB[0000129-141126162609456-oozie-oozi-W]
>> > ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
>> > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action updated in
>> DB!
>> > 2014-11-27 05:37:03,151 INFO ActionStartXCommand:543 - SERVER[
>> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
>> > JOB[0000129-141126162609456-oozie-oozi-W]
>> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Start action
>> > [0000129-141126162609456-oozie-oozi-W@shell-node] with user-retry
>> state :
>> > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
>> > 2014-11-27 05:37:05,377 INFO ShellActionExecutor:543 - SERVER[
>> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
>> > JOB[0000129-141126162609456-oozie-oozi-W]
>> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] checking
>> action,
>> > hadoop job ID [job_1417019189193_0117] status [RUNNING]
>> > 2014-11-27 05:37:05,378 INFO ActionStartXCommand:543 - SERVER[
>> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
>> > JOB[0000129-141126162609456-oozie-oozi-W]
>> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
>> > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action
>> > status=RUNNING
>> > 2014-11-27 05:37:05,379 INFO ActionStartXCommand:543 - SERVER[
>> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
>> > JOB[0000129-141126162609456-oozie-oozi-W]
>> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
>> > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action updated
>> in
>> > DB!
>> > 2014-11-27 05:37:49,260 INFO CallbackServlet:543 - SERVER[
>> > sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-]
>> > JOB[0000129-141126162609456-oozie-oozi-W]
>> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] callback for
>> action
>> > [0000129-141126162609456-oozie-oozi-W@shell-node]
>> > 2014-11-27 05:37:50,753 INFO ShellActionExecutor:543 - SERVER[
>> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
>> > JOB[0000129-141126162609456-oozie-oozi-W]
>> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] action
>> completed,
>> > external ID [job_1417019189193_0117]
>> > 2014-11-27 05:37:50,761 WARN ShellActionExecutor:546 - SERVER[
>> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
>> > JOB[0000129-141126162609456-oozie-oozi-W]
>> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Launcher ERROR,
>> > reason: Main class [org.apache.oozie.action.hadoop.ShellMain], exit code
>> > [1]
>> > 2014-11-27 05:37:50,865 INFO ActionEndXCommand:543 - SERVER[
>> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
>> > JOB[0000129-141126162609456-oozie-oozi-W]
>> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] ERROR is
>> considered
>> > as FAILED for SLA
>> > 2014-11-27 05:37:50,941 INFO ActionStartXCommand:543 - SERVER[
>> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
>> > JOB[0000129-141126162609456-oozie-oozi-W]
>> > ACTION[0000129-141126162609456-oozie-oozi-W@fail] Start action
>> > [0000129-141126162609456-oozie-oozi-W@fail] with user-retry state :
>> > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
>> > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
>> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
>> > JOB[0000129-141126162609456-oozie-oozi-W]
>> > ACTION[0000129-141126162609456-oozie-oozi-W@fail]
>> > [***0000129-141126162609456-oozie-oozi-W@fail***]Action status=DONE
>> > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
>> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
>> > JOB[0000129-141126162609456-oozie-oozi-W]
>> > ACTION[0000129-141126162609456-oozie-oozi-W@fail]
>> > [***0000129-141126162609456-oozie-oozi-W@fail***]Action updated in DB!
>> >
>> > On Thu, Nov 27, 2014 at 10:25 AM, Shwetha GS <sh...@inmobi.com>
>> wrote:
>> >
>> > > <exec>ingest.sh</exec>
>> > > Try using absolute path.
>> > >
>> > > None of the logs imply any error. Can you attach all the oozie logs
>> for the
>> > > workflow?
>> > >
>> > >
>> > >
>> > >
>> > > On Thu, Nov 27, 2014 at 7:50 AM, Gaurav Agarwal <gz...@gmail.com>
>> > > wrote:
>> > >
>> > > > Hello I am running apache falcon and oozie with shell script to run
>> the
>> > > > workflow.
>> > > >
>> > > >
>> > > >
>> > > > getting this error
>> > > >
>> > > > *ell-node] Launcher ERROR, reason: Main class
>> > > > [org.apache.oozie.action.hadoop.ShellMain], exit code [1]*
>> > > > *2014-11-26 17:24:06,645 INFO ActionEndXCommand:543 -
>> > > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
>> > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
>> > > > JOB[0000020-141126162609456-oozie-oozi-W]
>> > > > ACTION[0000020-141126162609456-oozie-oozi-W@shell-node] ERROR is
>> > > > considered
>> > > > as FAILED for SLA*
>> > > > *2014-11-26 17:24:06,702 INFO ActionStartXCommand:543 -
>> > > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
>> > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
>> > > > JOB[0000020-141126162609456-oozie-oozi-W]
>> > > > ACTION[0000020-141126162609456-oozie-oozi-W@fail] Start action
>> > > > [0000020-141126162609456-oozie-*
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > *Workflow*
>> > > >
>> > > > <!--
>> > > > Licensed to the Apache Software Foundation (ASF) under one
>> > > > or more contributor license agreements. See the NOTICE file
>> > > > distributed with this work for additional information
>> > > > regarding copyright ownership. The ASF licenses this file
>> > > > to you under the Apache License, Version 2.0 (the
>> > > > "License"); you may not use this file except in compliance
>> > > > with the License. You may obtain a copy of the License at
>> > > >
>> > > > http://www.apache.org/licenses/LICENSE-2.0
>> > > >
>> > > > Unless required by applicable law or agreed to in writing,
>> software
>> > > > distributed under the License is distributed on an "AS IS" BASIS,
>> > > > WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>> > > implied.
>> > > > See the License for the specific language governing permissions
>> and
>> > > > limitations under the License.
>> > > > -->
>> > > > <workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf">
>> > > > <start to="shell-node"/>
>> > > > <action name="shell-node">
>> > > > <shell xmlns="uri:oozie:shell-action:0.2">
>> > > > <job-tracker>${jobTracker}</job-tracker>
>> > > > <name-node>${nameNode}</name-node>
>> > > > <configuration>
>> > > > <property>
>> > > > <name>mapred.job.queue.name</name>
>> > > > <value>${queueName}</value>
>> > > > </property>
>> > > > </configuration>
>> > > > <exec>ingest.sh</exec>
>> > > > <argument>${feedInstancePaths}</argument>
>> > > > <file>${wf:appPath()}/ingest.sh#ingest.sh</file>
>> > > > <!-- <file>/tmp/ingest.sh#ingest.sh</file> -->
>> > > > <!-- <capture-output/> -->
>> > > > </shell>
>> > > > <ok to="end"/>
>> > > > <error to="fail"/>
>> > > > </action>
>> > > > <kill name="fail">
>> > > > <message>Shell action failed, error
>> > > > message[${wf:errorMessage(wf:lastErrorNode())}]</message>
>> > > > </kill>
>> > > > <end name="end"/>
>> > > > </workflow-app>
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > This is the shell script
>> > > >
>> > > >
>> > > >
>> > > > #!/bin/bash
>> > > >
>> > > >
>> > > > java -cp
>> > > >
>> > >
>> "/media/sf_ToSandbox/BigData-0.0.1-SNAPSHOT.jar:/usr/hdp/2.2.0.0-913/sqoop/lib/*:/usr/hdp/2.2.0.0-913/hadoop/lib/*"
>> > > > com.bigdata.falcon.StaticDataProvider
>> > > >
>> > > >
>> > > >
>> > > > These are the feed and process and cluster information
>> > > >
>> > > >
>> > > >
>> > > > <feed description="Raw customer email feed" name="FINALrawEmailFeed"
>> > > > xmlns="uri:falcon:feed:0.1">
>> > > >
>> > > >
>> <tags>externalSystem=USWestEmailServers,classification=secure</tags>
>> > > > <groups>churnAnalysisDataPipeline</groups>
>> > > > <frequency>minutes(3)</frequency>
>> > > > <late-arrival cut-off="hours(4)"/>
>> > > > <clusters>
>> > > > <cluster name="SprimaryCluster" type="source">
>> > > > <validity start="2014-11-21T00:00Z"
>> end="2016-03-31T00:00Z"/>
>> > > > <retention limit="days(90)" action="delete"/>
>> > > > </cluster>
>> > > > </clusters>
>> > > >
>> > > > <locations>
>> > > > <location type="data"
>> > > >
>> > > >
>> > > >
>> > >
>> path="/user/gaurav123/falcon/demo/primary/input/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
>> > > > <location type="stats" path="/none"/>
>> > > > <location type="meta" path="/none"/>
>> > > > </locations>
>> > > >
>> > > > <ACL owner="ambari-qa" group="users" permission="0777"/>
>> > > > <schema location="/none" provider="none"/>
>> > > > </feed>
>> > > >
>> > > >
>> > > >
>> > > > *Process*
>> > > >
>> > > > *-----------*
>> > > >
>> > > >
>> > > >
>> > > > <?xml version="1.0" encoding="UTF-8"?>
>> > > > <!--
>> > > > Hourly process, ingests raw emails from all US west email servers
>> > > > -->
>> > > > <process name="IngestProcess" xmlns="uri:falcon:process:0.1">
>> > > >
>> > > >
>> > > >
>> > > >
>> > >
>> <tags>pipeline=churnAnalysisDataPipeline,owner=ETLGroup,externalSystem=USWestEmailServers</tags>
>> > > >
>> > > >
>> > > > <clusters>
>> > > > <cluster name="SprimaryCluster">
>> > > > <validity start="2014-11-24T09:35Z"
>> end="2014-11-25T10:00Z"/>
>> > > > </cluster>
>> > > > </clusters>
>> > > >
>> > > > <parallel>1</parallel>
>> > > > <order>FIFO</order>
>> > > > <frequency>minutes(3)</frequency>
>> > > >
>> > > > <outputs>
>> > > > <output name="output" feed="FINALrawEmailFeed"
>> > > instance="now(0,0)"
>> > > > />
>> > > > </outputs>
>> > > >
>> > > > <workflow name="IngestWorkflow" version="2.0.0"
>> > > > engine="oozie"
>> path="/user/gaurav123/falcon/demo/apps/ingest/fs" />
>> > > >
>> > > > <retry policy="periodic" delay="minutes(15)" attempts="3" />
>> > > >
>> > > > </process>
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > Can you please let me know what is the issue , I have tried since
>> long
>> > > two
>> > > > days? but unable to run the workflow with falcon process
>> > > >
>> > > > Also got this exception
>> > > >
>> > > > 2014-11-26 17:47:42,548 ERROR CompletedActionXCommand:540 -
>> > > > SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[] APP[-]
>> > > > JOB[0000099-141126162609456-oozie-oozi-W]
>> > > > ACTION[0000099-141126162609456-oozie-oozi-W@user-action]
>> XException,
>> > > > org.apache.oozie.command.CommandException: E0800: Action it is not
>> > > > running its in [PREP] state, action
>> > > > [0000099-141126162609456-oozie-oozi-W@user-action]
>> > > >
>> > >
>> > > --
>> > > _____________________________________________________________
>> > > The information contained in this communication is intended solely
>> for the
>> > > use of the individual or entity to whom it is addressed and others
>> > > authorized to receive it. It may contain confidential or legally
>> privileged
>> > > information. If you are not the intended recipient you are hereby
>> notified
>> > > that any disclosure, copying, distribution or taking any action in
>> reliance
>> > > on the contents of this information is strictly prohibited and may be
>> > > unlawful. If you have received this communication in error, please
>> notify
>> > > us immediately by responding to this email and then delete it from
>> your
>> > > system. The firm is neither liable for the proper and complete
>> transmission
>> > > of the information contained in this communication nor for any delay
>> in its
>> > > receipt.
>> > >
>>
>>
>
>
Re: Need help for the problem that i faced
Posted by Gaurav Agarwal <gz...@gmail.com>.
Srikanth
What do we mean by that in oozie context?
Thanks
On Thu, Nov 27, 2014 at 11:36 AM, Srikanth Sundarrajan <sr...@hotmail.com>
wrote:
> Looks like the script ingest.sh is returning non-zero exit code.
>
> Regards
> Srikanth Sundarrajan
>
> > Date: Thu, 27 Nov 2014 11:31:30 +0530
> > Subject: Re: Need help for the problem that i faced
> > From: gzagarwal@gmail.com
> > To: dev@falcon.incubator.apache.org
> >
> > These are the log
> >
> > 2014-11-27 05:37:02,726 INFO ActionStartXCommand:543 - SERVER[
> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > JOB[0000129-141126162609456-oozie-oozi-W]
> > ACTION[0000129-141126162609456-oozie-oozi-W@:start:] Start action
> > [0000129-141126162609456-oozie-oozi-W@:start:] with user-retry state :
> > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > JOB[0000129-141126162609456-oozie-oozi-W]
> > ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
> > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action status=DONE
> > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > JOB[0000129-141126162609456-oozie-oozi-W]
> > ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
> > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action updated in
> DB!
> > 2014-11-27 05:37:03,151 INFO ActionStartXCommand:543 - SERVER[
> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > JOB[0000129-141126162609456-oozie-oozi-W]
> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Start action
> > [0000129-141126162609456-oozie-oozi-W@shell-node] with user-retry state
> :
> > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> > 2014-11-27 05:37:05,377 INFO ShellActionExecutor:543 - SERVER[
> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > JOB[0000129-141126162609456-oozie-oozi-W]
> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] checking action,
> > hadoop job ID [job_1417019189193_0117] status [RUNNING]
> > 2014-11-27 05:37:05,378 INFO ActionStartXCommand:543 - SERVER[
> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > JOB[0000129-141126162609456-oozie-oozi-W]
> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
> > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action
> > status=RUNNING
> > 2014-11-27 05:37:05,379 INFO ActionStartXCommand:543 - SERVER[
> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > JOB[0000129-141126162609456-oozie-oozi-W]
> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
> > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action updated in
> > DB!
> > 2014-11-27 05:37:49,260 INFO CallbackServlet:543 - SERVER[
> > sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-]
> > JOB[0000129-141126162609456-oozie-oozi-W]
> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] callback for
> action
> > [0000129-141126162609456-oozie-oozi-W@shell-node]
> > 2014-11-27 05:37:50,753 INFO ShellActionExecutor:543 - SERVER[
> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > JOB[0000129-141126162609456-oozie-oozi-W]
> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] action
> completed,
> > external ID [job_1417019189193_0117]
> > 2014-11-27 05:37:50,761 WARN ShellActionExecutor:546 - SERVER[
> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > JOB[0000129-141126162609456-oozie-oozi-W]
> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Launcher ERROR,
> > reason: Main class [org.apache.oozie.action.hadoop.ShellMain], exit code
> > [1]
> > 2014-11-27 05:37:50,865 INFO ActionEndXCommand:543 - SERVER[
> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > JOB[0000129-141126162609456-oozie-oozi-W]
> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] ERROR is
> considered
> > as FAILED for SLA
> > 2014-11-27 05:37:50,941 INFO ActionStartXCommand:543 - SERVER[
> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > JOB[0000129-141126162609456-oozie-oozi-W]
> > ACTION[0000129-141126162609456-oozie-oozi-W@fail] Start action
> > [0000129-141126162609456-oozie-oozi-W@fail] with user-retry state :
> > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > JOB[0000129-141126162609456-oozie-oozi-W]
> > ACTION[0000129-141126162609456-oozie-oozi-W@fail]
> > [***0000129-141126162609456-oozie-oozi-W@fail***]Action status=DONE
> > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > JOB[0000129-141126162609456-oozie-oozi-W]
> > ACTION[0000129-141126162609456-oozie-oozi-W@fail]
> > [***0000129-141126162609456-oozie-oozi-W@fail***]Action updated in DB!
> >
> > On Thu, Nov 27, 2014 at 10:25 AM, Shwetha GS <sh...@inmobi.com>
> wrote:
> >
> > > <exec>ingest.sh</exec>
> > > Try using absolute path.
> > >
> > > None of the logs imply any error. Can you attach all the oozie logs
> for the
> > > workflow?
> > >
> > >
> > >
> > >
> > > On Thu, Nov 27, 2014 at 7:50 AM, Gaurav Agarwal <gz...@gmail.com>
> > > wrote:
> > >
> > > > Hello I am running apache falcon and oozie with shell script to run
> the
> > > > workflow.
> > > >
> > > >
> > > >
> > > > getting this error
> > > >
> > > > *ell-node] Launcher ERROR, reason: Main class
> > > > [org.apache.oozie.action.hadoop.ShellMain], exit code [1]*
> > > > *2014-11-26 17:24:06,645 INFO ActionEndXCommand:543 -
> > > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
> > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > > > JOB[0000020-141126162609456-oozie-oozi-W]
> > > > ACTION[0000020-141126162609456-oozie-oozi-W@shell-node] ERROR is
> > > > considered
> > > > as FAILED for SLA*
> > > > *2014-11-26 17:24:06,702 INFO ActionStartXCommand:543 -
> > > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
> > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > > > JOB[0000020-141126162609456-oozie-oozi-W]
> > > > ACTION[0000020-141126162609456-oozie-oozi-W@fail] Start action
> > > > [0000020-141126162609456-oozie-*
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > *Workflow*
> > > >
> > > > <!--
> > > > Licensed to the Apache Software Foundation (ASF) under one
> > > > or more contributor license agreements. See the NOTICE file
> > > > distributed with this work for additional information
> > > > regarding copyright ownership. The ASF licenses this file
> > > > to you under the Apache License, Version 2.0 (the
> > > > "License"); you may not use this file except in compliance
> > > > with the License. You may obtain a copy of the License at
> > > >
> > > > http://www.apache.org/licenses/LICENSE-2.0
> > > >
> > > > Unless required by applicable law or agreed to in writing, software
> > > > distributed under the License is distributed on an "AS IS" BASIS,
> > > > WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> > > implied.
> > > > See the License for the specific language governing permissions and
> > > > limitations under the License.
> > > > -->
> > > > <workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf">
> > > > <start to="shell-node"/>
> > > > <action name="shell-node">
> > > > <shell xmlns="uri:oozie:shell-action:0.2">
> > > > <job-tracker>${jobTracker}</job-tracker>
> > > > <name-node>${nameNode}</name-node>
> > > > <configuration>
> > > > <property>
> > > > <name>mapred.job.queue.name</name>
> > > > <value>${queueName}</value>
> > > > </property>
> > > > </configuration>
> > > > <exec>ingest.sh</exec>
> > > > <argument>${feedInstancePaths}</argument>
> > > > <file>${wf:appPath()}/ingest.sh#ingest.sh</file>
> > > > <!-- <file>/tmp/ingest.sh#ingest.sh</file> -->
> > > > <!-- <capture-output/> -->
> > > > </shell>
> > > > <ok to="end"/>
> > > > <error to="fail"/>
> > > > </action>
> > > > <kill name="fail">
> > > > <message>Shell action failed, error
> > > > message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> > > > </kill>
> > > > <end name="end"/>
> > > > </workflow-app>
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > This is the shell script
> > > >
> > > >
> > > >
> > > > #!/bin/bash
> > > >
> > > >
> > > > java -cp
> > > >
> > >
> "/media/sf_ToSandbox/BigData-0.0.1-SNAPSHOT.jar:/usr/hdp/2.2.0.0-913/sqoop/lib/*:/usr/hdp/2.2.0.0-913/hadoop/lib/*"
> > > > com.bigdata.falcon.StaticDataProvider
> > > >
> > > >
> > > >
> > > > These are the feed and process and cluster information
> > > >
> > > >
> > > >
> > > > <feed description="Raw customer email feed" name="FINALrawEmailFeed"
> > > > xmlns="uri:falcon:feed:0.1">
> > > >
> > > >
> <tags>externalSystem=USWestEmailServers,classification=secure</tags>
> > > > <groups>churnAnalysisDataPipeline</groups>
> > > > <frequency>minutes(3)</frequency>
> > > > <late-arrival cut-off="hours(4)"/>
> > > > <clusters>
> > > > <cluster name="SprimaryCluster" type="source">
> > > > <validity start="2014-11-21T00:00Z"
> end="2016-03-31T00:00Z"/>
> > > > <retention limit="days(90)" action="delete"/>
> > > > </cluster>
> > > > </clusters>
> > > >
> > > > <locations>
> > > > <location type="data"
> > > >
> > > >
> > > >
> > >
> path="/user/gaurav123/falcon/demo/primary/input/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
> > > > <location type="stats" path="/none"/>
> > > > <location type="meta" path="/none"/>
> > > > </locations>
> > > >
> > > > <ACL owner="ambari-qa" group="users" permission="0777"/>
> > > > <schema location="/none" provider="none"/>
> > > > </feed>
> > > >
> > > >
> > > >
> > > > *Process*
> > > >
> > > > *-----------*
> > > >
> > > >
> > > >
> > > > <?xml version="1.0" encoding="UTF-8"?>
> > > > <!--
> > > > Hourly process, ingests raw emails from all US west email servers
> > > > -->
> > > > <process name="IngestProcess" xmlns="uri:falcon:process:0.1">
> > > >
> > > >
> > > >
> > > >
> > >
> <tags>pipeline=churnAnalysisDataPipeline,owner=ETLGroup,externalSystem=USWestEmailServers</tags>
> > > >
> > > >
> > > > <clusters>
> > > > <cluster name="SprimaryCluster">
> > > > <validity start="2014-11-24T09:35Z"
> end="2014-11-25T10:00Z"/>
> > > > </cluster>
> > > > </clusters>
> > > >
> > > > <parallel>1</parallel>
> > > > <order>FIFO</order>
> > > > <frequency>minutes(3)</frequency>
> > > >
> > > > <outputs>
> > > > <output name="output" feed="FINALrawEmailFeed"
> > > instance="now(0,0)"
> > > > />
> > > > </outputs>
> > > >
> > > > <workflow name="IngestWorkflow" version="2.0.0"
> > > > engine="oozie" path="/user/gaurav123/falcon/demo/apps/ingest/fs"
> />
> > > >
> > > > <retry policy="periodic" delay="minutes(15)" attempts="3" />
> > > >
> > > > </process>
> > > >
> > > >
> > > >
> > > >
> > > > Can you please let me know what is the issue , I have tried since
> long
> > > two
> > > > days? but unable to run the workflow with falcon process
> > > >
> > > > Also got this exception
> > > >
> > > > 2014-11-26 17:47:42,548 ERROR CompletedActionXCommand:540 -
> > > > SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[] APP[-]
> > > > JOB[0000099-141126162609456-oozie-oozi-W]
> > > > ACTION[0000099-141126162609456-oozie-oozi-W@user-action] XException,
> > > > org.apache.oozie.command.CommandException: E0800: Action it is not
> > > > running its in [PREP] state, action
> > > > [0000099-141126162609456-oozie-oozi-W@user-action]
> > > >
> > >
> > > --
> > > _____________________________________________________________
> > > The information contained in this communication is intended solely for
> the
> > > use of the individual or entity to whom it is addressed and others
> > > authorized to receive it. It may contain confidential or legally
> privileged
> > > information. If you are not the intended recipient you are hereby
> notified
> > > that any disclosure, copying, distribution or taking any action in
> reliance
> > > on the contents of this information is strictly prohibited and may be
> > > unlawful. If you have received this communication in error, please
> notify
> > > us immediately by responding to this email and then delete it from your
> > > system. The firm is neither liable for the proper and complete
> transmission
> > > of the information contained in this communication nor for any delay
> in its
> > > receipt.
> > >
>
>
RE: Need help for the problem that i faced
Posted by Srikanth Sundarrajan <sr...@hotmail.com>.
Looks like the script ingest.sh is returning non-zero exit code.
Regards
Srikanth Sundarrajan
> Date: Thu, 27 Nov 2014 11:31:30 +0530
> Subject: Re: Need help for the problem that i faced
> From: gzagarwal@gmail.com
> To: dev@falcon.incubator.apache.org
>
> These are the log
>
> 2014-11-27 05:37:02,726 INFO ActionStartXCommand:543 - SERVER[
> sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000129-141126162609456-oozie-oozi-W]
> ACTION[0000129-141126162609456-oozie-oozi-W@:start:] Start action
> [0000129-141126162609456-oozie-oozi-W@:start:] with user-retry state :
> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
> sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000129-141126162609456-oozie-oozi-W]
> ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
> [***0000129-141126162609456-oozie-oozi-W@:start:***]Action status=DONE
> 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
> sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000129-141126162609456-oozie-oozi-W]
> ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
> [***0000129-141126162609456-oozie-oozi-W@:start:***]Action updated in DB!
> 2014-11-27 05:37:03,151 INFO ActionStartXCommand:543 - SERVER[
> sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000129-141126162609456-oozie-oozi-W]
> ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Start action
> [0000129-141126162609456-oozie-oozi-W@shell-node] with user-retry state :
> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> 2014-11-27 05:37:05,377 INFO ShellActionExecutor:543 - SERVER[
> sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000129-141126162609456-oozie-oozi-W]
> ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] checking action,
> hadoop job ID [job_1417019189193_0117] status [RUNNING]
> 2014-11-27 05:37:05,378 INFO ActionStartXCommand:543 - SERVER[
> sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000129-141126162609456-oozie-oozi-W]
> ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
> [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action
> status=RUNNING
> 2014-11-27 05:37:05,379 INFO ActionStartXCommand:543 - SERVER[
> sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000129-141126162609456-oozie-oozi-W]
> ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
> [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action updated in
> DB!
> 2014-11-27 05:37:49,260 INFO CallbackServlet:543 - SERVER[
> sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-]
> JOB[0000129-141126162609456-oozie-oozi-W]
> ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] callback for action
> [0000129-141126162609456-oozie-oozi-W@shell-node]
> 2014-11-27 05:37:50,753 INFO ShellActionExecutor:543 - SERVER[
> sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000129-141126162609456-oozie-oozi-W]
> ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] action completed,
> external ID [job_1417019189193_0117]
> 2014-11-27 05:37:50,761 WARN ShellActionExecutor:546 - SERVER[
> sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000129-141126162609456-oozie-oozi-W]
> ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Launcher ERROR,
> reason: Main class [org.apache.oozie.action.hadoop.ShellMain], exit code
> [1]
> 2014-11-27 05:37:50,865 INFO ActionEndXCommand:543 - SERVER[
> sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000129-141126162609456-oozie-oozi-W]
> ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] ERROR is considered
> as FAILED for SLA
> 2014-11-27 05:37:50,941 INFO ActionStartXCommand:543 - SERVER[
> sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000129-141126162609456-oozie-oozi-W]
> ACTION[0000129-141126162609456-oozie-oozi-W@fail] Start action
> [0000129-141126162609456-oozie-oozi-W@fail] with user-retry state :
> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
> sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000129-141126162609456-oozie-oozi-W]
> ACTION[0000129-141126162609456-oozie-oozi-W@fail]
> [***0000129-141126162609456-oozie-oozi-W@fail***]Action status=DONE
> 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
> sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000129-141126162609456-oozie-oozi-W]
> ACTION[0000129-141126162609456-oozie-oozi-W@fail]
> [***0000129-141126162609456-oozie-oozi-W@fail***]Action updated in DB!
>
> On Thu, Nov 27, 2014 at 10:25 AM, Shwetha GS <sh...@inmobi.com> wrote:
>
> > <exec>ingest.sh</exec>
> > Try using absolute path.
> >
> > None of the logs imply any error. Can you attach all the oozie logs for the
> > workflow?
> >
> >
> >
> >
> > On Thu, Nov 27, 2014 at 7:50 AM, Gaurav Agarwal <gz...@gmail.com>
> > wrote:
> >
> > > Hello I am running apache falcon and oozie with shell script to run the
> > > workflow.
> > >
> > >
> > >
> > > getting this error
> > >
> > > *ell-node] Launcher ERROR, reason: Main class
> > > [org.apache.oozie.action.hadoop.ShellMain], exit code [1]*
> > > *2014-11-26 17:24:06,645 INFO ActionEndXCommand:543 -
> > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
> > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > > JOB[0000020-141126162609456-oozie-oozi-W]
> > > ACTION[0000020-141126162609456-oozie-oozi-W@shell-node] ERROR is
> > > considered
> > > as FAILED for SLA*
> > > *2014-11-26 17:24:06,702 INFO ActionStartXCommand:543 -
> > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
> > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > > JOB[0000020-141126162609456-oozie-oozi-W]
> > > ACTION[0000020-141126162609456-oozie-oozi-W@fail] Start action
> > > [0000020-141126162609456-oozie-*
> > >
> > >
> > >
> > >
> > >
> > > *Workflow*
> > >
> > > <!--
> > > Licensed to the Apache Software Foundation (ASF) under one
> > > or more contributor license agreements. See the NOTICE file
> > > distributed with this work for additional information
> > > regarding copyright ownership. The ASF licenses this file
> > > to you under the Apache License, Version 2.0 (the
> > > "License"); you may not use this file except in compliance
> > > with the License. You may obtain a copy of the License at
> > >
> > > http://www.apache.org/licenses/LICENSE-2.0
> > >
> > > Unless required by applicable law or agreed to in writing, software
> > > distributed under the License is distributed on an "AS IS" BASIS,
> > > WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> > implied.
> > > See the License for the specific language governing permissions and
> > > limitations under the License.
> > > -->
> > > <workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf">
> > > <start to="shell-node"/>
> > > <action name="shell-node">
> > > <shell xmlns="uri:oozie:shell-action:0.2">
> > > <job-tracker>${jobTracker}</job-tracker>
> > > <name-node>${nameNode}</name-node>
> > > <configuration>
> > > <property>
> > > <name>mapred.job.queue.name</name>
> > > <value>${queueName}</value>
> > > </property>
> > > </configuration>
> > > <exec>ingest.sh</exec>
> > > <argument>${feedInstancePaths}</argument>
> > > <file>${wf:appPath()}/ingest.sh#ingest.sh</file>
> > > <!-- <file>/tmp/ingest.sh#ingest.sh</file> -->
> > > <!-- <capture-output/> -->
> > > </shell>
> > > <ok to="end"/>
> > > <error to="fail"/>
> > > </action>
> > > <kill name="fail">
> > > <message>Shell action failed, error
> > > message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> > > </kill>
> > > <end name="end"/>
> > > </workflow-app>
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > This is the shell script
> > >
> > >
> > >
> > > #!/bin/bash
> > >
> > >
> > > java -cp
> > >
> > "/media/sf_ToSandbox/BigData-0.0.1-SNAPSHOT.jar:/usr/hdp/2.2.0.0-913/sqoop/lib/*:/usr/hdp/2.2.0.0-913/hadoop/lib/*"
> > > com.bigdata.falcon.StaticDataProvider
> > >
> > >
> > >
> > > These are the feed and process and cluster information
> > >
> > >
> > >
> > > <feed description="Raw customer email feed" name="FINALrawEmailFeed"
> > > xmlns="uri:falcon:feed:0.1">
> > >
> > > <tags>externalSystem=USWestEmailServers,classification=secure</tags>
> > > <groups>churnAnalysisDataPipeline</groups>
> > > <frequency>minutes(3)</frequency>
> > > <late-arrival cut-off="hours(4)"/>
> > > <clusters>
> > > <cluster name="SprimaryCluster" type="source">
> > > <validity start="2014-11-21T00:00Z" end="2016-03-31T00:00Z"/>
> > > <retention limit="days(90)" action="delete"/>
> > > </cluster>
> > > </clusters>
> > >
> > > <locations>
> > > <location type="data"
> > >
> > >
> > >
> > path="/user/gaurav123/falcon/demo/primary/input/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
> > > <location type="stats" path="/none"/>
> > > <location type="meta" path="/none"/>
> > > </locations>
> > >
> > > <ACL owner="ambari-qa" group="users" permission="0777"/>
> > > <schema location="/none" provider="none"/>
> > > </feed>
> > >
> > >
> > >
> > > *Process*
> > >
> > > *-----------*
> > >
> > >
> > >
> > > <?xml version="1.0" encoding="UTF-8"?>
> > > <!--
> > > Hourly process, ingests raw emails from all US west email servers
> > > -->
> > > <process name="IngestProcess" xmlns="uri:falcon:process:0.1">
> > >
> > >
> > >
> > >
> > <tags>pipeline=churnAnalysisDataPipeline,owner=ETLGroup,externalSystem=USWestEmailServers</tags>
> > >
> > >
> > > <clusters>
> > > <cluster name="SprimaryCluster">
> > > <validity start="2014-11-24T09:35Z" end="2014-11-25T10:00Z"/>
> > > </cluster>
> > > </clusters>
> > >
> > > <parallel>1</parallel>
> > > <order>FIFO</order>
> > > <frequency>minutes(3)</frequency>
> > >
> > > <outputs>
> > > <output name="output" feed="FINALrawEmailFeed"
> > instance="now(0,0)"
> > > />
> > > </outputs>
> > >
> > > <workflow name="IngestWorkflow" version="2.0.0"
> > > engine="oozie" path="/user/gaurav123/falcon/demo/apps/ingest/fs" />
> > >
> > > <retry policy="periodic" delay="minutes(15)" attempts="3" />
> > >
> > > </process>
> > >
> > >
> > >
> > >
> > > Can you please let me know what is the issue , I have tried since long
> > two
> > > days? but unable to run the workflow with falcon process
> > >
> > > Also got this exception
> > >
> > > 2014-11-26 17:47:42,548 ERROR CompletedActionXCommand:540 -
> > > SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[] APP[-]
> > > JOB[0000099-141126162609456-oozie-oozi-W]
> > > ACTION[0000099-141126162609456-oozie-oozi-W@user-action] XException,
> > > org.apache.oozie.command.CommandException: E0800: Action it is not
> > > running its in [PREP] state, action
> > > [0000099-141126162609456-oozie-oozi-W@user-action]
> > >
> >
> > --
> > _____________________________________________________________
> > The information contained in this communication is intended solely for the
> > use of the individual or entity to whom it is addressed and others
> > authorized to receive it. It may contain confidential or legally privileged
> > information. If you are not the intended recipient you are hereby notified
> > that any disclosure, copying, distribution or taking any action in reliance
> > on the contents of this information is strictly prohibited and may be
> > unlawful. If you have received this communication in error, please notify
> > us immediately by responding to this email and then delete it from your
> > system. The firm is neither liable for the proper and complete transmission
> > of the information contained in this communication nor for any delay in its
> > receipt.
> >
Re: Need help for the problem that i faced
Posted by Gaurav Agarwal <gz...@gmail.com>.
These are the log
2014-11-27 05:37:02,726 INFO ActionStartXCommand:543 - SERVER[
sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
JOB[0000129-141126162609456-oozie-oozi-W]
ACTION[0000129-141126162609456-oozie-oozi-W@:start:] Start action
[0000129-141126162609456-oozie-oozi-W@:start:] with user-retry state :
userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
JOB[0000129-141126162609456-oozie-oozi-W]
ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
[***0000129-141126162609456-oozie-oozi-W@:start:***]Action status=DONE
2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
JOB[0000129-141126162609456-oozie-oozi-W]
ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
[***0000129-141126162609456-oozie-oozi-W@:start:***]Action updated in DB!
2014-11-27 05:37:03,151 INFO ActionStartXCommand:543 - SERVER[
sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
JOB[0000129-141126162609456-oozie-oozi-W]
ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Start action
[0000129-141126162609456-oozie-oozi-W@shell-node] with user-retry state :
userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2014-11-27 05:37:05,377 INFO ShellActionExecutor:543 - SERVER[
sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
JOB[0000129-141126162609456-oozie-oozi-W]
ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] checking action,
hadoop job ID [job_1417019189193_0117] status [RUNNING]
2014-11-27 05:37:05,378 INFO ActionStartXCommand:543 - SERVER[
sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
JOB[0000129-141126162609456-oozie-oozi-W]
ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
[***0000129-141126162609456-oozie-oozi-W@shell-node***]Action
status=RUNNING
2014-11-27 05:37:05,379 INFO ActionStartXCommand:543 - SERVER[
sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
JOB[0000129-141126162609456-oozie-oozi-W]
ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
[***0000129-141126162609456-oozie-oozi-W@shell-node***]Action updated in
DB!
2014-11-27 05:37:49,260 INFO CallbackServlet:543 - SERVER[
sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-]
JOB[0000129-141126162609456-oozie-oozi-W]
ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] callback for action
[0000129-141126162609456-oozie-oozi-W@shell-node]
2014-11-27 05:37:50,753 INFO ShellActionExecutor:543 - SERVER[
sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
JOB[0000129-141126162609456-oozie-oozi-W]
ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] action completed,
external ID [job_1417019189193_0117]
2014-11-27 05:37:50,761 WARN ShellActionExecutor:546 - SERVER[
sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
JOB[0000129-141126162609456-oozie-oozi-W]
ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Launcher ERROR,
reason: Main class [org.apache.oozie.action.hadoop.ShellMain], exit code
[1]
2014-11-27 05:37:50,865 INFO ActionEndXCommand:543 - SERVER[
sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
JOB[0000129-141126162609456-oozie-oozi-W]
ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] ERROR is considered
as FAILED for SLA
2014-11-27 05:37:50,941 INFO ActionStartXCommand:543 - SERVER[
sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
JOB[0000129-141126162609456-oozie-oozi-W]
ACTION[0000129-141126162609456-oozie-oozi-W@fail] Start action
[0000129-141126162609456-oozie-oozi-W@fail] with user-retry state :
userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
JOB[0000129-141126162609456-oozie-oozi-W]
ACTION[0000129-141126162609456-oozie-oozi-W@fail]
[***0000129-141126162609456-oozie-oozi-W@fail***]Action status=DONE
2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
JOB[0000129-141126162609456-oozie-oozi-W]
ACTION[0000129-141126162609456-oozie-oozi-W@fail]
[***0000129-141126162609456-oozie-oozi-W@fail***]Action updated in DB!
On Thu, Nov 27, 2014 at 10:25 AM, Shwetha GS <sh...@inmobi.com> wrote:
> <exec>ingest.sh</exec>
> Try using absolute path.
>
> None of the logs imply any error. Can you attach all the oozie logs for the
> workflow?
>
>
>
>
> On Thu, Nov 27, 2014 at 7:50 AM, Gaurav Agarwal <gz...@gmail.com>
> wrote:
>
> > Hello I am running apache falcon and oozie with shell script to run the
> > workflow.
> >
> >
> >
> > getting this error
> >
> > *ell-node] Launcher ERROR, reason: Main class
> > [org.apache.oozie.action.hadoop.ShellMain], exit code [1]*
> > *2014-11-26 17:24:06,645 INFO ActionEndXCommand:543 -
> > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
> > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > JOB[0000020-141126162609456-oozie-oozi-W]
> > ACTION[0000020-141126162609456-oozie-oozi-W@shell-node] ERROR is
> > considered
> > as FAILED for SLA*
> > *2014-11-26 17:24:06,702 INFO ActionStartXCommand:543 -
> > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
> > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > JOB[0000020-141126162609456-oozie-oozi-W]
> > ACTION[0000020-141126162609456-oozie-oozi-W@fail] Start action
> > [0000020-141126162609456-oozie-*
> >
> >
> >
> >
> >
> > *Workflow*
> >
> > <!--
> > Licensed to the Apache Software Foundation (ASF) under one
> > or more contributor license agreements. See the NOTICE file
> > distributed with this work for additional information
> > regarding copyright ownership. The ASF licenses this file
> > to you under the Apache License, Version 2.0 (the
> > "License"); you may not use this file except in compliance
> > with the License. You may obtain a copy of the License at
> >
> > http://www.apache.org/licenses/LICENSE-2.0
> >
> > Unless required by applicable law or agreed to in writing, software
> > distributed under the License is distributed on an "AS IS" BASIS,
> > WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> > See the License for the specific language governing permissions and
> > limitations under the License.
> > -->
> > <workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf">
> > <start to="shell-node"/>
> > <action name="shell-node">
> > <shell xmlns="uri:oozie:shell-action:0.2">
> > <job-tracker>${jobTracker}</job-tracker>
> > <name-node>${nameNode}</name-node>
> > <configuration>
> > <property>
> > <name>mapred.job.queue.name</name>
> > <value>${queueName}</value>
> > </property>
> > </configuration>
> > <exec>ingest.sh</exec>
> > <argument>${feedInstancePaths}</argument>
> > <file>${wf:appPath()}/ingest.sh#ingest.sh</file>
> > <!-- <file>/tmp/ingest.sh#ingest.sh</file> -->
> > <!-- <capture-output/> -->
> > </shell>
> > <ok to="end"/>
> > <error to="fail"/>
> > </action>
> > <kill name="fail">
> > <message>Shell action failed, error
> > message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> > </kill>
> > <end name="end"/>
> > </workflow-app>
> >
> >
> >
> >
> >
> >
> >
> > This is the shell script
> >
> >
> >
> > #!/bin/bash
> >
> >
> > java -cp
> >
> "/media/sf_ToSandbox/BigData-0.0.1-SNAPSHOT.jar:/usr/hdp/2.2.0.0-913/sqoop/lib/*:/usr/hdp/2.2.0.0-913/hadoop/lib/*"
> > com.bigdata.falcon.StaticDataProvider
> >
> >
> >
> > These are the feed and process and cluster information
> >
> >
> >
> > <feed description="Raw customer email feed" name="FINALrawEmailFeed"
> > xmlns="uri:falcon:feed:0.1">
> >
> > <tags>externalSystem=USWestEmailServers,classification=secure</tags>
> > <groups>churnAnalysisDataPipeline</groups>
> > <frequency>minutes(3)</frequency>
> > <late-arrival cut-off="hours(4)"/>
> > <clusters>
> > <cluster name="SprimaryCluster" type="source">
> > <validity start="2014-11-21T00:00Z" end="2016-03-31T00:00Z"/>
> > <retention limit="days(90)" action="delete"/>
> > </cluster>
> > </clusters>
> >
> > <locations>
> > <location type="data"
> >
> >
> >
> path="/user/gaurav123/falcon/demo/primary/input/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
> > <location type="stats" path="/none"/>
> > <location type="meta" path="/none"/>
> > </locations>
> >
> > <ACL owner="ambari-qa" group="users" permission="0777"/>
> > <schema location="/none" provider="none"/>
> > </feed>
> >
> >
> >
> > *Process*
> >
> > *-----------*
> >
> >
> >
> > <?xml version="1.0" encoding="UTF-8"?>
> > <!--
> > Hourly process, ingests raw emails from all US west email servers
> > -->
> > <process name="IngestProcess" xmlns="uri:falcon:process:0.1">
> >
> >
> >
> >
> <tags>pipeline=churnAnalysisDataPipeline,owner=ETLGroup,externalSystem=USWestEmailServers</tags>
> >
> >
> > <clusters>
> > <cluster name="SprimaryCluster">
> > <validity start="2014-11-24T09:35Z" end="2014-11-25T10:00Z"/>
> > </cluster>
> > </clusters>
> >
> > <parallel>1</parallel>
> > <order>FIFO</order>
> > <frequency>minutes(3)</frequency>
> >
> > <outputs>
> > <output name="output" feed="FINALrawEmailFeed"
> instance="now(0,0)"
> > />
> > </outputs>
> >
> > <workflow name="IngestWorkflow" version="2.0.0"
> > engine="oozie" path="/user/gaurav123/falcon/demo/apps/ingest/fs" />
> >
> > <retry policy="periodic" delay="minutes(15)" attempts="3" />
> >
> > </process>
> >
> >
> >
> >
> > Can you please let me know what is the issue , I have tried since long
> two
> > days? but unable to run the workflow with falcon process
> >
> > Also got this exception
> >
> > 2014-11-26 17:47:42,548 ERROR CompletedActionXCommand:540 -
> > SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[] APP[-]
> > JOB[0000099-141126162609456-oozie-oozi-W]
> > ACTION[0000099-141126162609456-oozie-oozi-W@user-action] XException,
> > org.apache.oozie.command.CommandException: E0800: Action it is not
> > running its in [PREP] state, action
> > [0000099-141126162609456-oozie-oozi-W@user-action]
> >
>
> --
> _____________________________________________________________
> The information contained in this communication is intended solely for the
> use of the individual or entity to whom it is addressed and others
> authorized to receive it. It may contain confidential or legally privileged
> information. If you are not the intended recipient you are hereby notified
> that any disclosure, copying, distribution or taking any action in reliance
> on the contents of this information is strictly prohibited and may be
> unlawful. If you have received this communication in error, please notify
> us immediately by responding to this email and then delete it from your
> system. The firm is neither liable for the proper and complete transmission
> of the information contained in this communication nor for any delay in its
> receipt.
>
Re: Need help for the problem that i faced
Posted by Shwetha GS <sh...@inmobi.com>.
<exec>ingest.sh</exec>
Try using absolute path.
None of the logs imply any error. Can you attach all the oozie logs for the
workflow?
On Thu, Nov 27, 2014 at 7:50 AM, Gaurav Agarwal <gz...@gmail.com> wrote:
> Hello I am running apache falcon and oozie with shell script to run the
> workflow.
>
>
>
> getting this error
>
> *ell-node] Launcher ERROR, reason: Main class
> [org.apache.oozie.action.hadoop.ShellMain], exit code [1]*
> *2014-11-26 17:24:06,645 INFO ActionEndXCommand:543 -
> SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
> USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000020-141126162609456-oozie-oozi-W]
> ACTION[0000020-141126162609456-oozie-oozi-W@shell-node] ERROR is
> considered
> as FAILED for SLA*
> *2014-11-26 17:24:06,702 INFO ActionStartXCommand:543 -
> SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
> USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000020-141126162609456-oozie-oozi-W]
> ACTION[0000020-141126162609456-oozie-oozi-W@fail] Start action
> [0000020-141126162609456-oozie-*
>
>
>
>
>
> *Workflow*
>
> <!--
> Licensed to the Apache Software Foundation (ASF) under one
> or more contributor license agreements. See the NOTICE file
> distributed with this work for additional information
> regarding copyright ownership. The ASF licenses this file
> to you under the Apache License, Version 2.0 (the
> "License"); you may not use this file except in compliance
> with the License. You may obtain a copy of the License at
>
> http://www.apache.org/licenses/LICENSE-2.0
>
> Unless required by applicable law or agreed to in writing, software
> distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the License for the specific language governing permissions and
> limitations under the License.
> -->
> <workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf">
> <start to="shell-node"/>
> <action name="shell-node">
> <shell xmlns="uri:oozie:shell-action:0.2">
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
> <configuration>
> <property>
> <name>mapred.job.queue.name</name>
> <value>${queueName}</value>
> </property>
> </configuration>
> <exec>ingest.sh</exec>
> <argument>${feedInstancePaths}</argument>
> <file>${wf:appPath()}/ingest.sh#ingest.sh</file>
> <!-- <file>/tmp/ingest.sh#ingest.sh</file> -->
> <!-- <capture-output/> -->
> </shell>
> <ok to="end"/>
> <error to="fail"/>
> </action>
> <kill name="fail">
> <message>Shell action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> </kill>
> <end name="end"/>
> </workflow-app>
>
>
>
>
>
>
>
> This is the shell script
>
>
>
> #!/bin/bash
>
>
> java -cp
> "/media/sf_ToSandbox/BigData-0.0.1-SNAPSHOT.jar:/usr/hdp/2.2.0.0-913/sqoop/lib/*:/usr/hdp/2.2.0.0-913/hadoop/lib/*"
> com.bigdata.falcon.StaticDataProvider
>
>
>
> These are the feed and process and cluster information
>
>
>
> <feed description="Raw customer email feed" name="FINALrawEmailFeed"
> xmlns="uri:falcon:feed:0.1">
>
> <tags>externalSystem=USWestEmailServers,classification=secure</tags>
> <groups>churnAnalysisDataPipeline</groups>
> <frequency>minutes(3)</frequency>
> <late-arrival cut-off="hours(4)"/>
> <clusters>
> <cluster name="SprimaryCluster" type="source">
> <validity start="2014-11-21T00:00Z" end="2016-03-31T00:00Z"/>
> <retention limit="days(90)" action="delete"/>
> </cluster>
> </clusters>
>
> <locations>
> <location type="data"
>
>
> path="/user/gaurav123/falcon/demo/primary/input/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
> <location type="stats" path="/none"/>
> <location type="meta" path="/none"/>
> </locations>
>
> <ACL owner="ambari-qa" group="users" permission="0777"/>
> <schema location="/none" provider="none"/>
> </feed>
>
>
>
> *Process*
>
> *-----------*
>
>
>
> <?xml version="1.0" encoding="UTF-8"?>
> <!--
> Hourly process, ingests raw emails from all US west email servers
> -->
> <process name="IngestProcess" xmlns="uri:falcon:process:0.1">
>
>
>
> <tags>pipeline=churnAnalysisDataPipeline,owner=ETLGroup,externalSystem=USWestEmailServers</tags>
>
>
> <clusters>
> <cluster name="SprimaryCluster">
> <validity start="2014-11-24T09:35Z" end="2014-11-25T10:00Z"/>
> </cluster>
> </clusters>
>
> <parallel>1</parallel>
> <order>FIFO</order>
> <frequency>minutes(3)</frequency>
>
> <outputs>
> <output name="output" feed="FINALrawEmailFeed" instance="now(0,0)"
> />
> </outputs>
>
> <workflow name="IngestWorkflow" version="2.0.0"
> engine="oozie" path="/user/gaurav123/falcon/demo/apps/ingest/fs" />
>
> <retry policy="periodic" delay="minutes(15)" attempts="3" />
>
> </process>
>
>
>
>
> Can you please let me know what is the issue , I have tried since long two
> days? but unable to run the workflow with falcon process
>
> Also got this exception
>
> 2014-11-26 17:47:42,548 ERROR CompletedActionXCommand:540 -
> SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[] APP[-]
> JOB[0000099-141126162609456-oozie-oozi-W]
> ACTION[0000099-141126162609456-oozie-oozi-W@user-action] XException,
> org.apache.oozie.command.CommandException: E0800: Action it is not
> running its in [PREP] state, action
> [0000099-141126162609456-oozie-oozi-W@user-action]
>
--
_____________________________________________________________
The information contained in this communication is intended solely for the
use of the individual or entity to whom it is addressed and others
authorized to receive it. It may contain confidential or legally privileged
information. If you are not the intended recipient you are hereby notified
that any disclosure, copying, distribution or taking any action in reliance
on the contents of this information is strictly prohibited and may be
unlawful. If you have received this communication in error, please notify
us immediately by responding to this email and then delete it from your
system. The firm is neither liable for the proper and complete transmission
of the information contained in this communication nor for any delay in its
receipt.