You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Something Something <ma...@gmail.com> on 2014/02/14 01:06:33 UTC
Oozie-Sqoop job not working
When I run ‘Sqoop’ from command line it works fine, but when I run it via
Oozie job it just keeps showing the following message:
2014-02-12 19:04:47,854 DEBUG DefinitionXCommand:545 – USER[root] GROUP[-]
TOKEN[] APP[sqoop] JOB[0000004-140211210850200-oozie-oozi-W] ACTION[-]
Execute command [definition] key [0000004-140211210850200-oozie-oozi-W]
My workflow.xml looks like this:
<workflow-app name=”sqoop” xmlns=”uri:oozie:workflow:0.1″>
<start to=”etl” />
<action name=”etl”>
<sqoop xmlns=”uri:oozie:sqoop-action:0.2″>
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<prepare>
<delete path=”${outputBaseDir}/${tableName}/${period}” />
</prepare>
<configuration>
<property>
<name>mapred.compress.map.output</name>
<value>true</value>
</property>
</configuration>
<arg>import</arg>
<arg>–connect</arg>
<arg>jdbc:mysql://${mysqlHost}/${mysqlDb}</arg>
<arg>–username</arg>
<arg>root</arg>
<arg>–target-dir</arg>
<arg>${outputBaseDir}/${tableName}/${period}</arg>
<arg>–query</arg>
<arg>${query}</arg>
<arg>–split-by</arg>
<arg>${splitBy}</arg>
<arg>-m</arg>
<arg>1</arg>
</sqoop>
<ok to=”end” />
<error to=”fail” />
</action>
<kill name=”fail”>
<message>Script failed, error
message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name=’end’ />
</workflow-app>
job.properties looks like this:
nameNode=hdfs://10.0.2.15:8020
jobTracker=10.0.2.15:8021
queueName=default
mysqlHost=10.0.2.15
mysqlDb=test
outputBaseDir=/user/ajay
tableName=temperatures
period=2014010200
query=select *, DATE_FORMAT(created_time, ‘%Y%m%d%H’) as period from
${tableName} where DATE_FORMAT(created_time, ‘%Y%m%d%H’) = ${period} and
\\$CONDITIONS
splitBy=id
oozie.wf.application.path=${nameNode}/user/ajay/workflow.xml
outputDir=sqoop-output
Started the job under Sandbox like this:
oozie job -config ./job.properties -run
The following works from command line:
sqoop import –connect jdbc:mysql://10.0.2.15/test –username root
–target-dir $OUTPUT_DIR –query “select *, DATE_FORMAT(created_time,
‘%Y%m%d%H’) as period from $TABLE_NAME where DATE_FORMAT(created_time,
‘%Y%m%d%H’) = $PERIOD and \$CONDITIONS” –split-by id
What am I doing wrong? Please help. Thanks.
Re: Oozie-Sqoop job not working
Posted by Harsh J <ha...@cloudera.com>.
Are you seeing your action not execute? How many Map and Reduce slots
are currently configured on your cluster at http://10.0.2.15:50030?
On Fri, Feb 14, 2014 at 5:36 AM, Something Something
<ma...@gmail.com> wrote:
> When I run ‘Sqoop’ from command line it works fine, but when I run it via
> Oozie job it just keeps showing the following message:
>
> 2014-02-12 19:04:47,854 DEBUG DefinitionXCommand:545 – USER[root] GROUP[-]
> TOKEN[] APP[sqoop] JOB[0000004-140211210850200-oozie-oozi-W] ACTION[-]
> Execute command [definition] key [0000004-140211210850200-oozie-oozi-W]
>
> My workflow.xml looks like this:
>
> <workflow-app name=”sqoop” xmlns=”uri:oozie:workflow:0.1″>
> <start to=”etl” />
> <action name=”etl”>
> <sqoop xmlns=”uri:oozie:sqoop-action:0.2″>
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
> <prepare>
> <delete path=”${outputBaseDir}/${tableName}/${period}” />
> </prepare>
> <configuration>
> <property>
> <name>mapred.compress.map.output</name>
> <value>true</value>
> </property>
> </configuration>
> <arg>import</arg>
> <arg>–connect</arg>
> <arg>jdbc:mysql://${mysqlHost}/${mysqlDb}</arg>
> <arg>–username</arg>
> <arg>root</arg>
> <arg>–target-dir</arg>
> <arg>${outputBaseDir}/${tableName}/${period}</arg>
> <arg>–query</arg>
> <arg>${query}</arg>
> <arg>–split-by</arg>
> <arg>${splitBy}</arg>
> <arg>-m</arg>
> <arg>1</arg>
> </sqoop>
> <ok to=”end” />
> <error to=”fail” />
> </action>
> <kill name=”fail”>
> <message>Script failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> </kill>
> <end name=’end’ />
> </workflow-app>
>
> job.properties looks like this:
>
> nameNode=hdfs://10.0.2.15:8020
> jobTracker=10.0.2.15:8021
> queueName=default
>
> mysqlHost=10.0.2.15
> mysqlDb=test
>
> outputBaseDir=/user/ajay
> tableName=temperatures
> period=2014010200
> query=select *, DATE_FORMAT(created_time, ‘%Y%m%d%H’) as period from
> ${tableName} where DATE_FORMAT(created_time, ‘%Y%m%d%H’) = ${period} and
> \\$CONDITIONS
> splitBy=id
>
> oozie.wf.application.path=${nameNode}/user/ajay/workflow.xml
> outputDir=sqoop-output
>
> Started the job under Sandbox like this:
> oozie job -config ./job.properties -run
>
> The following works from command line:
> sqoop import –connect jdbc:mysql://10.0.2.15/test –username root
> –target-dir $OUTPUT_DIR –query “select *, DATE_FORMAT(created_time,
> ‘%Y%m%d%H’) as period from $TABLE_NAME where DATE_FORMAT(created_time,
> ‘%Y%m%d%H’) = $PERIOD and \$CONDITIONS” –split-by id
>
> What am I doing wrong? Please help. Thanks.
--
Harsh J