You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Alberto Crespi <cr...@gmail.com> on 2014/06/09 13:05:09 UTC
Oozie launcher never ends
Hello,
I'm using CDH 5.0.1 and oozie-4.0.0
I'm doing a simple oozie's workflow with sqoop's action.
this is the xml's code:
<workflow-app name="Sqoop" xmlns="uri:oozie:workflow:0.4">
<start to="Sqoop"/>
<action name="Sqoop">
<sqoop xmlns="uri:oozie:sqoop-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<command>import --connect jdbc:mysql://10.0.0.221/db
--username cloudera --password cloudera --table test -m 1 --target-dir
/user/albert</command>
</sqoop>
<ok to="end"/>
<error to="kill"/>
</action>
<kill name="kill">
<message>Action failed, error
message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>
When i launch my job with hue or line-command, starts the oozie-launcher.
My oozie laucher, never finish, it is always stay at 95%.
If i kill the job oozie-launcher, starts the oozie-sqoop and it do his
work very good.
But i don't understand why the oozie-launcher doesnt' work well!
This is my output
>>> Invoking Sqoop command line now >>>
12280 [main] WARN org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR
has not been set in the environment. Cannot check for additional
configuration.
12510 [main] INFO org.apache.sqoop.Sqoop - Running Sqoop version:
1.4.4-cdh5.0.1
12640 [main] WARN org.apache.sqoop.tool.BaseSqoopTool - Setting your
password on the command-line is insecure. Consider using -P instead.
12748 [main] WARN org.apache.sqoop.ConnFactory - $SQOOP_CONF_DIR has
not been set in the environment. Cannot check for additional
configuration.
13131 [main] INFO org.apache.sqoop.manager.MySQLManager - Preparing
to use a MySQL streaming resultset.
13131 [main] INFO org.apache.sqoop.tool.CodeGenTool - Beginning code
generation
15107 [main] INFO org.apache.sqoop.manager.SqlManager - Executing
SQL statement: SELECT t.* FROM `test` AS t LIMIT 1
15202 [main] INFO org.apache.sqoop.manager.SqlManager - Executing
SQL statement: SELECT t.* FROM `test` AS t LIMIT 1
15206 [main] INFO org.apache.sqoop.orm.CompilationManager -
HADOOP_MAPRED_HOME is
/opt/cloudera/parcels/CDH-5.0.1-1.cdh5.0.1.p0.47/lib/hadoop-mapreduce
23112 [main] INFO org.apache.sqoop.orm.CompilationManager - Writing
jar file: /tmp/sqoop-yarn/compile/ab1d242ba1b3a0869bab06c1eb20c02f/test.jar
23140 [main] WARN org.apache.sqoop.manager.MySQLManager - It looks
like you are importing from mysql.
23140 [main] WARN org.apache.sqoop.manager.MySQLManager - This
transfer can be faster! Use the --direct
23140 [main] WARN org.apache.sqoop.manager.MySQLManager - option to
exercise a MySQL-specific fast path.
23140 [main] INFO org.apache.sqoop.manager.MySQLManager - Setting
zero DATETIME behavior to convertToNull (mysql)
23151 [main] INFO org.apache.sqoop.mapreduce.ImportJobBase -
Beginning import of test
23232 [main] WARN org.apache.sqoop.mapreduce.JobBase - SQOOP_HOME is
unset. May not be able to find all job dependencies.
25731 [main] INFO org.apache.sqoop.mapreduce.db.DBInputFormat -
Using read commited transaction isolation
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Someone can help me?
Thank-you
Re: Oozie launcher never ends
Posted by Mona Chitnis <ch...@yahoo-inc.com.INVALID>.
I didn¹t quite understand one part. You mentioned that after you kill the
launcher your sqoop action starts? I¹m not sure if Sqoop action works the
same as Pig action - where launcher is alive for the entire duration of
the child jobs. If it is similar then that is expected behavior.
On 6/9/14, 4:05 AM, "Alberto Crespi" <cr...@gmail.com> wrote:
>Hello,
>
>I'm using CDH 5.0.1 and oozie-4.0.0
>
>I'm doing a simple oozie's workflow with sqoop's action.
>this is the xml's code:
>
><workflow-app name="Sqoop" xmlns="uri:oozie:workflow:0.4">
> <start to="Sqoop"/>
> <action name="Sqoop">
> <sqoop xmlns="uri:oozie:sqoop-action:0.2">
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
> <command>import --connect jdbc:mysql://10.0.0.221/db
>--username cloudera --password cloudera --table test -m 1 --target-dir
>/user/albert</command>
> </sqoop>
> <ok to="end"/>
> <error to="kill"/>
> </action>
> <kill name="kill">
> <message>Action failed, error
>message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> </kill>
> <end name="end"/>
></workflow-app>
>
>
>When i launch my job with hue or line-command, starts the oozie-launcher.
>My oozie laucher, never finish, it is always stay at 95%.
>If i kill the job oozie-launcher, starts the oozie-sqoop and it do his
>work very good.
>But i don't understand why the oozie-launcher doesnt' work well!
>
>
>
>This is my output
>
>
>>>> Invoking Sqoop command line now >>>
>
>12280 [main] WARN org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR
>has not been set in the environment. Cannot check for additional
>configuration.
>12510 [main] INFO org.apache.sqoop.Sqoop - Running Sqoop version:
>1.4.4-cdh5.0.1
>12640 [main] WARN org.apache.sqoop.tool.BaseSqoopTool - Setting your
>password on the command-line is insecure. Consider using -P instead.
>12748 [main] WARN org.apache.sqoop.ConnFactory - $SQOOP_CONF_DIR has
>not been set in the environment. Cannot check for additional
>configuration.
>13131 [main] INFO org.apache.sqoop.manager.MySQLManager - Preparing
>to use a MySQL streaming resultset.
>13131 [main] INFO org.apache.sqoop.tool.CodeGenTool - Beginning code
>generation
>15107 [main] INFO org.apache.sqoop.manager.SqlManager - Executing
>SQL statement: SELECT t.* FROM `test` AS t LIMIT 1
>15202 [main] INFO org.apache.sqoop.manager.SqlManager - Executing
>SQL statement: SELECT t.* FROM `test` AS t LIMIT 1
>15206 [main] INFO org.apache.sqoop.orm.CompilationManager -
>HADOOP_MAPRED_HOME is
>/opt/cloudera/parcels/CDH-5.0.1-1.cdh5.0.1.p0.47/lib/hadoop-mapreduce
>23112 [main] INFO org.apache.sqoop.orm.CompilationManager - Writing
>jar file:
>/tmp/sqoop-yarn/compile/ab1d242ba1b3a0869bab06c1eb20c02f/test.jar
>23140 [main] WARN org.apache.sqoop.manager.MySQLManager - It looks
>like you are importing from mysql.
>23140 [main] WARN org.apache.sqoop.manager.MySQLManager - This
>transfer can be faster! Use the --direct
>23140 [main] WARN org.apache.sqoop.manager.MySQLManager - option to
>exercise a MySQL-specific fast path.
>23140 [main] INFO org.apache.sqoop.manager.MySQLManager - Setting
>zero DATETIME behavior to convertToNull (mysql)
>23151 [main] INFO org.apache.sqoop.mapreduce.ImportJobBase -
>Beginning import of test
>23232 [main] WARN org.apache.sqoop.mapreduce.JobBase - SQOOP_HOME is
>unset. May not be able to find all job dependencies.
>25731 [main] INFO org.apache.sqoop.mapreduce.db.DBInputFormat -
>Using read commited transaction isolation
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>Heart beat
>
>
>Someone can help me?
>
>Thank-you
Re: Oozie launcher never ends
Posted by Alberto Crespi <cr...@gmail.com>.
Hi,
I don't care if the launcher stay alive while the action starts or ends.
My problem is that the launcher stay append at 95% forever, and the action
sqoop never starts.
Something about nodemanager memory? or no-slot for map-reduce?
2014-06-09 13:05 GMT+02:00 Alberto Crespi <cr...@gmail.com>:
> Hello,
>
> I'm using CDH 5.0.1 and oozie-4.0.0
>
> I'm doing a simple oozie's workflow with sqoop's action.
> this is the xml's code:
>
> <workflow-app name="Sqoop" xmlns="uri:oozie:workflow:0.4">
> <start to="Sqoop"/>
> <action name="Sqoop">
> <sqoop xmlns="uri:oozie:sqoop-action:0.2">
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
> <command>import --connect jdbc:mysql://10.0.0.221/db --username cloudera --password cloudera --table test -m 1 --target-dir /user/albert</command>
> </sqoop>
> <ok to="end"/>
> <error to="kill"/>
> </action>
> <kill name="kill">
> <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> </kill>
> <end name="end"/>
> </workflow-app>
>
>
> When i launch my job with hue or line-command, starts the oozie-launcher.
> My oozie laucher, never finish, it is always stay at 95%.
> If i kill the job oozie-launcher, starts the oozie-sqoop and it do his work very good.
>
> But i don't understand why the oozie-launcher doesnt' work well!
>
>
>
> This is my output
>
>
> >>> Invoking Sqoop command line now >>>
>
> 12280 [main] WARN org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
> 12510 [main] INFO org.apache.sqoop.Sqoop - Running Sqoop version: 1.4.4-cdh5.0.1
> 12640 [main] WARN org.apache.sqoop.tool.BaseSqoopTool - Setting your password on the command-line is insecure. Consider using -P instead.
> 12748 [main] WARN org.apache.sqoop.ConnFactory - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
> 13131 [main] INFO org.apache.sqoop.manager.MySQLManager - Preparing to use a MySQL streaming resultset.
> 13131 [main] INFO org.apache.sqoop.tool.CodeGenTool - Beginning code generation
> 15107 [main] INFO org.apache.sqoop.manager.SqlManager - Executing SQL statement: SELECT t.* FROM `test` AS t LIMIT 1
> 15202 [main] INFO org.apache.sqoop.manager.SqlManager - Executing SQL statement: SELECT t.* FROM `test` AS t LIMIT 1
> 15206 [main] INFO org.apache.sqoop.orm.CompilationManager - HADOOP_MAPRED_HOME is /opt/cloudera/parcels/CDH-5.0.1-1.cdh5.0.1.p0.47/lib/hadoop-mapreduce
> 23112 [main] INFO org.apache.sqoop.orm.CompilationManager - Writing jar file: /tmp/sqoop-yarn/compile/ab1d242ba1b3a0869bab06c1eb20c02f/test.jar
> 23140 [main] WARN org.apache.sqoop.manager.MySQLManager - It looks like you are importing from mysql.
> 23140 [main] WARN org.apache.sqoop.manager.MySQLManager - This transfer can be faster! Use the --direct
> 23140 [main] WARN org.apache.sqoop.manager.MySQLManager - option to exercise a MySQL-specific fast path.
> 23140 [main] INFO org.apache.sqoop.manager.MySQLManager - Setting zero DATETIME behavior to convertToNull (mysql)
> 23151 [main] INFO org.apache.sqoop.mapreduce.ImportJobBase - Beginning import of test
> 23232 [main] WARN org.apache.sqoop.mapreduce.JobBase - SQOOP_HOME is unset. May not be able to find all job dependencies.
> 25731 [main] INFO org.apache.sqoop.mapreduce.db.DBInputFormat - Using read commited transaction isolation
> Heart beat
> Heart beat
> Heart beat
> Heart beat
> Heart beat
> Heart beat
> Heart beat
> Heart beat
> Heart beat
> Heart beat
> Heart beat
> Heart beat
> Heart beat
> Heart beat
> Heart beat
> Heart beat
> Heart beat
> Heart beat
> Heart beat
> Heart beat
> Heart beat
> Heart beat
> Heart beat
>
>
> Someone can help me?
>
> Thank-you
>
>