You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Parrot Silk <si...@gmail.com> on 2013/03/12 19:03:19 UTC
Use sqoop action to export to 3rd party database
Hi,
I am tying invoke oozie workflow containing a sqoop action.
First I try the sqoop command from command line and it works. The command
is as follows:
sqoop export --connect jdbc:filemaker://192.168.14.123/mydb --driver
com.filemaker.jdbc.Driver --username Admin --export-dir
/user/hive/warehouse/table1 --table targetTable
--input-fields-terminated-by '\001'
Next I try to embed the command into a workflow.xml as follow but can't
make it work
<command>export --connect jdbc:filemaker://192.168.14.123/mydb --driver
com.filemaker.jdbc.Driver --username Admin --export-dir
/user/hive/warehouse/table1 --table targetTable
--input-fields-terminated-by '\001'</command>
The job log outputs this:
2013-03-11 16:44:08,743 WARN SqoopActionExecutor:542 - USER[cloudera]
GROUP[-] TOKEN[] APP[sqoop-mine-wf]
JOB[0000027-130308164316356-oozie-oozi-W]
ACTION[0000027-130308164316356-oozie-oozi-W@sqoop-node] Launcher ERROR,
reason: Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]
2013-03-11 16:44:08,894 INFO ActionEndXCommand:539 - USER[cloudera]
GROUP[-] TOKEN[] APP[sqoop-mine-wf]
JOB[0000027-130308164316356-oozie-oozi-W]
ACTION[0000027-130308164316356-oozie-oozi-W@sqoop-node] ERROR is considered
as FAILED for SLA
2013-03-11 16:44:08,937 INFO ActionStartXCommand:539 - USER[cloudera]
GROUP[-] TOKEN[] APP[sqoop-mine-wf]
JOB[0000027-130308164316356-oozie-oozi-W]
ACTION[0000027-130308164316356-oozie-oozi-W@fail] Start action
[0000027-130308164316356-oozie-oozi-W@fail] with user-retry state :
userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2013-03-11 16:44:08,937 WARN ActionStartXCommand:542 - USER[cloudera]
GROUP[-] TOKEN[] APP[sqoop-mine-wf]
JOB[0000027-130308164316356-oozie-oozi-W]
ACTION[0000027-130308164316356-oozie-oozi-W@fail]
[***0000027-130308164316356-oozie-oozi-W@fail***]Action status=DONE
2013-03-11 16:44:08,937 WARN ActionStartXCommand:542 - USER[cloudera]
GROUP[-] TOKEN[] APP[sqoop-mine-wf]
JOB[0000027-130308164316356-oozie-oozi-W]
ACTION[0000027-130308164316356-oozie-oozi-W@fail]
[***0000027-130308164316356-oozie-oozi-W@fail***]Action updated in DB!
2013-03-11 16:44:09,005 WARN CoordActionUpdateXCommand:542 -
USER[cloudera] GROUP[-] TOKEN[] APP[sqoop-mine-wf]
JOB[0000027-130308164316356-oozie-oozi-W] ACTION[-] E1100: Command
precondition does not hold before execution, [, coord action is null],
Error Code: E1100
I also check the sqoop-node action(which indicates the failure) and
follow the console URL. However, that page actually shows the operation is
successful.
I did include jdbc.jar in the lib.However I still get the error.
Any help would be appreciated and I am wondering if there is any
other logs I can check.
Thanks
Ryan
Re: Use sqoop action to export to 3rd party database
Posted by Robert Kanter <rk...@cloudera.com>.
Hi,
Can you check the log in the JobTracker UI for the launcher job and paste
any error messages here?
thanks
- Robert
On Tue, Mar 12, 2013 at 11:03 AM, Parrot Silk <si...@gmail.com> wrote:
> Hi,
>
> I am tying invoke oozie workflow containing a sqoop action.
>
> First I try the sqoop command from command line and it works. The command
> is as follows:
>
> sqoop export --connect jdbc:filemaker://192.168.14.123/mydb --driver
> com.filemaker.jdbc.Driver --username Admin --export-dir
> /user/hive/warehouse/table1 --table targetTable
> --input-fields-terminated-by '\001'
>
> Next I try to embed the command into a workflow.xml as follow but can't
> make it work
>
> <command>export --connect jdbc:filemaker://192.168.14.123/mydb --driver
> com.filemaker.jdbc.Driver --username Admin --export-dir
> /user/hive/warehouse/table1 --table targetTable
> --input-fields-terminated-by '\001'</command>
>
>
> The job log outputs this:
>
> 2013-03-11 16:44:08,743 WARN SqoopActionExecutor:542 - USER[cloudera]
> GROUP[-] TOKEN[] APP[sqoop-mine-wf]
> JOB[0000027-130308164316356-oozie-oozi-W]
> ACTION[0000027-130308164316356-oozie-oozi-W@sqoop-node] Launcher ERROR,
> reason: Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code
> [1]
> 2013-03-11 16:44:08,894 INFO ActionEndXCommand:539 - USER[cloudera]
> GROUP[-] TOKEN[] APP[sqoop-mine-wf]
> JOB[0000027-130308164316356-oozie-oozi-W]
> ACTION[0000027-130308164316356-oozie-oozi-W@sqoop-node] ERROR is
> considered
> as FAILED for SLA
> 2013-03-11 16:44:08,937 INFO ActionStartXCommand:539 - USER[cloudera]
> GROUP[-] TOKEN[] APP[sqoop-mine-wf]
> JOB[0000027-130308164316356-oozie-oozi-W]
> ACTION[0000027-130308164316356-oozie-oozi-W@fail] Start action
> [0000027-130308164316356-oozie-oozi-W@fail] with user-retry state :
> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> 2013-03-11 16:44:08,937 WARN ActionStartXCommand:542 - USER[cloudera]
> GROUP[-] TOKEN[] APP[sqoop-mine-wf]
> JOB[0000027-130308164316356-oozie-oozi-W]
> ACTION[0000027-130308164316356-oozie-oozi-W@fail]
> [***0000027-130308164316356-oozie-oozi-W@fail***]Action status=DONE
> 2013-03-11 16:44:08,937 WARN ActionStartXCommand:542 - USER[cloudera]
> GROUP[-] TOKEN[] APP[sqoop-mine-wf]
> JOB[0000027-130308164316356-oozie-oozi-W]
> ACTION[0000027-130308164316356-oozie-oozi-W@fail]
> [***0000027-130308164316356-oozie-oozi-W@fail***]Action updated in DB!
> 2013-03-11 16:44:09,005 WARN CoordActionUpdateXCommand:542 -
> USER[cloudera] GROUP[-] TOKEN[] APP[sqoop-mine-wf]
> JOB[0000027-130308164316356-oozie-oozi-W] ACTION[-] E1100: Command
> precondition does not hold before execution, [, coord action is null],
> Error Code: E1100
>
>
> I also check the sqoop-node action(which indicates the failure) and
> follow the console URL. However, that page actually shows the operation is
> successful.
>
> I did include jdbc.jar in the lib.However I still get the error.
>
> Any help would be appreciated and I am wondering if there is any
> other logs I can check.
>
>
> Thanks
> Ryan
>
Re: Use sqoop action to export to 3rd party database
Posted by Parrot Silk <si...@gmail.com>.
Hi Robert,
Thanks for the reply. After checking the log and I think I find the cause.
On the command line, the following works:
sqoop export --connect jdbc:filemaker://192.168.14.123/mydb --driver
com.filemaker.jdbc.Driver --username Admin --export-dir
/user/hive/warehouse/table1 --table targetTable
--input-fields-terminated-by '\001'
However, if put in the workflow.xml, I need to use :
<command>export --connect jdbc:filemaker://192.168.14.123/mydb --driver
com.filemaker.jdbc.Driver --username Admin --export-dir
/user/hive/warehouse/table1 --table targetTable
--input-fields-terminated-by \001</command>
Please note that there is no single quotation mark around \001.
Thanks
Ryan
2013/3/12 Parrot Silk <si...@gmail.com>
> Hi,
>
> I am tying invoke oozie workflow containing a sqoop action.
>
> First I try the sqoop command from command line and it works. The
> command is as follows:
>
> sqoop export --connect jdbc:filemaker://192.168.14.123/mydb --driver
> com.filemaker.jdbc.Driver --username Admin --export-dir
> /user/hive/warehouse/table1 --table targetTable
> --input-fields-terminated-by '\001'
>
> Next I try to embed the command into a workflow.xml as follow but can't
> make it work
>
> <command>export --connect jdbc:filemaker://192.168.14.123/mydb --driver
> com.filemaker.jdbc.Driver --username Admin --export-dir
> /user/hive/warehouse/table1 --table targetTable
> --input-fields-terminated-by '\001'</command>
>
>
> The job log outputs this:
>
> 2013-03-11 16:44:08,743 WARN SqoopActionExecutor:542 - USER[cloudera]
> GROUP[-] TOKEN[] APP[sqoop-mine-wf]
> JOB[0000027-130308164316356-oozie-oozi-W]
> ACTION[0000027-130308164316356-oozie-oozi-W@sqoop-node] Launcher ERROR,
> reason: Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]
> 2013-03-11 16:44:08,894 INFO ActionEndXCommand:539 - USER[cloudera]
> GROUP[-] TOKEN[] APP[sqoop-mine-wf]
> JOB[0000027-130308164316356-oozie-oozi-W]
> ACTION[0000027-130308164316356-oozie-oozi-W@sqoop-node] ERROR is
> considered as FAILED for SLA
> 2013-03-11 16:44:08,937 INFO ActionStartXCommand:539 - USER[cloudera]
> GROUP[-] TOKEN[] APP[sqoop-mine-wf]
> JOB[0000027-130308164316356-oozie-oozi-W]
> ACTION[0000027-130308164316356-oozie-oozi-W@fail] Start action
> [0000027-130308164316356-oozie-oozi-W@fail] with user-retry state :
> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> 2013-03-11 16:44:08,937 WARN ActionStartXCommand:542 - USER[cloudera]
> GROUP[-] TOKEN[] APP[sqoop-mine-wf]
> JOB[0000027-130308164316356-oozie-oozi-W]
> ACTION[0000027-130308164316356-oozie-oozi-W@fail]
> [***0000027-130308164316356-oozie-oozi-W@fail***]Action status=DONE
> 2013-03-11 16:44:08,937 WARN ActionStartXCommand:542 - USER[cloudera]
> GROUP[-] TOKEN[] APP[sqoop-mine-wf]
> JOB[0000027-130308164316356-oozie-oozi-W]
> ACTION[0000027-130308164316356-oozie-oozi-W@fail]
> [***0000027-130308164316356-oozie-oozi-W@fail***]Action updated in DB!
> 2013-03-11 16:44:09,005 WARN CoordActionUpdateXCommand:542 -
> USER[cloudera] GROUP[-] TOKEN[] APP[sqoop-mine-wf]
> JOB[0000027-130308164316356-oozie-oozi-W] ACTION[-] E1100: Command
> precondition does not hold before execution, [, coord action is null],
> Error Code: E1100
>
>
> I also check the sqoop-node action(which indicates the failure) and
> follow the console URL. However, that page actually shows the operation is
> successful.
>
> I did include jdbc.jar in the lib.However I still get the error.
>
> Any help would be appreciated and I am wondering if there is any
> other logs I can check.
>
>
> Thanks
> Ryan
>