You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by "hui.pan@wbkit.com" <hu...@wbkit.com> on 2017/05/09 13:13:22 UTC
A problem about oozie workflow about shell Action.
Dear All:
I meet a problem about oozie workflow about shell Action. I use hue to control my oozie.
The step is as follows:
1. I create my workflow using shell and using a exist shell below：
#!/bin/sh
DB_USER=test
DB_PWD=test
DB_SERV=192.168.1.207/razor
sqlplus64 -s "$DB_USER/$DB_PWD@$DB_SERV" << EOF
spool /opt/extract_data.sql
set trimspool on
set linesize 8000
set pagesize 50000
set newpage 1
set heading off
set term off
set feedback off
set sqlblankline off
SELECT * FROM RAZOR_WORKS WHERE STATUS='FAIL';
spool off
exit
EOF
2. I run this and got this error:
2017-05-09 19:50:46,238 INFO ActionStartXCommand:520 - SERVER[tw-master] USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-170506114458023-oozie-oozi-W@:start:] Start action [0000127-170506114458023-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2017-05-09 19:50:46,245 INFO ActionStartXCommand:520 - SERVER[tw-master] USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-170506114458023-oozie-oozi-W@:start:] [***0000127-170506114458023-oozie-oozi-W@:start:***]Action status=DONE
2017-05-09 19:50:46,246 INFO ActionStartXCommand:520 - SERVER[tw-master] USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-170506114458023-oozie-oozi-W@:start:] [***0000127-170506114458023-oozie-oozi-W@:start:***]Action updated in DB!
2017-05-09 19:50:46,363 INFO WorkflowNotificationXCommand:520 - SERVER[tw-master] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-170506114458023-oozie-oozi-W@:start:] No Notification URL is defined. Therefore nothing to notify for job 0000127-170506114458023-oozie-oozi-W@:start:
2017-05-09 19:50:46,363 INFO WorkflowNotificationXCommand:520 - SERVER[tw-master] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000127-170506114458023-oozie-oozi-W
2017-05-09 19:50:46,399 INFO ActionStartXCommand:520 - SERVER[tw-master] USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-170506114458023-oozie-oozi-W@shell-077a] Start action [0000127-170506114458023-oozie-oozi-W@shell-077a] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2017-05-09 19:50:48,543 INFO ShellActionExecutor:520 - SERVER[tw-master] USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-170506114458023-oozie-oozi-W@shell-077a] checking action, hadoop job ID [job_1494042098609_0071] status [RUNNING]
2017-05-09 19:50:48,547 INFO ActionStartXCommand:520 - SERVER[tw-master] USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-170506114458023-oozie-oozi-W@shell-077a] [***0000127-170506114458023-oozie-oozi-W@shell-077a***]Action status=RUNNING
2017-05-09 19:50:48,547 INFO ActionStartXCommand:520 - SERVER[tw-master] USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-170506114458023-oozie-oozi-W@shell-077a] [***0000127-170506114458023-oozie-oozi-W@shell-077a***]Action updated in DB!
2017-05-09 19:50:48,585 INFO WorkflowNotificationXCommand:520 - SERVER[tw-master] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-170506114458023-oozie-oozi-W@shell-077a] No Notification URL is defined. Therefore nothing to notify for job 0000127-170506114458023-oozie-oozi-W@shell-077a
2017-05-09 19:51:06,432 INFO CallbackServlet:520 - SERVER[tw-master] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-170506114458023-oozie-oozi-W@shell-077a] callback for action [0000127-170506114458023-oozie-oozi-W@shell-077a]
2017-05-09 19:51:06,601 INFO ShellActionExecutor:520 - SERVER[tw-master] USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-170506114458023-oozie-oozi-W@shell-077a] action completed, external ID [job_1494042098609_0071]
2017-05-09 19:51:06,611 WARN ShellActionExecutor:523 - SERVER[tw-master] USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-170506114458023-oozie-oozi-W@shell-077a] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]
2017-05-09 19:51:06,711 INFO ActionEndXCommand:520 - SERVER[tw-master] USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-170506114458023-oozie-oozi-W@shell-077a] ERROR is considered as FAILED for SLA
2017-05-09 19:51:06,853 INFO ActionStartXCommand:520 - SERVER[tw-master] USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-170506114458023-oozie-oozi-W@Kill] Start action [0000127-170506114458023-oozie-oozi-W@Kill] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2017-05-09 19:51:06,864 INFO ActionStartXCommand:520 - SERVER[tw-master] USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-170506114458023-oozie-oozi-W@Kill] [***0000127-170506114458023-oozie-oozi-W@Kill***]Action status=DONE
2017-05-09 19:51:06,864 INFO ActionStartXCommand:520 - SERVER[tw-master] USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-170506114458023-oozie-oozi-W@Kill] [***0000127-170506114458023-oozie-oozi-W@Kill***]Action updated in DB!
2017-05-09 19:51:06,994 INFO WorkflowNotificationXCommand:520 - SERVER[tw-master] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-170506114458023-oozie-oozi-W@Kill] No Notification URL is defined. Therefore nothing to notify for job 0000127-170506114458023-oozie-oozi-W@Kill
2017-05-09 19:51:06,994 INFO WorkflowNotificationXCommand:520 - SERVER[tw-master] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-170506114458023-oozie-oozi-W@shell-077a] No Notification URL is defined. Therefore nothing to notify for job 0000127-170506114458023-oozie-oozi-W@shell-077a
2017-05-09 19:51:06,995 INFO WorkflowNotificationXCommand:520 - SERVER[tw-master] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000127-170506114458023-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000127-170506114458023-oozie-oozi-W
3. I don't know how to do with ERROR is considered as FAILED for SLA.Is there something wrong with my permission,or the solution is not supported.
Thanks
Best Regards,
Hollis
Re: A problem about oozie workflow about shell Action.
Posted by Peter Cseh <ge...@cloudera.com>.
Hi,
For some reason, the shell script is exiting with an error code.
please check the logs for the launcher mapper. The root cause of the issue
should be visible there.
gp
On Tue, May 9, 2017 at 6:13 AM, hui.pan@wbkit.com <hu...@wbkit.com> wrote:
> Dear All:
> I meet a problem about oozie workflow about shell Action. I use
> hue to control my oozie.
> The step is as follows:
> 1. I create my workflow using shell and using a exist shell
> below：
>
> #!/bin/sh
>
> DB_USER=test
> DB_PWD=test
> DB_SERV=192.168.1.207/razor
> sqlplus64 -s "$DB_USER/$DB_PWD@$DB_SERV" << EOF
> spool /opt/extract_data.sql
> set trimspool on
> set linesize 8000
> set pagesize 50000
> set newpage 1
> set heading off
> set term off
> set feedback off
> set sqlblankline off
> SELECT * FROM RAZOR_WORKS WHERE STATUS='FAIL';
> spool off
> exit
> EOF
> 2. I run this and got this error:
> 2017-05-09 19:50:46,238 INFO ActionStartXCommand:520 - SERVER[tw-master]
> USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W]
> ACTION[0000127-170506114458023-oozie-oozi-W@:start:] Start action
> [0000127-170506114458023-oozie-oozi-W@:start:] with user-retry state :
> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> 2017-05-09 19:50:46,245 INFO ActionStartXCommand:520 - SERVER[tw-master]
> USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W]
> ACTION[0000127-170506114458023-oozie-oozi-W@:start:]
> [***0000127-170506114458023-oozie-oozi-W@:start:***]Action status=DONE
> 2017-05-09 19:50:46,246 INFO ActionStartXCommand:520 - SERVER[tw-master]
> USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W]
> ACTION[0000127-170506114458023-oozie-oozi-W@:start:]
> [***0000127-170506114458023-oozie-oozi-W@:start:***]Action updated in DB!
> 2017-05-09 19:50:46,363 INFO WorkflowNotificationXCommand:520 -
> SERVER[tw-master] USER[-] GROUP[-] TOKEN[-] APP[-]
> JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-
> 170506114458023-oozie-oozi-W@:start:] No Notification URL is defined.
> Therefore nothing to notify for job 0000127-170506114458023-oozie-oozi-W@
> :start:
> 2017-05-09 19:50:46,363 INFO WorkflowNotificationXCommand:520 -
> SERVER[tw-master] USER[-] GROUP[-] TOKEN[-] APP[-]
> JOB[0000127-170506114458023-oozie-oozi-W] ACTION[] No Notification URL is
> defined. Therefore nothing to notify for job 0000127-170506114458023-oozie-
> oozi-W
> 2017-05-09 19:50:46,399 INFO ActionStartXCommand:520 - SERVER[tw-master]
> USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W]
> ACTION[0000127-170506114458023-oozie-oozi-W@shell-077a] Start action
> [0000127-170506114458023-oozie-oozi-W@shell-077a] with user-retry state :
> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> 2017-05-09 19:50:48,543 INFO ShellActionExecutor:520 - SERVER[tw-master]
> USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W]
> ACTION[0000127-170506114458023-oozie-oozi-W@shell-077a] checking action,
> hadoop job ID [job_1494042098609_0071] status [RUNNING]
> 2017-05-09 19:50:48,547 INFO ActionStartXCommand:520 - SERVER[tw-master]
> USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W]
> ACTION[0000127-170506114458023-oozie-oozi-W@shell-077a]
> [***0000127-170506114458023-oozie-oozi-W@shell-077a***]Action
> status=RUNNING
> 2017-05-09 19:50:48,547 INFO ActionStartXCommand:520 - SERVER[tw-master]
> USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W]
> ACTION[0000127-170506114458023-oozie-oozi-W@shell-077a]
> [***0000127-170506114458023-oozie-oozi-W@shell-077a***]Action updated in
> DB!
> 2017-05-09 19:50:48,585 INFO WorkflowNotificationXCommand:520 -
> SERVER[tw-master] USER[-] GROUP[-] TOKEN[-] APP[-]
> JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-
> 170506114458023-oozie-oozi-W@shell-077a] No Notification URL is defined.
> Therefore nothing to notify for job 0000127-170506114458023-oozie-
> oozi-W@shell-077a
> 2017-05-09 19:51:06,432 INFO CallbackServlet:520 - SERVER[tw-master]
> USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000127-170506114458023-oozie-oozi-W]
> ACTION[0000127-170506114458023-oozie-oozi-W@shell-077a] callback for
> action [0000127-170506114458023-oozie-oozi-W@shell-077a]
> 2017-05-09 19:51:06,601 INFO ShellActionExecutor:520 - SERVER[tw-master]
> USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W]
> ACTION[0000127-170506114458023-oozie-oozi-W@shell-077a] action completed,
> external ID [job_1494042098609_0071]
> 2017-05-09 19:51:06,611 WARN ShellActionExecutor:523 - SERVER[tw-master]
> USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W]
> ACTION[0000127-170506114458023-oozie-oozi-W@shell-077a] Launcher ERROR,
> reason: Main class [org.apache.oozie.action.hadoop.ShellMain], exit code
> [1]
> 2017-05-09 19:51:06,711 INFO ActionEndXCommand:520 - SERVER[tw-master]
> USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W]
> ACTION[0000127-170506114458023-oozie-oozi-W@shell-077a] ERROR is
> considered as FAILED for SLA
> 2017-05-09 19:51:06,853 INFO ActionStartXCommand:520 - SERVER[tw-master]
> USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W]
> ACTION[0000127-170506114458023-oozie-oozi-W@Kill] Start action
> [0000127-170506114458023-oozie-oozi-W@Kill] with user-retry state :
> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> 2017-05-09 19:51:06,864 INFO ActionStartXCommand:520 - SERVER[tw-master]
> USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W]
> ACTION[0000127-170506114458023-oozie-oozi-W@Kill]
> [***0000127-170506114458023-oozie-oozi-W@Kill***]Action status=DONE
> 2017-05-09 19:51:06,864 INFO ActionStartXCommand:520 - SERVER[tw-master]
> USER[root] GROUP[-] TOKEN[] APP[SHELL_AGAIN] JOB[0000127-170506114458023-oozie-oozi-W]
> ACTION[0000127-170506114458023-oozie-oozi-W@Kill]
> [***0000127-170506114458023-oozie-oozi-W@Kill***]Action updated in DB!
> 2017-05-09 19:51:06,994 INFO WorkflowNotificationXCommand:520 -
> SERVER[tw-master] USER[-] GROUP[-] TOKEN[-] APP[-]
> JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-
> 170506114458023-oozie-oozi-W@Kill] No Notification URL is defined.
> Therefore nothing to notify for job 0000127-170506114458023-oozie-
> oozi-W@Kill
> 2017-05-09 19:51:06,994 INFO WorkflowNotificationXCommand:520 -
> SERVER[tw-master] USER[-] GROUP[-] TOKEN[-] APP[-]
> JOB[0000127-170506114458023-oozie-oozi-W] ACTION[0000127-
> 170506114458023-oozie-oozi-W@shell-077a] No Notification URL is defined.
> Therefore nothing to notify for job 0000127-170506114458023-oozie-
> oozi-W@shell-077a
> 2017-05-09 19:51:06,995 INFO WorkflowNotificationXCommand:520 -
> SERVER[tw-master] USER[-] GROUP[-] TOKEN[-] APP[-]
> JOB[0000127-170506114458023-oozie-oozi-W] ACTION[] No Notification URL is
> defined. Therefore nothing to notify for job 0000127-170506114458023-oozie-
> oozi-W
> 3. I don't know how to do with ERROR is considered as FAILED
> for SLA.Is there something wrong with my permission,or the solution is not
> supported.
> Thanks
>
> Best Regards,
> Hollis
>
>
--
Peter Cseh
Software Engineer
<http://www.cloudera.com>