You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by M Gates <ga...@gmail.com> on 2016/06/22 00:35:21 UTC
Oozie error with pig
Hi
Looking for some help with Oozie and Pig. I have the setup below but keep getting the error below: 'The application does not define formal parameters in its XML definition’.
I have my test.pig script in hdfs with the workflow.xml - any idea what I am missing in the scripts below or what might cause this error?
Thanks,
Mark G.
Workflow.xml
<workflow-app name="customer_journey_event_app" xmlns="uri:oozie:workflow:0.4">
<start to="test"/>
<action name="test">
<pig>
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<prepare>
<delete path="${hdfs_temp_dir}/pig"/>
</prepare>
<configuration>
<property>
<name>mapred.compress.map.output</name>
<value>true</value>
</property>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<script>test.pig</script>
<argument>-useHCatalog</argument>
<argument>-param</argument>
<argument>output=${hdfs_temp_dir}/pig</argument>
</pig>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Pig failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>
Coordinator.xml
<coordinator-app name="${app_name}"
frequency="${coord:days(1)}"
start="${start_time}"
end="${end_time}"
timezone=“America/Montreal"
xmlns="uri:oozie:coordinator:0.2">
<controls>
<concurrency>1</concurrency>
<execution>FIFO</execution>
</controls>
<action>
<workflow>
<app-path>${app_path}/workflow</app-path>
<configuration>
<property>
<name>nominal_time</name>
<value>${coord:nominalTime()}</value>
</property>
</configuration>
</workflow>
</action>
</coordinator-app>
Log Output:
2016-06-21 19:33:16,766 INFO CoordMaterializeTransitionXCommand:543 - SERVER[hadoop.oss.ads] USER[markg] GROUP[-] TOKEN[] APP[customer_journey_event_app] JOB[0002430-160613201309706-oozie-oozi-C] ACTION[-] [0002430-160613201309706-oozie-oozi-C]: all actions have been materialized, set pending to true
2016-06-21 19:33:16,766 INFO CoordMaterializeTransitionXCommand:543 - SERVER[hadoop.oss.ads] USER[markg] GROUP[-] TOKEN[] APP[customer_journey_event_app] JOB[0002430-160613201309706-oozie-oozi-C] ACTION[-] Coord Job status updated to = RUNNING
2016-06-21 19:33:16,870 INFO CoordActionNotificationXCommand:543 - SERVER[hadoop.oss.ads] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0002430-160613201309706-oozie-oozi-C] ACTION[0002430-160613201309706-oozie-oozi-C@1] STARTED Coordinator Notification actionId=0002430-160613201309706-oozie-oozi-C@1 : WAITING
2016-06-21 19:33:16,871 INFO CoordActionNotificationXCommand:543 - SERVER[hadoop.oss.ads] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0002430-160613201309706-oozie-oozi-C] ACTION[0002430-160613201309706-oozie-oozi-C@1] No Notification URL is defined. Therefore nothing to notify for job 0002430-160613201309706-oozie-oozi-C action ID 0002430-160613201309706-oozie-oozi-C@1
2016-06-21 19:33:16,871 INFO CoordActionNotificationXCommand:543 - SERVER[hadoop.oss.ads] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0002430-160613201309706-oozie-oozi-C] ACTION[0002430-160613201309706-oozie-oozi-C@1] ENDED Coordinator Notification actionId=0002430-160613201309706-oozie-oozi-C@1
2016-06-21 19:33:34,573 INFO CoordActionInputCheckXCommand:543 - SERVER[hadoop.oss.ads] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0002430-160613201309706-oozie-oozi-C] ACTION[0002430-160613201309706-oozie-oozi-C@1] [0002430-160613201309706-oozie-oozi-C@1]::CoordActionInputCheck:: Missing deps:
2016-06-21 19:33:34,619 WARN ParameterVerifier:546 - SERVER[hadoop.oss.ads] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0002430-160613201309706-oozie-oozi-C] ACTION[0002430-160613201309706-oozie-oozi-C@1] The application does not define formal parameters in its XML definition
2016-06-21 19:33:54,696 INFO CoordActionUpdateXCommand:543 - SERVER[hadoop.oss.ads] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0002430-160613201309706-oozie-oozi-C] ACTION[0002430-160613201309706-oozie-oozi-C@1] Updating Coordintaor action id :0002430-160613201309706-oozie-oozi-C@1 status to KILLED, pending = 0
Re: Oozie error with pig
Posted by Peter Cseh <ge...@cloudera.com>.
Hi,
There are several parameters defined in the workflow (${app_name},
${start_time} etc.).
All of them should have defined values at the time of workflow submission.
From the message it seems like some of the parameters are not defined in
the coordinator job.
Regards,
Peter
On Wed, Jun 22, 2016 at 2:35 AM, M Gates <ga...@gmail.com> wrote:
>
> Hi
>
> Looking for some help with Oozie and Pig. I have the setup below but keep
> getting the error below: 'The application does not define formal
> parameters in its XML definition’.
>
>
> I have my test.pig script in hdfs with the workflow.xml - any idea what I
> am missing in the scripts below or what might cause this error?
>
> Thanks,
> Mark G.
>
>
> Workflow.xml
>
>
> <workflow-app name="customer_journey_event_app"
> xmlns="uri:oozie:workflow:0.4">
> <start to="test"/>
>
> <action name="test">
> <pig>
>
> <job-tracker>${jobTracker}</job-tracker>
>
> <name-node>${nameNode}</name-node>
> <prepare>
> <delete
> path="${hdfs_temp_dir}/pig"/>
> </prepare>
> <configuration>
> <property>
>
> <name>mapred.compress.map.output</name>
>
> <value>true</value>
> </property>
> <property>
>
> <name>mapred.job.queue.name</name>
>
> <value>${queueName}</value>
> </property>
> </configuration>
> <script>test.pig</script>
>
> <argument>-useHCatalog</argument>
> <argument>-param</argument>
>
> <argument>output=${hdfs_temp_dir}/pig</argument>
> </pig>
> <ok to="end"/>
> <error to="fail"/>
> </action>
>
> <kill name="fail">
> <message>Pig failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> </kill>
>
> <end name="end"/>
> </workflow-app>
>
>
> Coordinator.xml
>
>
> <coordinator-app name="${app_name}"
> frequency="${coord:days(1)}"
> start="${start_time}"
> end="${end_time}"
> timezone=“America/Montreal"
> xmlns="uri:oozie:coordinator:0.2">
> <controls>
> <concurrency>1</concurrency>
> <execution>FIFO</execution>
> </controls>
>
> <action>
> <workflow>
>
> <app-path>${app_path}/workflow</app-path>
> <configuration>
> <property>
>
> <name>nominal_time</name>
>
> <value>${coord:nominalTime()}</value>
> </property>
> </configuration>
> </workflow>
> </action>
> </coordinator-app>
>
>
> Log Output:
>
>
> 2016-06-21 19:33:16,766 INFO CoordMaterializeTransitionXCommand:543 -
> SERVER[hadoop.oss.ads] USER[markg] GROUP[-] TOKEN[]
> APP[customer_journey_event_app] JOB[0002430-160613201309706-oozie-oozi-C]
> ACTION[-] [0002430-160613201309706-oozie-oozi-C]: all actions have been
> materialized, set pending to true
> 2016-06-21 19:33:16,766 INFO CoordMaterializeTransitionXCommand:543 -
> SERVER[hadoop.oss.ads] USER[markg] GROUP[-] TOKEN[]
> APP[customer_journey_event_app] JOB[0002430-160613201309706-oozie-oozi-C]
> ACTION[-] Coord Job status updated to = RUNNING
> 2016-06-21 19:33:16,870 INFO CoordActionNotificationXCommand:543 -
> SERVER[hadoop.oss.ads] USER[-] GROUP[-] TOKEN[-] APP[-]
> JOB[0002430-160613201309706-oozie-oozi-C]
> ACTION[0002430-160613201309706-oozie-oozi-C@1] STARTED Coordinator
> Notification actionId=0002430-160613201309706-oozie-oozi-C@1 : WAITING
> 2016-06-21 19:33:16,871 INFO CoordActionNotificationXCommand:543 -
> SERVER[hadoop.oss.ads] USER[-] GROUP[-] TOKEN[-] APP[-]
> JOB[0002430-160613201309706-oozie-oozi-C]
> ACTION[0002430-160613201309706-oozie-oozi-C@1] No Notification URL is
> defined. Therefore nothing to notify for job
> 0002430-160613201309706-oozie-oozi-C action ID
> 0002430-160613201309706-oozie-oozi-C@1
> 2016-06-21 19:33:16,871 INFO CoordActionNotificationXCommand:543 -
> SERVER[hadoop.oss.ads] USER[-] GROUP[-] TOKEN[-] APP[-]
> JOB[0002430-160613201309706-oozie-oozi-C]
> ACTION[0002430-160613201309706-oozie-oozi-C@1] ENDED Coordinator
> Notification actionId=0002430-160613201309706-oozie-oozi-C@1
> 2016-06-21 19:33:34,573 INFO CoordActionInputCheckXCommand:543 -
> SERVER[hadoop.oss.ads] USER[-] GROUP[-] TOKEN[-] APP[-]
> JOB[0002430-160613201309706-oozie-oozi-C]
> ACTION[0002430-160613201309706-oozie-oozi-C@1]
> [0002430-160613201309706-oozie-oozi-C@1]::CoordActionInputCheck:: Missing
> deps:
> 2016-06-21 19:33:34,619 WARN ParameterVerifier:546 -
> SERVER[hadoop.oss.ads] USER[-] GROUP[-] TOKEN[-] APP[-]
> JOB[0002430-160613201309706-oozie-oozi-C]
> ACTION[0002430-160613201309706-oozie-oozi-C@1] The application does not
> define formal parameters in its XML definition
> 2016-06-21 19:33:54,696 INFO CoordActionUpdateXCommand:543 -
> SERVER[hadoop.oss.ads] USER[-] GROUP[-] TOKEN[-] APP[-]
> JOB[0002430-160613201309706-oozie-oozi-C]
> ACTION[0002430-160613201309706-oozie-oozi-C@1] Updating Coordintaor
> action id :0002430-160613201309706-oozie-oozi-C@1 status to KILLED,
> pending = 0
>
--
Peter Cseh
Software Engineer
<http://www.cloudera.com>