[jira] [Reopened] (OOZIE-2873) Check El Functions before submitting the coordinator

["Satish Subhashrao Saley (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/OOZIE-2873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Satish Subhashrao Saley reopened OOZIE-2873:
--------------------------------------------

reopening to revert the changes.

> Check El Functions before submitting the coordinator
> ----------------------------------------------------
>
>                 Key: OOZIE-2873
>                 URL: https://issues.apache.org/jira/browse/OOZIE-2873
>             Project: Oozie
>          Issue Type: Bug
>            Reporter: Satish Subhashrao Saley
>            Assignee: Satish Subhashrao Saley
>            Priority: Minor
>         Attachments: OOZIE-2873-1.patch
>
>
> Oozie doesn't check for el functions while submitting the coordinator job. Later on the coordinator action(s) can remain in WAITING state if there user has messed up the el functions.
> For Example.
> {code}
> <?xml version="1.0" encoding="UTF-8"?>
> <coordinator-app xmlns="uri:oozie:coordinator:0.4" name="my_coord" frequency="${coord:hours(1)}" start="${startTime}" end="${endTime}" timezone="UTC">
>     <controls>
>         <concurrency>1</concurrency>
>         <execution>FIFO</execution>
>     </controls>
>     <datasets>
>         <dataset name="my_dataset" frequency="${coord:hours(1)}" initial-instance="${initInstanceTime}" timezone="UTC">
>             <uri-template>hcat://${HCAT_SERVER}/${HCAT_DB_NAME}/${TABLE_NAME}/dt=${YEAR}${MONTH}${DAY};hr=${HOUR}</uri-template>
>         </dataset>
>     </datasets>
>     <input-events>
>         <data-in name="my_dataset_name" dataset="my_dataset">
>             <instance>${coord:current(0)}</instance>
>         </data-in>
>      
>     </input-events>
>     <action>
>         <workflow>
>             <app-path>${oozieAppWorkflowPath}/my_workflow.xml</app-path>
>             <configuration>
>                 <property>
>                     <name>yyyymmdd</name>
>                     <value>${coord:formatTime(coord:nominalTime(), 'DAY')}</value>
>                 </property>
>                 <property>
>                     <name>hh</name>
>                     <value>${coord:formatTime(coord:nominalTime(),'HH')}</value>
>                 </property>
>             </configuration>
>         </workflow>
>     </action>
> </coordinator-app>
> {code}
> After Oozie finds out the dependency.
> {code}
> 2017-04-25 16:51:53,503 DEBUG DependencyChecker:526 [pool-11-thread-66] - SERVER[localhost] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0100010100101-saley-C] ACTION[0100010100101-saley-C@1] Dependency [hcat://localhost:9098/my_database/my_table/dt=20170411;hr=02] is available
> {code}
> The issue is with el function
> {code}
> 2017-04-25 16:51:53,506 ERROR CoordPushDependencyCheckXCommand:517 [pool-11-thread-66] - SERVER[localhost] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0100010100101-saley-C] ACTION[0100010100101-saley-C@1] XException, 
> org.apache.oozie.command.CommandException: E1021: Coord Action Input Check Error: E1021: Coord Action Input Check Error: Unable to evaluate :${coord:formatTime(coord:nominalTime(), 'DAY')}:
> <configuration>
>                 <property>
>                     <name>yyyymmdd</name>
>                     <value>${coord:formatTime(coord:nominalTime(), 'DAY')}</value>
>                 </property>
> {code}
> The coord action remained in WAITING state.
> Solution:
> We should error out at the time of job submission. Currently users are supposed to run dry run on the coordinator before actually running it. But everybody wants to run directly. We should run dry run by default to catch such errors. While working the fix, I have found some buggy test cases which would have been caught if we run dry run first. Fixing those cases as well.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)