You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by Mona Chitnis <mo...@yahoo.in> on 2013/10/29 22:32:19 UTC
Re: Review Request 14239: OOZIE-1504 parameterize coord EL functions.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14239/#review27730
-----------------------------------------------------------
In addition to phase 2 (ph2_*) functions which do the actual resolution, add phase 1 (ph1_*) counterparts to echo the function name. This is required during initial submission and materialization phase.
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java
<https://reviews.apache.org/r/14239/#comment53852>
slightly confusing condition. Throw exception if the date is not absolute?
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java
<https://reviews.apache.org/r/14239/#comment53854>
Also print the startTime and endTime (indexes) here so we know what exactly threw the exception
http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/resources/cord-action-for-parameterization-1.xml
<https://reviews.apache.org/r/14239/#comment53856>
same comment as above
http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/resources/cord-action-for-parameterization-1.xml
<https://reviews.apache.org/r/14239/#comment53858>
dataset name='logs'
http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/resources/cord-action-for-parameterization-1.xml
<https://reviews.apache.org/r/14239/#comment53857>
should be dataIn('input') i.e. name given to the data-in
http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/resources/cord-action-for-parameterization-2.xml
<https://reviews.apache.org/r/14239/#comment53861>
same as above
http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/resources/cord-action-for-parameterization-2.xml
<https://reviews.apache.org/r/14239/#comment53860>
same comment as above
http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/resources/cord-action-for-parameterization-3.xml
<https://reviews.apache.org/r/14239/#comment53862>
same as above
http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/resources/cord-action-for-parameterization-3.xml
<https://reviews.apache.org/r/14239/#comment53867>
do you need two dataset definitions here? I see both are the same
http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/resources/cord-action-for-parameterization-3.xml
<https://reviews.apache.org/r/14239/#comment53863>
same comment as above
http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/resources/cord-action-for-parameterization-4.xml
<https://reviews.apache.org/r/14239/#comment53864>
same as above
http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/resources/cord-action-for-parameterization-4.xml
<https://reviews.apache.org/r/14239/#comment53865>
same as above
http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/resources/cord-action-for-parameterization.xml
<https://reviews.apache.org/r/14239/#comment53855>
maintain the original structure in this comment block, by reverting the auto format here
http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/resources/cord-action-for-parameterization.xml
<https://reviews.apache.org/r/14239/#comment53866>
do you need two dataset definitions here? I see both are the same
http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/resources/cord-action-for-parameterization.xml
<https://reviews.apache.org/r/14239/#comment53859>
should be dataIn('input') i.e. name given to the data-in
- Mona Chitnis
On Sept. 19, 2013, 10:20 p.m., Purshotam Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14239/
> -----------------------------------------------------------
>
> (Updated Sept. 19, 2013, 10:20 p.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1504
> https://issues.apache.org/jira/browse/OOZIE-1504
>
>
> Repository: oozie
>
>
> Description
> -------
>
> Use-case
> -----------------------------
> A workflow that processes input datasets for the past 30 days:
> <input-events>
> <data-in name="event_input_path_format1" dataset="EVENT_INPUT_FORMAT1">
> <start-instance>$
> {coord:current(-30)}
> </start-instance>
> <end-instance>$
> {coord:current(-1)}
> </end-instance>
> </data-in>
> <data-in name="event_input_path_format2" dataset="EVENT_INPUT_FORMAT2">
> <start-instance>$
> {coord:current(-30)}
> </start-instance>
> <end-instance>$
> {coord:current(-1)}
> </end-instance>
> </data-in>
> </input-events>
> Instead, one wants it to start on the same specific date so that each day, it
> processes one more data set than the last day (the datasets are daily). Something like the following is not supported
> <input-events>
> <data-in name="event_input_path_format1" dataset="EVENT_INPUT_FORMAT1">
> <start-instance>2013-03-15T00:00Z</start-instance>
> <end-instance>$
> {coord:current(-1)}
> </end-instance>
> </data-in>
> <data-in name="event_input_path_format2" dataset="EVENT_INPUT_FORMAT2">
> <start-instance>2013-03-15T00:00Z</start-instance>
> <end-instance>$
> {coord:current(-1)}
> </end-instance>
> </data-in>
> </input-events>
> ---------------------------------
>
>
> Fix support providing absolute value for instance range. One can have a combination of absolute date with coord:current().
> Other EL function are not supported with absolute date.
>
> coord:Current() can be parameterize.
>
> like
> <end-instance>${coord:latest(end_date)}</end-instance> // where end_date is one of the specified property.
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java 1523758
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java 1523758
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/resources/oozie-default.xml 1523758
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordCommandUtils.java 1523758
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordMaterializeTransitionXCommand.java 1523758
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/java/org/apache/oozie/coord/TestCoordELEvaluator.java 1523758
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/java/org/apache/oozie/coord/TestCoordELFunctions.java 1523758
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/resources/cord-action-for-parameterization-1.xml PRE-CREATION
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/resources/cord-action-for-parameterization-2.xml PRE-CREATION
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/resources/cord-action-for-parameterization-3.xml PRE-CREATION
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/resources/cord-action-for-parameterization-4.xml PRE-CREATION
> http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/resources/cord-action-for-parameterization.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/14239/diff/
>
>
> Testing
> -------
>
> Added multiple test cases to verify combination of scenarios.
>
>
> Thanks,
>
> Purshotam Shah
>
>