You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by Mona Chitnis <ch...@yahoo-inc.com> on 2011/12/07 23:58:35 UTC
Review Request: OOZIE-629 Oozie server to prevent usage of dataset
initial-instance earlier than the system-defined default value
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3068/
-----------------------------------------------------------
Review request for oozie, Mohammad Islam and Angelo K. Huang.
Summary
-------
Oozie server uses a system configurable property to define the default dataset initial-instance, i.e., UTC 1970-01-01T00:00Z.
Oozie will prevent launching job that mentions any dataset initial instance earlier than the system default (e.g. 1970-01-01T00:00Z). Firstly it makes for good sanity check, and secondly it will avoid the situation where it takes a really long time to compute and resolve dataset instances if the date is older than the above default one - the oldest date permitted.
This addresses bug OOZIE-629.
https://issues.apache.org/jira/browse/OOZIE-629
Diffs
-----
trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 1211626
trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordSubmitXCommand.java 1211626
Diff: https://reviews.apache.org/r/3068/diff
Testing
-------
yes
Thanks,
Mona
Re: Review Request: OOZIE-629 Oozie server to prevent usage of dataset
initial-instance earlier than the system-defined default value
Posted by Mona Chitnis <ch...@yahoo-inc.com>.
> On 2011-12-08 05:35:44, Santhosh Srinivasan wrote:
> > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java, line 724
> > <https://reviews.apache.org/r/3068/diff/1/?file=63040#file63040line724>
> >
> > Initialization to null is not required. By default, objects are initialized to null.
done
> On 2011-12-08 05:35:44, Santhosh Srinivasan wrote:
> > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java, line 732
> > <https://reviews.apache.org/r/3068/diff/1/?file=63040#file63040line732>
> >
> > A couple of comments:
> > 1. Why are you not throwing CoordinatorJobException?
> >
> > 2. The error message has to be more informative. Please include the givenInstance to help the user identify the source of the problem. Additional information about why parsing failed will make it complete.
1. This failure should be treated separately as this is caused by parsing error of the date and the other case we are testing for is comparison with a date in correct format. Throwing IllegalArgumentException here gets bubbled up as ErrorCode.1003 - invalid coordinator application attributes, which is more apt for the string that has parsing error. This will show the difference from CoordinatorJobException thrown by the comparison block which bubbles up as ErrorCode.1021. In both cases, the job is failed and does not get submitted.
- Mona
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3068/#review3730
-----------------------------------------------------------
On 2011-12-07 22:58:35, Mona Chitnis wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3068/
> -----------------------------------------------------------
>
> (Updated 2011-12-07 22:58:35)
>
>
> Review request for oozie, Mohammad Islam and Angelo K. Huang.
>
>
> Summary
> -------
>
> Oozie server uses a system configurable property to define the default dataset initial-instance, i.e., UTC 1970-01-01T00:00Z.
>
> Oozie will prevent launching job that mentions any dataset initial instance earlier than the system default (e.g. 1970-01-01T00:00Z). Firstly it makes for good sanity check, and secondly it will avoid the situation where it takes a really long time to compute and resolve dataset instances if the date is older than the above default one - the oldest date permitted.
>
>
> This addresses bug OOZIE-629.
> https://issues.apache.org/jira/browse/OOZIE-629
>
>
> Diffs
> -----
>
> trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 1211626
> trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordSubmitXCommand.java 1211626
>
> Diff: https://reviews.apache.org/r/3068/diff
>
>
> Testing
> -------
>
> yes
>
>
> Thanks,
>
> Mona
>
>
Re: Review Request: OOZIE-629 Oozie server to prevent usage of dataset
initial-instance earlier than the system-defined default value
Posted by Santhosh Srinivasan <sm...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3068/#review3730
-----------------------------------------------------------
trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java
<https://reviews.apache.org/r/3068/#comment8385>
Initialization to null is not required. By default, objects are initialized to null.
trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java
<https://reviews.apache.org/r/3068/#comment8381>
A couple of comments:
1. Why are you not throwing CoordinatorJobException?
2. The error message has to be more informative. Please include the givenInstance to help the user identify the source of the problem. Additional information about why parsing failed will make it complete.
trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordSubmitXCommand.java
<https://reviews.apache.org/r/3068/#comment8382>
Based on changes to the error message, the check here has to change.
- Santhosh
On 2011-12-07 22:58:35, Mona Chitnis wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3068/
> -----------------------------------------------------------
>
> (Updated 2011-12-07 22:58:35)
>
>
> Review request for oozie, Mohammad Islam and Angelo K. Huang.
>
>
> Summary
> -------
>
> Oozie server uses a system configurable property to define the default dataset initial-instance, i.e., UTC 1970-01-01T00:00Z.
>
> Oozie will prevent launching job that mentions any dataset initial instance earlier than the system default (e.g. 1970-01-01T00:00Z). Firstly it makes for good sanity check, and secondly it will avoid the situation where it takes a really long time to compute and resolve dataset instances if the date is older than the above default one - the oldest date permitted.
>
>
> This addresses bug OOZIE-629.
> https://issues.apache.org/jira/browse/OOZIE-629
>
>
> Diffs
> -----
>
> trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 1211626
> trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordSubmitXCommand.java 1211626
>
> Diff: https://reviews.apache.org/r/3068/diff
>
>
> Testing
> -------
>
> yes
>
>
> Thanks,
>
> Mona
>
>
Re: Review Request: OOZIE-629 Oozie server to prevent usage of dataset
initial-instance earlier than the system-defined default value
Posted by Mona Chitnis <ch...@yahoo-inc.com>.
> On 2011-12-09 07:01:35, Santhosh Srinivasan wrote:
> > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java, line 732
> > <https://reviews.apache.org/r/3068/diff/3/?file=63441#file63441line732>
> >
> > Instead of concatenating the exception message, chain the exception.
> >
> > throw new IllegalArgumentException("msg", e);
> >
> > The previous message "Unable to parse dataset initial-instance string " + val + " to Date object" was better phrased.
good point. done
- Mona
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3068/#review3776
-----------------------------------------------------------
On 2011-12-09 03:43:34, Mona Chitnis wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3068/
> -----------------------------------------------------------
>
> (Updated 2011-12-09 03:43:34)
>
>
> Review request for oozie, Mohammad Islam and Angelo K. Huang.
>
>
> Summary
> -------
>
> Oozie server uses a system configurable property to define the default dataset initial-instance, i.e., UTC 1970-01-01T00:00Z.
>
> Oozie will prevent launching job that mentions any dataset initial instance earlier than the system default (e.g. 1970-01-01T00:00Z). Firstly it makes for good sanity check, and secondly it will avoid the situation where it takes a really long time to compute and resolve dataset instances if the date is older than the above default one - the oldest date permitted.
>
>
> This addresses bug OOZIE-629.
> https://issues.apache.org/jira/browse/OOZIE-629
>
>
> Diffs
> -----
>
> trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 1212186
> trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordSubmitXCommand.java 1212186
> trunk/core/src/test/resources/coord-multiple-data-instances.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/3068/diff
>
>
> Testing
> -------
>
> yes
>
>
> Thanks,
>
> Mona
>
>
Re: Review Request: OOZIE-629 Oozie server to prevent usage of dataset
initial-instance earlier than the system-defined default value
Posted by Santhosh Srinivasan <sm...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3068/#review3776
-----------------------------------------------------------
Just one change required.
trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java
<https://reviews.apache.org/r/3068/#comment8549>
Instead of concatenating the exception message, chain the exception.
throw new IllegalArgumentException("msg", e);
The previous message "Unable to parse dataset initial-instance string " + val + " to Date object" was better phrased.
- Santhosh
On 2011-12-09 03:43:34, Mona Chitnis wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3068/
> -----------------------------------------------------------
>
> (Updated 2011-12-09 03:43:34)
>
>
> Review request for oozie, Mohammad Islam and Angelo K. Huang.
>
>
> Summary
> -------
>
> Oozie server uses a system configurable property to define the default dataset initial-instance, i.e., UTC 1970-01-01T00:00Z.
>
> Oozie will prevent launching job that mentions any dataset initial instance earlier than the system default (e.g. 1970-01-01T00:00Z). Firstly it makes for good sanity check, and secondly it will avoid the situation where it takes a really long time to compute and resolve dataset instances if the date is older than the above default one - the oldest date permitted.
>
>
> This addresses bug OOZIE-629.
> https://issues.apache.org/jira/browse/OOZIE-629
>
>
> Diffs
> -----
>
> trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 1212186
> trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordSubmitXCommand.java 1212186
> trunk/core/src/test/resources/coord-multiple-data-instances.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/3068/diff
>
>
> Testing
> -------
>
> yes
>
>
> Thanks,
>
> Mona
>
>
Re: Review Request: OOZIE-629 Oozie server to prevent usage of dataset
initial-instance earlier than the system-defined default value
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3068/#review3832
-----------------------------------------------------------
Ship it!
+1
Committed.
- Mohammad
On 2011-12-11 02:33:47, Mona Chitnis wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3068/
> -----------------------------------------------------------
>
> (Updated 2011-12-11 02:33:47)
>
>
> Review request for oozie, Mohammad Islam and Angelo K. Huang.
>
>
> Summary
> -------
>
> Oozie server uses a system configurable property to define the default dataset initial-instance, i.e., UTC 1970-01-01T00:00Z.
>
> Oozie will prevent launching job that mentions any dataset initial instance earlier than the system default (e.g. 1970-01-01T00:00Z). Firstly it makes for good sanity check, and secondly it will avoid the situation where it takes a really long time to compute and resolve dataset instances if the date is older than the above default one - the oldest date permitted.
>
>
> This addresses bug OOZIE-629.
> https://issues.apache.org/jira/browse/OOZIE-629
>
>
> Diffs
> -----
>
> trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordSubmitXCommand.java 1212959
> trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 1212959
> trunk/core/src/test/resources/coord-dataset-initial-instance.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/3068/diff
>
>
> Testing
> -------
>
> yes
>
>
> Thanks,
>
> Mona
>
>
Re: Review Request: OOZIE-629 Oozie server to prevent usage of dataset
initial-instance earlier than the system-defined default value
Posted by Mona Chitnis <ch...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3068/
-----------------------------------------------------------
(Updated 2011-12-11 02:33:47.445709)
Review request for oozie, Mohammad Islam and Angelo K. Huang.
Changes
-------
diff after updating repo to latest version. Moved out the logic block in CoordSubmitXCommand to a separate method.
Summary
-------
Oozie server uses a system configurable property to define the default dataset initial-instance, i.e., UTC 1970-01-01T00:00Z.
Oozie will prevent launching job that mentions any dataset initial instance earlier than the system default (e.g. 1970-01-01T00:00Z). Firstly it makes for good sanity check, and secondly it will avoid the situation where it takes a really long time to compute and resolve dataset instances if the date is older than the above default one - the oldest date permitted.
This addresses bug OOZIE-629.
https://issues.apache.org/jira/browse/OOZIE-629
Diffs (updated)
-----
trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordSubmitXCommand.java 1212959
trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 1212959
trunk/core/src/test/resources/coord-dataset-initial-instance.xml PRE-CREATION
Diff: https://reviews.apache.org/r/3068/diff
Testing
-------
yes
Thanks,
Mona
Re: Review Request: OOZIE-629 Oozie server to prevent usage of dataset
initial-instance earlier than the system-defined default value
Posted by Mona Chitnis <ch...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3068/
-----------------------------------------------------------
(Updated 2011-12-09 19:04:04.148510)
Review request for oozie, Mohammad Islam and Angelo K. Huang.
Changes
-------
fixed typo in test case and the name of the test resource file
Summary
-------
Oozie server uses a system configurable property to define the default dataset initial-instance, i.e., UTC 1970-01-01T00:00Z.
Oozie will prevent launching job that mentions any dataset initial instance earlier than the system default (e.g. 1970-01-01T00:00Z). Firstly it makes for good sanity check, and secondly it will avoid the situation where it takes a really long time to compute and resolve dataset instances if the date is older than the above default one - the oldest date permitted.
This addresses bug OOZIE-629.
https://issues.apache.org/jira/browse/OOZIE-629
Diffs (updated)
-----
trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 1212186
trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordSubmitXCommand.java 1212186
trunk/core/src/test/resources/coord-dataset-initial-instance.xml PRE-CREATION
Diff: https://reviews.apache.org/r/3068/diff
Testing
-------
yes
Thanks,
Mona
Re: Review Request: OOZIE-629 Oozie server to prevent usage of dataset
initial-instance earlier than the system-defined default value
Posted by Mona Chitnis <ch...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3068/
-----------------------------------------------------------
(Updated 2011-12-09 18:04:12.555575)
Review request for oozie, Mohammad Islam and Angelo K. Huang.
Changes
-------
review feedback incorporated
Summary
-------
Oozie server uses a system configurable property to define the default dataset initial-instance, i.e., UTC 1970-01-01T00:00Z.
Oozie will prevent launching job that mentions any dataset initial instance earlier than the system default (e.g. 1970-01-01T00:00Z). Firstly it makes for good sanity check, and secondly it will avoid the situation where it takes a really long time to compute and resolve dataset instances if the date is older than the above default one - the oldest date permitted.
This addresses bug OOZIE-629.
https://issues.apache.org/jira/browse/OOZIE-629
Diffs (updated)
-----
trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 1212186
trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordSubmitXCommand.java 1212186
trunk/core/src/test/resources/coord-multiple-data-instances.xml PRE-CREATION
Diff: https://reviews.apache.org/r/3068/diff
Testing
-------
yes
Thanks,
Mona
Re: Review Request: OOZIE-629 Oozie server to prevent usage of dataset
initial-instance earlier than the system-defined default value
Posted by Mona Chitnis <ch...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3068/
-----------------------------------------------------------
(Updated 2011-12-09 03:43:34.262831)
Review request for oozie, Mohammad Islam and Angelo K. Huang.
Changes
-------
removed accidental whitespaces in the xml resource file
Summary
-------
Oozie server uses a system configurable property to define the default dataset initial-instance, i.e., UTC 1970-01-01T00:00Z.
Oozie will prevent launching job that mentions any dataset initial instance earlier than the system default (e.g. 1970-01-01T00:00Z). Firstly it makes for good sanity check, and secondly it will avoid the situation where it takes a really long time to compute and resolve dataset instances if the date is older than the above default one - the oldest date permitted.
This addresses bug OOZIE-629.
https://issues.apache.org/jira/browse/OOZIE-629
Diffs (updated)
-----
trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 1212186
trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordSubmitXCommand.java 1212186
trunk/core/src/test/resources/coord-multiple-data-instances.xml PRE-CREATION
Diff: https://reviews.apache.org/r/3068/diff
Testing
-------
yes
Thanks,
Mona
Re: Review Request: OOZIE-629 Oozie server to prevent usage of dataset
initial-instance earlier than the system-defined default value
Posted by Mona Chitnis <ch...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3068/
-----------------------------------------------------------
(Updated 2011-12-09 03:41:19.379953)
Review request for oozie, Mohammad Islam and Angelo K. Huang.
Summary
-------
Oozie server uses a system configurable property to define the default dataset initial-instance, i.e., UTC 1970-01-01T00:00Z.
Oozie will prevent launching job that mentions any dataset initial instance earlier than the system default (e.g. 1970-01-01T00:00Z). Firstly it makes for good sanity check, and secondly it will avoid the situation where it takes a really long time to compute and resolve dataset instances if the date is older than the above default one - the oldest date permitted.
This addresses bug OOZIE-629.
https://issues.apache.org/jira/browse/OOZIE-629
Diffs (updated)
-----
trunk/core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 1212186
trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordSubmitXCommand.java 1212186
trunk/core/src/test/resources/coord-multiple-data-instances.xml PRE-CREATION
Diff: https://reviews.apache.org/r/3068/diff
Testing
-------
yes
Thanks,
Mona