You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Rohini Palaniswamy (JIRA)" <ji...@apache.org> on 2012/11/16 20:38:12 UTC
[jira] [Created] (OOZIE-1073) Optimize latest and future EL
resolution in case of start-instance and end-instance
Rohini Palaniswamy created OOZIE-1073:
-----------------------------------------
Summary: Optimize latest and future EL resolution in case of start-instance and end-instance
Key: OOZIE-1073
URL: https://issues.apache.org/jira/browse/OOZIE-1073
Project: Oozie
Issue Type: Improvement
Reporter: Rohini Palaniswamy
<start-instance>${coord:latest(-23)}</start-instance>
<end-instance>${coord:latest(0)}</end-instance>
Assuming all 24 instances are available, Oozie makes 300(1+2+3+..24) calls instead of 24 calls. This leads to some very high CPU usage when number of latest instances is high.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OOZIE-1073) Optimize latest and future EL
resolution in case of start-instance and end-instance
Posted by "Rohini Palaniswamy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rohini Palaniswamy updated OOZIE-1073:
--------------------------------------
Attachment: OOZIE-1073-1.patch
Attaching initial patch. Newly added tests testActionInputCheckLatest and testActionInputCheckFuture in TestCoordActionInputCheckXCommand fail for the extending test class TestCoordActionInputCheckXCommandNonUTC. This is not due to the newly added code but that of the existing behaviour. Waiting for a clarification from [~tucu00] on what is the expected behaviour in case of latest and future in case of a NonUTC oozie timezone.
> Optimize latest and future EL resolution in case of start-instance and end-instance
> -----------------------------------------------------------------------------------
>
> Key: OOZIE-1073
> URL: https://issues.apache.org/jira/browse/OOZIE-1073
> Project: Oozie
> Issue Type: Improvement
> Reporter: Rohini Palaniswamy
> Attachments: OOZIE-1073-1.patch
>
>
> <start-instance>${coord:latest(-23)}</start-instance>
> <end-instance>${coord:latest(0)}</end-instance>
> Assuming all 24 instances are available, Oozie makes 300(1+2+3+..24) calls instead of 24 calls. This leads to some very high CPU usage when number of latest instances is high.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1073) Optimize latest and future EL
resolution in case of start-instance and end-instance
Posted by "Rohini Palaniswamy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506900#comment-13506900 ]
Rohini Palaniswamy commented on OOZIE-1073:
-------------------------------------------
Review request in https://reviews.apache.org/r/8283/. Removed the patch with fix for bug in latest and future when the oozie processing timezone was different than UTC. Handling that in a separate jira - OOZIE-1106.
> Optimize latest and future EL resolution in case of start-instance and end-instance
> -----------------------------------------------------------------------------------
>
> Key: OOZIE-1073
> URL: https://issues.apache.org/jira/browse/OOZIE-1073
> Project: Oozie
> Issue Type: Improvement
> Reporter: Rohini Palaniswamy
> Attachments: OOZIE-1073-1.patch
>
>
> <start-instance>${coord:latest(-23)}</start-instance>
> <end-instance>${coord:latest(0)}</end-instance>
> Assuming all 24 instances are available, Oozie makes 300(1+2+3+..24) calls instead of 24 calls. This leads to some very high CPU usage when number of latest instances is high.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OOZIE-1073) Optimize latest and future EL
resolution in case of start-instance and end-instance
Posted by "Rohini Palaniswamy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rohini Palaniswamy updated OOZIE-1073:
--------------------------------------
Attachment: OOZIE-1073-3.patch
Incorporated Virag's review comments
> Optimize latest and future EL resolution in case of start-instance and end-instance
> -----------------------------------------------------------------------------------
>
> Key: OOZIE-1073
> URL: https://issues.apache.org/jira/browse/OOZIE-1073
> Project: Oozie
> Issue Type: Improvement
> Affects Versions: 3.2.0
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: trunk
>
> Attachments: OOZIE-1073-1.patch, OOZIE-1073-2.patch, OOZIE-1073-3.patch
>
>
> <start-instance>${coord:latest(-23)}</start-instance>
> <end-instance>${coord:latest(0)}</end-instance>
> Assuming all 24 instances are available, Oozie makes 300(1+2+3+..24) calls instead of 24 calls. This leads to some very high CPU usage when number of latest instances is high.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OOZIE-1073) Optimize latest and future EL
resolution in case of start-instance and end-instance
Posted by "Rohini Palaniswamy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rohini Palaniswamy updated OOZIE-1073:
--------------------------------------
Attachment: OOZIE-1073-2.patch
Updated patch that fixes the bug in latest and future when Oozie processing time zone is different than UTC
> Optimize latest and future EL resolution in case of start-instance and end-instance
> -----------------------------------------------------------------------------------
>
> Key: OOZIE-1073
> URL: https://issues.apache.org/jira/browse/OOZIE-1073
> Project: Oozie
> Issue Type: Improvement
> Reporter: Rohini Palaniswamy
> Attachments: OOZIE-1073-1.patch, OOZIE-1073-2.patch
>
>
> <start-instance>${coord:latest(-23)}</start-instance>
> <end-instance>${coord:latest(0)}</end-instance>
> Assuming all 24 instances are available, Oozie makes 300(1+2+3+..24) calls instead of 24 calls. This leads to some very high CPU usage when number of latest instances is high.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1073) Optimize latest and future EL
resolution in case of start-instance and end-instance
Posted by "Virag Kothari (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510008#comment-13510008 ]
Virag Kothari commented on OOZIE-1073:
--------------------------------------
+1..Committed to trunk and hcat-intre
> Optimize latest and future EL resolution in case of start-instance and end-instance
> -----------------------------------------------------------------------------------
>
> Key: OOZIE-1073
> URL: https://issues.apache.org/jira/browse/OOZIE-1073
> Project: Oozie
> Issue Type: Improvement
> Affects Versions: 3.2.0
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: trunk
>
> Attachments: OOZIE-1073-1.patch, OOZIE-1073-2.patch, OOZIE-1073-3.patch
>
>
> <start-instance>${coord:latest(-23)}</start-instance>
> <end-instance>${coord:latest(0)}</end-instance>
> Assuming all 24 instances are available, Oozie makes 300(1+2+3+..24) calls instead of 24 calls. This leads to some very high CPU usage when number of latest instances is high.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1073) Optimize latest and future EL
resolution in case of start-instance and end-instance
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507410#comment-13507410 ]
Hadoop QA commented on OOZIE-1073:
----------------------------------
Testing JIRA OOZIE-1073
Cleaning local svn workspace
----------------------------
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
. {color:green}+1{color} the patch does not introduce any @author tags
. {color:green}+1{color} the patch does not introduce any tabs
. {color:green}+1{color} the patch does not introduce any trailing spaces
. {color:green}+1{color} the patch does not introduce any line longer than 132
. {color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
. {color:green}+1{color} the patch does not seem to introduce new RAT warnings
{color:green}+1 JAVADOC{color}
. {color:green}+1{color} the patch does not seem to introduce new Javadoc warnings
{color:green}+1 COMPILE{color}
. {color:green}+1{color} HEAD compiles
. {color:green}+1{color} patch compiles
. {color:green}+1{color} the patch does not seem to introduce new javac warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
. {color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
. {color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
. Tests run: 929
{color:green}+1 DISTRO{color}
. {color:green}+1{color} distro tarball builds with the patch
----------------------------
{color:green}*+1 Overall result, good!, no -1s*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/223/
> Optimize latest and future EL resolution in case of start-instance and end-instance
> -----------------------------------------------------------------------------------
>
> Key: OOZIE-1073
> URL: https://issues.apache.org/jira/browse/OOZIE-1073
> Project: Oozie
> Issue Type: Improvement
> Affects Versions: 3.2.0
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: trunk
>
> Attachments: OOZIE-1073-1.patch, OOZIE-1073-2.patch
>
>
> <start-instance>${coord:latest(-23)}</start-instance>
> <end-instance>${coord:latest(0)}</end-instance>
> Assuming all 24 instances are available, Oozie makes 300(1+2+3+..24) calls instead of 24 calls. This leads to some very high CPU usage when number of latest instances is high.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1073) Optimize latest and future EL
resolution in case of start-instance and end-instance
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507033#comment-13507033 ]
Hadoop QA commented on OOZIE-1073:
----------------------------------
Testing JIRA OOZIE-1073
Cleaning local svn workspace
----------------------------
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
. {color:green}+1{color} the patch does not introduce any @author tags
. {color:green}+1{color} the patch does not introduce any tabs
. {color:green}+1{color} the patch does not introduce any trailing spaces
. {color:red}-1{color} the patch contains 3 line(s) longer than 132 characters
. {color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
. {color:green}+1{color} the patch does not seem to introduce new RAT warnings
{color:green}+1 JAVADOC{color}
. {color:green}+1{color} the patch does not seem to introduce new Javadoc warnings
{color:green}+1 COMPILE{color}
. {color:green}+1{color} HEAD compiles
. {color:green}+1{color} patch compiles
. {color:green}+1{color} the patch does not seem to introduce new javac warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
. {color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
. {color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
. Tests run: 926
{color:green}+1 DISTRO{color}
. {color:green}+1{color} distro tarball builds with the patch
----------------------------
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/220/
> Optimize latest and future EL resolution in case of start-instance and end-instance
> -----------------------------------------------------------------------------------
>
> Key: OOZIE-1073
> URL: https://issues.apache.org/jira/browse/OOZIE-1073
> Project: Oozie
> Issue Type: Improvement
> Affects Versions: 3.2.0
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: trunk
>
> Attachments: OOZIE-1073-1.patch
>
>
> <start-instance>${coord:latest(-23)}</start-instance>
> <end-instance>${coord:latest(0)}</end-instance>
> Assuming all 24 instances are available, Oozie makes 300(1+2+3+..24) calls instead of 24 calls. This leads to some very high CPU usage when number of latest instances is high.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OOZIE-1073) Optimize latest and future EL
resolution in case of start-instance and end-instance
Posted by "Rohini Palaniswamy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rohini Palaniswamy updated OOZIE-1073:
--------------------------------------
Attachment: OOZIE-1073-2.patch
Updated patch fixing Raw patch analysis error.
> Optimize latest and future EL resolution in case of start-instance and end-instance
> -----------------------------------------------------------------------------------
>
> Key: OOZIE-1073
> URL: https://issues.apache.org/jira/browse/OOZIE-1073
> Project: Oozie
> Issue Type: Improvement
> Affects Versions: 3.2.0
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: trunk
>
> Attachments: OOZIE-1073-1.patch, OOZIE-1073-2.patch
>
>
> <start-instance>${coord:latest(-23)}</start-instance>
> <end-instance>${coord:latest(0)}</end-instance>
> Assuming all 24 instances are available, Oozie makes 300(1+2+3+..24) calls instead of 24 calls. This leads to some very high CPU usage when number of latest instances is high.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OOZIE-1073) Optimize latest and future EL
resolution in case of start-instance and end-instance
Posted by "Rohini Palaniswamy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rohini Palaniswamy updated OOZIE-1073:
--------------------------------------
Attachment: (was: OOZIE-1073-2.patch)
> Optimize latest and future EL resolution in case of start-instance and end-instance
> -----------------------------------------------------------------------------------
>
> Key: OOZIE-1073
> URL: https://issues.apache.org/jira/browse/OOZIE-1073
> Project: Oozie
> Issue Type: Improvement
> Reporter: Rohini Palaniswamy
> Attachments: OOZIE-1073-1.patch
>
>
> <start-instance>${coord:latest(-23)}</start-instance>
> <end-instance>${coord:latest(0)}</end-instance>
> Assuming all 24 instances are available, Oozie makes 300(1+2+3+..24) calls instead of 24 calls. This leads to some very high CPU usage when number of latest instances is high.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira