You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Mona Chitnis (JIRA)" <ji...@apache.org> on 2012/10/09 00:02:02 UTC
[jira] [Created] (OOZIE-1015) HadoopAccessorService jobtracker
validation should not have hardcoded conf key
Mona Chitnis created OOZIE-1015:
-----------------------------------
Summary: HadoopAccessorService jobtracker validation should not have hardcoded conf key
Key: OOZIE-1015
URL: https://issues.apache.org/jira/browse/OOZIE-1015
Project: Oozie
Issue Type: Bug
Reporter: Mona Chitnis
While validating the jobtracker address against whitelist in HadoopAccessorService, the conf key to be looked up is a string value "mapred.job.tracker". Instead we should use static variables for robustness (mapred.job.tracker for hadoop-1 and yarn.resourcemanager.address for hadoop-2)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1015) HadoopAccessorService jobtracker
validation should not have hardcoded conf key
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13482138#comment-13482138 ]
Hadoop QA commented on OOZIE-1015:
----------------------------------
Testing JIRA OOZIE-1015
Cleaning local svn workspace
----------------------------
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
. {color:green}+1{color} the patch does not introduce any @author tags
. {color:green}+1{color} the patch does not introduce any tabs
. {color:green}+1{color} the patch does not introduce any trailing spaces
. {color:green}+1{color} the patch does not introduce any line longer than 132
. {color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
. {color:green}+1{color} the patch does not seem to introduce new RAT warnings
{color:green}+1 JAVADOC{color}
. {color:green}+1{color} the patch does not seem to introduce new Javadoc warnings
{color:green}+1 COMPILE{color}
. {color:green}+1{color} HEAD compiles
. {color:green}+1{color} patch compiles
. {color:green}+1{color} the patch does not seem to introduce new javac warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
. {color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
. {color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
. Tests run: 913
{color:green}+1 DISTRO{color}
. {color:green}+1{color} distro tarball builds with the patch
----------------------------
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/154/
> HadoopAccessorService jobtracker validation should not have hardcoded conf key
> ------------------------------------------------------------------------------
>
> Key: OOZIE-1015
> URL: https://issues.apache.org/jira/browse/OOZIE-1015
> Project: Oozie
> Issue Type: Bug
> Affects Versions: 3.3.0
> Reporter: Mona Chitnis
> Assignee: Mona Chitnis
> Fix For: trunk
>
> Attachments: OOZIE-1015.patch
>
>
> While validating the jobtracker address against whitelist in HadoopAccessorService, the conf key to be looked up is a string value "mapred.job.tracker". Instead we should use static variables for robustness (mapred.job.tracker for hadoop-1 and yarn.resourcemanager.address for hadoop-2)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1015) HadoopAccessorService jobtracker
validation should not have hardcoded conf key
Posted by "Mona Chitnis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13491976#comment-13491976 ]
Mona Chitnis commented on OOZIE-1015:
-------------------------------------
new testcase n/a since this is only variable change
> HadoopAccessorService jobtracker validation should not have hardcoded conf key
> ------------------------------------------------------------------------------
>
> Key: OOZIE-1015
> URL: https://issues.apache.org/jira/browse/OOZIE-1015
> Project: Oozie
> Issue Type: Bug
> Affects Versions: 3.3.0
> Reporter: Mona Chitnis
> Assignee: Mona Chitnis
> Fix For: trunk
>
> Attachments: OOZIE-1015.patch
>
>
> While validating the jobtracker address against whitelist in HadoopAccessorService, the conf key to be looked up is a string value "mapred.job.tracker". Instead we should use static variables for robustness (mapred.job.tracker for hadoop-1 and yarn.resourcemanager.address for hadoop-2)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1015) HadoopAccessorService jobtracker
validation should not have hardcoded conf key
Posted by "Mona Chitnis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504994#comment-13504994 ]
Mona Chitnis commented on OOZIE-1015:
-------------------------------------
existing unit tests all pass, and new test n/a. Committing patch.
> HadoopAccessorService jobtracker validation should not have hardcoded conf key
> ------------------------------------------------------------------------------
>
> Key: OOZIE-1015
> URL: https://issues.apache.org/jira/browse/OOZIE-1015
> Project: Oozie
> Issue Type: Bug
> Affects Versions: 3.3.0
> Reporter: Mona Chitnis
> Assignee: Mona Chitnis
> Fix For: trunk
>
> Attachments: OOZIE-1015.patch, OOZIE-1015v2.patch
>
>
> While validating the jobtracker address against whitelist in HadoopAccessorService, the conf key to be looked up is a string value "mapred.job.tracker". Instead we should use static variables for robustness (mapred.job.tracker for hadoop-1 and yarn.resourcemanager.address for hadoop-2)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1015) HadoopAccessorService jobtracker
validation should not have hardcoded conf key
Posted by "Mona Chitnis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494285#comment-13494285 ]
Mona Chitnis commented on OOZIE-1015:
-------------------------------------
Thanks Harsh,
1. Only difference is HADOOP_JOB_TRACKER (and _2) in JAE are private. Changing those to public seems ok right? Then better not to bring in any confusing XOozieClient parameters.
2. Seems doable to me. Will assess impact in other places in code and will need end-to-end test just to be sure.
> HadoopAccessorService jobtracker validation should not have hardcoded conf key
> ------------------------------------------------------------------------------
>
> Key: OOZIE-1015
> URL: https://issues.apache.org/jira/browse/OOZIE-1015
> Project: Oozie
> Issue Type: Bug
> Affects Versions: 3.3.0
> Reporter: Mona Chitnis
> Assignee: Mona Chitnis
> Fix For: trunk
>
> Attachments: OOZIE-1015.patch
>
>
> While validating the jobtracker address against whitelist in HadoopAccessorService, the conf key to be looked up is a string value "mapred.job.tracker". Instead we should use static variables for robustness (mapred.job.tracker for hadoop-1 and yarn.resourcemanager.address for hadoop-2)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OOZIE-1015) HadoopAccessorService jobtracker
validation should not have hardcoded conf key
Posted by "Mona Chitnis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mona Chitnis updated OOZIE-1015:
--------------------------------
Attachment: OOZIE-1015.patch
> HadoopAccessorService jobtracker validation should not have hardcoded conf key
> ------------------------------------------------------------------------------
>
> Key: OOZIE-1015
> URL: https://issues.apache.org/jira/browse/OOZIE-1015
> Project: Oozie
> Issue Type: Bug
> Affects Versions: 3.3.0
> Reporter: Mona Chitnis
> Assignee: Mona Chitnis
> Fix For: trunk
>
> Attachments: OOZIE-1015.patch
>
>
> While validating the jobtracker address against whitelist in HadoopAccessorService, the conf key to be looked up is a string value "mapred.job.tracker". Instead we should use static variables for robustness (mapred.job.tracker for hadoop-1 and yarn.resourcemanager.address for hadoop-2)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1015) HadoopAccessorService jobtracker
validation should not have hardcoded conf key
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492515#comment-13492515 ]
Harsh J commented on OOZIE-1015:
--------------------------------
Patch looks OK to me. I just have a few mostly off-topic questions:
# What is the difference between using XOozieClient.JT in the AccessorService vs. constant HADOOP_JOB_TRACKER (and _2?) in JavaActionExecutor?
# Also, the renewer string we pass for tokens, "mr token", perhaps we can make it "oozie mr token" for better traceability, if thats a compatible change to make?
> HadoopAccessorService jobtracker validation should not have hardcoded conf key
> ------------------------------------------------------------------------------
>
> Key: OOZIE-1015
> URL: https://issues.apache.org/jira/browse/OOZIE-1015
> Project: Oozie
> Issue Type: Bug
> Affects Versions: 3.3.0
> Reporter: Mona Chitnis
> Assignee: Mona Chitnis
> Fix For: trunk
>
> Attachments: OOZIE-1015.patch
>
>
> While validating the jobtracker address against whitelist in HadoopAccessorService, the conf key to be looked up is a string value "mapred.job.tracker". Instead we should use static variables for robustness (mapred.job.tracker for hadoop-1 and yarn.resourcemanager.address for hadoop-2)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OOZIE-1015) HadoopAccessorService jobtracker
validation should not have hardcoded conf key
Posted by "Mona Chitnis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mona Chitnis updated OOZIE-1015:
--------------------------------
Attachment: OOZIE-1015v2.patch
Incorporated Harsh's comments
> HadoopAccessorService jobtracker validation should not have hardcoded conf key
> ------------------------------------------------------------------------------
>
> Key: OOZIE-1015
> URL: https://issues.apache.org/jira/browse/OOZIE-1015
> Project: Oozie
> Issue Type: Bug
> Affects Versions: 3.3.0
> Reporter: Mona Chitnis
> Assignee: Mona Chitnis
> Fix For: trunk
>
> Attachments: OOZIE-1015.patch, OOZIE-1015v2.patch
>
>
> While validating the jobtracker address against whitelist in HadoopAccessorService, the conf key to be looked up is a string value "mapred.job.tracker". Instead we should use static variables for robustness (mapred.job.tracker for hadoop-1 and yarn.resourcemanager.address for hadoop-2)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1015) HadoopAccessorService jobtracker
validation should not have hardcoded conf key
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13495928#comment-13495928 ]
Hadoop QA commented on OOZIE-1015:
----------------------------------
Testing JIRA OOZIE-1015
Cleaning local svn workspace
----------------------------
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
. {color:green}+1{color} the patch does not introduce any @author tags
. {color:green}+1{color} the patch does not introduce any tabs
. {color:green}+1{color} the patch does not introduce any trailing spaces
. {color:red}-1{color} the patch contains 1 line(s) longer than 132 characters
. {color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
. {color:green}+1{color} the patch does not seem to introduce new RAT warnings
{color:green}+1 JAVADOC{color}
. {color:green}+1{color} the patch does not seem to introduce new Javadoc warnings
{color:green}+1 COMPILE{color}
. {color:green}+1{color} HEAD compiles
. {color:green}+1{color} patch compiles
. {color:green}+1{color} the patch does not seem to introduce new javac warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
. {color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
. {color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
. Tests run: 916
{color:green}+1 DISTRO{color}
. {color:green}+1{color} distro tarball builds with the patch
----------------------------
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/174/
> HadoopAccessorService jobtracker validation should not have hardcoded conf key
> ------------------------------------------------------------------------------
>
> Key: OOZIE-1015
> URL: https://issues.apache.org/jira/browse/OOZIE-1015
> Project: Oozie
> Issue Type: Bug
> Affects Versions: 3.3.0
> Reporter: Mona Chitnis
> Assignee: Mona Chitnis
> Fix For: trunk
>
> Attachments: OOZIE-1015.patch, OOZIE-1015v2.patch
>
>
> While validating the jobtracker address against whitelist in HadoopAccessorService, the conf key to be looked up is a string value "mapred.job.tracker". Instead we should use static variables for robustness (mapred.job.tracker for hadoop-1 and yarn.resourcemanager.address for hadoop-2)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (OOZIE-1015) HadoopAccessorService jobtracker
validation should not have hardcoded conf key
Posted by "Mona Chitnis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mona Chitnis reassigned OOZIE-1015:
-----------------------------------
Assignee: Mona Chitnis
> HadoopAccessorService jobtracker validation should not have hardcoded conf key
> ------------------------------------------------------------------------------
>
> Key: OOZIE-1015
> URL: https://issues.apache.org/jira/browse/OOZIE-1015
> Project: Oozie
> Issue Type: Bug
> Reporter: Mona Chitnis
> Assignee: Mona Chitnis
>
> While validating the jobtracker address against whitelist in HadoopAccessorService, the conf key to be looked up is a string value "mapred.job.tracker". Instead we should use static variables for robustness (mapred.job.tracker for hadoop-1 and yarn.resourcemanager.address for hadoop-2)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira