You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by "Eric Yang (JIRA)" <ji...@apache.org> on 2009/05/07 04:29:30 UTC
[jira] Created: (CHUKWA-216) postProcess.sh should locate
hadoop-site.xml in HADOOP_CONF_DIR
postProcess.sh should locate hadoop-site.xml in HADOOP_CONF_DIR
---------------------------------------------------------------
Key: CHUKWA-216
URL: https://issues.apache.org/jira/browse/CHUKWA-216
Project: Hadoop Chukwa
Issue Type: Bug
Components: Data Processors
Environment: Redhat EL 5.1, Java 6
Reporter: Eric Yang
Priority: Blocker
postProcess.sh is currently hard coded to CHUKWA_HOME/conf as class path for locating configuration.
postProcessManager should locate name node address and job tracker address from HADOOP_CONF_DIR or class path.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CHUKWA-216) postProcess.sh should
locate hadoop-site.xml in HADOOP_CONF_DIR
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708602#action_12708602 ]
Jerome Boulon edited comment on CHUKWA-216 at 5/12/09 2:50 PM:
---------------------------------------------------------------
>>PostProcess.sh is currently hard coded to CHUKWA_HOME/conf as class path for locating configuration.
This is a duplicate of CHUKWA-215
was (Author: jboulon):
>>PostProcess.sh is currently hard coded to CHUKWA_HOME/conf as class path for locating configuration.
This is a duplicate of CHUKWA-216
> postProcess.sh should locate hadoop-site.xml in HADOOP_CONF_DIR
> ---------------------------------------------------------------
>
> Key: CHUKWA-216
> URL: https://issues.apache.org/jira/browse/CHUKWA-216
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Jerome Boulon
> Priority: Blocker
>
> postProcess.sh is currently hard coded to CHUKWA_HOME/conf as class path for locating configuration.
> postProcessManager should locate name node address and job tracker address from HADOOP_CONF_DIR or class path.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-216) postProcess.sh should locate
hadoop-site.xml in HADOOP_CONF_DIR
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708618#action_12708618 ]
Eric Yang commented on CHUKWA-216:
----------------------------------
This code should be deprecated because it's incompatible going forward with newer version of hadoop. The proper fix is to ensure HADOOP_CONF_DIR as part of the class path, hence, we do not need to maintain any change for hadoop configuration change. namenode configuration should be added by the Configuration object itself rather than source hadoop-site.xml externally for all backend services.
> postProcess.sh should locate hadoop-site.xml in HADOOP_CONF_DIR
> ---------------------------------------------------------------
>
> Key: CHUKWA-216
> URL: https://issues.apache.org/jira/browse/CHUKWA-216
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Jerome Boulon
> Priority: Blocker
>
> postProcess.sh is currently hard coded to CHUKWA_HOME/conf as class path for locating configuration.
> postProcessManager should locate name node address and job tracker address from HADOOP_CONF_DIR or class path.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-216) postProcess.sh should locate
hadoop-site.xml in HADOOP_CONF_DIR
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated CHUKWA-216:
-----------------------------
Fix Version/s: 0.2.0
Affects Version/s: 0.2.0
0.1.2
> postProcess.sh should locate hadoop-site.xml in HADOOP_CONF_DIR
> ---------------------------------------------------------------
>
> Key: CHUKWA-216
> URL: https://issues.apache.org/jira/browse/CHUKWA-216
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Affects Versions: 0.1.2, 0.2.0
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Jerome Boulon
> Priority: Blocker
> Fix For: 0.2.0
>
> Attachments: CHUKWA-216.patch
>
>
> postProcess.sh is currently hard coded to CHUKWA_HOME/conf as class path for locating configuration.
> postProcessManager should locate name node address and job tracker address from HADOOP_CONF_DIR or class path.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-216) postProcess.sh should locate
hadoop-site.xml in HADOOP_CONF_DIR
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jerome Boulon updated CHUKWA-216:
---------------------------------
Attachment: CHUKWA-216.patch
- Remove the add resoure for Hadoop and let HadoopConfiguration do the default, so for .20:
--> addDefaultResource("core-default.xml");
--> addDefaultResource("core-site.xml");
> postProcess.sh should locate hadoop-site.xml in HADOOP_CONF_DIR
> ---------------------------------------------------------------
>
> Key: CHUKWA-216
> URL: https://issues.apache.org/jira/browse/CHUKWA-216
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Jerome Boulon
> Priority: Blocker
> Attachments: CHUKWA-216.patch
>
>
> postProcess.sh is currently hard coded to CHUKWA_HOME/conf as class path for locating configuration.
> postProcessManager should locate name node address and job tracker address from HADOOP_CONF_DIR or class path.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-216) postProcess.sh should locate
hadoop-site.xml in HADOOP_CONF_DIR
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716963#action_12716963 ]
Hudson commented on CHUKWA-216:
-------------------------------
Integrated in Chukwa-trunk #45 (See [http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/45/])
> postProcess.sh should locate hadoop-site.xml in HADOOP_CONF_DIR
> ---------------------------------------------------------------
>
> Key: CHUKWA-216
> URL: https://issues.apache.org/jira/browse/CHUKWA-216
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Affects Versions: 0.1.2, 0.2.0
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Jerome Boulon
> Priority: Blocker
> Fix For: 0.2.0
>
> Attachments: CHUKWA-216.patch
>
>
> postProcess.sh is currently hard coded to CHUKWA_HOME/conf as class path for locating configuration.
> postProcessManager should locate name node address and job tracker address from HADOOP_CONF_DIR or class path.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-216) postProcess.sh should locate
hadoop-site.xml in HADOOP_CONF_DIR
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715204#action_12715204 ]
Eric Yang commented on CHUKWA-216:
----------------------------------
+1 Looks good
> postProcess.sh should locate hadoop-site.xml in HADOOP_CONF_DIR
> ---------------------------------------------------------------
>
> Key: CHUKWA-216
> URL: https://issues.apache.org/jira/browse/CHUKWA-216
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Affects Versions: 0.1.2, 0.2.0
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Jerome Boulon
> Priority: Blocker
> Fix For: 0.2.0
>
> Attachments: CHUKWA-216.patch
>
>
> postProcess.sh is currently hard coded to CHUKWA_HOME/conf as class path for locating configuration.
> postProcessManager should locate name node address and job tracker address from HADOOP_CONF_DIR or class path.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-216) postProcess.sh should locate
hadoop-site.xml in HADOOP_CONF_DIR
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708614#action_12708614 ]
Jerome Boulon commented on CHUKWA-216:
--------------------------------------
>> postProcessManager should locate name node address and job tracker address from HADOOP_CONF_DIR or class path.
PostProcessManager is currently getting the configuration from ChukwaConfiguration class and loading hadoop-site.xml from the correct location.
So could you be more specific on what's problem you are trying to solve?
// Hadoop config
String hadoopConfDir = System.getenv("HADOOP_CONF_DIR");
if (hadoopConfDir != null) {
super.addResource(new Path(hadoopConfDir + "/hadoop-site.xml"));
log.debug("added " + hadoopConfDir + "/hadoop-site.xml to ChukwaConfiguration");
} else {
String hadoopHome = System.getenv("HADOOP_HOME");
if (hadoopHome != null) {
super.addResource(new Path(hadoopHome + "/conf/hadoop-site.xml"));
log.debug("added " + hadoopHome + "/hadoop-site.xml to ChukwaConfiguration");
}
}
> postProcess.sh should locate hadoop-site.xml in HADOOP_CONF_DIR
> ---------------------------------------------------------------
>
> Key: CHUKWA-216
> URL: https://issues.apache.org/jira/browse/CHUKWA-216
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Jerome Boulon
> Priority: Blocker
>
> postProcess.sh is currently hard coded to CHUKWA_HOME/conf as class path for locating configuration.
> postProcessManager should locate name node address and job tracker address from HADOOP_CONF_DIR or class path.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-216) postProcess.sh should locate
hadoop-site.xml in HADOOP_CONF_DIR
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716002#action_12716002 ]
Hudson commented on CHUKWA-216:
-------------------------------
Integrated in Chukwa-trunk #43 (See [http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/43/])
. postProcess.sh should use HADOOP_CONF_DIR to find hadoop-site.xml. Contributed by Jerome Boulon
> postProcess.sh should locate hadoop-site.xml in HADOOP_CONF_DIR
> ---------------------------------------------------------------
>
> Key: CHUKWA-216
> URL: https://issues.apache.org/jira/browse/CHUKWA-216
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Affects Versions: 0.1.2, 0.2.0
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Jerome Boulon
> Priority: Blocker
> Fix For: 0.2.0
>
> Attachments: CHUKWA-216.patch
>
>
> postProcess.sh is currently hard coded to CHUKWA_HOME/conf as class path for locating configuration.
> postProcessManager should locate name node address and job tracker address from HADOOP_CONF_DIR or class path.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-216) postProcess.sh should locate
hadoop-site.xml in HADOOP_CONF_DIR
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708602#action_12708602 ]
Jerome Boulon commented on CHUKWA-216:
--------------------------------------
>>PostProcess.sh is currently hard coded to CHUKWA_HOME/conf as class path for locating configuration.
This is a duplicate of CHUKWA-216
> postProcess.sh should locate hadoop-site.xml in HADOOP_CONF_DIR
> ---------------------------------------------------------------
>
> Key: CHUKWA-216
> URL: https://issues.apache.org/jira/browse/CHUKWA-216
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Jerome Boulon
> Priority: Blocker
>
> postProcess.sh is currently hard coded to CHUKWA_HOME/conf as class path for locating configuration.
> postProcessManager should locate name node address and job tracker address from HADOOP_CONF_DIR or class path.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-216) postProcess.sh should locate
hadoop-site.xml in HADOOP_CONF_DIR
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714886#action_12714886 ]
Ari Rabkin commented on CHUKWA-216:
-----------------------------------
This looks like a good idea. Have you folks tested this operationally?
> postProcess.sh should locate hadoop-site.xml in HADOOP_CONF_DIR
> ---------------------------------------------------------------
>
> Key: CHUKWA-216
> URL: https://issues.apache.org/jira/browse/CHUKWA-216
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Jerome Boulon
> Priority: Blocker
> Attachments: CHUKWA-216.patch
>
>
> postProcess.sh is currently hard coded to CHUKWA_HOME/conf as class path for locating configuration.
> postProcessManager should locate name node address and job tracker address from HADOOP_CONF_DIR or class path.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-216) postProcess.sh should locate
hadoop-site.xml in HADOOP_CONF_DIR
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jerome Boulon updated CHUKWA-216:
---------------------------------
Status: Patch Available (was: Open)
> postProcess.sh should locate hadoop-site.xml in HADOOP_CONF_DIR
> ---------------------------------------------------------------
>
> Key: CHUKWA-216
> URL: https://issues.apache.org/jira/browse/CHUKWA-216
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Jerome Boulon
> Priority: Blocker
> Attachments: CHUKWA-216.patch
>
>
> postProcess.sh is currently hard coded to CHUKWA_HOME/conf as class path for locating configuration.
> postProcessManager should locate name node address and job tracker address from HADOOP_CONF_DIR or class path.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-216) postProcess.sh should locate
hadoop-site.xml in HADOOP_CONF_DIR
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714608#action_12714608 ]
Jerome Boulon commented on CHUKWA-216:
--------------------------------------
+ add HADOOP_CONF_DIR to the classpath
> postProcess.sh should locate hadoop-site.xml in HADOOP_CONF_DIR
> ---------------------------------------------------------------
>
> Key: CHUKWA-216
> URL: https://issues.apache.org/jira/browse/CHUKWA-216
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Jerome Boulon
> Priority: Blocker
> Attachments: CHUKWA-216.patch
>
>
> postProcess.sh is currently hard coded to CHUKWA_HOME/conf as class path for locating configuration.
> postProcessManager should locate name node address and job tracker address from HADOOP_CONF_DIR or class path.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-216) postProcess.sh should locate
hadoop-site.xml in HADOOP_CONF_DIR
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715189#action_12715189 ]
Eric Yang commented on CHUKWA-216:
----------------------------------
Remove this issue as a blocker for 0.1.2 release.
> postProcess.sh should locate hadoop-site.xml in HADOOP_CONF_DIR
> ---------------------------------------------------------------
>
> Key: CHUKWA-216
> URL: https://issues.apache.org/jira/browse/CHUKWA-216
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Affects Versions: 0.1.2, 0.2.0
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Jerome Boulon
> Priority: Blocker
> Fix For: 0.2.0
>
> Attachments: CHUKWA-216.patch
>
>
> postProcess.sh is currently hard coded to CHUKWA_HOME/conf as class path for locating configuration.
> postProcessManager should locate name node address and job tracker address from HADOOP_CONF_DIR or class path.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CHUKWA-216) postProcess.sh should locate
hadoop-site.xml in HADOOP_CONF_DIR
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang reassigned CHUKWA-216:
--------------------------------
Assignee: Jerome Boulon
> postProcess.sh should locate hadoop-site.xml in HADOOP_CONF_DIR
> ---------------------------------------------------------------
>
> Key: CHUKWA-216
> URL: https://issues.apache.org/jira/browse/CHUKWA-216
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Jerome Boulon
> Priority: Blocker
>
> postProcess.sh is currently hard coded to CHUKWA_HOME/conf as class path for locating configuration.
> postProcessManager should locate name node address and job tracker address from HADOOP_CONF_DIR or class path.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-216) postProcess.sh should locate
hadoop-site.xml in HADOOP_CONF_DIR
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ari Rabkin updated CHUKWA-216:
------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this to TRUNK. Thanks, Jerome!
> postProcess.sh should locate hadoop-site.xml in HADOOP_CONF_DIR
> ---------------------------------------------------------------
>
> Key: CHUKWA-216
> URL: https://issues.apache.org/jira/browse/CHUKWA-216
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Affects Versions: 0.1.2, 0.2.0
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Jerome Boulon
> Priority: Blocker
> Fix For: 0.2.0
>
> Attachments: CHUKWA-216.patch
>
>
> postProcess.sh is currently hard coded to CHUKWA_HOME/conf as class path for locating configuration.
> postProcessManager should locate name node address and job tracker address from HADOOP_CONF_DIR or class path.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.