You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by "Eric Yang (JIRA)" <ji...@apache.org> on 2009/04/12 09:19:14 UTC
[jira] Created: (CHUKWA-123) regression on chukwa-hadoop-client.jar
regression on chukwa-hadoop-client.jar
--------------------------------------
Key: CHUKWA-123
URL: https://issues.apache.org/jira/browse/CHUKWA-123
Project: Hadoop Chukwa
Issue Type: Bug
Components: input tools
Affects Versions: 0.1.2
Environment: Redhat 5.1, Java 6
Reporter: Eric Yang
Priority: Blocker
Fix For: 0.1.2
There is currently no metrics file being generated by Log4jMetricsContext plugin.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-123) regression on chukwa-hadoop-client.jar
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated CHUKWA-123:
-----------------------------
Attachment: (was: CHUKWA-123.patch)
> regression on chukwa-hadoop-client.jar
> --------------------------------------
>
> Key: CHUKWA-123
> URL: https://issues.apache.org/jira/browse/CHUKWA-123
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: input tools
> Affects Versions: 0.1.2
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Priority: Blocker
> Fix For: 0.1.2
>
> Attachments: CHUKWA-123.patch
>
>
> There is currently no metrics file being generated by Log4jMetricsContext plugin.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-123) regression on chukwa-hadoop-client.jar
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated CHUKWA-123:
-----------------------------
Attachment: CHUKWA-123.patch
Patch for logging hadoop metrics to CHUKWA_LOG_DIR/metrics and package configuration with chukwa-hadoop-client.jar.
> regression on chukwa-hadoop-client.jar
> --------------------------------------
>
> Key: CHUKWA-123
> URL: https://issues.apache.org/jira/browse/CHUKWA-123
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: input tools
> Affects Versions: 0.1.2
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Priority: Blocker
> Fix For: 0.1.2
>
> Attachments: CHUKWA-123.patch
>
>
> There is currently no metrics file being generated by Log4jMetricsContext plugin.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-123) regression on
chukwa-hadoop-client.jar
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12698289#action_12698289 ]
Eric Yang commented on CHUKWA-123:
----------------------------------
Loos like the Log4JMetricsContext has a different implementation than before. From the current code, I can understand that it tries to set different record type for each of the metrics before sending. However, the implementation lacks of details on directory handling. The previous was more solid, and it could do the same with RECORD_TYPE separation with minor tweaks.
I am not confident with the current implementation, since it seems like it hasn't been tested. We should discuss what is the right thing to do for this problem until streaming adaptor is ready.
> regression on chukwa-hadoop-client.jar
> --------------------------------------
>
> Key: CHUKWA-123
> URL: https://issues.apache.org/jira/browse/CHUKWA-123
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: input tools
> Affects Versions: 0.1.2
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Priority: Blocker
> Fix For: 0.1.2
>
> Attachments: CHUKWA-123.patch
>
>
> There is currently no metrics file being generated by Log4jMetricsContext plugin.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-123) regression on chukwa-hadoop-client.jar
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated CHUKWA-123:
-----------------------------
Attachment: CHUKWA-123.patch
Add additional slash.
> regression on chukwa-hadoop-client.jar
> --------------------------------------
>
> Key: CHUKWA-123
> URL: https://issues.apache.org/jira/browse/CHUKWA-123
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: input tools
> Affects Versions: 0.1.2
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Priority: Blocker
> Fix For: 0.1.2
>
> Attachments: CHUKWA-123.patch
>
>
> There is currently no metrics file being generated by Log4jMetricsContext plugin.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-123) regression on
chukwa-hadoop-client.jar
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12698441#action_12698441 ]
Jerome Boulon commented on CHUKWA-123:
--------------------------------------
-1 I don't understand what this patch tries to solve
Also could you be more concrete.
What's made you not confident with the current implementation?
Everything is configured from one single file hadoop-metrics.properties instead of an hidden log4.properties file embedded in chukwa-hadoop-client.jar.
You can take a look at conf/hadoop-metrics.properties.template, I've put some info on how to use it at the same time of the Jira.
Example:
mapred.class=org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext
mapred.period=60
mapred.directory=/tmp/
mapred.uuid=true
The UUID flag will add a timestamp at the end of the filename. So you need to use it when there's more than one process, running on the same machine and outputting the same kind of metrics like jvm or mapred.
> regression on chukwa-hadoop-client.jar
> --------------------------------------
>
> Key: CHUKWA-123
> URL: https://issues.apache.org/jira/browse/CHUKWA-123
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: input tools
> Affects Versions: 0.1.2
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Priority: Blocker
> Fix For: 0.1.2
>
> Attachments: CHUKWA-123.patch
>
>
> There is currently no metrics file being generated by Log4jMetricsContext plugin.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CHUKWA-123) regression on
chukwa-hadoop-client.jar
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang resolved CHUKWA-123.
------------------------------
Resolution: Invalid
By using the new config parameter, this issue is not a problem anymore.
> regression on chukwa-hadoop-client.jar
> --------------------------------------
>
> Key: CHUKWA-123
> URL: https://issues.apache.org/jira/browse/CHUKWA-123
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: input tools
> Affects Versions: 0.1.2
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Priority: Blocker
> Fix For: 0.1.2
>
> Attachments: CHUKWA-123.patch
>
>
> There is currently no metrics file being generated by Log4jMetricsContext plugin.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CHUKWA-123) regression on
chukwa-hadoop-client.jar
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang reassigned CHUKWA-123:
--------------------------------
Assignee: Eric Yang
> regression on chukwa-hadoop-client.jar
> --------------------------------------
>
> Key: CHUKWA-123
> URL: https://issues.apache.org/jira/browse/CHUKWA-123
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: input tools
> Affects Versions: 0.1.2
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Priority: Blocker
> Fix For: 0.1.2
>
> Attachments: CHUKWA-123.patch
>
>
> There is currently no metrics file being generated by Log4jMetricsContext plugin.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-123) regression on
chukwa-hadoop-client.jar
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12698177#action_12698177 ]
Eric Yang commented on CHUKWA-123:
----------------------------------
Looks like the output directory is not set in chukwa-hadoop-metrics.log4j.properties.
2009-04-12 07:00:47,535 WARN org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext: Log4JMetricsContext.dfs.directory is null
2009-04-12 07:00:47,536 ERROR org.apache.hadoop.util.MetricsUtil: Unable to create metrics context dfs^D
org.apache.hadoop.metrics.MetricsException: Invalid output directory: null^D
at org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext.init(Log4JMetricsContext.java:75)^D
at org.apache.hadoop.metrics.ContextFactory.getContext(ContextFactory.java:134)^D
at org.apache.hadoop.metrics.MetricsUtil.getContext(MetricsUtil.java:56)^D
at org.apache.hadoop.metrics.MetricsUtil.getContext(MetricsUtil.java:45)^D
at org.apache.hadoop.dfs.datanode.metrics.DataNodeMetrics.<init>(DataNodeMetrics.java:100)^D
at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:387)^D
at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:224)^D
at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:3070)^D
at org.apache.hadoop.dfs.DataNode.instantiateDataNode(DataNode.java:3025)^D
at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:3033)^D
at org.apache.hadoop.dfs.DataNode.main(DataNode.java:3155)
> regression on chukwa-hadoop-client.jar
> --------------------------------------
>
> Key: CHUKWA-123
> URL: https://issues.apache.org/jira/browse/CHUKWA-123
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: input tools
> Affects Versions: 0.1.2
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Priority: Blocker
> Fix For: 0.1.2
>
>
> There is currently no metrics file being generated by Log4jMetricsContext plugin.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.