You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by "Eric Yang (JIRA)" <ji...@apache.org> on 2010/11/07 19:14:05 UTC
[jira] Created: (CHUKWA-545) Add ability to stream Hadoop metrics
through syslog appender
Add ability to stream Hadoop metrics through syslog appender
-------------------------------------------------------------
Key: CHUKWA-545
URL: https://issues.apache.org/jira/browse/CHUKWA-545
Project: Chukwa
Issue Type: New Feature
Components: data collection
Reporter: Eric Yang
Assignee: Eric Yang
Fix For: 0.5.0
Hadoop metrics is currently streaming through Chukwa's customized log4j appender to delegate the registration of metrics file with Chukwa agent. Both Chukwa agent and chukwa hadoop client keeps track of list of registered adaptors. It is error prone to have adaptor list tracked in two different places. In addition, the management of metrics files is also a problem. Hence, I propose to remove both limitation by using syslog appeender to send hadoop metrics to Chukwa. This approach eliminates the adaptor and file management problems.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-545) Add ability to stream Hadoop metrics
through syslog appender
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated CHUKWA-545:
-----------------------------
Attachment: CHUKWA-545.patch
- Converted Log4jMetricsContext to use syslog appender.
- Modified HadoopMetricsProcessor to parse syslog message.
> Add ability to stream Hadoop metrics through syslog appender
> -------------------------------------------------------------
>
> Key: CHUKWA-545
> URL: https://issues.apache.org/jira/browse/CHUKWA-545
> Project: Chukwa
> Issue Type: New Feature
> Components: data collection
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.5.0
>
> Attachments: CHUKWA-545.patch
>
>
> Hadoop metrics is currently streaming through Chukwa's customized log4j appender to delegate the registration of metrics file with Chukwa agent. Both Chukwa agent and chukwa hadoop client keeps track of list of registered adaptors. It is error prone to have adaptor list tracked in two different places. In addition, the management of metrics files is also a problem. Hence, I propose to remove both limitation by using syslog appeender to send hadoop metrics to Chukwa. This approach eliminates the adaptor and file management problems.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-545) Add ability to stream Hadoop metrics
through syslog appender
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated CHUKWA-545:
-----------------------------
Status: Patch Available (was: Open)
> Add ability to stream Hadoop metrics through syslog appender
> -------------------------------------------------------------
>
> Key: CHUKWA-545
> URL: https://issues.apache.org/jira/browse/CHUKWA-545
> Project: Chukwa
> Issue Type: New Feature
> Components: data collection
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.5.0
>
> Attachments: CHUKWA-545.patch
>
>
> Hadoop metrics is currently streaming through Chukwa's customized log4j appender to delegate the registration of metrics file with Chukwa agent. Both Chukwa agent and chukwa hadoop client keeps track of list of registered adaptors. It is error prone to have adaptor list tracked in two different places. In addition, the management of metrics files is also a problem. Hence, I propose to remove both limitation by using syslog appeender to send hadoop metrics to Chukwa. This approach eliminates the adaptor and file management problems.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-545) Add ability to stream Hadoop metrics
through syslog appender
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated CHUKWA-545:
-----------------------------
Resolution: Won't Fix
Status: Resolved (was: Patch Available)
Due to limitation of fragmenting SyslogAppender message at 1024 bytes, this is no longer a good approach to collect Hadoop metrics.
Deprecated by CHUKWA-546.
Reference:
https://issues.apache.org/bugzilla/show_bug.cgi?id=49177
> Add ability to stream Hadoop metrics through syslog appender
> -------------------------------------------------------------
>
> Key: CHUKWA-545
> URL: https://issues.apache.org/jira/browse/CHUKWA-545
> Project: Chukwa
> Issue Type: New Feature
> Components: data collection
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.5.0
>
> Attachments: CHUKWA-545.patch
>
>
> Hadoop metrics is currently streaming through Chukwa's customized log4j appender to delegate the registration of metrics file with Chukwa agent. Both Chukwa agent and chukwa hadoop client keeps track of list of registered adaptors. It is error prone to have adaptor list tracked in two different places. In addition, the management of metrics files is also a problem. Hence, I propose to remove both limitation by using syslog appeender to send hadoop metrics to Chukwa. This approach eliminates the adaptor and file management problems.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.