You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by "Eric Yang (JIRA)" <ji...@apache.org> on 2010/12/30 06:49:45 UTC
[jira] Created: (CHUKWA-571) Add digested system metrics to
SystemMetrics demux parser
Add digested system metrics to SystemMetrics demux parser
---------------------------------------------------------
Key: CHUKWA-571
URL: https://issues.apache.org/jira/browse/CHUKWA-571
Project: Chukwa
Issue Type: New Feature
Components: MR Data Processors
Environment: Java 6, Mac OS X 10.6
Reporter: Eric Yang
Assignee: Eric Yang
For now, the system metrics demux parser are storing raw metrics. Each system components are measured independently. It would be nice to make a summary grouping for cpu, disk, network. Instead of storing cpu:combined.0 ... cpu:combined.8, for 8 cpu cores in the system, it would be nice to have a average number for all cpus combined. The digested metrics are easier to aggregate at cluster level.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-571) Add digested system metrics to
SystemMetrics demux parser
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated CHUKWA-571:
-----------------------------
Status: Patch Available (was: Open)
Patch for CHUKWA-572 should be applied with this patch for the metrics to generate properly.
> Add digested system metrics to SystemMetrics demux parser
> ---------------------------------------------------------
>
> Key: CHUKWA-571
> URL: https://issues.apache.org/jira/browse/CHUKWA-571
> Project: Chukwa
> Issue Type: New Feature
> Components: MR Data Processors
> Environment: Java 6, Mac OS X 10.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Attachments: CHUKWA-571.patch
>
>
> For now, the system metrics demux parser are storing raw metrics. Each system components are measured independently. It would be nice to make a summary grouping for cpu, disk, network. Instead of storing cpu:combined.0 ... cpu:combined.8, for 8 cpu cores in the system, it would be nice to have a average number for all cpus combined. The digested metrics are easier to aggregate at cluster level.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-571) Add digested system metrics to
SystemMetrics demux parser
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12976470#action_12976470 ]
Ari Rabkin commented on CHUKWA-571:
-----------------------------------
Looks good; +1 to commit.
> Add digested system metrics to SystemMetrics demux parser
> ---------------------------------------------------------
>
> Key: CHUKWA-571
> URL: https://issues.apache.org/jira/browse/CHUKWA-571
> Project: Chukwa
> Issue Type: New Feature
> Components: MR Data Processors
> Environment: Java 6, Mac OS X 10.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Attachments: CHUKWA-571.patch
>
>
> For now, the system metrics demux parser are storing raw metrics. Each system components are measured independently. It would be nice to make a summary grouping for cpu, disk, network. Instead of storing cpu:combined.0 ... cpu:combined.8, for 8 cpu cores in the system, it would be nice to have a average number for all cpus combined. The digested metrics are easier to aggregate at cluster level.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-571) Add digested system metrics to
SystemMetrics demux parser
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated CHUKWA-571:
-----------------------------
Resolution: Fixed
Fix Version/s: 0.5.0
Status: Resolved (was: Patch Available)
Thanks Ari. I just committed this.
> Add digested system metrics to SystemMetrics demux parser
> ---------------------------------------------------------
>
> Key: CHUKWA-571
> URL: https://issues.apache.org/jira/browse/CHUKWA-571
> Project: Chukwa
> Issue Type: New Feature
> Components: MR Data Processors
> Environment: Java 6, Mac OS X 10.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.5.0
>
> Attachments: CHUKWA-571.patch
>
>
> For now, the system metrics demux parser are storing raw metrics. Each system components are measured independently. It would be nice to make a summary grouping for cpu, disk, network. Instead of storing cpu:combined.0 ... cpu:combined.8, for 8 cpu cores in the system, it would be nice to have a average number for all cpus combined. The digested metrics are easier to aggregate at cluster level.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-571) Add digested system metrics to
SystemMetrics demux parser
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated CHUKWA-571:
-----------------------------
Attachment: CHUKWA-571.patch
Digest metrics is parse and stored without number suffix. For example, the total bytes received by all network card is in network:RxBytes column. Network Loopback device is omitted from the summary number.
Full list of digest metrics:
* Averaged CPU utilization:
* cpu:combined (user+sys)
* cpu:idle
* cpu:sys
* cpu:user
Sum of network devices throughput stats at interval:
* network:RxBytes
* network:RxDropped
* network:RxErrors
* network:RxPackets
* network:TxBytes
* network:TxColisions
* network:TxErrors
* network:TxPackets
Sum of disk devices throughput stats at interval:
* disk:ReadBytes
* disk:Reads
* disk:WriteBytes
* disk:Writes
> Add digested system metrics to SystemMetrics demux parser
> ---------------------------------------------------------
>
> Key: CHUKWA-571
> URL: https://issues.apache.org/jira/browse/CHUKWA-571
> Project: Chukwa
> Issue Type: New Feature
> Components: MR Data Processors
> Environment: Java 6, Mac OS X 10.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Attachments: CHUKWA-571.patch
>
>
> For now, the system metrics demux parser are storing raw metrics. Each system components are measured independently. It would be nice to make a summary grouping for cpu, disk, network. Instead of storing cpu:combined.0 ... cpu:combined.8, for 8 cpu cores in the system, it would be nice to have a average number for all cpus combined. The digested metrics are easier to aggregate at cluster level.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.