You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Nigel Daley (JIRA)" <ji...@apache.org> on 2007/01/30 00:08:49 UTC
[jira] Created: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Metrics should offer complete set of static report methods or none at all
-------------------------------------------------------------------------
Key: HADOOP-954
URL: https://issues.apache.org/jira/browse/HADOOP-954
Project: Hadoop
Issue Type: Improvement
Components: metrics
Affects Versions: 0.10.1
Reporter: Nigel Daley
Assigned To: David Bowen
Priority: Minor
org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nigel Daley updated HADOOP-954:
-------------------------------
Attachment: HADOOP-954-2.patch
Updated patch. Includes job and shuffle metrics. Also fixes some bugs in previous patch.
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
> Attachments: HADOOP-954-2.patch, hadoop-954.patch
>
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "David Bowen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Bowen updated HADOOP-954:
-------------------------------
Status: Patch Available (was: Open)
Here's a patch that modifies how the metrics package is used. It compiles, but has not been tested yet. Comments and questions are invited.
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12469254 ]
Doug Cutting commented on HADOOP-954:
-------------------------------------
> Comments and questions are invited.
Looks like a good direction to me. Thanks!
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
> Attachments: hadoop-954.patch
>
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "David Bowen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Bowen updated HADOOP-954:
-------------------------------
Attachment: hadoop-954.patch
Oops, I botched a file renaming (Metrics.java ->MetricsUtil.java). I'm not used to svn and nor is Netbeans.
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
> Attachments: hadoop-954.patch
>
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12468769 ]
Doug Cutting commented on HADOOP-954:
-------------------------------------
> The preferable way to use the API is to have each "event" simply increment a counter
This class is the result of someone trying to figure out the preferable way. So let's not throw it out, but rather improve it. There should be an object that encapsulates these counters, right? If that's not a MetricsRecord, what should it be? Perhaps we need a standard wrapper on a MetricsRecord? Please advise.
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-954:
--------------------------------
Resolution: Fixed
Fix Version/s: 0.12.0
Status: Resolved (was: Patch Available)
I just committed this. Thanks, David & Nigel!
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
> Fix For: 0.12.0
>
> Attachments: HADOOP-954-2.patch, HADOOP-954-3.patch, hadoop-954.patch
>
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "David Bowen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12471151 ]
David Bowen commented on HADOOP-954:
------------------------------------
+1. These changes look fine.
I think we are going to hit overflows with counting bytes. We should change this to counting megabytes (using a float rather than an int), but that can be another Jira ticket.
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
> Attachments: HADOOP-954-2.patch, HADOOP-954-3.patch, hadoop-954.patch
>
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "David Bowen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Bowen updated HADOOP-954:
-------------------------------
The comments on this Metrics class say that it is a utility class (and the log name is org.apache.hadoop.util.MetricsUtil), but the name of the class implies that it is meant to be the main interface to the package.
I think that the comments at least should be changed to say that this class is intended to provide a simplified interface to a subset of the package's functionality. It assumes that there is always exactly one tag in a record, defaulting to the hostname.
It also seems to assume that there is only one metric in a record, and that it is a gauge of type long. This takes the simplified interface approach to a bit of an extreme.
My preference would be to move this class out of the metrics package and put it in dfs. It doesn't make sense to have the apparent main interface to the metrics package omit most of its functionality.
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12469253 ]
Hadoop QA commented on HADOOP-954:
----------------------------------
-1, because the patch command could not apply the latest attachment (http://issues.apache.org) as a patch to trunk revision r502030. Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
> Attachments: hadoop-954.patch
>
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12471193 ]
Hadoop QA commented on HADOOP-954:
----------------------------------
+1, because http://issues.apache.org/jira/secure/attachment/12350600/HADOOP-954-3.patch applied and successfully tested against trunk revision r504682.
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
> Attachments: HADOOP-954-2.patch, HADOOP-954-3.patch, hadoop-954.patch
>
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12469137 ]
Doug Cutting commented on HADOOP-954:
-------------------------------------
> if Hadoop currently needs only a subset of the functionality, it would make sense to create any
encapsulation of that subset somewhere outside of the metrics package
The metrics package is a part of Hadoop. It's architecturally wise to make it standalone, but if there are utilities based on it that are generally useful, they should be included with metrics, and not in some other utility package. If these utilities are poorly designed then they should be fixed or discarded, not simply moved elsewhere.
> We should get rid of the report method (there are about 2 dozen calls) and use the MetricsRecord API instead.
I'm okay with that, so long as it doesn't lead to a bunch of duplicated code. If there are more than a few lines that are duplicated by different uses of the metrics API, then these should be shared through a utility class, no?
> Should I submit a patch with these changes?
Yes, please. Thanks! It would be great to have the designer of the metrics system make Hadoop's use of it exemplary.
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nigel Daley updated HADOOP-954:
-------------------------------
Attachment: HADOOP-954-3.patch
New patch fixes initialization of map and reduce metrics.
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
> Attachments: HADOOP-954-2.patch, HADOOP-954-3.patch, hadoop-954.patch
>
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-954:
--------------------------------
Status: Open (was: Patch Available)
I'll mark this as "Open" until the patch is ready for commit.
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
> Attachments: hadoop-954.patch
>
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nigel Daley updated HADOOP-954:
-------------------------------
Status: Patch Available (was: Open)
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
> Attachments: HADOOP-954-2.patch, HADOOP-954-3.patch, hadoop-954.patch
>
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "David Bowen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12471180 ]
David Bowen commented on HADOOP-954:
------------------------------------
> Of course it will overflow while counting bytes. However, I strongly disagree on replacing the int
> with a float. It is far better to support longs.
This conversation perhaps belongs somewhere else.
Longs aren't currently supported in hadoop metrics, so perhaps we need a Jira ticket for that. But before deciding to make that a priority, I think it is worth trying to use the functionality that already exists. It may be sufficient.
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
> Attachments: HADOOP-954-2.patch, HADOOP-954-3.patch, hadoop-954.patch
>
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "David Bowen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12469259 ]
David Bowen commented on HADOOP-954:
------------------------------------
What's up with this? My workspace is up-to-date at revision 502037.
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
> Attachments: hadoop-954.patch
>
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12471155 ]
Owen O'Malley commented on HADOOP-954:
--------------------------------------
Of course it will overflow while counting bytes. However, I strongly disagree on replacing the int with a float. It is far better to support longs.
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
> Attachments: HADOOP-954-2.patch, HADOOP-954-3.patch, hadoop-954.patch
>
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "David Bowen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12468788 ]
David Bowen commented on HADOOP-954:
------------------------------------
Another goal is to keep the metrics package usable stand-alone,
independent of Hadoop. Therefore, if Hadoop currently needs only a
subset of the functionality, it would make sense to create any
encapsulation of that subset somewhere outside of the metrics package,
unless it is clear that a large percentage of non-Hadoop users will want
that particular subset. But let us put aside this API issue for now,
and focus on the higher priority issue which is getting useful metric
data out of Hadoop. For that, we need to use counters, and counters
won't work with the Metrics.report method.
We have a few classes which each wrap a MetricsRecord: MapTaskMetrics,
TaskTrackerMetrics etc. Some use Metrics.report and others use the
MetricsRecord API directly. The latter should be OK. We should get rid
of the report method (there are about 2 dozen calls) and use the
MetricsRecord API instead. As an alternative to using a callback, we
could just split the MetricsRecords into smaller ones, where each
corresponds to an "event". E.g. we currently have a MetricsRecord
called "map" containing metrics input_records, input_bytes,
output_records and output_bytes. This could be split into two, one for
input data and one for output data. This makes sense because the input
related metrics and output related metrics are updated at different times.
This splitting approach would work for everything, but would be a bit
inelegant in the case of e.g. JobTrackerMetrics where there are 6
counters that need to be independently incremented. We could shorten
the code by using a callback.
Should I submit a patch with these changes?
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "David Bowen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Bowen updated HADOOP-954:
-------------------------------
Attachment: (was: hadoop-954.patch)
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
> Attachments: hadoop-954.patch
>
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12468743 ]
Doug Cutting commented on HADOOP-954:
-------------------------------------
Since it is used in more than one place (dfs and mapred) then it should probably remain in the metrics package. (The "util" package should only be used as a last resort.) But improving its documentation and making it more general both sound like good moves.
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "David Bowen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12468765 ]
David Bowen commented on HADOOP-954:
------------------------------------
A bigger question here is whether this simplified interface is
appropriate even for dfs and mapred. There are a couple of issues:
1. Most or all of the data being reported should be using counters
rather than gauges. This would allow us to get graphs of things
like bytes-read per second, as opposed to simply a graph of the
cumulative number of bytes read (which will soon overflow).
2. The current code is a little inefficient if used in inner loops.
Each call to the report method (for a single metric) will cause
all the metrics in that record to be copied from the MetricsRecord
into the library's internal table. Also, the report method cannot
be easily modified to work with counters, since it would be
necessary to reinitialize the record every time.
The preferable way to use the API is to have each "event" (counter
increment) simply increment a counter, and to use a call-back to report
the counter values to the metrics library periodically (every 5 seconds
in our case).
- David
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-954) Metrics should offer complete set of
static report methods or none at all
Posted by "David Bowen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Bowen updated HADOOP-954:
-------------------------------
Attachment: hadoop-954.patch
Oops. I used "submit patch" when I meant "attach file".
> Metrics should offer complete set of static report methods or none at all
> -------------------------------------------------------------------------
>
> Key: HADOOP-954
> URL: https://issues.apache.org/jira/browse/HADOOP-954
> Project: Hadoop
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.10.1
> Reporter: Nigel Daley
> Assigned To: David Bowen
> Priority: Minor
> Attachments: hadoop-954.patch
>
>
> org.apache.hadoop.metrics.Metrics currently has one report method. I should either have report methods for all underlying MetricsRecord or no report methods at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.