You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Kevin Wilfong (Created) (JIRA)" <ji...@apache.org> on 2011/10/07 19:54:30 UTC
[jira] [Created] (HIVE-2490) Add reset operation to Metrics MBean.
Add reset operation to Metrics MBean.
-------------------------------------
Key: HIVE-2490
URL: https://issues.apache.org/jira/browse/HIVE-2490
Project: Hive
Issue Type: New Feature
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
We should add a reset operation to the Metrics MBean, which will set all the counters to 0.
Note: Deleting the counters from the map of attributes was not suggested because that could break any scripts that get the list of attributes from the bean and then the values of each attribute. Also, 0 is unlikely to be an actual value for any counter, and it will not break the increment functionality.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2490) Add reset operation and average time
attribute to Metrics MBean.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214099#comment-13214099 ]
jiraposter@reviews.apache.org commented on HIVE-2490:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2290/#review5298
-----------------------------------------------------------
Ship it!
+1 Looks good. Can you upload the patch on jira so that I can test and commit.
- Ashutosh
On 2011-10-07 19:42:21, Kevin Wilfong wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2290/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-10-07 19:42:21)
bq.
bq.
bq. Review request for hive and Paul Yang.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. I added a reset operation which sets the value of all attributes to 0, see https://issues.apache.org/jira/browse/HIVE-2490 for an explanation of why 0 was chosen and why the attributes are not deleted.
bq.
bq. I also added an average time attribute in addition to number and total time. This seemed natural as we have all the data we need to calculate it.
bq.
bq. Previously, because we were incrementing the number at the beginning of the function, and the total time at the end of the function, it was impossible to guarantee an accurate average. In particular if the frequency of function calls was high and/or the execution time of the function was very high, the average that could be determined based on the given attributes could be very inaccurate.
bq.
bq. I also added some synchronizations to the Metrics class where I thought it did not appear to be thread safe.
bq.
bq. I based the average and reset API on what is done by Zookeeper in Hadoop.
bq.
bq.
bq. This addresses bug HIVE-2490.
bq. https://issues.apache.org/jira/browse/HIVE-2490
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/common/src/java/org/apache/hadoop/hive/common/metrics/Metrics.java 1179884
bq. trunk/common/src/java/org/apache/hadoop/hive/common/metrics/MetricsMBeanImpl.java 1179884
bq.
bq. Diff: https://reviews.apache.org/r/2290/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. I ran an instance of the Metastore, and executed queries against it from multiple clients, and verified the average attribute and reset operation behaved as expected.
bq.
bq.
bq. Thanks,
bq.
bq. Kevin
bq.
bq.
> Add reset operation and average time attribute to Metrics MBean.
> ----------------------------------------------------------------
>
> Key: HIVE-2490
> URL: https://issues.apache.org/jira/browse/HIVE-2490
> Project: Hive
> Issue Type: New Feature
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
>
> We should add a reset operation to the Metrics MBean, which will set all the counters to 0.
> Note: Deleting the counters from the map of attributes was not suggested because that could break any scripts that get the list of attributes from the bean and then the values of each attribute. Also, 0 is unlikely to be an actual value for any counter, and it will not break the increment functionality.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2490) Add reset operation and average time
attribute to Metrics MBean.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123115#comment-13123115 ]
jiraposter@reviews.apache.org commented on HIVE-2490:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2290/
-----------------------------------------------------------
Review request for hive and Paul Yang.
Summary
-------
I added a reset operation which sets the value of all attributes to 0, see https://issues.apache.org/jira/browse/HIVE-2490 for an explanation of why 0 was chosen and why the attributes are not deleted.
I also added an average time attribute in addition to number and total time. This seemed natural as we have all the data we need to calculate it.
Previously, because we were incrementing the number at the beginning of the function, and the total time at the end of the function, it was impossible to guarantee an accurate average. In particular if the frequency of function calls was high and/or the execution time of the function was very high, the average that could be determined based on the given attributes could be very inaccurate.
I also added some synchronizations to the Metrics class where I thought it did not appear to be thread safe.
I based the average and reset API on what is done by Zookeeper in Hadoop.
This addresses bug HIVE-2490.
https://issues.apache.org/jira/browse/HIVE-2490
Diffs
-----
trunk/common/src/java/org/apache/hadoop/hive/common/metrics/Metrics.java 1179884
trunk/common/src/java/org/apache/hadoop/hive/common/metrics/MetricsMBeanImpl.java 1179884
Diff: https://reviews.apache.org/r/2290/diff
Testing
-------
I ran an instance of the Metastore, and executed queries against it from multiple clients, and verified the average attribute and reset operation behaved as expected.
Thanks,
Kevin
> Add reset operation and average time attribute to Metrics MBean.
> ----------------------------------------------------------------
>
> Key: HIVE-2490
> URL: https://issues.apache.org/jira/browse/HIVE-2490
> Project: Hive
> Issue Type: New Feature
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
>
> We should add a reset operation to the Metrics MBean, which will set all the counters to 0.
> Note: Deleting the counters from the map of attributes was not suggested because that could break any scripts that get the list of attributes from the bean and then the values of each attribute. Also, 0 is unlikely to be an actual value for any counter, and it will not break the increment functionality.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2490) Add reset operation and average time
attribute to Metrics MBean.
Posted by "Kevin Wilfong (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Wilfong updated HIVE-2490:
--------------------------------
Summary: Add reset operation and average time attribute to Metrics MBean. (was: Add reset operation to Metrics MBean.)
> Add reset operation and average time attribute to Metrics MBean.
> ----------------------------------------------------------------
>
> Key: HIVE-2490
> URL: https://issues.apache.org/jira/browse/HIVE-2490
> Project: Hive
> Issue Type: New Feature
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
>
> We should add a reset operation to the Metrics MBean, which will set all the counters to 0.
> Note: Deleting the counters from the map of attributes was not suggested because that could break any scripts that get the list of attributes from the bean and then the values of each attribute. Also, 0 is unlikely to be an actual value for any counter, and it will not break the increment functionality.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2490) Add reset operation and average time
attribute to Metrics MBean.
Posted by "Ashutosh Chauhan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2490:
-----------------------------------
Resolution: Fixed
Fix Version/s: 0.9.0
Status: Resolved (was: Patch Available)
Committed to trunk.
> Add reset operation and average time attribute to Metrics MBean.
> ----------------------------------------------------------------
>
> Key: HIVE-2490
> URL: https://issues.apache.org/jira/browse/HIVE-2490
> Project: Hive
> Issue Type: New Feature
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Fix For: 0.9.0
>
> Attachments: HIVE-2490.1.patch.txt
>
>
> We should add a reset operation to the Metrics MBean, which will set all the counters to 0.
> Note: Deleting the counters from the map of attributes was not suggested because that could break any scripts that get the list of attributes from the bean and then the values of each attribute. Also, 0 is unlikely to be an actual value for any counter, and it will not break the increment functionality.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2490) Add reset operation and average time
attribute to Metrics MBean.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13216224#comment-13216224 ]
Hudson commented on HIVE-2490:
------------------------------
Integrated in Hive-trunk-h0.21 #1274 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1274/])
HIVE-2490: Add reset operation and average time attribute to Metrics MBean. (kevinwilfong via hashutosh) (Revision 1293352)
Result = FAILURE
hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293352
Files :
* /hive/trunk/common/src/java/org/apache/hadoop/hive/common/metrics/Metrics.java
* /hive/trunk/common/src/java/org/apache/hadoop/hive/common/metrics/MetricsMBeanImpl.java
> Add reset operation and average time attribute to Metrics MBean.
> ----------------------------------------------------------------
>
> Key: HIVE-2490
> URL: https://issues.apache.org/jira/browse/HIVE-2490
> Project: Hive
> Issue Type: New Feature
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Fix For: 0.9.0
>
> Attachments: HIVE-2490.1.patch.txt
>
>
> We should add a reset operation to the Metrics MBean, which will set all the counters to 0.
> Note: Deleting the counters from the map of attributes was not suggested because that could break any scripts that get the list of attributes from the bean and then the values of each attribute. Also, 0 is unlikely to be an actual value for any counter, and it will not break the increment functionality.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2490) Add reset operation and average time
attribute to Metrics MBean.
Posted by "Kevin Wilfong (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Wilfong updated HIVE-2490:
--------------------------------
Status: Patch Available (was: Open)
> Add reset operation and average time attribute to Metrics MBean.
> ----------------------------------------------------------------
>
> Key: HIVE-2490
> URL: https://issues.apache.org/jira/browse/HIVE-2490
> Project: Hive
> Issue Type: New Feature
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2490.1.patch.txt
>
>
> We should add a reset operation to the Metrics MBean, which will set all the counters to 0.
> Note: Deleting the counters from the map of attributes was not suggested because that could break any scripts that get the list of attributes from the bean and then the values of each attribute. Also, 0 is unlikely to be an actual value for any counter, and it will not break the increment functionality.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2490) Add reset operation and average time
attribute to Metrics MBean.
Posted by "Kevin Wilfong (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Wilfong updated HIVE-2490:
--------------------------------
Attachment: HIVE-2490.1.patch.txt
> Add reset operation and average time attribute to Metrics MBean.
> ----------------------------------------------------------------
>
> Key: HIVE-2490
> URL: https://issues.apache.org/jira/browse/HIVE-2490
> Project: Hive
> Issue Type: New Feature
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2490.1.patch.txt
>
>
> We should add a reset operation to the Metrics MBean, which will set all the counters to 0.
> Note: Deleting the counters from the map of attributes was not suggested because that could break any scripts that get the list of attributes from the bean and then the values of each attribute. Also, 0 is unlikely to be an actual value for any counter, and it will not break the increment functionality.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira