You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Sushanth Sowmyan (JIRA)" <ji...@apache.org> on 2010/12/01 00:26:11 UTC
[jira] Created: (HIVE-1818) Call frequency and duration metrics for
HiveMetaStore via jmx
Call frequency and duration metrics for HiveMetaStore via jmx
-------------------------------------------------------------
Key: HIVE-1818
URL: https://issues.apache.org/jira/browse/HIVE-1818
Project: Hive
Issue Type: New Feature
Components: Metastore
Reporter: Sushanth Sowmyan
Priority: Minor
As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1818) Call frequency and duration metrics for
HiveMetaStore via jmx
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang updated HIVE-1818:
----------------------------
Status: Open (was: Patch Available)
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1818) Call frequency and duration metrics
for HiveMetaStore via jmx
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12987311#action_12987311 ]
Sushanth Sowmyan commented on HIVE-1818:
----------------------------------------
Paul, could you please take a look at it? Since this touches on a lot of functions(and changes the logStartFunction semantics) on HiveMetaStore (even if trivially), it is vulnerable to other changes in HiveMetaStore.
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1818) Call frequency and duration metrics for
HiveMetaStore via jmx
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang updated HIVE-1818:
----------------------------
Status: Open (was: Patch Available)
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Work stopped: (HIVE-1818) Call frequency and duration
metrics for HiveMetaStore via jmx
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HIVE-1818 stopped by Sushanth Sowmyan.
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1818) Call frequency and duration metrics
for HiveMetaStore via jmx
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12991757#comment-12991757 ]
Sushanth Sowmyan commented on HIVE-1818:
----------------------------------------
Patch updated to reflect latest suggestion.
Thanks!
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1818) Call frequency and duration metrics for
HiveMetaStore via jmx
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-1818:
-----------------------------
Status: Patch Available (was: Open)
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Priority: Minor
> Attachments: HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1818) Call frequency and duration metrics
for HiveMetaStore via jmx
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989812#comment-12989812 ]
Sushanth Sowmyan commented on HIVE-1818:
----------------------------------------
Hi,
Thanks for the review, I updated the diff on reviewboard (again, not sure whether you got a notification from reviewboard, or whether I should update that diff here too) to reflect your comments/feedback.
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1818) Call frequency and duration metrics
for HiveMetaStore via jmx
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12974785#action_12974785 ]
Paul Yang commented on HIVE-1818:
---------------------------------
Taking a look
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Priority: Minor
> Attachments: HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-1818) Call frequency and duration metrics
for HiveMetaStore via jmx
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sushanth Sowmyan resolved HIVE-1818.
------------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1818) Call frequency and duration metrics
for HiveMetaStore via jmx
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986171#action_12986171 ]
Sushanth Sowmyan commented on HIVE-1818:
----------------------------------------
Review board link : https://reviews.apache.org/r/352/
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1818) Call frequency and duration metrics for
HiveMetaStore via jmx
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang updated HIVE-1818:
----------------------------
Status: Open (was: Patch Available)
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Attachments: HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1818) Call frequency and duration metrics for
HiveMetaStore via jmx
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sushanth Sowmyan updated HIVE-1818:
-----------------------------------
Attachment: HIVE-1818-vs-1054860.patch
+Updated patch to work against current trunk.
+Also, made scopes ThreadLocal inside Metrics, which was necessary
+Renamed/removed some of the logStart* functions in HiveMetaStore so as to make sure new contributions use the start* functions instead which also do the required logging in addition to jmx tracking.
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1818) Call frequency and duration metrics
for HiveMetaStore via jmx
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992322#comment-12992322 ]
Sushanth Sowmyan commented on HIVE-1818:
----------------------------------------
Thanks!
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1818) Call frequency and duration metrics for
HiveMetaStore via jmx
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sushanth Sowmyan updated HIVE-1818:
-----------------------------------
Attachment: HIVE-1818.patch
Simple patch implementing a jmx metrics subsystem
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Priority: Minor
> Attachments: HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Work started: (HIVE-1818) Call frequency and duration
metrics for HiveMetaStore via jmx
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HIVE-1818 started by Sushanth Sowmyan.
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1818) Call frequency and duration metrics for
HiveMetaStore via jmx
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sushanth Sowmyan updated HIVE-1818:
-----------------------------------
Release Note: jmx instrumentation for HiveMetaStore call frequency and duration
Status: Patch Available (was: Open)
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1818) Call frequency and duration metrics
for HiveMetaStore via jmx
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990749#comment-12990749 ]
Paul Yang commented on HIVE-1818:
---------------------------------
In case it was missed - posted 1 additional suggestion on reviewboard.
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1818) Call frequency and duration metrics
for HiveMetaStore via jmx
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12981991#action_12981991 ]
Paul Yang commented on HIVE-1818:
---------------------------------
Looks pretty good, though I'm not too familiar with MBeans
A few changes:
* Can you print the exception stack if Metrics.init() fails in HiveMetaStore.init()? See MetastoreUtils for some helper methods
* Metrics.incrementCounter(String, int) - with a new name, it should set the value to the value of increment, not 1
* doesn't MetricsMBeanImpl.metricsMap need to be synchronized since it could be hit by multiple threads simultaneously?
* Can you run checkstyle on the new files? I saw issues like
{code}
if (!initialized){ return null; }
{code}
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1818) Call frequency and duration metrics
for HiveMetaStore via jmx
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12987322#action_12987322 ]
Paul Yang commented on HIVE-1818:
---------------------------------
Taking a look..
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1818) Call frequency and duration metrics
for HiveMetaStore via jmx
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992247#comment-12992247 ]
Paul Yang commented on HIVE-1818:
---------------------------------
+1 tests passed
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1818) Call frequency and duration metrics for
HiveMetaStore via jmx
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-1818:
-----------------------------------
Fix Version/s: 0.7.0
Status: Patch Available (was: Open)
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1818) Call frequency and duration metrics
for HiveMetaStore via jmx
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992249#comment-12992249 ]
Paul Yang commented on HIVE-1818:
---------------------------------
Committed. Thanks Sushanth!
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1818) Call frequency and duration metrics for
HiveMetaStore via jmx
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sushanth Sowmyan updated HIVE-1818:
-----------------------------------
Attachment: HIVE-1818-vs-1063088.patch
> Can you print the exception stack if Metrics.init() fails in HiveMetaStore.init()? See MetastoreUtils for some helper methods
> Metrics.incrementCounter(String, int) - with a new name, it should set the value to the value of increment, not 1
> doesn't MetricsMBeanImpl.metricsMap need to be synchronized since it could be hit by multiple threads simultaneously?
Done. Good spots.
> Can you run checkstyle on the new files? I saw issues like : if (!initialized){ return null; }
Fixed instances of single line ifs, but checkstyle did not raise that as an issue for me. Weird.
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1818) Call frequency and duration metrics
for HiveMetaStore via jmx
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12988177#action_12988177 ]
Paul Yang commented on HIVE-1818:
---------------------------------
Not sure how reviewboard does notifications, but I posted some comments yesterday.
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HIVE-1818) Call frequency and duration metrics
for HiveMetaStore via jmx
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang reassigned HIVE-1818:
-------------------------------
Assignee: Sushanth Sowmyan
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Attachments: HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1818) Call frequency and duration metrics
for HiveMetaStore via jmx
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12974786#action_12974786 ]
Paul Yang commented on HIVE-1818:
---------------------------------
Sushanth, can you regenerate this patch against the current trunk?
> Call frequency and duration metrics for HiveMetaStore via jmx
> -------------------------------------------------------------
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Priority: Minor
> Attachments: HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the HiveMetaStore had some sort of instrumentation capability so as to measure frequency of calls to various calls on the HiveMetaStore and the duration of time spent in these calls.
> There are already incrementCounter() and logStartFunction() / logStartTableFunction() ,etc calls in HiveMetaStore, and they could be refactored/repurposed to make calls that expose JMX MBeans as well. Or, a Metrics subsystem could be introduced which made calls to incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics subsystem could use to determine whether or not to be enabled, and if so, on to what port. And once we have the capability to instrument and expose MBeans, it might also be possible for other subsystems to also adopt and use this system.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.