You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Mikhail Bautin (Created) (JIRA)" <ji...@apache.org> on 2011/11/17 05:50:51 UTC
[jira] [Created] (HBASE-4809) Per-CF set RPC metrics
Per-CF set RPC metrics
----------------------
Key: HBASE-4809
URL: https://issues.apache.org/jira/browse/HBASE-4809
Project: HBase
Issue Type: Improvement
Reporter: Mikhail Bautin
Priority: Minor
Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each "mutation signature" (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4809) Per-CF set RPC metrics
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154285#comment-13154285 ]
Phabricator commented on HBASE-4809:
------------------------------------
nspiegelberg has accepted the revision "[jira] [HBASE-4809] Per-CF set RPC metrics".
could you put the test under TestHeapSize & I'll commit
INLINE COMMENTS
src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaMetrics.java:220-222 I think it's better to be consistent than optimal. Right now, heapsize is easy to refactor because it's done the same way for all classes. I'm +1 on a heapsize refactor, but I say we put that in another JIRA. It's easier to review 2 JIRAs, feature + refactor, than it is to combine the two.
REVISION DETAIL
https://reviews.facebook.net/D483
> Per-CF set RPC metrics
> ----------------------
>
> Key: HBASE-4809
> URL: https://issues.apache.org/jira/browse/HBASE-4809
> Project: HBase
> Issue Type: Improvement
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
> Attachments: D483.1.patch, D483.2.patch, D483.3.patch, HBASE-4809_Per_CF_set_RPC_metrics.patch
>
>
> Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each "mutation signature" (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4809) Per-CF set RPC metrics
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154888#comment-13154888 ]
Hadoop QA commented on HBASE-4809:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504694/D483.5.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 15 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -162 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 66 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.replication.TestReplication
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/327//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/327//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/327//console
This message is automatically generated.
> Per-CF set RPC metrics
> ----------------------
>
> Key: HBASE-4809
> URL: https://issues.apache.org/jira/browse/HBASE-4809
> Project: HBase
> Issue Type: Improvement
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: D483.1.patch, D483.2.patch, D483.3.patch, D483.4.patch, D483.5.patch, HBASE-4809.patch, HBASE-4809_Per_CF_set_RPC_metrics.patch
>
>
> Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each "mutation signature" (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4809) Per-CF set RPC metrics
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151847#comment-13151847 ]
Phabricator commented on HBASE-4809:
------------------------------------
nspiegelberg has commented on the revision "[jira] [HBASE-4809] Per-CF set RPC metrics".
lgtm for the most part. pretty straightforward
INLINE COMMENTS
src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaMetrics.java:220-222 any reason why this isn't in TestHeapSize? so other classes work this way?
src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java:534-535 nit pick: isEmpty() is guaranteed O(1). size() is not and may be O(n) depending on the container. That said, I'm pretty sure that it's O(1) in this case and too many other things are CPU intensive, so I consider it an optional change.
src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java:547 for accuracy, should this condition be:
MAX_SIZE -= "__more".length()
if (sb.length() + Bytes.toString(family).length() > MAX_SIZE)
right now, it's not really a MAX_SIZE, it's more TRUNCATE_AFTER
REVISION DETAIL
https://reviews.facebook.net/D483
> Per-CF set RPC metrics
> ----------------------
>
> Key: HBASE-4809
> URL: https://issues.apache.org/jira/browse/HBASE-4809
> Project: HBase
> Issue Type: Improvement
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
> Attachments: D483.1.patch
>
>
> Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each "mutation signature" (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4809) Per-CF set RPC metrics
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154849#comment-13154849 ]
Phabricator commented on HBASE-4809:
------------------------------------
mbautin has committed the revision "[jira] [HBASE-4809] Per-CF set RPC metrics".
REVISION DETAIL
https://reviews.facebook.net/D483
COMMIT
https://reviews.facebook.net/rHBASE1204799
> Per-CF set RPC metrics
> ----------------------
>
> Key: HBASE-4809
> URL: https://issues.apache.org/jira/browse/HBASE-4809
> Project: HBase
> Issue Type: Improvement
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: D483.1.patch, D483.2.patch, D483.3.patch, D483.4.patch, D483.5.patch, HBASE-4809.patch, HBASE-4809_Per_CF_set_RPC_metrics.patch
>
>
> Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each "mutation signature" (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4809) Per-CF set RPC metrics
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HBASE-4809:
-------------------------------
Attachment: D483.2.patch
mbautin updated the revision "[jira] [HBASE-4809] Per-CF set RPC metrics".
Reviewers: nspiegelberg, JIRA, Kannan, Karthik
Addressing Nicolas's comments and making a round of refactoring, code deduplication, and cleanup. Most unit tests pass -- I will post another update with final results.
REVISION DETAIL
https://reviews.facebook.net/D483
AFFECTED FILES
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java
src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaMetrics.java
> Per-CF set RPC metrics
> ----------------------
>
> Key: HBASE-4809
> URL: https://issues.apache.org/jira/browse/HBASE-4809
> Project: HBase
> Issue Type: Improvement
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
> Attachments: D483.1.patch, D483.2.patch
>
>
> Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each "mutation signature" (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4809) Per-CF set RPC metrics
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13153930#comment-13153930 ]
Hadoop QA commented on HBASE-4809:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504450/HBASE-4809_Per_CF_set_RPC_metrics.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 4 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -166 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 61 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.regionserver.wal.TestLogRolling
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/316//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/316//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/316//console
This message is automatically generated.
> Per-CF set RPC metrics
> ----------------------
>
> Key: HBASE-4809
> URL: https://issues.apache.org/jira/browse/HBASE-4809
> Project: HBase
> Issue Type: Improvement
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
> Attachments: D483.1.patch, D483.2.patch, D483.3.patch, HBASE-4809_Per_CF_set_RPC_metrics.patch
>
>
> Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each "mutation signature" (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4809) Per-CF set RPC metrics
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HBASE-4809:
-------------------------------
Attachment: D483.5.patch
mbautin updated the revision "[jira] [HBASE-4809] Per-CF set RPC metrics".
Reviewers: nspiegelberg, JIRA, Kannan, Karthik
My previous bugfix broke TestMemStore and TestStoreScanner. Fixing them by setting the showTableName flag explicitly.
REVISION DETAIL
https://reviews.facebook.net/D483
AFFECTED FILES
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java
src/test/java/org/apache/hadoop/hbase/io/TestHeapSize.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStore.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreScanner.java
src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaConfigured.java
src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaMetrics.java
> Per-CF set RPC metrics
> ----------------------
>
> Key: HBASE-4809
> URL: https://issues.apache.org/jira/browse/HBASE-4809
> Project: HBase
> Issue Type: Improvement
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
> Attachments: D483.1.patch, D483.2.patch, D483.3.patch, D483.4.patch, D483.5.patch, HBASE-4809_Per_CF_set_RPC_metrics.patch
>
>
> Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each "mutation signature" (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4809) Per-CF set RPC metrics
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154853#comment-13154853 ]
Hadoop QA commented on HBASE-4809:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504694/D483.5.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 15 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -162 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 66 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplication
org.apache.hadoop.hbase.client.TestAdmin
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/325//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/325//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/325//console
This message is automatically generated.
> Per-CF set RPC metrics
> ----------------------
>
> Key: HBASE-4809
> URL: https://issues.apache.org/jira/browse/HBASE-4809
> Project: HBase
> Issue Type: Improvement
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: D483.1.patch, D483.2.patch, D483.3.patch, D483.4.patch, D483.5.patch, HBASE-4809.patch, HBASE-4809_Per_CF_set_RPC_metrics.patch
>
>
> Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each "mutation signature" (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4809) Per-CF set RPC metrics
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HBASE-4809:
-------------------------------
Attachment: D483.4.patch
mbautin updated the revision "[jira] [HBASE-4809] Per-CF set RPC metrics".
Reviewers: nspiegelberg, JIRA, Kannan, Karthik
Addressing Nicolas's comments. Cluster testing revealed another bug -- the hbase.metrics.showTableName conf option was not honored in RPC metrics. This is now fixed. I will post the final patch on the JIRA.
REVISION DETAIL
https://reviews.facebook.net/D483
AFFECTED FILES
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java
src/test/java/org/apache/hadoop/hbase/io/TestHeapSize.java
src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaConfigured.java
src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaMetrics.java
> Per-CF set RPC metrics
> ----------------------
>
> Key: HBASE-4809
> URL: https://issues.apache.org/jira/browse/HBASE-4809
> Project: HBase
> Issue Type: Improvement
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
> Attachments: D483.1.patch, D483.2.patch, D483.3.patch, D483.4.patch, HBASE-4809_Per_CF_set_RPC_metrics.patch
>
>
> Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each "mutation signature" (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4809) Per-CF set RPC metrics
Posted by "Mikhail Bautin (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mikhail Bautin updated HBASE-4809:
----------------------------------
Attachment: HBASE-4809_Per_CF_set_RPC_metrics.patch
This corresponds to D483.3.patch.
> Per-CF set RPC metrics
> ----------------------
>
> Key: HBASE-4809
> URL: https://issues.apache.org/jira/browse/HBASE-4809
> Project: HBase
> Issue Type: Improvement
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
> Attachments: D483.1.patch, D483.2.patch, D483.3.patch, HBASE-4809_Per_CF_set_RPC_metrics.patch
>
>
> Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each "mutation signature" (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4809) Per-CF set RPC metrics
Posted by "Mikhail Bautin (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mikhail Bautin updated HBASE-4809:
----------------------------------
Release Note: Testing the patch on Hudson.
Status: Patch Available (was: Open)
> Per-CF set RPC metrics
> ----------------------
>
> Key: HBASE-4809
> URL: https://issues.apache.org/jira/browse/HBASE-4809
> Project: HBase
> Issue Type: Improvement
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
> Attachments: D483.1.patch, D483.2.patch, D483.3.patch, HBASE-4809_Per_CF_set_RPC_metrics.patch
>
>
> Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each "mutation signature" (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4809) Per-CF set RPC metrics
Posted by "Mikhail Bautin (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mikhail Bautin reassigned HBASE-4809:
-------------------------------------
Assignee: Mikhail Bautin
> Per-CF set RPC metrics
> ----------------------
>
> Key: HBASE-4809
> URL: https://issues.apache.org/jira/browse/HBASE-4809
> Project: HBase
> Issue Type: Improvement
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
> Attachments: D483.1.patch
>
>
> Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each "mutation signature" (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4809) Per-CF set RPC metrics
Posted by "Nicolas Spiegelberg (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nicolas Spiegelberg updated HBASE-4809:
---------------------------------------
Resolution: Fixed
Fix Version/s: 0.94.0
Status: Resolved (was: Patch Available)
> Per-CF set RPC metrics
> ----------------------
>
> Key: HBASE-4809
> URL: https://issues.apache.org/jira/browse/HBASE-4809
> Project: HBase
> Issue Type: Improvement
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: D483.1.patch, D483.2.patch, D483.3.patch, D483.4.patch, D483.5.patch, HBASE-4809.patch, HBASE-4809_Per_CF_set_RPC_metrics.patch
>
>
> Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each "mutation signature" (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4809) Per-CF set RPC metrics
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13153373#comment-13153373 ]
Phabricator commented on HBASE-4809:
------------------------------------
mbautin has commented on the revision "[jira] [HBASE-4809] Per-CF set RPC metrics".
INLINE COMMENTS
src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaMetrics.java:220-222 Do you think we should put all heap-size related stuff into TestHeapSize? If we keep these in their respective classes' unit tests, fewer diffs will overlap.
src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java:534-535 Done.
src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java:547 Done.
REVISION DETAIL
https://reviews.facebook.net/D483
> Per-CF set RPC metrics
> ----------------------
>
> Key: HBASE-4809
> URL: https://issues.apache.org/jira/browse/HBASE-4809
> Project: HBase
> Issue Type: Improvement
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
> Attachments: D483.1.patch, D483.2.patch
>
>
> Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each "mutation signature" (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4809) Per-CF set RPC metrics
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154905#comment-13154905 ]
Hudson commented on HBASE-4809:
-------------------------------
Integrated in HBase-TRUNK #2472 (See [https://builds.apache.org/job/HBase-TRUNK/2472/])
[jira] [HBASE-4809] Per-CF set RPC metrics
Summary: Porting RPC metrics incremented for each column family set touched by a
request from 0.89-fb.
Test Plan:
Unit tests, 5-node cluster. Observe metrics through JMX.
The current version of patch compiles but no testing done yet.
Reviewers: nspiegelberg, JIRA, Kannan, Karthik
Reviewed By: nspiegelberg
CC: nspiegelberg, mbautin
Differential Revision: 483
nspiegelberg :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/TestHeapSize.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStore.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreScanner.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaConfigured.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaMetrics.java
> Per-CF set RPC metrics
> ----------------------
>
> Key: HBASE-4809
> URL: https://issues.apache.org/jira/browse/HBASE-4809
> Project: HBase
> Issue Type: Improvement
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: D483.1.patch, D483.2.patch, D483.3.patch, D483.4.patch, D483.5.patch, HBASE-4809.patch, HBASE-4809_Per_CF_set_RPC_metrics.patch
>
>
> Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each "mutation signature" (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4809) Per-CF set RPC metrics
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HBASE-4809:
-------------------------------
Attachment: D483.1.patch
mbautin requested code review of "[jira] [HBASE-4809] Per-CF set RPC metrics".
Reviewers: nspiegelberg, JIRA, Kannan, Karthik
Porting RPC metrics incremented for each column family set touched by a request from 0.89-fb.
TEST PLAN
Unit tests, 5-node cluster. Observe metrics through JMX.
The current version of patch compiles but no testing done yet.
REVISION DETAIL
https://reviews.facebook.net/D483
AFFECTED FILES
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java
src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaMetrics.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/1011/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> Per-CF set RPC metrics
> ----------------------
>
> Key: HBASE-4809
> URL: https://issues.apache.org/jira/browse/HBASE-4809
> Project: HBase
> Issue Type: Improvement
> Reporter: Mikhail Bautin
> Priority: Minor
> Attachments: D483.1.patch
>
>
> Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each "mutation signature" (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4809) Per-CF set RPC metrics
Posted by "Nicolas Spiegelberg (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nicolas Spiegelberg updated HBASE-4809:
---------------------------------------
Attachment: HBASE-4809.patch
> Per-CF set RPC metrics
> ----------------------
>
> Key: HBASE-4809
> URL: https://issues.apache.org/jira/browse/HBASE-4809
> Project: HBase
> Issue Type: Improvement
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
> Attachments: D483.1.patch, D483.2.patch, D483.3.patch, D483.4.patch, D483.5.patch, HBASE-4809.patch, HBASE-4809_Per_CF_set_RPC_metrics.patch
>
>
> Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each "mutation signature" (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4809) Per-CF set RPC metrics
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HBASE-4809:
-------------------------------
Attachment: D483.3.patch
mbautin updated the revision "[jira] [HBASE-4809] Per-CF set RPC metrics".
Reviewers: nspiegelberg, JIRA, Kannan, Karthik
Rebasing on most recent changes to the trunk and fixing a bug in StoreScanner. Unit tests pass, I will start a run on Hudson. Cluster testing is still to be done.
REVISION DETAIL
https://reviews.facebook.net/D483
AFFECTED FILES
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java
src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaMetrics.java
> Per-CF set RPC metrics
> ----------------------
>
> Key: HBASE-4809
> URL: https://issues.apache.org/jira/browse/HBASE-4809
> Project: HBase
> Issue Type: Improvement
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
> Attachments: D483.1.patch, D483.2.patch, D483.3.patch
>
>
> Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each "mutation signature" (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira