You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Lars Hofhansl (JIRA)" <ji...@apache.org> on 2012/10/11 02:35:03 UTC
[jira] [Created] (HBASE-6974) Metric blocked updates
Lars Hofhansl created HBASE-6974:
------------------------------------
Summary: Metric blocked updates
Key: HBASE-6974
URL: https://issues.apache.org/jira/browse/HBASE-6974
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Fix For: 0.94.3, 0.96.0
When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
(HRegion.checkResources).
It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475968#comment-13475968 ]
Lars Hofhansl commented on HBASE-6974:
--------------------------------------
@Anoop: Yes that's the idea.
I think you are right and #2 is probably checked somewhere else.
#3 should lead to #1.
Thanks for checking Anoop!
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13476706#comment-13476706 ]
Lars Hofhansl commented on HBASE-6974:
--------------------------------------
Drz, are you planning to work on this. I'm happy to do a quick patch.
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479493#comment-13479493 ]
Hudson commented on HBASE-6974:
-------------------------------
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #227 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/227/])
HBASE-6974 Metric for blocked updates (Michael Drzal) (Revision 1399880)
Result = FAILURE
larsh :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 6974-v5.txt, HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch, HBASE-6974-v4.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Drzal updated HBASE-6974:
---------------------------------
Status: Patch Available (was: Open)
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch, HBASE-6974-v4.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478472#comment-13478472 ]
Lars Hofhansl commented on HBASE-6974:
--------------------------------------
Actually... Should probably name the metric more appropriately.
How about "memstoreSizeBlockedSeconds", and "memstoreHighWaterBlockedSeconds"?
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch, HBASE-6974-v4.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6974) Metric for blocked updates
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6974:
---------------------------------
Summary: Metric for blocked updates (was: Metric blocked updates)
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Fix For: 0.94.3, 0.96.0
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Drzal updated HBASE-6974:
---------------------------------
Attachment: HBASE-6974.patch
First shot at this.
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479465#comment-13479465 ]
Hudson commented on HBASE-6974:
-------------------------------
Integrated in HBase-0.94 #541 (See [https://builds.apache.org/job/HBase-0.94/541/])
HBASE-6974 Metric for blocked updates (Michael Drzal) (Revision 1399881)
Result = FAILURE
larsh :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 6974-v5.txt, HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch, HBASE-6974-v4.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478089#comment-13478089 ]
Lars Hofhansl commented on HBASE-6974:
--------------------------------------
What I meant was something like this:
{code}
boolean blocked = false;
+ long startTime = 0;
while (this.memstoreSize.get() > this.blockingMemStoreSize) {
requestFlush();
if (!blocked) {
+ startTime = EnvironmentEdgeManager.currentTimeMillis();
{code}
That way we only call currentTimeMillis when we're actually blocking.
I think in the MemstoreFlusher we should have the same logic with the blocked flag, otherwise we'd get inundated with log messages that we need to flush.
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Drzal updated HBASE-6974:
---------------------------------
Status: Open (was: Patch Available)
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479229#comment-13479229 ]
Lars Hofhansl commented on HBASE-6974:
--------------------------------------
Only if you want to, I can make these changes upon commmit.
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch, HBASE-6974-v4.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477905#comment-13477905 ]
Michael Drzal commented on HBASE-6974:
--------------------------------------
[~lhofhansl]
- 1024 ms to s conversion fixed. I think I spent too much time converting bytes around, sorry
- I'll add in the memstore flusher metric and post a patch in a bit
- I'll look into EnvironmentEdge.currentTimeMillis
- I understand what you are saying about currentTimeMillis, but let me try to restate what you said so that I am sure that we are on the same page:
If I move the call to currentTimeMillis inside the while loop, that means that I will have to keep another variable off to the side to keep track of the total, plus another one to accomplish the swap. Doing this, I would call currentTimeMillis once for each time through the loop, correct? If you think that is a better way to go, I can do that.
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HBASE-6974 started by Michael Drzal.
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6974) Metric for blocked updates
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6974:
---------------------------------
Priority: Critical (was: Major)
Upping to critical, so it goes into 0.94.3
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Drzal updated HBASE-6974:
---------------------------------
Status: Open (was: Patch Available)
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475971#comment-13475971 ]
Lars Hofhansl commented on HBASE-6974:
--------------------------------------
In any case the #2 should be covered by a different metric, I think.
#1 (and #3) are caused by the IO system not being able to keep up. #2 is caused by insufficient memory (which may also in part stem from the IO system being too slow, but it also may have other reasons).
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479462#comment-13479462 ]
Hudson commented on HBASE-6974:
-------------------------------
Integrated in HBase-TRUNK #3459 (See [https://builds.apache.org/job/HBase-TRUNK/3459/])
HBASE-6974 Metric for blocked updates (Michael Drzal) (Revision 1399880)
Result = FAILURE
larsh :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 6974-v5.txt, HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch, HBASE-6974-v4.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Anoop Sam John (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475981#comment-13475981 ]
Anoop Sam John commented on HBASE-6974:
---------------------------------------
Yes Lars, you are correct, the reasons for #1(#3) and #2 will be normally different... And seperate metric can be better.. In some way user should be able to know why the updates are blocked.. :)
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479409#comment-13479409 ]
Hadoop QA commented on HBASE-6974:
----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12549750/6974-v5.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 82 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3082//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3082//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3082//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3082//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3082//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3082//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3082//console
This message is automatically generated.
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 6974-v5.txt, HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch, HBASE-6974-v4.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Drzal updated HBASE-6974:
---------------------------------
Status: Patch Available (was: Open)
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Drzal updated HBASE-6974:
---------------------------------
Attachment: HBASE-6974-v3.patch
updated moving currentTimeMillis into while loop
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Drzal updated HBASE-6974:
---------------------------------
Status: Patch Available (was: Open)
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475976#comment-13475976 ]
Lars Hofhansl commented on HBASE-6974:
--------------------------------------
And while we're at that, let's remove the synchronized from reclaimMemStoreMemory... Looks like this a left over from the past.
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478465#comment-13478465 ]
Lars Hofhansl commented on HBASE-6974:
--------------------------------------
+1 on v4
(super-duper minor nit: the blocked flag can be pulled into the if statement in the memstore flusher - I'll do that on commit, no need for a new patch).
Thanks Drz!
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch, HBASE-6974-v4.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Anoop Sam John (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475964#comment-13475964 ]
Anoop Sam John commented on HBASE-6974:
---------------------------------------
Lars
Updates will be blocked because more than one reason right
1. As in checkResources when the memstore size became too much (as per the hbase.hregion.memstore.block.multiplier value and memstore size)
2. Because of the global heap space usage for all the memstore in the RS
3. Becuase of the store files count being more (hbase.hstore.blockingStoreFiles).. Well this is not directly affect.. This will make the flush to wait which in turn can block the updates also as per the point 1.. So may be not to consider..
So your idea is that in all case where the block is happened the wait time will be captured and updated in the metric right? May be reason also we can capture so that after seeing this metric user can think about changing some configs if needed etc...
Just for confirming I asked.. Will be very much useful I guess...
Nice work Lars... We would like to use this...
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479138#comment-13479138 ]
Lars Hofhansl commented on HBASE-6974:
--------------------------------------
Same here. Seems in each patch that is the hardest part :)
How about "updatesBlockedSeconds" and "updatesBlockedSecondsHighWater"?
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch, HBASE-6974-v4.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478913#comment-13478913 ]
Michael Drzal commented on HBASE-6974:
--------------------------------------
I hate naming things, and I really don't care what they are named. I did think having HighWater in the name was useful as that term is used in other places in programming/computer science and is generally well understood.
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch, HBASE-6974-v4.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477961#comment-13477961 ]
Michael Drzal commented on HBASE-6974:
--------------------------------------
I think I addressed all of your concerns except for the placement of currentTimeMillis. Let me know if I understood you correctly, and if so, I can make the switch.
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13490361#comment-13490361 ]
Hudson commented on HBASE-6974:
-------------------------------
Integrated in HBase-0.94-security-on-Hadoop-23 #9 (See [https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/9/])
HBASE-6974 Metric for blocked updates (Michael Drzal) (Revision 1399881)
Result = FAILURE
larsh :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 6974-v5.txt, HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch, HBASE-6974-v4.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478260#comment-13478260 ]
Michael Drzal commented on HBASE-6974:
--------------------------------------
Let me know if that works.
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Drzal updated HBASE-6974:
---------------------------------
Status: Patch Available (was: In Progress)
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6974) Metric for blocked updates
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6974:
---------------------------------
Attachment: 6974-v5.txt
Here's what I am going to commit soon.
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 6974-v5.txt, HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch, HBASE-6974-v4.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478468#comment-13478468 ]
Lars Hofhansl commented on HBASE-6974:
--------------------------------------
This is a low risk change, going to commit soon, unless I hear an objections.
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch, HBASE-6974-v4.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475975#comment-13475975 ]
Lars Hofhansl commented on HBASE-6974:
--------------------------------------
#2 is checked in MemStoreFlusher.reclaimMemStoreMemory. Currently there is not even a log message when that happens :(
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Drzal updated HBASE-6974:
---------------------------------
Attachment: HBASE-6974-v2.patch
Patch updated based on comments from Lars
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478093#comment-13478093 ]
Michael Drzal commented on HBASE-6974:
--------------------------------------
Sure, that works as long as you are ok with missing any time used by the initial call to requestFlush.
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478469#comment-13478469 ]
Hadoop QA commented on HBASE-6974:
----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12549585/HBASE-6974-v4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 82 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3070//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3070//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3070//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3070//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3070//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3070//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3070//console
This message is automatically generated.
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch, HBASE-6974-v4.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477397#comment-13477397 ]
Lars Hofhansl commented on HBASE-6974:
--------------------------------------
Looks good. Few minor comments:
* I think you snug a divider by 1024 in there to convert from ms to s :)
* We should also collect another metric when this situation happens in the memstore flusher (here it happens because of global memory pressure)
* Let's use EnvironmentEdge.currentTimeMillis()
* Nit: a call to currentTimeMillis is not free, we should only call it in the !blocked part inside the while loop (which means it cannot be final, has to be initialized with 0, etc)
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Drzal updated HBASE-6974:
---------------------------------
Attachment: HBASE-6974-v4.patch
fixing a test failure
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch, HBASE-6974-v4.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477416#comment-13477416 ]
Hadoop QA commented on HBASE-6974:
----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12549387/HBASE-6974.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 82 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3058//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3058//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3058//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3058//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3058//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3058//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3058//console
This message is automatically generated.
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6974) Metric for blocked updates
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6974:
---------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to 0.94 and 0.96.
Thanks for the patch, Drz. :)
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 6974-v5.txt, HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch, HBASE-6974-v4.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479192#comment-13479192 ]
Michael Drzal commented on HBASE-6974:
--------------------------------------
Sure, that is fine. Do you want me to spin a new patch?
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch, HBASE-6974-v4.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478010#comment-13478010 ]
Hadoop QA commented on HBASE-6974:
----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12549522/HBASE-6974-v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 82 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3064//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3064//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3064//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3064//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3064//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3064//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3064//console
This message is automatically generated.
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Drzal reassigned HBASE-6974:
------------------------------------
Assignee: Michael Drzal
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Fix For: 0.94.3, 0.96.0
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478320#comment-13478320 ]
Hadoop QA commented on HBASE-6974:
----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12549554/HBASE-6974-v3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 82 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3067//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3067//console
This message is automatically generated.
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475927#comment-13475927 ]
Lars Hofhansl commented on HBASE-6974:
--------------------------------------
It might be good to track the total amount of a time (seconds is probably the most useful resolution here) that writes are blocked (i.e. not just a counter).
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Fix For: 0.94.3, 0.96.0
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Drzal updated HBASE-6974:
---------------------------------
Status: Open (was: Patch Available)
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
> Attachments: HBASE-6974.patch, HBASE-6974-v2.patch, HBASE-6974-v3.patch
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6974) Metric for blocked updates
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13476960#comment-13476960 ]
Michael Drzal commented on HBASE-6974:
--------------------------------------
I'll try to get a patch up later today.
> Metric for blocked updates
> --------------------------
>
> Key: HBASE-6974
> URL: https://issues.apache.org/jira/browse/HBASE-6974
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Michael Drzal
> Priority: Critical
> Fix For: 0.94.3, 0.96.0
>
>
> When the disc subsystem cannot keep up with a sustained high write load, a region will eventually block updates to throttle clients.
> (HRegion.checkResources).
> It would be nice to have a metric for this, so that these occurrences can be tracked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira