You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Matteo Bertozzi (JIRA)" <ji...@apache.org> on 2012/04/26 21:28:31 UTC
[jira] [Created] (HBASE-5886) Add new metric for possible data loss
due to puts without WAL
Matteo Bertozzi created HBASE-5886:
--------------------------------------
Summary: Add new metric for possible data loss due to puts without WAL
Key: HBASE-5886
URL: https://issues.apache.org/jira/browse/HBASE-5886
Project: HBase
Issue Type: New Feature
Components: metrics, regionserver
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5886) Add new metric for possible data loss
due to puts without WAL
Posted by "Matteo Bertozzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matteo Bertozzi updated HBASE-5886:
-----------------------------------
Status: Patch Available (was: Open)
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5886) Add new metric for possible data loss
due to puts without WAL
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5886:
---------------------------------
Fix Version/s: 0.94.1
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 5886-0.94.patch, HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch, HBASE-5886-v3.patch, HBASE-5886-v4.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5886) Add new metric for possible data loss
due to puts without WAL
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl closed HBASE-5886.
--------------------------------
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Fix For: 0.94.1, 0.96.0
>
> Attachments: 5886-0.94.patch, HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch, HBASE-5886-v3.patch, HBASE-5886-v4.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268835#comment-13268835 ]
Hudson commented on HBASE-5886:
-------------------------------
Integrated in HBase-0.94-security #26 (See [https://builds.apache.org/job/HBase-0.94-security/26/])
HBASE-5886 Add new metric for possible data loss due to puts without WAL (Revision 1333689)
Result = SUCCESS
larsh :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 5886-0.94.patch, HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch, HBASE-5886-v3.patch, HBASE-5886-v4.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "Matteo Bertozzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266124#comment-13266124 ]
Matteo Bertozzi commented on HBASE-5886:
----------------------------------------
INFO level is good for me. The ideas is having something to grep.
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "Matteo Bertozzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13265995#comment-13265995 ]
Matteo Bertozzi commented on HBASE-5886:
----------------------------------------
Any other feedback on the patch?
Add a config option "hbase.warn.nowal" to write the warn message only if the option is set?
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5886) Add new metric for possible data loss
due to puts without WAL
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5886:
---------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to 0.94 too.
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 5886-0.94.patch, HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch, HBASE-5886-v3.patch, HBASE-5886-v4.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263111#comment-13263111 ]
Hadoop QA commented on HBASE-5886:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12524562/HBASE-5886-v0.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.io.TestHeapSize
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1659//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1659//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1659//console
This message is automatically generated.
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5886) Add new metric for possible data loss
due to puts without WAL
Posted by "Matteo Bertozzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matteo Bertozzi updated HBASE-5886:
-----------------------------------
Attachment: HBASE-5886-v1.patch
Renamed metrics and function, removed unused DEFAULT_ constant and the log on flush.
I'm not sure about how to get the client ip from the region, since the region shouldn't be aware of the client. Maybe moving something in Region Server?
heapSize() reflect the in memory state, so you've pointer size and so on... maybe using kv.getValueLength() + kv.getKeyLength() or just valueLength()?
I've to look at OperationMetrics, but seems that updatePutMetrics() is not called for doMiniBatchPut()... (I've to take a look at that)
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263939#comment-13263939 ]
Hadoop QA commented on HBASE-5886:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12524910/HBASE-5886-v2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 hadoop23. The patch compiles against the hadoop 0.23.x profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.io.TestHeapSize
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1671//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1671//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1671//console
This message is automatically generated.
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5886) Add new metric for possible data loss
due to puts without WAL
Posted by "Matteo Bertozzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matteo Bertozzi updated HBASE-5886:
-----------------------------------
Attachment: HBASE-5886-v0.patch
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5886) Add new metric for possible data loss
due to puts without WAL
Posted by "Matteo Bertozzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matteo Bertozzi updated HBASE-5886:
-----------------------------------
Attachment: (was: HBASE-5886-v2.patch)
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-5886) Add new metric for
possible data loss due to puts without WAL
Posted by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13265055#comment-13265055 ]
Nicolas Spiegelberg edited comment on HBASE-5886 at 4/30/12 5:55 PM:
---------------------------------------------------------------------
I'm confused about why this metric is useful. This metric is never accurate at determining data loss because querying it is async from the Put path. If you are looking for a restart point, you should have another thread call HTable.flush() and checkpoint or add an API to query for the latest timestamp in a CF's storefile.
Edit: s/and/at
was (Author: nspiegelberg):
I'm confused about why this metric is useful. This metric is never accurate and determining data loss because querying it is async from the Put path. If you are looking for a restart point, you should have another thread call HTable.flush() and checkpoint or add an API to query for the latest timestamp in a CF's storefile.
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5886) Add new metric for possible data loss
due to puts without WAL
Posted by "Matteo Bertozzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matteo Bertozzi updated HBASE-5886:
-----------------------------------
Attachment: HBASE-5886-v4.patch
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch, HBASE-5886-v3.patch, HBASE-5886-v4.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267953#comment-13267953 ]
Hudson commented on HBASE-5886:
-------------------------------
Integrated in HBase-TRUNK #2845 (See [https://builds.apache.org/job/HBase-TRUNK/2845/])
HBASE-5886 Add new metric for possible data loss due to puts without WAL (Revision 1333676)
Result = SUCCESS
stack :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 5886-0.94.patch, HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch, HBASE-5886-v3.patch, HBASE-5886-v4.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266102#comment-13266102 ]
Todd Lipcon commented on HBASE-5886:
------------------------------------
Or maybe drop it to INFO level?
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5886) Add new metric for possible data loss
due to puts without WAL
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-5886:
-------------------------
Status: Patch Available (was: Open)
Retrying to see that I got the heap sizings right.
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch, HBASE-5886-v3.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5886) Add new metric for possible data loss
due to puts without WAL
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5886:
---------------------------------
Attachment: 5886-0.94.patch
Yeah, would be good in 0.94. Here's the 0.94 patch.
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 5886-0.94.patch, HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch, HBASE-5886-v3.patch, HBASE-5886-v4.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13265055#comment-13265055 ]
Nicolas Spiegelberg commented on HBASE-5886:
--------------------------------------------
I'm confused about why this metric is useful. This metric is never accurate and determining data loss because querying it is async from the Put path. If you are looking for a restart point, you should have another thread call HTable.flush() and checkpoint or add an API to query for the latest timestamp in a CF's storefile.
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267642#comment-13267642 ]
Hadoop QA commented on HBASE-5886:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12525470/HBASE-5886-v4.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 hadoop23. The patch compiles against the hadoop 0.23.x profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1746//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1746//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1746//console
This message is automatically generated.
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch, HBASE-5886-v3.patch, HBASE-5886-v4.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263575#comment-13263575 ]
Hadoop QA commented on HBASE-5886:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12524847/HBASE-5886-v1.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 hadoop23. The patch compiles against the hadoop 0.23.x profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.io.TestHeapSize
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1668//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1668//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1668//console
This message is automatically generated.
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5886) Add new metric for possible data loss
due to puts without WAL
Posted by "Matteo Bertozzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matteo Bertozzi updated HBASE-5886:
-----------------------------------
Attachment: HBASE-5886-v1.patch
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5886) Add new metric for possible data loss
due to puts without WAL
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-5886:
-------------------------
Status: Open (was: Patch Available)
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch, HBASE-5886-v3.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5886) Add new metric for possible data loss
due to puts without WAL
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-5886:
-------------------------
Attachment: HBASE-5886-v3.patch
Hmm... trunk has changed. Here is an updated patch.
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch, HBASE-5886-v3.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13265425#comment-13265425 ]
Todd Lipcon commented on HBASE-5886:
------------------------------------
To fill in some color, we had a customer complaining of data loss after a crash. We spent many hours doing post mortem analysis, and only after we ruled out lots and lots of corner case possibilities did we come to find out that the user's application had set the writeToWAL flag false in a small number of their puts. Having the log message and the metric make it easy to say "yea, you got data loss, but here's a smoking gun which shows you explicitly disabled WAL for some puts in that region". This is important in lots of environments where HBase ops/support may be distinct from the users writing applications, and may not even have access to the source code of those applications.
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268154#comment-13268154 ]
Hudson commented on HBASE-5886:
-------------------------------
Integrated in HBase-TRUNK-security #191 (See [https://builds.apache.org/job/HBase-TRUNK-security/191/])
HBASE-5886 Add new metric for possible data loss due to puts without WAL (Revision 1333676)
Result = SUCCESS
stack :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 5886-0.94.patch, HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch, HBASE-5886-v3.patch, HBASE-5886-v4.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267919#comment-13267919 ]
stack commented on HBASE-5886:
------------------------------
Committed to trunk. Want this is 0.94 Lars?
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Fix For: 0.96.0
>
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch, HBASE-5886-v3.patch, HBASE-5886-v4.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263994#comment-13263994 ]
Hadoop QA commented on HBASE-5886:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12524913/HBASE-5886-v2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 hadoop23. The patch compiles against the hadoop 0.23.x profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1672//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1672//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1672//console
This message is automatically generated.
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268009#comment-13268009 ]
Hudson commented on HBASE-5886:
-------------------------------
Integrated in HBase-0.94 #177 (See [https://builds.apache.org/job/HBase-0.94/177/])
HBASE-5886 Add new metric for possible data loss due to puts without WAL (Revision 1333689)
Result = SUCCESS
larsh :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 5886-0.94.patch, HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch, HBASE-5886-v3.patch, HBASE-5886-v4.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "Matteo Bertozzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13265071#comment-13265071 ]
Matteo Bertozzi commented on HBASE-5886:
----------------------------------------
@Nicolas the metric is not meant to be precise but just to give an hint about possible data loss.
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262994#comment-13262994 ]
Todd Lipcon commented on HBASE-5886:
------------------------------------
I think the metrics names could be improved here -- "putWithoutWAL" could be "numPutsWithoutWAL" and "possibleDataLossSize" perhaps "mbInMemoryWithoutWAL"?
- in recordPossibleDataLossNoWal, use {{getAndIncrement}} instead of {{incrementAndGet}}, and compare the result to 0 -- otherwise you're doing an extra atomic op and have a potential race
- the warning message should include the client IP address as well as the region ID. Perhaps something like "Client 123.123.123.123 writing data to region abcdef12345 with WAL disabled. Data may be lost in the event of a crash. Will not log further warnings for this region."
- the debug message when setting possibleDataLossSize back to 0 seems unnecessary.
- DEFAULT_WARN_NO_WAL_INTERVAL is unused
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "Elliott Clark (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263129#comment-13263129 ]
Elliott Clark commented on HBASE-5886:
--------------------------------------
* This patch fails unit tests. org.apache.hadoop.hbase.io.TestHeapSize
* recordPossibleDataLossNoWal seems like the wrong name. There's no possible data loss until a region server crashes without flushing. Maybe something like recordPutWithoutWal.
* I don't think the warn in recordPossibleDataLossNoWal is the right level. Nothing has gone wrong; someone has just decided to opt for a speedy put vs a safe one. I'm not even sure that a log is needed here.
* How far off of the actual size of the data is the heapSize of the kv ?
* Putting data about puts without WAL into org.apache.hadoop.hbase.regionserver.metrics.OperationMetrics seems like it would be good.
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267951#comment-13267951 ]
Hadoop QA commented on HBASE-5886:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12525532/5886-0.94.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1756//console
This message is automatically generated.
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 5886-0.94.patch, HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch, HBASE-5886-v3.patch, HBASE-5886-v4.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5886) Add new metric for possible data loss
due to puts without WAL
Posted by "Matteo Bertozzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matteo Bertozzi updated HBASE-5886:
-----------------------------------
Attachment: (was: HBASE-5886-v1.patch)
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5886) Add new metric for possible data loss
due to puts without WAL
Posted by "Matteo Bertozzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matteo Bertozzi updated HBASE-5886:
-----------------------------------
Attachment: HBASE-5886-v2.patch
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data
loss due to puts without WAL
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266117#comment-13266117 ]
stack commented on HBASE-5886:
------------------------------
I love the war story. Any more where that came from for our entertainment?
Patch looks good to me. I can drop the log level on commit if thats what you want Matteo.
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5886) Add new metric for possible data loss
due to puts without WAL
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-5886:
-------------------------
Status: Patch Available (was: Open)
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch, HBASE-5886-v3.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5886) Add new metric for possible data loss
due to puts without WAL
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-5886:
-------------------------
Fix Version/s: 0.96.0
Leaving open till we figure if it belongs in 0.94.1
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Fix For: 0.96.0
>
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch, HBASE-5886-v3.patch, HBASE-5886-v4.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5886) Add new metric for possible data loss
due to puts without WAL
Posted by "Matteo Bertozzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matteo Bertozzi updated HBASE-5886:
-----------------------------------
Attachment: HBASE-5886-v2.patch
> Add new metric for possible data loss due to puts without WAL
> --------------------------------------------------------------
>
> Key: HBASE-5886
> URL: https://issues.apache.org/jira/browse/HBASE-5886
> Project: HBase
> Issue Type: New Feature
> Components: metrics, regionserver
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Labels: metrics
> Attachments: HBASE-5886-v0.patch, HBASE-5886-v1.patch, HBASE-5886-v2.patch
>
>
> Add a metrics to keep track of puts without WAL and possible data loss size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira