You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hbase.apache.org by st...@apache.org on 2016/11/14 18:28:14 UTC
hbase git commit: HBASE-17089 Add doc on experience running with
hedged reads
Repository: hbase
Updated Branches:
refs/heads/master c3685760f -> 6ee2dcf48
HBASE-17089 Add doc on experience running with hedged reads
Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/6ee2dcf4
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/6ee2dcf4
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/6ee2dcf4
Branch: refs/heads/master
Commit: 6ee2dcf480dd95877a20e33086a020eb1a19e41f
Parents: c368576
Author: Michael Stack <st...@apache.org>
Authored: Mon Nov 14 10:27:58 2016 -0800
Committer: Michael Stack <st...@apache.org>
Committed: Mon Nov 14 10:28:09 2016 -0800
----------------------------------------------------------------------
src/main/asciidoc/_chapters/performance.adoc | 12 +++++++++++-
src/main/asciidoc/_chapters/protobuf.adoc | 5 +++--
2 files changed, 14 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hbase/blob/6ee2dcf4/src/main/asciidoc/_chapters/performance.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/performance.adoc b/src/main/asciidoc/_chapters/performance.adoc
index 5f27640..bc8b48a 100644
--- a/src/main/asciidoc/_chapters/performance.adoc
+++ b/src/main/asciidoc/_chapters/performance.adoc
@@ -156,6 +156,10 @@ See <<block.cache>>
See <<recommended_configurations>>.
+[[perf.99th.percentile]]
+=== Improving the 99th Percentile
+Try link:[hedged_reads].
+
[[perf.compactions.and.splits]]
=== Managing Compactions
@@ -751,9 +755,10 @@ Most people should leave this alone.
Default = 7, or can collapse to at least 1/128th of original size.
See the _Development Process_ section of the document link:https://issues.apache.org/jira/secure/attachment/12444007/Bloom_Filters_in_HBase.pdf[BloomFilters in HBase] for more on what this option means.
+[[hedged.reads]]
=== Hedged Reads
-Hedged reads are a feature of HDFS, introduced in link:https://issues.apache.org/jira/browse/HDFS-5776[HDFS-5776].
+Hedged reads are a feature of HDFS, introduced in Hadoop 2.4.0 with link:https://issues.apache.org/jira/browse/HDFS-5776[HDFS-5776].
Normally, a single thread is spawned for each read request.
However, if hedged reads are enabled, the client waits some configurable amount of time, and if the read does not return, the client spawns a second read request, against a different block replica of the same data.
Whichever read returns first is used, and the other read request is discarded.
@@ -790,6 +795,11 @@ See <<hbase_metrics>> for more information.
* hedgeReadOpsWin - the number of times the hedged read thread was faster than the original thread.
This could indicate that a given RegionServer is having trouble servicing requests.
+Concerns while running with hedged reads enabled include:
+
+* They may lead to network congestion. See link:https://issues.apache.org/jira/browse/HBASE-17083[HBASE-17083]
+* Make sure you set the thread pool large enough so as blocking on the pool does not become a bottleneck (Again see link:https://issues.apache.org/jira/browse/HBASE-17083[HBASE-17083])
+
[[perf.deleting]]
== Deleting from HBase
http://git-wip-us.apache.org/repos/asf/hbase/blob/6ee2dcf4/src/main/asciidoc/_chapters/protobuf.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/protobuf.adoc b/src/main/asciidoc/_chapters/protobuf.adoc
index fa63127..1c2cc47 100644
--- a/src/main/asciidoc/_chapters/protobuf.adoc
+++ b/src/main/asciidoc/_chapters/protobuf.adoc
@@ -27,6 +27,8 @@
:icons: font
:experimental:
+
+== Protobuf
HBase uses Google's link:http://protobuf.protobufs[protobufs] wherever
it persists metadata -- in the tail of hfiles or Cells written by
HBase into the system hbase;meta table or when HBase writes znodes
@@ -36,12 +38,11 @@ Interfaces (Services) we expose to clients, for example the `Admin` and `Client`
Interfaces that the RegionServer fields,
or specifying the arbitrary extensions added by developers via our
xref:cp[Coprocessor Endpoint] mechanism.
+
In this chapter we go into detail for developers who are looking to
understand better how it all works. This chapter is of particular
use to those who would amend or extend HBase functionality.
-== Protobuf
-
With protobuf, you describe serializations and services in a `.protos` file.
You then feed these descriptors to a protobuf tool, the `protoc` binary,
to generate classes that can marshall and unmarshall the described serializations