You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hbase.apache.org by ap...@apache.org on 2015/08/16 20:24:52 UTC
[1/2] hbase git commit: Update POM and CHANGES.txt for 0.98.14
Repository: hbase
Updated Branches:
refs/heads/0.98 a4a003fc6 -> db2ea9780
Update POM and CHANGES.txt for 0.98.14
Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/bd72eacb
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/bd72eacb
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/bd72eacb
Branch: refs/heads/0.98
Commit: bd72eacb1eafde538ee79060a360423ad20ef89b
Parents: a4a003f
Author: Andrew Purtell <ap...@apache.org>
Authored: Fri Aug 14 15:51:53 2015 -0700
Committer: Andrew Purtell <ap...@apache.org>
Committed: Fri Aug 14 18:20:23 2015 -0700
----------------------------------------------------------------------
CHANGES.txt | 91 ++++++++++++++++++++++++++++++++++++++
hbase-annotations/pom.xml | 2 +-
hbase-assembly/pom.xml | 2 +-
hbase-checkstyle/pom.xml | 4 +-
hbase-client/pom.xml | 2 +-
hbase-common/pom.xml | 2 +-
hbase-examples/pom.xml | 2 +-
hbase-hadoop-compat/pom.xml | 2 +-
hbase-hadoop1-compat/pom.xml | 2 +-
hbase-hadoop2-compat/pom.xml | 2 +-
hbase-it/pom.xml | 2 +-
hbase-prefix-tree/pom.xml | 2 +-
hbase-protocol/pom.xml | 2 +-
hbase-resource-bundle/pom.xml | 2 +-
hbase-rest/pom.xml | 2 +-
hbase-server/pom.xml | 2 +-
hbase-shell/pom.xml | 2 +-
hbase-testing-util/pom.xml | 2 +-
hbase-thrift/pom.xml | 2 +-
pom.xml | 2 +-
20 files changed, 111 insertions(+), 20 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/CHANGES.txt
----------------------------------------------------------------------
diff --git a/CHANGES.txt b/CHANGES.txt
index d5abaf4..09f59c5 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,96 @@
HBase Change Log
+Release 0.98.14 - 8/21/2015
+
+** Sub-task
+ * [HBASE-7847] - Use zookeeper multi to clear znodes
+ * [HBASE-13730] - Backport HBASE-13576 (Failure in checking one region should not fail the entire HBCK operation) to 0.98
+ * [HBASE-13743] - Backport HBASE-13709 (Updates to meta table server columns may be eclipsed) to 0.98
+ * [HBASE-13899] - Jacoco instrumentation fails under jdk8
+ * [HBASE-13937] - Partially revert HBASE-13172
+ * [HBASE-13963] - avoid leaking jdk.tools
+ * [HBASE-14086] - remove unused bundled dependencies
+ * [HBASE-14105] - Add shell tests for Snapshot
+ * [HBASE-14176] - Add missing headers to META-INF files
+
+** Bug
+ * [HBASE-5878] - Use getVisibleLength public api from HdfsDataInputStream from Hadoop-2.
+ * [HBASE-10844] - Coprocessor failure during batchmutation leaves the memstore datastructs in an inconsistent state
+ * [HBASE-12865] - WALs may be deleted before they are replicated to peers
+ * [HBASE-12967] - Invalid FQCNs in alter table command leaves the table unusable
+ * [HBASE-13561] - ITBLL.Verify doesn't actually evaluate counters after job completes
+ * [HBASE-13646] - HRegion#execService should not try to build incomplete messages
+ * [HBASE-13706] - CoprocessorClassLoader should not exempt Hive classes
+ * [HBASE-13825] - Use ProtobufUtil#mergeFrom and ProtobufUtil#mergeDelimitedFrom in place of builder methods of same name
+ * [HBASE-13835] - KeyValueHeap.current might be in heap when exception happens in pollRealKV
+ * [HBASE-13849] - Remove restore and clone snapshot from the WebUI
+ * [HBASE-13861] - BucketCacheTmpl.jamon has wrong bucket free and used labels
+ * [HBASE-13865] - Increase the default value for hbase.hregion.memstore.block.multipler from 2 to 4 (part 2)
+ * [HBASE-13877] - Interrupt to flush from TableFlushProcedure causes dataloss in ITBLL
+ * [HBASE-13878] - Set hbase.fs.tmp.dir config in HBaseTestingUtility.java for Phoenix UT to use
+ * [HBASE-13881] - Bug in HTable#incrementColumnValue implementation
+ * [HBASE-13885] - ZK watches leaks during snapshots
+ * [HBASE-13892] - Scanner with all results filtered out results in NPE
+ * [HBASE-13897] - OOM may occur when Import imports a row with too many KeyValues
+ * [HBASE-13901] - Error while calling watcher on creating and deleting an HBase table
+ * [HBASE-13918] - Fix hbase:namespace description in webUI
+ * [HBASE-13923] - Loaded region coprocessors are not reported in shell status command
+ * [HBASE-13933] - DBE's seekBefore with tags corrupts the tag's offset information thus leading to incorrect results
+ * [HBASE-13935] - Orphaned namespace table ZK node should not prevent master to start
+ * [HBASE-13938] - Deletes done during the region merge transaction may get eclipsed
+ * [HBASE-13945] - Prefix_Tree seekBefore() does not work correctly
+ * [HBASE-13958] - RESTApiClusterManager calls kill() instead of suspend() and resume()
+ * [HBASE-13959] - Region splitting uses a single thread in most common cases
+ * [HBASE-13969] - AuthenticationTokenSecretManager is never stopped in RPCServer
+ * [HBASE-13970] - NPE during compaction in trunk
+ * [HBASE-13978] - Variable never assigned in SimpleTotalOrderPartitioner.getPartition()
+ * [HBASE-13988] - Add exception handler for lease thread
+ * [HBASE-13995] - ServerName is not fully case insensitive
+ * [HBASE-14005] - Set permission to .top hfile in LoadIncrementalHFiles
+ * [HBASE-14022] - TestMultiTableSnapshotInputFormatImpl uses a class only available in JRE 1.7+
+ * [HBASE-14042] - Fix FATAL level logging in FSHLog where logged for non fatal exceptions
+ * [HBASE-14048] - isPBMagicPrefix should always check for null data
+ * [HBASE-14050] - NPE in org.apache.hadoop.hbase.ipc.RpcServer$Connection.readAndProcess
+ * [HBASE-14054] - Acknowledged writes may get lost if regionserver clock is set backwards
+ * [HBASE-14089] - Remove unnecessary draw of system entropy from RecoverableZooKeeper
+ * [HBASE-14146] - Once replication sees an error it slows down forever
+ * [HBASE-14157] - Interfaces implemented by subclasses should be checked when registering CoprocessorService
+ * [HBASE-14168] - Avoid useless retry for DoNotRetryIOException in TableRecordReaderImpl
+ * [HBASE-14178] - regionserver blocks because of waiting for offsetLock
+ * [HBASE-14195] - Closed connections are not removed from ConnectionCache (Thrift, REST)
+ * [HBASE-14196] - Thrift server idle connection timeout issue
+ * [HBASE-14199] - maven-remote-resources-plugin failure processing NOTICE.vm in hbase-assembly
+ * [HBASE-14209] - TestShell visibility tests failing
+ * [HBASE-14214] - list_labels shouldn't raise ArgumentError if no labels are defined
+
+** Improvement
+ * [HBASE-8642] - [Snapshot] List and delete snapshot by table
+ * [HBASE-12596] - bulkload needs to follow locality
+ * [HBASE-13755] - Provide single super user check implementation
+ * [HBASE-13925] - Use zookeeper multi to clear znodes in ZKProcedureUtil
+ * [HBASE-13961] - SnapshotManager#initialize should set snapshotLayoutVersion if it allows to create snapshot with old layout format
+ * [HBASE-13985] - Add configuration to skip validating HFile format when bulk loading
+ * [HBASE-14002] - Add --noReplicationSetup option to IntegrationTestReplication
+ * [HBASE-14097] - Log link to client scan troubleshooting section when scanner exceptions happen.
+ * [HBASE-14122] - Client API for determining if server side supports cell level security
+
+** New Feature
+ * [HBASE-13356] - HBase should provide an InputFormat supporting multiple scans in mapreduce jobs over snapshots
+ * [HBASE-13639] - SyncTable - rsync for HBase tables
+ * [HBASE-14154] - DFS Replication should be configurable at column family level
+
+** Task
+ * [HBASE-11276] - Add back support for running ChaosMonkey as standalone tool
+ * [HBASE-13666] - book.pdf is not renamed during site build
+ * [HBASE-13994] - Backport HBASE-13917 (Remove string comparison to identify request priority) to 0.98
+ * [HBASE-14057] - HBase shell user_permission should list super users defined on hbase-site.xml
+ * [HBASE-14085] - Correct LICENSE and NOTICE files in artifacts
+
+** Test
+ * [HBASE-11819] - Unit test for CoprocessorHConnection
+ * [HBASE-13940] - IntegrationTestBulkLoad needs option to specify output folders used by test
+
+
Release 0.98.13 - 6/18/2015
** Sub-task
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/hbase-annotations/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-annotations/pom.xml b/hbase-annotations/pom.xml
index 40ff44b..4e1f9b1 100644
--- a/hbase-annotations/pom.xml
+++ b/hbase-annotations/pom.xml
@@ -23,7 +23,7 @@
<parent>
<artifactId>hbase</artifactId>
<groupId>org.apache.hbase</groupId>
- <version>0.98.14-SNAPSHOT</version>
+ <version>0.98.14</version>
<relativePath>..</relativePath>
</parent>
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/hbase-assembly/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-assembly/pom.xml b/hbase-assembly/pom.xml
index 4570f64..708c29f 100644
--- a/hbase-assembly/pom.xml
+++ b/hbase-assembly/pom.xml
@@ -23,7 +23,7 @@
<parent>
<artifactId>hbase</artifactId>
<groupId>org.apache.hbase</groupId>
- <version>0.98.14-SNAPSHOT</version>
+ <version>0.98.14</version>
<relativePath>..</relativePath>
</parent>
<artifactId>hbase-assembly</artifactId>
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/hbase-checkstyle/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-checkstyle/pom.xml b/hbase-checkstyle/pom.xml
index e73e48c..ab8bd3b 100644
--- a/hbase-checkstyle/pom.xml
+++ b/hbase-checkstyle/pom.xml
@@ -24,14 +24,14 @@
<modelVersion>4.0.0</modelVersion>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-checkstyle</artifactId>
-<version>0.98.14-SNAPSHOT</version>
+<version>0.98.14</version>
<name>Apache HBase - Checkstyle</name>
<description>Module to hold Checkstyle properties for HBase.</description>
<parent>
<artifactId>hbase</artifactId>
<groupId>org.apache.hbase</groupId>
- <version>0.98.14-SNAPSHOT</version>
+ <version>0.98.14</version>
<relativePath>..</relativePath>
</parent>
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/hbase-client/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-client/pom.xml b/hbase-client/pom.xml
index 268a1f7..7d78acd 100644
--- a/hbase-client/pom.xml
+++ b/hbase-client/pom.xml
@@ -24,7 +24,7 @@
<parent>
<artifactId>hbase</artifactId>
<groupId>org.apache.hbase</groupId>
- <version>0.98.14-SNAPSHOT</version>
+ <version>0.98.14</version>
<relativePath>..</relativePath>
</parent>
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/hbase-common/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-common/pom.xml b/hbase-common/pom.xml
index 8c015fa..4790823 100644
--- a/hbase-common/pom.xml
+++ b/hbase-common/pom.xml
@@ -23,7 +23,7 @@
<parent>
<artifactId>hbase</artifactId>
<groupId>org.apache.hbase</groupId>
- <version>0.98.14-SNAPSHOT</version>
+ <version>0.98.14</version>
<relativePath>..</relativePath>
</parent>
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/hbase-examples/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-examples/pom.xml b/hbase-examples/pom.xml
index 7a4fb13..8e48aae 100644
--- a/hbase-examples/pom.xml
+++ b/hbase-examples/pom.xml
@@ -23,7 +23,7 @@
<parent>
<artifactId>hbase</artifactId>
<groupId>org.apache.hbase</groupId>
- <version>0.98.14-SNAPSHOT</version>
+ <version>0.98.14</version>
<relativePath>..</relativePath>
</parent>
<artifactId>hbase-examples</artifactId>
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/hbase-hadoop-compat/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-hadoop-compat/pom.xml b/hbase-hadoop-compat/pom.xml
index 7915c4c..c6ac876 100644
--- a/hbase-hadoop-compat/pom.xml
+++ b/hbase-hadoop-compat/pom.xml
@@ -23,7 +23,7 @@
<parent>
<artifactId>hbase</artifactId>
<groupId>org.apache.hbase</groupId>
- <version>0.98.14-SNAPSHOT</version>
+ <version>0.98.14</version>
<relativePath>..</relativePath>
</parent>
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/hbase-hadoop1-compat/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-hadoop1-compat/pom.xml b/hbase-hadoop1-compat/pom.xml
index abf28a9..6e5c610 100644
--- a/hbase-hadoop1-compat/pom.xml
+++ b/hbase-hadoop1-compat/pom.xml
@@ -21,7 +21,7 @@ limitations under the License.
<parent>
<artifactId>hbase</artifactId>
<groupId>org.apache.hbase</groupId>
- <version>0.98.14-SNAPSHOT</version>
+ <version>0.98.14</version>
<relativePath>..</relativePath>
</parent>
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/hbase-hadoop2-compat/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-hadoop2-compat/pom.xml b/hbase-hadoop2-compat/pom.xml
index 10ff4f4..639bc4e 100644
--- a/hbase-hadoop2-compat/pom.xml
+++ b/hbase-hadoop2-compat/pom.xml
@@ -21,7 +21,7 @@ limitations under the License.
<parent>
<artifactId>hbase</artifactId>
<groupId>org.apache.hbase</groupId>
- <version>0.98.14-SNAPSHOT</version>
+ <version>0.98.14</version>
<relativePath>..</relativePath>
</parent>
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/hbase-it/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-it/pom.xml b/hbase-it/pom.xml
index 21c4986..758da3c 100644
--- a/hbase-it/pom.xml
+++ b/hbase-it/pom.xml
@@ -23,7 +23,7 @@
<parent>
<artifactId>hbase</artifactId>
<groupId>org.apache.hbase</groupId>
- <version>0.98.14-SNAPSHOT</version>
+ <version>0.98.14</version>
<relativePath>..</relativePath>
</parent>
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/hbase-prefix-tree/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-prefix-tree/pom.xml b/hbase-prefix-tree/pom.xml
index 5fb38f7..dfa5f7b 100644
--- a/hbase-prefix-tree/pom.xml
+++ b/hbase-prefix-tree/pom.xml
@@ -23,7 +23,7 @@
<parent>
<artifactId>hbase</artifactId>
<groupId>org.apache.hbase</groupId>
- <version>0.98.14-SNAPSHOT</version>
+ <version>0.98.14</version>
<relativePath>..</relativePath>
</parent>
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/hbase-protocol/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-protocol/pom.xml b/hbase-protocol/pom.xml
index a081720..94a43a7 100644
--- a/hbase-protocol/pom.xml
+++ b/hbase-protocol/pom.xml
@@ -23,7 +23,7 @@
<parent>
<artifactId>hbase</artifactId>
<groupId>org.apache.hbase</groupId>
- <version>0.98.14-SNAPSHOT</version>
+ <version>0.98.14</version>
<relativePath>..</relativePath>
</parent>
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/hbase-resource-bundle/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-resource-bundle/pom.xml b/hbase-resource-bundle/pom.xml
index 08ffe64..428ec47 100644
--- a/hbase-resource-bundle/pom.xml
+++ b/hbase-resource-bundle/pom.xml
@@ -23,7 +23,7 @@
<parent>
<artifactId>hbase</artifactId>
<groupId>org.apache.hbase</groupId>
- <version>0.98.14-SNAPSHOT</version>
+ <version>0.98.14</version>
<relativePath>..</relativePath>
</parent>
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/hbase-rest/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-rest/pom.xml b/hbase-rest/pom.xml
index 788a2fd..f567551 100644
--- a/hbase-rest/pom.xml
+++ b/hbase-rest/pom.xml
@@ -25,7 +25,7 @@
<parent>
<artifactId>hbase</artifactId>
<groupId>org.apache.hbase</groupId>
- <version>0.98.14-SNAPSHOT</version>
+ <version>0.98.14</version>
<relativePath>..</relativePath>
</parent>
<artifactId>hbase-rest</artifactId>
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/hbase-server/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-server/pom.xml b/hbase-server/pom.xml
index 417e983..b1f8e5d 100644
--- a/hbase-server/pom.xml
+++ b/hbase-server/pom.xml
@@ -23,7 +23,7 @@
<parent>
<artifactId>hbase</artifactId>
<groupId>org.apache.hbase</groupId>
- <version>0.98.14-SNAPSHOT</version>
+ <version>0.98.14</version>
<relativePath>..</relativePath>
</parent>
<artifactId>hbase-server</artifactId>
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/hbase-shell/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-shell/pom.xml b/hbase-shell/pom.xml
index 4e6bfa9..83d2135 100644
--- a/hbase-shell/pom.xml
+++ b/hbase-shell/pom.xml
@@ -23,7 +23,7 @@
<parent>
<artifactId>hbase</artifactId>
<groupId>org.apache.hbase</groupId>
- <version>0.98.14-SNAPSHOT</version>
+ <version>0.98.14</version>
<relativePath>..</relativePath>
</parent>
<artifactId>hbase-shell</artifactId>
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/hbase-testing-util/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-testing-util/pom.xml b/hbase-testing-util/pom.xml
index caa2cf6..350007d 100644
--- a/hbase-testing-util/pom.xml
+++ b/hbase-testing-util/pom.xml
@@ -23,7 +23,7 @@
<parent>
<artifactId>hbase</artifactId>
<groupId>org.apache.hbase</groupId>
- <version>0.98.14-SNAPSHOT</version>
+ <version>0.98.14</version>
<relativePath>..</relativePath>
</parent>
<artifactId>hbase-testing-util</artifactId>
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/hbase-thrift/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-thrift/pom.xml b/hbase-thrift/pom.xml
index 4a07ca7..054a75a 100644
--- a/hbase-thrift/pom.xml
+++ b/hbase-thrift/pom.xml
@@ -23,7 +23,7 @@
<parent>
<artifactId>hbase</artifactId>
<groupId>org.apache.hbase</groupId>
- <version>0.98.14-SNAPSHOT</version>
+ <version>0.98.14</version>
<relativePath>..</relativePath>
</parent>
<artifactId>hbase-thrift</artifactId>
http://git-wip-us.apache.org/repos/asf/hbase/blob/bd72eacb/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index f752b93..29e6022 100644
--- a/pom.xml
+++ b/pom.xml
@@ -39,7 +39,7 @@
<groupId>org.apache.hbase</groupId>
<artifactId>hbase</artifactId>
<packaging>pom</packaging>
- <version>0.98.14-SNAPSHOT</version>
+ <version>0.98.14</version>
<name>Apache HBase</name>
<description>
Apache HBase is the Hadoop database. Use it when you need
[2/2] hbase git commit: Update docs from master
Posted by ap...@apache.org.
Update docs from master
Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/db2ea978
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/db2ea978
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/db2ea978
Branch: refs/heads/0.98
Commit: db2ea9780adc94ed31203a55980f5ddc25cddfd0
Parents: bd72eac
Author: Andrew Purtell <ap...@apache.org>
Authored: Fri Aug 14 16:09:05 2015 -0700
Committer: Andrew Purtell <ap...@apache.org>
Committed: Fri Aug 14 18:20:29 2015 -0700
----------------------------------------------------------------------
src/main/asciidoc/_chapters/architecture.adoc | 177 +++++++++-----
src/main/asciidoc/_chapters/community.adoc | 10 +-
src/main/asciidoc/_chapters/configuration.adoc | 38 ++-
src/main/asciidoc/_chapters/developer.adoc | 38 ++-
.../asciidoc/_chapters/getting_started.adoc | 9 +-
src/main/asciidoc/_chapters/hbase_mob.adoc | 236 +++++++++++++++++++
src/main/asciidoc/_chapters/ops_mgt.adoc | 14 +-
src/main/asciidoc/_chapters/performance.adoc | 2 +-
src/main/asciidoc/_chapters/rpc.adoc | 3 +-
src/main/asciidoc/_chapters/schema_design.adoc | 16 +-
src/main/asciidoc/_chapters/security.adoc | 18 ++
.../asciidoc/_chapters/troubleshooting.adoc | 33 +++
src/main/asciidoc/_chapters/upgrading.adoc | 8 +
src/main/asciidoc/book.adoc | 1 +
src/main/site/asciidoc/sponsors.adoc | 1 +
src/main/site/resources/doap_Hbase.rdf | 14 +-
src/main/site/site.xml | 4 +-
src/main/site/xdoc/index.xml | 1 +
18 files changed, 513 insertions(+), 110 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hbase/blob/db2ea978/src/main/asciidoc/_chapters/architecture.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/architecture.adoc b/src/main/asciidoc/_chapters/architecture.adoc
index 659c4ee..740b585 100644
--- a/src/main/asciidoc/_chapters/architecture.adoc
+++ b/src/main/asciidoc/_chapters/architecture.adoc
@@ -1016,16 +1016,7 @@ For background, see link:https://issues.apache.org/jira/browse/HBASE-2643[HBASE-
===== Performance Improvements during Log Splitting
-WAL log splitting and recovery can be resource intensive and take a long time, depending on the number of RegionServers involved in the crash and the size of the regions. <<distributed.log.splitting>> and <<distributed.log.replay>> were developed to improve performance during log splitting.
-
-[[distributed.log.splitting]]
-====== Distributed Log Splitting
-
-_Distributed Log Splitting_ was added in HBase version 0.92 (link:https://issues.apache.org/jira/browse/HBASE-1364[HBASE-1364]) by Prakash Khemani from Facebook.
-It reduces the time to complete log splitting dramatically, improving the availability of regions and tables.
-For example, recovering a crashed cluster took around 9 hours with single-threaded log splitting, but only about six minutes with distributed log splitting.
-
-The information in this section is sourced from Jimmy Xiang's blog post at http://blog.cloudera.com/blog/2012/07/hbase-log-splitting/.
+WAL log splitting and recovery can be resource intensive and take a long time, depending on the number of RegionServers involved in the crash and the size of the regions. <<distributed.log.splitting>> was developed to improve performance during log splitting.
.Enabling or Disabling Distributed Log Splitting
@@ -2222,18 +2213,6 @@ The region replica having replica_id==0 is called the primary region, and the ot
Only the primary can accept writes from the client, and the primary will always contain the latest changes.
Since all writes still have to go through the primary region, the writes are not highly-available (meaning they might block for some time if the region becomes unavailable).
-The writes are asynchronously sent to the secondary region replicas using an _Async WAL replication_ feature.
-This works similarly to HBase's multi-datacenter replication, but instead the data from a region is replicated to the secondary regions.
-Each secondary replica always receives and observes the writes in the same order that the primary region committed them.
-This ensures that the secondaries won't diverge from the primary regions data, but since the log replication is asnyc, the data might be stale in secondary regions.
-In some sense, this design can be thought of as _in-cluster replication_, where instead of replicating to a different datacenter, the data goes to a secondary region to keep secondary region's in-memory state up to date.
-The data files are shared between the primary region and the other replicas, so that there is no extra storage overhead.
-However, the secondary regions will have recent non-flushed data in their MemStores, which increases the memory overhead.
-
-
-Async WAL replication feature is being implemented in Phase 2 of issue HBASE-10070.
-Before this, region replicas will only be updated with flushed data files from the primary (see `hbase.regionserver.storefile.refresh.period` below). It is also possible to use this without setting `storefile.refresh.period` for read only tables.
-
=== Timeline Consistency
@@ -2273,8 +2252,8 @@ In terms of semantics, TIMELINE consistency as implemented by HBase differs from
There is no stickiness to region replicas or a transaction-id based guarantee.
If required, this can be implemented later though.
-.HFile Version 1
-image::timeline_consistency.png[HFile Version 1]
+.Timeline Consistency
+image::timeline_consistency.png[Timeline Consistency]
To better understand the TIMELINE semantics, lets look at the above diagram.
Lets say that there are two clients, and the first one writes x=1 at first, then x=2 and x=3 later.
@@ -2309,11 +2288,52 @@ Following are advantages and disadvantages.
To serve the region data from multiple replicas, HBase opens the regions in secondary mode in the region servers.
The regions opened in secondary mode will share the same data files with the primary region replica, however each secondary region replica will have its own MemStore to keep the unflushed data (only primary region can do flushes). Also to serve reads from secondary regions, the blocks of data files may be also cached in the block caches for the secondary regions.
+=== Where is the code
+This feature is delivered in two phases, Phase 1 and 2. The first phase is done in time for HBase-1.0.0 release. Meaning that using HBase-1.0.x, you can use all the features that are marked for Phase 1. Phase 2 is committed in HBase-1.1.0, meaning all HBase versions after 1.1.0 should contain Phase 2 items.
+
+=== Propagating writes to region replicas
+As discussed above writes only go to the primary region replica. For propagating the writes from the primary region replica to the secondaries, there are two different mechanisms. For read-only tables, you do not need to use any of the following methods. Disabling and enabling the table should make the data available in all region replicas. For mutable tables, you have to use *only* one of the following mechanisms: storefile refresher, or async wal replication. The latter is recommeded.
+
+==== StoreFile Refresher
+The first mechanism is store file refresher which is introduced in HBase-1.0+. Store file refresher is a thread per region server, which runs periodically, and does a refresh operation for the store files of the primary region for the secondary region replicas. If enabled, the refresher will ensure that the secondary region replicas see the new flushed, compacted or bulk loaded files from the primary region in a timely manner. However, this means that only flushed data can be read back from the secondary region replicas, and after the refresher is run, making the secondaries lag behind the primary for an a longer time.
+
+For turning this feature on, you should configure `hbase.regionserver.storefile.refresh.period` to a non-zero value. See Configuration section below.
+
+==== Asnyc WAL replication
+The second mechanism for propagation of writes to secondaries is done via “Async WAL Replication” feature and is only available in HBase-1.1+. This works similarly to HBase’s multi-datacenter replication, but instead the data from a region is replicated to the secondary regions. Each secondary replica always receives and observes the writes in the same order that the primary region committed them. In some sense, this design can be thought of as “in-cluster replication”, where instead of replicating to a different datacenter, the data goes to secondary regions to keep secondary region’s in-memory state up to date. The data files are shared between the primary region and the other replicas, so that there is no extra storage overhead. However, the secondary regions will have recent non-flushed data in their memstores, which increases the memory overhead. The primary region writes flush, compaction, and bulk load events to its WAL as well, which are also replicated through w
al replication to secondaries. When they observe the flush/compaction or bulk load event, the secondary regions replay the event to pick up the new files and drop the old ones.
+
+Committing writes in the same order as in primary ensures that the secondaries won’t diverge from the primary regions data, but since the log replication is asynchronous, the data might still be stale in secondary regions. Since this feature works as a replication endpoint, the performance and latency characteristics is expected to be similar to inter-cluster replication.
+
+Async WAL Replication is *disabled* by default. You can enable this feature by setting `hbase.region.replica.replication.enabled` to `true`.
+Asyn WAL Replication feature will add a new replication peer named `region_replica_replication` as a replication peer when you create a table with region replication > 1 for the first time. Once enabled, if you want to disable this feature, you need to do two actions:
+* Set configuration property `hbase.region.replica.replication.enabled` to false in `hbase-site.xml` (see Configuration section below)
+* Disable the replication peer named `region_replica_replication` in the cluster using hbase shell or `ReplicationAdmin` class:
+[source,bourne]
+----
+ hbase> disable_peer 'region_replica_replication'
+----
+
+=== Store File TTL
+In both of the write propagation approaches mentioned above, store files of the primary will be opened in secondaries independent of the primary region. So for files that the primary compacted away, the secondaries might still be referring to these files for reading. Both features are using HFileLinks to refer to files, but there is no protection (yet) for guaranteeing that the file will not be deleted prematurely. Thus, as a guard, you should set the configuration property `hbase.master.hfilecleaner.ttl` to a larger value, such as 1 hour to guarantee that you will not receive IOExceptions for requests going to replicas.
+
+=== Region replication for META table’s region
+Currently, Async WAL Replication is not done for the META table’s WAL. The meta table’s secondary replicas still refreshes themselves from the persistent store files. Hence the `hbase.regionserver.meta.storefile.refresh.period` needs to be set to a certain non-zero value for refreshing the meta store files. Note that this configuration is configured differently than
+`hbase.regionserver.storefile.refresh.period`.
+
+=== Memory accounting
+The secondary region replicas refer to the data files of the primary region replica, but they have their own memstores (in HBase-1.1+) and uses block cache as well. However, one distinction is that the secondary region replicas cannot flush the data when there is memory pressure for their memstores. They can only free up memstore memory when the primary region does a flush and this flush is replicated to the secondary. Since in a region server hosting primary replicas for some regions and secondaries for some others, the secondaries might cause extra flushes to the primary regions in the same host. In extreme situations, there can be no memory left for adding new writes coming from the primary via wal replication. For unblocking this situation (and since secondary cannot flush by itself), the secondary is allowed to do a “store file refresh” by doing a file system list operation to pick up new files from primary, and possibly dropping its memstore. This refresh will only be perf
ormed if the memstore size of the biggest secondary region replica is at least `hbase.region.replica.storefile.refresh.memstore.multiplier` (default 4) times bigger than the biggest memstore of a primary replica. One caveat is that if this is performed, the secondary can observe partial row updates across column families (since column families are flushed independently). The default should be good to not do this operation frequently. You can set this value to a large number to disable this feature if desired, but be warned that it might cause the replication to block forever.
+
+=== Secondary replica failover
+When a secondary region replica first comes online, or fails over, it may have served some edits from it’s memstore. Since the recovery is handled differently for secondary replicas, the secondary has to ensure that it does not go back in time before it starts serving requests after assignment. For doing that, the secondary waits until it observes a full flush cycle (start flush, commit flush) or a “region open event” replicated from the primary. Until this happens, the secondary region replica will reject all read requests by throwing an IOException with message “The region's reads are disabled”. However, the other replicas will probably still be available to read, thus not causing any impact for the rpc with TIMELINE consistency. To facilitate faster recovery, the secondary region will trigger a flush request from the primary when it is opened. The configuration property `hbase.region.replica.wait.for.primary.flush` (enabled by default) can be used to disable this featur
e if needed.
+
+
+
+
=== Configuration properties
-To use highly available reads, you should set the following properties in hbase-site.xml file.
+To use highly available reads, you should set the following properties in `hbase-site.xml` file.
There is no specific configuration to enable or disable region replicas.
-Instead you can change the number of region replicas per table to increase or decrease at the table creation or with alter table.
+Instead you can change the number of region replicas per table to increase or decrease at the table creation or with alter table. The following configuration is for using async wal replication and using meta replicas of 3.
==== Server side properties
@@ -2321,11 +2341,27 @@ Instead you can change the number of region replicas per table to increase or de
[source,xml]
----
<property>
- <name>hbase.regionserver.storefile.refresh.period</name>
- <value>0</value>
- <description>
- The period (in milliseconds) for refreshing the store files for the secondary regions. 0 means this feature is disabled. Secondary regions sees new files (from flushes and compactions) from primary once the secondary region refreshes the list of files in the region. But too frequent refreshes might cause extra Namenode pressure. If the files cannot be refreshed for longer than HFile TTL (hbase.master.hfilecleaner.ttl) the requests are rejected. Configuring HFile TTL to a larger value is also recommended with this setting.
- </description>
+ <name>hbase.regionserver.storefile.refresh.period</name>
+ <value>0</value>
+ <description>
+ The period (in milliseconds) for refreshing the store files for the secondary regions. 0 means this feature is disabled. Secondary regions sees new files (from flushes and compactions) from primary once the secondary region refreshes the list of files in the region (there is no notification mechanism). But too frequent refreshes might cause extra Namenode pressure. If the files cannot be refreshed for longer than HFile TTL (hbase.master.hfilecleaner.ttl) the requests are rejected. Configuring HFile TTL to a larger value is also recommended with this setting.
+ </description>
+</property>
+
+<property>
+ <name>hbase.regionserver.meta.storefile.refresh.period</name>
+ <value>300000</value>
+ <description>
+ The period (in milliseconds) for refreshing the store files for the hbase:meta tables secondary regions. 0 means this feature is disabled. Secondary regions sees new files (from flushes and compactions) from primary once the secondary region refreshes the list of files in the region (there is no notification mechanism). But too frequent refreshes might cause extra Namenode pressure. If the files cannot be refreshed for longer than HFile TTL (hbase.master.hfilecleaner.ttl) the requests are rejected. Configuring HFile TTL to a larger value is also recommended with this setting. This should be a non-zero number if meta replicas are enabled (via hbase.meta.replica.count set to greater than 1).
+ </description>
+</property>
+
+<property>
+ <name>hbase.region.replica.replication.enabled</name>
+ <value>true</value>
+ <description>
+ Whether asynchronous WAL replication to the secondary region replicas is enabled or not. If this is enabled, a replication peer named "region_replica_replication" will be created which will tail the logs and replicate the mutatations to region replicas for tables that have region replication > 1. If this is enabled once, disabling this replication also requires disabling the replication peer using shell or ReplicationAdmin java class. Replication to secondary region replicas works over standard inter-cluster replication. So replication, if disabled explicitly, also has to be enabled by setting "hbase.replication"· to true for this feature to work.
+ </description>
</property>
<property>
<name>hbase.region.replica.replication.memstore.enabled</name>
@@ -2341,6 +2377,38 @@ Instead you can change the number of region replicas per table to increase or de
of row-level consistency, even when the read requests `Consistency.TIMELINE`.
</description>
</property>
+
+<property>
+ <name>hbase.master.hfilecleaner.ttl</name>
+ <value>3600000</value>
+ <description>
+ The period (in milliseconds) to keep store files in the archive folder before deleting them from the file system.</description>
+</property>
+
+<property>
+ <name>hbase.meta.replica.count</name>
+ <value>3</value>
+ <description>
+ Region replication count for the meta regions. Defaults to 1.
+ </description>
+</property>
+
+
+<property>
+ <name>hbase.region.replica.storefile.refresh.memstore.multiplier</name>
+ <value>4</value>
+ <description>
+ The multiplier for a “store file refresh” operation for the secondary region replica. If a region server has memory pressure, the secondary region will refresh it’s store files if the memstore size of the biggest secondary replica is bigger this many times than the memstore size of the biggest primary replica. Set this to a very big value to disable this feature (not recommended).
+ </description>
+</property>
+
+<property>
+ <name>hbase.region.replica.wait.for.primary.flush</name>
+ <value>true</value>
+ <description>
+ Whether to wait for observing a full flush cycle from the primary before start serving data in a secondary. Disabling this might cause the secondary region replicas to go back in time for reads between region movements.
+ </description>
+</property>
----
One thing to keep in mind also is that, region replica placement policy is only enforced by the `StochasticLoadBalancer` which is the default balancer.
@@ -2353,11 +2421,11 @@ Ensure to set the following for all clients (and servers) that will use region r
[source,xml]
----
<property>
- <name>hbase.ipc.client.allowsInterrupt</name>
- <value>true</value>
- <description>
- Whether to enable interruption of RPC threads at the client side. This is required for region replicas with fallback RPC’s to secondary regions.
- </description>
+ <name>hbase.ipc.client.specificThreadForWriting</name>
+ <value>true</value>
+ <description>
+ Whether to enable interruption of RPC threads at the client side. This is required for region replicas with fallback RPC’s to secondary regions.
+ </description>
</property>
<property>
<name>hbase.client.primaryCallTimeout.get</name>
@@ -2380,13 +2448,29 @@ Ensure to set the following for all clients (and servers) that will use region r
The timeout (in microseconds), before secondary fallback RPC’s are submitted for scan requests with Consistency.TIMELINE to the secondary replicas of the regions. Defaults to 1 sec. Setting this lower will increase the number of RPC’s, but will lower the p99 latencies.
</description>
</property>
+<property>
+ <name>hbase.meta.replicas.use</name>
+ <value>true</value>
+ <description>
+ Whether to use meta table replicas or not. Default is false.
+ </description>
+</property>
----
+Note HBase-1.0.x users should use `hbase.ipc.client.allowsInterrupt` rather than `hbase.ipc.client.specificThreadForWriting`.
+
+=== User Interface
+
+In the masters user interface, the region replicas of a table are also shown together with the primary regions.
+You can notice that the replicas of a region will share the same start and end keys and the same region name prefix.
+The only difference would be the appended replica_id (which is encoded as hex), and the region encoded name will be different.
+You can also see the replica ids shown explicitly in the UI.
+
=== Creating a table with region replication
Region replication is a per-table property.
-All tables have REGION_REPLICATION = 1 by default, which means that there is only one replica per region.
-You can set and change the number of replicas per region of a table by supplying the REGION_REPLICATION property in the table descriptor.
+All tables have `REGION_REPLICATION = 1` by default, which means that there is only one replica per region.
+You can set and change the number of replicas per region of a table by supplying the `REGION_REPLICATION` property in the table descriptor.
==== Shell
@@ -2414,21 +2498,8 @@ admin.createTable(htd);
You can also use `setRegionReplication()` and alter table to increase, decrease the region replication for a table.
-=== Region splits and merges
-
-Region splits and merges are not compatible with regions with replicas yet.
-So you have to pre-split the table, and disable the region splits.
-Also you should not execute region merges on tables with region replicas.
-To disable region splits you can use DisabledRegionSplitPolicy as the split policy.
-
-=== User Interface
-
-In the masters user interface, the region replicas of a table are also shown together with the primary regions.
-You can notice that the replicas of a region will share the same start and end keys and the same region name prefix.
-The only difference would be the appended replica_id (which is encoded as hex), and the region encoded name will be different.
-You can also see the replica ids shown explicitly in the UI.
-=== API and Usage
+=== Read API and Usage
==== Shell
@@ -2490,7 +2561,7 @@ scan.setConsistency(Consistency.TIMELINE);
ResultScanner scanner = table.getScanner(scan);
----
-You can inspect whether the results are coming from primary region or not by calling the Result.isStale() method:
+You can inspect whether the results are coming from primary region or not by calling the `Result.isStale()` method:
[source,java]
----
http://git-wip-us.apache.org/repos/asf/hbase/blob/db2ea978/src/main/asciidoc/_chapters/community.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/community.adoc b/src/main/asciidoc/_chapters/community.adoc
index 4b91b0d..573fb49 100644
--- a/src/main/asciidoc/_chapters/community.adoc
+++ b/src/main/asciidoc/_chapters/community.adoc
@@ -62,11 +62,11 @@ Any -1 on a patch by anyone vetos a patch; it cannot be committed until the just
.How to set fix version in JIRA on issue resolve
Here is how link:http://search-hadoop.com/m/azemIi5RCJ1[we agreed] to set versions in JIRA when we resolve an issue.
-If trunk is going to be 0.98.0 then:
+If master is going to be 0.98.0 then:
-* Commit only to trunk: Mark with 0.98
-* Commit to 0.95 and trunk : Mark with 0.98, and 0.95.x
-* Commit to 0.94.x and 0.95, and trunk: Mark with 0.98, 0.95.x, and 0.94.x
+* Commit only to master: Mark with 0.98
+* Commit to 0.95 and master: Mark with 0.98, and 0.95.x
+* Commit to 0.94.x and 0.95, and master: Mark with 0.98, 0.95.x, and 0.94.x
* Commit to 89-fb: Mark with 89-fb.
* Commit site fixes: no version
@@ -103,7 +103,7 @@ Owners do not need to be committers.
[[hbase.commit.msg.format]]
== Commit Message format
-We link:http://search-hadoop.com/m/Gwxwl10cFHa1[agreed] to the following SVN commit message format:
+We link:http://search-hadoop.com/m/Gwxwl10cFHa1[agreed] to the following Git commit message format:
[source]
----
HBASE-xxxxx <title>. (<contributor>)
http://git-wip-us.apache.org/repos/asf/hbase/blob/db2ea978/src/main/asciidoc/_chapters/configuration.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/configuration.adoc b/src/main/asciidoc/_chapters/configuration.adoc
index 01f2eb7..35ce042 100644
--- a/src/main/asciidoc/_chapters/configuration.adoc
+++ b/src/main/asciidoc/_chapters/configuration.adoc
@@ -98,6 +98,11 @@ This section lists required services and some required system configuration.
|JDK 7
|JDK 8
+|1.2
+|link:http://search-hadoop.com/m/DHED4Zlz0R1[Not Supported]
+|yes
+|yes
+
|1.1
|link:http://search-hadoop.com/m/DHED4Zlz0R1[Not Supported]
|yes
@@ -116,11 +121,6 @@ deprecated `remove()` method of the `PoolMap` class and is under consideration.
link:https://issues.apache.org/jira/browse/HBASE-7608[HBASE-7608] for more information about JDK 8
support.
-|0.96
-|yes
-|yes
-|N/A
-
|0.94
|yes
|yes
@@ -210,22 +210,20 @@ Use the following legend to interpret this table:
* "X" = not supported
* "NT" = Not tested
-[cols="1,1,1,1,1,1,1", options="header"]
+[cols="1,1,1,1,1,1", options="header"]
|===
-| | HBase-0.92.x | HBase-0.94.x | HBase-0.96.x | HBase-0.98.x (Support for Hadoop 1.1+ is deprecated.) | HBase-1.0.x (Hadoop 1.x is NOT supported) | HBase-1.1.x
-|Hadoop-0.20.205 | S | X | X | X | X | X
-|Hadoop-0.22.x | S | X | X | X | X | X
-|Hadoop-1.0.x |X | X | X | X | X | X
-|Hadoop-1.1.x | NT | S | S | NT | X | X
-|Hadoop-0.23.x | X | S | NT | X | X | X
-|Hadoop-2.0.x-alpha | X | NT | X | X | X | X
-|Hadoop-2.1.0-beta | X | NT | S | X | X | X
-|Hadoop-2.2.0 | X | NT | S | S | NT | NT
-|Hadoop-2.3.x | X | NT | S | S | NT | NT
-|Hadoop-2.4.x | X | NT | S | S | S | S
-|Hadoop-2.5.x | X | NT | S | S | S | S
-|Hadoop-2.6.x | X | NT | NT | NT | S | S
-|Hadoop-2.7.x | X | NT | NT | NT | NT | NT
+| | HBase-0.94.x | HBase-0.98.x (Support for Hadoop 1.1+ is deprecated.) | HBase-1.0.x (Hadoop 1.x is NOT supported) | HBase-1.1.x | HBase-1.2.x
+|Hadoop-1.0.x | X | X | X | X | X
+|Hadoop-1.1.x | S | NT | X | X | X
+|Hadoop-0.23.x | S | X | X | X | X
+|Hadoop-2.0.x-alpha | NT | X | X | X | X
+|Hadoop-2.1.0-beta | NT | X | X | X | X
+|Hadoop-2.2.0 | NT | S | NT | NT | NT
+|Hadoop-2.3.x | NT | S | NT | NT | NT
+|Hadoop-2.4.x | NT | S | S | S | S
+|Hadoop-2.5.x | NT | S | S | S | S
+|Hadoop-2.6.x | NT | NT | S | S | S
+|Hadoop-2.7.x | NT | NT | NT | NT | NT
|===
.Replace the Hadoop Bundled With HBase!
http://git-wip-us.apache.org/repos/asf/hbase/blob/db2ea978/src/main/asciidoc/_chapters/developer.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/developer.adoc b/src/main/asciidoc/_chapters/developer.adoc
index ce66a90..409efbd 100644
--- a/src/main/asciidoc/_chapters/developer.adoc
+++ b/src/main/asciidoc/_chapters/developer.adoc
@@ -569,8 +569,8 @@ Checkin the _CHANGES.txt_ and any version changes.
. Update the documentation.
+
-Update the documentation under _src/main/docbkx_.
-This usually involves copying the latest from trunk and making version-particular adjustments to suit this release candidate version.
+Update the documentation under _src/main/asciidoc_.
+This usually involves copying the latest from master and making version-particular adjustments to suit this release candidate version.
. Build the source tarball.
+
@@ -643,7 +643,7 @@ This profile invokes the Apache pom referenced by our pom files, and also signs
[source,bourne]
----
-$ mvn deploy -DskipTests -Papache-release
+$ mvn deploy -DskipTests -Papache-release -Prelease
----
+
This command copies all artifacts up to a temporary staging Apache mvn repository in an 'open' state.
@@ -1336,6 +1336,13 @@ NOTE: End-of-life releases are not included in this list.
| 1.0
| Enis Soztutar
+
+| 1.1
+| Nick Dimiduk
+
+| 1.2
+| Sean Busbey
+
|===
[[code.standards]]
@@ -1682,7 +1689,9 @@ $ make_patch.sh [-a] [-p <patch_dir>]
If you decline, the script uses +git diff+ instead.
The patch is saved in a configurable directory and is ready to be attached to your JIRA.
-* .Patching WorkflowAlways patch against the master branch first, even if you want to patch in another branch.
+.Patching Workflow
+
+* Always patch against the master branch first, even if you want to patch in another branch.
HBase committers always apply patches first to the master branch, and backport if necessary.
* Submit one single patch for a fix.
If necessary, squash local commits to merge local commits into a single one first.
@@ -1718,17 +1727,20 @@ Please understand that not every patch may get committed, and that feedback will
However, at times it is easier to refer to different version of a patch if you add `-vX`, where the [replaceable]_X_ is the version (starting with 2).
* If you need to submit your patch against multiple branches, rather than just master, name each version of the patch with the branch it is for, following the naming conventions in <<submitting.patches.create,submitting.patches.create>>.
-.Methods to Create PatchesEclipse::
+.Methods to Create Patches
+Eclipse::
Select the menu item.
Git::
- `git format-patch` is preferred because it preserves commit messages.
+ `git format-patch` is preferred:
+ - It preserves the committer and commit message.
+ - It handles binary files by default, whereas `git diff` ignores them unless
+ you use the `--binary` option.
Use `git rebase -i` first, to combine (squash) smaller commits into a single larger one.
Subversion::
-
-Make sure you review <<eclipse.code.formatting,eclipse.code.formatting>> and <<common.patch.feedback,common.patch.feedback>> for code style.
-If your patch was generated incorrectly or your code does not adhere to the code formatting guidelines, you may be asked to redo some work.
+ Make sure you review <<eclipse.code.formatting,eclipse.code.formatting>> and <<common.patch.feedback,common.patch.feedback>> for code style.
+ If your patch was generated incorrectly or your code does not adhere to the code formatting guidelines, you may be asked to redo some work.
[[submitting.patches.tests]]
==== Unit Tests
@@ -1839,13 +1851,13 @@ The instructions and preferences around the way to create patches have changed,
This is the indication that the patch was not created with `--no-prefix`.
+
----
-diff --git a/src/main/docbkx/developer.xml b/src/main/docbkx/developer.xml
+diff --git a/src/main/asciidoc/_chapters/developer.adoc b/src/main/asciidoc/_chapters/developer.adoc
----
* If the first line of the patch looks similar to the following (without the `a` and `b`), the patch was created with +git diff --no-prefix+ and you need to add `-p0` to the +git apply+ command below.
+
----
-diff --git src/main/docbkx/developer.xml src/main/docbkx/developer.xml
+diff --git src/main/asciidoc/_chapters/developer.adoc src/main/asciidoc/_chapters/developer.adoc
----
+
@@ -1923,7 +1935,7 @@ If the contributor used +git format-patch+ to generate the patch, their commit m
[[committer.amending.author]]
====== Add Amending-Author when a conflict cherrypick backporting
-We've established the practice of committing to trunk and then cherry picking back to branches whenever possible.
+We've established the practice of committing to master and then cherry picking back to branches whenever possible.
When there is a minor conflict we can fix it up and just proceed with the commit.
The resulting commit retains the original author.
When the amending author is different from the original committer, add notice of this at the end of the commit message as: `Amending-Author: Author
@@ -1944,7 +1956,7 @@ A committer should.
In the thread link:http://search-hadoop.com/m/DHED4EiwOz[HBase, mail # dev - ANNOUNCEMENT: Git Migration In Progress (WAS =>
Re: Git Migration)], it was agreed on the following patch flow
-. Develop and commit the patch against trunk/master first.
+. Develop and commit the patch against master first.
. Try to cherry-pick the patch when backporting if possible.
. If this does not work, manually commit the patch to the branch.
http://git-wip-us.apache.org/repos/asf/hbase/blob/db2ea978/src/main/asciidoc/_chapters/getting_started.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/getting_started.adoc b/src/main/asciidoc/_chapters/getting_started.adoc
index 7839bad..4134831 100644
--- a/src/main/asciidoc/_chapters/getting_started.adoc
+++ b/src/main/asciidoc/_chapters/getting_started.adoc
@@ -80,15 +80,16 @@ See <<java,Java>> for information about supported JDK versions.
This will take you to a mirror of _HBase
Releases_.
Click on the folder named _stable_ and then download the binary file that ends in _.tar.gz_ to your local filesystem.
- Be sure to choose the version that corresponds with the version of Hadoop you are likely to use later.
- In most cases, you should choose the file for Hadoop 2, which will be called something like _hbase-0.98.3-hadoop2-bin.tar.gz_.
+ Prior to 1.x version, be sure to choose the version that corresponds with the version of Hadoop you are
+ likely to use later (in most cases, you should choose the file for Hadoop 2, which will be called
+ something like _hbase-0.98.13-hadoop2-bin.tar.gz_).
Do not download the file ending in _src.tar.gz_ for now.
. Extract the downloaded file, and change to the newly-created directory.
+
----
-$ tar xzvf hbase-<?eval ${project.version}?>-hadoop2-bin.tar.gz
-$ cd hbase-<?eval ${project.version}?>-hadoop2/
+$ tar xzvf hbase-<?eval ${project.version}?>-bin.tar.gz
+$ cd hbase-<?eval ${project.version}?>/
----
. For HBase 0.98.5 and later, you are required to set the `JAVA_HOME` environment variable before starting HBase.
http://git-wip-us.apache.org/repos/asf/hbase/blob/db2ea978/src/main/asciidoc/_chapters/hbase_mob.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/hbase_mob.adoc b/src/main/asciidoc/_chapters/hbase_mob.adoc
new file mode 100644
index 0000000..e843d0c
--- /dev/null
+++ b/src/main/asciidoc/_chapters/hbase_mob.adoc
@@ -0,0 +1,236 @@
+////
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+////
+
+[[hbase_mob]]
+== Storing Medium-sized Objects (MOB)
+:doctype: book
+:numbered:
+:toc: left
+:icons: font
+:experimental:
+:toc: left
+:source-language: java
+
+Data comes in many sizes, and saving all of your data in HBase, including binary
+data such as images and documents, is ideal. While HBase can technically handle
+binary objects with cells that are larger than 100 KB in size, HBase's normal
+read and write paths are optimized for values smaller than 100KB in size. When
+HBase deals with large numbers of objects over this threshold, referred to here
+as medium objects, or MOBs, performance is degraded due to write amplification
+caused by splits and compactions. When using MOBs, ideally your objects will be between
+100KB and 10MB. HBase ***FIX_VERSION_NUMBER*** adds support
+for better managing large numbers of MOBs while maintaining performance,
+consistency, and low operational overhead. MOB support is provided by the work
+done in link:https://issues.apache.org/jira/browse/HBASE-11339[HBASE-11339]. To
+take advantage of MOB, you need to use <<hfilev3,HFile version 3>>. Optionally,
+configure the MOB file reader's cache settings for each RegionServer (see
+<<mob.cache.configure>>), then configure specific columns to hold MOB data.
+Client code does not need to change to take advantage of HBase MOB support. The
+feature is transparent to the client.
+
+=== Configuring Columns for MOB
+
+You can configure columns to support MOB during table creation or alteration,
+either in HBase Shell or via the Java API. The two relevant properties are the
+boolean `IS_MOB` and the `MOB_THRESHOLD`, which is the number of bytes at which
+an object is considered to be a MOB. Only `IS_MOB` is required. If you do not
+specify the `MOB_THRESHOLD`, the default threshold value of 100 KB is used.
+
+.Configure a Column for MOB Using HBase Shell
+====
+----
+hbase> create 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 102400}
+hbase> alter 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 102400}
+----
+====
+
+.Configure a Column for MOB Using the Java API
+====
+[source,java]
+----
+...
+HColumnDescriptor hcd = new HColumnDescriptor(“f”);
+hcd.setMobEnabled(true);
+...
+hcd.setMobThreshold(102400L);
+...
+----
+====
+
+
+=== Testing MOB
+
+The utility `org.apache.hadoop.hbase.IntegrationTestIngestMOB` is provided to assist with testing
+the MOB feature. The utility is run as follows:
+[source,bash]
+----
+$ sudo -u hbase hbase org.apache.hadoop.hbase.IntegrationTestIngestMOB \
+ -threshold 102400 \
+ -minMobDataSize 512 \
+ -maxMobDataSize 5120
+----
+
+* `*threshold*` is the threshold at which cells are considered to be MOBs.
+ The default is 1 kB, expressed in bytes.
+* `*minMobDataSize*` is the minimum value for the size of MOB data.
+ The default is 512 B, expressed in bytes.
+* `*maxMobDataSize*` is the maximum value for the size of MOB data.
+ The default is 5 kB, expressed in bytes.
+
+
+[[mob.cache.configure]]
+=== Configuring the MOB Cache
+
+
+Because there can be a large number of MOB files at any time, as compared to the number of HFiles,
+MOB files are not always kept open. The MOB file reader cache is a LRU cache which keeps the most
+recently used MOB files open. To configure the MOB file reader's cache on each RegionServer, add
+the following properties to the RegionServer's `hbase-site.xml`, customize the configuration to
+suit your environment, and restart or rolling restart the RegionServer.
+
+.Example MOB Cache Configuration
+====
+[source,xml]
+----
+<property>
+ <name>hbase.mob.file.cache.size</name>
+ <value>1000</value>
+ <description>
+ Number of opened file handlers to cache.
+ A larger value will benefit reads by provinding more file handlers per mob
+ file cache and would reduce frequent file opening and closing.
+ However, if this is set too high, this could lead to a "too many opened file handers"
+ The default value is 1000.
+ </description>
+</property>
+<property>
+ <name>hbase.mob.cache.evict.period</name>
+ <value>3600</value>
+ <description>
+ The amount of time in seconds after which an unused file is evicted from the
+ MOB cache. The default value is 3600 seconds.
+ </description>
+</property>
+<property>
+ <name>hbase.mob.cache.evict.remain.ratio</name>
+ <value>0.5f</value>
+ <description>
+ A multiplier (between 0.0 and 1.0), which determines how many files remain cached
+ after the threshold of files that remains cached after a cache eviction occurs
+ which is triggered by reaching the `hbase.mob.file.cache.size` threshold.
+ The default value is 0.5f, which means that half the files (the least-recently-used
+ ones) are evicted.
+ </description>
+</property>
+----
+====
+
+=== MOB Optimization Tasks
+
+==== Manually Compacting MOB Files
+
+To manually compact MOB files, rather than waiting for the
+<<mob.cache.configure,configuration>> to trigger compaction, use the
+`compact_mob` or `major_compact_mob` HBase shell commands. These commands
+require the first argument to be the table name, and take an optional column
+family as the second argument. If the column family is omitted, all MOB-enabled
+column families are compacted.
+
+----
+hbase> compact_mob 't1', 'c1'
+hbase> compact_mob 't1'
+hbase> major_compact_mob 't1', 'c1'
+hbase> major_compact_mob 't1'
+----
+
+These commands are also available via `Admin.compactMob` and
+`Admin.majorCompactMob` methods.
+
+==== MOB Sweeper
+
+HBase MOB a MapReduce job called the Sweeper tool for
+optimization. The Sweeper tool oalesces small MOB files or MOB files with many
+deletions or updates. The Sweeper tool is not required if you use native MOB compaction, which
+does not rely on MapReduce.
+
+To configure the Sweeper tool, set the following options:
+
+[source,xml]
+----
+<property>
+ <name>hbase.mob.sweep.tool.compaction.ratio</name>
+ <value>0.5f</value>
+ <description>
+ If there are too many cells deleted in a mob file, it's regarded
+ as an invalid file and needs to be merged.
+ If existingCellsSize/mobFileSize is less than ratio, it's regarded
+ as an invalid file. The default value is 0.5f.
+ </description>
+</property>
+<property>
+ <name>hbase.mob.sweep.tool.compaction.mergeable.size</name>
+ <value>134217728</value>
+ <description>
+ If the size of a mob file is less than this value, it's regarded as a small
+ file and needs to be merged. The default value is 128MB.
+ </description>
+</property>
+<property>
+ <name>hbase.mob.sweep.tool.compaction.memstore.flush.size</name>
+ <value>134217728</value>
+ <description>
+ The flush size for the memstore used by sweep job. Each sweep reducer owns such a memstore.
+ The default value is 128MB.
+ </description>
+</property>
+<property>
+ <name>hbase.master.mob.ttl.cleaner.period</name>
+ <value>86400</value>
+ <description>
+ The period that ExpiredMobFileCleanerChore runs. The unit is second.
+ The default value is one day.
+ </description>
+</property>
+----
+
+Next, add the HBase install directory, _`$HBASE_HOME`/*_, and HBase library directory to
+_yarn-site.xml_ Adjust this example to suit your environment.
+[source,xml]
+----
+<property>
+ <description>Classpath for typical applications.</description>
+ <name>yarn.application.classpath</name>
+ <value>
+ $HADOOP_CONF_DIR,
+ $HADOOP_COMMON_HOME/*,$HADOOP_COMMON_HOME/lib/*,
+ $HADOOP_HDFS_HOME/*,$HADOOP_HDFS_HOME/lib/*,
+ $HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HOME/lib/*,
+ $HADOOP_YARN_HOME/*,$HADOOP_YARN_HOME/lib/*,
+ $HBASE_HOME/*, $HBASE_HOME/lib/*
+ </value>
+</property>
+----
+
+Finally, run the `sweeper` tool for each column which is configured for MOB.
+[source,bash]
+----
+$ org.apache.hadoop.hbase.mob.compactions.Sweeper _tableName_ _familyName_
+----
http://git-wip-us.apache.org/repos/asf/hbase/blob/db2ea978/src/main/asciidoc/_chapters/ops_mgt.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/ops_mgt.adoc b/src/main/asciidoc/_chapters/ops_mgt.adoc
index 3c4a73b..514003d 100644
--- a/src/main/asciidoc/_chapters/ops_mgt.adoc
+++ b/src/main/asciidoc/_chapters/ops_mgt.adoc
@@ -1622,7 +1622,7 @@ handling multiple workloads:
HBASE-11598 introduces quotas, which allow you to throttle requests based on
the following limits:
-. <<request-quotas,The number or size of requests in a given timeframe>>
+. <<request-quotas,The number or size of requests(read, write, or read+write) in a given timeframe>>
. <<namespace-quotas,The number of tables allowed in a namespace>>
These limits can be enforced for a specified user, table, or namespace.
@@ -1633,6 +1633,7 @@ Quotas are disabled by default. To enable the feature, set the `hbase.quota.enab
property to `true` in _hbase-site.xml_ file for all cluster nodes.
.General Quota Syntax
+. THROTTLE_TYPE can be expressed as READ, WRITE, or the default type(read + write).
. Timeframes can be expressed in the following units: `sec`, `min`, `hour`, `day`
. Request sizes can be expressed in the following units: `B` (bytes), `K` (kilobytes),
`M` (megabytes), `G` (gigabytes), `T` (terabytes), `P` (petabytes)
@@ -1652,17 +1653,26 @@ in `hbase-site.xml`. This property is expressed in milliseconds and defaults to
# Limit user u1 to 10 requests per second
hbase> set_quota TYPE => THROTTLE, USER => 'u1', LIMIT => '10req/sec'
+# Limit user u1 to 10 read requests per second
+hbase> set_quota TYPE => THROTTLE, THROTTLE_TYPE => READ, USER => 'u1', LIMIT => '10req/sec'
+
# Limit user u1 to 10 M per day everywhere
hbase> set_quota TYPE => THROTTLE, USER => 'u1', LIMIT => '10M/day'
+# Limit user u1 to 10 M write size per sec
+hbase> set_quota TYPE => THROTTLE, THROTTLE_TYPE => WRITE, USER => 'u1', LIMIT => '10M/sec'
+
# Limit user u1 to 5k per minute on table t2
hbase> set_quota TYPE => THROTTLE, USER => 'u1', TABLE => 't2', LIMIT => '5K/min'
+# Limit user u1 to 10 read requests per sec on table t2
+hbase> set_quota TYPE => THROTTLE, THROTTLE_TYPE => READ, USER => 'u1', TABLE => 't2', LIMIT => '10req/sec'
+
# Remove an existing limit from user u1 on namespace ns2
hbase> set_quota TYPE => THROTTLE, USER => 'u1', NAMESPACE => 'ns2', LIMIT => NONE
# Limit all users to 10 requests per hour on namespace ns1
-hbase> set_quota TYPE => THROTTLE, NAMESPACE => 'ns1', LIMIT => '10req/shour'
+hbase> set_quota TYPE => THROTTLE, NAMESPACE => 'ns1', LIMIT => '10req/hour'
# Limit all users to 10 T per hour on table t1
hbase> set_quota TYPE => THROTTLE, TABLE => 't1', LIMIT => '10T/hour'
http://git-wip-us.apache.org/repos/asf/hbase/blob/db2ea978/src/main/asciidoc/_chapters/performance.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/performance.adoc b/src/main/asciidoc/_chapters/performance.adoc
index 2155d52..f213175 100644
--- a/src/main/asciidoc/_chapters/performance.adoc
+++ b/src/main/asciidoc/_chapters/performance.adoc
@@ -546,7 +546,7 @@ To disable the WAL, see <<wal.disable>>.
=== HBase Client: Group Puts by RegionServer
In addition to using the writeBuffer, grouping `Put`s by RegionServer can reduce the number of client RPC calls per writeBuffer flush.
-There is a utility `HTableUtil` currently on TRUNK that does this, but you can either copy that or implement your own version for those still on 0.90.x or earlier.
+There is a utility `HTableUtil` currently on MASTER that does this, but you can either copy that or implement your own version for those still on 0.90.x or earlier.
[[perf.hbase.write.mr.reducer]]
=== MapReduce: Skip The Reducer
http://git-wip-us.apache.org/repos/asf/hbase/blob/db2ea978/src/main/asciidoc/_chapters/rpc.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/rpc.adoc b/src/main/asciidoc/_chapters/rpc.adoc
index 43e7156..c319f39 100644
--- a/src/main/asciidoc/_chapters/rpc.adoc
+++ b/src/main/asciidoc/_chapters/rpc.adoc
@@ -71,7 +71,8 @@ Optionally, Cells(KeyValues) can be passed outside of protobufs in follow-behind
-For more detail on the protobufs involved, see the link:http://svn.apache.org/viewvc/hbase/trunk/hbase-protocol/src/main/protobuf/RPC.proto?view=markup[RPC.proto] file in trunk.
+For more detail on the protobufs involved, see the
+link:https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=blob;f=hbase-protocol/src/main/protobuf/RPC.proto;hb=HEAD[RPC.proto] file in master.
==== Connection Setup
http://git-wip-us.apache.org/repos/asf/hbase/blob/db2ea978/src/main/asciidoc/_chapters/schema_design.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/schema_design.adoc b/src/main/asciidoc/_chapters/schema_design.adoc
index 9319c65..a212a5c 100644
--- a/src/main/asciidoc/_chapters/schema_design.adoc
+++ b/src/main/asciidoc/_chapters/schema_design.adoc
@@ -27,7 +27,19 @@
:icons: font
:experimental:
-A good general introduction on the strength and weaknesses modelling on the various non-rdbms datastores is Ian Varley's Master thesis, link:http://ianvarley.com/UT/MR/Varley_MastersReport_Full_2009-08-07.pdf[No Relation: The Mixed Blessings of Non-Relational Databases]. Also, read <<keyvalue,keyvalue>> for how HBase stores data internally, and the section on <<schema.casestudies,schema.casestudies>>.
+A good introduction on the strength and weaknesses modelling on the various non-rdbms datastores is
+to be found in Ian Varley's Master thesis,
+link:http://ianvarley.com/UT/MR/Varley_MastersReport_Full_2009-08-07.pdf[No Relation: The Mixed Blessings of Non-Relational Databases].
+It is a little dated now but a good background read if you have a moment on how HBase schema modeling
+differs from how it is done in an RDBMS. Also,
+read <<keyvalue,keyvalue>> for how HBase stores data internally, and the section on <<schema.casestudies,schema.casestudies>>.
+
+The documentation on the Cloud Bigtable website, link:https://cloud.google.com/bigtable/docs/schema-design[Designing Your Schema],
+is pertinent and nicely done and lessons learned there equally apply here in HBase land; just divide
+any quoted values by ~10 to get what works for HBase: e.g. where it says individual values can be ~10MBs in size, HBase can do similar -- perhaps best
+to go smaller if you can -- and where it says a maximum of 100 column families in Cloud Bigtable, think ~10 when
+modeling on HBase.
+
[[schema.creation]]
== Schema Creation
@@ -700,7 +712,7 @@ See https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Scan.html#se
====
[[schema.casestudies.log_timeseries.varkeys]]
-==== Variangle Length or Fixed Length Rowkeys?
+==== Variable Length or Fixed Length Rowkeys?
It is critical to remember that rowkeys are stamped on every column in HBase.
If the hostname is `a` and the event type is `e1` then the resulting rowkey would be quite small.
http://git-wip-us.apache.org/repos/asf/hbase/blob/db2ea978/src/main/asciidoc/_chapters/security.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/security.adoc b/src/main/asciidoc/_chapters/security.adoc
index f9a4b8a..40e8e16 100644
--- a/src/main/asciidoc/_chapters/security.adoc
+++ b/src/main/asciidoc/_chapters/security.adoc
@@ -175,6 +175,15 @@ Add the following to the `hbase-site.xml` file for every Thrift gateway:
You may have to put the concrete full hostname.
-->
</property>
+<!-- Add these if you need to configure a different DNS interface from the default -->
+<property>
+ <name>hbase.thrift.dns.interface</name>
+ <value>default</value>
+</property>
+<property>
+ <name>hbase.thrift.dns.nameserver</name>
+ <value>default</value>
+</property>
----
Substitute the appropriate credential and keytab for _$USER_ and _$KEYTAB_ respectively.
@@ -308,6 +317,15 @@ To enable REST gateway Kerberos authentication for client access, add the follow
<name>hbase.rest.authentication.kerberos.keytab</name>
<value>$KEYTAB</value>
</property>
+<!-- Add these if you need to configure a different DNS interface from the default -->
+<property>
+ <name>hbase.rest.dns.interface</name>
+ <value>default</value>
+</property>
+<property>
+ <name>hbase.rest.dns.nameserver</name>
+ <value>default</value>
+</property>
----
Substitute the keytab for HTTP for _$KEYTAB_.
http://git-wip-us.apache.org/repos/asf/hbase/blob/db2ea978/src/main/asciidoc/_chapters/troubleshooting.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/troubleshooting.adoc b/src/main/asciidoc/_chapters/troubleshooting.adoc
index 1776c9e..8ae61b4 100644
--- a/src/main/asciidoc/_chapters/troubleshooting.adoc
+++ b/src/main/asciidoc/_chapters/troubleshooting.adoc
@@ -559,6 +559,14 @@ You can also tail all the logs at the same time, edit files, etc.
For more information on the HBase client, see <<client,client>>.
+=== Missed Scan Results Due To Mismatch Of `hbase.client.scanner.max.result.size` Between Client and Server
+If either the client or server version is lower than 0.98.11/1.0.0 and the server
+has a smaller value for `hbase.client.scanner.max.result.size` than the client, scan
+requests that reach the server's `hbase.client.scanner.max.result.size` are likely
+to miss data. In particular, 0.98.11 defaults `hbase.client.scanner.max.result.size`
+to 2 MB but other versions default to larger values. For this reason, be very careful
+using 0.98.11 servers with any other client version.
+
[[trouble.client.scantimeout]]
=== ScannerTimeoutException or UnknownScannerException
@@ -834,6 +842,31 @@ Two common use-cases for querying HDFS for HBase objects is research the degree
If there are a large number of StoreFiles for each ColumnFamily it could indicate the need for a major compaction.
Additionally, after a major compaction if the resulting StoreFile is "small" it could indicate the need for a reduction of ColumnFamilies for the table.
+=== Unexpected Filesystem Growth
+
+If you see an unexpected spike in filesystem usage by HBase, two possible culprits
+are snapshots and WALs.
+
+Snapshots::
+ When you create a snapshot, HBase retains everything it needs to recreate the table's
+ state at that time of tne snapshot. This includes deleted cells or expired versions.
+ For this reason, your snapshot usage pattern should be well-planned, and you should
+ prune snapshots that you no longer need. Snapshots are stored in `/hbase/.snapshots`,
+ and archives needed to restore snapshots are stored in
+ `/hbase/.archive/<_tablename_>/<_region_>/<_column_family_>/`.
+
+ *Do not* manage snapshots or archives manually via HDFS. HBase provides APIs and
+ HBase Shell commands for managing them. For more information, see <<ops.snapshots>>.
+
+WAL::
+ Write-ahead logs (WALs) are stored in subdirectories of `/hbase/.logs/`, depending
+ on their status. Already-processed WALs are stored in `/hbase/.logs/oldWALs/` and
+ corrupt WALs are stored in `/hbase/.logs/.corrupt/` for examination.
+ If the size of any subdirectory of `/hbase/.logs/` is growing, examine the HBase
+ server logs to find the root cause for why WALs are not being processed correctly.
+
+*Do not* manage WALs manually via HDFS.
+
[[trouble.network]]
== Network
http://git-wip-us.apache.org/repos/asf/hbase/blob/db2ea978/src/main/asciidoc/_chapters/upgrading.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/upgrading.adoc b/src/main/asciidoc/_chapters/upgrading.adoc
index 6b63833..9742654 100644
--- a/src/main/asciidoc/_chapters/upgrading.adoc
+++ b/src/main/asciidoc/_chapters/upgrading.adoc
@@ -204,6 +204,14 @@ See the release notes on the issue link:https://issues.apache.org/jira/browse/HB
.Distributed Log Replay
<<distributed.log.replay>> is off by default in HBase 1.0.0. Enabling it can make a big difference improving HBase MTTR. Enable this feature if you are doing a clean stop/start when you are upgrading. You cannot rolling upgrade to this feature (caveat if you are running on a version of HBase in excess of HBase 0.98.4 -- see link:https://issues.apache.org/jira/browse/HBASE-12577[HBASE-12577 Disable distributed log replay by default] for more).
+.Mismatch Of `hbase.client.scanner.max.result.size` Between Client and Server
+If either the client or server version is lower than 0.98.11/1.0.0 and the server
+has a smaller value for `hbase.client.scanner.max.result.size` than the client, scan
+requests that reach the server's `hbase.client.scanner.max.result.size` are likely
+to miss data. In particular, 0.98.11 defaults `hbase.client.scanner.max.result.size`
+to 2 MB but other versions default to larger values. For this reason, be very careful
+using 0.98.11 servers with any other client version.
+
[[upgrade1.0.rolling.upgrade]]
==== Rolling upgrade from 0.98.x to HBase 1.0.0
.From 0.96.x to 1.0.0
http://git-wip-us.apache.org/repos/asf/hbase/blob/db2ea978/src/main/asciidoc/book.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/book.adoc b/src/main/asciidoc/book.adoc
index b2bd151..d030c38 100644
--- a/src/main/asciidoc/book.adoc
+++ b/src/main/asciidoc/book.adoc
@@ -61,6 +61,7 @@ include::_chapters/schema_design.adoc[]
include::_chapters/mapreduce.adoc[]
include::_chapters/security.adoc[]
include::_chapters/architecture.adoc[]
+include::_chapters/hbase_mob.adoc[]
include::_chapters/hbase_apis.adoc[]
include::_chapters/external_apis.adoc[]
include::_chapters/thrift_filter_language.adoc[]
http://git-wip-us.apache.org/repos/asf/hbase/blob/db2ea978/src/main/site/asciidoc/sponsors.adoc
----------------------------------------------------------------------
diff --git a/src/main/site/asciidoc/sponsors.adoc b/src/main/site/asciidoc/sponsors.adoc
index 443d46b..3ad1e0e 100644
--- a/src/main/site/asciidoc/sponsors.adoc
+++ b/src/main/site/asciidoc/sponsors.adoc
@@ -29,6 +29,7 @@ The below companies have been gracious enough to provide their commerical tool o
* link:http://www.yourkit.com[YourKit] allows us to use their link:http://www.yourkit.com/overview/index.jsp[Java Profiler].
* Some of us use link:http://www.jetbrains.com/idea[IntelliJ IDEA] thanks to link:http://www.jetbrains.com/[JetBrains].
+* Thank you to Boris at link:http://www.vectorportal.com/idea[Vector Portal] for granting us a license on the image on which our logo is based.
== Sponsoring the Apache Software Foundation">
To contribute to the Apache Software Foundation, a good idea in our opinion, see the link:http://www.apache.org/foundation/sponsorship.html[ASF Sponsorship] page.
http://git-wip-us.apache.org/repos/asf/hbase/blob/db2ea978/src/main/site/resources/doap_Hbase.rdf
----------------------------------------------------------------------
diff --git a/src/main/site/resources/doap_Hbase.rdf b/src/main/site/resources/doap_Hbase.rdf
index 08e9bc0..46082a1 100644
--- a/src/main/site/resources/doap_Hbase.rdf
+++ b/src/main/site/resources/doap_Hbase.rdf
@@ -36,16 +36,16 @@
<category rdf:resource="http://projects.apache.org/category/database" />
<release>
<Version>
- <name>Apache hbase 0.92.1</name>
- <created>2012-03-19</created>
- <revision>0.92.1</revision>
+ <name>Apache hbase </name>
+ <created>2015-07-23</created>
+ <revision>2.0.0-SNAPSHOT</revision>
</Version>
</release>
<repository>
- <SVNRepository>
- <location rdf:resource="http://svn.apache.org/repos/asf/hbase"/>
- <browse rdf:resource="http://svn.apache.org/viewvc/hbase"/>
- </SVNRepository>
+ <GitRepository>
+ <location rdf:resource="git://git.apache.org/hbase.git"/>
+ <browse rdf:resource="https://git-wip-us.apache.org/repos/asf?p=hbase.git"/>
+ </GitRepository>
</repository>
<maintainer>
<foaf:Person>
http://git-wip-us.apache.org/repos/asf/hbase/blob/db2ea978/src/main/site/site.xml
----------------------------------------------------------------------
diff --git a/src/main/site/site.xml b/src/main/site/site.xml
index b7debd3..6c7e8d2 100644
--- a/src/main/site/site.xml
+++ b/src/main/site/site.xml
@@ -44,8 +44,8 @@
</bannerLeft>
<bannerRight>
<name>Apache HBase Orca</name>
- <src>images/hbasecon2015.30percent.png</src>
- <href>http://hbasecon.com/</href>
+ <src>images/jumping-orca_rotated_25percent.png</src>
+ <href>http://hbase.apache.org/</href>
</bannerRight>
<body>
<menu name="Apache HBase Project">
http://git-wip-us.apache.org/repos/asf/hbase/blob/db2ea978/src/main/site/xdoc/index.xml
----------------------------------------------------------------------
diff --git a/src/main/site/xdoc/index.xml b/src/main/site/xdoc/index.xml
index ca64d75..86a9ba9 100644
--- a/src/main/site/xdoc/index.xml
+++ b/src/main/site/xdoc/index.xml
@@ -76,6 +76,7 @@ Apache HBase is an open-source, distributed, versioned, non-relational database
</p>
</section>
<section name="News">
+ <p>June 25th, 2015 <a href="http://www.zusaar.com/event/14057003">HBase Summer Meetup 2015</a> in Tokyo</p>
<p>May 7th, 2015 <a href="http://hbasecon.com/">HBaseCon2015</a> in San Francisco</p>
<p>February 17th, 2015 <a href="http://www.meetup.com/hbaseusergroup/events/219260093/">HBase meetup around Strata+Hadoop World</a> in San Jose</p>
<p>January 15th, 2015 <a href="http://www.meetup.com/hbaseusergroup/events/218744798/">HBase meetup @ AppDynamics</a> in San Francisco</p>