You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hbase.apache.org by nd...@apache.org on 2016/08/28 17:06:57 UTC
[1/2] hbase git commit: updating docs from master
Repository: hbase
Updated Branches:
refs/heads/branch-1.1 dca60dbc5 -> 2039d9678
updating docs from master
Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/45f9865e
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/45f9865e
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/45f9865e
Branch: refs/heads/branch-1.1
Commit: 45f9865ef916f064f2b5f492190cc07cf6854d96
Parents: dca60db
Author: Nick Dimiduk <nd...@apache.org>
Authored: Sun Aug 28 10:06:04 2016 -0700
Committer: Nick Dimiduk <nd...@apache.org>
Committed: Sun Aug 28 10:06:04 2016 -0700
----------------------------------------------------------------------
src/main/asciidoc/_chapters/architecture.adoc | 3 +--
src/main/asciidoc/_chapters/configuration.adoc | 21 +-------------------
src/main/asciidoc/_chapters/developer.adoc | 8 +++++++-
.../asciidoc/_chapters/getting_started.adoc | 5 +++--
src/main/asciidoc/_chapters/mapreduce.adoc | 2 +-
src/main/asciidoc/_chapters/ops_mgt.adoc | 2 +-
src/main/asciidoc/_chapters/other_info.adoc | 4 ++--
src/main/asciidoc/_chapters/security.adoc | 4 ++--
.../asciidoc/_chapters/troubleshooting.adoc | 4 ++--
src/main/asciidoc/_chapters/upgrading.adoc | 7 ++++---
10 files changed, 24 insertions(+), 36 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hbase/blob/45f9865e/src/main/asciidoc/_chapters/architecture.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/architecture.adoc b/src/main/asciidoc/_chapters/architecture.adoc
index 4b88665..cfdd638 100644
--- a/src/main/asciidoc/_chapters/architecture.adoc
+++ b/src/main/asciidoc/_chapters/architecture.adoc
@@ -1412,8 +1412,7 @@ admin.createTable(tableDesc);
[source]
.Configuring the Split Policy On a Table Using HBase Shell
----
-hbase> create 'test', {METHOD => 'table_att', CONFIG => {'SPLIT_POLICY' => 'org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy'}},
-{NAME => 'cf1'}
+hbase> create 'test', {METADATA => {'SPLIT_POLICY' => 'org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy'}},{NAME => 'cf1'}
----
The default split policy can be overwritten using a custom
http://git-wip-us.apache.org/repos/asf/hbase/blob/45f9865e/src/main/asciidoc/_chapters/configuration.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/configuration.adoc b/src/main/asciidoc/_chapters/configuration.adoc
index dd253d7..8dc3e8a 100644
--- a/src/main/asciidoc/_chapters/configuration.adoc
+++ b/src/main/asciidoc/_chapters/configuration.adoc
@@ -337,26 +337,7 @@ Do not move to Apache HBase 0.96.x if you cannot upgrade your Hadoop. See link:h
[[hadoop.older.versions]]
==== Hadoop versions 0.20.x - 1.x
-HBase will lose data unless it is running on an HDFS that has a durable `sync` implementation.
-DO NOT use Hadoop 0.20.2, Hadoop 0.20.203.0, and Hadoop 0.20.204.0 which DO NOT have this attribute.
-Currently only Hadoop versions 0.20.205.x or any release in excess of this version -- this includes hadoop-1.0.0 -- have a working, durable sync.
-The Cloudera blog post link:http://www.cloudera.com/blog/2012/01/an-update-on-apache-hadoop-1-0/[An
- update on Apache Hadoop 1.0] by Charles Zedlweski has a nice exposition on how all the Hadoop versions relate.
-It's worth checking out if you are having trouble making sense of the Hadoop version morass.
-
-Sync has to be explicitly enabled by setting `dfs.support.append` equal to true on both the client side -- in _hbase-site.xml_ -- and on the serverside in _hdfs-site.xml_ (The sync facility HBase needs is a subset of the append code path).
-
-[source,xml]
-----
-
-<property>
- <name>dfs.support.append</name>
- <value>true</value>
-</property>
-----
-
-You will have to restart your cluster after making this edit.
-Ignore the chicken-little comment you'll find in the _hdfs-default.xml_ in the description for the `dfs.support.append` configuration.
+DO NOT use Hadoop versions older than 2.2.0 for HBase versions greater than 1.0. Check release documentation if you are using an older version of HBase for Hadoop related information.
[[hadoop.security]]
==== Apache HBase on Secure Hadoop
http://git-wip-us.apache.org/repos/asf/hbase/blob/45f9865e/src/main/asciidoc/_chapters/developer.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/developer.adoc b/src/main/asciidoc/_chapters/developer.adoc
index 74ce3df..0a29864 100644
--- a/src/main/asciidoc/_chapters/developer.adoc
+++ b/src/main/asciidoc/_chapters/developer.adoc
@@ -351,6 +351,10 @@ mvn -Dhadoop.profile=22 ...
You may need to change the protobuf definitions that reside in the _hbase-protocol_ module or other modules.
+Previous to hbase-2.0.0, protobuf definition files were sprinkled across all hbase modules but now all
+to do with protobuf must reside in the hbase-protocol module; we are trying to contain our protobuf
+use so we can freely change versions without upsetting any downstream project use of protobuf.
+
The protobuf files are located in _hbase-protocol/src/main/protobuf_.
For the change to be effective, you will need to regenerate the classes.
You can use maven profile `compile-protobuf` to do this.
@@ -1587,7 +1591,9 @@ value="HE_EQUALS_USE_HASHCODE",
justification="I know what I'm doing")
----
-It is important to use the Apache-licensed version of the annotations.
+It is important to use the Apache-licensed version of the annotations. That generally means using
+annotations in the `edu.umd.cs.findbugs.annotations` package so that we can rely on the cleanroom
+reimplementation rather than annotations in the `javax.annotations` package.
[[common.patch.feedback.javadoc.defaults]]
===== Javadoc - Useless Defaults
http://git-wip-us.apache.org/repos/asf/hbase/blob/45f9865e/src/main/asciidoc/_chapters/getting_started.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/getting_started.adoc b/src/main/asciidoc/_chapters/getting_started.adoc
index 276a908..26af568 100644
--- a/src/main/asciidoc/_chapters/getting_started.adoc
+++ b/src/main/asciidoc/_chapters/getting_started.adoc
@@ -87,10 +87,11 @@ See <<java,Java>> for information about supported JDK versions.
Do not download the file ending in _src.tar.gz_ for now.
. Extract the downloaded file, and change to the newly-created directory.
+
+[source,subs="attributes"]
----
-$ tar xzvf hbase-<?eval ${project.version}?>-bin.tar.gz
-$ cd hbase-<?eval ${project.version}?>/
+$ tar xzvf hbase-{Version}-bin.tar.gz
+$ cd hbase-{Version}/
----
. For HBase 0.98.5 and later, you are required to set the `JAVA_HOME` environment variable before starting HBase.
http://git-wip-us.apache.org/repos/asf/hbase/blob/45f9865e/src/main/asciidoc/_chapters/mapreduce.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/mapreduce.adoc b/src/main/asciidoc/_chapters/mapreduce.adoc
index 75718fd..dfa843a 100644
--- a/src/main/asciidoc/_chapters/mapreduce.adoc
+++ b/src/main/asciidoc/_chapters/mapreduce.adoc
@@ -73,7 +73,7 @@ This example assumes you use a BASH-compatible shell.
$ HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase classpath` ${HADOOP_HOME}/bin/hadoop jar ${HBASE_HOME}/lib/hbase-server-VERSION.jar rowcounter usertable
----
-When the command runs, internally, the HBase JAR finds the dependencies it needs for ZooKeeper, Guava, and its other dependencies on the passed `HADOOP_CLASSPATH` and adds the JARs to the MapReduce job configuration.
+When the command runs, internally, the HBase JAR finds the dependencies it needs and adds them to the MapReduce job configuration.
See the source at `TableMapReduceUtil#addDependencyJars(org.apache.hadoop.mapreduce.Job)` for how this is done.
The command `hbase mapredcp` can also help you dump the CLASSPATH entries required by MapReduce, which are the same jars `TableMapReduceUtil#addDependencyJars` would add.
http://git-wip-us.apache.org/repos/asf/hbase/blob/45f9865e/src/main/asciidoc/_chapters/ops_mgt.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/ops_mgt.adoc b/src/main/asciidoc/_chapters/ops_mgt.adoc
index 6e84237..0d184a7 100644
--- a/src/main/asciidoc/_chapters/ops_mgt.adoc
+++ b/src/main/asciidoc/_chapters/ops_mgt.adoc
@@ -437,7 +437,7 @@ Caching for the input Scan is configured via `hbase.client.scanner.caching`
By default, CopyTable utility only copies the latest version of row cells unless `--versions=n` is explicitly specified in the command.
====
-See Jonathan Hsieh's link:http://www.cloudera.com/blog/2012/06/online-hbase-backups-with-copytable-2/[Online
+See Jonathan Hsieh's link:https://blog.cloudera.com/blog/2012/06/online-hbase-backups-with-copytable-2/[Online
HBase Backups with CopyTable] blog post for more on `CopyTable`.
[[export]]
http://git-wip-us.apache.org/repos/asf/hbase/blob/45f9865e/src/main/asciidoc/_chapters/other_info.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/other_info.adoc b/src/main/asciidoc/_chapters/other_info.adoc
index 6143876..8bcbe0f 100644
--- a/src/main/asciidoc/_chapters/other_info.adoc
+++ b/src/main/asciidoc/_chapters/other_info.adoc
@@ -59,9 +59,9 @@ link:http://ianvarley.com/UT/MR/Varley_MastersReport_Full_2009-08-07.pdf[No Rela
[[other.info.sites]]
=== HBase Sites
-link:http://www.cloudera.com/blog/category/hbase/[Cloudera's HBase Blog] has a lot of links to useful HBase information.
+link:https://blog.cloudera.com/blog/category/hbase/[Cloudera's HBase Blog] has a lot of links to useful HBase information.
-* link:http://www.cloudera.com/blog/2010/04/cap-confusion-problems-with-partition-tolerance/[CAP Confusion] is a relevant entry for background information on distributed storage systems.
+* link:https://blog.cloudera.com/blog/2010/04/cap-confusion-problems-with-partition-tolerance/[CAP Confusion] is a relevant entry for background information on distributed storage systems.
link:http://wiki.apache.org/hadoop/HBase/HBasePresentations[HBase Wiki] has a page with a number of presentations.
http://git-wip-us.apache.org/repos/asf/hbase/blob/45f9865e/src/main/asciidoc/_chapters/security.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/security.adoc b/src/main/asciidoc/_chapters/security.adoc
index 85e503c..0ed9ba2 100644
--- a/src/main/asciidoc/_chapters/security.adoc
+++ b/src/main/asciidoc/_chapters/security.adoc
@@ -126,7 +126,7 @@ A number of properties exist to configure SPNEGO authentication for the web serv
== Secure Client Access to Apache HBase
Newer releases of Apache HBase (>= 0.92) support optional SASL authentication of clients.
-See also Matteo Bertozzi's article on link:http://www.cloudera.com/blog/2012/09/understanding-user-authentication-and-authorization-in-apache-hbase/[Understanding User Authentication and Authorization in Apache HBase].
+See also Matteo Bertozzi's article on link:https://blog.cloudera.com/blog/2012/09/understanding-user-authentication-and-authorization-in-apache-hbase/[Understanding User Authentication and Authorization in Apache HBase].
This describes how to set up Apache HBase and clients for connection to secure HBase resources.
@@ -451,7 +451,7 @@ Substitute the keytab for HTTP for _$KEYTAB_.
== Simple User Access to Apache HBase
Newer releases of Apache HBase (>= 0.92) support optional SASL authentication of clients.
-See also Matteo Bertozzi's article on link:http://www.cloudera.com/blog/2012/09/understanding-user-authentication-and-authorization-in-apache-hbase/[Understanding User Authentication and Authorization in Apache HBase].
+See also Matteo Bertozzi's article on link:https://blog.cloudera.com/blog/2012/09/understanding-user-authentication-and-authorization-in-apache-hbase/[Understanding User Authentication and Authorization in Apache HBase].
This describes how to set up Apache HBase and clients for simple user access to HBase resources.
http://git-wip-us.apache.org/repos/asf/hbase/blob/45f9865e/src/main/asciidoc/_chapters/troubleshooting.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/troubleshooting.adoc b/src/main/asciidoc/_chapters/troubleshooting.adoc
index fc9aadb..ce47423 100644
--- a/src/main/asciidoc/_chapters/troubleshooting.adoc
+++ b/src/main/asciidoc/_chapters/troubleshooting.adoc
@@ -41,7 +41,7 @@ RegionServer suicides are 'normal', as this is what they do when something goes
For example, if ulimit and max transfer threads (the two most important initial settings, see <<ulimit>> and <<dfs.datanode.max.transfer.threads>>) aren't changed, it will make it impossible at some point for DataNodes to create new threads that from the HBase point of view is seen as if HDFS was gone.
Think about what would happen if your MySQL database was suddenly unable to access files on your local file system, well it's the same with HBase and HDFS.
Another very common reason to see RegionServers committing seppuku is when they enter prolonged garbage collection pauses that last longer than the default ZooKeeper session timeout.
-For more information on GC pauses, see the link:http://www.cloudera.com/blog/2011/02/avoiding-full-gcs-in-hbase-with-memstore-local-allocation-buffers-part-1/[3 part blog post] by Todd Lipcon and <<gcpause>> above.
+For more information on GC pauses, see the link:https://blog.cloudera.com/blog/2011/02/avoiding-full-gcs-in-hbase-with-memstore-local-allocation-buffers-part-1/[3 part blog post] by Todd Lipcon and <<gcpause>> above.
[[trouble.log]]
== Logs
@@ -211,7 +211,7 @@ Similarly, to enable GC logging for client processes, uncomment one of the below
# If <FILE-PATH> is not replaced, the log file(.gc) would be generated in the HBASE_LOG_DIR .
----
-For more information on GC pauses, see the link:http://www.cloudera.com/blog/2011/02/avoiding-full-gcs-in-hbase-with-memstore-local-allocation-buffers-part-1/[3 part blog post] by Todd Lipcon and <<gcpause>> above.
+For more information on GC pauses, see the link:https://blog.cloudera.com/blog/2011/02/avoiding-full-gcs-in-hbase-with-memstore-local-allocation-buffers-part-1/[3 part blog post] by Todd Lipcon and <<gcpause>> above.
[[trouble.resources]]
== Resources
http://git-wip-us.apache.org/repos/asf/hbase/blob/45f9865e/src/main/asciidoc/_chapters/upgrading.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/upgrading.adoc b/src/main/asciidoc/_chapters/upgrading.adoc
index d731542..9552024 100644
--- a/src/main/asciidoc/_chapters/upgrading.adoc
+++ b/src/main/asciidoc/_chapters/upgrading.adoc
@@ -71,9 +71,10 @@ In addition to the usual API versioning considerations HBase has other compatibi
.Client API compatibility
* Allow changing or removing existing client APIs.
-* An API needs to deprecated for a major version before we will change/remove it.
+* An API needs to be deprecated for a major version before we will change/remove it.
* APIs available in a patch version will be available in all later patch versions. However, new APIs may be added which will not be available in earlier patch versions.
-* Example: A user using a newly deprecated api does not need to modify application code with hbase api calls until the next major version.
+* New APIs introduced in a patch version will only be added in a source compatible way footnote:[See 'Source Compatibility' https://blogs.oracle.com/darcy/entry/kinds_of_compatibility]: i.e. code that implements public APIs will continue to compile.
+* Example: A user using a newly deprecated API does not need to modify application code with HBase API calls until the next major version.
.Client Binary compatibility
* Client code written to APIs available in a given patch release can run unchanged (no recompilation needed) against the new jars of later patch versions.
@@ -95,7 +96,7 @@ In addition to the usual API versioning considerations HBase has other compatibi
* JMX APIs exposed via the `/jmx/` endpoint
.Summary
-* A patch upgrade is a drop-in replacement. Any change that is not Java binary compatible would not be allowed.footnote:[See http://docs.oracle.com/javase/specs/jls/se7/html/jls-13.html.]. Downgrading versions within patch releases may not be compatible.
+* A patch upgrade is a drop-in replacement. Any change that is not Java binary and source compatible would not be allowed.footnote:[See http://docs.oracle.com/javase/specs/jls/se7/html/jls-13.html.] Downgrading versions within patch releases may not be compatible.
* A minor upgrade requires no application/client code modification. Ideally it would be a drop-in replacement but client code, coprocessors, filters, etc might have to be recompiled if new jars are used.
[2/2] hbase git commit: update CHANGES.txt for 1.1.6rc2
Posted by nd...@apache.org.
update CHANGES.txt for 1.1.6rc2
Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/2039d967
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/2039d967
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/2039d967
Branch: refs/heads/branch-1.1
Commit: 2039d967835afd066676b83f0bcb722cc55fdde3
Parents: 45f9865
Author: Nick Dimiduk <nd...@apache.org>
Authored: Sun Aug 28 10:06:28 2016 -0700
Committer: Nick Dimiduk <nd...@apache.org>
Committed: Sun Aug 28 10:06:28 2016 -0700
----------------------------------------------------------------------
CHANGES.txt | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hbase/blob/2039d967/CHANGES.txt
----------------------------------------------------------------------
diff --git a/CHANGES.txt b/CHANGES.txt
index d7358d6..1476991 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,7 +1,7 @@
HBase Change Log
-Release Notes - HBase - Version 1.1.6 08/13/2016
+Release Notes - HBase - Version 1.1.6 08/28/2016
** Sub-task
* [HBASE-15878] - Deprecate doBulkLoad(Path hfofDir, final HTable table) in branch-1 (even though its 'late')
@@ -10,7 +10,9 @@ Release Notes - HBase - Version 1.1.6 08/13/2016
* [HBASE-16194] - Should count in MSLAB chunk allocation into heap size change when adding duplicate cells
* [HBASE-16195] - Should not add chunk into chunkQueue if not using chunk pool in HeapMemStoreLAB
* [HBASE-16317] - revert all ESAPI changes
+ * [HBASE-16318] - fail build if license isn't in whitelist
* [HBASE-16321] - Ensure findbugs jsr305 jar isn't present
+ * [HBASE-16452] - Procedure v2 - Make ProcedureWALPrettyPrinter extend Tool
@@ -21,6 +23,7 @@ Release Notes - HBase - Version 1.1.6 08/13/2016
** Bug
* [HBASE-11625] - Reading datablock throws "Invalid HFile block magic" and can not switch to hdfs checksum
* [HBASE-15615] - Wrong sleep time when RegionServerCallable need retry
+ * [HBASE-15635] - Mean age of Blocks in cache (seconds) on webUI should be greater than zero
* [HBASE-15698] - Increment TimeRange not serialized to server
* [HBASE-15801] - Upgrade checkstyle for all branches
* [HBASE-15811] - Batch Get after batch Put does not fetch all Cells
@@ -49,6 +52,7 @@ Release Notes - HBase - Version 1.1.6 08/13/2016
* [HBASE-16207] - can't restore snapshot without "Admin" permission
* [HBASE-16237] - Blocks for hbase:meta table are not cached in L1 cache
* [HBASE-16238] - It's useless to catch SESSIONEXPIRED exception and retry in RecoverableZooKeeper
+ * [HBASE-16270] - Handle duplicate clearing of snapshot in region replicas
* [HBASE-16272] - Overflow in ServerName's compareTo method
* [HBASE-16281] - TestMasterReplication is flaky
* [HBASE-16288] - HFile intermediate block level indexes might recurse forever creating multi TB files
@@ -57,7 +61,12 @@ Release Notes - HBase - Version 1.1.6 08/13/2016
* [HBASE-16315] - RegionSizeCalculator prints region names as binary without escapes
* [HBASE-16319] - Fix TestCacheOnWrite after HBASE-16288
* [HBASE-16350] - Undo server abort from HBASE-14968
+ * [HBASE-16360] - TableMapReduceUtil addHBaseDependencyJars has the wrong class name for PrefixTreeCodec
* [HBASE-16368] - test*WhenRegionMove in TestPartialResultsFromClientSide is flaky
+ * [HBASE-16420] - Fix source incompatibility of Table interface
+ * [HBASE-16429] - FSHLog: deadlock if rollWriter called when ring buffer filled with appends
+ * [HBASE-16464] - archive folder grows bigger and bigger due to corrupt snapshot under tmp dir
+ * [HBASE-16471] - Region Server metrics context will be wrong when machine hostname contain "master" word
@@ -99,6 +108,8 @@ Release Notes - HBase - Version 1.1.6 08/13/2016
* [HBASE-15895] - remove unmaintained jenkins build analysis tool.
* [HBASE-16073] - update compatibility_checker for jacc dropping comma sep args
* [HBASE-16154] - bring non-master branches up to date wrt check_compatibility script
+ * [HBASE-16376] - Document implicit side-effects on partial results when calling Scan#setBatch(int)
+ * [HBASE-16467] - Move AbstractHBaseTool to hbase-common