You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hbase.apache.org by zh...@apache.org on 2020/04/18 01:34:42 UTC
[hbase] branch master updated: HBASE-23800 Add documentation about
the CECPs changes (#1487)
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new 2c8d104 HBASE-23800 Add documentation about the CECPs changes (#1487)
2c8d104 is described below
commit 2c8d104d71c996a797426fd7f0ef0921a1657f8a
Author: Duo Zhang <zh...@apache.org>
AuthorDate: Sat Apr 18 09:34:31 2020 +0800
HBASE-23800 Add documentation about the CECPs changes (#1487)
Signed-off-by: stack <st...@apache.org>
---
src/main/asciidoc/_chapters/cp.adoc | 31 +++++++++++--
src/main/asciidoc/_chapters/protobuf.adoc | 72 +++++++++++++++++++++++++++++++
2 files changed, 99 insertions(+), 4 deletions(-)
diff --git a/src/main/asciidoc/_chapters/cp.adoc b/src/main/asciidoc/_chapters/cp.adoc
index d2e424b..43aa551 100644
--- a/src/main/asciidoc/_chapters/cp.adoc
+++ b/src/main/asciidoc/_chapters/cp.adoc
@@ -178,11 +178,28 @@ average or summation for an entire table which spans hundreds of regions.
In contrast to observer coprocessors, where your code is run transparently, endpoint
coprocessors must be explicitly invoked using the
-link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/Table.html#coprocessorService-java.lang.Class-byte:A-byte:A-org.apache.hadoop.hbase.client.coprocessor.Batch.Call-[CoprocessorService()]
+link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/AsyncTable.html#coprocessorService-java.util.function.Function-org.apache.hadoop.hbase.client.ServiceCaller-byte:A-[CoprocessorService()]
method available in
-link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/Table.html[Table]
-or
-link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/HTable.html[HTable].
+link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/AsyncTable.html[AsyncTable].
+
+[WARNING]
+.On using coprocessorService method with sync client
+====
+The coprocessorService method in link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/Table.html[Table]
+has been deprecated.
+
+In link:https://issues.apache.org/jira/browse/HBASE-21512[HBASE-21512]
+we reimplement the sync client based on the async client. The coprocessorService
+method defined in `Table` interface directly references a method from protobuf's
+`BlockingInterface`, which means we need to use a separate thread pool to execute
+the method so we avoid blocking the async client(We want to avoid blocking calls in
+our async implementation).
+
+Since coprocessor is an advanced feature, we believe it is OK for coprocessor users to
+instead switch over to use `AsyncTable`. There is a lightweight
+link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/Connection.html#toAsyncConnection--[toAsyncConnection]
+method to get an `AsyncConnection` from `Connection` if needed.
+====
Starting with HBase 0.96, endpoint coprocessors are implemented using Google Protocol
Buffers (protobuf). For more details on protobuf, see Google's
@@ -193,6 +210,12 @@ link:https://issues.apache.org/jira/browse/HBASE-5448[HBASE-5448]). To upgrade y
HBase cluster from 0.94 or earlier to 0.96 or later, you need to reimplement your
coprocessor.
+In HBase 2.x, we made use of a shaded version of protobuf 3.x, but kept the
+protobuf for coprocessors on 2.5.0. In HBase 3.0.0, we removed all dependencies on
+non-shaded protobuf so you need to reimplement your coprocessor to make use of the
+shaded protobuf version provided in hbase-thirdparty. Please see
+the <<protobuf,protobuf>> section for more details.
+
Coprocessor Endpoints should make no use of HBase internals and
only avail of public APIs; ideally a CPEP should depend on Interfaces
and data structures only. This is not always possible but beware
diff --git a/src/main/asciidoc/_chapters/protobuf.adoc b/src/main/asciidoc/_chapters/protobuf.adoc
index ad7e378..7b26f97 100644
--- a/src/main/asciidoc/_chapters/protobuf.adoc
+++ b/src/main/asciidoc/_chapters/protobuf.adoc
@@ -148,3 +148,75 @@ consider extending it also in
Going forward, we will provide a new module of common types for use
by CPEPs that will have the same guarantees against change as does our
public API. TODO.
+
+=== protobuf changes for hbase-3.0.0 (HBASE-23797)
+Since hadoop(start from 3.3.x) also shades protobuf and bumps the version to
+3.x, there is no reason for us to stay on protobuf 2.5.0 any more.
+
+In HBase 3.0.0, the hbase-protocol module has been purged, the CPEP
+implementation should use the protos in hbase-protocol-shaded module, and also
+make use of the shaded protobuf in hbase-thirdparty. In general, we will keep
+the protobuf version compatible for a whole major release, unless there are
+critical problems, for example, a critical CVE on protobuf.
+
+Add this dependency to your pom:
+[source,xml]
+----
+<dependency>
+ <groupId>org.apache.hbase.thirdparty</groupId>
+ <artifactId>hbase-shaded-protobuf</artifactId>
+ <!-- use the version that your target hbase cluster uses -->
+ <version>${hbase-thirdparty.version}</version>
+ <scope>provided</scope>
+</dependency>
+----
+
+And typically you also need to add this plugin to your pom to make your
+generated protobuf code also use the shaded and relocated protobuf version
+in hbase-thirdparty.
+[source,xml]
+----
+<plugin>
+ <groupId>com.google.code.maven-replacer-plugin</groupId>
+ <artifactId>replacer</artifactId>
+ <version>1.5.3</version>
+ <executions>
+ <execution>
+ <phase>process-sources</phase>
+ <goals>
+ <goal>replace</goal>
+ </goals>
+ </execution>
+ </executions>
+ <configuration>
+ <basedir>${basedir}/target/generated-sources/</basedir>
+ <includes>
+ <include>**/*.java</include>
+ </includes>
+ <!-- Ignore errors when missing files, because it means this build
+ was run with -Dprotoc.skip and there is no -Dreplacer.skip -->
+ <ignoreErrors>true</ignoreErrors>
+ <replacements>
+ <replacement>
+ <token>([^\.])com.google.protobuf</token>
+ <value>$1org.apache.hbase.thirdparty.com.google.protobuf</value>
+ </replacement>
+ <replacement>
+ <token>(public)(\W+static)?(\W+final)?(\W+class)</token>
+ <value>@javax.annotation.Generated("proto") $1$2$3$4</value>
+ </replacement>
+ <!-- replacer doesn't support anchoring or negative lookbehind -->
+ <replacement>
+ <token>(@javax.annotation.Generated\("proto"\) ){2}</token>
+ <value>$1</value>
+ </replacement>
+ </replacements>
+ </configuration>
+</plugin>
+----
+
+In hbase-examples module, we have some examples under the
+`org.apache.hadoop.hbase.coprocessor.example` package. You can see
+`BulkDeleteEndpoint` and `BulkDelete.proto` for more details, and you can also
+check the `pom.xml` of hbase-examples module to see how to make use of the above
+plugin.