You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gearpump.apache.org by ap...@apache.org on 2016/04/11 20:26:28 UTC
[26/50] incubator-gearpump git commit: A few documentation points
A few documentation points
Project: http://git-wip-us.apache.org/repos/asf/incubator-gearpump/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-gearpump/commit/f64af570
Tree: http://git-wip-us.apache.org/repos/asf/incubator-gearpump/tree/f64af570
Diff: http://git-wip-us.apache.org/repos/asf/incubator-gearpump/diff/f64af570
Branch: refs/heads/master
Commit: f64af570ed8f5dbed8a3bb942f753b0d656642a2
Parents: 1445d12
Author: Riccardo V. Vincelli <rv...@github.com>
Authored: Tue Feb 9 13:10:13 2016 +0100
Committer: Riccardo V. Vincelli <rv...@github.com>
Committed: Tue Feb 9 13:18:26 2016 +0100
----------------------------------------------------------------------
docs/deployment-security.md | 2 +-
docs/dev-connectors.md | 17 ++++++++++++++++-
docs/dev-custom-serializer.md | 13 +++++++++++++
3 files changed, 30 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-gearpump/blob/f64af570/docs/deployment-security.md
----------------------------------------------------------------------
diff --git a/docs/deployment-security.md b/docs/deployment-security.md
index 331aeba..4be1346 100644
--- a/docs/deployment-security.md
+++ b/docs/deployment-security.md
@@ -55,7 +55,7 @@ provided for the gearpump-hbase connector. Specifically, the `UserConfig` object
```scala
val principal = "gearpump/fully.qualified.domain.name@YOUR-REALM.COM"
-val keytabContent = Files.toByteArray(new File("path_to_keytab_file))
+val keytabContent = Files.toByteArray(new File("path_to_keytab_file"))
val appConfig = UserConfig.empty
.withString("gearpump.kerberos.principal", principal)
.withBytes("gearpump.keytab.file", keytabContent)
http://git-wip-us.apache.org/repos/asf/incubator-gearpump/blob/f64af570/docs/dev-connectors.md
----------------------------------------------------------------------
diff --git a/docs/dev-connectors.md b/docs/dev-connectors.md
index 458956b..a9983fc 100644
--- a/docs/dev-connectors.md
+++ b/docs/dev-connectors.md
@@ -104,7 +104,7 @@ To use `HBaseSink` in your application, you first need to add the `gearpump-exte
To connect to HBase, you need to provide following info:
- the HBase configuration to tell which HBase service to connect
- - the table name
+ - the table name (you must create the table yourself, see the [HBase documentation](https://hbase.apache.org/book.html))
Then, you can use `HBaseSink` in your application:
@@ -123,6 +123,21 @@ Then, you can use `HBaseSink` in your application:
You can tune the connection to HBase via the HBase configuration passed in. If not passed, Gearpump will try to check local classpath to find a valid HBase configuration (`hbase-site.xml`).
+Attention, due to the issue discussed [here](http://stackoverflow.com/questions/24456484/hbase-managed-zookeeper-suddenly-trying-to-connect-to-localhost-instead-of-zooke) you may need to create additional configuration for your HBase sink:
+
+```scala
+ def hadoopConfig = {
+ val conf = new Configuration()
+ conf.set("hbase.zookeeper.quorum", "zookeeperHost")
+ conf.set("hbase.zookeeper.property.clientPort", "2181")
+ conf
+ }
+```
+
+```scala
+ val sink = HBaseSink(UserConfig.empty, tableName, hadoopConfig)
+```
+
## How to implement your own `DataSource`
To implement your own `DataSource`, you need to implement two things:
http://git-wip-us.apache.org/repos/asf/incubator-gearpump/blob/f64af570/docs/dev-custom-serializer.md
----------------------------------------------------------------------
diff --git a/docs/dev-custom-serializer.md b/docs/dev-custom-serializer.md
index a35162b..ca0a332 100644
--- a/docs/dev-custom-serializer.md
+++ b/docs/dev-custom-serializer.md
@@ -29,6 +29,19 @@ When you decide that you want to define a custom serializer, you can do this in
Please note that Gearpump shaded the original Kryo dependency. The package name ```com.esotericsoftware``` was relocated to ```io.gearpump.esotericsoftware```. So in the following customization, you should import corresponding shaded classes, the example code will show that part.
+In general you should use the shaded version of a library whenever possible in order to avoid binary incompatibilities, eg don't use:
+
+```scala
+ import com.google.common.io.Files
+```
+
+but rather
+
+```scala
+ import io.gearpump.google.common.io.Files
+```
+
+
##### System Level Serializer
If the serializer is widely used, you can define a global serializer which is available to all applications(or worker or master) in the system.