You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by ja...@apache.org on 2018/10/05 18:02:47 UTC
samza git commit: Adding connectors overview doc
Repository: samza
Updated Branches:
refs/heads/master 6b3189436 -> 4baaddbbb
Adding connectors overview doc
Author: Aditya Toomula <at...@linkedin.com>
Reviewers: Jagadish<ja...@apache.org>
Closes #689 from atoomula/connector
Project: http://git-wip-us.apache.org/repos/asf/samza/repo
Commit: http://git-wip-us.apache.org/repos/asf/samza/commit/4baaddbb
Tree: http://git-wip-us.apache.org/repos/asf/samza/tree/4baaddbb
Diff: http://git-wip-us.apache.org/repos/asf/samza/diff/4baaddbb
Branch: refs/heads/master
Commit: 4baaddbbbb665fefac821eb0aecd1608d5a6bdf4
Parents: 6b31894
Author: Aditya Toomula <at...@linkedin.com>
Authored: Fri Oct 5 11:02:45 2018 -0700
Committer: Jagadish <jv...@linkedin.com>
Committed: Fri Oct 5 11:02:45 2018 -0700
----------------------------------------------------------------------
.../documentation/versioned/api/table-api.md | 2 +-
.../versioned/connectors/overview.md | 32 ++++++++++++++++++--
2 files changed, 31 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/samza/blob/4baaddbb/docs/learn/documentation/versioned/api/table-api.md
----------------------------------------------------------------------
diff --git a/docs/learn/documentation/versioned/api/table-api.md b/docs/learn/documentation/versioned/api/table-api.md
index d95dfac..f5efa88 100644
--- a/docs/learn/documentation/versioned/api/table-api.md
+++ b/docs/learn/documentation/versioned/api/table-api.md
@@ -191,7 +191,7 @@ The code snippet below illustrates the usage of table in Samza low level API.
5 DelegatingSystemDescriptor ksd = new DelegatingSystemDescriptor("mySystem");
6
7 TableDescriptor<Integer, Profile> tableDesc = new InMemoryTableDescriptor(
- 8 "T1", KVSerde.of(new IntegerSerde(), new ProfileJsonSerde()));
+ 8 "t1", KVSerde.of(new IntegerSerde(), new ProfileJsonSerde()));
9 GenericInputDescriptor<Profile> profileISD = ksd.getInputDescriptor("Profile", new NoOpSerde<>());
10
11 appDesc.addTable(tableDesc);
http://git-wip-us.apache.org/repos/asf/samza/blob/4baaddbb/docs/learn/documentation/versioned/connectors/overview.md
----------------------------------------------------------------------
diff --git a/docs/learn/documentation/versioned/connectors/overview.md b/docs/learn/documentation/versioned/connectors/overview.md
index 579c494..5b6ba39 100644
--- a/docs/learn/documentation/versioned/connectors/overview.md
+++ b/docs/learn/documentation/versioned/connectors/overview.md
@@ -19,6 +19,34 @@ title: Connectors overview
limitations under the License.
-->
-# Section 1
-# Section 2
+Stream processing applications often read data from external sources like Kafka or HDFS. Likewise, they require processed
+results to be written to external system or data stores. As of the 1.0 release, Samza integrates with the following systems
+out-of-the-box:
+
+- [Apache Kafka](kafka) (consumer/producer)
+- [Microsoft Azure Eventhubs](eventhubs) (consumer/producer)
+- [Amazon AWS Kinesis Streams](kinesis) (consumer)
+- [Hadoop Filesystem](hdfs) (consumer/producer)
+- [Elasticsearch](https://github.com/apache/samza/blob/master/samza-elasticsearch/src/main/java/org/apache/samza/system/elasticsearch/ElasticsearchSystemProducer.java) (producer)
+
+Instructions on how to use these connectors can be found in the corresponding subsections. Please note that the
+connector API is different from [Samza Table API](../api/table-api), where the data could be read from and written to
+data stores.
+
+Samza is pluggable and designed to support a variety of producers and consumers. You can provide your own producer or
+consumer by implementing the SystemFactory interface.
+
+To associate a system with a Samza Connector, the user needs to set the following config:
+
+{% highlight jproperties %}
+systems.<system-name>.samza.factory=org.apache.samza.system.kafka.KafkaSystemFactory
+{% endhighlight %}
+
+Any system specific configs, could be defined as below:
+
+{% highlight jproperties %}
+systems.<system-name>.param1=value1
+systems.<system-name>.consumer.param2=value2
+systems.<system-name>.producer.param3=value3
+{% endhighlight %}