You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by ac...@apache.org on 2020/11/10 08:48:50 UTC
[camel-kafka-connector-examples] branch ckc-google-mail-stream
created (now 44f995d)
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a change to branch ckc-google-mail-stream
in repository https://gitbox.apache.org/repos/asf/camel-kafka-connector-examples.git.
at 44f995d Added a Google Mail Stream Source Connector example
This branch includes the following new commits:
new 44f995d Added a Google Mail Stream Source Connector example
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[camel-kafka-connector-examples] 01/01: Added a Google Mail Stream
Source Connector example
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch ckc-google-mail-stream
in repository https://gitbox.apache.org/repos/asf/camel-kafka-connector-examples.git
commit 44f995df42844d1f3aa29e54ded33dbe19166296
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Tue Nov 10 09:48:15 2020 +0100
Added a Google Mail Stream Source Connector example
---
.../google-mail-stream-source/README.adoc | 108 +++++++++++++++++++++
...CamelGoogleMailStreamSourceConnector.properties | 35 +++++++
2 files changed, 143 insertions(+)
diff --git a/google-mail-stream/google-mail-stream-source/README.adoc b/google-mail-stream/google-mail-stream-source/README.adoc
new file mode 100644
index 0000000..07bccfe
--- /dev/null
+++ b/google-mail-stream/google-mail-stream-source/README.adoc
@@ -0,0 +1,108 @@
+= Camel-Kafka-connector Google Mail Stream Source Example
+
+This is an example for Camel-Kafka-connector Google Mail Stream Source Example
+
+== Standalone
+
+=== What is needed
+
+- A google account
+- A google application authorized to deal with gmail API
+
+=== Setting up Google Application
+
+The step-by-step guide is a bit long, so follow it with attention:
+
+1. Create a project
+2. Note down clientSecret and clientId
+2. Create an OAuth consent screen
+3. Create OAuth Client credentials (OAuth for WebApp)
+4. Add https://developers.google.com/oauthplayground to Authorized redirect URIs
+5. Setup https://developers.google.com/oauthplayground using credentials from above
+6. Authorize APIs for Gmail API v1 scope https://mail.google.com
+7. Exchange authorization code for tokens
+8. Note down Access token & Refresh token
+9. Mark some email as unread in your account
+
+We auto-sent an email with the following body "Camel Kafka Connector is great!" and marked it as unread.
+
+=== Running Kafka
+
+[source]
+----
+$KAFKA_HOME/bin/zookeeper-server-start.sh $KAFKA_HOME/config/zookeeper.properties
+$KAFKA_HOME/bin/kafka-server-start.sh $KAFKA_HOME/config/server.properties
+$KAFKA_HOME/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic mytopic
+----
+
+=== Download the connector package
+
+Download the connector package zip and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/`
+
+[source]
+----
+> cd /home/oscerd/connectors/
+> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-google-mail-stream-kafka-connector/0.6.0/camel-google-mail-stream-kafka-connector-0.6.0-package.zip
+> unzip camel-google-mail-stream-kafka-connector-0.6.0-package.zip
+----
+
+=== Configuring Kafka Connect
+
+You'll need to set up the `plugin.path` property in your kafka
+
+Open the `$KAFKA_HOME/config/connect-standalone.properties` and set the `plugin.path` property to your choosen location:
+
+[source]
+----
+...
+plugin.path=/home/oscerd/connectors
+...
+----
+
+=== Setup the connectors
+
+Open the Git configuration file at `$EXAMPLES/google-mail-stream/google-mail-stream-source/config/CamelGoogleMailStreamSourceConnector.properties`
+
+[source]
+----
+name=CamelGoogle-mail-streamSourceConnector
+connector.class=org.apache.camel.kafkaconnector.googlemailstream.CamelGooglemailstreamSourceConnector
+tasks.max=1
+
+key.converter=org.apache.kafka.connect.storage.StringConverter
+value.converter=org.apache.kafka.connect.storage.StringConverter
+
+topics=mytopic
+
+camel.source.path.index=ckc
+camel.source.endpoint.clientId=<client_id>
+camel.source.endpoint.accessToken=<access_token>
+camel.source.endpoint.refreshToken=<refresh_token>
+camel.source.endpoint.clientSecret=<client_secret>
+camel.source.endpoint.applicationName=ckc
+camel.source.endpoint.delay=60000
+camel.source.endpoint.markAsRead=true
+camel.source.endpoint.labels=inbox
+camel.source.endpoint.query=is:unread -category:(promotions OR social)
+camel.source.endpoint.maxResults=20
+----
+
+=== Running the example
+
+Run the kafka connect with the Google Mail Stream Source connector:
+
+[source]
+----
+$KAFKA_HOME/bin/connect-standalone.sh $KAFKA_HOME/config/connect-standalone.properties $EXAMPLES/google-mail-stream/google-mail-stream-source/config/CamelGoogleMailStreamSourceConnector.properties
+----
+
+On a different terminal run the kafkacat consumer
+
+[source]
+----
+> ./kafkacat -b localhost:9092 -t mytopic
+Camel Kafka Connector is great!
+% Reached end of topic mytopic [0] at offset 1
+----
+
+In the google mail account you should now see the email marked as read.
diff --git a/google-mail-stream/google-mail-stream-source/config/CamelGoogleMailStreamSourceConnector.properties b/google-mail-stream/google-mail-stream-source/config/CamelGoogleMailStreamSourceConnector.properties
new file mode 100644
index 0000000..d888ab9
--- /dev/null
+++ b/google-mail-stream/google-mail-stream-source/config/CamelGoogleMailStreamSourceConnector.properties
@@ -0,0 +1,35 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+name=CamelGoogle-mail-streamSourceConnector
+connector.class=org.apache.camel.kafkaconnector.googlemailstream.CamelGooglemailstreamSourceConnector
+key.converter=org.apache.kafka.connect.storage.StringConverter
+value.converter=org.apache.kafka.connect.storage.StringConverter
+
+topics=mytopic
+
+camel.source.path.index=ckc
+camel.source.endpoint.clientId=<client_id>
+camel.source.endpoint.accessToken=<access_token>
+camel.source.endpoint.refreshToken=<refresh_token>
+camel.source.endpoint.clientSecret=<client_secret>
+camel.source.endpoint.applicationName=ckc
+camel.source.endpoint.delay=60000
+camel.source.endpoint.markAsRead=true
+camel.source.endpoint.labels=inbox
+camel.source.endpoint.query=is:unread -category:(promotions OR social)
+camel.source.endpoint.maxResults=20