[camel-kafka-connector-examples] branch oc-aws2-sqs created (now 9333c0c)

[ac...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

acosentino pushed a change to branch oc-aws2-sqs
in repository https://gitbox.apache.org/repos/asf/camel-kafka-connector-examples.git.


      at 9333c0c  AWS2-SQS Source example: Adding Openshift instuctions

This branch includes the following new commits:

     new 9333c0c  AWS2-SQS Source example: Adding Openshift instuctions

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.

[camel-kafka-connector-examples] 01/01: AWS2-SQS Source example: Adding Openshift instuctions

[ac...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

acosentino pushed a commit to branch oc-aws2-sqs
in repository https://gitbox.apache.org/repos/asf/camel-kafka-connector-examples.git

commit 9333c0cad86aeeebbc7a5f002f9b4e2028af6c5f
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Sep 16 11:36:56 2020 +0200

    AWS2-SQS Source example: Adding Openshift instuctions
---
 aws2-sqs/aws2-sqs-source/README.adoc | 135 +++++++++++++++++++++++++++++++++--
 1 file changed, 130 insertions(+), 5 deletions(-)

diff --git a/aws2-sqs/aws2-sqs-source/README.adoc b/aws2-sqs/aws2-sqs-source/README.adoc
index c2512fa..37d7c05 100644
--- a/aws2-sqs/aws2-sqs-source/README.adoc
+++ b/aws2-sqs/aws2-sqs-source/README.adoc
@@ -1,14 +1,14 @@
 # Camel-Kafka-connector AWS2 SQS Source
 
-## Introduction
-
 This is an example for Camel-Kafka-connector AW2-SQS
 
-## What is needed
+## Standalone 
+
+### What is needed
 
 - An AWS SQS queue
 
-## Running Kafka
+### Running Kafka
 
 ```
 $KAFKA_HOME/bin/zookeeper-server-start.sh config/zookeeper.properties
@@ -16,7 +16,7 @@ $KAFKA_HOME/bin/kafka-server-start.sh config/server.properties
 $KAFKA_HOME/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic mytopic
 ```
 
-## Setting up the needed bits and running the example
+### Setting up the needed bits and running the example
 
 You'll need to setup the plugin.path property in your kafka
 
@@ -71,3 +71,128 @@ SQS to Kafka through Camel
 SQS to Kafka through Camel
 ```
 
+## Openshift
+
+### What is needed
+
+- An AWS SQS queue
+- An Openshift instance
+
+### Running Kafka using Strimzi Operator
+
+First we install the Strimzi operator and use it to deploy the Kafka broker and Kafka Connect into our OpenShift project.
+We need to create security objects as part of installation so it is necessary to switch to admin user.
+If you use Minishift, you can do it with the following command:
+
+[source,bash,options="nowrap"]
+----
+oc login -u system:admin
+----
+
+We will use OpenShift project `myproject`.
+If it doesn't exist yet, you can create it using following command:
+
+[source,bash,options="nowrap"]
+----
+oc new-project myproject
+----
+
+If the project already exists, you can switch to it with:
+
+[source,bash,options="nowrap"]
+----
+oc project myproject
+----
+
+We can now install the Strimzi operator into this project:
+
+[source,bash,options="nowrap",subs="attributes"]
+----
+oc apply -f https://github.com/strimzi/strimzi-kafka-operator/releases/download/0.19.0/strimzi-cluster-operator-0.19.0.yaml
+----
+
+Next we will deploy a Kafka broker cluster and a Kafka Connect cluster and then create a Kafka Connect image with the Debezium connectors installed:
+
+[source,bash,options="nowrap",subs="attributes"]
+----
+# Deploy a single node Kafka broker
+oc apply -f https://github.com/strimzi/strimzi-kafka-operator/raw/0.19.0/examples/kafka/kafka-persistent-single.yaml
+
+# Deploy a single instance of Kafka Connect with no plug-in installed
+oc apply -f https://github.com/strimzi/strimzi-kafka-operator/raw/0.19.0/examples/connect/kafka-connect-s2i-single-node-kafka.yaml
+----
+
+Optionally enable the possibility to instantiate Kafka Connectors through specific custom resource:
+[source,bash,options="nowrap"]
+----
+oc annotate kafkaconnects2is my-connect-cluster strimzi.io/use-connector-resources=true
+----
+
+### Add Camel Kafka connector binaries
+
+Strimzi uses `Source2Image` builds to allow users to add their own connectors to the existing Strimzi Docker images.
+We now need to build the connectors and add them to the image,
+if you have built the whole project (`mvn clean package`) decompress the connectors you need in a folder (i.e. like `my-connectors/`)
+so that each one is in its own subfolder
+(alternatively you can download the latest officially released and packaged connectors from maven):
+
+[source,bash,options="nowrap"]
+----
+oc start-build my-connect-cluster-connect --from-dir=./my-connectors/ --follow
+----
+
+We should now wait for the rollout of the new image to finish and the replica set with the new connector to become ready.
+Once it is done, we can check that the connectors are available in our Kafka Connect cluster.
+Strimzi is running Kafka Connect in a distributed mode.
+
+### Create connector instance
+
+Now we can create some instance of a AWS2-SQS source connector:
+
+You need to have enabled `use-connector-resources`, you can create the connector instance by creating a specific custom resource:
+
+[source,bash,options="nowrap"]
+----
+oc apply -f - << EOF
+apiVersion: kafka.strimzi.io/v1alpha1
+kind: KafkaConnector
+metadata:
+  name: sqs-source-connector
+  namespace: oscerd-ckc
+  labels:
+    strimzi.io/cluster: my-connect-cluster
+spec:
+  class: org.apache.camel.kafkaconnector.aws2sqs.CamelAws2sqsSourceConnector
+  tasksMax: 1
+  config:
+    key.converter: org.apache.kafka.connect.storage.StringConverter
+    value.converter: org.apache.kafka.connect.storage.StringConverter
+    topics: sqs-topic
+    camel.source.path.queueNameOrArn: camel-connector-test
+    camel.source.maxPollDuration: 10000
+    camel.component.aws2-sqs.accessKey: xxxx
+    camel.component.aws2-sqs.secretKey: yyyy
+    camel.component.aws2-sqs.region: region
+EOF
+----
+
+You can check the status of the connector using
+
+[source,bash,options="nowrap"]
+----
+oc exec -i -c kafka my-cluster-kafka-0 -- curl -s http://my-connect-cluster-connect-api:8083/connectors/sqs-source-connector/status
+----
+
+Just connect to your AWS Console and send message to the camel-connector-test, through the AWS Console.
+
+### Check received messages
+
+You can also run the Kafka console consumer to see the messages received from the topic:
+
+[source,bash,options="nowrap"]
+----
+oc exec -i -c kafka my-cluster-kafka-0 -- bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic sqs-topic --from-beginning
+SQS to Kafka through Camel
+SQS to Kafka through Camel
+----
+