You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by ac...@apache.org on 2019/12/12 18:57:55 UTC
[camel-kafka-connector] branch cassandraql-source created (now
bb575f6)
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a change to branch cassandraql-source
in repository https://gitbox.apache.org/repos/asf/camel-kafka-connector.git.
at bb575f6 A bit more details in the cassandraql source connector example
This branch includes the following new commits:
new dbdac7e Added a first example of camel-cassandraql source connector
new 97209d8 Added detailed instructions on how to run the example for camel-cassandraql source connector
new bb575f6 A bit more details in the cassandraql source connector example
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[camel-kafka-connector] 02/03: Added detailed instructions on how
to run the example for camel-cassandraql source connector
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch cassandraql-source
in repository https://gitbox.apache.org/repos/asf/camel-kafka-connector.git
commit 97209d8fc92053ad8e0ad30c782ff2d3fd383893
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Thu Dec 12 19:53:29 2019 +0100
Added detailed instructions on how to run the example for camel-cassandraql source connector
---
docs/try-it-out-locally.adoc | 59 +++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 58 insertions(+), 1 deletion(-)
diff --git a/docs/try-it-out-locally.adoc b/docs/try-it-out-locally.adoc
index 5141669..67b0b3b 100644
--- a/docs/try-it-out-locally.adoc
+++ b/docs/try-it-out-locally.adoc
@@ -69,6 +69,63 @@ export CLASSPATH="$(find core/target/ -type f -name '*.jar'| grep '\-package' |
$KAFKA_HOME/bin/connect-standalone.sh $KAFKA_HOME/config/connect-standalone.properties examples/CamelTelegramSourceConnector.properties
----
+.Run the cassandraql source:
+Change your telegram bot token in `examples/CamelCassandraQLSourceConnector.properties`
+
+To run this example you'll need a bit more work:
+
+First you'll need to run a cassandra instance:
+
+[source,bash]
+----
+docker run --name master_node --env MAX_HEAP_SIZE='800M' -dt oscerd/cassandra
+----
+
+Check everything is fine:
+
+[source,bash]
+----
+docker exec -ti master_node /opt/cassandra/bin/nodetool status
+Datacenter: datacenter1
+=======================
+Status=Up/Down
+|/ State=Normal/Leaving/Joining/Moving
+-- Address Load Tokens Owns (effective) Host ID Rack
+UN 172.17.0.2 251.32 KiB 256 100.0% 5126aaad-f143-43e9-920a-0f9540a93967 rack1
+----
+
+Now we can populate the database:
+
+[source,bash]
+----
+<LOCAL_CASSANDRA_HOME>/bin/cqlsh $(docker inspect --format='{{ .NetworkSettings.IPAddress }}' master_node)
+----
+
+and run the script:
+
+[source,bash]
+----
+create keyspace test with replication = {'class':'SimpleStrategy', 'replication_factor':3};
+use test;
+create table users ( id int primary key, name text );
+insert into users (id,name) values (1, 'oscerd');
+quit;
+----
+
+The output of the following command should be used in the configuration of CamelCassandraQLSourceConnector.properties
+
+[source,bash]
+----
+<LOCAL_CASSANDRA_HOME>/bin/cqlsh $(docker inspect --format='{{ .NetworkSettings.IPAddress }}' master_node)
+----
+
+in particular it should be used as address instead of localhost in the `camel.source.url`
+[source,bash]
+----
+export CLASSPATH="$(find core/target/ -type f -name '*.jar'| grep '\-package' | tr '\n' ':')"
+$KAFKA_HOME/bin/connect-standalone.sh $KAFKA_HOME/config/connect-standalone.properties examples/CamelCassandraQLSourceConnector.properties
+----
+
.Run the file sink, just a camel file appending to /tmp/kafkaconnect.txt:
[source,bash]
----
@@ -168,4 +225,4 @@ $KAFKA_HOME/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --to
[source,bash]
----
$KAFKA_HOME/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic mytopic
-----
\ No newline at end of file
+----
[camel-kafka-connector] 01/03: Added a first example of
camel-cassandraql source connector
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch cassandraql-source
in repository https://gitbox.apache.org/repos/asf/camel-kafka-connector.git
commit dbdac7e5870f488309df48e3b5cb726d3c3455ec
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Thu Dec 12 19:45:48 2019 +0100
Added a first example of camel-cassandraql source connector
---
core/pom.xml | 11 ++++++++
.../CamelCassandraQLSourceConnector.properties | 30 ++++++++++++++++++++++
parent/pom.xml | 23 ++++++++++++++++-
tests/pom.xml | 10 ++++++++
4 files changed, 73 insertions(+), 1 deletion(-)
diff --git a/core/pom.xml b/core/pom.xml
index ab2e4d8..576fd61 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -47,6 +47,17 @@
<!-- Camel components -->
<dependency>
<groupId>org.apache.camel</groupId>
+ <artifactId>camel-cassandraql</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.camel</groupId>
<artifactId>camel-telegram</artifactId>
<scope>provided</scope>
</dependency>
diff --git a/examples/CamelCassandraQLSourceConnector.properties b/examples/CamelCassandraQLSourceConnector.properties
new file mode 100644
index 0000000..c407f05
--- /dev/null
+++ b/examples/CamelCassandraQLSourceConnector.properties
@@ -0,0 +1,30 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+name=CamelCassandraQLSourceConnector
+connector.class=org.apache.camel.kafkaconnector.CamelSourceConnector
+key.converter=org.apache.kafka.connect.storage.StringConverter
+value.converter=org.apache.kafka.connect.storage.StringConverter
+#transforms=S3ObjectTransformer
+#transforms.S3ObjectTransformer.type=org.apache.camel.kafkaconnector.converters.S3ObjectTransformer
+
+camel.source.maxPollDuration=10000
+
+camel.source.kafka.topic=mytopic
+
+camel.source.url=cql://172.17.0.2/test?cql=select * from users&resultSetConversionStrategy=ONE
+
diff --git a/parent/pom.xml b/parent/pom.xml
index a5da3a4..7f32be9 100644
--- a/parent/pom.xml
+++ b/parent/pom.xml
@@ -110,6 +110,17 @@
<artifactId>camel-main</artifactId>
<version>${camel.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.apache.camel</groupId>
+ <artifactId>camel-cassandraql</artifactId>
+ <version>${camel.version}</version>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-telegram</artifactId>
@@ -160,7 +171,17 @@
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-cbor</artifactId>
- <version>2.9.9</version>
+ <version>2.10.1</version>
+ </dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.core</groupId>
+ <artifactId>jackson-core</artifactId>
+ <version>2.10.1</version>
+ </dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.databind</groupId>
+ <artifactId>jackson-databind</artifactId>
+ <version>2.10.1</version>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
diff --git a/tests/pom.xml b/tests/pom.xml
index 7c3a7a2..ebcaac3 100644
--- a/tests/pom.xml
+++ b/tests/pom.xml
@@ -48,6 +48,16 @@
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
+ <artifactId>camel-cassandraql</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.camel</groupId>
<artifactId>camel-aws-sqs</artifactId>
</dependency>
<dependency>
[camel-kafka-connector] 03/03: A bit more details in the
cassandraql source connector example
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch cassandraql-source
in repository https://gitbox.apache.org/repos/asf/camel-kafka-connector.git
commit bb575f62dca81620c110729a0a3cd01cd6615fad
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Thu Dec 12 19:57:21 2019 +0100
A bit more details in the cassandraql source connector example
---
docs/try-it-out-locally.adoc | 1 +
1 file changed, 1 insertion(+)
diff --git a/docs/try-it-out-locally.adoc b/docs/try-it-out-locally.adoc
index 67b0b3b..3da3943 100644
--- a/docs/try-it-out-locally.adoc
+++ b/docs/try-it-out-locally.adoc
@@ -94,6 +94,7 @@ Status=Up/Down
UN 172.17.0.2 251.32 KiB 256 100.0% 5126aaad-f143-43e9-920a-0f9540a93967 rack1
----
+You'll need a local installation of cassandra, in particular the 3.11.4.
Now we can populate the database:
[source,bash]