You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by ac...@apache.org on 2021/03/01 17:34:35 UTC
[camel-kafka-connector-examples] branch infinispan-fix-1 created
(now c308cb3)
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a change to branch infinispan-fix-1
in repository https://gitbox.apache.org/repos/asf/camel-kafka-connector-examples.git.
at c308cb3 Review Infinispan examples after component splitting - Infinispan-sink example
This branch includes the following new commits:
new c308cb3 Review Infinispan examples after component splitting - Infinispan-sink example
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[camel-kafka-connector-examples] 01/01: Review Infinispan examples
after component splitting - Infinispan-sink example
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch infinispan-fix-1
in repository https://gitbox.apache.org/repos/asf/camel-kafka-connector-examples.git
commit c308cb3ba3138f0ca5047afdfdeefdd42c52b05f
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Mon Mar 1 18:34:03 2021 +0100
Review Infinispan examples after component splitting - Infinispan-sink example
---
infinispan/infinispan-sink/README.adoc | 130 +++++++++++++++++----------------
1 file changed, 68 insertions(+), 62 deletions(-)
diff --git a/infinispan/infinispan-sink/README.adoc b/infinispan/infinispan-sink/README.adoc
index 19e53e8..e576f6b 100644
--- a/infinispan/infinispan-sink/README.adoc
+++ b/infinispan/infinispan-sink/README.adoc
@@ -10,29 +10,29 @@ This is an example for Camel-Kafka-connector Infinispan Sink
### Setting up Infinispan
-As first step you need to download the Infinispan Server with version 11.0.3.Final.
+As first step you need to download the Infinispan Server with version 12.0.0.Final.
-Infinispan 11.x is secured by default. For the purpose of this example we'll remove the security check, so we won't need any authentication.
+Infinispan 12.x is secured by default. For the purpose of this example we'll remove the security check, so we won't need any authentication.
If you have Infinispan untar.gzped in $INFINISPAN_HOME, you'll need to:
-- Edit $INFINISPAN_HOME/server/config/infinispan.xml
+- Edit $INFINISPAN_HOME/server/conf/infinispan.xml
and the file content should be
```
<infinispan
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:schemaLocation="urn:infinispan:config:11.0 https://infinispan.org/schemas/infinispan-config-11.0.xsd
- urn:infinispan:server:11.0 https://infinispan.org/schemas/infinispan-server-11.0.xsd"
- xmlns="urn:infinispan:config:11.0"
- xmlns:server="urn:infinispan:server:11.0">
+ xsi:schemaLocation="urn:infinispan:config:12.0 https://infinispan.org/schemas/infinispan-config-12.0.xsd
+ urn:infinispan:server:12.0 https://infinispan.org/schemas/infinispan-server-12.0.xsd"
+ xmlns="urn:infinispan:config:12.0"
+ xmlns:server="urn:infinispan:server:12.0">
<cache-container name="default" statistics="true">
- <transport cluster="${infinispan.cluster.name}" stack="${infinispan.cluster.stack:tcp}" node-name="${infinispan.node.name:}"/>
+ <transport cluster="${infinispan.cluster.name:cluster}" stack="${infinispan.cluster.stack:tcp}" node-name="${infinispan.node.name:}"/>
</cache-container>
- <server xmlns="urn:infinispan:server:11.0">
+ <server xmlns="urn:infinispan:server:12.0">
<interfaces>
<interface name="public">
<inet-address value="${infinispan.bind.address:127.0.0.1}"/>
@@ -45,8 +45,8 @@ and the file content should be
</socket-bindings>
<endpoints socket-binding="default">
- <hotrod-connector name="hotrod"/>
- <rest-connector name="rest"/>
+ <hotrod-connector/>
+ <rest-connector/>
</endpoints>
</server>
</infinispan>
@@ -59,30 +59,40 @@ You can now start your server
```
> $INFINISPAN_HOME/bin/server.sh
bin/server.sh
-06:57:51,378 INFO (main) [BOOT] JVM OpenJDK 64-Bit Server VM AdoptOpenJDK 25.252-b09
-06:57:51,395 INFO (main) [BOOT] JVM arguments = [-Xms64m, -Xmx512m, -XX:MetaspaceSize=64M, -Djava.net.preferIPv4Stack=true, -Djava.awt.headless=true, -Dvisualvm.display.name=infinispan-server, -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager, -Dinfinispan.server.home.path=/home/oscerd/playground/infinispan-server-11.0.3.Final]
-06:57:51,396 INFO (main) [BOOT] PID = 9678
-06:57:51,441 INFO (main) [org.infinispan.SERVER] ISPN080000: Infinispan Server starting
-06:57:51,441 INFO (main) [org.infinispan.SERVER] ISPN080017: Server configuration: /home/oscerd/playground/infinispan-server-11.0.3.Final/server/conf/infinispan.xml
-06:57:51,441 INFO (main) [org.infinispan.SERVER] ISPN080032: Logging configuration: /home/oscerd/playground/infinispan-server-11.0.3.Final/server/conf/log4j2.xml
-06:57:51,959 INFO (main) [org.infinispan.SERVER] ISPN080027: Loaded extension 'query-dsl-filter-converter-factory'
-06:57:51,960 INFO (main) [org.infinispan.SERVER] ISPN080027: Loaded extension 'continuous-query-filter-converter-factory'
-06:57:51,961 INFO (main) [org.infinispan.SERVER] ISPN080027: Loaded extension 'iteration-filter-converter-factory'
-06:57:51,961 INFO (main) [org.infinispan.SERVER] ISPN080027: Loaded extension 'jdk.nashorn.api.scripting.NashornScriptEngineFactory'
-06:57:52,367 WARN (main) [org.infinispan.PERSISTENCE] ISPN000554: jboss-marshalling is deprecated and planned for removal
-06:57:52,919 INFO (main) [org.infinispan.CONTAINER] ISPN000128: Infinispan version: Infinispan 'Corona Extra' 11.0.3.Final
-06:57:52,921 INFO (main) [org.infinispan.CONTAINER] ISPN000389: Loaded global state, version=11.0.3.Final timestamp=2020-09-30T21:04:46.511Z
-06:57:53,046 INFO (main) [org.infinispan.CLUSTER] ISPN000078: Starting JGroups channel cluster with stack tcp
-06:57:55,138 INFO (main) [org.jgroups.protocols.pbcast.GMS] ghost-35169: no members discovered after 2001 ms: creating cluster as coordinator
-06:57:55,150 INFO (main) [org.infinispan.CLUSTER] ISPN000094: Received new cluster view for channel cluster: [ghost-35169|0] (1) [ghost-35169]
-06:57:55,156 INFO (main) [org.infinispan.CLUSTER] ISPN000079: Channel cluster local address is ghost-35169, physical addresses are [192.168.1.15:7800]
-06:57:55,810 INFO (main) [org.infinispan.CONTAINER] ISPN000104: Using EmbeddedTransactionManager
+18:07:32,435 INFO (main) [BOOT] JVM OpenJDK 64-Bit Server VM AdoptOpenJDK 11.0.7+10
+18:07:32,441 INFO (main) [BOOT] JVM arguments = [-server, -Xlog:gc*:file=/home/oscerd/playground/infinispan-server-12.0.0.Final/server/log/gc.log:time,uptimemillis:filecount=5,filesize=3M, -Xms64m, -Xmx512m, -XX:MetaspaceSize=64M, -Djava.net.preferIPv4Stack=true, -Djava.awt.headless=true, -Dvisualvm.display.name=infinispan-server, -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager, -Dinfinispan.server.home.path=/home/oscerd/playground/infinispan-server-12.0.0.Final, -cl [...]
+18:07:32,441 INFO (main) [BOOT] PID = 21270
+18:07:32,466 INFO (main) [org.infinispan.SERVER] ISPN080000: Infinispan Server starting
+18:07:32,466 INFO (main) [org.infinispan.SERVER] ISPN080017: Server configuration: infinispan.xml
+18:07:32,466 INFO (main) [org.infinispan.SERVER] ISPN080032: Logging configuration: /home/oscerd/playground/infinispan-server-12.0.0.Final/server/conf/log4j2.xml
+18:07:32,823 INFO (main) [org.infinispan.SERVER] ISPN080027: Loaded extension 'query-dsl-filter-converter-factory'
+18:07:32,823 INFO (main) [org.infinispan.SERVER] ISPN080027: Loaded extension 'continuous-query-filter-converter-factory'
+18:07:32,825 INFO (main) [org.infinispan.SERVER] ISPN080027: Loaded extension 'iteration-filter-converter-factory'
+18:07:32,825 INFO (main) [org.infinispan.SERVER] ISPN080027: Loaded extension 'jdk.nashorn.api.scripting.NashornScriptEngineFactory'
+18:07:33,172 INFO (main) [org.infinispan.CONTAINER] ISPN000556: Starting user marshaller 'org.infinispan.commons.marshall.ImmutableProtoStreamMarshaller'
+18:07:33,262 WARN (main) [org.infinispan.PERSISTENCE] ISPN000554: jboss-marshalling is deprecated and planned for removal
+18:07:33,545 INFO (main) [org.infinispan.query.remote.impl.ProtobufMetadataManagerImpl] ISPN028019: Registering protostream serialization context initializer: org.infinispan.query.core.stats.impl.PersistenceContextInitializerImpl
+18:07:33,547 INFO (main) [org.infinispan.CONTAINER] ISPN000128: Infinispan version: Infinispan 'Lockdown' 12.0.0.Final
+18:07:33,687 INFO (main) [org.infinispan.CLUSTER] ISPN000078: Starting JGroups channel cluster with stack tcp
+18:07:35,796 INFO (main) [org.jgroups.protocols.pbcast.GMS] ghost-43669: no members discovered after 2002 ms: creating cluster as coordinator
+18:07:35,832 INFO (main) [org.infinispan.CLUSTER] ISPN000094: Received new cluster view for channel cluster: [ghost-43669|0] (1) [ghost-43669]
+18:07:35,843 INFO (main) [org.infinispan.CLUSTER] ISPN000079: Channel cluster local address is ghost-43669, physical addresses are [10.36.115.182:7800]
+18:07:35,878 INFO (main) [org.infinispan.CONTAINER] ISPN000390: Persisted state, version=12.0.0.Final timestamp=2021-03-01T17:07:35.875687Z
+18:07:36,081 INFO (main) [org.jboss.threads] JBoss Threads version 2.3.3.Final
+18:07:36,150 INFO (main) [org.infinispan.CONTAINER] ISPN000104: Using EmbeddedTransactionManager
+18:07:36,447 INFO (main) [org.infinispan.server.core.RequestTracer] OpenTracing integration is disabled
+18:07:36,510 INFO (ForkJoinPool.commonPool-worker-3) [org.infinispan.SERVER] ISPN080018: Started connector HotRod (internal)
+18:07:36,601 INFO (main) [org.infinispan.SERVER] ISPN080018: Started connector REST (internal)
+18:07:36,762 INFO (main) [org.infinispan.SERVER] ISPN080004: Connector SINGLE_PORT (default) listening on 127.0.0.1:11222
+18:07:36,762 INFO (main) [org.infinispan.SERVER] ISPN080034: Server 'ghost-43669' listening on http://127.0.0.1:11222
+18:07:36,763 INFO (main) [org.infinispan.SERVER] ISPN080001: Infinispan Server 12.0.0.Final started in 4296ms
+
```
So, you'll need to run
```
-> cd $INFINISPAN_HOME/bin/cli.sh
+> $INFINISPAN_HOME/bin/cli.sh
[disconnected]> connect
[ghost-35169@cluster//containers/default]> create cache --template=org.infinispan.DIST_SYNC mycache
[ghost-35169@cluster//containers/default]> describe caches/mycache
@@ -93,9 +103,6 @@ So, you'll need to run
"state-transfer" : {
"timeout" : 60000
},
- "transaction" : {
- "mode" : "NONE"
- },
"locking" : {
"concurrency-level" : 1000,
"acquire-timeout" : 15000,
@@ -104,7 +111,30 @@ So, you'll need to run
"statistics" : true
}
}
+
```
+
+It's important to add encoding to your cache configuration, otherwise consuming events won't work.
+You'll need to:
+
+- Edit $INFINISPAN_HOME/server/data/caches.xml
+
+and add the encoding section in the configuration
+
+```
+<infinispan xmlns="urn:infinispan:config:12.0">
+ <cache-container>
+ <distributed-cache mode="SYNC" remote-timeout="17500" name="mycache" statistics="true">
+ <encoding>
+ <key media-type="text/plain; charset=UTF-8"/>
+ <value media-type="text/plain; charset=UTF-8"/>
+ </encoding>
+ <locking concurrency-level="1000" acquire-timeout="15000" striping="false"/>
+ <state-transfer timeout="60000"/>
+ </distributed-cache>
+ </cache-container></infinispan>
+```
+
Now we should be ready to work on this.
### Running Kafka
@@ -133,7 +163,7 @@ In this example we'll use `/home/oscerd/connectors/`
Now it's time to setup the connectors
-Open the AWS2 SNS configuration file
+Open the Infinispan Sink connector configuration
```
name=CamelInfinispanSinkConnector
@@ -153,12 +183,10 @@ Now you can run the example
$KAFKA_HOME/bin/connect-standalone.sh $KAFKA_HOME/config/connect-standalone.properties config/CamelInfinispanSinkConnector.properties
```
-On a different terminal run the kafka-producer and send messages to your Kafka Broker.
+On a different terminal run the kafkacat and send messages to your Kafka Broker.
```
-bin/kafka-console-producer.sh --bootstrap-server localhost:9092 --topic mytopic
-Kafka to Infinispan message 1
-Kafka to Infinispan message 2
+echo "Test" | ./kafkacat -b localhost:9092 -t mytopic -H "CamelHeader.CamelInfinispanKey=Roma" -H "CamelHeader.CamelInfinispanValue=Italy"
```
You should see the stats of cache changing. You can check this by running
@@ -167,29 +195,7 @@ You should see the stats of cache changing. You can check this by running
> $INFINISPAN_HOME/bin/cli.sh
[disconnected]> connect
[ghost-35169@cluster//containers/default]> cache mycache
-[ghost-35169@cluster//containers/default/caches/mycache]> stats
-{
- "total_number_of_entries" : 0,
- "off_heap_memory_used" : 0,
- "time_since_start" : 350,
- "time_since_reset" : 350,
- "stores" : 0,
- "current_number_of_entries" : 0,
- "data_memory_used" : 0,
- "misses" : 0,
- "remove_hits" : 0,
- "remove_misses" : 0,
- "evictions" : 0,
- "average_read_time" : 0,
- "average_read_time_nanos" : 0,
- "average_write_time" : 0,
- "average_write_time_nanos" : 0,
- "average_remove_time" : 0,
- "average_remove_time_nanos" : 0,
- "required_minimum_number_of_nodes" : 1,
- "current_number_of_entries_in_memory" : 0,
- "retrievals" : 0,
- "hits" : 0
-}
+[ghost-35169@cluster//containers/default/caches/mycache]> get Roma
+Italy
```