You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by ac...@apache.org on 2023/08/28 06:12:06 UTC
[camel-kamelets-examples] 01/01: Added Opensearch Search Example
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch os-search
in repository https://gitbox.apache.org/repos/asf/camel-kamelets-examples.git
commit 789192a1e7c97e660086498df62c531bd11bb662
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Mon Aug 28 08:11:41 2023 +0200
Added Opensearch Search Example
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
jbang/opensearch-search-log/README.adoc | 200 ++++++++++++++++++++++++
jbang/opensearch-search-log/os-search-log.yaml | 34 ++++
jbang/opensearch-search-log/timer-os-index.yaml | 31 ++++
3 files changed, 265 insertions(+)
diff --git a/jbang/opensearch-search-log/README.adoc b/jbang/opensearch-search-log/README.adoc
new file mode 100644
index 0000000..7995bc1
--- /dev/null
+++ b/jbang/opensearch-search-log/README.adoc
@@ -0,0 +1,200 @@
+== Timer to Opensearch Search Source
+
+In this sample you'll use the Opensearch Index Sink Kamelet and Opensearch Search Source Kamelet based on camel-opensearch component.
+
+=== Install JBang
+
+First install JBang according to https://www.jbang.dev
+
+When JBang is installed then you should be able to run from a shell:
+
+[source,sh]
+----
+$ jbang --version
+----
+
+This will output the version of JBang.
+
+To run this example you can either install Camel on JBang via:
+
+[source,sh]
+----
+$ jbang app install camel@apache/camel
+----
+
+Which allows to run CamelJBang with `camel` as shown below.
+
+=== Setup Opensearch
+
+We are going to use the official Docker image for Opensearch.
+
+We can run the following:
+
+[source,sh]
+----
+docker run -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" --name opensearch-node -e "DISABLE_SECURITY_PLUGIN=true" -d opensearchproject/opensearch:2.9.0
+----
+
+To verify the connection is fine run the following command:
+
+[source,sh]
+----
+[oscerd@ghost timer-opensearch-index]$ curl -X GET "http://localhost:9200/"
+{
+ "name" : "0f14d266d0bb",
+ "cluster_name" : "docker-cluster",
+ "cluster_uuid" : "w_Qoavz0SKuYcj4onSoWdw",
+ "version" : {
+ "distribution" : "opensearch",
+ "number" : "2.9.0",
+ "build_type" : "tar",
+ "build_hash" : "1164221ee2b8ba3560f0ff492309867beea28433",
+ "build_date" : "2023-07-18T21:23:29.367080729Z",
+ "build_snapshot" : false,
+ "lucene_version" : "9.7.0",
+ "minimum_wire_compatibility_version" : "7.10.0",
+ "minimum_index_compatibility_version" : "7.0.0"
+ },
+ "tagline" : "The OpenSearch Project: https://opensearch.org/"
+}
+
+----
+
+=== How to run
+
+Then you can run this example using:
+
+[source,sh]
+----
+$ jbang run --fresh camel@apache/camel run --local-kamelet-dir=<path_to_local_kamelet_dir> timer-os-index.yaml
+----
+
+=== Developer Web Console
+
+You can enable the developer console via `--console` flag as show:
+
+[source,sh]
+----
+$ jbang run --fresh camel@apache/camel run --local-kamelet-dir=<path_to_local_kamelet_dir> timer-os-index.yaml --console
+----
+
+Then you can browse: http://localhost:8080/q/dev to introspect the running Camel applicaton.
+
+=== Running sink for populating index
+
+We are going to insert just one record into the index.
+
+[source,sh]
+----
+[jbang] Resolving dependencies...
+[jbang] org.apache.camel:camel-bom:4.0.0@pom
+[jbang] org.apache.camel:camel-jbang-core:4.0.0
+[jbang] org.apache.camel.kamelets:camel-kamelets:4.0.0-RC1
+[jbang] Dependencies resolved
+[jbang] Building jar...
+2023-08-25 17:09:13.246 INFO 87439 --- [ main] org.apache.camel.main.MainSupport : Apache Camel (JBang) 4.0.0 is starting
+2023-08-25 17:09:13.445 INFO 87439 --- [ main] org.apache.camel.main.MainSupport : Using Java 17.0.8 with PID 87439. Started by oscerd in /home/oscerd/workspace/apache-camel/camel-kamelets-examples/jbang/timer-opensearch-index
+2023-08-25 17:09:14.158 INFO 87439 --- [ main] mel.cli.connector.LocalCliConnector : Camel CLI enabled (local)
+2023-08-25 17:09:15.532 INFO 87439 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 4.0.0 (timer-os-index) is starting
+2023-08-25 17:09:15.747 INFO 87439 --- [ main] .core.spi.resolver.ResolverProvider : Using the default address resolver as the dns resolver could not be loaded
+2023-08-25 17:09:15.870 INFO 87439 --- [ntloop-thread-0] .http.vertx.VertxPlatformHttpServer : Vert.x HttpServer started on 0.0.0.0:8080
+2023-08-25 17:09:16.068 INFO 87439 --- [ main] onent.opensearch.OpensearchProducer : Connecting to the OpenSearch cluster: opensearch-cluster
+2023-08-25 17:09:16.305 INFO 87439 --- [ main] g.apache.camel.main.BaseMainSupport : Property-placeholders summary
+2023-08-25 17:09:16.307 INFO 87439 --- [ main] g.apache.camel.main.BaseMainSupport : [timer-source.kamelet.yaml] repeatCount=1
+2023-08-25 17:09:16.307 INFO 87439 --- [ main] g.apache.camel.main.BaseMainSupport : [timer-source.kamelet.yaml] message={"id":"2","message":"Camel Rocks"}
+2023-08-25 17:09:16.307 INFO 87439 --- [ main] g.apache.camel.main.BaseMainSupport : [timer-source.kamelet.yaml] contentType=application/json
+2023-08-25 17:09:16.307 INFO 87439 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] local-opensearch=local-opensearch-1
+2023-08-25 17:09:16.307 INFO 87439 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] clusterName=opensearch-cluster
+2023-08-25 17:09:16.308 INFO 87439 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] indexName=test_index
+2023-08-25 17:09:16.308 INFO 87439 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] hostAddresses=localhost
+2023-08-25 17:09:16.308 INFO 87439 --- [ main] g.apache.camel.main.BaseMainSupport : [OS Environment Variable] user=xxxxxx
+2023-08-25 17:09:16.329 INFO 87439 --- [ main] el.impl.engine.AbstractCamelContext : Routes startup (started:3)
+2023-08-25 17:09:16.330 INFO 87439 --- [ main] el.impl.engine.AbstractCamelContext : Started route1 (kamelet://timer-source)
+2023-08-25 17:09:16.331 INFO 87439 --- [ main] el.impl.engine.AbstractCamelContext : Started timer-source-1 (timer://tick)
+2023-08-25 17:09:16.331 INFO 87439 --- [ main] el.impl.engine.AbstractCamelContext : Started opensearch-index-sink-2 (kamelet://source)
+2023-08-25 17:09:16.331 INFO 87439 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 4.0.0 (timer-os-index) started in 798ms (build:0ms init:0ms start:798ms)
+----
+
+=== Verify the sink worked
+
+To verify everything is fine we should run the following command
+
+[source,sh]
+----
+$ curl http://localhost:9200/test_index/_search
+{"took":1,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":1,"relation":"eq"},"max_score":1.0,"hits":[{"_index":"test_index","_id":"sUM-LYoBQGW75SJUE41U","_score":1.0,"_source":{"id":"1","message":"Camel Rocks"}}]}}
+----
+
+=== How to run the source
+
+Then you can run this example using:
+
+[source,sh]
+----
+$ jbang run --fresh camel@apache/camel run --local-kamelet-dir=<path_to_local_kamelet_dir> os-search-log.yaml
+----
+
+=== Developer Web Console
+
+You can enable the developer console via `--console` flag as show:
+
+[source,sh]
+----
+$ jbang run --fresh camel@apache/camel run --local-kamelet-dir=<path_to_local_kamelet_dir> os-search-log.yaml --console
+----
+
+=== Running source for searching index
+
+We are going to search for one record into the index with id '1'.
+
+[source,sh]
+----
+[jbang] Resolving dependencies...
+[jbang] org.apache.camel:camel-bom:4.0.0@pom
+[jbang] org.apache.camel:camel-jbang-core:4.0.0
+[jbang] org.apache.camel.kamelets:camel-kamelets:4.0.0-RC1
+[jbang] Dependencies resolved
+[jbang] Building jar...
+2023-08-28 08:08:17.916 INFO 37447 --- [ main] org.apache.camel.main.MainSupport : Apache Camel (JBang) 4.0.0 is starting
+2023-08-28 08:08:18.100 INFO 37447 --- [ main] org.apache.camel.main.MainSupport : Using Java 17.0.8 with PID 37447. Started by oscerd in /home/oscerd/workspace/apache-camel/camel-kamelets-examples/jbang/opensearch-search-log
+2023-08-28 08:08:18.759 INFO 37447 --- [ main] mel.cli.connector.LocalCliConnector : Camel CLI enabled (local)
+2023-08-28 08:08:20.078 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 4.0.0 (os-search-log) is starting
+2023-08-28 08:08:20.252 INFO 37447 --- [ main] .core.spi.resolver.ResolverProvider : Using the default address resolver as the dns resolver could not be loaded
+2023-08-28 08:08:20.336 INFO 37447 --- [ntloop-thread-0] .http.vertx.VertxPlatformHttpServer : Vert.x HttpServer started on 0.0.0.0:8080
+2023-08-28 08:08:20.375 INFO 37447 --- [ main] onent.opensearch.OpensearchProducer : Connecting to the OpenSearch cluster: opensearch-cluster
+2023-08-28 08:08:20.607 INFO 37447 --- [ main] g.apache.camel.main.BaseMainSupport : Property-placeholders summary
+2023-08-28 08:08:20.607 INFO 37447 --- [ main] g.apache.camel.main.BaseMainSupport : [h-search-source.kamelet.yaml] query={ "query": { "match": { "id": "1" } } }
+2023-08-28 08:08:20.607 INFO 37447 --- [ main] g.apache.camel.main.BaseMainSupport : [h-search-source.kamelet.yaml] local-opensearch=local-opensearch-1
+2023-08-28 08:08:20.608 INFO 37447 --- [ main] g.apache.camel.main.BaseMainSupport : [h-search-source.kamelet.yaml] clusterName=opensearch-cluster
+2023-08-28 08:08:20.608 INFO 37447 --- [ main] g.apache.camel.main.BaseMainSupport : [h-search-source.kamelet.yaml] indexName=test_index
+2023-08-28 08:08:20.608 INFO 37447 --- [ main] g.apache.camel.main.BaseMainSupport : [h-search-source.kamelet.yaml] hostAddresses=localhost
+2023-08-28 08:08:20.608 INFO 37447 --- [ main] g.apache.camel.main.BaseMainSupport : [OS Environment Variable] user=xxxxxx
+2023-08-28 08:08:20.629 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Routes startup (started:3)
+2023-08-28 08:08:20.630 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Started route1 (kamelet://opensearch-search-source)
+2023-08-28 08:08:20.630 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Started opensearch-search-source-1 (timer://tick)
+2023-08-28 08:08:20.630 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Started log-sink-2 (kamelet://source)
+2023-08-28 08:08:20.630 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 4.0.0 (os-search-log) started in 551ms (build:0ms init:0ms start:551ms)
+2023-08-28 08:08:20.632 INFO 37447 --- [ main] t.platform.http.main.MainHttpServer : HTTP endpoints summary
+2023-08-28 08:08:20.637 INFO 37447 --- [ main] t.platform.http.main.MainHttpServer : http://0.0.0.0:8080/q/dev
+2023-08-28 08:08:22.002 INFO 37447 --- [/O dispatcher 1] log-sink : Exchange[ExchangePattern: InOnly, BodyType: org.apache.camel.converter.stream.InputStreamCache, Body: {"total":{"relation":"Eq","value":2},"hits":[{"index":"test_index","id":"YpDBOooBhbqWEogQAwfy","score":0.18232156,"fields":{},"highlight":{},"innerHits":{},"matchedQueries":[],"ignored":[],"source":{"_children":{"id":{"_value":"1"},"message":{"_value":"Camel Rocks"}},"_nodeFactory":{"_cfgBigDe [...]
+2023-08-28 08:08:22.624 INFO 37447 --- [/O dispatcher 1] log-sink : Exchange[ExchangePattern: InOnly, BodyType: org.apache.camel.converter.stream.InputStreamCache, Body: {"total":{"relation":"Eq","value":2},"hits":[{"index":"test_index","id":"YpDBOooBhbqWEogQAwfy","score":0.18232156,"fields":{},"highlight":{},"innerHits":{},"matchedQueries":[],"ignored":[],"source":{"_children":{"id":{"_value":"1"},"message":{"_value":"Camel Rocks"}},"_nodeFactory":{"_cfgBigDe [...]
+^C2023-08-28 08:08:23.222 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 4.0.0 (os-search-log) is shutting down (timeout:10s)
+2023-08-28 08:08:23.232 INFO 37447 --- [ - ShutdownTask] onent.opensearch.OpensearchProducer : Disconnecting from OpenSearch cluster: opensearch-cluster
+2023-08-28 08:08:23.247 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Routes stopped (stopped:3)
+2023-08-28 08:08:23.248 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Stopped log-sink-2 (kamelet://source)
+2023-08-28 08:08:23.248 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Stopped opensearch-search-source-1 (timer://tick)
+2023-08-28 08:08:23.249 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Stopped route1 (kamelet://opensearch-search-source)
+2023-08-28 08:08:23.273 INFO 37447 --- [ntloop-thread-0] .http.vertx.VertxPlatformHttpServer : Vert.x HttpServer stopped
+2023-08-28 08:08:23.280 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 4.0.0 (os-search-log) shutdown in 57ms (uptime:3s)
+2023-08-28 08:08:23.280 INFO 37447 --- [ main] org.apache.camel.main.MainSupport : Apache Camel (JBang) 4.0.0 shutdown
+----
+
+=== Help and contributions
+
+If you hit any problem using Camel or have some feedback, then please
+https://camel.apache.org/community/support/[let us know].
+
+We also love contributors, so
+https://camel.apache.org/community/contributing/[get involved] :-)
+
+The Camel riders!
diff --git a/jbang/opensearch-search-log/os-search-log.yaml b/jbang/opensearch-search-log/os-search-log.yaml
new file mode 100644
index 0000000..1f016b1
--- /dev/null
+++ b/jbang/opensearch-search-log/os-search-log.yaml
@@ -0,0 +1,34 @@
+## ---------------------------------------------------------------------------
+## Licensed to the Apache Software Foundation (ASF) under one or more
+## contributor license agreements. See the NOTICE file distributed with
+## this work for additional information regarding copyright ownership.
+## The ASF licenses this file to You under the Apache License, Version 2.0
+## (the "License"); you may not use this file except in compliance with
+## the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing, software
+## distributed under the License is distributed on an "AS IS" BASIS,
+## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+## See the License for the specific language governing permissions and
+## limitations under the License.
+## ---------------------------------------------------------------------------
+
+- route:
+ from:
+ uri: "kamelet:opensearch-search-source"
+ parameters:
+ hostAddresses: "localhost"
+ clusterName: "opensearch-cluster"
+ indexName: "test_index"
+ query: '{
+ "query": {
+ "match": {
+ "id": "1"
+ }
+ }
+ }'
+ steps:
+ - to:
+ uri: "kamelet:log-sink"
diff --git a/jbang/opensearch-search-log/timer-os-index.yaml b/jbang/opensearch-search-log/timer-os-index.yaml
new file mode 100644
index 0000000..02c2f06
--- /dev/null
+++ b/jbang/opensearch-search-log/timer-os-index.yaml
@@ -0,0 +1,31 @@
+## ---------------------------------------------------------------------------
+## Licensed to the Apache Software Foundation (ASF) under one or more
+## contributor license agreements. See the NOTICE file distributed with
+## this work for additional information regarding copyright ownership.
+## The ASF licenses this file to You under the Apache License, Version 2.0
+## (the "License"); you may not use this file except in compliance with
+## the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing, software
+## distributed under the License is distributed on an "AS IS" BASIS,
+## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+## See the License for the specific language governing permissions and
+## limitations under the License.
+## ---------------------------------------------------------------------------
+
+- route:
+ from:
+ uri: "kamelet:timer-source"
+ parameters:
+ message: '{"id":"1","message":"Camel Rocks"}'
+ contentType: "application/json"
+ repeatCount: 1
+ steps:
+ - to:
+ uri: "kamelet:opensearch-index-sink"
+ parameters:
+ hostAddresses: "localhost"
+ clusterName: "opensearch-cluster"
+ indexName: "test_index"