You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by ac...@apache.org on 2023/08/28 06:12:05 UTC
[camel-kamelets-examples] branch os-search created (now 789192a)
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a change to branch os-search
in repository https://gitbox.apache.org/repos/asf/camel-kamelets-examples.git
at 789192a Added Opensearch Search Example
This branch includes the following new commits:
new 789192a Added Opensearch Search Example
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[camel-kamelets-examples] 01/01: Added Opensearch Search Example
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch os-search
in repository https://gitbox.apache.org/repos/asf/camel-kamelets-examples.git
commit 789192a1e7c97e660086498df62c531bd11bb662
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Mon Aug 28 08:11:41 2023 +0200
Added Opensearch Search Example
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
jbang/opensearch-search-log/README.adoc | 200 ++++++++++++++++++++++++
jbang/opensearch-search-log/os-search-log.yaml | 34 ++++
jbang/opensearch-search-log/timer-os-index.yaml | 31 ++++
3 files changed, 265 insertions(+)
diff --git a/jbang/opensearch-search-log/README.adoc b/jbang/opensearch-search-log/README.adoc
new file mode 100644
index 0000000..7995bc1
--- /dev/null
+++ b/jbang/opensearch-search-log/README.adoc
@@ -0,0 +1,200 @@
+== Timer to Opensearch Search Source
+
+In this sample you'll use the Opensearch Index Sink Kamelet and Opensearch Search Source Kamelet based on camel-opensearch component.
+
+=== Install JBang
+
+First install JBang according to https://www.jbang.dev
+
+When JBang is installed then you should be able to run from a shell:
+
+[source,sh]
+----
+$ jbang --version
+----
+
+This will output the version of JBang.
+
+To run this example you can either install Camel on JBang via:
+
+[source,sh]
+----
+$ jbang app install camel@apache/camel
+----
+
+Which allows to run CamelJBang with `camel` as shown below.
+
+=== Setup Opensearch
+
+We are going to use the official Docker image for Opensearch.
+
+We can run the following:
+
+[source,sh]
+----
+docker run -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" --name opensearch-node -e "DISABLE_SECURITY_PLUGIN=true" -d opensearchproject/opensearch:2.9.0
+----
+
+To verify the connection is fine run the following command:
+
+[source,sh]
+----
+[oscerd@ghost timer-opensearch-index]$ curl -X GET "http://localhost:9200/"
+{
+ "name" : "0f14d266d0bb",
+ "cluster_name" : "docker-cluster",
+ "cluster_uuid" : "w_Qoavz0SKuYcj4onSoWdw",
+ "version" : {
+ "distribution" : "opensearch",
+ "number" : "2.9.0",
+ "build_type" : "tar",
+ "build_hash" : "1164221ee2b8ba3560f0ff492309867beea28433",
+ "build_date" : "2023-07-18T21:23:29.367080729Z",
+ "build_snapshot" : false,
+ "lucene_version" : "9.7.0",
+ "minimum_wire_compatibility_version" : "7.10.0",
+ "minimum_index_compatibility_version" : "7.0.0"
+ },
+ "tagline" : "The OpenSearch Project: https://opensearch.org/"
+}
+
+----
+
+=== How to run
+
+Then you can run this example using:
+
+[source,sh]
+----
+$ jbang run --fresh camel@apache/camel run --local-kamelet-dir=<path_to_local_kamelet_dir> timer-os-index.yaml
+----
+
+=== Developer Web Console
+
+You can enable the developer console via `--console` flag as show:
+
+[source,sh]
+----
+$ jbang run --fresh camel@apache/camel run --local-kamelet-dir=<path_to_local_kamelet_dir> timer-os-index.yaml --console
+----
+
+Then you can browse: http://localhost:8080/q/dev to introspect the running Camel applicaton.
+
+=== Running sink for populating index
+
+We are going to insert just one record into the index.
+
+[source,sh]
+----
+[jbang] Resolving dependencies...
+[jbang] org.apache.camel:camel-bom:4.0.0@pom
+[jbang] org.apache.camel:camel-jbang-core:4.0.0
+[jbang] org.apache.camel.kamelets:camel-kamelets:4.0.0-RC1
+[jbang] Dependencies resolved
+[jbang] Building jar...
+2023-08-25 17:09:13.246 INFO 87439 --- [ main] org.apache.camel.main.MainSupport : Apache Camel (JBang) 4.0.0 is starting
+2023-08-25 17:09:13.445 INFO 87439 --- [ main] org.apache.camel.main.MainSupport : Using Java 17.0.8 with PID 87439. Started by oscerd in /home/oscerd/workspace/apache-camel/camel-kamelets-examples/jbang/timer-opensearch-index
+2023-08-25 17:09:14.158 INFO 87439 --- [ main] mel.cli.connector.LocalCliConnector : Camel CLI enabled (local)
+2023-08-25 17:09:15.532 INFO 87439 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 4.0.0 (timer-os-index) is starting
+2023-08-25 17:09:15.747 INFO 87439 --- [ main] .core.spi.resolver.ResolverProvider : Using the default address resolver as the dns resolver could not be loaded
+2023-08-25 17:09:15.870 INFO 87439 --- [ntloop-thread-0] .http.vertx.VertxPlatformHttpServer : Vert.x HttpServer started on 0.0.0.0:8080
+2023-08-25 17:09:16.068 INFO 87439 --- [ main] onent.opensearch.OpensearchProducer : Connecting to the OpenSearch cluster: opensearch-cluster
+2023-08-25 17:09:16.305 INFO 87439 --- [ main] g.apache.camel.main.BaseMainSupport : Property-placeholders summary
+2023-08-25 17:09:16.307 INFO 87439 --- [ main] g.apache.camel.main.BaseMainSupport : [timer-source.kamelet.yaml] repeatCount=1
+2023-08-25 17:09:16.307 INFO 87439 --- [ main] g.apache.camel.main.BaseMainSupport : [timer-source.kamelet.yaml] message={"id":"2","message":"Camel Rocks"}
+2023-08-25 17:09:16.307 INFO 87439 --- [ main] g.apache.camel.main.BaseMainSupport : [timer-source.kamelet.yaml] contentType=application/json
+2023-08-25 17:09:16.307 INFO 87439 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] local-opensearch=local-opensearch-1
+2023-08-25 17:09:16.307 INFO 87439 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] clusterName=opensearch-cluster
+2023-08-25 17:09:16.308 INFO 87439 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] indexName=test_index
+2023-08-25 17:09:16.308 INFO 87439 --- [ main] g.apache.camel.main.BaseMainSupport : [arch-index-sink.kamelet.yaml] hostAddresses=localhost
+2023-08-25 17:09:16.308 INFO 87439 --- [ main] g.apache.camel.main.BaseMainSupport : [OS Environment Variable] user=xxxxxx
+2023-08-25 17:09:16.329 INFO 87439 --- [ main] el.impl.engine.AbstractCamelContext : Routes startup (started:3)
+2023-08-25 17:09:16.330 INFO 87439 --- [ main] el.impl.engine.AbstractCamelContext : Started route1 (kamelet://timer-source)
+2023-08-25 17:09:16.331 INFO 87439 --- [ main] el.impl.engine.AbstractCamelContext : Started timer-source-1 (timer://tick)
+2023-08-25 17:09:16.331 INFO 87439 --- [ main] el.impl.engine.AbstractCamelContext : Started opensearch-index-sink-2 (kamelet://source)
+2023-08-25 17:09:16.331 INFO 87439 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 4.0.0 (timer-os-index) started in 798ms (build:0ms init:0ms start:798ms)
+----
+
+=== Verify the sink worked
+
+To verify everything is fine we should run the following command
+
+[source,sh]
+----
+$ curl http://localhost:9200/test_index/_search
+{"took":1,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":1,"relation":"eq"},"max_score":1.0,"hits":[{"_index":"test_index","_id":"sUM-LYoBQGW75SJUE41U","_score":1.0,"_source":{"id":"1","message":"Camel Rocks"}}]}}
+----
+
+=== How to run the source
+
+Then you can run this example using:
+
+[source,sh]
+----
+$ jbang run --fresh camel@apache/camel run --local-kamelet-dir=<path_to_local_kamelet_dir> os-search-log.yaml
+----
+
+=== Developer Web Console
+
+You can enable the developer console via `--console` flag as show:
+
+[source,sh]
+----
+$ jbang run --fresh camel@apache/camel run --local-kamelet-dir=<path_to_local_kamelet_dir> os-search-log.yaml --console
+----
+
+=== Running source for searching index
+
+We are going to search for one record into the index with id '1'.
+
+[source,sh]
+----
+[jbang] Resolving dependencies...
+[jbang] org.apache.camel:camel-bom:4.0.0@pom
+[jbang] org.apache.camel:camel-jbang-core:4.0.0
+[jbang] org.apache.camel.kamelets:camel-kamelets:4.0.0-RC1
+[jbang] Dependencies resolved
+[jbang] Building jar...
+2023-08-28 08:08:17.916 INFO 37447 --- [ main] org.apache.camel.main.MainSupport : Apache Camel (JBang) 4.0.0 is starting
+2023-08-28 08:08:18.100 INFO 37447 --- [ main] org.apache.camel.main.MainSupport : Using Java 17.0.8 with PID 37447. Started by oscerd in /home/oscerd/workspace/apache-camel/camel-kamelets-examples/jbang/opensearch-search-log
+2023-08-28 08:08:18.759 INFO 37447 --- [ main] mel.cli.connector.LocalCliConnector : Camel CLI enabled (local)
+2023-08-28 08:08:20.078 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 4.0.0 (os-search-log) is starting
+2023-08-28 08:08:20.252 INFO 37447 --- [ main] .core.spi.resolver.ResolverProvider : Using the default address resolver as the dns resolver could not be loaded
+2023-08-28 08:08:20.336 INFO 37447 --- [ntloop-thread-0] .http.vertx.VertxPlatformHttpServer : Vert.x HttpServer started on 0.0.0.0:8080
+2023-08-28 08:08:20.375 INFO 37447 --- [ main] onent.opensearch.OpensearchProducer : Connecting to the OpenSearch cluster: opensearch-cluster
+2023-08-28 08:08:20.607 INFO 37447 --- [ main] g.apache.camel.main.BaseMainSupport : Property-placeholders summary
+2023-08-28 08:08:20.607 INFO 37447 --- [ main] g.apache.camel.main.BaseMainSupport : [h-search-source.kamelet.yaml] query={ "query": { "match": { "id": "1" } } }
+2023-08-28 08:08:20.607 INFO 37447 --- [ main] g.apache.camel.main.BaseMainSupport : [h-search-source.kamelet.yaml] local-opensearch=local-opensearch-1
+2023-08-28 08:08:20.608 INFO 37447 --- [ main] g.apache.camel.main.BaseMainSupport : [h-search-source.kamelet.yaml] clusterName=opensearch-cluster
+2023-08-28 08:08:20.608 INFO 37447 --- [ main] g.apache.camel.main.BaseMainSupport : [h-search-source.kamelet.yaml] indexName=test_index
+2023-08-28 08:08:20.608 INFO 37447 --- [ main] g.apache.camel.main.BaseMainSupport : [h-search-source.kamelet.yaml] hostAddresses=localhost
+2023-08-28 08:08:20.608 INFO 37447 --- [ main] g.apache.camel.main.BaseMainSupport : [OS Environment Variable] user=xxxxxx
+2023-08-28 08:08:20.629 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Routes startup (started:3)
+2023-08-28 08:08:20.630 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Started route1 (kamelet://opensearch-search-source)
+2023-08-28 08:08:20.630 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Started opensearch-search-source-1 (timer://tick)
+2023-08-28 08:08:20.630 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Started log-sink-2 (kamelet://source)
+2023-08-28 08:08:20.630 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 4.0.0 (os-search-log) started in 551ms (build:0ms init:0ms start:551ms)
+2023-08-28 08:08:20.632 INFO 37447 --- [ main] t.platform.http.main.MainHttpServer : HTTP endpoints summary
+2023-08-28 08:08:20.637 INFO 37447 --- [ main] t.platform.http.main.MainHttpServer : http://0.0.0.0:8080/q/dev
+2023-08-28 08:08:22.002 INFO 37447 --- [/O dispatcher 1] log-sink : Exchange[ExchangePattern: InOnly, BodyType: org.apache.camel.converter.stream.InputStreamCache, Body: {"total":{"relation":"Eq","value":2},"hits":[{"index":"test_index","id":"YpDBOooBhbqWEogQAwfy","score":0.18232156,"fields":{},"highlight":{},"innerHits":{},"matchedQueries":[],"ignored":[],"source":{"_children":{"id":{"_value":"1"},"message":{"_value":"Camel Rocks"}},"_nodeFactory":{"_cfgBigDe [...]
+2023-08-28 08:08:22.624 INFO 37447 --- [/O dispatcher 1] log-sink : Exchange[ExchangePattern: InOnly, BodyType: org.apache.camel.converter.stream.InputStreamCache, Body: {"total":{"relation":"Eq","value":2},"hits":[{"index":"test_index","id":"YpDBOooBhbqWEogQAwfy","score":0.18232156,"fields":{},"highlight":{},"innerHits":{},"matchedQueries":[],"ignored":[],"source":{"_children":{"id":{"_value":"1"},"message":{"_value":"Camel Rocks"}},"_nodeFactory":{"_cfgBigDe [...]
+^C2023-08-28 08:08:23.222 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 4.0.0 (os-search-log) is shutting down (timeout:10s)
+2023-08-28 08:08:23.232 INFO 37447 --- [ - ShutdownTask] onent.opensearch.OpensearchProducer : Disconnecting from OpenSearch cluster: opensearch-cluster
+2023-08-28 08:08:23.247 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Routes stopped (stopped:3)
+2023-08-28 08:08:23.248 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Stopped log-sink-2 (kamelet://source)
+2023-08-28 08:08:23.248 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Stopped opensearch-search-source-1 (timer://tick)
+2023-08-28 08:08:23.249 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Stopped route1 (kamelet://opensearch-search-source)
+2023-08-28 08:08:23.273 INFO 37447 --- [ntloop-thread-0] .http.vertx.VertxPlatformHttpServer : Vert.x HttpServer stopped
+2023-08-28 08:08:23.280 INFO 37447 --- [ main] el.impl.engine.AbstractCamelContext : Apache Camel 4.0.0 (os-search-log) shutdown in 57ms (uptime:3s)
+2023-08-28 08:08:23.280 INFO 37447 --- [ main] org.apache.camel.main.MainSupport : Apache Camel (JBang) 4.0.0 shutdown
+----
+
+=== Help and contributions
+
+If you hit any problem using Camel or have some feedback, then please
+https://camel.apache.org/community/support/[let us know].
+
+We also love contributors, so
+https://camel.apache.org/community/contributing/[get involved] :-)
+
+The Camel riders!
diff --git a/jbang/opensearch-search-log/os-search-log.yaml b/jbang/opensearch-search-log/os-search-log.yaml
new file mode 100644
index 0000000..1f016b1
--- /dev/null
+++ b/jbang/opensearch-search-log/os-search-log.yaml
@@ -0,0 +1,34 @@
+## ---------------------------------------------------------------------------
+## Licensed to the Apache Software Foundation (ASF) under one or more
+## contributor license agreements. See the NOTICE file distributed with
+## this work for additional information regarding copyright ownership.
+## The ASF licenses this file to You under the Apache License, Version 2.0
+## (the "License"); you may not use this file except in compliance with
+## the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing, software
+## distributed under the License is distributed on an "AS IS" BASIS,
+## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+## See the License for the specific language governing permissions and
+## limitations under the License.
+## ---------------------------------------------------------------------------
+
+- route:
+ from:
+ uri: "kamelet:opensearch-search-source"
+ parameters:
+ hostAddresses: "localhost"
+ clusterName: "opensearch-cluster"
+ indexName: "test_index"
+ query: '{
+ "query": {
+ "match": {
+ "id": "1"
+ }
+ }
+ }'
+ steps:
+ - to:
+ uri: "kamelet:log-sink"
diff --git a/jbang/opensearch-search-log/timer-os-index.yaml b/jbang/opensearch-search-log/timer-os-index.yaml
new file mode 100644
index 0000000..02c2f06
--- /dev/null
+++ b/jbang/opensearch-search-log/timer-os-index.yaml
@@ -0,0 +1,31 @@
+## ---------------------------------------------------------------------------
+## Licensed to the Apache Software Foundation (ASF) under one or more
+## contributor license agreements. See the NOTICE file distributed with
+## this work for additional information regarding copyright ownership.
+## The ASF licenses this file to You under the Apache License, Version 2.0
+## (the "License"); you may not use this file except in compliance with
+## the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing, software
+## distributed under the License is distributed on an "AS IS" BASIS,
+## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+## See the License for the specific language governing permissions and
+## limitations under the License.
+## ---------------------------------------------------------------------------
+
+- route:
+ from:
+ uri: "kamelet:timer-source"
+ parameters:
+ message: '{"id":"1","message":"Camel Rocks"}'
+ contentType: "application/json"
+ repeatCount: 1
+ steps:
+ - to:
+ uri: "kamelet:opensearch-index-sink"
+ parameters:
+ hostAddresses: "localhost"
+ clusterName: "opensearch-cluster"
+ indexName: "test_index"