You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by pc...@apache.org on 2023/03/09 15:10:31 UTC

[camel-website] 01/02: feat: Camel-k telemetry

This is an automated email from the ASF dual-hosted git repository.

pcongiusti pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-website.git

commit 73c131d8c8efc7bd339e0efdc8e107b2d8e25150
Author: Gaelle Fournier <ga...@gmail.com>
AuthorDate: Mon Mar 6 11:21:53 2023 +0100

    feat: Camel-k telemetry
---
 .../compare_tracing_telemetry.png                  | Bin 0 -> 79994 bytes
 .../blog/2023/03/camel-k-telemetry/featured.png    | Bin 0 -> 191399 bytes
 .../blog/2023/03/camel-k-telemetry/full_trace.png  | Bin 0 -> 68084 bytes
 content/blog/2023/03/camel-k-telemetry/index.md    | 420 +++++++++++++++++++++
 .../2023/03/camel-k-telemetry/inventory_traces.png | Bin 0 -> 158586 bytes
 5 files changed, 420 insertions(+)

diff --git a/content/blog/2023/03/camel-k-telemetry/compare_tracing_telemetry.png b/content/blog/2023/03/camel-k-telemetry/compare_tracing_telemetry.png
new file mode 100644
index 00000000..a6910339
Binary files /dev/null and b/content/blog/2023/03/camel-k-telemetry/compare_tracing_telemetry.png differ
diff --git a/content/blog/2023/03/camel-k-telemetry/featured.png b/content/blog/2023/03/camel-k-telemetry/featured.png
new file mode 100644
index 00000000..0feb4b4b
Binary files /dev/null and b/content/blog/2023/03/camel-k-telemetry/featured.png differ
diff --git a/content/blog/2023/03/camel-k-telemetry/full_trace.png b/content/blog/2023/03/camel-k-telemetry/full_trace.png
new file mode 100644
index 00000000..6b1ffac8
Binary files /dev/null and b/content/blog/2023/03/camel-k-telemetry/full_trace.png differ
diff --git a/content/blog/2023/03/camel-k-telemetry/index.md b/content/blog/2023/03/camel-k-telemetry/index.md
new file mode 100644
index 00000000..c425aebf
--- /dev/null
+++ b/content/blog/2023/03/camel-k-telemetry/index.md
@@ -0,0 +1,420 @@
+---
+title: "Camel K Observability: Distributed Tracing"
+date: 2023-03-25
+draft: false
+authors: [gansheer]
+categories: ["Camel K", "Features"]
+preview: "Camel K: how to configure Distributed Tracing for an Integration"
+---
+
+Tracing is an important approach for controlling and monitoring the experience of users, it allows us to gather more information about an integration's performance. 
+
+Camel K has long been providing support for distributed tracing using OpenTracing. At the beginning of 2022, the [CNCF](https://www.cncf.io) announced that there were [archiving the OpenTracing project](https://www.cncf.io/blog/2022/01/31/cncf-archives-the-opentracing-project/) in favor of the [OpenTelemetry project](https://opentelemetry.io/). OpenTelemetry is the latest solution created by merging OpenTracing and OpenCensus. As a result, we decided in Camel K 1.12 to introduce the [`te [...]
+
+I'll walk you through the configurations needed to support Distributed tracing for your camel integrations in this blog post.
+
+## Distributed Tracing tools
+
+OpenTelemetry defines the [OpenTelemetry Protocol (OTLP)](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/otlp.md) that allows any Distributed Tracing tool that provides a OTLP compatible collector API to be used. While OpenTelemetry is still CNCF incubating project, its has become an industry-standard and is now natively supported by many Distributed Tracings tools.
+
+The `telemetry` trait has been tested on Jaeger (v1.35+), OpenTelemetry Collector and Grafana Tempo to this day.
+
+We will be using Jaeger for this blog post manipulations.
+
+## Camel K 
+
+The `telemetry` trait is available starting from Camel K version 1.12, so you should first check your kamel client version
+
+```sh
+$ kamel version
+Camel K Client 1.12.0
+```
+
+To install the Camel K operator just execute
+```sh
+$ kamel install
+```
+
+Check if the Camel K operator is available
+```sh
+$ kubectl get pods -l name=camel-k-operator
+NAME                                READY   STATUS    RESTARTS   AGE
+camel-k-operator-5b897ddcdd-4qwqb   1/1     Running   0          50s
+```
+
+For more informations see the [documentation](https://camel.apache.org/camel-k/1.12.x/installation/installation.html)
+
+## Install Jaeger
+
+For this installation, you should firt ensure that ingress is available on your kubernetes. If you are working on minikube `minikube addons enable ingress` should be enough.
+
+If Jaeger is not already available on your kubernetes instance, an easy way to install it on a kubernetes is to install the Jaeger operator (see https://www.jaegertracing.io/docs).
+
+In short, to install the Jaeger operator you should:
+
+* Install cert-manager if it is not installed as it is required 
+```sh
+ # The cert manager is a pre-requise to the Jaeger operator (see https://cert-manager.io/docs/installation/)
+$ kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.10.1/cert-manager.yaml
+```
+Wait 1 or 2 minutes, we want to ensure the cert-manager has had enough time to be fully deployed
+
+```sh
+ # see https://www.jaegertracing.io/docs/1.42/operator/#installing-the-operator-on-kubernetes
+$ kubectl create namespace observability
+$ kubectl create -f https://github.com/jaegertracing/jaeger-operator/releases/download/v1.42.0/jaeger-operator.yaml -n observability
+```
+
+Check if Jaeger-operator is available:
+```sh
+$ kubectl get deployment jaeger-operator -n observability
+
+NAME              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
+jaeger-operator   1         1         1            1           48s
+```
+
+To use Jaeger, you can install the AllInOne image:
+
+```sh
+$ kubectl apply -f - << EOF
+apiVersion: jaegertracing.io/v1
+kind: Jaeger
+metadata:
+  name: instance
+EOF
+```
+
+Check the presence of the Jaeger instance
+
+```sh
+$ kubectl get jaeger
+NAME       STATUS    VERSION   STRATEGY   STORAGE   AGE
+instance   Running   1.42.0    allinone   memory    8s
+```
+
+
+## Run some Camel K Integrations with Telemetry trait
+
+Now that we have Jaeger available on the Kubernetes cluster, the next step is to run some integrations in a Kubernetes cluster where Camel K is installed.
+You can find the source code in the [Camel K example github repository](https://github.com/apache/camel-k-examples/tree/main/generic-examples/traits/telemetry).
+
+We will be creating two integration routes as the distributed services:
+
+* *Inventory* which is also a rest service in charge of product's inventory management
+
+**`InventoryService.java`**
+```java 
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.model.rest.RestBindingMode;
+import org.apache.camel.component.jackson.JacksonDataFormat;
+import java.text.SimpleDateFormat;
+import org.apache.camel.Exchange;
+import java.util.Date;
+import java.util.Map;
+
+
+public class InventoryService extends RouteBuilder {
+
+    
+    @Override
+    public void configure() throws Exception {
+        restConfiguration()
+            .enableCORS(true)
+            .bindingMode(RestBindingMode.json);
+
+        rest()
+            .post("/notify/order/place")
+                .to("direct:notify");
+
+        
+        JacksonDataFormat invDataFormat = new JacksonDataFormat();
+        invDataFormat.setUnmarshalType(InventoryNotification.class);
+
+        from("direct:notify")
+            .log("notifyorder--> ${body}")
+            .setHeader(Exchange.CONTENT_TYPE, constant("application/json"))
+            .bean(InventoryNotification.class, "getInventoryNotification(${body['orderId']},${body['itemId']},${body['quantity']} )")
+            .marshal(invDataFormat)
+            .log("Inventory Notified ${body}")
+            .convertBodyTo(String.class)
+        ;
+    }
+
+    private static class InventoryNotification {
+        private Integer orderId;
+        private Integer itemId;
+        private Integer quantity;
+        private String department;
+        private Date datetime;
+
+        public static InventoryNotification getInventoryNotification(Integer orderId, Integer itemId, Integer quantity ){
+            InventoryNotification invenNotification  = new InventoryNotification();
+            invenNotification.setOrderId(orderId);
+            invenNotification.setItemId(itemId);
+            invenNotification.setQuantity(quantity);
+            invenNotification.setDepartment("inventory");
+            SimpleDateFormat formatter= new SimpleDateFormat("yyyy-MM-dd 'at' HH:mm:ss z");
+            invenNotification.setDatetime(new Date(System.currentTimeMillis()));
+            return invenNotification;
+        }
+
+
+        public void setOrderId(Integer orderId){
+            this.orderId=orderId;
+        }
+        public void setItemId(Integer itemId){
+            this.itemId=itemId;
+        }
+        public void setQuantity(Integer quantity){
+            this.quantity=quantity;
+        }
+        public Integer getOrderId(){
+            return this.orderId;
+        }
+        public Integer getItemId(){
+            return this.itemId;
+        }
+        public Integer getQuantity(){
+            return this.quantity;
+        }
+        public String getDepartment() {
+            return department;
+        }
+        public void setDepartment(String department) {
+            this.department = department;
+        }
+        public Date getDatetime() {
+            return datetime;
+        }
+    
+        public void setDatetime(Date datetime) {
+            this.datetime = datetime;
+        }
+    }
+    
+}
+```
+
+* *Order* which is a rest service called by a user to place an order for an product, and will notify the *Inventory* service
+
+**`OrderService.java`**
+```java
+import java.util.HashMap;
+import org.apache.camel.Processor;
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.model.rest.RestBindingMode;
+import org.apache.camel.Exchange;
+import org.apache.camel.component.jackson.JacksonDataFormat;
+import org.apache.camel.processor.aggregate.GroupedBodyAggregationStrategy;
+
+public class OrderService extends RouteBuilder {
+
+    @Override
+    public void configure() throws Exception {
+        
+        restConfiguration()
+            .enableCORS(true)
+            .bindingMode(RestBindingMode.json);
+
+        rest()
+            .post("/place")
+                .to("direct:placeorder");
+
+        JacksonDataFormat jacksonDataFormat = new JacksonDataFormat();
+        jacksonDataFormat.setUnmarshalType(Order.class);
+
+        from("direct:placeorder")
+            .log("-----IN ${headers}")
+            .marshal(jacksonDataFormat)
+            .log("inputBody --> ${body}")
+            .to("http://inventory/notify/order?bridgeEndpoint=true")
+            .removeHeaders("*")
+            .log("responseBody from inventory --> ${body}")
+            .setHeader(Exchange.CONTENT_TYPE, constant("application/json"))
+            .setBody(simple("{\"inventory\":${body}}"))
+            .unmarshal().json()
+        ;
+    }
+    
+    private static class Order implements java.io.Serializable{
+        private static final long serialVersionUID = 1L;
+        
+        private Integer orderId;
+        private Integer itemId;
+        private Integer quantity;
+
+        private String orderItemName;
+        private Integer price;
+
+        public void setOrderId(Integer orderId){
+            this.orderId=orderId;
+        }
+        public void setItemId(Integer itemId){
+            this.itemId=itemId;
+        }
+        public void setQuantity(Integer quantity){
+            this.quantity=quantity;
+        }
+
+        public void setOrderItemName(String orderItemName){
+            this.orderItemName=orderItemName;
+        }
+        public void setPrice(Integer price){
+            this.price=price;
+        }
+        
+        public Integer getOrderId(){
+            return this.orderId;
+        }
+        public Integer getItemId(){
+            return this.itemId;
+        }
+        public Integer getQuantity(){
+            return this.quantity;
+        }
+
+        public String getOrderItemName(){
+            return this.orderItemName;
+        }
+        public Integer getPrice(){
+            return this.price;
+        }
+
+    }
+}
+```
+
+Let's run our camel integrations:
+
+* the *Inventory* integration
+```sh
+$ kamel run InventoryService.java --name inventory \
+   -d camel-jackson \
+   -t telemetry.enabled=true \
+   -t telemetry.sampler=on \
+   -t ingress.enabled=true \
+   -t ingress.host=inventory.local
+Integration "inventory" created
+```
+* the *Order* integration
+```sh
+$ kamel run OrderService.java --name order \
+   -d camel-jackson \
+   -t telemetry.enabled=true \
+   -t telemetry.sampler=on \
+   -t telemetry.service-name=external-order \
+   -t ingress.enabled=true \
+   -t ingress.host=order.local
+Integration "order" created
+```
+
+This configuration will:
+* enable tracing for our integrations
+* sample all traces
+* send traces to OTLP API endpoint
+
+We are also using inventory.local and order.local host in ingress for the rest services so make sure that your DNS pointing to your cluster for both hostnames. You can execute `echo "$(minikube ip) order.local inventory.local" | sudo tee -a /etc/hosts` if you are working on minikube.
+
+As you can see we did not provide the OTLP API endpoint as Camel K automaticly discover the **Jaeger** OTLP API endpoint. For any other Distributed Tracing tool you will have to declare it through `-t telemetry.endpoint=xxxx`. The `telemetry` trait is expecting an [OTLP trace endpoint](https://opentelemetry.io/docs/concepts/sdk-configuration/otlp-exporter-configuration/#otel_exporter_otlp_traces_endpoint).
+
+Now that everything is up let's make somes traces !
+
+## Generate some integration traces
+
+1. Let's place some order:
+
+```sh
+$ curl  http://order.local/place -d '{"orderId":1, "itemId":5, "quantity":1, "orderItemName":"best product", "price":99}' -H "Content-Type: application/json"
+{"inventory":"{\"orderId\":1,\"itemId\":5,\"quantity\":1,\"department\":\"inventory\",\"datetime\":1678293274691}"}
+```
+
+If the call went well, you should have this response indicating that the inventory service has been called.
+
+In Jaeger's interface we should see the call with the spans referencing to the rest services and camel components.
+
+![Jaeger full trace](full_trace.png)
+
+
+2. Now, we will stop collecting the traces for the *Order* integration:
+```
+kamel run OrderService.java --name order \
+   -d camel-jackson \
+   -t ingress.enabled=true \
+   -t ingress.host=order.local
+Integration "order" updated
+```
+
+If we create another order:
+```sh
+$ curl  http://order.local/place -d '{"orderId":2, "itemId":18, "quantity":1, "orderItemName":"another product", "price":5}' -H "Content-Type: application/json"
+{"inventory":"{\"orderId\":2,\"itemId\":18,\"quantity\":1,\"department\":\"inventory\",\"datetime\":1678293771675}"}
+```
+
+As we can see in Jaeger's interface looking for the inventory traces the call will only show the spans for inventory and no longer the order parent spans.
+
+![Jaeger inventory trace](inventory_traces.png)
+
+As you can see, it is quite easy to improve the observability of your integrations using the new `telemetry` trait.
+
+## From tracing to telemetry
+
+We are encouraging migration from the `tracing` trait to the `telemetry` trait.
+
+To give you some inputs on your migration, I will show you how to collect and compare the traces usings both traits.
+
+Let's deploy two different *Order* integrations to compare the traces generated :
+* stop collecting the traces from *Inventory* integration
+```sh
+kamel run InventoryService.java --name inventory \
+   -d camel-jackson
+Integration "inventory" updated
+```
+* update the *Order* integration using the `telemetry` trait
+```sh
+$ kamel run OrderService.java --name order \
+   -d camel-jackson \
+   -t telemetry.enabled=true \
+   -t telemetry.sampler=on \
+   -t telemetry.service-name=order-external-telemetry \
+   -t ingress.enabled=true \
+   -t ingress.host=order.local
+Integration "order" updated
+```
+* the *Order* integration using the `tracing` trait
+```sh
+$ kamel run OrderService.java --name order-tracing \
+   -d camel-jackson \
+   -t tracing.enabled=true \
+   -t tracing.sampler-type=const \
+   -t tracing.sampler-param=1 \
+   -t tracing.service-name=order-external-tracing \
+   -t ingress.enabled=true \
+   -t ingress.host=order-tracing.local
+Integration "order-tracing" created
+```
+
+Don't forget to ensure the new order-tracing.local hostname is configured for your kubernetes cluster !
+
+Let's create the same orders with the two order services:
+* one with the order service using the new `telemetry` trait
+```sh
+$ curl  http://order.local/place -d '{"orderId":20, "itemId":5, "quantity":1, "orderItemName":"strange product", "price":10}' -H "Content-Type: application/json"
+{"inventory":"{\"orderId\":20,\"itemId\":5,\"quantity\":1,\"department\":\"inventory\",\"datetime\":1678296117315}"}
+```
+* another with the order services using the deprecated `tracing` trait
+```sh
+$ curl  http://order-tracing.local/place -d '{"orderId":21, "itemId":5, "quantity":1, "orderItemName":"strange product", "price":10}' -H "Content-Type: application/json"
+{"inventory":"{\"orderId\":21,\"itemId\":5,\"quantity\":1,\"department\":\"inventory\",\"datetime\":1678296184049}"}
+```
+
+As you can see, going from the `trancing` to the `telemetry` is quite effortless. On the trace side, from one protocol to another you should expect some differences in your traces:
+
+![Difference between OpenTracing and OpenTelemtry traces](compare_tracing_telemetry.png)
+
+
+# Conclusion
+
+This blog post showed how you can use the new `telemetry` trait and how to work with the Jager Distributed Tracing tool. We also took the time to see how you can migrate from the deprecated `tracing` trait to the `telemetry` trait.
+
+
+If you have any feedback, ideas or find a new issue, please [create a new issue report in GitHub](https://github.com/apache/camel-karavan/issues)!
diff --git a/content/blog/2023/03/camel-k-telemetry/inventory_traces.png b/content/blog/2023/03/camel-k-telemetry/inventory_traces.png
new file mode 100644
index 00000000..499e2c57
Binary files /dev/null and b/content/blog/2023/03/camel-k-telemetry/inventory_traces.png differ