You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by pc...@apache.org on 2023/03/09 15:10:31 UTC
[camel-website] 01/02: feat: Camel-k telemetry
This is an automated email from the ASF dual-hosted git repository.
pcongiusti pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-website.git
commit 73c131d8c8efc7bd339e0efdc8e107b2d8e25150
Author: Gaelle Fournier <ga...@gmail.com>
AuthorDate: Mon Mar 6 11:21:53 2023 +0100
feat: Camel-k telemetry
---
.../compare_tracing_telemetry.png | Bin 0 -> 79994 bytes
.../blog/2023/03/camel-k-telemetry/featured.png | Bin 0 -> 191399 bytes
.../blog/2023/03/camel-k-telemetry/full_trace.png | Bin 0 -> 68084 bytes
content/blog/2023/03/camel-k-telemetry/index.md | 420 +++++++++++++++++++++
.../2023/03/camel-k-telemetry/inventory_traces.png | Bin 0 -> 158586 bytes
5 files changed, 420 insertions(+)
diff --git a/content/blog/2023/03/camel-k-telemetry/compare_tracing_telemetry.png b/content/blog/2023/03/camel-k-telemetry/compare_tracing_telemetry.png
new file mode 100644
index 00000000..a6910339
Binary files /dev/null and b/content/blog/2023/03/camel-k-telemetry/compare_tracing_telemetry.png differ
diff --git a/content/blog/2023/03/camel-k-telemetry/featured.png b/content/blog/2023/03/camel-k-telemetry/featured.png
new file mode 100644
index 00000000..0feb4b4b
Binary files /dev/null and b/content/blog/2023/03/camel-k-telemetry/featured.png differ
diff --git a/content/blog/2023/03/camel-k-telemetry/full_trace.png b/content/blog/2023/03/camel-k-telemetry/full_trace.png
new file mode 100644
index 00000000..6b1ffac8
Binary files /dev/null and b/content/blog/2023/03/camel-k-telemetry/full_trace.png differ
diff --git a/content/blog/2023/03/camel-k-telemetry/index.md b/content/blog/2023/03/camel-k-telemetry/index.md
new file mode 100644
index 00000000..c425aebf
--- /dev/null
+++ b/content/blog/2023/03/camel-k-telemetry/index.md
@@ -0,0 +1,420 @@
+---
+title: "Camel K Observability: Distributed Tracing"
+date: 2023-03-25
+draft: false
+authors: [gansheer]
+categories: ["Camel K", "Features"]
+preview: "Camel K: how to configure Distributed Tracing for an Integration"
+---
+
+Tracing is an important approach for controlling and monitoring the experience of users, it allows us to gather more information about an integration's performance.
+
+Camel K has long been providing support for distributed tracing using OpenTracing. At the beginning of 2022, the [CNCF](https://www.cncf.io) announced that there were [archiving the OpenTracing project](https://www.cncf.io/blog/2022/01/31/cncf-archives-the-opentracing-project/) in favor of the [OpenTelemetry project](https://opentelemetry.io/). OpenTelemetry is the latest solution created by merging OpenTracing and OpenCensus. As a result, we decided in Camel K 1.12 to introduce the [`te [...]
+
+I'll walk you through the configurations needed to support Distributed tracing for your camel integrations in this blog post.
+
+## Distributed Tracing tools
+
+OpenTelemetry defines the [OpenTelemetry Protocol (OTLP)](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/otlp.md) that allows any Distributed Tracing tool that provides a OTLP compatible collector API to be used. While OpenTelemetry is still CNCF incubating project, its has become an industry-standard and is now natively supported by many Distributed Tracings tools.
+
+The `telemetry` trait has been tested on Jaeger (v1.35+), OpenTelemetry Collector and Grafana Tempo to this day.
+
+We will be using Jaeger for this blog post manipulations.
+
+## Camel K
+
+The `telemetry` trait is available starting from Camel K version 1.12, so you should first check your kamel client version
+
+```sh
+$ kamel version
+Camel K Client 1.12.0
+```
+
+To install the Camel K operator just execute
+```sh
+$ kamel install
+```
+
+Check if the Camel K operator is available
+```sh
+$ kubectl get pods -l name=camel-k-operator
+NAME READY STATUS RESTARTS AGE
+camel-k-operator-5b897ddcdd-4qwqb 1/1 Running 0 50s
+```
+
+For more informations see the [documentation](https://camel.apache.org/camel-k/1.12.x/installation/installation.html)
+
+## Install Jaeger
+
+For this installation, you should firt ensure that ingress is available on your kubernetes. If you are working on minikube `minikube addons enable ingress` should be enough.
+
+If Jaeger is not already available on your kubernetes instance, an easy way to install it on a kubernetes is to install the Jaeger operator (see https://www.jaegertracing.io/docs).
+
+In short, to install the Jaeger operator you should:
+
+* Install cert-manager if it is not installed as it is required
+```sh
+ # The cert manager is a pre-requise to the Jaeger operator (see https://cert-manager.io/docs/installation/)
+$ kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.10.1/cert-manager.yaml
+```
+Wait 1 or 2 minutes, we want to ensure the cert-manager has had enough time to be fully deployed
+
+```sh
+ # see https://www.jaegertracing.io/docs/1.42/operator/#installing-the-operator-on-kubernetes
+$ kubectl create namespace observability
+$ kubectl create -f https://github.com/jaegertracing/jaeger-operator/releases/download/v1.42.0/jaeger-operator.yaml -n observability
+```
+
+Check if Jaeger-operator is available:
+```sh
+$ kubectl get deployment jaeger-operator -n observability
+
+NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
+jaeger-operator 1 1 1 1 48s
+```
+
+To use Jaeger, you can install the AllInOne image:
+
+```sh
+$ kubectl apply -f - << EOF
+apiVersion: jaegertracing.io/v1
+kind: Jaeger
+metadata:
+ name: instance
+EOF
+```
+
+Check the presence of the Jaeger instance
+
+```sh
+$ kubectl get jaeger
+NAME STATUS VERSION STRATEGY STORAGE AGE
+instance Running 1.42.0 allinone memory 8s
+```
+
+
+## Run some Camel K Integrations with Telemetry trait
+
+Now that we have Jaeger available on the Kubernetes cluster, the next step is to run some integrations in a Kubernetes cluster where Camel K is installed.
+You can find the source code in the [Camel K example github repository](https://github.com/apache/camel-k-examples/tree/main/generic-examples/traits/telemetry).
+
+We will be creating two integration routes as the distributed services:
+
+* *Inventory* which is also a rest service in charge of product's inventory management
+
+**`InventoryService.java`**
+```java
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.model.rest.RestBindingMode;
+import org.apache.camel.component.jackson.JacksonDataFormat;
+import java.text.SimpleDateFormat;
+import org.apache.camel.Exchange;
+import java.util.Date;
+import java.util.Map;
+
+
+public class InventoryService extends RouteBuilder {
+
+
+ @Override
+ public void configure() throws Exception {
+ restConfiguration()
+ .enableCORS(true)
+ .bindingMode(RestBindingMode.json);
+
+ rest()
+ .post("/notify/order/place")
+ .to("direct:notify");
+
+
+ JacksonDataFormat invDataFormat = new JacksonDataFormat();
+ invDataFormat.setUnmarshalType(InventoryNotification.class);
+
+ from("direct:notify")
+ .log("notifyorder--> ${body}")
+ .setHeader(Exchange.CONTENT_TYPE, constant("application/json"))
+ .bean(InventoryNotification.class, "getInventoryNotification(${body['orderId']},${body['itemId']},${body['quantity']} )")
+ .marshal(invDataFormat)
+ .log("Inventory Notified ${body}")
+ .convertBodyTo(String.class)
+ ;
+ }
+
+ private static class InventoryNotification {
+ private Integer orderId;
+ private Integer itemId;
+ private Integer quantity;
+ private String department;
+ private Date datetime;
+
+ public static InventoryNotification getInventoryNotification(Integer orderId, Integer itemId, Integer quantity ){
+ InventoryNotification invenNotification = new InventoryNotification();
+ invenNotification.setOrderId(orderId);
+ invenNotification.setItemId(itemId);
+ invenNotification.setQuantity(quantity);
+ invenNotification.setDepartment("inventory");
+ SimpleDateFormat formatter= new SimpleDateFormat("yyyy-MM-dd 'at' HH:mm:ss z");
+ invenNotification.setDatetime(new Date(System.currentTimeMillis()));
+ return invenNotification;
+ }
+
+
+ public void setOrderId(Integer orderId){
+ this.orderId=orderId;
+ }
+ public void setItemId(Integer itemId){
+ this.itemId=itemId;
+ }
+ public void setQuantity(Integer quantity){
+ this.quantity=quantity;
+ }
+ public Integer getOrderId(){
+ return this.orderId;
+ }
+ public Integer getItemId(){
+ return this.itemId;
+ }
+ public Integer getQuantity(){
+ return this.quantity;
+ }
+ public String getDepartment() {
+ return department;
+ }
+ public void setDepartment(String department) {
+ this.department = department;
+ }
+ public Date getDatetime() {
+ return datetime;
+ }
+
+ public void setDatetime(Date datetime) {
+ this.datetime = datetime;
+ }
+ }
+
+}
+```
+
+* *Order* which is a rest service called by a user to place an order for an product, and will notify the *Inventory* service
+
+**`OrderService.java`**
+```java
+import java.util.HashMap;
+import org.apache.camel.Processor;
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.model.rest.RestBindingMode;
+import org.apache.camel.Exchange;
+import org.apache.camel.component.jackson.JacksonDataFormat;
+import org.apache.camel.processor.aggregate.GroupedBodyAggregationStrategy;
+
+public class OrderService extends RouteBuilder {
+
+ @Override
+ public void configure() throws Exception {
+
+ restConfiguration()
+ .enableCORS(true)
+ .bindingMode(RestBindingMode.json);
+
+ rest()
+ .post("/place")
+ .to("direct:placeorder");
+
+ JacksonDataFormat jacksonDataFormat = new JacksonDataFormat();
+ jacksonDataFormat.setUnmarshalType(Order.class);
+
+ from("direct:placeorder")
+ .log("-----IN ${headers}")
+ .marshal(jacksonDataFormat)
+ .log("inputBody --> ${body}")
+ .to("http://inventory/notify/order?bridgeEndpoint=true")
+ .removeHeaders("*")
+ .log("responseBody from inventory --> ${body}")
+ .setHeader(Exchange.CONTENT_TYPE, constant("application/json"))
+ .setBody(simple("{\"inventory\":${body}}"))
+ .unmarshal().json()
+ ;
+ }
+
+ private static class Order implements java.io.Serializable{
+ private static final long serialVersionUID = 1L;
+
+ private Integer orderId;
+ private Integer itemId;
+ private Integer quantity;
+
+ private String orderItemName;
+ private Integer price;
+
+ public void setOrderId(Integer orderId){
+ this.orderId=orderId;
+ }
+ public void setItemId(Integer itemId){
+ this.itemId=itemId;
+ }
+ public void setQuantity(Integer quantity){
+ this.quantity=quantity;
+ }
+
+ public void setOrderItemName(String orderItemName){
+ this.orderItemName=orderItemName;
+ }
+ public void setPrice(Integer price){
+ this.price=price;
+ }
+
+ public Integer getOrderId(){
+ return this.orderId;
+ }
+ public Integer getItemId(){
+ return this.itemId;
+ }
+ public Integer getQuantity(){
+ return this.quantity;
+ }
+
+ public String getOrderItemName(){
+ return this.orderItemName;
+ }
+ public Integer getPrice(){
+ return this.price;
+ }
+
+ }
+}
+```
+
+Let's run our camel integrations:
+
+* the *Inventory* integration
+```sh
+$ kamel run InventoryService.java --name inventory \
+ -d camel-jackson \
+ -t telemetry.enabled=true \
+ -t telemetry.sampler=on \
+ -t ingress.enabled=true \
+ -t ingress.host=inventory.local
+Integration "inventory" created
+```
+* the *Order* integration
+```sh
+$ kamel run OrderService.java --name order \
+ -d camel-jackson \
+ -t telemetry.enabled=true \
+ -t telemetry.sampler=on \
+ -t telemetry.service-name=external-order \
+ -t ingress.enabled=true \
+ -t ingress.host=order.local
+Integration "order" created
+```
+
+This configuration will:
+* enable tracing for our integrations
+* sample all traces
+* send traces to OTLP API endpoint
+
+We are also using inventory.local and order.local host in ingress for the rest services so make sure that your DNS pointing to your cluster for both hostnames. You can execute `echo "$(minikube ip) order.local inventory.local" | sudo tee -a /etc/hosts` if you are working on minikube.
+
+As you can see we did not provide the OTLP API endpoint as Camel K automaticly discover the **Jaeger** OTLP API endpoint. For any other Distributed Tracing tool you will have to declare it through `-t telemetry.endpoint=xxxx`. The `telemetry` trait is expecting an [OTLP trace endpoint](https://opentelemetry.io/docs/concepts/sdk-configuration/otlp-exporter-configuration/#otel_exporter_otlp_traces_endpoint).
+
+Now that everything is up let's make somes traces !
+
+## Generate some integration traces
+
+1. Let's place some order:
+
+```sh
+$ curl http://order.local/place -d '{"orderId":1, "itemId":5, "quantity":1, "orderItemName":"best product", "price":99}' -H "Content-Type: application/json"
+{"inventory":"{\"orderId\":1,\"itemId\":5,\"quantity\":1,\"department\":\"inventory\",\"datetime\":1678293274691}"}
+```
+
+If the call went well, you should have this response indicating that the inventory service has been called.
+
+In Jaeger's interface we should see the call with the spans referencing to the rest services and camel components.
+
+
+
+
+2. Now, we will stop collecting the traces for the *Order* integration:
+```
+kamel run OrderService.java --name order \
+ -d camel-jackson \
+ -t ingress.enabled=true \
+ -t ingress.host=order.local
+Integration "order" updated
+```
+
+If we create another order:
+```sh
+$ curl http://order.local/place -d '{"orderId":2, "itemId":18, "quantity":1, "orderItemName":"another product", "price":5}' -H "Content-Type: application/json"
+{"inventory":"{\"orderId\":2,\"itemId\":18,\"quantity\":1,\"department\":\"inventory\",\"datetime\":1678293771675}"}
+```
+
+As we can see in Jaeger's interface looking for the inventory traces the call will only show the spans for inventory and no longer the order parent spans.
+
+
+
+As you can see, it is quite easy to improve the observability of your integrations using the new `telemetry` trait.
+
+## From tracing to telemetry
+
+We are encouraging migration from the `tracing` trait to the `telemetry` trait.
+
+To give you some inputs on your migration, I will show you how to collect and compare the traces usings both traits.
+
+Let's deploy two different *Order* integrations to compare the traces generated :
+* stop collecting the traces from *Inventory* integration
+```sh
+kamel run InventoryService.java --name inventory \
+ -d camel-jackson
+Integration "inventory" updated
+```
+* update the *Order* integration using the `telemetry` trait
+```sh
+$ kamel run OrderService.java --name order \
+ -d camel-jackson \
+ -t telemetry.enabled=true \
+ -t telemetry.sampler=on \
+ -t telemetry.service-name=order-external-telemetry \
+ -t ingress.enabled=true \
+ -t ingress.host=order.local
+Integration "order" updated
+```
+* the *Order* integration using the `tracing` trait
+```sh
+$ kamel run OrderService.java --name order-tracing \
+ -d camel-jackson \
+ -t tracing.enabled=true \
+ -t tracing.sampler-type=const \
+ -t tracing.sampler-param=1 \
+ -t tracing.service-name=order-external-tracing \
+ -t ingress.enabled=true \
+ -t ingress.host=order-tracing.local
+Integration "order-tracing" created
+```
+
+Don't forget to ensure the new order-tracing.local hostname is configured for your kubernetes cluster !
+
+Let's create the same orders with the two order services:
+* one with the order service using the new `telemetry` trait
+```sh
+$ curl http://order.local/place -d '{"orderId":20, "itemId":5, "quantity":1, "orderItemName":"strange product", "price":10}' -H "Content-Type: application/json"
+{"inventory":"{\"orderId\":20,\"itemId\":5,\"quantity\":1,\"department\":\"inventory\",\"datetime\":1678296117315}"}
+```
+* another with the order services using the deprecated `tracing` trait
+```sh
+$ curl http://order-tracing.local/place -d '{"orderId":21, "itemId":5, "quantity":1, "orderItemName":"strange product", "price":10}' -H "Content-Type: application/json"
+{"inventory":"{\"orderId\":21,\"itemId\":5,\"quantity\":1,\"department\":\"inventory\",\"datetime\":1678296184049}"}
+```
+
+As you can see, going from the `trancing` to the `telemetry` is quite effortless. On the trace side, from one protocol to another you should expect some differences in your traces:
+
+
+
+
+# Conclusion
+
+This blog post showed how you can use the new `telemetry` trait and how to work with the Jager Distributed Tracing tool. We also took the time to see how you can migrate from the deprecated `tracing` trait to the `telemetry` trait.
+
+
+If you have any feedback, ideas or find a new issue, please [create a new issue report in GitHub](https://github.com/apache/camel-karavan/issues)!
diff --git a/content/blog/2023/03/camel-k-telemetry/inventory_traces.png b/content/blog/2023/03/camel-k-telemetry/inventory_traces.png
new file mode 100644
index 00000000..499e2c57
Binary files /dev/null and b/content/blog/2023/03/camel-k-telemetry/inventory_traces.png differ