You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "gansheer (via GitHub)" <gi...@apache.org> on 2023/03/09 09:16:36 UTC
[GitHub] [camel-website] gansheer opened a new pull request, #985: feat: Camel-k telemetry
gansheer opened a new pull request, #985:
URL: https://github.com/apache/camel-website/pull/985
Blog post for camel-k on how to use the `telemetry` trait.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-website] squakez commented on pull request #985: feat: Camel-k telemetry
Posted by "squakez (via GitHub)" <gi...@apache.org>.
squakez commented on PR #985:
URL: https://github.com/apache/camel-website/pull/985#issuecomment-1462213568
All is in good shape. @gansheer please confirm if you want this to be merged already, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-website] squakez commented on a diff in pull request #985: feat: Camel-k telemetry
Posted by "squakez (via GitHub)" <gi...@apache.org>.
squakez commented on code in PR #985:
URL: https://github.com/apache/camel-website/pull/985#discussion_r1130744838
##########
content/blog/2023/03/camel-k-telemetry/index.md:
##########
@@ -0,0 +1,420 @@
+---
+title: "Camel K Observability: Distributed Tracing"
+date: 2023-03-25
Review Comment:
IMO the blog is already good to go. We don't have any other blog in the pipeline and the last one was published a couple of weeks ago, so we can publish today itself.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-website] gansheer commented on pull request #985: feat: Camel-k telemetry
Posted by "gansheer (via GitHub)" <gi...@apache.org>.
gansheer commented on PR #985:
URL: https://github.com/apache/camel-website/pull/985#issuecomment-1462214812
LGTM, ready for merge :stars: !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-website] squakez commented on a diff in pull request #985: feat: Camel-k telemetry
Posted by "squakez (via GitHub)" <gi...@apache.org>.
squakez commented on code in PR #985:
URL: https://github.com/apache/camel-website/pull/985#discussion_r1130699519
##########
content/blog/2023/03/camel-k-telemetry/index.md:
##########
@@ -0,0 +1,420 @@
+---
+title: "Camel K Observability: Distributed Tracing"
+date: 2023-03-25
Review Comment:
Any reason why to post this in 2 weeks!?
##########
content/blog/2023/03/camel-k-telemetry/index.md:
##########
@@ -0,0 +1,420 @@
+---
+title: "Camel K Observability: Distributed Tracing"
+date: 2023-03-25
+draft: false
+authors: [gansheer]
+categories: ["Camel K", "Features"]
Review Comment:
Let's also add "devops" category
##########
content/blog/2023/03/camel-k-telemetry/index.md:
##########
@@ -0,0 +1,420 @@
+---
+title: "Camel K Observability: Distributed Tracing"
+date: 2023-03-25
+draft: false
+authors: [gansheer]
+categories: ["Camel K", "Features"]
+preview: "Camel K: how to configure Distributed Tracing for an Integration"
+---
+
+Tracing is an important approach for controlling and monitoring the experience of users, it allows us to gather more information about an integration's performance.
+
+Camel K has long been providing support for distributed tracing using OpenTracing. At the beginning of 2022, the [CNCF](https://www.cncf.io) announced that there were [archiving the OpenTracing project](https://www.cncf.io/blog/2022/01/31/cncf-archives-the-opentracing-project/) in favor of the [OpenTelemetry project](https://opentelemetry.io/). OpenTelemetry is the latest solution created by merging OpenTracing and OpenCensus. As a result, we decided in Camel K 1.12 to introduce the [`telemetry` trait](https://camel.apache.org/camel-k/1.12.x/traits/telemetry.html) based on OpenTelemtry. At the same time we decides to deprecate the [`tracing` trait](https://camel.apache.org/camel-k/1.12.x/traits/tracing.html) based on OpenTracing.
+
+I'll walk you through the configurations needed to support Distributed tracing for your camel integrations in this blog post.
+
+## Distributed Tracing tools
+
+OpenTelemetry defines the [OpenTelemetry Protocol (OTLP)](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/otlp.md) that allows any Distributed Tracing tool that provides a OTLP compatible collector API to be used. While OpenTelemetry is still CNCF incubating project, its has become an industry-standard and is now natively supported by many Distributed Tracings tools.
+
+The `telemetry` trait has been tested on Jaeger (v1.35+), OpenTelemetry Collector and Grafana Tempo to this day.
+
+We will be using Jaeger for this blog post manipulations.
+
+## Camel K
+
+The `telemetry` trait is available starting from Camel K version 1.12, so you should first check your kamel client version
+
+```sh
+$ kamel version
+Camel K Client 1.12.0
+```
+
+To install the Camel K operator just execute
+```sh
+$ kamel install
+```
+
+Check if the Camel K operator is available
+```sh
+$ kubectl get pods -l name=camel-k-operator
+NAME READY STATUS RESTARTS AGE
+camel-k-operator-5b897ddcdd-4qwqb 1/1 Running 0 50s
+```
+
+For more informations see the [documentation](https://camel.apache.org/camel-k/1.12.x/installation/installation.html)
+
+## Install Jaeger
+
+For this installation, you should firt ensure that ingress is available on your kubernetes. If you are working on minikube `minikube addons enable ingress` should be enough.
+
+If Jaeger is not already available on your kubernetes instance, an easy way to install it on a kubernetes is to install the Jaeger operator (see https://www.jaegertracing.io/docs).
+
+In short, to install the Jaeger operator you should:
+
+* Install cert-manager if it is not installed as it is required
+```sh
+ # The cert manager is a pre-requise to the Jaeger operator (see https://cert-manager.io/docs/installation/)
+$ kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.10.1/cert-manager.yaml
+```
+Wait 1 or 2 minutes, we want to ensure the cert-manager has had enough time to be fully deployed
+
+```sh
+ # see https://www.jaegertracing.io/docs/1.42/operator/#installing-the-operator-on-kubernetes
+$ kubectl create namespace observability
+$ kubectl create -f https://github.com/jaegertracing/jaeger-operator/releases/download/v1.42.0/jaeger-operator.yaml -n observability
+```
+
+Check if Jaeger-operator is available:
+```sh
+$ kubectl get deployment jaeger-operator -n observability
+
+NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
+jaeger-operator 1 1 1 1 48s
+```
+
+To use Jaeger, you can install the AllInOne image:
+
+```sh
+$ kubectl apply -f - << EOF
+apiVersion: jaegertracing.io/v1
+kind: Jaeger
+metadata:
+ name: instance
+EOF
+```
+
+Check the presence of the Jaeger instance
+
+```sh
+$ kubectl get jaeger
+NAME STATUS VERSION STRATEGY STORAGE AGE
+instance Running 1.42.0 allinone memory 8s
+```
+
+
+## Run some Camel K Integrations with Telemetry trait
+
+Now that we have Jaeger available on the Kubernetes cluster, the next step is to run some integrations in a Kubernetes cluster where Camel K is installed.
+You can find the source code in the [Camel K example github repository](https://github.com/apache/camel-k-examples/tree/main/generic-examples/traits/telemetry).
+
+We will be creating two integration routes as the distributed services:
+
+* *Inventory* which is also a rest service in charge of product's inventory management
+
+**`InventoryService.java`**
+```java
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.model.rest.RestBindingMode;
+import org.apache.camel.component.jackson.JacksonDataFormat;
+import java.text.SimpleDateFormat;
+import org.apache.camel.Exchange;
+import java.util.Date;
+import java.util.Map;
+
+
+public class InventoryService extends RouteBuilder {
+
+
+ @Override
+ public void configure() throws Exception {
+ restConfiguration()
+ .enableCORS(true)
+ .bindingMode(RestBindingMode.json);
+
+ rest()
+ .post("/notify/order/place")
+ .to("direct:notify");
+
+
+ JacksonDataFormat invDataFormat = new JacksonDataFormat();
+ invDataFormat.setUnmarshalType(InventoryNotification.class);
+
+ from("direct:notify")
+ .log("notifyorder--> ${body}")
+ .setHeader(Exchange.CONTENT_TYPE, constant("application/json"))
+ .bean(InventoryNotification.class, "getInventoryNotification(${body['orderId']},${body['itemId']},${body['quantity']} )")
+ .marshal(invDataFormat)
+ .log("Inventory Notified ${body}")
+ .convertBodyTo(String.class)
+ ;
+ }
+
+ private static class InventoryNotification {
+ private Integer orderId;
+ private Integer itemId;
+ private Integer quantity;
+ private String department;
+ private Date datetime;
+
+ public static InventoryNotification getInventoryNotification(Integer orderId, Integer itemId, Integer quantity ){
+ InventoryNotification invenNotification = new InventoryNotification();
+ invenNotification.setOrderId(orderId);
+ invenNotification.setItemId(itemId);
+ invenNotification.setQuantity(quantity);
+ invenNotification.setDepartment("inventory");
+ SimpleDateFormat formatter= new SimpleDateFormat("yyyy-MM-dd 'at' HH:mm:ss z");
+ invenNotification.setDatetime(new Date(System.currentTimeMillis()));
+ return invenNotification;
+ }
+
+
+ public void setOrderId(Integer orderId){
+ this.orderId=orderId;
+ }
+ public void setItemId(Integer itemId){
+ this.itemId=itemId;
+ }
+ public void setQuantity(Integer quantity){
+ this.quantity=quantity;
+ }
+ public Integer getOrderId(){
+ return this.orderId;
+ }
+ public Integer getItemId(){
+ return this.itemId;
+ }
+ public Integer getQuantity(){
+ return this.quantity;
+ }
+ public String getDepartment() {
+ return department;
+ }
+ public void setDepartment(String department) {
+ this.department = department;
+ }
+ public Date getDatetime() {
+ return datetime;
+ }
+
+ public void setDatetime(Date datetime) {
+ this.datetime = datetime;
+ }
+ }
+
+}
+```
+
+* *Order* which is a rest service called by a user to place an order for an product, and will notify the *Inventory* service
+
+**`OrderService.java`**
+```java
+import java.util.HashMap;
+import org.apache.camel.Processor;
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.model.rest.RestBindingMode;
+import org.apache.camel.Exchange;
+import org.apache.camel.component.jackson.JacksonDataFormat;
+import org.apache.camel.processor.aggregate.GroupedBodyAggregationStrategy;
+
+public class OrderService extends RouteBuilder {
+
+ @Override
+ public void configure() throws Exception {
+
+ restConfiguration()
+ .enableCORS(true)
+ .bindingMode(RestBindingMode.json);
+
+ rest()
+ .post("/place")
+ .to("direct:placeorder");
+
+ JacksonDataFormat jacksonDataFormat = new JacksonDataFormat();
+ jacksonDataFormat.setUnmarshalType(Order.class);
+
+ from("direct:placeorder")
+ .log("-----IN ${headers}")
+ .marshal(jacksonDataFormat)
+ .log("inputBody --> ${body}")
+ .to("http://inventory/notify/order?bridgeEndpoint=true")
+ .removeHeaders("*")
+ .log("responseBody from inventory --> ${body}")
+ .setHeader(Exchange.CONTENT_TYPE, constant("application/json"))
+ .setBody(simple("{\"inventory\":${body}}"))
+ .unmarshal().json()
+ ;
+ }
+
+ private static class Order implements java.io.Serializable{
+ private static final long serialVersionUID = 1L;
+
+ private Integer orderId;
+ private Integer itemId;
+ private Integer quantity;
+
+ private String orderItemName;
+ private Integer price;
+
+ public void setOrderId(Integer orderId){
+ this.orderId=orderId;
+ }
+ public void setItemId(Integer itemId){
+ this.itemId=itemId;
+ }
+ public void setQuantity(Integer quantity){
+ this.quantity=quantity;
+ }
+
+ public void setOrderItemName(String orderItemName){
+ this.orderItemName=orderItemName;
+ }
+ public void setPrice(Integer price){
+ this.price=price;
+ }
+
+ public Integer getOrderId(){
+ return this.orderId;
+ }
+ public Integer getItemId(){
+ return this.itemId;
+ }
+ public Integer getQuantity(){
+ return this.quantity;
+ }
+
+ public String getOrderItemName(){
+ return this.orderItemName;
+ }
+ public Integer getPrice(){
+ return this.price;
+ }
+
+ }
+}
+```
+
+Let's run our camel integrations:
+
+* the *Inventory* integration
+```sh
+$ kamel run InventoryService.java --name inventory \
+ -d camel-jackson \
+ -t telemetry.enabled=true \
+ -t telemetry.sampler=on \
+ -t ingress.enabled=true \
+ -t ingress.host=inventory.local
+Integration "inventory" created
+```
+* the *Order* integration
+```sh
+$ kamel run OrderService.java --name order \
+ -d camel-jackson \
+ -t telemetry.enabled=true \
+ -t telemetry.sampler=on \
+ -t telemetry.service-name=external-order \
+ -t ingress.enabled=true \
+ -t ingress.host=order.local
+Integration "order" created
+```
+
+This configuration will:
+* enable tracing for our integrations
+* sample all traces
+* send traces to OTLP API endpoint
+
+We are also using inventory.local and order.local host in ingress for the rest services so make sure that your DNS pointing to your cluster for both hostnames. You can execute `echo "$(minikube ip) order.local inventory.local" | sudo tee -a /etc/hosts` if you are working on minikube.
+
+As you can see we did not provide the OTLP API endpoint as Camel K automaticly discover the **Jaeger** OTLP API endpoint. For any other Distributed Tracing tool you will have to declare it through `-t telemetry.endpoint=xxxx`. The `telemetry` trait is expecting an [OTLP trace endpoint](https://opentelemetry.io/docs/concepts/sdk-configuration/otlp-exporter-configuration/#otel_exporter_otlp_traces_endpoint).
+
+Now that everything is up let's make somes traces !
+
+## Generate some integration traces
+
+1. Let's place some order:
+
+```sh
+$ curl http://order.local/place -d '{"orderId":1, "itemId":5, "quantity":1, "orderItemName":"best product", "price":99}' -H "Content-Type: application/json"
+{"inventory":"{\"orderId\":1,\"itemId\":5,\"quantity\":1,\"department\":\"inventory\",\"datetime\":1678293274691}"}
+```
+
+If the call went well, you should have this response indicating that the inventory service has been called.
+
+In Jaeger's interface we should see the call with the spans referencing to the rest services and camel components.
+
+
+
+
+2. Now, we will stop collecting the traces for the *Order* integration:
+```
+kamel run OrderService.java --name order \
+ -d camel-jackson \
+ -t ingress.enabled=true \
+ -t ingress.host=order.local
+Integration "order" updated
+```
+
+If we create another order:
+```sh
+$ curl http://order.local/place -d '{"orderId":2, "itemId":18, "quantity":1, "orderItemName":"another product", "price":5}' -H "Content-Type: application/json"
+{"inventory":"{\"orderId\":2,\"itemId\":18,\"quantity\":1,\"department\":\"inventory\",\"datetime\":1678293771675}"}
+```
+
+As we can see in Jaeger's interface looking for the inventory traces the call will only show the spans for inventory and no longer the order parent spans.
+
+
+
+As you can see, it is quite easy to improve the observability of your integrations using the new `telemetry` trait.
+
+## From tracing to telemetry
+
+We are encouraging migration from the `tracing` trait to the `telemetry` trait.
+
+To give you some inputs on your migration, I will show you how to collect and compare the traces usings both traits.
+
+Let's deploy two different *Order* integrations to compare the traces generated :
+* stop collecting the traces from *Inventory* integration
+```sh
+kamel run InventoryService.java --name inventory \
+ -d camel-jackson
+Integration "inventory" updated
+```
+* update the *Order* integration using the `telemetry` trait
+```sh
+$ kamel run OrderService.java --name order \
+ -d camel-jackson \
+ -t telemetry.enabled=true \
+ -t telemetry.sampler=on \
+ -t telemetry.service-name=order-external-telemetry \
+ -t ingress.enabled=true \
+ -t ingress.host=order.local
+Integration "order" updated
+```
+* the *Order* integration using the `tracing` trait
+```sh
+$ kamel run OrderService.java --name order-tracing \
+ -d camel-jackson \
+ -t tracing.enabled=true \
+ -t tracing.sampler-type=const \
+ -t tracing.sampler-param=1 \
+ -t tracing.service-name=order-external-tracing \
+ -t ingress.enabled=true \
+ -t ingress.host=order-tracing.local
+Integration "order-tracing" created
+```
+
+Don't forget to ensure the new order-tracing.local hostname is configured for your kubernetes cluster !
+
+Let's create the same orders with the two order services:
+* one with the order service using the new `telemetry` trait
+```sh
+$ curl http://order.local/place -d '{"orderId":20, "itemId":5, "quantity":1, "orderItemName":"strange product", "price":10}' -H "Content-Type: application/json"
+{"inventory":"{\"orderId\":20,\"itemId\":5,\"quantity\":1,\"department\":\"inventory\",\"datetime\":1678296117315}"}
+```
+* another with the order services using the deprecated `tracing` trait
+```sh
+$ curl http://order-tracing.local/place -d '{"orderId":21, "itemId":5, "quantity":1, "orderItemName":"strange product", "price":10}' -H "Content-Type: application/json"
+{"inventory":"{\"orderId\":21,\"itemId\":5,\"quantity\":1,\"department\":\"inventory\",\"datetime\":1678296184049}"}
+```
+
+As you can see, going from the `trancing` to the `telemetry` is quite effortless. On the trace side, from one protocol to another you should expect some differences in your traces:
Review Comment:
"trancing"
##########
content/blog/2023/03/camel-k-telemetry/index.md:
##########
@@ -0,0 +1,420 @@
+---
+title: "Camel K Observability: Distributed Tracing"
+date: 2023-03-25
+draft: false
+authors: [gansheer]
+categories: ["Camel K", "Features"]
+preview: "Camel K: how to configure Distributed Tracing for an Integration"
+---
+
+Tracing is an important approach for controlling and monitoring the experience of users, it allows us to gather more information about an integration's performance.
+
+Camel K has long been providing support for distributed tracing using OpenTracing. At the beginning of 2022, the [CNCF](https://www.cncf.io) announced that there were [archiving the OpenTracing project](https://www.cncf.io/blog/2022/01/31/cncf-archives-the-opentracing-project/) in favor of the [OpenTelemetry project](https://opentelemetry.io/). OpenTelemetry is the latest solution created by merging OpenTracing and OpenCensus. As a result, we decided in Camel K 1.12 to introduce the [`telemetry` trait](https://camel.apache.org/camel-k/1.12.x/traits/telemetry.html) based on OpenTelemtry. At the same time we decides to deprecate the [`tracing` trait](https://camel.apache.org/camel-k/1.12.x/traits/tracing.html) based on OpenTracing.
Review Comment:
"Camel K has been providing support for distributed tracing using OpenTracing since long time" sounds better. Also there is some typo in "OpenTelemtry"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-website] gansheer commented on a diff in pull request #985: feat: Camel-k telemetry
Posted by "gansheer (via GitHub)" <gi...@apache.org>.
gansheer commented on code in PR #985:
URL: https://github.com/apache/camel-website/pull/985#discussion_r1131009739
##########
content/blog/2023/03/camel-k-telemetry/index.md:
##########
@@ -0,0 +1,420 @@
+---
+title: "Camel K Observability: Distributed Tracing"
+date: 2023-03-25
Review Comment:
:ok_hand:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-website] gansheer commented on a diff in pull request #985: feat: Camel-k telemetry
Posted by "gansheer (via GitHub)" <gi...@apache.org>.
gansheer commented on code in PR #985:
URL: https://github.com/apache/camel-website/pull/985#discussion_r1130718123
##########
content/blog/2023/03/camel-k-telemetry/index.md:
##########
@@ -0,0 +1,420 @@
+---
+title: "Camel K Observability: Distributed Tracing"
+date: 2023-03-25
Review Comment:
None, I was just giving myself enough time. But that's also something I wanted to discuss : the date and when to publish/communicate on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-website] squakez merged pull request #985: feat: Camel-k telemetry
Posted by "squakez (via GitHub)" <gi...@apache.org>.
squakez merged PR #985:
URL: https://github.com/apache/camel-website/pull/985
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-website] gansheer commented on pull request #985: feat: Camel-k telemetry
Posted by "gansheer (via GitHub)" <gi...@apache.org>.
gansheer commented on PR #985:
URL: https://github.com/apache/camel-website/pull/985#issuecomment-1461634190
@squakez @claudio4j @christophd Please don't hesitate to give me some pointer on how to improve this first blog post
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-website] github-actions[bot] commented on pull request #985: feat: Camel-k telemetry
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #985:
URL: https://github.com/apache/camel-website/pull/985#issuecomment-1462170794
🚀 Preview is available at https://pr-985--camel.netlify.app
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org