[GitHub] [camel-website] apupier commented on a change in pull request #489: Whats new in Camel 3.6.

[GitBox <gi...@apache.org>]

apupier commented on a change in pull request #489:
URL: https://github.com/apache/camel-website/pull/489#discussion_r508324055



##########
File path: content/blog/2020/10/Camel36-Whatsnew/index.md
##########
@@ -0,0 +1,130 @@
+---
+title: "Apache Camel 3.6 What's New"
+date: 2020-10-21
+authors: [davsclaus]
+categories: ["Releases"]
+preview: Details of what we have done in the Camel 3.6 release.
+---
+
+Apache Camel 3.6 has just been released.
+
+This is a non-LTS release which means we will not provide patch releases, but use the release _as-is_.
+The next planned LTS release is 3.7 scheduled towards end of the year.
+
+
+## So what's in this release?
+
+This release introduces new set of features and noticeable improvements that will we cover in this blog post.
+
+
+### Spring Boot
+
+We have upgraded to latest release at this time which is Spring Boot 2.3.4.
+Support for Spring Boot 2.4 is planned for Camel 3.7.
+
+
+### Optimizations
+
+To speedup startup we switched to a new uuid generator. The old (classic) generator was inherited from Apache ActiveMQ which needed to ensure its ids were unique in a network of brokers, and therefore to ensure this the generator was using the hostname as prefix in the id. This required on startup to do a network access to obtain this information which costs a little time. Also depending on networks this can be more restrictive and delay the startup. The new generator is a pure in-memory fast generator that was used by Camel K and Camel Quarkus.
+
+We also identified a few other spots during route initialization. For example one small change was to avoid doing some regular expression masking on route endpoints which wasn't necessary anymore.
+
+Now the bigger improvements are in the following areas
+
+#### Avoid throwing exceptions
+
+We identified on spring runtimes that Camel would query the spring bean registry for known beans by id, which the Spring framework would throw a NoSuchBeanDefinitionException if the bean is not present. As Camel does a bit of optional bean discovery during bootstrap, we found a way to avoid this which prevents this.
+
+#### Singleton languages
+
+Another related problem is that in Camel 3 due to the modularization then some of the languages (bean, simple, and others) have been changed from being a singleton to prototype scoped. This is in fact one of the biggest problems and we had a Camel user report a problem with thread contention in a high concurrent use-case would race for resolving languages (they are prototype scoped). So you would have this problem, and because the language resolver would query the registry first then Spring would throw that no such bean exception, and then Camel would resolve the language via its own classpath resolver. So all together this cost performance. We can see this in the screenshots from the profiler in the following.
+
+{{< image "350-blocked.png" "Camel 3.5 Blocked Threads" >}}
+
+{{< image "360-blocked.png" "Camel 3.6 Blocked Threads" >}}
+
+The top screenshot is using Camel 3.5 and the bottom 3.6. In the top we can see the threads are blocked in Camels resolveLanguage method. And in 3.6 then its actually the log4j logger that is blocking for writing to the log file. Both applications are using the same Camel application and have been running for about 8 minutes.
+
+#### Reduce object allocations
+
+The next screenshots are showing a sample of the object allocations.
+
+{{< image "350-allocations.png" "Camel 3.5 Average Object Allocations Per Seconds" >}}
+
+{{< image "360-allocations.png" "Camel 3.6 Average Object Allocations Per Seconds" >}}
+
+
+With Camel 3.5 we are average about 1000 obj/sec and with 3.6 we are down to about a 1/3th.
+
+One of the improvements to help reduce the object allocations was how parameters to languages was changed from using a Map to a plain object array. The Map takes up more memory and object allocations than a single fixed object array. 
+
+#### Do as much init as possible
+
+Another performance improvement that aids during runtime was that we moved as much we could from the evaluation to the initialization phase in the Camel languages (simple, bean, etc.). We did this by introducing the init phase and ensuring CamelContext was carried around in the interns so we can use the context during the init phase, where its really needed. This ensures the runtime evaluation is as fast as possible.
+
+#### Other smaller optimizations
+
+We also improved the simple language to be a bit smarter in its binary operators (such as header.foo > 100). Now the simple language has stronger types for numeric and boolean types during its parsing, which allows us to know better from the right and left hand side of the binary operator to do type coercion so the types are comparable by the JVM. Before we may end up with falling back to converting to string types on both sides. And there is more to come, I have some ideas how to work on a compiled simple language.
+
+The screenshots below shows a chart with the CPU, object allocations and thrown exceptions.
+
+{{< image "350-performance.png" "Camel 3.5 Performance Charts" >}}
+
+{{< image "360-performance.png" "Camel 3.6 Performance Charts" >}}
+
+
+As we can see this summarise what was mentioned was done to optimize. The number of exceptions has been reduced to 0 at runtime. There is about 3500 thrown during bootstrap (that is Java JAXB which is used for loading the spring XML file with the Camel routes used for the sample application). We do have a fast XML loader in Camel that is not using JAXB.
+
+Another improvement we did was to build a source code generator for a new UriFactory which allows each component to quickly build dynamic endpoint URIs from a Map of parameters. The previous solution was to use RuntimeCamelCatalog that was more generic and required loading component metadata from json descriptor files. A few components use this to optimize the toD (such as http components). By this change we avoid the runtime catalog as dependency (reduce JAR size) and the source code generated uri factory is much faster (its speedy plain Java). However the sample application used for this blog did not use toD nor the UriFactory.
+
+Source from [external blog post](http://www.davsclaus.com/2020/10/apache-camel-36-more-camel-core.html)
+
+
+### API Components overhaul
+
+There are a number of API based components which are source code generated from _external API_. We have overhauled
+the code generator which now scrapes and includes documentation and keep the documentation up to date as well.
+In addition we also include additional metadata for Camel tooling so they can provide code assistance when Camel
+end users are using these API based components. Some of those external APIs are huge and you can have hundess of APIs.

Review comment:
       ```suggestion
   end users are using these API based components. Some of those external APIs are huge and you can have hundreds of APIs.
   ```

##########
File path: content/blog/2020/10/Camel36-Whatsnew/index.md
##########
@@ -0,0 +1,130 @@
+---
+title: "Apache Camel 3.6 What's New"
+date: 2020-10-21
+authors: [davsclaus]
+categories: ["Releases"]
+preview: Details of what we have done in the Camel 3.6 release.
+---
+
+Apache Camel 3.6 has just been released.
+
+This is a non-LTS release which means we will not provide patch releases, but use the release _as-is_.
+The next planned LTS release is 3.7 scheduled towards end of the year.
+
+
+## So what's in this release?
+
+This release introduces new set of features and noticeable improvements that will we cover in this blog post.
+
+
+### Spring Boot
+
+We have upgraded to latest release at this time which is Spring Boot 2.3.4.
+Support for Spring Boot 2.4 is planned for Camel 3.7.
+
+
+### Optimizations
+
+To speedup startup we switched to a new uuid generator. The old (classic) generator was inherited from Apache ActiveMQ which needed to ensure its ids were unique in a network of brokers, and therefore to ensure this the generator was using the hostname as prefix in the id. This required on startup to do a network access to obtain this information which costs a little time. Also depending on networks this can be more restrictive and delay the startup. The new generator is a pure in-memory fast generator that was used by Camel K and Camel Quarkus.
+
+We also identified a few other spots during route initialization. For example one small change was to avoid doing some regular expression masking on route endpoints which wasn't necessary anymore.
+
+Now the bigger improvements are in the following areas
+
+#### Avoid throwing exceptions
+
+We identified on spring runtimes that Camel would query the spring bean registry for known beans by id, which the Spring framework would throw a NoSuchBeanDefinitionException if the bean is not present. As Camel does a bit of optional bean discovery during bootstrap, we found a way to avoid this which prevents this.
+
+#### Singleton languages
+
+Another related problem is that in Camel 3 due to the modularization then some of the languages (bean, simple, and others) have been changed from being a singleton to prototype scoped. This is in fact one of the biggest problems and we had a Camel user report a problem with thread contention in a high concurrent use-case would race for resolving languages (they are prototype scoped). So you would have this problem, and because the language resolver would query the registry first then Spring would throw that no such bean exception, and then Camel would resolve the language via its own classpath resolver. So all together this cost performance. We can see this in the screenshots from the profiler in the following.
+
+{{< image "350-blocked.png" "Camel 3.5 Blocked Threads" >}}
+
+{{< image "360-blocked.png" "Camel 3.6 Blocked Threads" >}}
+
+The top screenshot is using Camel 3.5 and the bottom 3.6. In the top we can see the threads are blocked in Camels resolveLanguage method. And in 3.6 then its actually the log4j logger that is blocking for writing to the log file. Both applications are using the same Camel application and have been running for about 8 minutes.
+
+#### Reduce object allocations
+
+The next screenshots are showing a sample of the object allocations.
+
+{{< image "350-allocations.png" "Camel 3.5 Average Object Allocations Per Seconds" >}}
+
+{{< image "360-allocations.png" "Camel 3.6 Average Object Allocations Per Seconds" >}}
+
+
+With Camel 3.5 we are average about 1000 obj/sec and with 3.6 we are down to about a 1/3th.
+
+One of the improvements to help reduce the object allocations was how parameters to languages was changed from using a Map to a plain object array. The Map takes up more memory and object allocations than a single fixed object array. 
+
+#### Do as much init as possible
+
+Another performance improvement that aids during runtime was that we moved as much we could from the evaluation to the initialization phase in the Camel languages (simple, bean, etc.). We did this by introducing the init phase and ensuring CamelContext was carried around in the interns so we can use the context during the init phase, where its really needed. This ensures the runtime evaluation is as fast as possible.
+
+#### Other smaller optimizations
+
+We also improved the simple language to be a bit smarter in its binary operators (such as header.foo > 100). Now the simple language has stronger types for numeric and boolean types during its parsing, which allows us to know better from the right and left hand side of the binary operator to do type coercion so the types are comparable by the JVM. Before we may end up with falling back to converting to string types on both sides. And there is more to come, I have some ideas how to work on a compiled simple language.
+
+The screenshots below shows a chart with the CPU, object allocations and thrown exceptions.
+
+{{< image "350-performance.png" "Camel 3.5 Performance Charts" >}}
+
+{{< image "360-performance.png" "Camel 3.6 Performance Charts" >}}
+
+
+As we can see this summarise what was mentioned was done to optimize. The number of exceptions has been reduced to 0 at runtime. There is about 3500 thrown during bootstrap (that is Java JAXB which is used for loading the spring XML file with the Camel routes used for the sample application). We do have a fast XML loader in Camel that is not using JAXB.
+
+Another improvement we did was to build a source code generator for a new UriFactory which allows each component to quickly build dynamic endpoint URIs from a Map of parameters. The previous solution was to use RuntimeCamelCatalog that was more generic and required loading component metadata from json descriptor files. A few components use this to optimize the toD (such as http components). By this change we avoid the runtime catalog as dependency (reduce JAR size) and the source code generated uri factory is much faster (its speedy plain Java). However the sample application used for this blog did not use toD nor the UriFactory.
+
+Source from [external blog post](http://www.davsclaus.com/2020/10/apache-camel-36-more-camel-core.html)
+
+
+### API Components overhaul
+
+There are a number of API based components which are source code generated from _external API_. We have overhauled
+the code generator which now scrapes and includes documentation and keep the documentation up to date as well.
+In addition we also include additional metadata for Camel tooling so they can provide code assistance when Camel

Review comment:
       ```suggestion
   In addition we also include additional metadata for Camel tooling so they can [provide code assistance](https://issues.redhat.com/browse/FUSETOOLS2-615) when Camel
   ```

##########
File path: content/blog/2020/10/Camel36-Whatsnew/index.md
##########
@@ -0,0 +1,130 @@
+---
+title: "Apache Camel 3.6 What's New"
+date: 2020-10-21
+authors: [davsclaus]
+categories: ["Releases"]
+preview: Details of what we have done in the Camel 3.6 release.
+---
+
+Apache Camel 3.6 has just been released.
+
+This is a non-LTS release which means we will not provide patch releases, but use the release _as-is_.
+The next planned LTS release is 3.7 scheduled towards end of the year.
+
+
+## So what's in this release?
+
+This release introduces new set of features and noticeable improvements that will we cover in this blog post.
+
+
+### Spring Boot
+
+We have upgraded to latest release at this time which is Spring Boot 2.3.4.
+Support for Spring Boot 2.4 is planned for Camel 3.7.
+
+
+### Optimizations
+
+To speedup startup we switched to a new uuid generator. The old (classic) generator was inherited from Apache ActiveMQ which needed to ensure its ids were unique in a network of brokers, and therefore to ensure this the generator was using the hostname as prefix in the id. This required on startup to do a network access to obtain this information which costs a little time. Also depending on networks this can be more restrictive and delay the startup. The new generator is a pure in-memory fast generator that was used by Camel K and Camel Quarkus.
+
+We also identified a few other spots during route initialization. For example one small change was to avoid doing some regular expression masking on route endpoints which wasn't necessary anymore.
+
+Now the bigger improvements are in the following areas
+
+#### Avoid throwing exceptions
+
+We identified on spring runtimes that Camel would query the spring bean registry for known beans by id, which the Spring framework would throw a NoSuchBeanDefinitionException if the bean is not present. As Camel does a bit of optional bean discovery during bootstrap, we found a way to avoid this which prevents this.
+
+#### Singleton languages
+
+Another related problem is that in Camel 3 due to the modularization then some of the languages (bean, simple, and others) have been changed from being a singleton to prototype scoped. This is in fact one of the biggest problems and we had a Camel user report a problem with thread contention in a high concurrent use-case would race for resolving languages (they are prototype scoped). So you would have this problem, and because the language resolver would query the registry first then Spring would throw that no such bean exception, and then Camel would resolve the language via its own classpath resolver. So all together this cost performance. We can see this in the screenshots from the profiler in the following.
+
+{{< image "350-blocked.png" "Camel 3.5 Blocked Threads" >}}
+
+{{< image "360-blocked.png" "Camel 3.6 Blocked Threads" >}}
+
+The top screenshot is using Camel 3.5 and the bottom 3.6. In the top we can see the threads are blocked in Camels resolveLanguage method. And in 3.6 then its actually the log4j logger that is blocking for writing to the log file. Both applications are using the same Camel application and have been running for about 8 minutes.
+
+#### Reduce object allocations
+
+The next screenshots are showing a sample of the object allocations.
+
+{{< image "350-allocations.png" "Camel 3.5 Average Object Allocations Per Seconds" >}}
+
+{{< image "360-allocations.png" "Camel 3.6 Average Object Allocations Per Seconds" >}}
+
+
+With Camel 3.5 we are average about 1000 obj/sec and with 3.6 we are down to about a 1/3th.
+
+One of the improvements to help reduce the object allocations was how parameters to languages was changed from using a Map to a plain object array. The Map takes up more memory and object allocations than a single fixed object array. 
+
+#### Do as much init as possible
+
+Another performance improvement that aids during runtime was that we moved as much we could from the evaluation to the initialization phase in the Camel languages (simple, bean, etc.). We did this by introducing the init phase and ensuring CamelContext was carried around in the interns so we can use the context during the init phase, where its really needed. This ensures the runtime evaluation is as fast as possible.
+
+#### Other smaller optimizations
+
+We also improved the simple language to be a bit smarter in its binary operators (such as header.foo > 100). Now the simple language has stronger types for numeric and boolean types during its parsing, which allows us to know better from the right and left hand side of the binary operator to do type coercion so the types are comparable by the JVM. Before we may end up with falling back to converting to string types on both sides. And there is more to come, I have some ideas how to work on a compiled simple language.
+
+The screenshots below shows a chart with the CPU, object allocations and thrown exceptions.
+
+{{< image "350-performance.png" "Camel 3.5 Performance Charts" >}}
+
+{{< image "360-performance.png" "Camel 3.6 Performance Charts" >}}
+
+
+As we can see this summarise what was mentioned was done to optimize. The number of exceptions has been reduced to 0 at runtime. There is about 3500 thrown during bootstrap (that is Java JAXB which is used for loading the spring XML file with the Camel routes used for the sample application). We do have a fast XML loader in Camel that is not using JAXB.
+
+Another improvement we did was to build a source code generator for a new UriFactory which allows each component to quickly build dynamic endpoint URIs from a Map of parameters. The previous solution was to use RuntimeCamelCatalog that was more generic and required loading component metadata from json descriptor files. A few components use this to optimize the toD (such as http components). By this change we avoid the runtime catalog as dependency (reduce JAR size) and the source code generated uri factory is much faster (its speedy plain Java). However the sample application used for this blog did not use toD nor the UriFactory.
+
+Source from [external blog post](http://www.davsclaus.com/2020/10/apache-camel-36-more-camel-core.html)
+
+
+### API Components overhaul
+
+There are a number of API based components which are source code generated from _external API_. We have overhauled
+the code generator which now scrapes and includes documentation and keep the documentation up to date as well.
+In addition we also include additional metadata for Camel tooling so they can provide code assistance when Camel
+end users are using these API based components. Some of those external APIs are huge and you can have hundess of APIs.
+
+The API based components are: AS2, Box, Braintree, FHir, Google Calendar/Driver/Mail/Sheets, Olingo, Twillio, and Zendesk.

Review comment:
       ```suggestion
   The API based components are: [AS2](https://camel.apache.org/components/latest/as2-component.html), [Box](https://camel.apache.org/components/latest/box-component.html), [Braintree](https://camel.apache.org/components/latest/braintree-component.html), [FHIR](https://camel.apache.org/components/latest/fhir-component.html), Google [Calendar](https://camel.apache.org/components/latest/google-calendar-component.html)/[Drive](https://camel.apache.org/components/latest/google-drive-component.html)/[Mail](https://camel.apache.org/components/latest/google-mail-component.html)/[Sheets](https://camel.apache.org/components/latest/google-sheets-component.html), [Olingo](https://camel.apache.org/components/latest/olingo4-component.html), [Twillio](https://camel.apache.org/components/latest/twilio-component.html), and [Zendesk](https://camel.apache.org/components/latest/zendesk-component.html).
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org