You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by "Stephan Siano (JIRA)" <ji...@apache.org> on 2010/09/14 15:27:40 UTC
[jira] Created: (CAMEL-3123) Performance/scalability issue for
converter lookup
Performance/scalability issue for converter lookup
--------------------------------------------------
Key: CAMEL-3123
URL: https://issues.apache.org/activemq/browse/CAMEL-3123
Project: Apache Camel
Issue Type: Bug
Components: camel-core
Affects Versions: 2.4.0
Environment: Camel 2.4.0 on apache karaf 2.0.0
Reporter: Stephan Siano
In a simple HTTP->HTTP proxy scenario (whith a Jetty or a servlet endpoint, that does not matter) I observe a very severe performance regression between Camel 2.2.0 (as in servicemix 4.2) and camel 2.4.0 running on a apache karaf 2.0.0 OSGi stack. On the same hardware I get a throughput of 3500 messages per second with Camel 2.2.0, but only 210 messages per second on Camel 2.4.0 (both servlet->HTTP). If I replace the http endpoint with a log endpoint the throughput will be about 1500 messages per second in both cases.
I have done some profiling for this: The active CPU times as shown in the profiler remain approximately the same for both versions, however if I monitor wait times, I get very long wait times for org.apache.camel.impl.DefaultMessage.getHeader(java.lang.String,java.lang.Class) calls. If I break this down I see the java.beans.PropertyEditorManager.findEditor(java.lang.Class) call in org.apache.camel.impl.converter.PropertyEditorTypeConverter.convertTo(java.lang.Class,java.lang.Object). The findEditor() method is synchronized and initializes some class loading which takes some time.
Why is it necessary to instantiate the type converter for each message?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-3123) Performance/scalability issue for
converter lookup
Posted by "Stephan Siano (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61948#action_61948 ]
Stephan Siano commented on CAMEL-3123:
--------------------------------------
Hi Claus,
sure, I can try that later today, however I do not expect much from it, because that didn't come up in my profiler run at all. In general sychronized methods are not evil as such, as long as they are really fast (total time (active cpu plus wait) of the synchronized stuff must be much smaller than the cpu time for the rest). With the trunk build my CPU time goes up to 100% so I don't expect another bottleneck (at least not on my dual core PC). When is the UUID generator used, for each message exchange?
According to the link, it is sufficient to add the line
<bean id="activeMQUuidGenerator" class="org.apache.camel.impl.ActiveMQUuidGenerator" />
to my spring config. Don't I have to reference it somehow in the camel context?
> Performance/scalability issue for converter lookup
> --------------------------------------------------
>
> Key: CAMEL-3123
> URL: https://issues.apache.org/activemq/browse/CAMEL-3123
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 2.4.0
> Environment: Camel 2.4.0 on apache karaf 2.0.0
> Reporter: Stephan Siano
> Assignee: Claus Ibsen
> Fix For: 2.5.0
>
>
> In a simple HTTP->HTTP proxy scenario (whith a Jetty or a servlet endpoint, that does not matter) I observe a very severe performance regression between Camel 2.2.0 (as in servicemix 4.2) and camel 2.4.0 running on a apache karaf 2.0.0 OSGi stack. On the same hardware I get a throughput of 3500 messages per second with Camel 2.2.0, but only 210 messages per second on Camel 2.4.0 (both servlet->HTTP). If I replace the http endpoint with a log endpoint the throughput will be about 1500 messages per second in both cases.
> I have done some profiling for this: The active CPU times as shown in the profiler remain approximately the same for both versions, however if I monitor wait times, I get very long wait times for org.apache.camel.impl.DefaultMessage.getHeader(java.lang.String,java.lang.Class) calls. If I break this down I see the java.beans.PropertyEditorManager.findEditor(java.lang.Class) call in org.apache.camel.impl.converter.PropertyEditorTypeConverter.convertTo(java.lang.Class,java.lang.Object). The findEditor() method is synchronized and initializes some class loading which takes some time.
> Why is it necessary to instantiate the type converter for each message?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-3123) Performance/scalability issue for
converter lookup
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61854#action_61854 ]
Claus Ibsen commented on CAMEL-3123:
------------------------------------
Thanks Stephan
If you do a straight proxy we could probably enhance jetty / http client combo to maybe be faster if we avoid loading body/headers into the Exchange. However there may be API implications between Jetty (server) and http client making this harder.
Also there are potential other http clients out there such as the new async http client we are looking to add in the future.
Jetty also has a http client etc.
Anyway glad the numbers is back to normal, or even a bit better.
> Performance/scalability issue for converter lookup
> --------------------------------------------------
>
> Key: CAMEL-3123
> URL: https://issues.apache.org/activemq/browse/CAMEL-3123
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 2.4.0
> Environment: Camel 2.4.0 on apache karaf 2.0.0
> Reporter: Stephan Siano
> Assignee: Claus Ibsen
> Fix For: 2.5.0
>
>
> In a simple HTTP->HTTP proxy scenario (whith a Jetty or a servlet endpoint, that does not matter) I observe a very severe performance regression between Camel 2.2.0 (as in servicemix 4.2) and camel 2.4.0 running on a apache karaf 2.0.0 OSGi stack. On the same hardware I get a throughput of 3500 messages per second with Camel 2.2.0, but only 210 messages per second on Camel 2.4.0 (both servlet->HTTP). If I replace the http endpoint with a log endpoint the throughput will be about 1500 messages per second in both cases.
> I have done some profiling for this: The active CPU times as shown in the profiler remain approximately the same for both versions, however if I monitor wait times, I get very long wait times for org.apache.camel.impl.DefaultMessage.getHeader(java.lang.String,java.lang.Class) calls. If I break this down I see the java.beans.PropertyEditorManager.findEditor(java.lang.Class) call in org.apache.camel.impl.converter.PropertyEditorTypeConverter.convertTo(java.lang.Class,java.lang.Object). The findEditor() method is synchronized and initializes some class loading which takes some time.
> Why is it necessary to instantiate the type converter for each message?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-3123) Performance/scalability issue for
converter lookup
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61843#action_61843 ]
Claus Ibsen commented on CAMEL-3123:
------------------------------------
trunk: 997262.
First part of optimization committed
> Performance/scalability issue for converter lookup
> --------------------------------------------------
>
> Key: CAMEL-3123
> URL: https://issues.apache.org/activemq/browse/CAMEL-3123
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 2.4.0
> Environment: Camel 2.4.0 on apache karaf 2.0.0
> Reporter: Stephan Siano
> Assignee: Claus Ibsen
>
> In a simple HTTP->HTTP proxy scenario (whith a Jetty or a servlet endpoint, that does not matter) I observe a very severe performance regression between Camel 2.2.0 (as in servicemix 4.2) and camel 2.4.0 running on a apache karaf 2.0.0 OSGi stack. On the same hardware I get a throughput of 3500 messages per second with Camel 2.2.0, but only 210 messages per second on Camel 2.4.0 (both servlet->HTTP). If I replace the http endpoint with a log endpoint the throughput will be about 1500 messages per second in both cases.
> I have done some profiling for this: The active CPU times as shown in the profiler remain approximately the same for both versions, however if I monitor wait times, I get very long wait times for org.apache.camel.impl.DefaultMessage.getHeader(java.lang.String,java.lang.Class) calls. If I break this down I see the java.beans.PropertyEditorManager.findEditor(java.lang.Class) call in org.apache.camel.impl.converter.PropertyEditorTypeConverter.convertTo(java.lang.Class,java.lang.Object). The findEditor() method is synchronized and initializes some class loading which takes some time.
> Why is it necessary to instantiate the type converter for each message?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-3123) Performance/scalability issue for
converter lookup
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61952#action_61952 ]
Claus Ibsen commented on CAMEL-3123:
------------------------------------
And it should log at INFO level that it uses that {{activeMQUuidGenerator}} bean as UUID, when Camel start up.
> Performance/scalability issue for converter lookup
> --------------------------------------------------
>
> Key: CAMEL-3123
> URL: https://issues.apache.org/activemq/browse/CAMEL-3123
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 2.4.0
> Environment: Camel 2.4.0 on apache karaf 2.0.0
> Reporter: Stephan Siano
> Assignee: Claus Ibsen
> Fix For: 2.5.0
>
>
> In a simple HTTP->HTTP proxy scenario (whith a Jetty or a servlet endpoint, that does not matter) I observe a very severe performance regression between Camel 2.2.0 (as in servicemix 4.2) and camel 2.4.0 running on a apache karaf 2.0.0 OSGi stack. On the same hardware I get a throughput of 3500 messages per second with Camel 2.2.0, but only 210 messages per second on Camel 2.4.0 (both servlet->HTTP). If I replace the http endpoint with a log endpoint the throughput will be about 1500 messages per second in both cases.
> I have done some profiling for this: The active CPU times as shown in the profiler remain approximately the same for both versions, however if I monitor wait times, I get very long wait times for org.apache.camel.impl.DefaultMessage.getHeader(java.lang.String,java.lang.Class) calls. If I break this down I see the java.beans.PropertyEditorManager.findEditor(java.lang.Class) call in org.apache.camel.impl.converter.PropertyEditorTypeConverter.convertTo(java.lang.Class,java.lang.Object). The findEditor() method is synchronized and initializes some class loading which takes some time.
> Why is it necessary to instantiate the type converter for each message?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-3123) Performance/scalability issue for
converter lookup
Posted by "Stephan Siano (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61853#action_61853 ]
Stephan Siano commented on CAMEL-3123:
--------------------------------------
I built from the trunk and tested the changes with my scenario: The performance improved from 210 msg/s with camel 2.4.0 to 3900 msg/s with the version from the trunk (and 160 concurrent clients), which is even better than the performance with camel 2.2.0. So I guess that these changes are a very good thing :-)
> Performance/scalability issue for converter lookup
> --------------------------------------------------
>
> Key: CAMEL-3123
> URL: https://issues.apache.org/activemq/browse/CAMEL-3123
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 2.4.0
> Environment: Camel 2.4.0 on apache karaf 2.0.0
> Reporter: Stephan Siano
> Assignee: Claus Ibsen
> Fix For: 2.5.0
>
>
> In a simple HTTP->HTTP proxy scenario (whith a Jetty or a servlet endpoint, that does not matter) I observe a very severe performance regression between Camel 2.2.0 (as in servicemix 4.2) and camel 2.4.0 running on a apache karaf 2.0.0 OSGi stack. On the same hardware I get a throughput of 3500 messages per second with Camel 2.2.0, but only 210 messages per second on Camel 2.4.0 (both servlet->HTTP). If I replace the http endpoint with a log endpoint the throughput will be about 1500 messages per second in both cases.
> I have done some profiling for this: The active CPU times as shown in the profiler remain approximately the same for both versions, however if I monitor wait times, I get very long wait times for org.apache.camel.impl.DefaultMessage.getHeader(java.lang.String,java.lang.Class) calls. If I break this down I see the java.beans.PropertyEditorManager.findEditor(java.lang.Class) call in org.apache.camel.impl.converter.PropertyEditorTypeConverter.convertTo(java.lang.Class,java.lang.Object). The findEditor() method is synchronized and initializes some class loading which takes some time.
> Why is it necessary to instantiate the type converter for each message?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-3123) Performance/scalability issue for
converter lookup
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen updated CAMEL-3123:
-------------------------------
Issue Type: Improvement (was: Bug)
> Performance/scalability issue for converter lookup
> --------------------------------------------------
>
> Key: CAMEL-3123
> URL: https://issues.apache.org/activemq/browse/CAMEL-3123
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 2.4.0
> Environment: Camel 2.4.0 on apache karaf 2.0.0
> Reporter: Stephan Siano
>
> In a simple HTTP->HTTP proxy scenario (whith a Jetty or a servlet endpoint, that does not matter) I observe a very severe performance regression between Camel 2.2.0 (as in servicemix 4.2) and camel 2.4.0 running on a apache karaf 2.0.0 OSGi stack. On the same hardware I get a throughput of 3500 messages per second with Camel 2.2.0, but only 210 messages per second on Camel 2.4.0 (both servlet->HTTP). If I replace the http endpoint with a log endpoint the throughput will be about 1500 messages per second in both cases.
> I have done some profiling for this: The active CPU times as shown in the profiler remain approximately the same for both versions, however if I monitor wait times, I get very long wait times for org.apache.camel.impl.DefaultMessage.getHeader(java.lang.String,java.lang.Class) calls. If I break this down I see the java.beans.PropertyEditorManager.findEditor(java.lang.Class) call in org.apache.camel.impl.converter.PropertyEditorTypeConverter.convertTo(java.lang.Class,java.lang.Object). The findEditor() method is synchronized and initializes some class loading which takes some time.
> Why is it necessary to instantiate the type converter for each message?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CAMEL-3123) Performance/scalability issue for
converter lookup
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen resolved CAMEL-3123.
--------------------------------
Fix Version/s: 2.5.0
Resolution: Fixed
trunk: 997297.
Improved PropertyEditorTypeConverter as well.
> Performance/scalability issue for converter lookup
> --------------------------------------------------
>
> Key: CAMEL-3123
> URL: https://issues.apache.org/activemq/browse/CAMEL-3123
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 2.4.0
> Environment: Camel 2.4.0 on apache karaf 2.0.0
> Reporter: Stephan Siano
> Assignee: Claus Ibsen
> Fix For: 2.5.0
>
>
> In a simple HTTP->HTTP proxy scenario (whith a Jetty or a servlet endpoint, that does not matter) I observe a very severe performance regression between Camel 2.2.0 (as in servicemix 4.2) and camel 2.4.0 running on a apache karaf 2.0.0 OSGi stack. On the same hardware I get a throughput of 3500 messages per second with Camel 2.2.0, but only 210 messages per second on Camel 2.4.0 (both servlet->HTTP). If I replace the http endpoint with a log endpoint the throughput will be about 1500 messages per second in both cases.
> I have done some profiling for this: The active CPU times as shown in the profiler remain approximately the same for both versions, however if I monitor wait times, I get very long wait times for org.apache.camel.impl.DefaultMessage.getHeader(java.lang.String,java.lang.Class) calls. If I break this down I see the java.beans.PropertyEditorManager.findEditor(java.lang.Class) call in org.apache.camel.impl.converter.PropertyEditorTypeConverter.convertTo(java.lang.Class,java.lang.Object). The findEditor() method is synchronized and initializes some class loading which takes some time.
> Why is it necessary to instantiate the type converter for each message?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-3123) Performance/scalability issue for
converter lookup
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61946#action_61946 ]
Claus Ibsen commented on CAMEL-3123:
------------------------------------
Stephan is it possible for you to run a performance test using a different UUID generator?
http://camel.apache.org/uuidgenerator.html
For example the {{ActiveMQUuidGenerator}}. The default uuid generator is synchronized as well and its part of the JDK. So switching uuid may give a little boost.
You can see on that link how to switch the generator used by Camel.
> Performance/scalability issue for converter lookup
> --------------------------------------------------
>
> Key: CAMEL-3123
> URL: https://issues.apache.org/activemq/browse/CAMEL-3123
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 2.4.0
> Environment: Camel 2.4.0 on apache karaf 2.0.0
> Reporter: Stephan Siano
> Assignee: Claus Ibsen
> Fix For: 2.5.0
>
>
> In a simple HTTP->HTTP proxy scenario (whith a Jetty or a servlet endpoint, that does not matter) I observe a very severe performance regression between Camel 2.2.0 (as in servicemix 4.2) and camel 2.4.0 running on a apache karaf 2.0.0 OSGi stack. On the same hardware I get a throughput of 3500 messages per second with Camel 2.2.0, but only 210 messages per second on Camel 2.4.0 (both servlet->HTTP). If I replace the http endpoint with a log endpoint the throughput will be about 1500 messages per second in both cases.
> I have done some profiling for this: The active CPU times as shown in the profiler remain approximately the same for both versions, however if I monitor wait times, I get very long wait times for org.apache.camel.impl.DefaultMessage.getHeader(java.lang.String,java.lang.Class) calls. If I break this down I see the java.beans.PropertyEditorManager.findEditor(java.lang.Class) call in org.apache.camel.impl.converter.PropertyEditorTypeConverter.convertTo(java.lang.Class,java.lang.Object). The findEditor() method is synchronized and initializes some class loading which takes some time.
> Why is it necessary to instantiate the type converter for each message?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-3123) Performance/scalability issue for
converter lookup
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61836#action_61836 ]
Claus Ibsen commented on CAMEL-3123:
------------------------------------
I think I got a little unit test reproducing the issue.
Its because Jetty has some types for the headers and whatnot which is to be converted to String.
Providing some Jetty specific type converters and also add a local cache to the PropertyEditorTypeConverter so the synchronized impact is reduced
> Performance/scalability issue for converter lookup
> --------------------------------------------------
>
> Key: CAMEL-3123
> URL: https://issues.apache.org/activemq/browse/CAMEL-3123
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 2.4.0
> Environment: Camel 2.4.0 on apache karaf 2.0.0
> Reporter: Stephan Siano
> Assignee: Claus Ibsen
>
> In a simple HTTP->HTTP proxy scenario (whith a Jetty or a servlet endpoint, that does not matter) I observe a very severe performance regression between Camel 2.2.0 (as in servicemix 4.2) and camel 2.4.0 running on a apache karaf 2.0.0 OSGi stack. On the same hardware I get a throughput of 3500 messages per second with Camel 2.2.0, but only 210 messages per second on Camel 2.4.0 (both servlet->HTTP). If I replace the http endpoint with a log endpoint the throughput will be about 1500 messages per second in both cases.
> I have done some profiling for this: The active CPU times as shown in the profiler remain approximately the same for both versions, however if I monitor wait times, I get very long wait times for org.apache.camel.impl.DefaultMessage.getHeader(java.lang.String,java.lang.Class) calls. If I break this down I see the java.beans.PropertyEditorManager.findEditor(java.lang.Class) call in org.apache.camel.impl.converter.PropertyEditorTypeConverter.convertTo(java.lang.Class,java.lang.Object). The findEditor() method is synchronized and initializes some class loading which takes some time.
> Why is it necessary to instantiate the type converter for each message?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-3123) Performance/scalability issue for
converter lookup
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61835#action_61835 ]
Claus Ibsen commented on CAMEL-3123:
------------------------------------
Can you post a small sample app / unit test which causes this on your system?
> Performance/scalability issue for converter lookup
> --------------------------------------------------
>
> Key: CAMEL-3123
> URL: https://issues.apache.org/activemq/browse/CAMEL-3123
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 2.4.0
> Environment: Camel 2.4.0 on apache karaf 2.0.0
> Reporter: Stephan Siano
>
> In a simple HTTP->HTTP proxy scenario (whith a Jetty or a servlet endpoint, that does not matter) I observe a very severe performance regression between Camel 2.2.0 (as in servicemix 4.2) and camel 2.4.0 running on a apache karaf 2.0.0 OSGi stack. On the same hardware I get a throughput of 3500 messages per second with Camel 2.2.0, but only 210 messages per second on Camel 2.4.0 (both servlet->HTTP). If I replace the http endpoint with a log endpoint the throughput will be about 1500 messages per second in both cases.
> I have done some profiling for this: The active CPU times as shown in the profiler remain approximately the same for both versions, however if I monitor wait times, I get very long wait times for org.apache.camel.impl.DefaultMessage.getHeader(java.lang.String,java.lang.Class) calls. If I break this down I see the java.beans.PropertyEditorManager.findEditor(java.lang.Class) call in org.apache.camel.impl.converter.PropertyEditorTypeConverter.convertTo(java.lang.Class,java.lang.Object). The findEditor() method is synchronized and initializes some class loading which takes some time.
> Why is it necessary to instantiate the type converter for each message?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-3123) Performance/scalability issue for
converter lookup
Posted by "Stephan Siano (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61840#action_61840 ]
Stephan Siano commented on CAMEL-3123:
--------------------------------------
Does your sample send these messages sequentially (in one thread)? If this is the case, the improvement will be even much larger if you have e.g, 10 threads sending 50 messages each (because the time you save is almost completely in a synchronized method).
I am still unsure about what this PropertyManagerEditor.findEditor() method does. In my example it is called six times for a message, two times each with org.mortbay.jetty.Request, org.mortbay.jetty.Response, and java.lang.StringBuffer. It seems always to trigger ClassLoader.loadClass() calls which trigger the following calls:
org.ops4j.pax.swissbox.core.BundleClassLoader.loadClass(java.lang.String,boolean)java.lang.Class
org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(java.lang.String,boolean)java.lang.Class
sun.misc.Launcher$AppClassLoader.loadClass(java.lang.String,boolean)java.lang.Class
I have actually no idea why this is necessary at all, but it might help to keep all the found editors with the PropertyEditorTypeConverter (is that what you did?). Might that break some OSGi bundle concept?
> Performance/scalability issue for converter lookup
> --------------------------------------------------
>
> Key: CAMEL-3123
> URL: https://issues.apache.org/activemq/browse/CAMEL-3123
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 2.4.0
> Environment: Camel 2.4.0 on apache karaf 2.0.0
> Reporter: Stephan Siano
> Assignee: Claus Ibsen
>
> In a simple HTTP->HTTP proxy scenario (whith a Jetty or a servlet endpoint, that does not matter) I observe a very severe performance regression between Camel 2.2.0 (as in servicemix 4.2) and camel 2.4.0 running on a apache karaf 2.0.0 OSGi stack. On the same hardware I get a throughput of 3500 messages per second with Camel 2.2.0, but only 210 messages per second on Camel 2.4.0 (both servlet->HTTP). If I replace the http endpoint with a log endpoint the throughput will be about 1500 messages per second in both cases.
> I have done some profiling for this: The active CPU times as shown in the profiler remain approximately the same for both versions, however if I monitor wait times, I get very long wait times for org.apache.camel.impl.DefaultMessage.getHeader(java.lang.String,java.lang.Class) calls. If I break this down I see the java.beans.PropertyEditorManager.findEditor(java.lang.Class) call in org.apache.camel.impl.converter.PropertyEditorTypeConverter.convertTo(java.lang.Class,java.lang.Object). The findEditor() method is synchronized and initializes some class loading which takes some time.
> Why is it necessary to instantiate the type converter for each message?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-3123) Performance/scalability issue for
converter lookup
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61950#action_61950 ]
Claus Ibsen commented on CAMEL-3123:
------------------------------------
Hi
Well the exchange id is created on demand, such as when being logged or accessed. So you may have logic where the id is not used at all.
However a component can also provide their own id, such as if the id was part of the message protocol.
I looked in the camel-jetty component and it relies on the default generator and hence it would trigger the slower JDK UUID generator if accessed.
I have also seen users with Spring Integration having performance degration because SI uses UUID by default.
And frankly we also only switch the UUID generator in Camel due Google App Engine compatibility. Now that Christian Mueller added the pluggable API for UUIDs
we can revert back to AMQ based as default, and then just plugin the JDK UUID based on camel-gae component.
See CAMEL-3139 for some numbers I ran on my local laptop, albeit using a single thread (easier and quick to setup).
> Performance/scalability issue for converter lookup
> --------------------------------------------------
>
> Key: CAMEL-3123
> URL: https://issues.apache.org/activemq/browse/CAMEL-3123
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 2.4.0
> Environment: Camel 2.4.0 on apache karaf 2.0.0
> Reporter: Stephan Siano
> Assignee: Claus Ibsen
> Fix For: 2.5.0
>
>
> In a simple HTTP->HTTP proxy scenario (whith a Jetty or a servlet endpoint, that does not matter) I observe a very severe performance regression between Camel 2.2.0 (as in servicemix 4.2) and camel 2.4.0 running on a apache karaf 2.0.0 OSGi stack. On the same hardware I get a throughput of 3500 messages per second with Camel 2.2.0, but only 210 messages per second on Camel 2.4.0 (both servlet->HTTP). If I replace the http endpoint with a log endpoint the throughput will be about 1500 messages per second in both cases.
> I have done some profiling for this: The active CPU times as shown in the profiler remain approximately the same for both versions, however if I monitor wait times, I get very long wait times for org.apache.camel.impl.DefaultMessage.getHeader(java.lang.String,java.lang.Class) calls. If I break this down I see the java.beans.PropertyEditorManager.findEditor(java.lang.Class) call in org.apache.camel.impl.converter.PropertyEditorTypeConverter.convertTo(java.lang.Class,java.lang.Object). The findEditor() method is synchronized and initializes some class loading which takes some time.
> Why is it necessary to instantiate the type converter for each message?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-3123) Performance/scalability issue for
converter lookup
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61841#action_61841 ]
Claus Ibsen commented on CAMEL-3123:
------------------------------------
Yeah its a sequential test, but its still faster because each message caused the property editor to be invoked about 4-6 times.
I add specialized type converters for StringBuffer/StringBuilder -> String + Jetty specific as well, which removes the usage of the property editor being used.
However I will add an internal cache to it because you may have other cases where you want to convert a custom XXX object -> String which potentially can use the property editor to find an editor, and hence the synchronized stuff and as you write above some load class stuff as well.
Will commit this stuff after tests run is complete. Then you can maybe try it out on your system? You need to build from the source yourself.
Yeah anything can break in OSGi. Don't assume it just works, that is the attitude I use after seining so many issue with it.
> Performance/scalability issue for converter lookup
> --------------------------------------------------
>
> Key: CAMEL-3123
> URL: https://issues.apache.org/activemq/browse/CAMEL-3123
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 2.4.0
> Environment: Camel 2.4.0 on apache karaf 2.0.0
> Reporter: Stephan Siano
> Assignee: Claus Ibsen
>
> In a simple HTTP->HTTP proxy scenario (whith a Jetty or a servlet endpoint, that does not matter) I observe a very severe performance regression between Camel 2.2.0 (as in servicemix 4.2) and camel 2.4.0 running on a apache karaf 2.0.0 OSGi stack. On the same hardware I get a throughput of 3500 messages per second with Camel 2.2.0, but only 210 messages per second on Camel 2.4.0 (both servlet->HTTP). If I replace the http endpoint with a log endpoint the throughput will be about 1500 messages per second in both cases.
> I have done some profiling for this: The active CPU times as shown in the profiler remain approximately the same for both versions, however if I monitor wait times, I get very long wait times for org.apache.camel.impl.DefaultMessage.getHeader(java.lang.String,java.lang.Class) calls. If I break this down I see the java.beans.PropertyEditorManager.findEditor(java.lang.Class) call in org.apache.camel.impl.converter.PropertyEditorTypeConverter.convertTo(java.lang.Class,java.lang.Object). The findEditor() method is synchronized and initializes some class loading which takes some time.
> Why is it necessary to instantiate the type converter for each message?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-3123) Performance/scalability issue for
converter lookup
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61951#action_61951 ]
Claus Ibsen commented on CAMEL-3123:
------------------------------------
And yes that Spring bean is all you need. Camel will auto lookup it
{code:xml}
<bean id="activeMQUuidGenerator" class="org.apache.camel.impl.ActiveMQUuidGenerator" />
{code}
> Performance/scalability issue for converter lookup
> --------------------------------------------------
>
> Key: CAMEL-3123
> URL: https://issues.apache.org/activemq/browse/CAMEL-3123
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 2.4.0
> Environment: Camel 2.4.0 on apache karaf 2.0.0
> Reporter: Stephan Siano
> Assignee: Claus Ibsen
> Fix For: 2.5.0
>
>
> In a simple HTTP->HTTP proxy scenario (whith a Jetty or a servlet endpoint, that does not matter) I observe a very severe performance regression between Camel 2.2.0 (as in servicemix 4.2) and camel 2.4.0 running on a apache karaf 2.0.0 OSGi stack. On the same hardware I get a throughput of 3500 messages per second with Camel 2.2.0, but only 210 messages per second on Camel 2.4.0 (both servlet->HTTP). If I replace the http endpoint with a log endpoint the throughput will be about 1500 messages per second in both cases.
> I have done some profiling for this: The active CPU times as shown in the profiler remain approximately the same for both versions, however if I monitor wait times, I get very long wait times for org.apache.camel.impl.DefaultMessage.getHeader(java.lang.String,java.lang.Class) calls. If I break this down I see the java.beans.PropertyEditorManager.findEditor(java.lang.Class) call in org.apache.camel.impl.converter.PropertyEditorTypeConverter.convertTo(java.lang.Class,java.lang.Object). The findEditor() method is synchronized and initializes some class loading which takes some time.
> Why is it necessary to instantiate the type converter for each message?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CAMEL-3123) Performance/scalability issue for
converter lookup
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen reassigned CAMEL-3123:
----------------------------------
Assignee: Claus Ibsen
> Performance/scalability issue for converter lookup
> --------------------------------------------------
>
> Key: CAMEL-3123
> URL: https://issues.apache.org/activemq/browse/CAMEL-3123
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 2.4.0
> Environment: Camel 2.4.0 on apache karaf 2.0.0
> Reporter: Stephan Siano
> Assignee: Claus Ibsen
>
> In a simple HTTP->HTTP proxy scenario (whith a Jetty or a servlet endpoint, that does not matter) I observe a very severe performance regression between Camel 2.2.0 (as in servicemix 4.2) and camel 2.4.0 running on a apache karaf 2.0.0 OSGi stack. On the same hardware I get a throughput of 3500 messages per second with Camel 2.2.0, but only 210 messages per second on Camel 2.4.0 (both servlet->HTTP). If I replace the http endpoint with a log endpoint the throughput will be about 1500 messages per second in both cases.
> I have done some profiling for this: The active CPU times as shown in the profiler remain approximately the same for both versions, however if I monitor wait times, I get very long wait times for org.apache.camel.impl.DefaultMessage.getHeader(java.lang.String,java.lang.Class) calls. If I break this down I see the java.beans.PropertyEditorManager.findEditor(java.lang.Class) call in org.apache.camel.impl.converter.PropertyEditorTypeConverter.convertTo(java.lang.Class,java.lang.Object). The findEditor() method is synchronized and initializes some class loading which takes some time.
> Why is it necessary to instantiate the type converter for each message?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-3123) Performance/scalability issue for
converter lookup
Posted by "Stephan Siano (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61965#action_61965 ]
Stephan Siano commented on CAMEL-3123:
--------------------------------------
I tried it, but the UUID generator does not seem to play a role for my scenario. I tried both the initial servlet -> http processing as well as a jetty -> http processing. The result remained at 3900 messages per second for the servlet -> http case and changed from 4430 to 4450 messages per second in the jetty -> http test, but that is pretty within the error margin for these measurements. (The info level log was there so the UUID generator was loaded).
If these scenarios uses UUIDs at all, the type of the generator does not seem to affect the performance even for this scenario.
> Performance/scalability issue for converter lookup
> --------------------------------------------------
>
> Key: CAMEL-3123
> URL: https://issues.apache.org/activemq/browse/CAMEL-3123
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 2.4.0
> Environment: Camel 2.4.0 on apache karaf 2.0.0
> Reporter: Stephan Siano
> Assignee: Claus Ibsen
> Fix For: 2.5.0
>
>
> In a simple HTTP->HTTP proxy scenario (whith a Jetty or a servlet endpoint, that does not matter) I observe a very severe performance regression between Camel 2.2.0 (as in servicemix 4.2) and camel 2.4.0 running on a apache karaf 2.0.0 OSGi stack. On the same hardware I get a throughput of 3500 messages per second with Camel 2.2.0, but only 210 messages per second on Camel 2.4.0 (both servlet->HTTP). If I replace the http endpoint with a log endpoint the throughput will be about 1500 messages per second in both cases.
> I have done some profiling for this: The active CPU times as shown in the profiler remain approximately the same for both versions, however if I monitor wait times, I get very long wait times for org.apache.camel.impl.DefaultMessage.getHeader(java.lang.String,java.lang.Class) calls. If I break this down I see the java.beans.PropertyEditorManager.findEditor(java.lang.Class) call in org.apache.camel.impl.converter.PropertyEditorTypeConverter.convertTo(java.lang.Class,java.lang.Object). The findEditor() method is synchronized and initializes some class loading which takes some time.
> Why is it necessary to instantiate the type converter for each message?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-3123) Performance/scalability issue for
converter lookup
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61838#action_61838 ]
Claus Ibsen commented on CAMEL-3123:
------------------------------------
I created a small sample when sends 500 http requests to a http proxy.
The total time to do this:
before: 8.5 sec
after: 3.5 sec
By introducing Jetty specific type convertes and letting property editor being invoked later.
This eliminates the need for property editor being in use in that particular example.
> Performance/scalability issue for converter lookup
> --------------------------------------------------
>
> Key: CAMEL-3123
> URL: https://issues.apache.org/activemq/browse/CAMEL-3123
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 2.4.0
> Environment: Camel 2.4.0 on apache karaf 2.0.0
> Reporter: Stephan Siano
> Assignee: Claus Ibsen
>
> In a simple HTTP->HTTP proxy scenario (whith a Jetty or a servlet endpoint, that does not matter) I observe a very severe performance regression between Camel 2.2.0 (as in servicemix 4.2) and camel 2.4.0 running on a apache karaf 2.0.0 OSGi stack. On the same hardware I get a throughput of 3500 messages per second with Camel 2.2.0, but only 210 messages per second on Camel 2.4.0 (both servlet->HTTP). If I replace the http endpoint with a log endpoint the throughput will be about 1500 messages per second in both cases.
> I have done some profiling for this: The active CPU times as shown in the profiler remain approximately the same for both versions, however if I monitor wait times, I get very long wait times for org.apache.camel.impl.DefaultMessage.getHeader(java.lang.String,java.lang.Class) calls. If I break this down I see the java.beans.PropertyEditorManager.findEditor(java.lang.Class) call in org.apache.camel.impl.converter.PropertyEditorTypeConverter.convertTo(java.lang.Class,java.lang.Object). The findEditor() method is synchronized and initializes some class loading which takes some time.
> Why is it necessary to instantiate the type converter for each message?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.