You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by "Jeff Genender (JIRA)" <ji...@apache.org> on 2011/08/18 07:19:27 UTC
[jira] [Created] (CAMEL-4345) Synchronized code causes long delays
and hangs for big applications especially with Blueprint
Synchronized code causes long delays and hangs for big applications especially with Blueprint
---------------------------------------------------------------------------------------------
Key: CAMEL-4345
URL: https://issues.apache.org/jira/browse/CAMEL-4345
Project: Camel
Issue Type: Bug
Components: camel-core
Affects Versions: 2.8.0
Environment: Linux and Mac multicore machines
Reporter: Jeff Genender
The DefaultCamelContext uses synchronized "endpoints" which ends up ultimately extending a LinkedHashMap through the LRUCache. The LinkedHashMap is obviously not thread safe, so it requires synchronized guards when accessing the endpoints object. This especially happens in the getEndpoint(s) calls in the DefaultCamelContext. In large systems with lots of routes and on multicore systems, dynamically created routes (and many routes) can cause long delays and hang for long times since route creation and the starting of the camel route can occur in unison with synchronization. In a blueprint container, such as Karaf, this can cause timeouts on the bundle and camel routes will appear to hang indefinately. Thread dumps show the hangs occur on the synchronized call in getEndpoint(s). The fix for this is to use concurrent apis as much as possible and remove the synchronized code. I refactored the LRUCache/LRUSoftCache to use Google's ConcurrentLinkedHashMap (ASL2 License http://code.google.com/p/concurrentlinkedhashmap) and removed the synchronized code that locks the endpoints object. This should remove the hangs since the locks are no longer required. Since COncurrentLinkedHashmap is not OSGi ready, I have shaded the classes in core. On my executions, all unit tests pass with this refactoring using the concurrent code. This should speed up Camel on multicore systems that have lots of routes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4345) Synchronized code causes long
delays and hangs for big applications especially with Blueprint
Posted by "Hadrian Zbarcea (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087141#comment-13087141 ]
Hadrian Zbarcea commented on CAMEL-4345:
----------------------------------------
@Ben, thanks for the offer and welcome to the Camel community :).
> Synchronized code causes long delays and hangs for big applications especially with Blueprint
> ---------------------------------------------------------------------------------------------
>
> Key: CAMEL-4345
> URL: https://issues.apache.org/jira/browse/CAMEL-4345
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.0
> Environment: Linux and Mac multicore machines
> Reporter: Jeff Genender
> Attachments: CAMEL-4345.patch
>
>
> The DefaultCamelContext uses synchronized "endpoints" which ends up ultimately extending a LinkedHashMap through the LRUCache. The LinkedHashMap is obviously not thread safe, so it requires synchronized guards when accessing the endpoints object. This especially happens in the getEndpoint(s) calls in the DefaultCamelContext. In large systems with lots of routes and on multicore systems, dynamically created routes (and many routes) can cause long delays and hang for long times since route creation and the starting of the camel route can occur in unison with synchronization. In a blueprint container, such as Karaf, this can cause timeouts on the bundle and camel routes will appear to hang indefinately. Thread dumps show the hangs occur on the synchronized call in getEndpoint(s). The fix for this is to use concurrent apis as much as possible and remove the synchronized code. I refactored the LRUCache/LRUSoftCache to use Google's ConcurrentLinkedHashMap (ASL2 License http://code.google.com/p/concurrentlinkedhashmap) and removed the synchronized code that locks the endpoints object. This should remove the hangs since the locks are no longer required. Since COncurrentLinkedHashmap is not OSGi ready, I have shaded the classes in core. On my executions, all unit tests pass with this refactoring using the concurrent code. This should speed up Camel on multicore systems that have lots of routes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CAMEL-4345) Synchronized code causes long delays
and hangs for big applications especially with Blueprint
Posted by "Jeff Genender (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeff Genender updated CAMEL-4345:
---------------------------------
Attachment: CAMEL-4345.patch
Patch attached against TRUNK as of r1158987
> Synchronized code causes long delays and hangs for big applications especially with Blueprint
> ---------------------------------------------------------------------------------------------
>
> Key: CAMEL-4345
> URL: https://issues.apache.org/jira/browse/CAMEL-4345
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.0
> Environment: Linux and Mac multicore machines
> Reporter: Jeff Genender
> Attachments: CAMEL-4345.patch
>
>
> The DefaultCamelContext uses synchronized "endpoints" which ends up ultimately extending a LinkedHashMap through the LRUCache. The LinkedHashMap is obviously not thread safe, so it requires synchronized guards when accessing the endpoints object. This especially happens in the getEndpoint(s) calls in the DefaultCamelContext. In large systems with lots of routes and on multicore systems, dynamically created routes (and many routes) can cause long delays and hang for long times since route creation and the starting of the camel route can occur in unison with synchronization. In a blueprint container, such as Karaf, this can cause timeouts on the bundle and camel routes will appear to hang indefinately. Thread dumps show the hangs occur on the synchronized call in getEndpoint(s). The fix for this is to use concurrent apis as much as possible and remove the synchronized code. I refactored the LRUCache/LRUSoftCache to use Google's ConcurrentLinkedHashMap (ASL2 License http://code.google.com/p/concurrentlinkedhashmap) and removed the synchronized code that locks the endpoints object. This should remove the hangs since the locks are no longer required. Since COncurrentLinkedHashmap is not OSGi ready, I have shaded the classes in core. On my executions, all unit tests pass with this refactoring using the concurrent code. This should speed up Camel on multicore systems that have lots of routes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4345) Synchronized code causes long
delays and hangs for big applications especially with Blueprint
Posted by "Ben Manes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087133#comment-13087133 ]
Ben Manes commented on CAMEL-4345:
----------------------------------
Please open an issue to OSGi-ify CLHM and I'll do it for the next release (no timeframe).
As Google Guava is OSGi ready and I helped port the algorithm into MapMaker, that may also be a reasonable choice if it becomes a concern.
I'll be meeting up with Doug Lea in Sept. to discuss caching for JDK8, so maybe we'll see something in the standard libs eventually.
Cheers,
Ben
(CLHM author, Googler)
P.S. Don't you love Google Alerts? ;)
> Synchronized code causes long delays and hangs for big applications especially with Blueprint
> ---------------------------------------------------------------------------------------------
>
> Key: CAMEL-4345
> URL: https://issues.apache.org/jira/browse/CAMEL-4345
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.0
> Environment: Linux and Mac multicore machines
> Reporter: Jeff Genender
> Attachments: CAMEL-4345.patch
>
>
> The DefaultCamelContext uses synchronized "endpoints" which ends up ultimately extending a LinkedHashMap through the LRUCache. The LinkedHashMap is obviously not thread safe, so it requires synchronized guards when accessing the endpoints object. This especially happens in the getEndpoint(s) calls in the DefaultCamelContext. In large systems with lots of routes and on multicore systems, dynamically created routes (and many routes) can cause long delays and hang for long times since route creation and the starting of the camel route can occur in unison with synchronization. In a blueprint container, such as Karaf, this can cause timeouts on the bundle and camel routes will appear to hang indefinately. Thread dumps show the hangs occur on the synchronized call in getEndpoint(s). The fix for this is to use concurrent apis as much as possible and remove the synchronized code. I refactored the LRUCache/LRUSoftCache to use Google's ConcurrentLinkedHashMap (ASL2 License http://code.google.com/p/concurrentlinkedhashmap) and removed the synchronized code that locks the endpoints object. This should remove the hangs since the locks are no longer required. Since COncurrentLinkedHashmap is not OSGi ready, I have shaded the classes in core. On my executions, all unit tests pass with this refactoring using the concurrent code. This should speed up Camel on multicore systems that have lots of routes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4345) Synchronized code causes long
delays and hangs for big applications especially with Blueprint
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087678#comment-13087678 ]
Claus Ibsen commented on CAMEL-4345:
------------------------------------
I have applied that last change in osgi imports. And ran the osgi tests. All seems fine. Even camel-cxf which uses LRUCache from camel-core. It works also in osgi with that change above.
> Synchronized code causes long delays and hangs for big applications especially with Blueprint
> ---------------------------------------------------------------------------------------------
>
> Key: CAMEL-4345
> URL: https://issues.apache.org/jira/browse/CAMEL-4345
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.0
> Environment: Linux and Mac multicore machines
> Reporter: Jeff Genender
> Assignee: Hadrian Zbarcea
> Fix For: 2.9.0
>
> Attachments: CAMEL-4345.patch
>
>
> The DefaultCamelContext uses synchronized "endpoints" which ends up ultimately extending a LinkedHashMap through the LRUCache. The LinkedHashMap is obviously not thread safe, so it requires synchronized guards when accessing the endpoints object. This especially happens in the getEndpoint(s) calls in the DefaultCamelContext. In large systems with lots of routes and on multicore systems, dynamically created routes (and many routes) can cause long delays and hang for long times since route creation and the starting of the camel route can occur in unison with synchronization. In a blueprint container, such as Karaf, this can cause timeouts on the bundle and camel routes will appear to hang indefinately. Thread dumps show the hangs occur on the synchronized call in getEndpoint(s). The fix for this is to use concurrent apis as much as possible and remove the synchronized code. I refactored the LRUCache/LRUSoftCache to use Google's ConcurrentLinkedHashMap (ASL2 License http://code.google.com/p/concurrentlinkedhashmap) and removed the synchronized code that locks the endpoints object. This should remove the hangs since the locks are no longer required. Since COncurrentLinkedHashmap is not OSGi ready, I have shaded the classes in core. On my executions, all unit tests pass with this refactoring using the concurrent code. This should speed up Camel on multicore systems that have lots of routes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4345) Synchronized code causes long
delays and hangs for big applications especially with Blueprint
Posted by "Daniel Kulp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087083#comment-13087083 ]
Daniel Kulp commented on CAMEL-4345:
------------------------------------
I have no problem shading it in, but does it need to be exported from the bundle in OSGi?
Since it's completely an internal field, I don't think it does. I'd definitely prefer it didn't get exported so if some other project has a bundle with it, we don't conflict and such.
> Synchronized code causes long delays and hangs for big applications especially with Blueprint
> ---------------------------------------------------------------------------------------------
>
> Key: CAMEL-4345
> URL: https://issues.apache.org/jira/browse/CAMEL-4345
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.0
> Environment: Linux and Mac multicore machines
> Reporter: Jeff Genender
> Attachments: CAMEL-4345.patch
>
>
> The DefaultCamelContext uses synchronized "endpoints" which ends up ultimately extending a LinkedHashMap through the LRUCache. The LinkedHashMap is obviously not thread safe, so it requires synchronized guards when accessing the endpoints object. This especially happens in the getEndpoint(s) calls in the DefaultCamelContext. In large systems with lots of routes and on multicore systems, dynamically created routes (and many routes) can cause long delays and hang for long times since route creation and the starting of the camel route can occur in unison with synchronization. In a blueprint container, such as Karaf, this can cause timeouts on the bundle and camel routes will appear to hang indefinately. Thread dumps show the hangs occur on the synchronized call in getEndpoint(s). The fix for this is to use concurrent apis as much as possible and remove the synchronized code. I refactored the LRUCache/LRUSoftCache to use Google's ConcurrentLinkedHashMap (ASL2 License http://code.google.com/p/concurrentlinkedhashmap) and removed the synchronized code that locks the endpoints object. This should remove the hangs since the locks are no longer required. Since COncurrentLinkedHashmap is not OSGi ready, I have shaded the classes in core. On my executions, all unit tests pass with this refactoring using the concurrent code. This should speed up Camel on multicore systems that have lots of routes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CAMEL-4345) Synchronized code causes long delays
and hangs for big applications especially with Blueprint
Posted by "Hadrian Zbarcea (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hadrian Zbarcea reassigned CAMEL-4345:
--------------------------------------
Assignee: Hadrian Zbarcea
> Synchronized code causes long delays and hangs for big applications especially with Blueprint
> ---------------------------------------------------------------------------------------------
>
> Key: CAMEL-4345
> URL: https://issues.apache.org/jira/browse/CAMEL-4345
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.0
> Environment: Linux and Mac multicore machines
> Reporter: Jeff Genender
> Assignee: Hadrian Zbarcea
> Attachments: CAMEL-4345.patch
>
>
> The DefaultCamelContext uses synchronized "endpoints" which ends up ultimately extending a LinkedHashMap through the LRUCache. The LinkedHashMap is obviously not thread safe, so it requires synchronized guards when accessing the endpoints object. This especially happens in the getEndpoint(s) calls in the DefaultCamelContext. In large systems with lots of routes and on multicore systems, dynamically created routes (and many routes) can cause long delays and hang for long times since route creation and the starting of the camel route can occur in unison with synchronization. In a blueprint container, such as Karaf, this can cause timeouts on the bundle and camel routes will appear to hang indefinately. Thread dumps show the hangs occur on the synchronized call in getEndpoint(s). The fix for this is to use concurrent apis as much as possible and remove the synchronized code. I refactored the LRUCache/LRUSoftCache to use Google's ConcurrentLinkedHashMap (ASL2 License http://code.google.com/p/concurrentlinkedhashmap) and removed the synchronized code that locks the endpoints object. This should remove the hangs since the locks are no longer required. Since COncurrentLinkedHashmap is not OSGi ready, I have shaded the classes in core. On my executions, all unit tests pass with this refactoring using the concurrent code. This should speed up Camel on multicore systems that have lots of routes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CAMEL-4345) Synchronized code causes long delays
and hangs for big applications especially with Blueprint
Posted by "Daniel Kulp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Kulp updated CAMEL-4345:
-------------------------------
Fix Version/s: 2.8.2
> Synchronized code causes long delays and hangs for big applications especially with Blueprint
> ---------------------------------------------------------------------------------------------
>
> Key: CAMEL-4345
> URL: https://issues.apache.org/jira/browse/CAMEL-4345
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.0
> Environment: Linux and Mac multicore machines
> Reporter: Jeff Genender
> Assignee: Hadrian Zbarcea
> Fix For: 2.8.2, 2.9.0
>
> Attachments: CAMEL-4345.patch
>
>
> The DefaultCamelContext uses synchronized "endpoints" which ends up ultimately extending a LinkedHashMap through the LRUCache. The LinkedHashMap is obviously not thread safe, so it requires synchronized guards when accessing the endpoints object. This especially happens in the getEndpoint(s) calls in the DefaultCamelContext. In large systems with lots of routes and on multicore systems, dynamically created routes (and many routes) can cause long delays and hang for long times since route creation and the starting of the camel route can occur in unison with synchronization. In a blueprint container, such as Karaf, this can cause timeouts on the bundle and camel routes will appear to hang indefinately. Thread dumps show the hangs occur on the synchronized call in getEndpoint(s). The fix for this is to use concurrent apis as much as possible and remove the synchronized code. I refactored the LRUCache/LRUSoftCache to use Google's ConcurrentLinkedHashMap (ASL2 License http://code.google.com/p/concurrentlinkedhashmap) and removed the synchronized code that locks the endpoints object. This should remove the hangs since the locks are no longer required. Since COncurrentLinkedHashmap is not OSGi ready, I have shaded the classes in core. On my executions, all unit tests pass with this refactoring using the concurrent code. This should speed up Camel on multicore systems that have lots of routes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4345) Synchronized code causes long
delays and hangs for big applications especially with Blueprint
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087605#comment-13087605 ]
Claus Ibsen commented on CAMEL-4345:
------------------------------------
The osgi manifest contains import of the google code
{code}
Import-Package: com.googlecode.concurrentlinkedhashmap,javax.activatio
n;resolution:=optional,javax.management,javax.management.modelmbean,j
avax.management.remote,javax.naming,javax.naming.spi,javax.net.ssl,ja
vax.xml.bind;resolution:=optional,javax.xml.bind.annotation;resolutio
n:=optional,javax.xml.bind.annotation.adapters;resolution:=optional,j
avax.xml.namespace,javax.xml.parsers,javax.xml.stream;resolution:=opt
ional,javax.xml.transform,javax.xml.transform.dom,javax.xml.transform
.sax,javax.xml.transform.stream,javax.xml.validation,javax.xml.xpath,
org.apache.xalan.xsltc.trax;resolution:=optional,org.osgi.framework;v
ersion="[1.4,2)",org.slf4j;version="[1.6,2)",org.w3c.dom,org.xml.sax
{code}
I cannot see that its being exported, so I would assume we are almost okay.
What if another bundle exports com.googlecode.concurrentlinkedhashmap,
then camel-core could potential pickup that instead from itself, or am I mistaking?
> Synchronized code causes long delays and hangs for big applications especially with Blueprint
> ---------------------------------------------------------------------------------------------
>
> Key: CAMEL-4345
> URL: https://issues.apache.org/jira/browse/CAMEL-4345
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.0
> Environment: Linux and Mac multicore machines
> Reporter: Jeff Genender
> Assignee: Hadrian Zbarcea
> Fix For: 2.9.0
>
> Attachments: CAMEL-4345.patch
>
>
> The DefaultCamelContext uses synchronized "endpoints" which ends up ultimately extending a LinkedHashMap through the LRUCache. The LinkedHashMap is obviously not thread safe, so it requires synchronized guards when accessing the endpoints object. This especially happens in the getEndpoint(s) calls in the DefaultCamelContext. In large systems with lots of routes and on multicore systems, dynamically created routes (and many routes) can cause long delays and hang for long times since route creation and the starting of the camel route can occur in unison with synchronization. In a blueprint container, such as Karaf, this can cause timeouts on the bundle and camel routes will appear to hang indefinately. Thread dumps show the hangs occur on the synchronized call in getEndpoint(s). The fix for this is to use concurrent apis as much as possible and remove the synchronized code. I refactored the LRUCache/LRUSoftCache to use Google's ConcurrentLinkedHashMap (ASL2 License http://code.google.com/p/concurrentlinkedhashmap) and removed the synchronized code that locks the endpoints object. This should remove the hangs since the locks are no longer required. Since COncurrentLinkedHashmap is not OSGi ready, I have shaded the classes in core. On my executions, all unit tests pass with this refactoring using the concurrent code. This should speed up Camel on multicore systems that have lots of routes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4345) Synchronized code causes long
delays and hangs for big applications especially with Blueprint
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087005#comment-13087005 ]
Claus Ibsen commented on CAMEL-4345:
------------------------------------
Nice patch.
By shading it into camel-core I like the fact that the camel-core dependency don't change.
However are there people who would not like shading this into camel-core?
And are there any Apache policy that would discourage this fact?
eg the JAR contains .class files which are not from Apache ?
If just the JDK had this class already. Even JDK7 dont have it, right?
> Synchronized code causes long delays and hangs for big applications especially with Blueprint
> ---------------------------------------------------------------------------------------------
>
> Key: CAMEL-4345
> URL: https://issues.apache.org/jira/browse/CAMEL-4345
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.0
> Environment: Linux and Mac multicore machines
> Reporter: Jeff Genender
> Attachments: CAMEL-4345.patch
>
>
> The DefaultCamelContext uses synchronized "endpoints" which ends up ultimately extending a LinkedHashMap through the LRUCache. The LinkedHashMap is obviously not thread safe, so it requires synchronized guards when accessing the endpoints object. This especially happens in the getEndpoint(s) calls in the DefaultCamelContext. In large systems with lots of routes and on multicore systems, dynamically created routes (and many routes) can cause long delays and hang for long times since route creation and the starting of the camel route can occur in unison with synchronization. In a blueprint container, such as Karaf, this can cause timeouts on the bundle and camel routes will appear to hang indefinately. Thread dumps show the hangs occur on the synchronized call in getEndpoint(s). The fix for this is to use concurrent apis as much as possible and remove the synchronized code. I refactored the LRUCache/LRUSoftCache to use Google's ConcurrentLinkedHashMap (ASL2 License http://code.google.com/p/concurrentlinkedhashmap) and removed the synchronized code that locks the endpoints object. This should remove the hangs since the locks are no longer required. Since COncurrentLinkedHashmap is not OSGi ready, I have shaded the classes in core. On my executions, all unit tests pass with this refactoring using the concurrent code. This should speed up Camel on multicore systems that have lots of routes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4345) Synchronized code causes long
delays and hangs for big applications especially with Blueprint
Posted by "Hadrian Zbarcea (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087427#comment-13087427 ]
Hadrian Zbarcea commented on CAMEL-4345:
----------------------------------------
Patch applied in r1159457 with thanks to Jeff. Jeff, we expect more from you :).
There are a few checkstyle issues in the patch I will correct in the next hour or so and mark this issue as resolved.
> Synchronized code causes long delays and hangs for big applications especially with Blueprint
> ---------------------------------------------------------------------------------------------
>
> Key: CAMEL-4345
> URL: https://issues.apache.org/jira/browse/CAMEL-4345
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.0
> Environment: Linux and Mac multicore machines
> Reporter: Jeff Genender
> Assignee: Hadrian Zbarcea
> Attachments: CAMEL-4345.patch
>
>
> The DefaultCamelContext uses synchronized "endpoints" which ends up ultimately extending a LinkedHashMap through the LRUCache. The LinkedHashMap is obviously not thread safe, so it requires synchronized guards when accessing the endpoints object. This especially happens in the getEndpoint(s) calls in the DefaultCamelContext. In large systems with lots of routes and on multicore systems, dynamically created routes (and many routes) can cause long delays and hang for long times since route creation and the starting of the camel route can occur in unison with synchronization. In a blueprint container, such as Karaf, this can cause timeouts on the bundle and camel routes will appear to hang indefinately. Thread dumps show the hangs occur on the synchronized call in getEndpoint(s). The fix for this is to use concurrent apis as much as possible and remove the synchronized code. I refactored the LRUCache/LRUSoftCache to use Google's ConcurrentLinkedHashMap (ASL2 License http://code.google.com/p/concurrentlinkedhashmap) and removed the synchronized code that locks the endpoints object. This should remove the hangs since the locks are no longer required. Since COncurrentLinkedHashmap is not OSGi ready, I have shaded the classes in core. On my executions, all unit tests pass with this refactoring using the concurrent code. This should speed up Camel on multicore systems that have lots of routes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4345) Synchronized code causes long
delays and hangs for big applications especially with Blueprint
Posted by "Edstrom Johan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087073#comment-13087073 ]
Edstrom Johan commented on CAMEL-4345:
--------------------------------------
At least the google code is ASF License, would that help some?
> Synchronized code causes long delays and hangs for big applications especially with Blueprint
> ---------------------------------------------------------------------------------------------
>
> Key: CAMEL-4345
> URL: https://issues.apache.org/jira/browse/CAMEL-4345
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.0
> Environment: Linux and Mac multicore machines
> Reporter: Jeff Genender
> Attachments: CAMEL-4345.patch
>
>
> The DefaultCamelContext uses synchronized "endpoints" which ends up ultimately extending a LinkedHashMap through the LRUCache. The LinkedHashMap is obviously not thread safe, so it requires synchronized guards when accessing the endpoints object. This especially happens in the getEndpoint(s) calls in the DefaultCamelContext. In large systems with lots of routes and on multicore systems, dynamically created routes (and many routes) can cause long delays and hang for long times since route creation and the starting of the camel route can occur in unison with synchronization. In a blueprint container, such as Karaf, this can cause timeouts on the bundle and camel routes will appear to hang indefinately. Thread dumps show the hangs occur on the synchronized call in getEndpoint(s). The fix for this is to use concurrent apis as much as possible and remove the synchronized code. I refactored the LRUCache/LRUSoftCache to use Google's ConcurrentLinkedHashMap (ASL2 License http://code.google.com/p/concurrentlinkedhashmap) and removed the synchronized code that locks the endpoints object. This should remove the hangs since the locks are no longer required. Since COncurrentLinkedHashmap is not OSGi ready, I have shaded the classes in core. On my executions, all unit tests pass with this refactoring using the concurrent code. This should speed up Camel on multicore systems that have lots of routes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4345) Synchronized code causes long
delays and hangs for big applications especially with Blueprint
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087076#comment-13087076 ]
Claus Ibsen commented on CAMEL-4345:
------------------------------------
Yeah the ASL license sure does :)
I think back in the days Spring framework included the aop alliance JAR in the Spring JAR directly as well.
> Synchronized code causes long delays and hangs for big applications especially with Blueprint
> ---------------------------------------------------------------------------------------------
>
> Key: CAMEL-4345
> URL: https://issues.apache.org/jira/browse/CAMEL-4345
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.0
> Environment: Linux and Mac multicore machines
> Reporter: Jeff Genender
> Attachments: CAMEL-4345.patch
>
>
> The DefaultCamelContext uses synchronized "endpoints" which ends up ultimately extending a LinkedHashMap through the LRUCache. The LinkedHashMap is obviously not thread safe, so it requires synchronized guards when accessing the endpoints object. This especially happens in the getEndpoint(s) calls in the DefaultCamelContext. In large systems with lots of routes and on multicore systems, dynamically created routes (and many routes) can cause long delays and hang for long times since route creation and the starting of the camel route can occur in unison with synchronization. In a blueprint container, such as Karaf, this can cause timeouts on the bundle and camel routes will appear to hang indefinately. Thread dumps show the hangs occur on the synchronized call in getEndpoint(s). The fix for this is to use concurrent apis as much as possible and remove the synchronized code. I refactored the LRUCache/LRUSoftCache to use Google's ConcurrentLinkedHashMap (ASL2 License http://code.google.com/p/concurrentlinkedhashmap) and removed the synchronized code that locks the endpoints object. This should remove the hangs since the locks are no longer required. Since COncurrentLinkedHashmap is not OSGi ready, I have shaded the classes in core. On my executions, all unit tests pass with this refactoring using the concurrent code. This should speed up Camel on multicore systems that have lots of routes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4345) Synchronized code causes long
delays and hangs for big applications especially with Blueprint
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087606#comment-13087606 ]
Claus Ibsen commented on CAMEL-4345:
------------------------------------
Did anyone create a ticket at Google CLHM to ask for osgify the distribution, as Ben said we should do?
> Synchronized code causes long delays and hangs for big applications especially with Blueprint
> ---------------------------------------------------------------------------------------------
>
> Key: CAMEL-4345
> URL: https://issues.apache.org/jira/browse/CAMEL-4345
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.0
> Environment: Linux and Mac multicore machines
> Reporter: Jeff Genender
> Assignee: Hadrian Zbarcea
> Fix For: 2.9.0
>
> Attachments: CAMEL-4345.patch
>
>
> The DefaultCamelContext uses synchronized "endpoints" which ends up ultimately extending a LinkedHashMap through the LRUCache. The LinkedHashMap is obviously not thread safe, so it requires synchronized guards when accessing the endpoints object. This especially happens in the getEndpoint(s) calls in the DefaultCamelContext. In large systems with lots of routes and on multicore systems, dynamically created routes (and many routes) can cause long delays and hang for long times since route creation and the starting of the camel route can occur in unison with synchronization. In a blueprint container, such as Karaf, this can cause timeouts on the bundle and camel routes will appear to hang indefinately. Thread dumps show the hangs occur on the synchronized call in getEndpoint(s). The fix for this is to use concurrent apis as much as possible and remove the synchronized code. I refactored the LRUCache/LRUSoftCache to use Google's ConcurrentLinkedHashMap (ASL2 License http://code.google.com/p/concurrentlinkedhashmap) and removed the synchronized code that locks the endpoints object. This should remove the hangs since the locks are no longer required. Since COncurrentLinkedHashmap is not OSGi ready, I have shaded the classes in core. On my executions, all unit tests pass with this refactoring using the concurrent code. This should speed up Camel on multicore systems that have lots of routes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Updated] (CAMEL-4345) Synchronized code causes long delays and hangs for big applications especially with Blueprint
Posted by Johan Edstrom <se...@gmail.com>.
Very unscientifically testing this a 200 route count with 20 prepped exchanges went from 20 secs to 8 seconds, more interestingly though a 400 route count with 500 exchanges each that previously caused either a hang or a oome now takes around 40 seconds with the majority of time spent on route building.
This is definitely far more concurrent and much closer to very dynamic.
/je
On Aug 18, 2011, at 20:00, "Hadrian Zbarcea (JIRA)" <ji...@apache.org> wrote:
>
> [ https://issues.apache.org/jira/browse/CAMEL-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Hadrian Zbarcea updated CAMEL-4345:
> -----------------------------------
>
> Fix Version/s: 2.9.0
>
>> Synchronized code causes long delays and hangs for big applications especially with Blueprint
>> ---------------------------------------------------------------------------------------------
>>
>> Key: CAMEL-4345
>> URL: https://issues.apache.org/jira/browse/CAMEL-4345
>> Project: Camel
>> Issue Type: Bug
>> Components: camel-core
>> Affects Versions: 2.8.0
>> Environment: Linux and Mac multicore machines
>> Reporter: Jeff Genender
>> Assignee: Hadrian Zbarcea
>> Fix For: 2.9.0
>>
>> Attachments: CAMEL-4345.patch
>>
>>
>> The DefaultCamelContext uses synchronized "endpoints" which ends up ultimately extending a LinkedHashMap through the LRUCache. The LinkedHashMap is obviously not thread safe, so it requires synchronized guards when accessing the endpoints object. This especially happens in the getEndpoint(s) calls in the DefaultCamelContext. In large systems with lots of routes and on multicore systems, dynamically created routes (and many routes) can cause long delays and hang for long times since route creation and the starting of the camel route can occur in unison with synchronization. In a blueprint container, such as Karaf, this can cause timeouts on the bundle and camel routes will appear to hang indefinately. Thread dumps show the hangs occur on the synchronized call in getEndpoint(s). The fix for this is to use concurrent apis as much as possible and remove the synchronized code. I refactored the LRUCache/LRUSoftCache to use Google's ConcurrentLinkedHashMap (ASL2 License http://code.google.com/p/concurrentlinkedhashmap) and removed the synchronized code that locks the endpoints object. This should remove the hangs since the locks are no longer required. Since COncurrentLinkedHashmap is not OSGi ready, I have shaded the classes in core. On my executions, all unit tests pass with this refactoring using the concurrent code. This should speed up Camel on multicore systems that have lots of routes.
>
> --
> This message is automatically generated by JIRA.
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
[jira] [Updated] (CAMEL-4345) Synchronized code causes long delays
and hangs for big applications especially with Blueprint
Posted by "Hadrian Zbarcea (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hadrian Zbarcea updated CAMEL-4345:
-----------------------------------
Fix Version/s: 2.9.0
> Synchronized code causes long delays and hangs for big applications especially with Blueprint
> ---------------------------------------------------------------------------------------------
>
> Key: CAMEL-4345
> URL: https://issues.apache.org/jira/browse/CAMEL-4345
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.0
> Environment: Linux and Mac multicore machines
> Reporter: Jeff Genender
> Assignee: Hadrian Zbarcea
> Fix For: 2.9.0
>
> Attachments: CAMEL-4345.patch
>
>
> The DefaultCamelContext uses synchronized "endpoints" which ends up ultimately extending a LinkedHashMap through the LRUCache. The LinkedHashMap is obviously not thread safe, so it requires synchronized guards when accessing the endpoints object. This especially happens in the getEndpoint(s) calls in the DefaultCamelContext. In large systems with lots of routes and on multicore systems, dynamically created routes (and many routes) can cause long delays and hang for long times since route creation and the starting of the camel route can occur in unison with synchronization. In a blueprint container, such as Karaf, this can cause timeouts on the bundle and camel routes will appear to hang indefinately. Thread dumps show the hangs occur on the synchronized call in getEndpoint(s). The fix for this is to use concurrent apis as much as possible and remove the synchronized code. I refactored the LRUCache/LRUSoftCache to use Google's ConcurrentLinkedHashMap (ASL2 License http://code.google.com/p/concurrentlinkedhashmap) and removed the synchronized code that locks the endpoints object. This should remove the hangs since the locks are no longer required. Since COncurrentLinkedHashmap is not OSGi ready, I have shaded the classes in core. On my executions, all unit tests pass with this refactoring using the concurrent code. This should speed up Camel on multicore systems that have lots of routes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CAMEL-4345) Synchronized code causes long delays
and hangs for big applications especially with Blueprint
Posted by "Hadrian Zbarcea (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hadrian Zbarcea resolved CAMEL-4345.
------------------------------------
Resolution: Fixed
Fixed checkstyle and also removed the OSGi export of the google stuff. The intent I believe was to make it accessible to components, but we can do that via a utility factory if/when needed.
Thanks Jeff!
> Synchronized code causes long delays and hangs for big applications especially with Blueprint
> ---------------------------------------------------------------------------------------------
>
> Key: CAMEL-4345
> URL: https://issues.apache.org/jira/browse/CAMEL-4345
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.0
> Environment: Linux and Mac multicore machines
> Reporter: Jeff Genender
> Assignee: Hadrian Zbarcea
> Attachments: CAMEL-4345.patch
>
>
> The DefaultCamelContext uses synchronized "endpoints" which ends up ultimately extending a LinkedHashMap through the LRUCache. The LinkedHashMap is obviously not thread safe, so it requires synchronized guards when accessing the endpoints object. This especially happens in the getEndpoint(s) calls in the DefaultCamelContext. In large systems with lots of routes and on multicore systems, dynamically created routes (and many routes) can cause long delays and hang for long times since route creation and the starting of the camel route can occur in unison with synchronization. In a blueprint container, such as Karaf, this can cause timeouts on the bundle and camel routes will appear to hang indefinately. Thread dumps show the hangs occur on the synchronized call in getEndpoint(s). The fix for this is to use concurrent apis as much as possible and remove the synchronized code. I refactored the LRUCache/LRUSoftCache to use Google's ConcurrentLinkedHashMap (ASL2 License http://code.google.com/p/concurrentlinkedhashmap) and removed the synchronized code that locks the endpoints object. This should remove the hangs since the locks are no longer required. Since COncurrentLinkedHashmap is not OSGi ready, I have shaded the classes in core. On my executions, all unit tests pass with this refactoring using the concurrent code. This should speed up Camel on multicore systems that have lots of routes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4345) Synchronized code causes long
delays and hangs for big applications especially with Blueprint
Posted by "Jeff Genender (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087662#comment-13087662 ]
Jeff Genender commented on CAMEL-4345:
--------------------------------------
Claus, yes I opened an issue to OSGI-ify ConcurrentLinkedHashMap.
Good catch on the import... it should not be in there.... clearly the Felix runs before the shade ;-)
The fix is in the camel-core pom, right under the <camel.osgi.import>, add this:
{code}
!com.googlecode.concurrentlinkedhashmap,
{code}
That will fix it.
> Synchronized code causes long delays and hangs for big applications especially with Blueprint
> ---------------------------------------------------------------------------------------------
>
> Key: CAMEL-4345
> URL: https://issues.apache.org/jira/browse/CAMEL-4345
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.0
> Environment: Linux and Mac multicore machines
> Reporter: Jeff Genender
> Assignee: Hadrian Zbarcea
> Fix For: 2.9.0
>
> Attachments: CAMEL-4345.patch
>
>
> The DefaultCamelContext uses synchronized "endpoints" which ends up ultimately extending a LinkedHashMap through the LRUCache. The LinkedHashMap is obviously not thread safe, so it requires synchronized guards when accessing the endpoints object. This especially happens in the getEndpoint(s) calls in the DefaultCamelContext. In large systems with lots of routes and on multicore systems, dynamically created routes (and many routes) can cause long delays and hang for long times since route creation and the starting of the camel route can occur in unison with synchronization. In a blueprint container, such as Karaf, this can cause timeouts on the bundle and camel routes will appear to hang indefinately. Thread dumps show the hangs occur on the synchronized call in getEndpoint(s). The fix for this is to use concurrent apis as much as possible and remove the synchronized code. I refactored the LRUCache/LRUSoftCache to use Google's ConcurrentLinkedHashMap (ASL2 License http://code.google.com/p/concurrentlinkedhashmap) and removed the synchronized code that locks the endpoints object. This should remove the hangs since the locks are no longer required. Since COncurrentLinkedHashmap is not OSGi ready, I have shaded the classes in core. On my executions, all unit tests pass with this refactoring using the concurrent code. This should speed up Camel on multicore systems that have lots of routes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (CAMEL-4345) Synchronized code causes long
delays and hangs for big applications especially with Blueprint
Posted by "Hadrian Zbarcea (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on CAMEL-4345 started by Hadrian Zbarcea.
> Synchronized code causes long delays and hangs for big applications especially with Blueprint
> ---------------------------------------------------------------------------------------------
>
> Key: CAMEL-4345
> URL: https://issues.apache.org/jira/browse/CAMEL-4345
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.0
> Environment: Linux and Mac multicore machines
> Reporter: Jeff Genender
> Assignee: Hadrian Zbarcea
> Attachments: CAMEL-4345.patch
>
>
> The DefaultCamelContext uses synchronized "endpoints" which ends up ultimately extending a LinkedHashMap through the LRUCache. The LinkedHashMap is obviously not thread safe, so it requires synchronized guards when accessing the endpoints object. This especially happens in the getEndpoint(s) calls in the DefaultCamelContext. In large systems with lots of routes and on multicore systems, dynamically created routes (and many routes) can cause long delays and hang for long times since route creation and the starting of the camel route can occur in unison with synchronization. In a blueprint container, such as Karaf, this can cause timeouts on the bundle and camel routes will appear to hang indefinately. Thread dumps show the hangs occur on the synchronized call in getEndpoint(s). The fix for this is to use concurrent apis as much as possible and remove the synchronized code. I refactored the LRUCache/LRUSoftCache to use Google's ConcurrentLinkedHashMap (ASL2 License http://code.google.com/p/concurrentlinkedhashmap) and removed the synchronized code that locks the endpoints object. This should remove the hangs since the locks are no longer required. Since COncurrentLinkedHashmap is not OSGi ready, I have shaded the classes in core. On my executions, all unit tests pass with this refactoring using the concurrent code. This should speed up Camel on multicore systems that have lots of routes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira