You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "Zach Calvert (Created) (JIRA)" <ji...@apache.org> on 2012/03/02 17:37:56 UTC
[jira] [Created] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Bug: Unique Endpoints Leaking in DefaultInflightRepository
----------------------------------------------------------
Key: CAMEL-5058
URL: https://issues.apache.org/jira/browse/CAMEL-5058
Project: Camel
Issue Type: Bug
Components: camel-core
Affects Versions: 2.9.0
Reporter: Zach Calvert
If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Zach Calvert (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224374#comment-13224374 ]
Zach Calvert commented on CAMEL-5058:
-------------------------------------
Done
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Attachments: fixLeak.diff, fixLeak.diff
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Claus Ibsen (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen resolved CAMEL-5058.
--------------------------------
Resolution: Fixed
Thanks for the patch.
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Attachments: fixLeak.diff, fixLeak.diff
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Babak Vahdat (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221644#comment-13221644 ]
Babak Vahdat commented on CAMEL-5058:
-------------------------------------
Just for the sake of not losing the revision history:
trunk: http://svn.apache.org/viewvc?view=revision&revision=1296653
2.9.x: http://svn.apache.org/viewvc?view=revision&revision=1296655
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Claus Ibsen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223140#comment-13223140 ]
Claus Ibsen commented on CAMEL-5058:
------------------------------------
Notice ideally the inflight registry would just track for routes only. But the producer template uses an UoW as well for some reason we had to do a while ago. So it will also enlist itself in the inflight registry (well its in fact also an inflight message you may say).
So we can either ensure to remove from the inflight map if it become zero as you suggest. This may require a bit of locking, as you need to remove and compare.
Alternative, as the map, does not need to be removed in real time, we can have a background thread, doing the cleanup in a scheduled fashion.
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Babak Vahdat (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225120#comment-13225120 ]
Babak Vahdat commented on CAMEL-5058:
-------------------------------------
I did polish the provided unit-test a bit mostly because of the required visibility of the failed flag (volatile keyword).
Especially required while running tests on the multi-core boxes.
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Attachments: fixLeak.diff, fixLeak.diff
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Zach Calvert (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zach Calvert updated CAMEL-5058:
--------------------------------
Attachment: fixLeak.diff
The diff to fix the leak. Note the unit test runs 2,000,000 operations to validate the fix. It runs in 1 second on my computer, so the bottleneck due to the synchronized is limited.
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Attachments: fixLeak.diff
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Zach Calvert (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223224#comment-13223224 ]
Zach Calvert commented on CAMEL-5058:
-------------------------------------
Claus, that is correct that I'm using the producer template to send the messages. I actually have a solution that I'm currently testing, but I've got to wait for our legal department to review the change to submit back to the community (we've got a big process I have to follow).
Hopefully I will be able to submit the patch, including a unit test, to validate the change.
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Zach Calvert (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222564#comment-13222564 ]
Zach Calvert commented on CAMEL-5058:
-------------------------------------
This actually doesn't solve the problem. There is still opportunity to leak references if the endpoint creation isn't part of a route.
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Claus Ibsen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223074#comment-13223074 ]
Claus Ibsen commented on CAMEL-5058:
------------------------------------
Zach,
Yes I suspected there was still a problem for you. But I didn't have a chance to ask how you send the messages. So just to be sure, you use a producer template to send to dynamic endpoint uris.
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Zach Calvert (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225201#comment-13225201 ]
Zach Calvert commented on CAMEL-5058:
-------------------------------------
Yea, my bad. I remembered I had some cleanup to do on that hack-n-slash unit test last night. Ugly, ugly unit test...
Thanks for the cleanup Babak!
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Attachments: fixLeak.diff, fixLeak.diff
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Claus Ibsen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224071#comment-13224071 ]
Claus Ibsen commented on CAMEL-5058:
------------------------------------
Zach thanks for the patch. Can you re-attach the patch and make sure the mark [x] in grant license to ASF in the file attachment wizard. We need this to be able to accept your patch.
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Attachments: fixLeak.diff
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Zach Calvert (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222566#comment-13222566 ]
Zach Calvert commented on CAMEL-5058:
-------------------------------------
I'm not quite sure this fixes our problem. We aren't creating a true route. We are using a ProducerTemplate to send a Message to an Endpoint, and the Endpoint is unique, created at runtime, and would essentially create a new entry in the ConcurrentHashMap stored in the DefaultInflightRepository.
Ideally, when remove is called with an Exchange, if the reference count becomes 0, then the entry is removed from the Map.
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Zach Calvert (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zach Calvert updated CAMEL-5058:
--------------------------------
Comment: was deleted
(was: This actually doesn't solve the problem. There is still opportunity to leak references if the endpoint creation isn't part of a route. )
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Claus Ibsen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221623#comment-13221623 ]
Claus Ibsen commented on CAMEL-5058:
------------------------------------
I have committed a fix that removes entries from the registry when removing a route.
We need to keep it endpoint key based due producer template setup a UoW which essentially is a exchange created outside a route, and therefore we do not have a routeId at this given moment.
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Claus Ibsen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221555#comment-13221555 ]
Claus Ibsen commented on CAMEL-5058:
------------------------------------
The inflight registry should be route based instead of endpoint, as that is what the graceful shutdown really needs.
So instead of being endpoint based, we can switch to be route id, based instead.
Can you post some more details about your use-case, I wonder why you get so many unique endpoints in the registry, as its consumer based. Are you adding a lot of new routes or the likes?
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Fix For: 2.10.0, 2.9.2
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Zach Calvert (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222358#comment-13222358 ]
Zach Calvert commented on CAMEL-5058:
-------------------------------------
We construct endpoints at runtime to send a message to the XMPP subsystem. It isn't ideal, but it works for us. With the memory leak, however, it has to be restarted once a week or so.
Thanks for the fast fix!
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Zach Calvert (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zach Calvert updated CAMEL-5058:
--------------------------------
Attachment: fixLeak.diff
Re-attaching code with agreement to allow the Apache License
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Attachments: fixLeak.diff, fixLeak.diff
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Claus Ibsen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221814#comment-13221814 ]
Claus Ibsen commented on CAMEL-5058:
------------------------------------
Ah I was 1000 numbers behind in the commit message :)
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Claus Ibsen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225277#comment-13225277 ]
Claus Ibsen commented on CAMEL-5058:
------------------------------------
I improved this to avoid the synchronization in CAMEL-5057. As well only to track in flight per routes, as per endpoint is not needed.
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Attachments: fixLeak.diff, fixLeak.diff
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Claus Ibsen (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen reassigned CAMEL-5058:
----------------------------------
Assignee: Claus Ibsen
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CAMEL-5058) Bug: Unique Endpoints Leaking in
DefaultInflightRepository
Posted by "Claus Ibsen (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen updated CAMEL-5058:
-------------------------------
Fix Version/s: 2.9.2
2.10.0
> Bug: Unique Endpoints Leaking in DefaultInflightRepository
> ----------------------------------------------------------
>
> Key: CAMEL-5058
> URL: https://issues.apache.org/jira/browse/CAMEL-5058
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.9.0
> Reporter: Zach Calvert
> Assignee: Claus Ibsen
> Fix For: 2.10.0, 2.9.2
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> If you have an endpoint protocol which uses unique URIs you will leak Strings in the HashMap stored in the DefaultInflightRepository (org.apache.camel.impl.DefaultInflightRepository)
> It seems there is a reference counting scheme in place, but it doesn't do a remove until the "stop" method is called to shut the system down. We are running XMPP endpoints, which use a protocol like xmpp://someaccount@domain/password?to=someOtherAccount
> When there are 10 million accounts, not all of which are active, but all of which may message at some time or another, no references are removed to the endpointCount.
> When the count becomes 0, the reference should be removed and the size method will still return the appropriate result.
> Please be careful in the implementation to synchronize on some object (perhaps the AtomicInteger) reflecting a read/write lock on the endpoint count modification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira