You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by "Daniel Lord (Created) (JIRA)" <ji...@apache.org> on 2011/11/29 03:05:41 UTC
[jira] [Created] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Creating a new ZooKeeper client can leak file handles
-----------------------------------------------------
Key: ZOOKEEPER-1309
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
Project: ZooKeeper
Issue Type: Bug
Components: java client
Affects Versions: 3.3.4
Reporter: Daniel Lord
Priority: Critical
If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
final Selector selector = Selector.open();
If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164056#comment-13164056 ]
Hadoop QA commented on ZOOKEEPER-1309:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12506345/zk-1309-1.patch
against trunk revision 1208979.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/806//console
This message is automatically generated.
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Assignee: Daniel Lord
> Priority: Critical
> Fix For: 3.5.0, 3.4.1, 3.3.5
>
> Attachments: zk-1309-1.patch, zk-1309-1.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Daniel Lord (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189261#comment-13189261 ]
Daniel Lord commented on ZOOKEEPER-1309:
----------------------------------------
Hi Mahadev, the latest patch is from the root check out directory -- the one that houses both trunk and branches. Is that still not correct? The patch was for a branch but I can make the same change to the trunk if needed.
We've been running the patch for a while on production and it works nicely.
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Assignee: Daniel Lord
> Priority: Critical
> Fix For: 3.4.3, 3.5.0
>
> Attachments: zk-1309-1.patch, zk-1309-1.patch, zk-1309-1.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Daniel Lord (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Lord updated ZOOKEEPER-1309:
-----------------------------------
Attachment: zk-1309-1.patch
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Priority: Critical
> Attachments: zk-1309-1.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Patrick Hunt (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191623#comment-13191623 ]
Patrick Hunt commented on ZOOKEEPER-1309:
-----------------------------------------
No. That was my point earlier about how this is just kluged together (if someone wants to help on this lmk). The bot uses the last patch you attached to the jira and applies it to trunk regardless of any other setting.
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Assignee: Daniel Lord
> Priority: Critical
> Fix For: 3.4.3, 3.5.0
>
> Attachments: zk-1309-1.patch, zk-1309-1.patch, zk-1309-1.patch, zk-1309-3.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Daniel Lord (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Lord updated ZOOKEEPER-1309:
-----------------------------------
Attachment: zk-1309-3.patch
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Assignee: Daniel Lord
> Priority: Critical
> Fix For: 3.4.3, 3.5.0
>
> Attachments: zk-1309-1.patch, zk-1309-1.patch, zk-1309-1.patch, zk-1309-3.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Patrick Hunt (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191542#comment-13191542 ]
Patrick Hunt commented on ZOOKEEPER-1309:
-----------------------------------------
Looks like trunk (auto patch testing only tests against trunk) is different enough from 3.3 that your now properly formatted patch is not applying due to conflicts. Typically what we do is create patches for each branch in this case - say we should include this in 3.3/3.4/3.5 you might need to create a separate patch for 3.4/3.5 than 3.3 (in some cases we end up with 3 patches). Also, we typically name the patches ZOOKEEPER-####.patch (ZOOKEEPER-####_br34.patch, etc...), jira handles uploading new versions with the same name properly. Also - qa bot always uses the most recent patch. So you need to upload 3.3/3.4 before 3.5 patch if you want to patch to be auto-tested correctly. (sorry about that but it's pretty bare bones and that's how it currently works).
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Assignee: Daniel Lord
> Priority: Critical
> Fix For: 3.4.3, 3.5.0
>
> Attachments: zk-1309-1.patch, zk-1309-1.patch, zk-1309-1.patch, zk-1309-3.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Daniel Lord (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160513#comment-13160513 ]
Daniel Lord commented on ZOOKEEPER-1309:
----------------------------------------
Sorry Mahadev, I may have jumped the gun on this patch a little bit. I'm working on getting corporate approval to submit patches for both bug fixes and enhancements now.
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Priority: Critical
> Attachments: zk-1309-1.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Daniel Lord (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191604#comment-13191604 ]
Daniel Lord commented on ZOOKEEPER-1309:
----------------------------------------
OK and then when I select "Patch available" and select the fix version the qabot will attempt the patch on that version(s)?
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Assignee: Daniel Lord
> Priority: Critical
> Fix For: 3.4.3, 3.5.0
>
> Attachments: zk-1309-1.patch, zk-1309-1.patch, zk-1309-1.patch, zk-1309-3.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Daniel Lord (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Lord updated ZOOKEEPER-1309:
-----------------------------------
Attachment: zk-1309-1.patch
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Priority: Critical
> Attachments: zk-1309-1.patch, zk-1309-1.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Patrick Hunt (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-1309:
------------------------------------
Fix Version/s: 3.3.5
3.4.1
3.5.0
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Assignee: Daniel Lord
> Priority: Critical
> Fix For: 3.5.0, 3.4.1, 3.3.5
>
> Attachments: zk-1309-1.patch, zk-1309-1.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191493#comment-13191493 ]
Hadoop QA commented on ZOOKEEPER-1309:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12511557/zk-1309-3.patch
against trunk revision 1234974.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/913//console
This message is automatically generated.
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Assignee: Daniel Lord
> Priority: Critical
> Fix For: 3.4.3, 3.5.0
>
> Attachments: zk-1309-1.patch, zk-1309-1.patch, zk-1309-1.patch, zk-1309-3.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Camille Fournier (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13216963#comment-13216963 ]
Camille Fournier commented on ZOOKEEPER-1309:
---------------------------------------------
Ran tests and they all passed, so I'm gonna check this in to 3.3.5.
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Assignee: Daniel Lord
> Priority: Critical
> Fix For: 3.3.5
>
> Attachments: zk-1309-1.patch, zk-1309-1.patch, zk-1309-1.patch, zk-1309-3.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Daniel Lord (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191803#comment-13191803 ]
Daniel Lord commented on ZOOKEEPER-1309:
----------------------------------------
It looks to me that this issue (and fix) don't apply to branches-3.4 and trunk. The structure of the zookeeper client seems to have changed a lot in 3.4. For example the ClientCnxn used to leak file handles because it initialized the Selector.open() before it attempted to resolve any InetAddresses for the ensemble. This is no longer the case. If you look at the constructor of ZooKeeper now the StaticHostProvider constructor is responsible for resolving the InetAddresses. If any of the addresses are bad then StaticHostProvider will throw an UnknownHostException and the whole ZooKeeper handle will fall apart and won't leak any resources.
My original test for this was to connect to a single host named "qwerty" where "qwerty" does not resolve to a valid host and to watch the file handles grow very quickly. I can see that in this test there is no file handle leak using the 3.4 zookeeper client.
Objections on this jira applying only to 3.3.x?
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Assignee: Daniel Lord
> Priority: Critical
> Fix For: 3.4.3, 3.5.0
>
> Attachments: zk-1309-1.patch, zk-1309-1.patch, zk-1309-1.patch, zk-1309-3.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159629#comment-13159629 ]
Hadoop QA commented on ZOOKEEPER-1309:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12505555/zk-1309-1.patch
against trunk revision 1202557.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/799//console
This message is automatically generated.
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Priority: Critical
> Attachments: zk-1309-1.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Daniel Lord (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163266#comment-13163266 ]
Daniel Lord commented on ZOOKEEPER-1309:
----------------------------------------
Hi Mahadev, I've received corporate approval to contribute this patch. What did you need me to do with the patch in order to make it official?
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Priority: Critical
> Attachments: zk-1309-1.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Patrick Hunt (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191592#comment-13191592 ]
Patrick Hunt commented on ZOOKEEPER-1309:
-----------------------------------------
qabot looks for patches of jiras in the "patch available" state (ones you submitted) and automatically verifies the patch when it (the most recent patch) changes (iirc on the change part, takes 5-10 minutes to notice, if not just cancel and re-submit).
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Assignee: Daniel Lord
> Priority: Critical
> Fix For: 3.4.3, 3.5.0
>
> Attachments: zk-1309-1.patch, zk-1309-1.patch, zk-1309-1.patch, zk-1309-3.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Daniel Lord (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Lord updated ZOOKEEPER-1309:
-----------------------------------
Attachment: zk-1309-1.patch
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Assignee: Daniel Lord
> Priority: Critical
> Fix For: 3.5.0, 3.3.5, 3.4.2
>
> Attachments: zk-1309-1.patch, zk-1309-1.patch, zk-1309-1.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Daniel Lord (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191590#comment-13191590 ]
Daniel Lord commented on ZOOKEEPER-1309:
----------------------------------------
OK thanks for the help Patrick. Just to clarify how do I make the qa bot apply to a branch build? I'll check out the other branches (and trunk) to see where a new patch needs to be created for those versions.
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Assignee: Daniel Lord
> Priority: Critical
> Fix For: 3.4.3, 3.5.0
>
> Attachments: zk-1309-1.patch, zk-1309-1.patch, zk-1309-1.patch, zk-1309-3.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160254#comment-13160254 ]
Mahadev konar commented on ZOOKEEPER-1309:
------------------------------------------
Daniel,
Can you please reattach the patch and grant license to apache for inclusion?
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Priority: Critical
> Attachments: zk-1309-1.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Patrick Hunt (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt reassigned ZOOKEEPER-1309:
---------------------------------------
Assignee: Daniel Lord
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Assignee: Daniel Lord
> Priority: Critical
> Attachments: zk-1309-1.patch, zk-1309-1.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1309) Creating a new ZooKeeper client
can leak file handles
Posted by "Patrick Hunt (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191416#comment-13191416 ]
Patrick Hunt commented on ZOOKEEPER-1309:
-----------------------------------------
Hi Daniel the issue is this:
bq. branches/branch-3.3/src/java/main/org/apache/zookeeper/ClientCnxn.java
our processes assume everything is rooted in a toplevel directory that contains "src" (e.g. "trunk"), in this case that's branches/branch-3.3 directory
see: https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToContribute
> Creating a new ZooKeeper client can leak file handles
> -----------------------------------------------------
>
> Key: ZOOKEEPER-1309
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1309
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
> Affects Versions: 3.3.4
> Reporter: Daniel Lord
> Assignee: Daniel Lord
> Priority: Critical
> Fix For: 3.4.3, 3.5.0
>
> Attachments: zk-1309-1.patch, zk-1309-1.patch, zk-1309-1.patch
>
>
> If there is an IOException thrown by the constructor of ClientCnxn then file handles are leaked because of the initialization of the Selector which is never closed.
> final Selector selector = Selector.open();
> If there is an abnormal exit from the constructor then the Selector is not closed and file handles are leaked. You can easily see this by setting the hosts string to garbage ("qwerty", "asdf", etc.) and then try to open a new ZooKeeper connection. I've observed the same behavior in production when there were DNS issues where the host names of the ensemble can no longer be resolved and the application servers quickly run out of handles attempting to (re)connect to zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira