You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Vijay (JIRA)" <ji...@apache.org> on 2011/09/25 19:26:26 UTC
[jira] [Created] (CASSANDRA-3257) Enabling SSL on a fairly light
cluster leaks Open files.
Enabling SSL on a fairly light cluster leaks Open files.
--------------------------------------------------------
Key: CASSANDRA-3257
URL: https://issues.apache.org/jira/browse/CASSANDRA-3257
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8.6
Environment: JVM on CentOS
Reporter: Vijay
Priority: Minor
To reproduce:
Enable SSL encryption and let the server be ideal for a day or so you will see the below....
[vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
16333
Sun Sep 25 17:23:29 UTC 2011
[vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ java -jar cmdline-jmxclient-0.10.3.jar - localhost:7501 java.lang:type=Memory gc
[vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
64
Sun Sep 25 17:23:53 UTC 2011
[vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$
After running GC manually the issue goes away.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3257) Enabling SSL on a fairly light
cluster leaks Open files.
Posted by "Vijay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13114747#comment-13114747 ]
Vijay commented on CASSANDRA-3257:
----------------------------------
This issue elevates when a node is down.
> Enabling SSL on a fairly light cluster leaks Open files.
> --------------------------------------------------------
>
> Key: CASSANDRA-3257
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3257
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.6
> Environment: JVM on CentOS
> Reporter: Vijay
> Priority: Minor
>
> To reproduce:
> Enable SSL encryption and let the server be idle for a day or so you will see the below....
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
> 16333
> Sun Sep 25 17:23:29 UTC 2011
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ java -jar cmdline-jmxclient-0.10.3.jar - localhost:7501 java.lang:type=Memory gc
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
> 64
> Sun Sep 25 17:23:53 UTC 2011
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$
> After running GC manually the issue goes away.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-3257) Enabling SSL on a fairly light
cluster leaks Open files.
Posted by "Vijay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vijay reassigned CASSANDRA-3257:
--------------------------------
Assignee: Vijay
> Enabling SSL on a fairly light cluster leaks Open files.
> --------------------------------------------------------
>
> Key: CASSANDRA-3257
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3257
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.6
> Environment: JVM on CentOS
> Reporter: Vijay
> Assignee: Vijay
> Priority: Minor
>
> To reproduce:
> Enable SSL encryption and let the server be idle for a day or so you will see the below....
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
> 16333
> Sun Sep 25 17:23:29 UTC 2011
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ java -jar cmdline-jmxclient-0.10.3.jar - localhost:7501 java.lang:type=Memory gc
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
> 64
> Sun Sep 25 17:23:53 UTC 2011
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$
> After running GC manually the issue goes away.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3257) Enabling SSL on a fairly light
cluster leaks Open files.
Posted by "Jonathan Ellis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115021#comment-13115021 ]
Jonathan Ellis commented on CASSANDRA-3257:
-------------------------------------------
should use FileUtils.closeQuietly so if one close throws, the other still happens
> Enabling SSL on a fairly light cluster leaks Open files.
> --------------------------------------------------------
>
> Key: CASSANDRA-3257
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3257
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.6, 1.0.0
> Environment: JVM on CentOS
> Reporter: Vijay
> Assignee: Vijay
> Priority: Minor
> Fix For: 0.8.7, 1.0.0
>
> Attachments: 0001-ssl-open-file-issue.patch
>
>
> To reproduce:
> Enable SSL encryption and let the server be idle for a day or so you will see the below....
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
> 16333
> Sun Sep 25 17:23:29 UTC 2011
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ java -jar cmdline-jmxclient-0.10.3.jar - localhost:7501 java.lang:type=Memory gc
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
> 64
> Sun Sep 25 17:23:53 UTC 2011
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$
> After running GC manually the issue goes away.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3257) Enabling SSL on a fairly light
cluster leaks Open files.
Posted by "Vijay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vijay updated CASSANDRA-3257:
-----------------------------
Attachment: 0001-ssl-open-file-issue.patch
Closing the FIS fixes the issue (in SSLFactory).
> Enabling SSL on a fairly light cluster leaks Open files.
> --------------------------------------------------------
>
> Key: CASSANDRA-3257
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3257
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.6, 1.0.0
> Environment: JVM on CentOS
> Reporter: Vijay
> Assignee: Vijay
> Priority: Minor
> Fix For: 0.8.7, 1.0.0
>
> Attachments: 0001-ssl-open-file-issue.patch
>
>
> To reproduce:
> Enable SSL encryption and let the server be idle for a day or so you will see the below....
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
> 16333
> Sun Sep 25 17:23:29 UTC 2011
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ java -jar cmdline-jmxclient-0.10.3.jar - localhost:7501 java.lang:type=Memory gc
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
> 64
> Sun Sep 25 17:23:53 UTC 2011
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$
> After running GC manually the issue goes away.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3257) Enabling SSL on a fairly light
cluster leaks Open files.
Posted by "Jonathan Ellis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115231#comment-13115231 ]
Jonathan Ellis commented on CASSANDRA-3257:
-------------------------------------------
done in r1176204. also cleaned up formatting.
> Enabling SSL on a fairly light cluster leaks Open files.
> --------------------------------------------------------
>
> Key: CASSANDRA-3257
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3257
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.6, 1.0.0
> Environment: JVM on CentOS
> Reporter: Vijay
> Assignee: Vijay
> Priority: Minor
> Fix For: 0.8.7, 1.0.0
>
> Attachments: 0001-ssl-open-file-issue.patch
>
>
> To reproduce:
> Enable SSL encryption and let the server be idle for a day or so you will see the below....
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
> 16333
> Sun Sep 25 17:23:29 UTC 2011
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ java -jar cmdline-jmxclient-0.10.3.jar - localhost:7501 java.lang:type=Memory gc
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
> 64
> Sun Sep 25 17:23:53 UTC 2011
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$
> After running GC manually the issue goes away.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3257) Enabling SSL on a fairly light
cluster leaks Open files.
Posted by "Vijay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vijay updated CASSANDRA-3257:
-----------------------------
Description:
To reproduce:
Enable SSL encryption and let the server be idle for a day or so you will see the below....
[vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
16333
Sun Sep 25 17:23:29 UTC 2011
[vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ java -jar cmdline-jmxclient-0.10.3.jar - localhost:7501 java.lang:type=Memory gc
[vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
64
Sun Sep 25 17:23:53 UTC 2011
[vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$
After running GC manually the issue goes away.
was:
To reproduce:
Enable SSL encryption and let the server be ideal for a day or so you will see the below....
[vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
16333
Sun Sep 25 17:23:29 UTC 2011
[vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ java -jar cmdline-jmxclient-0.10.3.jar - localhost:7501 java.lang:type=Memory gc
[vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
64
Sun Sep 25 17:23:53 UTC 2011
[vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$
After running GC manually the issue goes away.
> Enabling SSL on a fairly light cluster leaks Open files.
> --------------------------------------------------------
>
> Key: CASSANDRA-3257
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3257
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.6
> Environment: JVM on CentOS
> Reporter: Vijay
> Priority: Minor
>
> To reproduce:
> Enable SSL encryption and let the server be idle for a day or so you will see the below....
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
> 16333
> Sun Sep 25 17:23:29 UTC 2011
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ java -jar cmdline-jmxclient-0.10.3.jar - localhost:7501 java.lang:type=Memory gc
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
> 64
> Sun Sep 25 17:23:53 UTC 2011
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$
> After running GC manually the issue goes away.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3257) Enabling SSL on a fairly light
cluster leaks Open files.
Posted by "Vijay (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115256#comment-13115256 ]
Vijay commented on CASSANDRA-3257:
----------------------------------
Thanks Jonathan!, was just about to attach it...
> Enabling SSL on a fairly light cluster leaks Open files.
> --------------------------------------------------------
>
> Key: CASSANDRA-3257
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3257
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.6, 1.0.0
> Environment: JVM on CentOS
> Reporter: Vijay
> Assignee: Vijay
> Priority: Minor
> Fix For: 0.8.7, 1.0.0
>
> Attachments: 0001-ssl-open-file-issue.patch
>
>
> To reproduce:
> Enable SSL encryption and let the server be idle for a day or so you will see the below....
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
> 16333
> Sun Sep 25 17:23:29 UTC 2011
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ java -jar cmdline-jmxclient-0.10.3.jar - localhost:7501 java.lang:type=Memory gc
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
> 64
> Sun Sep 25 17:23:53 UTC 2011
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$
> After running GC manually the issue goes away.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3257) Enabling SSL on a fairly light
cluster leaks Open files.
Posted by "Vijay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13114756#comment-13114756 ]
Vijay commented on CASSANDRA-3257:
----------------------------------
Seems like the issue may be because we are opening the FIS and not closing it.... in SSLFactory.createSSLContext, testing it... patch soon.
> Enabling SSL on a fairly light cluster leaks Open files.
> --------------------------------------------------------
>
> Key: CASSANDRA-3257
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3257
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.6
> Environment: JVM on CentOS
> Reporter: Vijay
> Priority: Minor
>
> To reproduce:
> Enable SSL encryption and let the server be idle for a day or so you will see the below....
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
> 16333
> Sun Sep 25 17:23:29 UTC 2011
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ java -jar cmdline-jmxclient-0.10.3.jar - localhost:7501 java.lang:type=Memory gc
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$ /usr/sbin/lsof |grep -i cassandra-app.jks |wc -l ;date
> 64
> Sun Sep 25 17:23:53 UTC 2011
> [vijay_tcasstest@vijay_tcass--1c-i-1568885c ~]$
> After running GC manually the issue goes away.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira