You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jason Harvey (JIRA)" <ji...@apache.org> on 2011/03/10 01:55:59 UTC
[jira] Created: (CASSANDRA-2304) sstable2json dies with "Too many
open files", regardless of ulimit
sstable2json dies with "Too many open files", regardless of ulimit
------------------------------------------------------------------
Key: CASSANDRA-2304
URL: https://issues.apache.org/jira/browse/CASSANDRA-2304
Project: Cassandra
Issue Type: Bug
Reporter: Jason Harvey
Running sstable2json on the attached sstable eventually results in the following:
{code}
Exception in thread "main" java.io.IOError: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:68)
at org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:567)
at org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SSTableSliceIterator.java:49)
at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:68)
at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80)
at org.apache.cassandra.tools.SSTableExport.serializeRow(SSTableExport.java:187)
at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:355)
at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:377)
at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:390)
at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:448)
Caused by: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233)
at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:111)
at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:106)
at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:91)
at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:62)
{code}
Set my ulimit -n to 60000 and got the same result. Leaking file descriptors?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2304) sstable2json dies with "Too many
open files", regardless of ulimit
Posted by "Jason Harvey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Harvey updated CASSANDRA-2304:
------------------------------------
Comment: was deleted
(was: Unfortunately, the sstable it is dying on is 900MB. Unable to attach that. If there is somewhere I could upload it, let me know.)
> sstable2json dies with "Too many open files", regardless of ulimit
> ------------------------------------------------------------------
>
> Key: CASSANDRA-2304
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2304
> Project: Cassandra
> Issue Type: Bug
> Reporter: Jason Harvey
> Attachments: sstable.tar.bz2
>
>
> Running sstable2json on the attached sstable eventually results in the following:
> {code}
> Exception in thread "main" java.io.IOError: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
> at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:68)
> at org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:567)
> at org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SSTableSliceIterator.java:49)
> at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:68)
> at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80)
> at org.apache.cassandra.tools.SSTableExport.serializeRow(SSTableExport.java:187)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:355)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:377)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:390)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:448)
> Caused by: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:111)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:106)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:91)
> at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:62)
> {code}
> Set my ulimit -n to 60000 and got the same result. Leaking file descriptors?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2304) sstable2json dies with "Too many
open files", regardless of ulimit
Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pavel Yaskevich updated CASSANDRA-2304:
---------------------------------------
Attachment: CASSANDRA-2304-v2.patch
with number of the exported columns properly incremented. LGTM.
> sstable2json dies with "Too many open files", regardless of ulimit
> ------------------------------------------------------------------
>
> Key: CASSANDRA-2304
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2304
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Affects Versions: 0.7.1
> Reporter: Jason Harvey
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 0.7.4
>
> Attachments: 2304.txt, CASSANDRA-2304-v2.patch, sstable.tar.bz2
>
>
> Running sstable2json on the attached sstable eventually results in the following:
> {code}
> Exception in thread "main" java.io.IOError: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
> at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:68)
> at org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:567)
> at org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SSTableSliceIterator.java:49)
> at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:68)
> at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80)
> at org.apache.cassandra.tools.SSTableExport.serializeRow(SSTableExport.java:187)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:355)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:377)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:390)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:448)
> Caused by: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:111)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:106)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:91)
> at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:62)
> {code}
> Set my ulimit -n to 60000 and got the same result. Leaking file descriptors?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2304) sstable2json dies with "Too many
open files", regardless of ulimit
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13005198#comment-13005198 ]
Hudson commented on CASSANDRA-2304:
-----------------------------------
Integrated in Cassandra-0.7 #370 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/370/])
fix fd leak in sstable2json with non-mmap'd i/o
patch by jbellis; reviewed by Pavel Yaskevich for CASSANDRA-2304
> sstable2json dies with "Too many open files", regardless of ulimit
> ------------------------------------------------------------------
>
> Key: CASSANDRA-2304
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2304
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Affects Versions: 0.7.1
> Reporter: Jason Harvey
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 0.7.4
>
> Attachments: 2304.txt, CASSANDRA-2304-v2.patch, sstable.tar.bz2
>
>
> Running sstable2json on the attached sstable eventually results in the following:
> {code}
> Exception in thread "main" java.io.IOError: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
> at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:68)
> at org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:567)
> at org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SSTableSliceIterator.java:49)
> at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:68)
> at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80)
> at org.apache.cassandra.tools.SSTableExport.serializeRow(SSTableExport.java:187)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:355)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:377)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:390)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:448)
> Caused by: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:111)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:106)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:91)
> at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:62)
> {code}
> Set my ulimit -n to 60000 and got the same result. Leaking file descriptors?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2304) sstable2json dies with "Too many
open files", regardless of ulimit
Posted by "Jason Harvey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13004916#comment-13004916 ]
Jason Harvey commented on CASSANDRA-2304:
-----------------------------------------
Output from lsof. Thousands of lines of the following:
{code}
java 1766 root 1080r REG 251,0 11809597 32374948 /var/lib/cassandra/data/reddit/Hide-f-734-Data.db
java 1766 root 1081r REG 251,0 11809597 32374948 /var/lib/cassandra/data/reddit/Hide-f-734-Data.db
java 1766 root 1082r REG 251,0 11809597 32374948 /var/lib/cassandra/data/reddit/Hide-f-734-Data.db
java 1766 root 1083r REG 251,0 11809597 32374948 /var/lib/cassandra/data/reddit/Hide-f-734-Data.db
java 1766 root 1084r REG 251,0 11809597 32374948 /var/lib/cassandra/data/reddit/Hide-f-734-Data.db
java 1766 root 1085r REG 251,0 11809597 32374948 /var/lib/cassandra/data/reddit/Hide-f-734-Data.db
java 1766 root 1086r REG 251,0 11809597 32374948 /var/lib/cassandra/data/reddit/Hide-f-734-Data.db
java 1766 root 1087r REG 251,0 11809597 32374948 /var/lib/cassandra/data/reddit/Hide-f-734-Data.db
java 1766 root 1088r REG 251,0 11809597 32374948 /var/lib/cassandra/data/reddit/Hide-f-734-Data.db
java 1766 root 1089r REG 251,0 11809597 32374948 /var/lib/cassandra/data/reddit/Hide-f-734-Data.db
java 1766 root 1090r REG 251,0 11809597 32374948 /var/lib/cassandra/data/reddit/Hide-f-734-Data.db
java 1766 root 1091r REG 251,0 11809597 32374948 /var/lib/cassandra/data/reddit/Hide-f-734-Data.db
java 1766 root 1092r REG 251,0 11809597 32374948 /var/lib/cassandra/data/reddit/Hide-f-734-Data.db
java 1766 root 1093r REG 251,0 11809597 32374948 /var/lib/cassandra/data/reddit/Hide-f-734-Data.db
java 1766 root 1094r REG 251,0 11809597 32374948 /var/lib/cassandra/data/reddit/Hide-f-734-Data.db
java 1766 root 1095r REG 251,0 11809597 32374948 /var/lib/cassandra/data/reddit/Hide-f-734-Data.db
{code}
> sstable2json dies with "Too many open files", regardless of ulimit
> ------------------------------------------------------------------
>
> Key: CASSANDRA-2304
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2304
> Project: Cassandra
> Issue Type: Bug
> Reporter: Jason Harvey
> Attachments: sstable.tar.bz2
>
>
> Running sstable2json on the attached sstable eventually results in the following:
> {code}
> Exception in thread "main" java.io.IOError: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
> at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:68)
> at org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:567)
> at org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SSTableSliceIterator.java:49)
> at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:68)
> at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80)
> at org.apache.cassandra.tools.SSTableExport.serializeRow(SSTableExport.java:187)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:355)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:377)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:390)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:448)
> Caused by: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:111)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:106)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:91)
> at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:62)
> {code}
> Set my ulimit -n to 60000 and got the same result. Leaking file descriptors?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2304) sstable2json dies with "Too many
open files", regardless of ulimit
Posted by "Jason Harvey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Harvey updated CASSANDRA-2304:
------------------------------------
Attachment: sstable.tar.bz2
Sstable which causes sstable2json to die. Grabbed from an 0.7.3 node.
> sstable2json dies with "Too many open files", regardless of ulimit
> ------------------------------------------------------------------
>
> Key: CASSANDRA-2304
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2304
> Project: Cassandra
> Issue Type: Bug
> Reporter: Jason Harvey
> Attachments: sstable.tar.bz2
>
>
> Running sstable2json on the attached sstable eventually results in the following:
> {code}
> Exception in thread "main" java.io.IOError: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
> at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:68)
> at org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:567)
> at org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SSTableSliceIterator.java:49)
> at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:68)
> at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80)
> at org.apache.cassandra.tools.SSTableExport.serializeRow(SSTableExport.java:187)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:355)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:377)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:390)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:448)
> Caused by: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:111)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:106)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:91)
> at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:62)
> {code}
> Set my ulimit -n to 60000 and got the same result. Leaking file descriptors?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2304) sstable2json dies with "Too many
open files", regardless of ulimit
Posted by "Jason Harvey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13004907#comment-13004907 ]
Jason Harvey commented on CASSANDRA-2304:
-----------------------------------------
Unfortunately, the sstable it is dying on is 900MB. Unable to attach that. If there is somewhere I could upload it, let me know.
> sstable2json dies with "Too many open files", regardless of ulimit
> ------------------------------------------------------------------
>
> Key: CASSANDRA-2304
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2304
> Project: Cassandra
> Issue Type: Bug
> Reporter: Jason Harvey
>
> Running sstable2json on the attached sstable eventually results in the following:
> {code}
> Exception in thread "main" java.io.IOError: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
> at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:68)
> at org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:567)
> at org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SSTableSliceIterator.java:49)
> at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:68)
> at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80)
> at org.apache.cassandra.tools.SSTableExport.serializeRow(SSTableExport.java:187)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:355)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:377)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:390)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:448)
> Caused by: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:111)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:106)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:91)
> at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:62)
> {code}
> Set my ulimit -n to 60000 and got the same result. Leaking file descriptors?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2304) sstable2json dies with "Too many
open files", regardless of ulimit
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-2304:
--------------------------------------
Remaining Estimate: 1h
Original Estimate: 1h
> sstable2json dies with "Too many open files", regardless of ulimit
> ------------------------------------------------------------------
>
> Key: CASSANDRA-2304
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2304
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Affects Versions: 0.7.1
> Reporter: Jason Harvey
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 0.7.4
>
> Attachments: 2304.txt, CASSANDRA-2304-v2.patch, sstable.tar.bz2
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Running sstable2json on the attached sstable eventually results in the following:
> {code}
> Exception in thread "main" java.io.IOError: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
> at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:68)
> at org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:567)
> at org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SSTableSliceIterator.java:49)
> at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:68)
> at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80)
> at org.apache.cassandra.tools.SSTableExport.serializeRow(SSTableExport.java:187)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:355)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:377)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:390)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:448)
> Caused by: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:111)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:106)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:91)
> at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:62)
> {code}
> Set my ulimit -n to 60000 and got the same result. Leaking file descriptors?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2304) sstable2json dies with "Too many
open files", regardless of ulimit
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-2304:
--------------------------------------
Attachment: 2304.txt
patch to close column iterator and only do one pass per row
> sstable2json dies with "Too many open files", regardless of ulimit
> ------------------------------------------------------------------
>
> Key: CASSANDRA-2304
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2304
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Affects Versions: 0.7.1
> Reporter: Jason Harvey
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 0.7.4
>
> Attachments: 2304.txt, sstable.tar.bz2
>
>
> Running sstable2json on the attached sstable eventually results in the following:
> {code}
> Exception in thread "main" java.io.IOError: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
> at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:68)
> at org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:567)
> at org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SSTableSliceIterator.java:49)
> at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:68)
> at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80)
> at org.apache.cassandra.tools.SSTableExport.serializeRow(SSTableExport.java:187)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:355)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:377)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:390)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:448)
> Caused by: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:111)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:106)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:91)
> at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:62)
> {code}
> Set my ulimit -n to 60000 and got the same result. Leaking file descriptors?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2304) sstable2json dies with "Too many
open files", regardless of ulimit
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-2304:
--------------------------------------
Reviewer: xedin
Component/s: Tools
Priority: Minor (was: Major)
Affects Version/s: 0.7.1
Fix Version/s: 0.7.4
Assignee: Jonathan Ellis
This is a bug with non-mmap'd I/O.
> sstable2json dies with "Too many open files", regardless of ulimit
> ------------------------------------------------------------------
>
> Key: CASSANDRA-2304
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2304
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Affects Versions: 0.7.1
> Reporter: Jason Harvey
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 0.7.4
>
> Attachments: 2304.txt, sstable.tar.bz2
>
>
> Running sstable2json on the attached sstable eventually results in the following:
> {code}
> Exception in thread "main" java.io.IOError: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
> at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:68)
> at org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:567)
> at org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SSTableSliceIterator.java:49)
> at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:68)
> at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80)
> at org.apache.cassandra.tools.SSTableExport.serializeRow(SSTableExport.java:187)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:355)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:377)
> at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:390)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:448)
> Caused by: java.io.FileNotFoundException: /var/lib/cassandra/data/reddit/CommentSortsCache-f-9764-Data.db (Too many open files)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:111)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:106)
> at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:91)
> at org.apache.cassandra.io.util.BufferedSegmentedFile.getSegment(BufferedSegmentedFile.java:62)
> {code}
> Set my ulimit -n to 60000 and got the same result. Leaking file descriptors?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira