You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jonathan Ellis (JIRA)" <ji...@apache.org> on 2013/06/19 07:47:20 UTC
[jira] [Comment Edited] (CASSANDRA-5555) Allow sstableloader to
handle a larger number of files
[ https://issues.apache.org/jira/browse/CASSANDRA-5555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13687629#comment-13687629 ]
Jonathan Ellis edited comment on CASSANDRA-5555 at 6/19/13 5:46 AM:
--------------------------------------------------------------------
To clarify: Alex was still reproducing OOM, but his initial heap dump showed that IndexSummary was the primary culprit. Once we started running the code from current 1.2, it still OOMs but the culprit is now BufferedSegmentedFile. (Looks like our reader pooling is biting us in the butt.)
was (Author: jbellis):
To clarify: Alex was still reproducing OOM, but his initial heap dump showed that IndexSummary was the primary culprit. Once we started running the code from current 1.2, it still OOMs but the culprit is now BufferedSegmentedFile.
> Allow sstableloader to handle a larger number of files
> ------------------------------------------------------
>
> Key: CASSANDRA-5555
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5555
> Project: Cassandra
> Issue Type: Improvement
> Components: Core, Tools
> Reporter: Tyler Hobbs
> Assignee: Jonathan Ellis
> Fix For: 1.2.6
>
> Attachments: 5555-01.txt, 5555-02.txt, cass_5555_pic_8.png, CASSANDRA-5555.txt, CASSANDRA-5555.txt
>
>
> With the default heap size, sstableloader will OOM when there are roughly 25k files in the directory to load. It's easy to reach this number of files in a single LCS column family.
> By avoiding creating all SSTableReaders up front in SSTableLoader, we should be able to increase the number of files that sstableloader can handle considerably.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira