You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Mck SembWever (JIRA)" <ji...@apache.org> on 2011/09/23 13:04:26 UTC
[jira] [Issue Comment Edited] (CASSANDRA-3150)
ColumnFormatRecordReader loops forever (StorageService.getSplits(..) out of
whack)
[ https://issues.apache.org/jira/browse/CASSANDRA-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113326#comment-13113326 ]
Mck SembWever edited comment on CASSANDRA-3150 at 9/23/11 11:04 AM:
--------------------------------------------------------------------
This is persisting to be a problem. And continuously gets worse through the day.
I can see like half my cf being read from one split.
It doesn't matter if it's ByteOrderingPartition or RandomPartitioner.
I've attached two screenshots from hadoop webpages (you see 36million map input records when cassandra.input.split.size is set to 393216).
was (Author: michaelsembwever):
This is persisting to be a problem. And continuously gets worse through the day.
I can see like half my cf being read from one split.
It doesn't matter if it's ByteOrderingPartition or RandomPartitioner.
I've attached two screenshots from hadoop webpages.
> ColumnFormatRecordReader loops forever (StorageService.getSplits(..) out of whack)
> ----------------------------------------------------------------------------------
>
> Key: CASSANDRA-3150
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3150
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 0.8.4
> Reporter: Mck SembWever
> Assignee: Mck SembWever
> Priority: Critical
> Attachments: CASSANDRA-3150.patch, Screenshot-Counters for task_201109212019_1060_m_000029 - Mozilla Firefox.png, Screenshot-Hadoop map task list for job_201109212019_1060 on cassandra01 - Mozilla Firefox.png, attempt_201109071357_0044_m_003040_0.grep-get_range_slices.log
>
>
> From http://thread.gmane.org/gmane.comp.db.cassandra.user/20039
> {quote}
> bq. Cassandra-0.8.4 w/ ByteOrderedPartitioner
> bq. CFIF's inputSplitSize=196608
> bq. 3 map tasks (from 4013) is still running after read 25 million rows.
> bq. Can this be a bug in StorageService.getSplits(..) ?
> getSplits looks pretty foolproof to me but I guess we'd need to add
> more debug logging to rule out a bug there for sure.
> I guess the main alternative would be a bug in the recordreader paging.
> {quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira