You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org> on 2012/02/07 18:14:59 UTC
[jira] [Updated] (CASSANDRA-3859) Add Progress Reporting to
Cassandra SstableLoader for BulkOutputFormat
[ https://issues.apache.org/jira/browse/CASSANDRA-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3859:
----------------------------------------
Attachment: 0002-Add-progress-to-CFOF.txt
0001-add-progress-reporting-to-BOF.txt
First patch adds our own Progressable class so we can dodge most of the mapred/mapreduce mess and adds it to BOF in BRW where we report progress anytime the loader reports output. Second patch builds on this and adds similar support to CFOF, since it also doesn't report progress and times out on occasion.
> Add Progress Reporting to Cassandra SstableLoader for BulkOutputFormat
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-3859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3859
> Project: Cassandra
> Issue Type: Improvement
> Components: Hadoop, Tools
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Priority: Minor
> Labels: bulkloader, hadoop, mapreduce, sstableloader
> Fix For: 1.1
>
> Attachments: 0001-add-progress-reporting-to-BOF.txt, 0002-Add-progress-to-CFOF.txt
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> When we are using the BulkOutputFormat to load the data to cassandra. We should use the progress reporting to Hadoop Job within Sstable loader because while loading the data for particular task if streaming is taking more time and progress is not reported to Job it may kill the task with timeout exception.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira