You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hbase.apache.org by st...@apache.org on 2010/03/04 17:39:11 UTC
svn commit: r919058 - in /hadoop/hbase/trunk: CHANGES.txt
core/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
Author: stack
Date: Thu Mar 4 16:39:10 2010
New Revision: 919058
URL: http://svn.apache.org/viewvc?rev=919058&view=rev
Log:
HBASE-2063 For hfileoutputformat, on timeout/failure/kill clean up half-written hfile
Modified:
hadoop/hbase/trunk/CHANGES.txt
hadoop/hbase/trunk/core/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
Modified: hadoop/hbase/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/hbase/trunk/CHANGES.txt?rev=919058&r1=919057&r2=919058&view=diff
==============================================================================
--- hadoop/hbase/trunk/CHANGES.txt (original)
+++ hadoop/hbase/trunk/CHANGES.txt Thu Mar 4 16:39:10 2010
@@ -221,6 +221,8 @@
HBASE-2244 META gets inconsistent in a number of crash scenarios
HBASE-2284 fsWriteLatency metric may be incorrectly reported
(Kannan Muthukkaruppan via Stack)
+ HBASE-2063 For hfileoutputformat, on timeout/failure/kill clean up
+ half-written hfile (Ruslan Salyakhov via Stack)
IMPROVEMENTS
HBASE-1760 Cleanup TODOs in HTable
Modified: hadoop/hbase/trunk/core/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
URL: http://svn.apache.org/viewvc/hadoop/hbase/trunk/core/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java?rev=919058&r1=919057&r2=919058&view=diff
==============================================================================
--- hadoop/hbase/trunk/core/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java (original)
+++ hadoop/hbase/trunk/core/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java Thu Mar 4 16:39:10 2010
@@ -35,6 +35,7 @@
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.mapreduce.RecordWriter;
import org.apache.hadoop.mapreduce.TaskAttemptContext;
+import org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.mortbay.log.Log;
@@ -50,7 +51,8 @@
public RecordWriter<ImmutableBytesWritable, KeyValue> getRecordWriter(TaskAttemptContext context)
throws IOException, InterruptedException {
// Get the path of the temporary output file
- final Path outputdir = FileOutputFormat.getOutputPath(context);
+ final Path outputPath = FileOutputFormat.getOutputPath(context);
+ final Path outputdir = new FileOutputCommitter(outputPath, context).getWorkPath();
Configuration conf = context.getConfiguration();
final FileSystem fs = outputdir.getFileSystem(conf);
// These configs. are from hbase-*.xml