[jira] Resolved: (HBASE-2280) HFileOutputFormat writes output to "unsafe" directory

["Vladimir Klimontovich (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vladimir Klimontovich resolved HBASE-2280.
------------------------------------------

    Resolution: Duplicate

Duplicate of HBASE-2063

> HFileOutputFormat writes output to "unsafe" directory
> -----------------------------------------------------
>
>                 Key: HBASE-2280
>                 URL: https://issues.apache.org/jira/browse/HBASE-2280
>             Project: Hadoop HBase
>          Issue Type: Bug
>    Affects Versions: 0.20.3
>            Reporter: Vladimir Klimontovich
>            Priority: Critical
>             Fix For: 0.20.5, 0.21.0, 0.22.0
>
>
> HFileOutputFormat writes data direct to output folder. It's incorrect as failed (or killed, or interrupted) reducers leaves inconsistent files in output folder.
> The convinient way to ouput data from OutputFormat is to use "working directory". The content of this directory is being moved to output directory at the end of reducer process if only reducer succeeded (this process is called "output commit" or "atomic commit").
> If means that instead of
>  final Path outputdir = FileOutputFormat.getOutputPath(context);
> hbase should use
>  final Path outputdir = FileOutputFormat.getWorkOutputPath(context);

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.