[jira] [Commented] (HIVE-22887) MR Job cannot set custom OutputCommitter during job execution.

["Gopal Vijayaraghavan (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-22887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17036326#comment-17036326 ] 

Gopal Vijayaraghavan commented on HIVE-22887:
---------------------------------------------

bq. If someone wants to provide their own OutputCommitter then its not possible in the current implementation.

This has to do with Hive supporting partitioned table inserts - Hive FileSinkOperator does not use an MROutput, to allow it to write + commit data to multiple directories.

> MR Job cannot set custom OutputCommitter during job execution.
> --------------------------------------------------------------
>
>                 Key: HIVE-22887
>                 URL: https://issues.apache.org/jira/browse/HIVE-22887
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Renukaprasad C
>            Assignee: Renukaprasad C
>            Priority: Major
>
> MapRedTask set jobs OutputCommitter to NullOutputCommitter.
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.prepareJobOutput(JobConf) this  sets the OutputCommitter to NullOutputCommitter always.
> conf.setOutputCommitter(NullOutputCommitter.class);
> There are some cases where we need to customize the OutputCommitter. 
> Like on SUCCESS, they need to write some file or do some custom operation in theie own OutputCommitter
> If someone wants to provide their own OutputCommitter then its not possible in the current implementation.
> Related issues: MR - MAPREDUCE-1802, Hive - HIVE-1355.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)