You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Geri (JIRA)" <ji...@apache.org> on 2017/11/20 16:07:00 UTC
[jira] [Commented] (HIVE-3700) Hive ignores
'mapreduce.fileoutputcommitter.marksuccessfuljobs'
[ https://issues.apache.org/jira/browse/HIVE-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16259413#comment-16259413 ]
Geri commented on HIVE-3700:
----------------------------
hi, could you please fix this?
> Hive ignores 'mapreduce.fileoutputcommitter.marksuccessfuljobs'
> ---------------------------------------------------------------
>
> Key: HIVE-3700
> URL: https://issues.apache.org/jira/browse/HIVE-3700
> Project: Hive
> Issue Type: Bug
> Components: Shims
> Affects Versions: 0.9.0
> Reporter: Joe Crobak
> Priority: Minor
>
> For downstream computation, it can be useful to have a {{_SUCCESS}} flag dropped in the directory to signify that the data is completely written. This is typically enabled with the property {{mapreduce.fileoutputcommitter.marksuccessfuljobs}}, but hive ignores this. I believe it's because the shims layer {{src/common-secure/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java}} and {{shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java}} both supply a NullOutputCommitter that don't respect this property.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)