[jira] [Commented] (TEZ-1351) Need to invoke isCommitRequired inside MROutput.commit()

["Bikas Saha (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081626#comment-14081626 ] 

Bikas Saha commented on TEZ-1351:
---------------------------------

Should the fix be in ORC? Calling fileOutput.close() before fileOutput.commit() sounds incorrect in the sense they have been defined in Tez.

> Need to invoke isCommitRequired inside MROutput.commit()
> --------------------------------------------------------
>
>                 Key: TEZ-1351
>                 URL: https://issues.apache.org/jira/browse/TEZ-1351
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Daniel Dai
>            Assignee: Siddharth Seth
>             Fix For: 0.5.0
>
>
> In MROutput.commit, we need to check isCommitRequired before invoking commitTask.
> Currently we did this check inside Pig:
> {code}
>                 if (fileOutput.isCommitRequired()) {
>                     fileOutput.commit();
>                 }
> {code}
> However, in some loader, output file is generated only after fileOutput.close, which is part of fileOutput.commit. The isCommitRequired check is too early. A walk around is to invoke fileOutput.close before isCommitRequired:
> {code}
>                 fileOutput.close();
>                 if (fileOutput.isCommitRequired()) {
>                     fileOutput.commit();
>                 }
> {code}
> But we are told there is a plan to make MROutput.close private.



--
This message was sent by Atlassian JIRA
(v6.2#6252)