[jira] [Comment Edited] (TEZ-2937) Can Processor.close() be called after closing inputs and outputs?

["Rohini Palaniswamy (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15001433#comment-15001433 ] 

Rohini Palaniswamy edited comment on TEZ-2937 at 11/12/15 1:12 AM:
-------------------------------------------------------------------

Hive's TezProcessor and also MapProcessor/ReduceProcessor have empty implementations of close(). Not sure about cascading thought. After reading through TEZ-955 mentioned in the comment in hive's processor, I am not sure if changing order of close() will be easily agreed upon. New hook for a final cleanup is fine as well. Or you can shift processor.close() below as asked in this jira if we think no one is going to be impacted and provide a new hook to be called in its place for dealing with inputs and outputs before they are closed for any one wanting to do that in future.


was (Author: rohini):
Hive's TezProcessor and also MapProcessor/ReduceProcessor have empty implementations of close(). Not sure about cascading thought. After reading through TEZ-955 mentioned in the comment in hive's processor, I am not sure if changing order of close() will be easily agreed upon. New hook for a final cleanup is fine as well. Or you can shift processor.close() below as asked in this jira if we think no one is going to be impacted and provide a new hook to be called in its place for dealing with inputs and outputs before they are closed for any one wanting to do that.

> Can Processor.close() be called after closing inputs and outputs?
> -----------------------------------------------------------------
>
>                 Key: TEZ-2937
>                 URL: https://issues.apache.org/jira/browse/TEZ-2937
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Rohini Palaniswamy
>
> Pig hit PIG-4722 as processor.close() which clears some static thread local variables was called when SpillThread was still running the Combiner.  LogicalIOProcessorRuntimeTask.java calls processor.close() before closing the inputs and outputs in both close() and cleanup() methods. Can Tez change to closing the processor after the inputs and outputs are closed? 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)