[jira] [Commented] (SQOOP-2343) AsyncSqlRecordWriter stucks if any exception is thrown out in its close method

["Jarek Jarcec Cecho (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SQOOP-2343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521782#comment-14521782 ] 

Jarek Jarcec Cecho commented on SQOOP-2343:
-------------------------------------------

Thanks for the patch [~Yibing]. It seems weird to me that mapreduce would call {{close}} twice. Let me quickly ask some of the mapreduce experts to see if that is intentional.

> AsyncSqlRecordWriter stucks if any exception is thrown out in its close method
> ------------------------------------------------------------------------------
>
>                 Key: SQOOP-2343
>                 URL: https://issues.apache.org/jira/browse/SQOOP-2343
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors
>    Affects Versions: 1.4.5
>            Reporter: Yibing Shi
>         Attachments: SQOOP-2343.patch
>
>
> In class {{AsyncSqlRecordWriter}}, if any exception is thrown in its close method, the Hadoop MapTask will call this close method once more in case it hasn't been closed. Please see below code snippet (in method runNewMapper):
> {code}
>     try {
>       input.initialize(split, mapperContext);
>       mapper.run(mapperContext);
>       mapPhase.complete();
>       setPhase(TaskStatus.Phase.SORT);
>       statusUpdate(umbilical);
>       input.close();
>       input = null;
>       output.close(mapperContext);
>       output = null;
>     } finally {
>       closeQuietly(input);
>       closeQuietly(output, mapperContext);
>     }
> {code}
> The second time the close method is called, the main thread will stuck in executeUpdate when trying to put a new dbOp into the synchronous queue, because at this moment the worker thread has ended and thus not receiver will take that object, which makes the putter (main thread) stuck.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)