[jira] [Commented] (SQOOP-674) Sqoop2: Exceptions in special map reduce threads can cause mapreduce job to freeze

["Jarek Jarcec Cecho (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SQOOP-674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533259#comment-13533259 ] 

Jarek Jarcec Cecho commented on SQOOP-674:
------------------------------------------

I found one case when our threads will came to deadlock:

1) Reader (=loader) gets data from shared queue.
2) Writer (=extractor) will put additional data to shared queue.
3) Writer (=extractor) will try to put additional data into shared queue, but will be blocked by call free.acquire() in SqoopRecordWriter::write()
4) Reader (=loader) will throw an exception while processing data from 1)

In this case we have a deadlock as writer will be waiting on dead reader.
                
> Sqoop2: Exceptions in special map reduce threads can cause mapreduce job to freeze
> ----------------------------------------------------------------------------------
>
>                 Key: SQOOP-674
>                 URL: https://issues.apache.org/jira/browse/SQOOP-674
>             Project: Sqoop
>          Issue Type: Bug
>    Affects Versions: 2.0.0
>            Reporter: Jarek Jarcec Cecho
>            Assignee: Jarek Jarcec Cecho
>             Fix For: 2.0.0
>
>
> We need to properly handle exceptions in our map reduce threads as they can freeze running map reduce job.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira