You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Aaron Kimball (JIRA)" <ji...@apache.org> on 2010/01/21 23:31:55 UTC

[jira] Created: (MAPREDUCE-1395) Sqoop does not check return value of Job.waitForCompletion()

Sqoop does not check return value of Job.waitForCompletion()
------------------------------------------------------------

                 Key: MAPREDUCE-1395
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1395
             Project: Hadoop Map/Reduce
          Issue Type: Bug
            Reporter: Aaron Kimball
            Assignee: Aaron Kimball


Old code depended on JobClient.runJob() throwing IOException on failure. Job.waitForCompletion can fail in that manner, or it can fail by returning false. Sqoop needs to check for this condition.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-1395) Sqoop does not check return value of Job.waitForCompletion()

Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aaron Kimball updated MAPREDUCE-1395:
-------------------------------------

    Component/s: contrib/sqoop

> Sqoop does not check return value of Job.waitForCompletion()
> ------------------------------------------------------------
>
>                 Key: MAPREDUCE-1395
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1395
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: contrib/sqoop
>            Reporter: Aaron Kimball
>            Assignee: Aaron Kimball
>         Attachments: MAPREDUCE-1395.patch
>
>
> Old code depended on JobClient.runJob() throwing IOException on failure. Job.waitForCompletion can fail in that manner, or it can fail by returning false. Sqoop needs to check for this condition.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-1395) Sqoop does not check return value of Job.waitForCompletion()

Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aaron Kimball updated MAPREDUCE-1395:
-------------------------------------

    Attachment: MAPREDUCE-1395.patch

Attaching a patch which adds these checks. Includes testcases that trigger failures in jobs due to IOException as well as a "normal failure" condition. Refactored DataDrivenImportJob a bit to allow mapper dependency injection for this test.

> Sqoop does not check return value of Job.waitForCompletion()
> ------------------------------------------------------------
>
>                 Key: MAPREDUCE-1395
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1395
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Aaron Kimball
>            Assignee: Aaron Kimball
>         Attachments: MAPREDUCE-1395.patch
>
>
> Old code depended on JobClient.runJob() throwing IOException on failure. Job.waitForCompletion can fail in that manner, or it can fail by returning false. Sqoop needs to check for this condition.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAPREDUCE-1395) Sqoop does not check return value of Job.waitForCompletion()

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12803569#action_12803569 ] 

Hadoop QA commented on MAPREDUCE-1395:
--------------------------------------

+1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12431074/MAPREDUCE-1395.patch
  against trunk revision 901866.

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 14 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

    +1 core tests.  The patch passed core unit tests.

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/400/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/400/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/400/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/400/console

This message is automatically generated.

> Sqoop does not check return value of Job.waitForCompletion()
> ------------------------------------------------------------
>
>                 Key: MAPREDUCE-1395
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1395
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: contrib/sqoop
>            Reporter: Aaron Kimball
>            Assignee: Aaron Kimball
>         Attachments: MAPREDUCE-1395.patch
>
>
> Old code depended on JobClient.runJob() throwing IOException on failure. Job.waitForCompletion can fail in that manner, or it can fail by returning false. Sqoop needs to check for this condition.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAPREDUCE-1395) Sqoop does not check return value of Job.waitForCompletion()

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12828482#action_12828482 ] 

Hudson commented on MAPREDUCE-1395:
-----------------------------------

Integrated in Hadoop-Mapreduce-trunk-Commit #225 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/225/])
    

> Sqoop does not check return value of Job.waitForCompletion()
> ------------------------------------------------------------
>
>                 Key: MAPREDUCE-1395
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1395
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: contrib/sqoop
>            Reporter: Aaron Kimball
>            Assignee: Aaron Kimball
>             Fix For: 0.22.0
>
>         Attachments: MAPREDUCE-1395.patch
>
>
> Old code depended on JobClient.runJob() throwing IOException on failure. Job.waitForCompletion can fail in that manner, or it can fail by returning false. Sqoop needs to check for this condition.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAPREDUCE-1395) Sqoop does not check return value of Job.waitForCompletion()

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12806702#action_12806702 ] 

Hudson commented on MAPREDUCE-1395:
-----------------------------------

Integrated in Hadoop-Mapreduce-trunk #221 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/221/])
    

> Sqoop does not check return value of Job.waitForCompletion()
> ------------------------------------------------------------
>
>                 Key: MAPREDUCE-1395
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1395
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: contrib/sqoop
>            Reporter: Aaron Kimball
>            Assignee: Aaron Kimball
>             Fix For: 0.22.0
>
>         Attachments: MAPREDUCE-1395.patch
>
>
> Old code depended on JobClient.runJob() throwing IOException on failure. Job.waitForCompletion can fail in that manner, or it can fail by returning false. Sqoop needs to check for this condition.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-1395) Sqoop does not check return value of Job.waitForCompletion()

Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aaron Kimball updated MAPREDUCE-1395:
-------------------------------------

    Status: Patch Available  (was: Open)

> Sqoop does not check return value of Job.waitForCompletion()
> ------------------------------------------------------------
>
>                 Key: MAPREDUCE-1395
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1395
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Aaron Kimball
>            Assignee: Aaron Kimball
>         Attachments: MAPREDUCE-1395.patch
>
>
> Old code depended on JobClient.runJob() throwing IOException on failure. Job.waitForCompletion can fail in that manner, or it can fail by returning false. Sqoop needs to check for this condition.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-1395) Sqoop does not check return value of Job.waitForCompletion()

Posted by "Tom White (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tom White updated MAPREDUCE-1395:
---------------------------------

       Resolution: Fixed
    Fix Version/s: 0.22.0
     Hadoop Flags: [Reviewed]
           Status: Resolved  (was: Patch Available)

I've just committed this. Thanks Aaron!

> Sqoop does not check return value of Job.waitForCompletion()
> ------------------------------------------------------------
>
>                 Key: MAPREDUCE-1395
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1395
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: contrib/sqoop
>            Reporter: Aaron Kimball
>            Assignee: Aaron Kimball
>             Fix For: 0.22.0
>
>         Attachments: MAPREDUCE-1395.patch
>
>
> Old code depended on JobClient.runJob() throwing IOException on failure. Job.waitForCompletion can fail in that manner, or it can fail by returning false. Sqoop needs to check for this condition.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.