You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Bhallamudi Venkata Siva Kamesh (Created) (JIRA)" <ji...@apache.org> on 2011/11/17 19:21:51 UTC

[jira] [Created] (MAPREDUCE-3423) Fetcher Thread should log proper exception

Fetcher Thread should log proper exception
------------------------------------------

                 Key: MAPREDUCE-3423
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3423
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: mrv2
    Affects Versions: 0.24.0
            Reporter: Bhallamudi Venkata Siva Kamesh


Exception should be logged properly when there is any while deserializing Shuffle Header information .
    {code}
      if (mapId == null || mapOutput == null) {
        LOG.info("fetcher#" + id + " failed to read map header" + 
                 mapId + " decomp: " + 
                 decompressedLength + ", " + compressedLength, ioe);
        return false;
      }
    {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-3423) Fetcher Thread should log proper exception

Posted by "Bhallamudi Venkata Siva Kamesh (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158247#comment-13158247 ] 

Bhallamudi Venkata Siva Kamesh commented on MAPREDUCE-3423:
-----------------------------------------------------------

bq.Am not sure I get the changes in the second part of the patch:

I updated this part, thinking mapid won't be null, handling this in the first part and returning false in the error scenarios.This exception happens when MergeManager tries to allocate location for the map output,so,  logging the same.

                
> Fetcher Thread should log proper exception
> ------------------------------------------
>
>                 Key: MAPREDUCE-3423
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3423
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.24.0
>            Reporter: Bhallamudi Venkata Siva Kamesh
>         Attachments: MAPREDUCE-3423.patch
>
>
> Exception should be logged properly when there is any while deserializing Shuffle Header information .
>     {code}
>       if (mapId == null || mapOutput == null) {
>         LOG.info("fetcher#" + id + " failed to read map header" + 
>                  mapId + " decomp: " + 
>                  decompressedLength + ", " + compressedLength, ioe);
>         return false;
>       }
>     {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (MAPREDUCE-3423) Fetcher Thread should log proper exception

Posted by "Bh V S Kamesh (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bh V S Kamesh updated MAPREDUCE-3423:
-------------------------------------

    Status: Patch Available  (was: Open)
    
> Fetcher Thread should log proper exception
> ------------------------------------------
>
>                 Key: MAPREDUCE-3423
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3423
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.24.0
>            Reporter: Bh V S Kamesh
>         Attachments: MAPREDUCE-3423-1.patch, MAPREDUCE-3423.patch
>
>
> Exception should be logged properly when there is any while deserializing Shuffle Header information .
>     {code}
>       if (mapId == null || mapOutput == null) {
>         LOG.info("fetcher#" + id + " failed to read map header" + 
>                  mapId + " decomp: " + 
>                  decompressedLength + ", " + compressedLength, ioe);
>         return false;
>       }
>     {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (MAPREDUCE-3423) Fetcher Thread should log proper exception

Posted by "Bhallamudi Venkata Siva Kamesh (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bhallamudi Venkata Siva Kamesh updated MAPREDUCE-3423:
------------------------------------------------------

    Status: Patch Available  (was: Open)
    
> Fetcher Thread should log proper exception
> ------------------------------------------
>
>                 Key: MAPREDUCE-3423
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3423
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.24.0
>            Reporter: Bhallamudi Venkata Siva Kamesh
>         Attachments: MAPREDUCE-3423.patch
>
>
> Exception should be logged properly when there is any while deserializing Shuffle Header information .
>     {code}
>       if (mapId == null || mapOutput == null) {
>         LOG.info("fetcher#" + id + " failed to read map header" + 
>                  mapId + " decomp: " + 
>                  decompressedLength + ", " + compressedLength, ioe);
>         return false;
>       }
>     {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-3423) Fetcher Thread should log proper exception

Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173192#comment-13173192 ] 

Hadoop QA commented on MAPREDUCE-3423:
--------------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12505303/MAPREDUCE-3423-1.patch
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 eclipse:eclipse.  The patch built with eclipse:eclipse.

    +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

    +1 core tests.  The patch passed unit tests in .

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1479//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1479//console

This message is automatically generated.
                
> Fetcher Thread should log proper exception
> ------------------------------------------
>
>                 Key: MAPREDUCE-3423
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3423
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.24.0
>            Reporter: Bh V S Kamesh
>         Attachments: MAPREDUCE-3423-1.patch, MAPREDUCE-3423.patch
>
>
> Exception should be logged properly when there is any while deserializing Shuffle Header information .
>     {code}
>       if (mapId == null || mapOutput == null) {
>         LOG.info("fetcher#" + id + " failed to read map header" + 
>                  mapId + " decomp: " + 
>                  decompressedLength + ", " + compressedLength, ioe);
>         return false;
>       }
>     {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (MAPREDUCE-3423) Fetcher Thread should log proper exception

Posted by "Bhallamudi Venkata Siva Kamesh (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bhallamudi Venkata Siva Kamesh updated MAPREDUCE-3423:
------------------------------------------------------

    Attachment: MAPREDUCE-3423-1.patch


bq.Also, in the first part, wouldnt it be simpler if it was something like

updated this.
                
> Fetcher Thread should log proper exception
> ------------------------------------------
>
>                 Key: MAPREDUCE-3423
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3423
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.24.0
>            Reporter: Bhallamudi Venkata Siva Kamesh
>         Attachments: MAPREDUCE-3423-1.patch, MAPREDUCE-3423.patch
>
>
> Exception should be logged properly when there is any while deserializing Shuffle Header information .
>     {code}
>       if (mapId == null || mapOutput == null) {
>         LOG.info("fetcher#" + id + " failed to read map header" + 
>                  mapId + " decomp: " + 
>                  decompressedLength + ", " + compressedLength, ioe);
>         return false;
>       }
>     {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (MAPREDUCE-3423) Fetcher Thread should log proper exception

Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mahadev konar updated MAPREDUCE-3423:
-------------------------------------

    Status: Open  (was: Patch Available)

Bhallamudi,
 Am not sure I get the changes in the second part of the patch:

{code}
- if (mapId == null || mapOutput == null) {
+ if (mapOutput == null) {
{code}

Also, the in the first part, wouldnt it be simpler if it was something like:

{code}
try{
} catch(IllegalArgument){
} catch (IOException) {
} catch(Exception) {
} 
{code}



                
> Fetcher Thread should log proper exception
> ------------------------------------------
>
>                 Key: MAPREDUCE-3423
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3423
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.24.0
>            Reporter: Bhallamudi Venkata Siva Kamesh
>         Attachments: MAPREDUCE-3423.patch
>
>
> Exception should be logged properly when there is any while deserializing Shuffle Header information .
>     {code}
>       if (mapId == null || mapOutput == null) {
>         LOG.info("fetcher#" + id + " failed to read map header" + 
>                  mapId + " decomp: " + 
>                  decompressedLength + ", " + compressedLength, ioe);
>         return false;
>       }
>     {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-3423) Fetcher Thread should log proper exception

Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152234#comment-13152234 ] 

Hadoop QA commented on MAPREDUCE-3423:
--------------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12504087/MAPREDUCE-3423.patch
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

    +1 core tests.  The patch passed unit tests in .

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1317//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1317//console

This message is automatically generated.
                
> Fetcher Thread should log proper exception
> ------------------------------------------
>
>                 Key: MAPREDUCE-3423
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3423
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.24.0
>            Reporter: Bhallamudi Venkata Siva Kamesh
>         Attachments: MAPREDUCE-3423.patch
>
>
> Exception should be logged properly when there is any while deserializing Shuffle Header information .
>     {code}
>       if (mapId == null || mapOutput == null) {
>         LOG.info("fetcher#" + id + " failed to read map header" + 
>                  mapId + " decomp: " + 
>                  decompressedLength + ", " + compressedLength, ioe);
>         return false;
>       }
>     {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-3423) Fetcher Thread should log proper exception

Posted by "Bhallamudi Venkata Siva Kamesh (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177306#comment-13177306 ] 

Bhallamudi Venkata Siva Kamesh commented on MAPREDUCE-3423:
-----------------------------------------------------------

Hi Arun,
 Thanks for looking into this. I have not removed null check for mapOutput. I have removed only null check for mapId and mapId won't be null by the time outer catch block catches any ioe (considering this patch).
                
> Fetcher Thread should log proper exception
> ------------------------------------------
>
>                 Key: MAPREDUCE-3423
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3423
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.24.0
>            Reporter: Bhallamudi Venkata Siva Kamesh
>            Assignee: Bhallamudi Venkata Siva Kamesh
>         Attachments: MAPREDUCE-3423-1.patch, MAPREDUCE-3423.patch
>
>
> Exception should be logged properly when there is any while deserializing Shuffle Header information .
>     {code}
>       if (mapId == null || mapOutput == null) {
>         LOG.info("fetcher#" + id + " failed to read map header" + 
>                  mapId + " decomp: " + 
>                  decompressedLength + ", " + compressedLength, ioe);
>         return false;
>       }
>     {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (MAPREDUCE-3423) Fetcher Thread should log proper exception

Posted by "Bhallamudi Venkata Siva Kamesh (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bhallamudi Venkata Siva Kamesh updated MAPREDUCE-3423:
------------------------------------------------------

    Attachment: MAPREDUCE-3423.patch

Attaching a patch with proper logging
                
> Fetcher Thread should log proper exception
> ------------------------------------------
>
>                 Key: MAPREDUCE-3423
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3423
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.24.0
>            Reporter: Bhallamudi Venkata Siva Kamesh
>         Attachments: MAPREDUCE-3423.patch
>
>
> Exception should be logged properly when there is any while deserializing Shuffle Header information .
>     {code}
>       if (mapId == null || mapOutput == null) {
>         LOG.info("fetcher#" + id + " failed to read map header" + 
>                  mapId + " decomp: " + 
>                  decompressedLength + ", " + compressedLength, ioe);
>         return false;
>       }
>     {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (MAPREDUCE-3423) Fetcher Thread should log proper exception

Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Arun C Murthy updated MAPREDUCE-3423:
-------------------------------------

    Assignee: Bhallamudi Venkata Siva Kamesh
      Status: Open  (was: Patch Available)

Kamesh - you've removed the null check for mapOutput?
                
> Fetcher Thread should log proper exception
> ------------------------------------------
>
>                 Key: MAPREDUCE-3423
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3423
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.24.0
>            Reporter: Bhallamudi Venkata Siva Kamesh
>            Assignee: Bhallamudi Venkata Siva Kamesh
>         Attachments: MAPREDUCE-3423-1.patch, MAPREDUCE-3423.patch
>
>
> Exception should be logged properly when there is any while deserializing Shuffle Header information .
>     {code}
>       if (mapId == null || mapOutput == null) {
>         LOG.info("fetcher#" + id + " failed to read map header" + 
>                  mapId + " decomp: " + 
>                  decompressedLength + ", " + compressedLength, ioe);
>         return false;
>       }
>     {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira