[jira] [Comment Edited] (MAPREDUCE-5) Shuffle's getMapOutput() fails with EofException, followed by IllegalStateException

["Harsh J (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13756394#comment-13756394 ] 

Harsh J edited comment on MAPREDUCE-5 at 9/3/13 6:46 AM:
---------------------------------------------------------

This one's been stale for a long time, and given the below comment from Jothi, am marking it as Not A Problem.

{quote}
The Jetty EOFException is observed when the Reduce Task prematurely closes a connection to a jetty server. The RT might decide to do this when the expected map output size is less than its memory cache limit, but cannot fetch it now because it has already fetched several other map outputs to memory. So, it triggers a flush of the existing map outputs from memory to disk and once this flush is complete, it will attempt to fetch the map output again by reopening the connection to the Jetty. In short, it is OK to see such messages.
This should neither cause a task nor a job failure. Are you seeing failures? The reason why these error messages appear for some jobs and not for others is primarily got to do with the size of map outputs and the heap size allocated for the reducer.
{quote}

For suppressing the log, as discussed earlier, I've filed: MAPREDUCE-5492.
                
      was (Author: qwertymaniac):
    This one's been stale for a long time, and given the below comment from Jothi, am marking it as Not A Problem. For suppressing the log, as discussed earlier, I've filed: MAPREDUCE-5492.
                  
> Shuffle's getMapOutput() fails with EofException, followed by IllegalStateException
> -----------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 0.20.2, 1.1.1
>         Environment: Sun Java 1.6.0_13, OpenSolaris, running on a SunFire 4150 (x64) 10 node cluster
>            Reporter: George Porter
>         Attachments: temp.rar
>
>
> During the shuffle phase, I'm seeing a large sequence of the following actions:
> 1) WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_200905181452_0002_m_000010_0,0) failed : org.mortbay.jetty.EofException
> 2) WARN org.mortbay.log: Committed before 410 getMapOutput(attempt_200905181452_0002_m_000010_0,0) failed : org.mortbay.jetty.EofException
> 3) ERROR org.mortbay.log: /mapOutput java.lang.IllegalStateException: Committed
> The map phase completes with 100%, and then the reduce phase crawls along with the above errors in each of the TaskTracker logs.  None of the tasktrackers get lost.  When I run non-data jobs like the 'pi' test from the example jar, everything works fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira