You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Todd Lipcon (Created) (JIRA)" <ji...@apache.org> on 2012/03/09 09:42:56 UTC
[jira] [Created] (MAPREDUCE-3992) Reduce fetcher doesn't verify
HTTP status code of response
Reduce fetcher doesn't verify HTTP status code of response
----------------------------------------------------------
Key: MAPREDUCE-3992
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3992
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv1
Affects Versions: 1.0.1
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Currently, the reduce fetch code doesn't check the HTTP status code of the response. This can lead to the following situation:
- the map output servlet gets an IOException after setting the headers but before the first call to flush()
- this causes it to send a response with a non-OK result code, including the exception text as the response body (response.sendError() does this if the response isn't committed)
- it will still include the response headers indicating it's a valid response
In the case of a merge-to-memory, the compression codec might then try to interpret the HTML response as compressed data, resulting in either a huge allocation (OOME) or some other nasty error. This bug seems to be present in MR1, but haven't checked trunk/MR2 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-3992) Reduce fetcher doesn't verify
HTTP status code of response
Posted by "Todd Lipcon (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon resolved MAPREDUCE-3992.
------------------------------------
Resolution: Fixed
Fix Version/s: 1.1.0
Target Version/s: 0.24.0, 0.23.3, 1.0.3 (was: 1.0.3, 0.23.3, 0.24.0)
Hadoop Flags: Reviewed
Committed to branch-1
> Reduce fetcher doesn't verify HTTP status code of response
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3992
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3992
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv1
> Affects Versions: 0.23.1, 0.24.0, 1.0.1
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Fix For: 0.24.0, 1.1.0, 0.23.3
>
> Attachments: mr-3992-branch-1.txt, mr-3992.txt
>
>
> Currently, the reduce fetch code doesn't check the HTTP status code of the response. This can lead to the following situation:
> - the map output servlet gets an IOException after setting the headers but before the first call to flush()
> - this causes it to send a response with a non-OK result code, including the exception text as the response body (response.sendError() does this if the response isn't committed)
> - it will still include the response headers indicating it's a valid response
> In the case of a merge-to-memory, the compression codec might then try to interpret the HTML response as compressed data, resulting in either a huge allocation (OOME) or some other nasty error. This bug seems to be present in MR1, but haven't checked trunk/MR2 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira