You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-issues@hadoop.apache.org by "Allen Wittenauer (JIRA)" <ji...@apache.org> on 2014/07/19 00:16:17 UTC

[jira] [Resolved] (MAPREDUCE-1770) have option to limit stderr output from user scripts in hadoop streaming

     [ https://issues.apache.org/jira/browse/MAPREDUCE-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Allen Wittenauer resolved MAPREDUCE-1770.
-----------------------------------------

    Resolution: Fixed

Log limits were implemented generically eons ago.

> have option to limit stderr output from user scripts in hadoop streaming
> ------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1770
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1770
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Joydeep Sen Sarma
>
> People often echo to stderr to debug their programs.  We have had cases where someone inadvertently dumped large amounts of data to stderr. this created large userlog directories and caused quite a few mapred slaves to become dysfunctional (logs partition was full).
> Hence looking for a defensive measure against such errant scripts. We are currently putting in a limit on the number of bytes from the script's stderr that are forwarded to System.err.



--
This message was sent by Atlassian JIRA
(v6.2#6252)