You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Andy Isaacson (JIRA)" <ji...@apache.org> on 2012/11/12 21:19:12 UTC
[jira] [Created] (HDFS-4178) shell scripts should not close stderr
Andy Isaacson created HDFS-4178:
-----------------------------------
Summary: shell scripts should not close stderr
Key: HDFS-4178
URL: https://issues.apache.org/jira/browse/HDFS-4178
Project: Hadoop HDFS
Issue Type: Bug
Components: scripts
Affects Versions: 2.0.2-alpha
Reporter: Andy Isaacson
Assignee: Andy Isaacson
The {{start-dfs.sh}} and {{stop-dfs.sh}} scripts close stderr for some subprocesses using the construct
bq. {{2>&-}}
This is dangerous because child processes started up under this scenario will re-use filedescriptor 2 for opened files. Since libc and many other codepaths assume that filedescriptor 2 can be written to in error conditions, this can potentially result in data corruption.
Much better to redirect stderr using the construct {{2>/dev/null}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4178) shell scripts should not close
stderr
Posted by "Andy Isaacson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499263#comment-13499263 ]
Andy Isaacson commented on HDFS-4178:
-------------------------------------
[~daryn] I think this jira is ready to commit, do you agree?
> shell scripts should not close stderr
> -------------------------------------
>
> Key: HDFS-4178
> URL: https://issues.apache.org/jira/browse/HDFS-4178
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: scripts
> Affects Versions: 2.0.2-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs4178.txt
>
>
> The {{start-dfs.sh}} and {{stop-dfs.sh}} scripts close stderr for some subprocesses using the construct
> bq. {{2>&-}}
> This is dangerous because child processes started up under this scenario will re-use filedescriptor 2 for opened files. Since libc and many other codepaths assume that filedescriptor 2 can be written to in error conditions, this can potentially result in data corruption.
> Much better to redirect stderr using the construct {{2>/dev/null}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4178) shell scripts should not close
stderr
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13495906#comment-13495906 ]
Jason Lowe commented on HDFS-4178:
----------------------------------
The data corruption occurs when the process opens a new file descriptor for its data channel and gets file descriptor 2 (since it would be the next available due to the prior close). If code in libc and elsewhere is blindly assuming it can write messages to fd 2 and fd 2 is your custom data stream then that's data corruption.
> shell scripts should not close stderr
> -------------------------------------
>
> Key: HDFS-4178
> URL: https://issues.apache.org/jira/browse/HDFS-4178
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: scripts
> Affects Versions: 2.0.2-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs4178.txt
>
>
> The {{start-dfs.sh}} and {{stop-dfs.sh}} scripts close stderr for some subprocesses using the construct
> bq. {{2>&-}}
> This is dangerous because child processes started up under this scenario will re-use filedescriptor 2 for opened files. Since libc and many other codepaths assume that filedescriptor 2 can be written to in error conditions, this can potentially result in data corruption.
> Much better to redirect stderr using the construct {{2>/dev/null}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4178) shell scripts should not close
stderr
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500987#comment-13500987 ]
Hudson commented on HDFS-4178:
------------------------------
Integrated in Hadoop-Yarn-trunk #42 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/42/])
HDFS-4178. Shell scripts should not close stderr (Andy Isaacson via daryn) (Revision 1411229)
Result = SUCCESS
daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1411229
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-dfs.sh
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/stop-dfs.sh
> shell scripts should not close stderr
> -------------------------------------
>
> Key: HDFS-4178
> URL: https://issues.apache.org/jira/browse/HDFS-4178
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: scripts
> Affects Versions: 2.0.2-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Fix For: 3.0.0, 2.0.3-alpha
>
> Attachments: hdfs4178.txt
>
>
> The {{start-dfs.sh}} and {{stop-dfs.sh}} scripts close stderr for some subprocesses using the construct
> bq. {{2>&-}}
> This is dangerous because child processes started up under this scenario will re-use filedescriptor 2 for opened files. Since libc and many other codepaths assume that filedescriptor 2 can be written to in error conditions, this can potentially result in data corruption.
> Much better to redirect stderr using the construct {{2>/dev/null}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4178) shell scripts should not close
stderr
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13495672#comment-13495672 ]
Hadoop QA commented on HDFS-4178:
---------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12553175/hdfs4178.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3486//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3486//console
This message is automatically generated.
> shell scripts should not close stderr
> -------------------------------------
>
> Key: HDFS-4178
> URL: https://issues.apache.org/jira/browse/HDFS-4178
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: scripts
> Affects Versions: 2.0.2-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs4178.txt
>
>
> The {{start-dfs.sh}} and {{stop-dfs.sh}} scripts close stderr for some subprocesses using the construct
> bq. {{2>&-}}
> This is dangerous because child processes started up under this scenario will re-use filedescriptor 2 for opened files. Since libc and many other codepaths assume that filedescriptor 2 can be written to in error conditions, this can potentially result in data corruption.
> Much better to redirect stderr using the construct {{2>/dev/null}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4178) shell scripts should not close stderr
Posted by "Andy Isaacson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andy Isaacson updated HDFS-4178:
--------------------------------
Status: Patch Available (was: Open)
> shell scripts should not close stderr
> -------------------------------------
>
> Key: HDFS-4178
> URL: https://issues.apache.org/jira/browse/HDFS-4178
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: scripts
> Affects Versions: 2.0.2-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs4178.txt
>
>
> The {{start-dfs.sh}} and {{stop-dfs.sh}} scripts close stderr for some subprocesses using the construct
> bq. {{2>&-}}
> This is dangerous because child processes started up under this scenario will re-use filedescriptor 2 for opened files. Since libc and many other codepaths assume that filedescriptor 2 can be written to in error conditions, this can potentially result in data corruption.
> Much better to redirect stderr using the construct {{2>/dev/null}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4178) shell scripts should not close stderr
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HDFS-4178:
------------------------------
Resolution: Fixed
Fix Version/s: 2.0.3-alpha
3.0.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Yes, I have committed to trunk and branch-2. Thanks Andy! (I was under the misimpression that fd 0-2 were reserved unless explicit opened, or that they were redirected to /dev/null after dropping the controlling terminal)
> shell scripts should not close stderr
> -------------------------------------
>
> Key: HDFS-4178
> URL: https://issues.apache.org/jira/browse/HDFS-4178
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: scripts
> Affects Versions: 2.0.2-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Fix For: 3.0.0, 2.0.3-alpha
>
> Attachments: hdfs4178.txt
>
>
> The {{start-dfs.sh}} and {{stop-dfs.sh}} scripts close stderr for some subprocesses using the construct
> bq. {{2>&-}}
> This is dangerous because child processes started up under this scenario will re-use filedescriptor 2 for opened files. Since libc and many other codepaths assume that filedescriptor 2 can be written to in error conditions, this can potentially result in data corruption.
> Much better to redirect stderr using the construct {{2>/dev/null}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4178) shell scripts should not close
stderr
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13495727#comment-13495727 ]
Daryn Sharp commented on HDFS-4178:
-----------------------------------
+1 Although it's worth noting that I don't think data corruption will occur, just a EBADF (Bad file descriptor) error if anything attempts to write to stderr. I'll let there be a second opinion on this one.
> shell scripts should not close stderr
> -------------------------------------
>
> Key: HDFS-4178
> URL: https://issues.apache.org/jira/browse/HDFS-4178
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: scripts
> Affects Versions: 2.0.2-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs4178.txt
>
>
> The {{start-dfs.sh}} and {{stop-dfs.sh}} scripts close stderr for some subprocesses using the construct
> bq. {{2>&-}}
> This is dangerous because child processes started up under this scenario will re-use filedescriptor 2 for opened files. Since libc and many other codepaths assume that filedescriptor 2 can be written to in error conditions, this can potentially result in data corruption.
> Much better to redirect stderr using the construct {{2>/dev/null}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4178) shell scripts should not close stderr
Posted by "Andy Isaacson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andy Isaacson updated HDFS-4178:
--------------------------------
Attachment: hdfs4178.txt
Attaching patch to redirect to {{/dev/null}}.
> shell scripts should not close stderr
> -------------------------------------
>
> Key: HDFS-4178
> URL: https://issues.apache.org/jira/browse/HDFS-4178
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: scripts
> Affects Versions: 2.0.2-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs4178.txt
>
>
> The {{start-dfs.sh}} and {{stop-dfs.sh}} scripts close stderr for some subprocesses using the construct
> bq. {{2>&-}}
> This is dangerous because child processes started up under this scenario will re-use filedescriptor 2 for opened files. Since libc and many other codepaths assume that filedescriptor 2 can be written to in error conditions, this can potentially result in data corruption.
> Much better to redirect stderr using the construct {{2>/dev/null}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4178) shell scripts should not close
stderr
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501175#comment-13501175 ]
Hudson commented on HDFS-4178:
------------------------------
Integrated in Hadoop-Mapreduce-trunk #1263 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1263/])
HDFS-4178. Shell scripts should not close stderr (Andy Isaacson via daryn) (Revision 1411229)
Result = FAILURE
daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1411229
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-dfs.sh
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/stop-dfs.sh
> shell scripts should not close stderr
> -------------------------------------
>
> Key: HDFS-4178
> URL: https://issues.apache.org/jira/browse/HDFS-4178
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: scripts
> Affects Versions: 2.0.2-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Fix For: 3.0.0, 2.0.3-alpha
>
> Attachments: hdfs4178.txt
>
>
> The {{start-dfs.sh}} and {{stop-dfs.sh}} scripts close stderr for some subprocesses using the construct
> bq. {{2>&-}}
> This is dangerous because child processes started up under this scenario will re-use filedescriptor 2 for opened files. Since libc and many other codepaths assume that filedescriptor 2 can be written to in error conditions, this can potentially result in data corruption.
> Much better to redirect stderr using the construct {{2>/dev/null}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4178) shell scripts should not close
stderr
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500295#comment-13500295 ]
Hudson commented on HDFS-4178:
------------------------------
Integrated in Hadoop-trunk-Commit #3043 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3043/])
HDFS-4178. Shell scripts should not close stderr (Andy Isaacson via daryn) (Revision 1411229)
Result = SUCCESS
daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1411229
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-dfs.sh
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/stop-dfs.sh
> shell scripts should not close stderr
> -------------------------------------
>
> Key: HDFS-4178
> URL: https://issues.apache.org/jira/browse/HDFS-4178
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: scripts
> Affects Versions: 2.0.2-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs4178.txt
>
>
> The {{start-dfs.sh}} and {{stop-dfs.sh}} scripts close stderr for some subprocesses using the construct
> bq. {{2>&-}}
> This is dangerous because child processes started up under this scenario will re-use filedescriptor 2 for opened files. Since libc and many other codepaths assume that filedescriptor 2 can be written to in error conditions, this can potentially result in data corruption.
> Much better to redirect stderr using the construct {{2>/dev/null}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4178) shell scripts should not close
stderr
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501156#comment-13501156 ]
Hudson commented on HDFS-4178:
------------------------------
Integrated in Hadoop-Hdfs-trunk #1232 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1232/])
HDFS-4178. Shell scripts should not close stderr (Andy Isaacson via daryn) (Revision 1411229)
Result = SUCCESS
daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1411229
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-dfs.sh
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/stop-dfs.sh
> shell scripts should not close stderr
> -------------------------------------
>
> Key: HDFS-4178
> URL: https://issues.apache.org/jira/browse/HDFS-4178
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: scripts
> Affects Versions: 2.0.2-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Fix For: 3.0.0, 2.0.3-alpha
>
> Attachments: hdfs4178.txt
>
>
> The {{start-dfs.sh}} and {{stop-dfs.sh}} scripts close stderr for some subprocesses using the construct
> bq. {{2>&-}}
> This is dangerous because child processes started up under this scenario will re-use filedescriptor 2 for opened files. Since libc and many other codepaths assume that filedescriptor 2 can be written to in error conditions, this can potentially result in data corruption.
> Much better to redirect stderr using the construct {{2>/dev/null}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4178) shell scripts should not close
stderr
Posted by "Andy Isaacson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496622#comment-13496622 ]
Andy Isaacson commented on HDFS-4178:
-------------------------------------
bq. The data corruption occurs when the process opens a new file descriptor for its data channel and gets file descriptor 2
Yep, exactly. If an app does {{fopen("datafile", "w+")}} (or the Java equivalent) and is run under {{someapp 2>&-}} the file {{datafile}} will be opened for write as filedescriptor 2. If the app then triggers glibc's "bad free() detected" codepath (among many others), the error message will potentially be written to the current file position in {{datafile}}.
(glibc has a workaround for the most common case, it opens {{/dev/tty}} for the error message in interactive sessions, but it still does corrupt output files when run in a noninteractive session with {{cron}}, {{at}}, {{batch}}, or at startup time from {{/etc/init.d}}.)
> shell scripts should not close stderr
> -------------------------------------
>
> Key: HDFS-4178
> URL: https://issues.apache.org/jira/browse/HDFS-4178
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: scripts
> Affects Versions: 2.0.2-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs4178.txt
>
>
> The {{start-dfs.sh}} and {{stop-dfs.sh}} scripts close stderr for some subprocesses using the construct
> bq. {{2>&-}}
> This is dangerous because child processes started up under this scenario will re-use filedescriptor 2 for opened files. Since libc and many other codepaths assume that filedescriptor 2 can be written to in error conditions, this can potentially result in data corruption.
> Much better to redirect stderr using the construct {{2>/dev/null}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4178) shell scripts should not close
stderr
Posted by "Andy Isaacson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500496#comment-13500496 ]
Andy Isaacson commented on HDFS-4178:
-------------------------------------
bq. (I was under the misimpression that fd 0-2 were reserved unless explicit opened, or that they were redirected to /dev/null after dropping the controlling terminal)
Yeah, it's really surprising that this pitfall is left lurking for people to stumble into! There's no credible use case for leaving fd 0,1,2 closed during process startup and it would be a huge win for {{_start}} to open {{/dev/null}} as appropriate before running {{main()}}, but unfortunately I've confirmed that this is not done and I actually did experimentally trigger the "glibc detected bad free"-in-my-datafile failure mode under glibc 2.13 running with {{at}}.
Thanks for committing the fix!
> shell scripts should not close stderr
> -------------------------------------
>
> Key: HDFS-4178
> URL: https://issues.apache.org/jira/browse/HDFS-4178
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: scripts
> Affects Versions: 2.0.2-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Fix For: 3.0.0, 2.0.3-alpha
>
> Attachments: hdfs4178.txt
>
>
> The {{start-dfs.sh}} and {{stop-dfs.sh}} scripts close stderr for some subprocesses using the construct
> bq. {{2>&-}}
> This is dangerous because child processes started up under this scenario will re-use filedescriptor 2 for opened files. Since libc and many other codepaths assume that filedescriptor 2 can be written to in error conditions, this can potentially result in data corruption.
> Much better to redirect stderr using the construct {{2>/dev/null}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira