You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2010/09/28 21:48:33 UTC
[jira] Created: (MAPREDUCE-2096) Secure local filesystem IO from
symlink vulnerabilities
Secure local filesystem IO from symlink vulnerabilities
-------------------------------------------------------
Key: MAPREDUCE-2096
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobtracker, security, tasktracker
Affects Versions: 0.22.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Blocker
This JIRA is to contribute a patch developed on the private security@ mailing list.
The vulnerability is that MR daemons occasionally open files that are located in a path where the user has write access. A malicious user may place a symlink in place of the expected file in order to cause the daemon to instead read another file on the system -- one which the attacker may not naturally be able to access. This includes delegation tokens belong to other users, log files, keytabs, etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from
symlink vulnerabilities
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12929884#action_12929884 ]
Todd Lipcon commented on MAPREDUCE-2096:
----------------------------------------
Thanks for reminding me to upload the trunk work. I put a patch up on HADOOP-6978 which blocks this. Working on the MR trunk patch as well while that one gets reviewed/committed.
> Secure local filesystem IO from symlink vulnerabilities
> -------------------------------------------------------
>
> Key: MAPREDUCE-2096
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker, security, tasktracker
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Attachments: mapreduce-2096-index-oob.txt, secure-files-9.txt, secure-files-authorized-jvm-fix.txt
>
>
> This JIRA is to contribute a patch developed on the private security@ mailing list.
> The vulnerability is that MR daemons occasionally open files that are located in a path where the user has write access. A malicious user may place a symlink in place of the expected file in order to cause the daemon to instead read another file on the system -- one which the attacker may not naturally be able to access. This includes delegation tokens belong to other users, log files, keytabs, etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from
symlink vulnerabilities
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918656#action_12918656 ]
Devaraj Das commented on MAPREDUCE-2096:
----------------------------------------
bq. This patch fixes the above issue by having TaskRunner call to a version of reportDiagonsticInfo that doesn't authorize the caller.
The same should be done for fsError.
> Secure local filesystem IO from symlink vulnerabilities
> -------------------------------------------------------
>
> Key: MAPREDUCE-2096
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker, security, tasktracker
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Attachments: secure-files-9.txt, secure-files-authorized-jvm-fix.txt
>
>
> This JIRA is to contribute a patch developed on the private security@ mailing list.
> The vulnerability is that MR daemons occasionally open files that are located in a path where the user has write access. A malicious user may place a symlink in place of the expected file in order to cause the daemon to instead read another file on the system -- one which the attacker may not naturally be able to access. This includes delegation tokens belong to other users, log files, keytabs, etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2096) Secure local filesystem IO from
symlink vulnerabilities
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated MAPREDUCE-2096:
-----------------------------------
Attachment: mapreduce-2096-index-oob.txt
Another small bug fix - if a task fails before it produces any logs (eg if taskcontroller is misconfigured) then this patch will throw an IndexOutOfBoundsException in the TaskLogsTruncater. I think this patch should fix it.
> Secure local filesystem IO from symlink vulnerabilities
> -------------------------------------------------------
>
> Key: MAPREDUCE-2096
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker, security, tasktracker
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Attachments: mapreduce-2096-index-oob.txt, secure-files-9.txt, secure-files-authorized-jvm-fix.txt
>
>
> This JIRA is to contribute a patch developed on the private security@ mailing list.
> The vulnerability is that MR daemons occasionally open files that are located in a path where the user has write access. A malicious user may place a symlink in place of the expected file in order to cause the daemon to instead read another file on the system -- one which the attacker may not naturally be able to access. This includes delegation tokens belong to other users, log files, keytabs, etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2096) Secure local filesystem IO from
symlink vulnerabilities
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated MAPREDUCE-2096:
-----------------------------------
Attachment: secure-files-authorized-jvm-fix.txt
In testing I came upon one spot where the new JVM authorization in this patch was incorrectly triggered:
2010-10-03 22:30:36,252 WARN org.apache.hadoop.mapred.TaskRunner: attempt_201010032228_0001_m_000001_2 Reporting Diagnostics
java.io.IOException: JVM with mapred is not authorized for job_201010032228_0001
at org.apache.hadoop.mapred.TaskTracker.ensureAuthorizedJVM(TaskTracker.java:3096)
at org.apache.hadoop.mapred.TaskTracker.reportDiagnosticInfo(TaskTracker.java:3164)
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:232)
This patch fixes the above issue by having TaskRunner call to a version of reportDiagonsticInfo that doesn't authorize the caller.
> Secure local filesystem IO from symlink vulnerabilities
> -------------------------------------------------------
>
> Key: MAPREDUCE-2096
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker, security, tasktracker
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Attachments: secure-files-9.txt, secure-files-authorized-jvm-fix.txt
>
>
> This JIRA is to contribute a patch developed on the private security@ mailing list.
> The vulnerability is that MR daemons occasionally open files that are located in a path where the user has write access. A malicious user may place a symlink in place of the expected file in order to cause the daemon to instead read another file on the system -- one which the attacker may not naturally be able to access. This includes delegation tokens belong to other users, log files, keytabs, etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from
symlink vulnerabilities
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12929844#action_12929844 ]
Owen O'Malley commented on MAPREDUCE-2096:
------------------------------------------
How is the trunk patch going for this one?
> Secure local filesystem IO from symlink vulnerabilities
> -------------------------------------------------------
>
> Key: MAPREDUCE-2096
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker, security, tasktracker
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Attachments: mapreduce-2096-index-oob.txt, secure-files-9.txt, secure-files-authorized-jvm-fix.txt
>
>
> This JIRA is to contribute a patch developed on the private security@ mailing list.
> The vulnerability is that MR daemons occasionally open files that are located in a path where the user has write access. A malicious user may place a symlink in place of the expected file in order to cause the daemon to instead read another file on the system -- one which the attacker may not naturally be able to access. This includes delegation tokens belong to other users, log files, keytabs, etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from
symlink vulnerabilities
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12915919#action_12915919 ]
Owen O'Malley commented on MAPREDUCE-2096:
------------------------------------------
I think we need to move away from checking in any of the autoconf/automake generated files. Let's run autoreconf everytime and make the ignored by subversion and git.
It should also be noted that Devaraj contributed significantly to this patch.
> Secure local filesystem IO from symlink vulnerabilities
> -------------------------------------------------------
>
> Key: MAPREDUCE-2096
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker, security, tasktracker
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Attachments: secure-files-9.txt
>
>
> This JIRA is to contribute a patch developed on the private security@ mailing list.
> The vulnerability is that MR daemons occasionally open files that are located in a path where the user has write access. A malicious user may place a symlink in place of the expected file in order to cause the daemon to instead read another file on the system -- one which the attacker may not naturally be able to access. This includes delegation tokens belong to other users, log files, keytabs, etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2096) Secure local filesystem IO from
symlink vulnerabilities
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated MAPREDUCE-2096:
-----------------------------------
Attachment: secure-files-9.txt
Here's the 0.20-based patch that we developed on the security mailing list. We'll use this JIRA to update it to trunk (with the common half going to HADOOP-6978).
We also just identified one more place that we need to do this behavior - jobconf_history.jsp needs to use the secure IO code.
> Secure local filesystem IO from symlink vulnerabilities
> -------------------------------------------------------
>
> Key: MAPREDUCE-2096
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker, security, tasktracker
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Attachments: secure-files-9.txt
>
>
> This JIRA is to contribute a patch developed on the private security@ mailing list.
> The vulnerability is that MR daemons occasionally open files that are located in a path where the user has write access. A malicious user may place a symlink in place of the expected file in order to cause the daemon to instead read another file on the system -- one which the attacker may not naturally be able to access. This includes delegation tokens belong to other users, log files, keytabs, etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from
symlink vulnerabilities
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12929892#action_12929892 ]
Todd Lipcon commented on MAPREDUCE-2096:
----------------------------------------
Does anyone have a suggestion on how to get common's native library build onto mapreduce's library path post-split?
It seems we should be publishing a tarball of common/build/native into maven, and then retrieving it with ivy from mapreduce, perhaps? Does anyone have a better idea or should I open a JIRA to publish the native build as an artifact?
> Secure local filesystem IO from symlink vulnerabilities
> -------------------------------------------------------
>
> Key: MAPREDUCE-2096
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker, security, tasktracker
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Attachments: mapreduce-2096-index-oob.txt, secure-files-9.txt, secure-files-authorized-jvm-fix.txt
>
>
> This JIRA is to contribute a patch developed on the private security@ mailing list.
> The vulnerability is that MR daemons occasionally open files that are located in a path where the user has write access. A malicious user may place a symlink in place of the expected file in order to cause the daemon to instead read another file on the system -- one which the attacker may not naturally be able to access. This includes delegation tokens belong to other users, log files, keytabs, etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.