You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2010/09/28 21:48:33 UTC

[jira] Created: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities

Secure local filesystem IO from symlink vulnerabilities
-------------------------------------------------------

                 Key: MAPREDUCE-2096
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: jobtracker, security, tasktracker
    Affects Versions: 0.22.0
            Reporter: Todd Lipcon
            Assignee: Todd Lipcon
            Priority: Blocker


This JIRA is to contribute a patch developed on the private security@ mailing list.

The vulnerability is that MR daemons occasionally open files that are located in a path where the user has write access. A malicious user may place a symlink in place of the expected file in order to cause the daemon to instead read another file on the system -- one which the attacker may not naturally be able to access. This includes delegation tokens belong to other users, log files, keytabs, etc.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12929884#action_12929884 ] 

Todd Lipcon commented on MAPREDUCE-2096:
----------------------------------------

Thanks for reminding me to upload the trunk work. I put a patch up on HADOOP-6978 which blocks this. Working on the MR trunk patch as well while that one gets reviewed/committed.

> Secure local filesystem IO from symlink vulnerabilities
> -------------------------------------------------------
>
>                 Key: MAPREDUCE-2096
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker, security, tasktracker
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Blocker
>         Attachments: mapreduce-2096-index-oob.txt, secure-files-9.txt, secure-files-authorized-jvm-fix.txt
>
>
> This JIRA is to contribute a patch developed on the private security@ mailing list.
> The vulnerability is that MR daemons occasionally open files that are located in a path where the user has write access. A malicious user may place a symlink in place of the expected file in order to cause the daemon to instead read another file on the system -- one which the attacker may not naturally be able to access. This includes delegation tokens belong to other users, log files, keytabs, etc.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities

Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918656#action_12918656 ] 

Devaraj Das commented on MAPREDUCE-2096:
----------------------------------------

bq. This patch fixes the above issue by having TaskRunner call to a version of reportDiagonsticInfo that doesn't authorize the caller.
The same should be done for fsError.

> Secure local filesystem IO from symlink vulnerabilities
> -------------------------------------------------------
>
>                 Key: MAPREDUCE-2096
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker, security, tasktracker
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Blocker
>         Attachments: secure-files-9.txt, secure-files-authorized-jvm-fix.txt
>
>
> This JIRA is to contribute a patch developed on the private security@ mailing list.
> The vulnerability is that MR daemons occasionally open files that are located in a path where the user has write access. A malicious user may place a symlink in place of the expected file in order to cause the daemon to instead read another file on the system -- one which the attacker may not naturally be able to access. This includes delegation tokens belong to other users, log files, keytabs, etc.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Todd Lipcon updated MAPREDUCE-2096:
-----------------------------------

    Attachment: mapreduce-2096-index-oob.txt

Another small bug fix - if a task fails before it produces any logs (eg if taskcontroller is misconfigured) then this patch will throw an IndexOutOfBoundsException in the TaskLogsTruncater. I think this patch should fix it.

> Secure local filesystem IO from symlink vulnerabilities
> -------------------------------------------------------
>
>                 Key: MAPREDUCE-2096
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker, security, tasktracker
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Blocker
>         Attachments: mapreduce-2096-index-oob.txt, secure-files-9.txt, secure-files-authorized-jvm-fix.txt
>
>
> This JIRA is to contribute a patch developed on the private security@ mailing list.
> The vulnerability is that MR daemons occasionally open files that are located in a path where the user has write access. A malicious user may place a symlink in place of the expected file in order to cause the daemon to instead read another file on the system -- one which the attacker may not naturally be able to access. This includes delegation tokens belong to other users, log files, keytabs, etc.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Todd Lipcon updated MAPREDUCE-2096:
-----------------------------------

    Attachment: secure-files-authorized-jvm-fix.txt

In testing I came upon one spot where the new JVM authorization in this patch was incorrectly triggered:

2010-10-03 22:30:36,252 WARN org.apache.hadoop.mapred.TaskRunner: attempt_201010032228_0001_m_000001_2 Reporting Diagnostics
java.io.IOException: JVM with mapred is not authorized for job_201010032228_0001
        at org.apache.hadoop.mapred.TaskTracker.ensureAuthorizedJVM(TaskTracker.java:3096)
        at org.apache.hadoop.mapred.TaskTracker.reportDiagnosticInfo(TaskTracker.java:3164)
        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:232)

This patch fixes the above issue by having TaskRunner call to a version of reportDiagonsticInfo that doesn't authorize the caller.

> Secure local filesystem IO from symlink vulnerabilities
> -------------------------------------------------------
>
>                 Key: MAPREDUCE-2096
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker, security, tasktracker
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Blocker
>         Attachments: secure-files-9.txt, secure-files-authorized-jvm-fix.txt
>
>
> This JIRA is to contribute a patch developed on the private security@ mailing list.
> The vulnerability is that MR daemons occasionally open files that are located in a path where the user has write access. A malicious user may place a symlink in place of the expected file in order to cause the daemon to instead read another file on the system -- one which the attacker may not naturally be able to access. This includes delegation tokens belong to other users, log files, keytabs, etc.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities

Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12929844#action_12929844 ] 

Owen O'Malley commented on MAPREDUCE-2096:
------------------------------------------

How is the trunk patch going for this one?

> Secure local filesystem IO from symlink vulnerabilities
> -------------------------------------------------------
>
>                 Key: MAPREDUCE-2096
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker, security, tasktracker
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Blocker
>         Attachments: mapreduce-2096-index-oob.txt, secure-files-9.txt, secure-files-authorized-jvm-fix.txt
>
>
> This JIRA is to contribute a patch developed on the private security@ mailing list.
> The vulnerability is that MR daemons occasionally open files that are located in a path where the user has write access. A malicious user may place a symlink in place of the expected file in order to cause the daemon to instead read another file on the system -- one which the attacker may not naturally be able to access. This includes delegation tokens belong to other users, log files, keytabs, etc.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities

Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12915919#action_12915919 ] 

Owen O'Malley commented on MAPREDUCE-2096:
------------------------------------------

I think we need to move away from checking in any of the autoconf/automake generated files. Let's run autoreconf everytime and make the ignored by subversion and git.

It should also be noted that Devaraj contributed significantly to this patch.

> Secure local filesystem IO from symlink vulnerabilities
> -------------------------------------------------------
>
>                 Key: MAPREDUCE-2096
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker, security, tasktracker
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Blocker
>         Attachments: secure-files-9.txt
>
>
> This JIRA is to contribute a patch developed on the private security@ mailing list.
> The vulnerability is that MR daemons occasionally open files that are located in a path where the user has write access. A malicious user may place a symlink in place of the expected file in order to cause the daemon to instead read another file on the system -- one which the attacker may not naturally be able to access. This includes delegation tokens belong to other users, log files, keytabs, etc.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Todd Lipcon updated MAPREDUCE-2096:
-----------------------------------

    Attachment: secure-files-9.txt

Here's the 0.20-based patch that we developed on the security mailing list. We'll use this JIRA to update it to trunk (with the common half going to HADOOP-6978).

We also just identified one more place that we need to do this behavior - jobconf_history.jsp needs to use the secure IO code.

> Secure local filesystem IO from symlink vulnerabilities
> -------------------------------------------------------
>
>                 Key: MAPREDUCE-2096
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker, security, tasktracker
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Blocker
>         Attachments: secure-files-9.txt
>
>
> This JIRA is to contribute a patch developed on the private security@ mailing list.
> The vulnerability is that MR daemons occasionally open files that are located in a path where the user has write access. A malicious user may place a symlink in place of the expected file in order to cause the daemon to instead read another file on the system -- one which the attacker may not naturally be able to access. This includes delegation tokens belong to other users, log files, keytabs, etc.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAPREDUCE-2096) Secure local filesystem IO from symlink vulnerabilities

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12929892#action_12929892 ] 

Todd Lipcon commented on MAPREDUCE-2096:
----------------------------------------

Does anyone have a suggestion on how to get common's native library build onto mapreduce's library path post-split?

It seems we should be publishing a tarball of common/build/native into maven, and then retrieving it with ivy from mapreduce, perhaps? Does anyone have a better idea or should I open a JIRA to publish the native build as an artifact?

> Secure local filesystem IO from symlink vulnerabilities
> -------------------------------------------------------
>
>                 Key: MAPREDUCE-2096
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2096
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker, security, tasktracker
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Blocker
>         Attachments: mapreduce-2096-index-oob.txt, secure-files-9.txt, secure-files-authorized-jvm-fix.txt
>
>
> This JIRA is to contribute a patch developed on the private security@ mailing list.
> The vulnerability is that MR daemons occasionally open files that are located in a path where the user has write access. A malicious user may place a symlink in place of the expected file in order to cause the daemon to instead read another file on the system -- one which the attacker may not naturally be able to access. This includes delegation tokens belong to other users, log files, keytabs, etc.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.