You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2011/01/18 18:31:43 UTC

[jira] Created: (HADOOP-7110) Implement chmod with JNI

Implement chmod with JNI
------------------------

                 Key: HADOOP-7110
                 URL: https://issues.apache.org/jira/browse/HADOOP-7110
             Project: Hadoop Common
          Issue Type: Improvement
          Components: io, native
    Affects Versions: 0.22.0
            Reporter: Todd Lipcon
            Assignee: Todd Lipcon
             Fix For: 0.22.0


MapReduce is currently using a race-prone workaround to approximate chmod() because forking chmod is too expensive. This race is causing build failures (and probably task failures too). We should implement chmod in the NativeIO library so we can have good performance (ie not fork) and still not be prone to races.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7110) Implement chmod with JNI

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983583#action_12983583 ] 

Hadoop QA commented on HADOOP-7110:
-----------------------------------

+1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12468660/hadoop-7110.txt
  against trunk revision 1060632.

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 3 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

    +1 core tests.  The patch passed core unit tests.

    +1 contrib tests.  The patch passed contrib unit tests.

    +1 system test framework.  The patch passed system test framework compile.

Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/185//testReport/
Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/185//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/185//console

This message is automatically generated.

> Implement chmod with JNI
> ------------------------
>
>                 Key: HADOOP-7110
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7110
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, native
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0
>
>         Attachments: hadoop-7110.txt
>
>
> MapReduce is currently using a race-prone workaround to approximate chmod() because forking chmod is too expensive. This race is causing build failures (and probably task failures too). We should implement chmod in the NativeIO library so we can have good performance (ie not fork) and still not be prone to races.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7110) Implement chmod with JNI

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989131#comment-12989131 ] 

Hudson commented on HADOOP-7110:
--------------------------------

Integrated in Hadoop-Common-trunk-Commit #492 (See [https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/492/])
    

> Implement chmod with JNI
> ------------------------
>
>                 Key: HADOOP-7110
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7110
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, native
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0
>
>         Attachments: hadoop-7110.txt
>
>
> MapReduce is currently using a race-prone workaround to approximate chmod() because forking chmod is too expensive. This race is causing build failures (and probably task failures too). We should implement chmod in the NativeIO library so we can have good performance (ie not fork) and still not be prone to races.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] Commented: (HADOOP-7110) Implement chmod with JNI

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986338#action_12986338 ] 

Hudson commented on HADOOP-7110:
--------------------------------

Integrated in Hadoop-Common-trunk #586 (See [https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/586/])
    HADOOP-7110. Implement chmod with JNI. Contributed by Todd Lipcon


> Implement chmod with JNI
> ------------------------
>
>                 Key: HADOOP-7110
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7110
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, native
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0
>
>         Attachments: hadoop-7110.txt
>
>
> MapReduce is currently using a race-prone workaround to approximate chmod() because forking chmod is too expensive. This race is causing build failures (and probably task failures too). We should implement chmod in the NativeIO library so we can have good performance (ie not fork) and still not be prone to races.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7110) Implement chmod with JNI

Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983455#action_12983455 ] 

Konstantin Shvachko commented on HADOOP-7110:
---------------------------------------------

What is wrong with the current implementation and why native is better?
Could you please explain.

> Implement chmod with JNI
> ------------------------
>
>                 Key: HADOOP-7110
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7110
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, native
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0
>
>         Attachments: hadoop-7110.txt
>
>
> MapReduce is currently using a race-prone workaround to approximate chmod() because forking chmod is too expensive. This race is causing build failures (and probably task failures too). We should implement chmod in the NativeIO library so we can have good performance (ie not fork) and still not be prone to races.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7110) Implement chmod with JNI

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983335#action_12983335 ] 

Eli Collins commented on HADOOP-7110:
-------------------------------------

+1  looks great

> Implement chmod with JNI
> ------------------------
>
>                 Key: HADOOP-7110
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7110
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, native
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0
>
>         Attachments: hadoop-7110.txt
>
>
> MapReduce is currently using a race-prone workaround to approximate chmod() because forking chmod is too expensive. This race is causing build failures (and probably task failures too). We should implement chmod in the NativeIO library so we can have good performance (ie not fork) and still not be prone to races.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-7110) Implement chmod with JNI

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Todd Lipcon updated HADOOP-7110:
--------------------------------

    Status: Patch Available  (was: Open)

> Implement chmod with JNI
> ------------------------
>
>                 Key: HADOOP-7110
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7110
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, native
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0
>
>         Attachments: hadoop-7110.txt
>
>
> MapReduce is currently using a race-prone workaround to approximate chmod() because forking chmod is too expensive. This race is causing build failures (and probably task failures too). We should implement chmod in the NativeIO library so we can have good performance (ie not fork) and still not be prone to races.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7110) Implement chmod with JNI

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983805#action_12983805 ] 

Todd Lipcon commented on HADOOP-7110:
-------------------------------------

bq. So if it fails right after setting them to 000 it will be hard to remove such file or directory

It's not due to failure of the process doing the chmod. It's due to other processes or threads working inside the directory failing because there is a moment in which they can't traverse the path inside the directory. As we've seen in MR-2238 this race really does happen - we've got build failures on three separate environments (Apache Hudson, my Hudson, and Greg's build box) to prove it.

bq. So why don't we just set permissions to the values specified (in the right order, which is probably important) rather than introducing more JNI code

The API is not flexible enough to do this. If you have a file 755 and want it to be 700, the only way to get rid of the group/other bits is to first clear those bits for all, then add them back for user only.

bq. Nobody said we should support posix permissions

Since we now support security, we have to be able to make some files mode 700 and others 755, etc. There's no way around it. The question here is implementation method, not whether we should support an API since we've had since 2008. Removing the ability to chmod isn't an option, considering compatibility.

bq. Your patch is an alternative implementation of HADOOP-6304

And HADOOP-6304 is a sort-of-copy-paste of code originally introduced in MAPREDUCE-842. I filed this JIRA as part of the fix for MAPREDUCE-2238 (which is caused by the MR code that has the same problems)


Do you have a particular issue with the JNI code?

> Implement chmod with JNI
> ------------------------
>
>                 Key: HADOOP-7110
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7110
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, native
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0
>
>         Attachments: hadoop-7110.txt
>
>
> MapReduce is currently using a race-prone workaround to approximate chmod() because forking chmod is too expensive. This race is causing build failures (and probably task failures too). We should implement chmod in the NativeIO library so we can have good performance (ie not fork) and still not be prone to races.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7110) Implement chmod with JNI

Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983621#action_12983621 ] 

Konstantin Shvachko commented on HADOOP-7110:
---------------------------------------------

Thanks for explanation, Todd. I understand now. As I see it the problem you observed in MAPREDUCE-2238 is caused by that {{Localizer.PermissionsHandler}} first clears the permissions by setting them to 000, then setting the ones passed. So if it fails right after setting them to 000 it will be hard to remove such file or directory.
So why don't we just set permissions to the values specified (in the right order, which is probably important) rather than introducing more JNI code. Nobody said we should support posix permissions.
Your patch is an alternative implementation of HADOOP-6304. You should have attached it to that jira rather than creating the new one. Now the discussion is split into parts, making it hard to follow.

> Implement chmod with JNI
> ------------------------
>
>                 Key: HADOOP-7110
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7110
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, native
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0
>
>         Attachments: hadoop-7110.txt
>
>
> MapReduce is currently using a race-prone workaround to approximate chmod() because forking chmod is too expensive. This race is causing build failures (and probably task failures too). We should implement chmod in the NativeIO library so we can have good performance (ie not fork) and still not be prone to races.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7110) Implement chmod with JNI

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983453#action_12983453 ] 

Todd Lipcon commented on HADOOP-7110:
-------------------------------------

Ran tests with compile.native=1 and without compile.native, both passed (except for HADOOP-7111 issues)

I'll commit this this evening.

> Implement chmod with JNI
> ------------------------
>
>                 Key: HADOOP-7110
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7110
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, native
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0
>
>         Attachments: hadoop-7110.txt
>
>
> MapReduce is currently using a race-prone workaround to approximate chmod() because forking chmod is too expensive. This race is causing build failures (and probably task failures too). We should implement chmod in the NativeIO library so we can have good performance (ie not fork) and still not be prone to races.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7110) Implement chmod with JNI

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986929#action_12986929 ] 

Hudson commented on HADOOP-7110:
--------------------------------

Integrated in Hadoop-Common-22-branch #19 (See [https://hudson.apache.org/hudson/job/Hadoop-Common-22-branch/19/])
    HADOOP-7110. Implement chmod with JNI. Contributed by Todd Lipcon


> Implement chmod with JNI
> ------------------------
>
>                 Key: HADOOP-7110
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7110
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, native
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0
>
>         Attachments: hadoop-7110.txt
>
>
> MapReduce is currently using a race-prone workaround to approximate chmod() because forking chmod is too expensive. This race is causing build failures (and probably task failures too). We should implement chmod in the NativeIO library so we can have good performance (ie not fork) and still not be prone to races.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-7110) Implement chmod with JNI

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Todd Lipcon updated HADOOP-7110:
--------------------------------

    Attachment: hadoop-7110.txt

> Implement chmod with JNI
> ------------------------
>
>                 Key: HADOOP-7110
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7110
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, native
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0
>
>         Attachments: hadoop-7110.txt
>
>
> MapReduce is currently using a race-prone workaround to approximate chmod() because forking chmod is too expensive. This race is causing build failures (and probably task failures too). We should implement chmod in the NativeIO library so we can have good performance (ie not fork) and still not be prone to races.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7110) Implement chmod with JNI

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983474#action_12983474 ] 

Todd Lipcon commented on HADOOP-7110:
-------------------------------------

Konstantin: depends which "current implementation" you're referring to.

In trunk right now, RawLocalFS uses the equivalent of system("chmod ... <path>") to perform a chmod. This is very inefficient, since Java uses fork, not vfork, and thus incurs a cost to set up new vm_area structures for the child process, etc. As I imagine you've seen before, if the process has a very large heap (many GB) this can take several milliseconds.

HADOOP-6304 identifies this issue (in fact calls it an "emergency bug fix") and proposes using the Java setReadable|setWritable|setExecutable APIs instead. These don't have the cost of a fork, but have two problems:
1) They have limited power since in typical java fashion are a least-common-denominator API. That is to say, they don't support setting a file to arbitrary modes, have no concept of "group" ownership, etc. If you'll look at the implementation in that JIRA you'll see what I mean.
2) Because the APIs are awkward, the only way to get reasonably close to full chmod support is to first chmod a file down to 000 and then work back up to the correct permissions. This is race prone and has been causing build failures for several months (see MAPREDUCE-2238)

So, this JIRA adds the JNI call which fixes all three issues issues. It has the full ability to chmod to any mode, is atomic, and is very efficient.

Does that answer the question?

> Implement chmod with JNI
> ------------------------
>
>                 Key: HADOOP-7110
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7110
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, native
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0
>
>         Attachments: hadoop-7110.txt
>
>
> MapReduce is currently using a race-prone workaround to approximate chmod() because forking chmod is too expensive. This race is causing build failures (and probably task failures too). We should implement chmod in the NativeIO library so we can have good performance (ie not fork) and still not be prone to races.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7110) Implement chmod with JNI

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985486#action_12985486 ] 

Todd Lipcon commented on HADOOP-7110:
-------------------------------------

Thanks. I will commit this in the next day or two if there are no other objections.

> Implement chmod with JNI
> ------------------------
>
>                 Key: HADOOP-7110
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7110
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, native
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0
>
>         Attachments: hadoop-7110.txt
>
>
> MapReduce is currently using a race-prone workaround to approximate chmod() because forking chmod is too expensive. This race is causing build failures (and probably task failures too). We should implement chmod in the NativeIO library so we can have good performance (ie not fork) and still not be prone to races.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7110) Implement chmod with JNI

Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985485#action_12985485 ] 

Konstantin Shvachko commented on HADOOP-7110:
---------------------------------------------

JNI should be used as the last resort, imo.
And it seems this is the case in this situation. I don't see other reasonable way implementing permissions setting.
+1

> Implement chmod with JNI
> ------------------------
>
>                 Key: HADOOP-7110
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7110
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, native
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0
>
>         Attachments: hadoop-7110.txt
>
>
> MapReduce is currently using a race-prone workaround to approximate chmod() because forking chmod is too expensive. This race is causing build failures (and probably task failures too). We should implement chmod in the NativeIO library so we can have good performance (ie not fork) and still not be prone to races.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7110) Implement chmod with JNI

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983280#action_12983280 ] 

Todd Lipcon commented on HADOOP-7110:
-------------------------------------

Although it would be useful to add a bunch of other JNI wrappers (eg chgrp, stat, readlink) I'd like to just do chmod in this JIRA. The reasoning is that chmod in particular is necessary to fix the MR build, so this needs to go into 0.22, whereas the others are truly improvements.

If others disagree, happy to write the wrappers.

> Implement chmod with JNI
> ------------------------
>
>                 Key: HADOOP-7110
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7110
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, native
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0
>
>         Attachments: hadoop-7110.txt
>
>
> MapReduce is currently using a race-prone workaround to approximate chmod() because forking chmod is too expensive. This race is causing build failures (and probably task failures too). We should implement chmod in the NativeIO library so we can have good performance (ie not fork) and still not be prone to races.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-7110) Implement chmod with JNI

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Todd Lipcon updated HADOOP-7110:
--------------------------------

      Resolution: Fixed
    Hadoop Flags: [Reviewed]
          Status: Resolved  (was: Patch Available)

Committed to trunk and branch - branch because, though this is an "optimization", without this optimization serious issues can arise (Arun called HADOOP-6304 an "emergency bug fix" which served the same purpose)

> Implement chmod with JNI
> ------------------------
>
>                 Key: HADOOP-7110
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7110
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, native
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0
>
>         Attachments: hadoop-7110.txt
>
>
> MapReduce is currently using a race-prone workaround to approximate chmod() because forking chmod is too expensive. This race is causing build failures (and probably task failures too). We should implement chmod in the NativeIO library so we can have good performance (ie not fork) and still not be prone to races.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.