You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Jay Vyas <ja...@gmail.com> on 2013/04/12 13:11:37 UTC
JobSubmissionFiles: past , present, and future?
Hi guys:
I'm curious about the changes and future of the JobSubmissionFiles class.
Grepping around on the web I'm finding some code snippets that suggest that hadoop security is not handled the same way on the staging directory as before:
http://javasourcecode.org/html/open-source/hadoop/hadoop-0.20.203.0/org/apache/hadoop/mapreduce/JobSubmissionFiles.java.html
http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-user/201210.mbox/%3CCAOcnVr0EYLSckXaOCPNm7kBZWpHvCdjBBX5A+AzeS_S6PwsbkQ@mail.gmail.com%3E
But I'm having trouble definitively pinning this to versions.
Why the difference in the if/else logic here and what is the future
Of permissions on .staging?
Re: JobSubmissionFiles: past , present, and future?
Posted by Jay Vyas <ja...@gmail.com>.
To update on this, it was just pointed out to me by matt farrallee
that the auto fix of permissions is for a failsafe
in case of a race condition, and not meant to mend bad permissions in all cases:
https://github.com/apache/hadoop-common/commit/f25dc04795a0e9836e3f237c802bfc1fe8a243ad
Something to keep in mind - if you see the "fixing staging permissions" error message alot
Then there might be a more systemic problem in your fs... At least, that was the case for us.
On Apr 12, 2013, at 6:11 AM, Jay Vyas <ja...@gmail.com> wrote:
> Hi guys:
>
> I'm curious about the changes and future of the JobSubmissionFiles class.
>
> Grepping around on the web I'm finding some code snippets that suggest that hadoop security is not handled the same way on the staging directory as before:
>
> http://javasourcecode.org/html/open-source/hadoop/hadoop-0.20.203.0/org/apache/hadoop/mapreduce/JobSubmissionFiles.java.html
>
> http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-user/201210.mbox/%3CCAOcnVr0EYLSckXaOCPNm7kBZWpHvCdjBBX5A+AzeS_S6PwsbkQ@mail.gmail.com%3E
>
> But I'm having trouble definitively pinning this to versions.
>
> Why the difference in the if/else logic here and what is the future
> Of permissions on .staging?