You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Konstantin Boudnik <co...@apache.org> on 2015/11/21 04:05:28 UTC
Re: IGFS: properties to use for implicit mkdirs
On Tue, Oct 13, 2015 at 08:01PM, Ivan Veselovskiy wrote:
> Currently we have #create, #append , #mkdirs operations that implicitly
> create parent directories if they are absent.
> Now #mkdirs uses the properties passed in for the implicitly created
> directories if they are not null, and uses default properties (with 0777
> permission flag) if the properties are not given.
> #create & #append use for the implicitly created directories properties
> passed in for newly created file, if the passed in properties are not null,
> and use default properties (with 0777 permission flag) if they are not
> given.
> The question is: wouldn't it be more logical to always use defaults for the
> implicitly created directories?
Looks like this is way to late, but still... Perhaps more logical would be to
do what Unix does: if permissions aren't explicitly set during creation,
those should be inherited from the parent.
Does it makes sense?
Cos
Re: IGFS: properties to use for implicit mkdirs
Posted by Konstantin Boudnik <co...@apache.org>.
On Wed, Nov 25, 2015 at 09:26PM, Ivan V. wrote:
> umask is 0002 on my Linux. (This is why the middle directories have 775
> permission in the above experiment. 022 would lead to 755.)
Yeah, sorry - that's what I was going to say and the mixed up with my own ;)
Looks like the proposed fix makes sense, actually. Thanks for taking time
walking me through.
> On Mon, Nov 23, 2015 at 10:48 PM, Konstantin Boudnik <co...@apache.org> wrote:
>
> > May I ask what's is the output of umask command on your system?
> >
> > Cos
> >
> > P.S. I have a hunch though it is 0022
> >
> > On Mon, Nov 23, 2015 at 02:01PM, Ivan V. wrote:
> > > Hi, Konstantin,
> > >
> > > >Perhaps more logical would be to do what Unix does: if permissions
> > aren't
> > > explicitly set during creation, those should be inherited from the
> > parent.
> > >
> > > as per my observations on Linux, the implicitly created directories take
> > > default permissions 0777 & ~umask but not inherit them from parent.
> > > E.g.
> > > /bin/mkdir -m 700 a
> > > /bin/mkdir -p -m 770 a/b/c/d
> > >
> > > gives the following result:
> > > a 700
> > > a/b 775
> > > a/b/c 775
> > > a/b/c/d 770
> > >
> > > ('touch' Unix command does not have ability to create parent
> > directories.)
> > > Or you mean a behavior of a built-in system function, but not the
> > standard
> > > commands?
> > >
> > > Currently suggested patch (see
> > > https://issues.apache.org/jira/browse/IGNITE-1637) fixes the behavior
> > > exactly as it is implemented in HDFS (because main function of IGFS is to
> > > be a replacement or a caching layer over HDFS):
> > > --------------------------------
> > >
> > > If a file is created, the created directories get
> > > org.apache.hadoop.fs.permission.FsPermission#getDirDefault (0777)
> > > permission. Umask *is* applied to the created file, but is *not *applied
> > to
> > > the created directories.
> > >
> > > If a directory is created, all the parent directories are created with
> > the
> > > same permission that is requested for the directory itself. The only
> > > exception is that "wx" (write and execute) permissions are explicitly
> > added
> > > for the owner on the parent directories to be able to write child
> > > directories. Umask *is* applied to all the created directories.
> > > If umask conflicts with the added permissions , the added permissions
> > take
> > > priority.
> > > --------------------------------
> > >
> > > Though, there is no problem to re-implement this in another way. Just
> > would
> > > like to make sure we have a proposal that is argued and agreed over the
> > > community.
> > >
> > > On Sat, Nov 21, 2015 at 6:37 PM, Dmitriy Setrakyan <
> > dsetrakyan@apache.org>
> > > wrote:
> > >
> > > > On Fri, Nov 20, 2015 at 7:05 PM, Konstantin Boudnik <co...@apache.org>
> > > > wrote:
> > > >
> > > > > On Tue, Oct 13, 2015 at 08:01PM, Ivan Veselovskiy wrote:
> > > > > > Currently we have #create, #append , #mkdirs operations that
> > implicitly
> > > > > > create parent directories if they are absent.
> > > > > > Now #mkdirs uses the properties passed in for the implicitly
> > created
> > > > > > directories if they are not null, and uses default properties (with
> > > > 0777
> > > > > > permission flag) if the properties are not given.
> > > > > > #create & #append use for the implicitly created directories
> > properties
> > > > > > passed in for newly created file, if the passed in properties are
> > not
> > > > > null,
> > > > > > and use default properties (with 0777 permission flag) if they are
> > not
> > > > > > given.
> > > > > > The question is: wouldn't it be more logical to always use
> > defaults for
> > > > > the
> > > > > > implicitly created directories?
> > > > >
> > > > > Looks like this is way to late, but still... Perhaps more logical
> > would
> > > > be
> > > > > to
> > > > > do what Unix does: if permissions aren't explicitly set during
> > creation,
> > > > > those should be inherited from the parent.
> > > > >
> > > > > Does it makes sense?
> > > > >
> > > >
> > > > Makes sense to me.
> > > >
> > > >
> > > > > Cos
> > > > >
> > > >
> >
Re: IGFS: properties to use for implicit mkdirs
Posted by "Ivan V." <iv...@gridgain.com>.
umask is 0002 on my Linux. (This is why the middle directories have 775
permission in the above experiment. 022 would lead to 755.)
On Mon, Nov 23, 2015 at 10:48 PM, Konstantin Boudnik <co...@apache.org> wrote:
> May I ask what's is the output of umask command on your system?
>
> Cos
>
> P.S. I have a hunch though it is 0022
>
> On Mon, Nov 23, 2015 at 02:01PM, Ivan V. wrote:
> > Hi, Konstantin,
> >
> > >Perhaps more logical would be to do what Unix does: if permissions
> aren't
> > explicitly set during creation, those should be inherited from the
> parent.
> >
> > as per my observations on Linux, the implicitly created directories take
> > default permissions 0777 & ~umask but not inherit them from parent.
> > E.g.
> > /bin/mkdir -m 700 a
> > /bin/mkdir -p -m 770 a/b/c/d
> >
> > gives the following result:
> > a 700
> > a/b 775
> > a/b/c 775
> > a/b/c/d 770
> >
> > ('touch' Unix command does not have ability to create parent
> directories.)
> > Or you mean a behavior of a built-in system function, but not the
> standard
> > commands?
> >
> > Currently suggested patch (see
> > https://issues.apache.org/jira/browse/IGNITE-1637) fixes the behavior
> > exactly as it is implemented in HDFS (because main function of IGFS is to
> > be a replacement or a caching layer over HDFS):
> > --------------------------------
> >
> > If a file is created, the created directories get
> > org.apache.hadoop.fs.permission.FsPermission#getDirDefault (0777)
> > permission. Umask *is* applied to the created file, but is *not *applied
> to
> > the created directories.
> >
> > If a directory is created, all the parent directories are created with
> the
> > same permission that is requested for the directory itself. The only
> > exception is that "wx" (write and execute) permissions are explicitly
> added
> > for the owner on the parent directories to be able to write child
> > directories. Umask *is* applied to all the created directories.
> > If umask conflicts with the added permissions , the added permissions
> take
> > priority.
> > --------------------------------
> >
> > Though, there is no problem to re-implement this in another way. Just
> would
> > like to make sure we have a proposal that is argued and agreed over the
> > community.
> >
> > On Sat, Nov 21, 2015 at 6:37 PM, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> > wrote:
> >
> > > On Fri, Nov 20, 2015 at 7:05 PM, Konstantin Boudnik <co...@apache.org>
> > > wrote:
> > >
> > > > On Tue, Oct 13, 2015 at 08:01PM, Ivan Veselovskiy wrote:
> > > > > Currently we have #create, #append , #mkdirs operations that
> implicitly
> > > > > create parent directories if they are absent.
> > > > > Now #mkdirs uses the properties passed in for the implicitly
> created
> > > > > directories if they are not null, and uses default properties (with
> > > 0777
> > > > > permission flag) if the properties are not given.
> > > > > #create & #append use for the implicitly created directories
> properties
> > > > > passed in for newly created file, if the passed in properties are
> not
> > > > null,
> > > > > and use default properties (with 0777 permission flag) if they are
> not
> > > > > given.
> > > > > The question is: wouldn't it be more logical to always use
> defaults for
> > > > the
> > > > > implicitly created directories?
> > > >
> > > > Looks like this is way to late, but still... Perhaps more logical
> would
> > > be
> > > > to
> > > > do what Unix does: if permissions aren't explicitly set during
> creation,
> > > > those should be inherited from the parent.
> > > >
> > > > Does it makes sense?
> > > >
> > >
> > > Makes sense to me.
> > >
> > >
> > > > Cos
> > > >
> > >
>
Re: IGFS: properties to use for implicit mkdirs
Posted by Konstantin Boudnik <co...@apache.org>.
May I ask what's is the output of umask command on your system?
Cos
P.S. I have a hunch though it is 0022
On Mon, Nov 23, 2015 at 02:01PM, Ivan V. wrote:
> Hi, Konstantin,
>
> >Perhaps more logical would be to do what Unix does: if permissions aren't
> explicitly set during creation, those should be inherited from the parent.
>
> as per my observations on Linux, the implicitly created directories take
> default permissions 0777 & ~umask but not inherit them from parent.
> E.g.
> /bin/mkdir -m 700 a
> /bin/mkdir -p -m 770 a/b/c/d
>
> gives the following result:
> a 700
> a/b 775
> a/b/c 775
> a/b/c/d 770
>
> ('touch' Unix command does not have ability to create parent directories.)
> Or you mean a behavior of a built-in system function, but not the standard
> commands?
>
> Currently suggested patch (see
> https://issues.apache.org/jira/browse/IGNITE-1637) fixes the behavior
> exactly as it is implemented in HDFS (because main function of IGFS is to
> be a replacement or a caching layer over HDFS):
> --------------------------------
>
> If a file is created, the created directories get
> org.apache.hadoop.fs.permission.FsPermission#getDirDefault (0777)
> permission. Umask *is* applied to the created file, but is *not *applied to
> the created directories.
>
> If a directory is created, all the parent directories are created with the
> same permission that is requested for the directory itself. The only
> exception is that "wx" (write and execute) permissions are explicitly added
> for the owner on the parent directories to be able to write child
> directories. Umask *is* applied to all the created directories.
> If umask conflicts with the added permissions , the added permissions take
> priority.
> --------------------------------
>
> Though, there is no problem to re-implement this in another way. Just would
> like to make sure we have a proposal that is argued and agreed over the
> community.
>
> On Sat, Nov 21, 2015 at 6:37 PM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
>
> > On Fri, Nov 20, 2015 at 7:05 PM, Konstantin Boudnik <co...@apache.org>
> > wrote:
> >
> > > On Tue, Oct 13, 2015 at 08:01PM, Ivan Veselovskiy wrote:
> > > > Currently we have #create, #append , #mkdirs operations that implicitly
> > > > create parent directories if they are absent.
> > > > Now #mkdirs uses the properties passed in for the implicitly created
> > > > directories if they are not null, and uses default properties (with
> > 0777
> > > > permission flag) if the properties are not given.
> > > > #create & #append use for the implicitly created directories properties
> > > > passed in for newly created file, if the passed in properties are not
> > > null,
> > > > and use default properties (with 0777 permission flag) if they are not
> > > > given.
> > > > The question is: wouldn't it be more logical to always use defaults for
> > > the
> > > > implicitly created directories?
> > >
> > > Looks like this is way to late, but still... Perhaps more logical would
> > be
> > > to
> > > do what Unix does: if permissions aren't explicitly set during creation,
> > > those should be inherited from the parent.
> > >
> > > Does it makes sense?
> > >
> >
> > Makes sense to me.
> >
> >
> > > Cos
> > >
> >
Re: IGFS: properties to use for implicit mkdirs
Posted by "Ivan V." <iv...@gridgain.com>.
Hi, Konstantin,
>Perhaps more logical would be to do what Unix does: if permissions aren't
explicitly set during creation, those should be inherited from the parent.
as per my observations on Linux, the implicitly created directories take
default permissions 0777 & ~umask but not inherit them from parent.
E.g.
/bin/mkdir -m 700 a
/bin/mkdir -p -m 770 a/b/c/d
gives the following result:
a 700
a/b 775
a/b/c 775
a/b/c/d 770
('touch' Unix command does not have ability to create parent directories.)
Or you mean a behavior of a built-in system function, but not the standard
commands?
Currently suggested patch (see
https://issues.apache.org/jira/browse/IGNITE-1637) fixes the behavior
exactly as it is implemented in HDFS (because main function of IGFS is to
be a replacement or a caching layer over HDFS):
--------------------------------
If a file is created, the created directories get
org.apache.hadoop.fs.permission.FsPermission#getDirDefault (0777)
permission. Umask *is* applied to the created file, but is *not *applied to
the created directories.
If a directory is created, all the parent directories are created with the
same permission that is requested for the directory itself. The only
exception is that "wx" (write and execute) permissions are explicitly added
for the owner on the parent directories to be able to write child
directories. Umask *is* applied to all the created directories.
If umask conflicts with the added permissions , the added permissions take
priority.
--------------------------------
Though, there is no problem to re-implement this in another way. Just would
like to make sure we have a proposal that is argued and agreed over the
community.
On Sat, Nov 21, 2015 at 6:37 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:
> On Fri, Nov 20, 2015 at 7:05 PM, Konstantin Boudnik <co...@apache.org>
> wrote:
>
> > On Tue, Oct 13, 2015 at 08:01PM, Ivan Veselovskiy wrote:
> > > Currently we have #create, #append , #mkdirs operations that implicitly
> > > create parent directories if they are absent.
> > > Now #mkdirs uses the properties passed in for the implicitly created
> > > directories if they are not null, and uses default properties (with
> 0777
> > > permission flag) if the properties are not given.
> > > #create & #append use for the implicitly created directories properties
> > > passed in for newly created file, if the passed in properties are not
> > null,
> > > and use default properties (with 0777 permission flag) if they are not
> > > given.
> > > The question is: wouldn't it be more logical to always use defaults for
> > the
> > > implicitly created directories?
> >
> > Looks like this is way to late, but still... Perhaps more logical would
> be
> > to
> > do what Unix does: if permissions aren't explicitly set during creation,
> > those should be inherited from the parent.
> >
> > Does it makes sense?
> >
>
> Makes sense to me.
>
>
> > Cos
> >
>
Re: IGFS: properties to use for implicit mkdirs
Posted by Dmitriy Setrakyan <ds...@apache.org>.
On Fri, Nov 20, 2015 at 7:05 PM, Konstantin Boudnik <co...@apache.org> wrote:
> On Tue, Oct 13, 2015 at 08:01PM, Ivan Veselovskiy wrote:
> > Currently we have #create, #append , #mkdirs operations that implicitly
> > create parent directories if they are absent.
> > Now #mkdirs uses the properties passed in for the implicitly created
> > directories if they are not null, and uses default properties (with 0777
> > permission flag) if the properties are not given.
> > #create & #append use for the implicitly created directories properties
> > passed in for newly created file, if the passed in properties are not
> null,
> > and use default properties (with 0777 permission flag) if they are not
> > given.
> > The question is: wouldn't it be more logical to always use defaults for
> the
> > implicitly created directories?
>
> Looks like this is way to late, but still... Perhaps more logical would be
> to
> do what Unix does: if permissions aren't explicitly set during creation,
> those should be inherited from the parent.
>
> Does it makes sense?
>
Makes sense to me.
> Cos
>