You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@accumulo.apache.org by Bulldog20630405 <bu...@gmail.com> on 2022/02/28 15:12:25 UTC
accumulo doesnt respect umask
we have set our user umask to 0027; however, the "accumulo data" table
directories get written as 755 and rfiles as 644
how to you configure acucmulo so it will create data and write rfiles with
a specific permissions?
we want our accumulo to write as 750/640
Re: accumulo doesnt respect umask
Posted by Bulldog20630405 <bu...@gmail.com>.
thanx Dave...
we are using hadoop 3.x on a local file system
what we did to fix this problem:
add the following to core-site.xml:
*fs.permissions.umask-mode*
instead of setting *dfs.permissions.umask-mode* in hdfs-site.xml
On Thu, Mar 3, 2022 at 10:31 AM Dave Marion <dl...@comcast.net> wrote:
> So, the question of which local file system implementation may be moot.
> https://issues.apache.org/jira/browse/HADOOP-16582, which affects a wide
> array of versions, says:
>
> "When mkdirs(Path) is called against LocalFileSystem, the implementation
> in RawLocalFileSystem is called and the directory permission is
> determined by the umask."
>
> Are you using a version of Hadoop < 3.0.0 ?
>
> On 03/03/2022 9:26 AM Dave Marion <dl...@comcast.net> wrote:
>
>
> Which version of Hadoop, and are you using RawLocalFileSystem or
> LocalFileSystem. I see a reference to HADOOP-11347 resolving this issue in
> RawLocalFileSystem.
>
> On 03/02/2022 11:44 AM Bulldog20630405 <bu...@gmail.com> wrote:
>
>
> we are using accumulo 1.10.x
> we are using a local file system...
> but we also set it in hdfs-site.xml (just in case accumulo picks it up)
> and we have our umask set as 0027 in our bashrc and in our default
> supervisord conf (we use supervisord to manage our services)...
> note: spark respect our umask setting so they are working; just not for
> accumulo
>
> thanx for your info; that helps us try to track down a solution
>
>
>
>
> On Mon, Feb 28, 2022 at 4:56 PM Christopher <ct...@apache.org> wrote:
>
> I don't think we're doing anything specific to set the permissions, so
> they should respect your environment setup. Which version of Accumulo
> are you using? And, are you referring to permissions in HDFS, a local
> filesystem implementation of DFS, or something else?
>
> If you're referring to files created on HDFS, are you following the
> instructions on
>
> https://hadoop.apache.org/docs/r3.1.1/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html
> to configure HDFS?
>
> On Mon, Feb 28, 2022 at 10:12 AM Bulldog20630405
> <bu...@gmail.com> wrote:
> >
> >
> > we have set our user umask to 0027; however, the "accumulo data" table
> directories get written as 755 and rfiles as 644
> >
> > how to you configure acucmulo so it will create data and write rfiles
> with a specific permissions?
> > we want our accumulo to write as 750/640
> >
> >
>
>
Re: accumulo doesnt respect umask
Posted by Dave Marion <dl...@comcast.net>.
So, the question of which local file system implementation may be moot. https://issues.apache.org/jira/browse/HADOOP-16582, which affects a wide array of versions, says:
"When mkdirs(Path) is called against LocalFileSystem, the implementation in RawLocalFileSystem is called and the directory permission is determined by the umask."
Are you using a version of Hadoop < 3.0.0 ?
> On 03/03/2022 9:26 AM Dave Marion <dl...@comcast.net> wrote:
>
>
> Which version of Hadoop, and are you using RawLocalFileSystem or LocalFileSystem. I see a reference to HADOOP-11347 resolving this issue in RawLocalFileSystem.
>
> > > On 03/02/2022 11:44 AM Bulldog20630405 <bu...@gmail.com> wrote:
> >
> >
> > we are using accumulo 1.10.x
> > we are using a local file system...
> > but we also set it in hdfs-site.xml (just in case accumulo picks it up)
> > and we have our umask set as 0027 in our bashrc and in our default supervisord conf (we use supervisord to manage our services)...
> > note: spark respect our umask setting so they are working; just not for accumulo
> >
> > thanx for your info; that helps us try to track down a solution
> >
> >
> >
> >
> > On Mon, Feb 28, 2022 at 4:56 PM Christopher <ctubbsii@apache.org mailto:ctubbsii@apache.org > wrote:
> >
> > > > > I don't think we're doing anything specific to set the permissions, so
> > > they should respect your environment setup. Which version of Accumulo
> > > are you using? And, are you referring to permissions in HDFS, a local
> > > filesystem implementation of DFS, or something else?
> > >
> > > If you're referring to files created on HDFS, are you following the
> > > instructions on
> > > https://hadoop.apache.org/docs/r3.1.1/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html
> > > to configure HDFS?
> > >
> > > On Mon, Feb 28, 2022 at 10:12 AM Bulldog20630405
> > > <bulldog20630405@gmail.com mailto:bulldog20630405@gmail.com > wrote:
> > > >
> > > >
> > > > we have set our user umask to 0027; however, the "accumulo data" table directories get written as 755 and rfiles as 644
> > > >
> > > > how to you configure acucmulo so it will create data and write rfiles with a specific permissions?
> > > > we want our accumulo to write as 750/640
> > > >
> > > >
> > >
> > > > >
> > >
Re: accumulo doesnt respect umask
Posted by Dave Marion <dl...@comcast.net>.
Which version of Hadoop, and are you using RawLocalFileSystem or LocalFileSystem. I see a reference to HADOOP-11347 resolving this issue in RawLocalFileSystem.
> On 03/02/2022 11:44 AM Bulldog20630405 <bu...@gmail.com> wrote:
>
>
> we are using accumulo 1.10.x
> we are using a local file system...
> but we also set it in hdfs-site.xml (just in case accumulo picks it up)
> and we have our umask set as 0027 in our bashrc and in our default supervisord conf (we use supervisord to manage our services)...
> note: spark respect our umask setting so they are working; just not for accumulo
>
> thanx for your info; that helps us try to track down a solution
>
>
>
>
> On Mon, Feb 28, 2022 at 4:56 PM Christopher <ctubbsii@apache.org mailto:ctubbsii@apache.org > wrote:
>
> > > I don't think we're doing anything specific to set the permissions, so
> > they should respect your environment setup. Which version of Accumulo
> > are you using? And, are you referring to permissions in HDFS, a local
> > filesystem implementation of DFS, or something else?
> >
> > If you're referring to files created on HDFS, are you following the
> > instructions on
> > https://hadoop.apache.org/docs/r3.1.1/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html
> > to configure HDFS?
> >
> > On Mon, Feb 28, 2022 at 10:12 AM Bulldog20630405
> > <bulldog20630405@gmail.com mailto:bulldog20630405@gmail.com > wrote:
> > >
> > >
> > > we have set our user umask to 0027; however, the "accumulo data" table directories get written as 755 and rfiles as 644
> > >
> > > how to you configure acucmulo so it will create data and write rfiles with a specific permissions?
> > > we want our accumulo to write as 750/640
> > >
> > >
> >
> > >
Re: accumulo doesnt respect umask
Posted by Bulldog20630405 <bu...@gmail.com>.
we are using accumulo 1.10.x
we are using a local file system...
but we also set it in hdfs-site.xml (just in case accumulo picks it up)
and we have our umask set as 0027 in our bashrc and in our default
supervisord conf (we use supervisord to manage our services)...
note: spark respect our umask setting so they are working; just not for
accumulo
thanx for your info; that helps us try to track down a solution
On Mon, Feb 28, 2022 at 4:56 PM Christopher <ct...@apache.org> wrote:
> I don't think we're doing anything specific to set the permissions, so
> they should respect your environment setup. Which version of Accumulo
> are you using? And, are you referring to permissions in HDFS, a local
> filesystem implementation of DFS, or something else?
>
> If you're referring to files created on HDFS, are you following the
> instructions on
>
> https://hadoop.apache.org/docs/r3.1.1/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html
> to configure HDFS?
>
> On Mon, Feb 28, 2022 at 10:12 AM Bulldog20630405
> <bu...@gmail.com> wrote:
> >
> >
> > we have set our user umask to 0027; however, the "accumulo data" table
> directories get written as 755 and rfiles as 644
> >
> > how to you configure acucmulo so it will create data and write rfiles
> with a specific permissions?
> > we want our accumulo to write as 750/640
> >
> >
>
Re: accumulo doesnt respect umask
Posted by Christopher <ct...@apache.org>.
I don't think we're doing anything specific to set the permissions, so
they should respect your environment setup. Which version of Accumulo
are you using? And, are you referring to permissions in HDFS, a local
filesystem implementation of DFS, or something else?
If you're referring to files created on HDFS, are you following the
instructions on
https://hadoop.apache.org/docs/r3.1.1/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html
to configure HDFS?
On Mon, Feb 28, 2022 at 10:12 AM Bulldog20630405
<bu...@gmail.com> wrote:
>
>
> we have set our user umask to 0027; however, the "accumulo data" table directories get written as 755 and rfiles as 644
>
> how to you configure acucmulo so it will create data and write rfiles with a specific permissions?
> we want our accumulo to write as 750/640
>
>