You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@impala.apache.org by Sunil Parmar <su...@gmail.com> on 2017/12/05 21:27:06 UTC
INSERT query generated file permissions
Environment CDH 5.12
When running INSERT query on the table all files are always owned by user
impala i.e. 744, all for impala read for everyone else . We have externally
running compaction process which needs to read/write/replace this files. Is
there a way to change this default behavior to have the file permission
different than 744 ? I'd prefer if it's 764 ( group read/write ) so we can
add the user to the same group as Impala who run the compaction process.
Sunil Parmar
Re: INSERT query generated file permissions
Posted by Jim Apple <jb...@cloudera.com>.
I'm not sure. When you tried it, what did you see?
On Wed, Dec 6, 2017 at 10:49 AM, Sunil Parmar <su...@gmail.com> wrote:
> Sure got it!
> One more related Impala question. The documentation for insert inherit
> permissions says.
>
> "If true, new directories created by INSERTs will inherit the permissions of
> their parent directories"
>
> Does it also applies to the files ?
>
>
> Sunil Parmar
>
> On Wed, Dec 6, 2017 at 10:27 AM, Jim Apple <jb...@cloudera.com> wrote:
>>
>> That sounds right to me, but we try to keep this list focused on Apache
>> Impala and not any proprietary tools around it. You might have more success
>> here:
>> http://community.cloudera.com/t5/Interactive-Short-cycle-SQL/bd-p/Impala
>>
>> On Wed, Dec 6, 2017 at 10:13 AM, Sunil Parmar <su...@gmail.com>
>> wrote:
>>>
>>> Thanks !! How to set insert_inherit_permissions from Cloudera manager /
>>> which security valve config to use and what to set in the value ?
>>>
>>> Impala Daemon Environment Advanced Configuration Snippet (Safety Valve) ?
>>>
>>> Sunil Parmar
>>>
>>> On Wed, Dec 6, 2017 at 6:50 AM, Petter von Dolwitz (Hem)
>>> <pe...@gmail.com> wrote:
>>>>
>>>> You can also explore HDFS ACLs to solve this problem. This is something
>>>> we use to give different user groups different privileges. See
>>>> https://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#ACLs_Shell_Commands
>>>>
>>>> Br,
>>>> Petter
>>>>
>>>> 2017-12-06 1:12 GMT+01:00 Jim Apple <jb...@cloudera.com>:
>>>>>
>>>>> I'm not sure. Some things that come to mind:
>>>>>
>>>>> Did you try setting a umask for the impala user?
>>>>>
>>>>> Did you try setting the parent directory mode bits to 764 and then
>>>>> using the insert_inherit_permissions flag?
>>>>>
>>>>> On Tue, Dec 5, 2017 at 1:27 PM, Sunil Parmar <su...@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Environment CDH 5.12
>>>>>>
>>>>>> When running INSERT query on the table all files are always owned by
>>>>>> user impala i.e. 744, all for impala read for everyone else . We have
>>>>>> externally running compaction process which needs to read/write/replace this
>>>>>> files. Is there a way to change this default behavior to have the file
>>>>>> permission different than 744 ? I'd prefer if it's 764 ( group read/write )
>>>>>> so we can add the user to the same group as Impala who run the compaction
>>>>>> process.
>>>>>>
>>>>>> Sunil Parmar
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: INSERT query generated file permissions
Posted by Sunil Parmar <su...@gmail.com>.
Sure got it!
One more related Impala question. The documentation for insert inherit
permissions says.
"If true, new directories created by INSERTs will inherit the permissions
of their parent directories"
Does it also applies to the files ?
Sunil Parmar
On Wed, Dec 6, 2017 at 10:27 AM, Jim Apple <jb...@cloudera.com> wrote:
> That sounds right to me, but we try to keep this list focused on Apache
> Impala and not any proprietary tools around it. You might have more success
> here: http://community.cloudera.com/t5/Interactive-
> Short-cycle-SQL/bd-p/Impala
>
> On Wed, Dec 6, 2017 at 10:13 AM, Sunil Parmar <su...@gmail.com>
> wrote:
>
>> Thanks !! How to set insert_inherit_permissions from Cloudera manager /
>> which security valve config to use and what to set in the value ?
>>
>> Impala Daemon Environment Advanced Configuration Snippet (Safety Valve) ?
>>
>> Sunil Parmar
>>
>> On Wed, Dec 6, 2017 at 6:50 AM, Petter von Dolwitz (Hem) <
>> petter.von.dolwitz@gmail.com> wrote:
>>
>>> You can also explore HDFS ACLs to solve this problem. This is something
>>> we use to give different user groups different privileges. See
>>> https://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/ha
>>> doop-hdfs/HdfsPermissionsGuide.html#ACLs_Shell_Commands
>>>
>>> Br,
>>> Petter
>>>
>>> 2017-12-06 1:12 GMT+01:00 Jim Apple <jb...@cloudera.com>:
>>>
>>>> I'm not sure. Some things that come to mind:
>>>>
>>>> Did you try setting a umask for the impala user?
>>>>
>>>> Did you try setting the parent directory mode bits to 764 and then
>>>> using the insert_inherit_permissions flag?
>>>>
>>>> On Tue, Dec 5, 2017 at 1:27 PM, Sunil Parmar <su...@gmail.com>
>>>> wrote:
>>>>
>>>>> Environment CDH 5.12
>>>>>
>>>>> When running INSERT query on the table all files are always owned by
>>>>> user impala i.e. 744, all for impala read for everyone else . We have
>>>>> externally running compaction process which needs to read/write/replace
>>>>> this files. Is there a way to change this default behavior to have the file
>>>>> permission different than 744 ? I'd prefer if it's 764 ( group read/write )
>>>>> so we can add the user to the same group as Impala who run the compaction
>>>>> process.
>>>>>
>>>>> Sunil Parmar
>>>>>
>>>>
>>>>
>>>
>>
>
Re: INSERT query generated file permissions
Posted by Jim Apple <jb...@cloudera.com>.
That sounds right to me, but we try to keep this list focused on Apache
Impala and not any proprietary tools around it. You might have more success
here:
http://community.cloudera.com/t5/Interactive-Short-cycle-SQL/bd-p/Impala
On Wed, Dec 6, 2017 at 10:13 AM, Sunil Parmar <su...@gmail.com> wrote:
> Thanks !! How to set insert_inherit_permissions from Cloudera manager /
> which security valve config to use and what to set in the value ?
>
> Impala Daemon Environment Advanced Configuration Snippet (Safety Valve) ?
>
> Sunil Parmar
>
> On Wed, Dec 6, 2017 at 6:50 AM, Petter von Dolwitz (Hem) <
> petter.von.dolwitz@gmail.com> wrote:
>
>> You can also explore HDFS ACLs to solve this problem. This is something
>> we use to give different user groups different privileges. See
>> https://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/
>> hadoop-hdfs/HdfsPermissionsGuide.html#ACLs_Shell_Commands
>>
>> Br,
>> Petter
>>
>> 2017-12-06 1:12 GMT+01:00 Jim Apple <jb...@cloudera.com>:
>>
>>> I'm not sure. Some things that come to mind:
>>>
>>> Did you try setting a umask for the impala user?
>>>
>>> Did you try setting the parent directory mode bits to 764 and then using
>>> the insert_inherit_permissions flag?
>>>
>>> On Tue, Dec 5, 2017 at 1:27 PM, Sunil Parmar <su...@gmail.com>
>>> wrote:
>>>
>>>> Environment CDH 5.12
>>>>
>>>> When running INSERT query on the table all files are always owned by
>>>> user impala i.e. 744, all for impala read for everyone else . We have
>>>> externally running compaction process which needs to read/write/replace
>>>> this files. Is there a way to change this default behavior to have the file
>>>> permission different than 744 ? I'd prefer if it's 764 ( group read/write )
>>>> so we can add the user to the same group as Impala who run the compaction
>>>> process.
>>>>
>>>> Sunil Parmar
>>>>
>>>
>>>
>>
>
Re: INSERT query generated file permissions
Posted by Sunil Parmar <su...@gmail.com>.
Thanks !! How to set insert_inherit_permissions from Cloudera manager /
which security valve config to use and what to set in the value ?
Impala Daemon Environment Advanced Configuration Snippet (Safety Valve) ?
Sunil Parmar
On Wed, Dec 6, 2017 at 6:50 AM, Petter von Dolwitz (Hem) <
petter.von.dolwitz@gmail.com> wrote:
> You can also explore HDFS ACLs to solve this problem. This is something we
> use to give different user groups different privileges. See
> https://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/
> HdfsPermissionsGuide.html#ACLs_Shell_Commands
>
> Br,
> Petter
>
> 2017-12-06 1:12 GMT+01:00 Jim Apple <jb...@cloudera.com>:
>
>> I'm not sure. Some things that come to mind:
>>
>> Did you try setting a umask for the impala user?
>>
>> Did you try setting the parent directory mode bits to 764 and then using
>> the insert_inherit_permissions flag?
>>
>> On Tue, Dec 5, 2017 at 1:27 PM, Sunil Parmar <su...@gmail.com>
>> wrote:
>>
>>> Environment CDH 5.12
>>>
>>> When running INSERT query on the table all files are always owned by
>>> user impala i.e. 744, all for impala read for everyone else . We have
>>> externally running compaction process which needs to read/write/replace
>>> this files. Is there a way to change this default behavior to have the file
>>> permission different than 744 ? I'd prefer if it's 764 ( group read/write )
>>> so we can add the user to the same group as Impala who run the compaction
>>> process.
>>>
>>> Sunil Parmar
>>>
>>
>>
>
Re: INSERT query generated file permissions
Posted by "Petter von Dolwitz (Hem)" <pe...@gmail.com>.
You can also explore HDFS ACLs to solve this problem. This is something we
use to give different user groups different privileges. See
https://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#ACLs_Shell_Commands
Br,
Petter
2017-12-06 1:12 GMT+01:00 Jim Apple <jb...@cloudera.com>:
> I'm not sure. Some things that come to mind:
>
> Did you try setting a umask for the impala user?
>
> Did you try setting the parent directory mode bits to 764 and then using
> the insert_inherit_permissions flag?
>
> On Tue, Dec 5, 2017 at 1:27 PM, Sunil Parmar <su...@gmail.com>
> wrote:
>
>> Environment CDH 5.12
>>
>> When running INSERT query on the table all files are always owned by user
>> impala i.e. 744, all for impala read for everyone else . We have externally
>> running compaction process which needs to read/write/replace this files. Is
>> there a way to change this default behavior to have the file permission
>> different than 744 ? I'd prefer if it's 764 ( group read/write ) so we can
>> add the user to the same group as Impala who run the compaction process.
>>
>> Sunil Parmar
>>
>
>
Re: INSERT query generated file permissions
Posted by Jim Apple <jb...@cloudera.com>.
I'm not sure. Some things that come to mind:
Did you try setting a umask for the impala user?
Did you try setting the parent directory mode bits to 764 and then using
the insert_inherit_permissions flag?
On Tue, Dec 5, 2017 at 1:27 PM, Sunil Parmar <su...@gmail.com> wrote:
> Environment CDH 5.12
>
> When running INSERT query on the table all files are always owned by user
> impala i.e. 744, all for impala read for everyone else . We have externally
> running compaction process which needs to read/write/replace this files. Is
> there a way to change this default behavior to have the file permission
> different than 744 ? I'd prefer if it's 764 ( group read/write ) so we can
> add the user to the same group as Impala who run the compaction process.
>
> Sunil Parmar
>