Re: Spill Location, permissions and Authentication

[Boaz Ben-Zvi <bb...@mapr.com>]

The permissions on the spill locations are s part of the setup configuration — these base locations should have permissions matching the OS userid used for the DrillBit .
The DrillBit creates those directories (with that long name 274a41…….) with its userid.
So in the example below, the user zetasvcprod should have the permissions on the directory drillprod .
The newer versions of Drill ( 1.9, 1.10 ) should work the same way.

       Boaz

On Feb 28, 2017, at 10:24 AM, John Omernik <jo...@omernik.com>> wrote:

I am using 1.8 as of this time, if this is fixed in a newer version, please
let me know.

I am running drill as a master user. (mapr) and have enabled
authentication. When I authenticate to drill, and try to run a query,
specifically one with hash joins turned off (thus using spill to disk), I
am getting error that my user (that I authenticated to sqlline with) does
not have access to my drill spill location. (They are MapR "local" volumes
that I have setup myself).

So, I was going to go adjust permissions, but I started thinking more...
Why isn't the spill location, when authentication is running, or when it's
not, always reading/writing as the drill bit user?  This doesn't make a lot
of sense that users would get to read this information, why not remove the
impersonation on requests to read and write?

Now, I am trying to think this through, so there may be tons of reasons,
but I would love to understand this process better.

Thanks!

John



Caused by: org.apache.hadoop.security.AccessControlException: User
zetasvcprod(user id 2000000) does not have access to
/var/local/zeta8/local/drillspill/drillprod/274a41a8-ae92-6b0f-8dab-3b16841c0e35_majorfragment1_minorfragment11_operator10/0

at com.mapr.fs.MapRClientImpl.create(MapRClientImpl.java:197)
~[maprfs-5.2.0-mapr.jar:5.2.0-mapr]

at com.mapr.fs.MapRFileSystem.create(MapRFileSystem.java:849)
~[maprfs-5.2.0-mapr.jar:5.2.0-mapr]

at com.mapr.fs.MapRFileSystem.create(MapRFileSystem.java:891)
~[maprfs-5.2.0-mapr.jar:5.2.0-mapr]

Re: Spill Location, permissions and Authentication

[John Omernik <jo...@omernik.com>]

Can you define the setup configuration?  Is this a MapR thing or a Drill
thing?  In my setup, I manually create the local volumes, so if I need to
set permissions a certain way, that's fine, I still question having Drill
reading and writing spill data AS the impersonated user rather than as the
user it's running as...

On Wed, Mar 1, 2017 at 8:03 PM, Boaz Ben-Zvi <bb...@mapr.com> wrote:

> The permissions on the spill locations are s part of the setup
> configuration — these base locations should have permissions matching the
> OS userid used for the DrillBit .
> The DrillBit creates those directories (with that long name 274a41…….)
> with its userid.
> So in the example below, the user zetasvcprod should have the permissions
> on the directory drillprod .
> The newer versions of Drill ( 1.9, 1.10 ) should work the same way.
>
>        Boaz
>
> On Feb 28, 2017, at 10:24 AM, John Omernik <john@omernik.com<mailto:john@
> omernik.com>> wrote:
>
> I am using 1.8 as of this time, if this is fixed in a newer version, please
> let me know.
>
> I am running drill as a master user. (mapr) and have enabled
> authentication. When I authenticate to drill, and try to run a query,
> specifically one with hash joins turned off (thus using spill to disk), I
> am getting error that my user (that I authenticated to sqlline with) does
> not have access to my drill spill location. (They are MapR "local" volumes
> that I have setup myself).
>
> So, I was going to go adjust permissions, but I started thinking more...
> Why isn't the spill location, when authentication is running, or when it's
> not, always reading/writing as the drill bit user?  This doesn't make a lot
> of sense that users would get to read this information, why not remove the
> impersonation on requests to read and write?
>
> Now, I am trying to think this through, so there may be tons of reasons,
> but I would love to understand this process better.
>
> Thanks!
>
> John
>
>
>
> Caused by: org.apache.hadoop.security.AccessControlException: User
> zetasvcprod(user id 2000000) does not have access to
> /var/local/zeta8/local/drillspill/drillprod/274a41a8-
> ae92-6b0f-8dab-3b16841c0e35_majorfragment1_minorfragment11_operator10/0
>
> at com.mapr.fs.MapRClientImpl.create(MapRClientImpl.java:197)
> ~[maprfs-5.2.0-mapr.jar:5.2.0-mapr]
>
> at com.mapr.fs.MapRFileSystem.create(MapRFileSystem.java:849)
> ~[maprfs-5.2.0-mapr.jar:5.2.0-mapr]
>
> at com.mapr.fs.MapRFileSystem.create(MapRFileSystem.java:891)
> ~[maprfs-5.2.0-mapr.jar:5.2.0-mapr]
>
>