[jira] [Comment Edited] (DRILL-3584) Drill Kerberos HDFS Support / Documentation

["Sudheesh Katkam (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862111#comment-15862111 ] 

Sudheesh Katkam edited comment on DRILL-3584 at 2/13/17 9:22 PM:
-----------------------------------------------------------------

I may be wrong, but AFAIK, following Nathan's blogpost is not sufficient for Drill to authenticate to a Kerberized HDFS (specially since ticket's expire).

The following instructions SHOULD allow for end to end (user to HDFS) authentication, but I have not tested them since I do not have the infrastructure to setup KDC or HDFS (and not possible to write a unit test for this, MiniDFS and Drill use the same auth library which has static variables that MUST be different).

(1) Enable Drill login to KDC.

Once the changes in DRILL-4280 are merged (targeted for 1.10 release), the drillbit can be started with a Kerberos principal and keytab. Add appropriate config to drill-override.conf, something like:
{code}
drill.exec {
  security.auth.principal: “drill/_host@REALM”
  security.auth.keytab: “/etc/drill/conf/drill.keytab”
}
{code}
This assumes a service principal is created for Drill ([details here|http://www.microhowto.info/howto/create_a_service_principal_using_mit_kerberos.html]). Ensure "drill" as a [user identity|http://hadoop.apache.org/docs/r1.2.1/hdfs_permissions_guide.html#User+Identity] exists in HDFS (Drill uses HDFS for [Dynamic UDFs|http://drill.apache.org/docs/dynamic-udfs/], etc.). At startup, Drill will login to KDC, and when accessing HDFS, Drill uses the configured Kerberos credentials.

(2) Enable authentication from user to Drill.

This is strongly recommended, but optional.

(3) Enable Drill impersonation.

This is strongly recommended, but optional. Otherwise, access to HDFS happen as "drill".

(4) Enable secure impersonation in HDFS.

Setup "drill" as a proxy user for HDFS with the required privileges, [details here|http://hadoop.apache.org/docs/r1.2.1/Secure_Impersonation.html]. This will allow Drill to act on behalf of the end user ("bob") when accessing HDFS. For example, if "drill" is authorized to impersonate "bob", then while accessing the HDFS, access rights are checked for "bob" and authentication credentials of "drill" are verified.


was (Author: sudheeshkatkam):
I may be wrong, but AFAIK, following Nathan's blogpost is not sufficient for Drill to authenticate to a Kerberized HDFS (specially since ticket's expire).

The following instructions SHOULD allow for end to end (user to HDFS) authentication, but I have not tested them since I do not have the infrastructure to setup KDC or HDFS (and not possible to write a unit test for this, MiniDFS and Drill use the same auth library which has static variables that MUST be different).

(1) Enable Drill login to KDC.

Once the changes in DRILL-4280 are merged (targeted for 1.10 release), the drillbit can be started with a Kerberos principal and keytab. Add appropriate config drill-override.conf, something like:
{code}
drill.exec {
  security.auth.principal: “drill/_host@REALM”
  security.auth.keytab: “/etc/drill/conf/drill.keytab”
}
{code}
This assumes a service principal is created for Drill ([details here|http://www.microhowto.info/howto/create_a_service_principal_using_mit_kerberos.html]). Ensure "drill" as a [user identity|http://hadoop.apache.org/docs/r1.2.1/hdfs_permissions_guide.html#User+Identity] exists in HDFS (Drill uses HDFS for [Dynamic UDFs|http://drill.apache.org/docs/dynamic-udfs/], etc.). At startup, Drill will login to KDC, and when accessing HDFS, Drill uses the configured Kerberos credentials.

(2) Enable authentication from user to Drill.

This is strongly recommended, but optional.

(3) Enable Drill impersonation.

This is strongly recommended, but optional. Otherwise, access to HDFS happen as "drill".

(4) Enable secure impersonation in HDFS.

Setup "drill" as a proxy user for HDFS with the required privileges, [details here|http://hadoop.apache.org/docs/r1.2.1/Secure_Impersonation.html]. This will allow Drill to act on behalf of the end user ("bob") when accessing HDFS. For example, if "drill" is authorized to impersonate "bob", then while accessing the HDFS, access rights are checked for "bob" and authentication credentials of "drill" are verified.

> Drill Kerberos HDFS Support / Documentation
> -------------------------------------------
>
>                 Key: DRILL-3584
>                 URL: https://issues.apache.org/jira/browse/DRILL-3584
>             Project: Apache Drill
>          Issue Type: New Feature
>    Affects Versions: 1.1.0
>            Reporter: Hari Sekhon
>            Priority: Critical
>              Labels: security
>
> I'm trying to find Drill docs for Kerberos support for secure HDFS clusters and it doesn't appear to be well tested / supported / documented yet.
> This product is Dead-on-Arrival if it doesn't integrate well with secure Hadoop clusters, specifically HDFS + Kerberos (plus obviously secure kerberized Hive/HCatalog etc.)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)