You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Charles Givre (Jira)" <ji...@apache.org> on 2019/11/06 12:44:00 UTC
[jira] [Comment Edited] (DRILL-7149) Kerberos Code Missing from
Drill on YARN
[ https://issues.apache.org/jira/browse/DRILL-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16968335#comment-16968335 ]
Charles Givre edited comment on DRILL-7149 at 11/6/19 12:43 PM:
----------------------------------------------------------------
My engineers found the cause of this issue. There is a TODO here which looks like it was where security tokens should be passed but this was never implemented [1].
Is there any documentation or other information available about this as it seems like this should be an easy fix if you're familiar with YARN and Kerberos? I should add that this issue effectively blocks Drill from being deployed on a cluster with YARN and Kerberos, such as a CDH cluster.
Any help would be greatly appreciated.
[[1]:https://github.com/apache/drill/blob/9d8ac02d05cf6f23ddc80065066722b121577656/drill-yarn/src/main/java/org/apache/drill/yarn/core/AppSpec.java#L136-L140|https://github.com/apache/drill/blob/9d8ac02d05cf6f23ddc80065066722b121577656/drill-yarn/src/main/java/org/apache/drill/yarn/core/AppSpec.java#L136-L140]
was (Author: cgivre):
My engineers found the cause of this issue. There is a TODO here which looks like it was where security tokens should be passed but this was never implemented. [1].
[1]: https://github.com/apache/drill/blob/9d8ac02d05cf6f23ddc80065066722b121577656/drill-yarn/src/main/java/org/apache/drill/yarn/core/AppSpec.java#L136-L140
> Kerberos Code Missing from Drill on YARN
> ----------------------------------------
>
> Key: DRILL-7149
> URL: https://issues.apache.org/jira/browse/DRILL-7149
> Project: Apache Drill
> Issue Type: Bug
> Components: Security
> Affects Versions: 1.14.0
> Reporter: Charles Givre
> Priority: Blocker
>
> My company is trying to deploy Drill using the Drill on Yarn (DoY) and we have run into the issue that DoY does not seem to support passing Kerberos credentials in order to interact with HDFS.
> Upon checking the source code available in GIT (https://github.com/apache/drill/blob/1.14.0/drill-yarn/src/main/java/org/apache/drill/yarn/core/) and referring to Apache YARN documentation (https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YarnApplicationSecurity.html) , we saw no section for passing the security credentials needed by the application to interact with any Hadoop cluster services and applications.
> This we feel needs to be added to the source code so that delegation tokens can be passed inside the container for the process to be able access Drill archive on HDFS and start. It probably should be added to the ContainerLaunchContext within the ApplicationSubmissionContext for DoY as suggested under Apache documentation.
>
> We tried the same DoY utility on a non-kerberised cluster and the process started well. Although we ran into a different issue there of hosts getting blacklisted
> We tested with the Single Principal per cluster option.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)