You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Gézapeti (Jira)" <ji...@apache.org> on 2019/10/23 09:49:00 UTC
[jira] [Comment Edited] (OOZIE-3551) Configure working defaults for
Spark action in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-3551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957708#comment-16957708 ]
Gézapeti edited comment on OOZIE-3551 at 10/23/19 9:48 AM:
-----------------------------------------------------------
Thanks for reporting this!
We have a bunch of these already set in [SparkArgsExtractor|https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L58-L63]
They were added incrementally as Spark changed. The last iteration of this was in OOZIE-3036.
We currently have support for [HCatCredentials|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java] to talk to the metastore and [Hive2Credentials|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java] for HS2.
Do we need to add support for more kinds delegation tokens for Hive now?
was (Author: gezapeti):
Thanks for reporting this!
We have a bunch of these already set in [SparkArgsExtractor|https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L58-L63}
They were added incrementally as Spark changed. The last iteration of this was in OOZIE-3036.
We currently have support for [HCatCredentials|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java] to talk to the metastore and [Hive2Credentials|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java] for HS2.
Do we need to add support for more kinds delegation tokens for Hive now?
> Configure working defaults for Spark action in Oozie
> ----------------------------------------------------
>
> Key: OOZIE-3551
> URL: https://issues.apache.org/jira/browse/OOZIE-3551
> Project: Oozie
> Issue Type: Bug
> Reporter: Liz Szilagyi
> Assignee: Liz Szilagyi
> Priority: Major
>
> Task:
> Add the following to the spark opts section of the spark action
> --conf spark.yarn.security.tokens.hiveserver2.enabled=false --conf spark.yarn.security.tokens.hivestreaming.enabled=false
> Issue:
> If Spark is launched form Oozie then we need to turn these 2 and some others off: [http://spark.apache.org/docs/latest/running-on-yarn.html#launching-your-application-with-apache-oozie] and let Oozie obtain the tokens.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)