You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Saisai Shao (JIRA)" <ji...@apache.org> on 2016/07/01 07:25:11 UTC
[jira] [Created] (SPARK-16342) Add a new Configurable Token Manager
for Spark Running on YARN
Saisai Shao created SPARK-16342:
-----------------------------------
Summary: Add a new Configurable Token Manager for Spark Running on YARN
Key: SPARK-16342
URL: https://issues.apache.org/jira/browse/SPARK-16342
Project: Spark
Issue Type: New Feature
Components: YARN
Reporter: Saisai Shao
Current Spark on YARN token management has some problems:
1. Supported service is hard-coded, only HDFS, Hive and HBase are supported for token fetching. For other third-party services which need to be communicated with Spark in Kerberos way, currently the only way is to modify Spark code.
2. Current token renewal and update mechanism is also hard-coded, which means other third-party services cannot be benefited from this system and will be failed when token is expired.
3. Also In the code level, current token obtain and update codes are placed in several different places without elegant structured, which makes it hard to maintain and extend.
So here propose a new Configurable Token Manager class to solve the issues mentioned above. Design doc is attached with link (https://docs.google.com/document/d/1piUvrQywWXiSwyZM9alN6ilrdlX9ohlNOuP4_Q3A6dc/edit?usp=sharing), any suggestion and comment is greatly appreciated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org