[jira] [Created] (ZEPPELIN-3016) Zeppelin Spark Interpreter User Impersonation is not working

["Venkata Sudheer Kumar M (JIRA)" <ji...@apache.org>]

Venkata Sudheer Kumar M created ZEPPELIN-3016:
-------------------------------------------------

             Summary: Zeppelin Spark Interpreter User Impersonation is not working
                 Key: ZEPPELIN-3016
                 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3016
             Project: Zeppelin
          Issue Type: Bug
          Components: Interpreters
    Affects Versions: 0.7.3, 0.7.2, 0.7.1
            Reporter: Venkata Sudheer Kumar M


Hadoop 2.7.3
Spark 2.1.1
Zeppelin (0.7.2 and 0.7.3)

Working:

Install and configure zeppelin 0.7.2/0.7.3 with Spark 2.1.1 with HDP 2.6.2 (Hadoop 2.7.3) and enable node labels from YARN ( * spark-am-worker-nodes* ) along with Preemption and Map spark to launch Application master only on these node-labeled yarn nodes using *spark.yarn.am.nodeLabelExpression* and enable zeppelin -spark interpreter to use *Per User*  and *Isolated per user * with this i'm able to launch the job (run *_sc.version _*from *_%pyspark_*) with the default user *zeppelin*

Not Working:

Follow the same process as above and also enable the *User Impersonation* setting for Spark Interpreter then (sc.version) it is failing with:
{color:#d04437}
java.lang.NullPointerException
	at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38)
	at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33)
at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext_2(SparkInterpreter.java:385)
	at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:374)
	at org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:146)
	at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:819)
	at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
	at org.apache.zeppelin.spark.PySparkInterpreter.getSparkInterpreter(PySparkInterpreter.java:566)
	at org.apache.zeppelin.spark.PySparkInterpreter.createGatewayServerAndStartScript(PySparkInterpreter.java:209)
	at org.apache.zeppelin.spark.PySparkInterpreter.open(PySparkInterpreter.java:163)
	at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:493)
	at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
	at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745){color}

Not sure why impersonation is even though Zeppelin Shell interpreter is able to pick up my user and also i have even tried wit the below from zeppelin-env.sh but it is still the same:
export ZEPPELIN_IMPERSONATE_CMD='sudo -H -u ${ZEPPELIN_IMPERSONATE_USER} bash -c '
export ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER=true

is there any known workaround for this or any related information will help.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)