You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@livy.apache.org by "Nicola Marangoni (Jira)" <ji...@apache.org> on 2019/10/24 13:18:00 UTC
[jira] [Created] (LIVY-700) Connection refused from YARN when
impersonation enabled
Nicola Marangoni created LIVY-700:
-------------------------------------
Summary: Connection refused from YARN when impersonation enabled
Key: LIVY-700
URL: https://issues.apache.org/jira/browse/LIVY-700
Project: Livy
Issue Type: Bug
Components: Interpreter
Affects Versions: 0.6.0
Environment: Cloudera 5.7 Hadoop 2.6 Spark 2.4.x Centos7 Livy 0.6 Zeppelin 8.2
Reporter: Nicola Marangoni
I have Livy and Zeppelin running under the user "zeppelin" on an edge node.
The Cloudera-cluster is not kerberized. Livy is set in yarn client mode.
When the impersonation is not enabled the Livy interpreter works, and a YARN job ist started on the cluster using the user "zeppelin".
When *livy.impersonation.enabled = true* Livy cannot connect to the YARN RM:
{code:java}
19/10/24 14:40:58 INFO utils.LineBufferedStream: 19/10/24 14:40:58 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm271
19/10/24 14:40:58 INFO utils.LineBufferedStream: 19/10/24 14:40:58 INFO retry.RetryInvocationHandler: Exception while invoking getClusterMetrics of class ApplicationClientProtocolPBClientImpl over rm271 after 1 fail over attempts. Trying to fail over immediately.
19/10/24 14:40:58 INFO utils.LineBufferedStream: 19/10/24 14:40:58 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm270
19/10/24 14:40:58 INFO utils.LineBufferedStream: 19/10/24 14:40:58 INFO retry.RetryInvocationHandler: Exception while invoking getClusterMetrics of class ApplicationClientProtocolPBClientImpl over rm270 after 2 fail over attempts. Trying to fail over after sleeping for 374ms.
19/10/24 14:40:58 INFO utils.LineBufferedStream: java.net.ConnectException: Call From <localhost>/123.123.123.123 to yarn.rm.host.name:8032 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused{code}
This error happen after the spark driver is successfully started.
Spark simply doesn't get any resources from YARN.
Zeppelin is configured to use FreeIPA over LDAP for login. The user I use to login to the zeppelin UI has even more rights on the cluster and if I start a spark-shell on the same cluster with that user I get a YARN job without any issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)