[jira] [Created] (AMBARI-22594) Livy server start fails during EU with 'Address already in use' error

["Dmitry Lysnichenko (JIRA)" <ji...@apache.org>]

Dmitry Lysnichenko created AMBARI-22594:
-------------------------------------------

             Summary: Livy server start fails during EU with 'Address already in use' error
                 Key: AMBARI-22594
                 URL: https://issues.apache.org/jira/browse/AMBARI-22594
             Project: Ambari
          Issue Type: Bug
            Reporter: Dmitry Lysnichenko
            Assignee: Dmitry Lysnichenko
            Priority: Blocker



Observed this issue quite consistently in Ambari-2.6.1 Upgrade ST runs

*STR*
# Deployed cluster with Ambari version: 2.5.1.0-159 and HDP version: 2.6.1.0-129
# Upgrade Ambari to Target Version: 2.6.1.0-43 | Hash: acbce28fdd119c72625c6beff63fc169de58ba22
# Regenerate keytabs post Ambari upgrade and this step will restart all services. Here Livy server is operational and gets restarted fine (at timestamp: 09:29)
# Now register HDP-2.6.4.0-36 version and perform EU. During EU 'Restart Livy server' task happens and reports success (at timestamp: 10:26)
# However when checking the livy logs - Livy restart reported below exception as the previous process was not killed/stopped
{code}
17/11/21 10:26:22 WARN AbstractLifeCycle: FAILED org.eclipse.jetty.server.Server@3bc735b3: java.net.BindException: Address already in use
java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321)
at org.apache.livy.server.LivyServer.main(LivyServer.scala)
Exception in thread "main" java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
{code}

- Post Upgrade, I tried to stop/start Spark as well and Livy still gave same exception; although web Ui reports operation as success (at timestamp: 11:37)
- Finally the web UI shows Livy as down, even though the process is running from the initial step (at timestamp: 09:29)





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)