You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by "Prabhjyot Singh (JIRA)" <ji...@apache.org> on 2017/02/18 03:25:44 UTC
[jira] [Created] (ZEPPELIN-2133) All interpreters sometimes throw
random Connection refuse errors
Prabhjyot Singh created ZEPPELIN-2133:
-----------------------------------------
Summary: All interpreters sometimes throw random Connection refuse errors
Key: ZEPPELIN-2133
URL: https://issues.apache.org/jira/browse/ZEPPELIN-2133
Project: Zeppelin
Issue Type: Bug
Components: Interpreters
Reporter: Prabhjyot Singh
Assignee: Prabhjyot Singh
The interpreters sometimes randomly throw connection refused error. When I try to run a paragraph, it initially keeps on printing this:
{code:java}
DEBUG [2017-02-15 21:26:08,857] ({pool-2-thread-3} RemoteInterpreterUtils.java[checkIfRemoteEndpointAccessible]:53) - Remote endpoint 'localhost:39477' is not accessible (might be initializing): Connection refused (Connection refused)
DEBUG [2017-02-15 21:26:09,364] ({pool-2-thread-3} RemoteInterpreterUtils.java[checkIfRemoteEndpointAccessible]:53) - Remote endpoint 'localhost:39477' is not accessible (might be initializing): Connection refused (Connection refused)
DEBUG [2017-02-15 21:26:09,871] ({pool-2-thread-3} RemoteInterpreterUtils.java[checkIfRemoteEndpointAccessible]:53) - Remote endpoint 'localhost:39477' is not accessible (might be initializing): Connection refused (Connection refused)
DEBUG [2017-02-15 21:26:10,389] ({pool-2-thread-3} RemoteInterpreterUtils.java[checkIfRemoteEndpointAccessible]:53) - Remote endpoint 'localhost:39477' is not accessible (might be initializing): Connection refused (Connection refused)
{code}
And when I go to command line I see these zombie processes;
{code:java}
root@os-r7-zeppelin-we-sec-2 zeppelin# ps aux | grep interpreter
zeppelin 601 0.0 0.0 113124 1512 ? S 06:14 0:00 /bin/bash /usr/hdp/current/zeppelin-server/bin/interpreter.sh -d /usr/hdp/current/zeppelin-server/interpreter/livy -p 38959 -l /usr/hdp/current/zeppelin-server/local-repo/2C8CCPY3G
zeppelin 612 0.0 0.0 113120 844 ? S 06:14 0:00 /bin/bash /usr/hdp/current/zeppelin-server/bin/interpreter.sh -d /usr/hdp/current/zeppelin-server/interpreter/livy -p 38959 -l /usr/hdp/current/zeppelin-server/local-repo/2C8CCPY3G
zeppelin 613 0.9 0.2 3427080 42588 ? Sl 06:14 0:00 /usr/lib/jvm/java-openjdk/bin/java -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///usr/hdp/current/zeppelin-server/conf/log4j.properties -Dzeppelin.log.file=/grid/0/log/zeppelin/zeppelin-interpreter-livy-zeppelin-os-r7-zeppelin-we-sec-2.log -Xms1024m -Xmx1024m -XX:MaxPermSize=512m -cp ::/usr/hdp/current/zeppelin-server/interpreter/livy/:/usr/hdp/current/zeppelin-server/lib/interpreter/: org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer 38959
zeppelin 780 0.0 0.0 113124 1508 ? S 06:14 0:00 /bin/bash /usr/hdp/current/zeppelin-server/bin/interpreter.sh -d /usr/hdp/current/zeppelin-server/interpreter/livy -p 39741 -l /usr/hdp/current/zeppelin-server/local-repo/2C8J12HF5
zeppelin 794 0.0 0.0 113120 844 ? S 06:14 0:00 /bin/bash /usr/hdp/current/zeppelin-server/bin/interpreter.sh -d /usr/hdp/current/zeppelin-server/interpreter/livy -p 39741 -l /usr/hdp/current/zeppelin-server/local-repo/2C8J12HF5
zeppelin 795 0.9 0.2 3427080 42864 ? Sl 06:14 0:00 /usr/lib/jvm/java-openjdk/bin/java -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///usr/hdp/current/zeppelin-server/conf/log4j.properties -Dzeppelin.log.file=/grid/0/log/zeppelin/zeppelin-interpreter-livy-zeppelin-os-r7-zeppelin-we-sec-2.log -Xms1024m -Xmx1024m -XX:MaxPermSize=512m -cp ::/usr/hdp/current/zeppelin-server/interpreter/livy/:/usr/hdp/current/zeppelin-server/lib/interpreter/: org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer 39741
zeppelin 1255 0.0 0.0 113124 244 ? S Feb14 0:00 /bin/bash /usr/hdp/current/zeppelin-server/bin/interpreter.sh -d /usr/hdp/current/zeppelin-server/interpreter/livy -p 54900 -l /usr/hdp/current/zeppelin-server/local-repo/2C8J12HF5
zeppelin 1266 0.0 0.0 113120 244 ? S Feb14 0:00 /bin/bash /usr/hdp/current/zeppelin-server/bin/interpreter.sh -d /usr/hdp/current/zeppelin-server/interpreter/livy -p 54900 -l /usr/hdp/current/zeppelin-server/local-repo/2C8J12HF5
zeppelin 1267 0.1 0.7 3596948 124976 ? Sl Feb14 2:24 /usr/lib/jvm/java-openjdk/bin/java -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///usr/hdp/current/zeppelin-server/conf/log4j.properties -Dzeppelin.log.file=/grid/0/log/zeppelin/zeppelin-interpreter-livy-zeppelin-os-r7-zeppelin-we-sec-2.log -Xms1024m -Xmx1024m -XX:MaxPermSize=512m -cp ::/usr/hdp/current/zeppelin-server/interpreter/livy/:/usr/hdp/current/zeppelin-server/lib/interpreter/: org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer 54900
root 1739 0.0 0.0 112648 976 pts/0 S+ 06:15 0:00 grep --color=auto interpreter
zeppelin 2599 0.0 0.0 113124 240 ? S Feb14 0:00 /bin/bash /usr/hdp/current/zeppelin-server/bin/interpreter.sh -d /usr/hdp/current/zeppelin-server/interpreter/livy -p 59599 -l /usr/hdp/current/zeppelin-server/local-repo/2C8J12HF5
zeppelin 2610 0.0 0.0 113120 240 ? S Feb14 0:00 /bin/bash /usr/hdp/current/zeppelin-server/bin/interpreter.sh -d /usr/hdp/current/zeppelin-server/interpreter/livy -p 59599 -l /usr/hdp/current/zeppelin-server/local-repo/2C8J12HF5
zeppelin 2611 0.1 2.0 3603128 330476 ? Sl Feb14 2:15 /usr/lib/jvm/java-openjdk/bin/java -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///usr/hdp/current/zeppelin-server/conf/log4j.properties -Dzeppelin.log.file=/grid/0/log/zeppelin/zeppelin-interpreter-livy-zeppelin-os-r7-zeppelin-we-sec-2.log -Xms1024m -Xmx1024m -XX:MaxPermSize=512m -cp ::/usr/hdp/current/zeppelin-server/interpreter/livy/:/usr/hdp/current/zeppelin-server/lib/interpreter/: org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer 59599
zeppelin 2792 0.0 0.0 113124 244 ? S Feb15 0:00 /bin/bash /usr/hdp/current/zeppelin-server/bin/interpreter.sh -d /usr/hdp/current/zeppelin-server/interpreter/livy -p 34730 -l /usr/hdp/current/zeppelin-server/local-repo/2C9XK2KSH
zeppelin 2811 0.0 0.0 113120 244 ? S Feb15 0:00 /bin/bash /usr/hdp/current/zeppelin-server/bin/interpreter.sh -d /usr/hdp/current/zeppelin-server/interpreter/livy -p 34730 -l /usr/hdp/current/zeppelin-server/local-repo/2C9XK2KSH
zeppelin 2813 0.1 2.0 3598864 336480 ? Sl Feb15 2:12 /usr/lib/jvm/java-openjdk/bin/java -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///usr/hdp/current/zeppelin-server/conf/log4j.properties -Dzeppelin.log.file=/grid/0/log/zeppelin/zeppelin-interpreter-livy-zeppelin-os-r7-zeppelin-we-sec-2.log -Xms1024m -Xmx1024m -XX:MaxPermSize=512m -cp ::/usr/hdp/current/zeppelin-server/interpreter/livy/:/usr/hdp/current/zeppelin-server/lib/interpreter/: org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer 34730
zeppelin 5441 0.0 0.0 113124 240 ? S Feb15 0:00 /bin/bash /usr/hdp/current/zeppelin-server/bin/interpreter.sh -d /usr/hdp/current/zeppelin-server/interpreter/livy -p 50880 -l /usr/hdp/current/zeppelin-server/local-repo/2C9XK2KSH
zeppelin 5468 0.0 0.0 113120 240 ? S Feb15 0:00 /bin/bash /usr/hdp/current/zeppelin-server/bin/interpreter.sh -d /usr/hdp/current/zeppelin-server/interpreter/livy -p 50880 -l /usr/hdp/current/zeppelin-server/local-repo/2C9XK2KSH
zeppelin 5469 0.1 1.9 3600024 322828 ? Sl Feb15 2:10 /usr/lib/jvm/java-openjdk/bin/java -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///usr/hdp/current/zeppelin-server/conf/log4j.properties -Dzeppelin.log.file=/grid/0/log/zeppelin/zeppelin-interpreter-livy-zeppelin-os-r7-zeppelin-we-sec-2.log -Xms1024m -Xmx1024m -XX:MaxPermSize=512m -cp ::/usr/hdp/current/zeppelin-server/interpreter/livy/:/usr/hdp/current/zeppelin-server/lib/interpreter/: org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer 50880
zeppelin 9968 0.0 0.0 113124 240 ? S Feb15 0:00 /bin/bash /usr/hdp/current/zeppelin-server/bin/interpreter.sh -d /usr/hdp/current/zeppelin-server/interpreter/livy -p 33813 -l /usr/hdp/current/zeppelin-server/local-repo/2C9XK2KSH
zeppelin 9980 0.0 0.0 113120 240 ? S Feb15 0:00 /bin/bash /usr/hdp/current/zeppelin-server/bin/interpreter.sh -d /usr/hdp/current/zeppelin-server/interpreter/livy -p 33813 -l /usr/hdp/current/zeppelin-server/local-repo/2C9XK2KSH
zeppelin 9981 0.1 2.0 3598864 338768 ? Sl Feb15 2:13 /usr/lib/jvm/java-openjdk/bin/java -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///usr/hdp/current/zeppelin-server/conf/log4j.properties -Dzeppelin.log.file=/grid/0/log/zeppelin/zeppelin-interpreter-livy-zeppelin-os-r7-zeppelin-we-sec-2.log -Xms1024m -Xmx1024m -XX:MaxPermSize=512m -cp ::/usr/hdp/current/zeppelin-server/interpreter/livy/:/usr/hdp/current/zeppelin-server/lib/interpreter/: org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer 33813
zeppelin 10263 0.0 0.0 113124 244 ? S Feb14 0:00 /bin/bash /usr/hdp/current/zeppelin-server/bin/interpreter.sh -d /usr/hdp/current/zeppelin-server/interpreter/livy -p 43014 -l /usr/hdp/current/zeppelin-server/local-repo/2C9XK2KSH
zeppelin 10277 0.0 0.0 113120 244 ? S Feb14 0:00 /bin/bash /usr/hdp/current/zeppelin-server/bin/interpreter.sh -d /usr/hdp/current/zeppelin-server/interpreter/livy -p 43014 -l /usr/hdp/current/zeppelin-server/local-repo/2C9XK2KSH
zeppelin 10279 0.1 2.1 3601948 353012 ? Sl Feb14 2:16 /usr/lib/jvm/java-openjdk/bin/java -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///usr/hdp/current/zeppelin-server/conf/log4j.properties -Dzeppelin.log.file=/grid/0/log/zeppelin/zeppelin-interpreter-livy-zeppelin-os-r7-zeppelin-we-sec-2.log -Xms1024m -Xmx1024m -XX:MaxPermSize=512m -cp ::/usr/hdp/current/zeppelin-server/interpreter/livy/:/usr/hdp/current/zeppelin-server/lib/interpreter/: org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer 43014
zeppelin 15102 0.0 0.0 113124 240 ? S Feb15 0:00 /bin/bash /usr/hdp/current/zeppelin-server/bin/interpreter.sh -d /usr/hdp/current/zeppelin-server/interpreter/livy -p 60490 -l /usr/hdp/current/zeppelin-server/local-repo/2C8J12HF5
zeppelin 15113 0.0 0.0 113120 240 ? S Feb15 0:00 /bin/bash /usr/hdp/current/zeppelin-server/bin/interpreter.sh -d /usr/hdp/current/zeppelin-server/interpreter/livy -p 60490 -l /usr/hdp/current/zeppelin-server/local-repo/2C8J12HF5
zeppelin 15114 0.1 1.7 3600844 285192 ? Sl Feb15 2:11 /usr/lib/jvm/java-openjdk/bin/java -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///usr/hdp/current/zeppelin-server/conf/log4j.properties -Dzeppelin.log.file=/grid/0/log/zeppelin/zeppelin-interpreter-livy-zeppelin-os-r7-zeppelin-we-sec-2.log -Xms1024m -Xmx1024m -XX:MaxPermSize=512m -cp ::/usr/hdp/current/zeppelin-server/interpreter/livy/:/usr/hdp/current/zeppelin-server/lib/interpreter/: org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer 60490
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)