You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "Mark Owens (JIRA)" <ji...@apache.org> on 2017/10/11 15:10:03 UTC
[jira] [Comment Edited] (ACCUMULO-4561) Crash when using ping on a
non-existing server
[ https://issues.apache.org/jira/browse/ACCUMULO-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16200425#comment-16200425 ]
Mark Owens edited comment on ACCUMULO-4561 at 10/11/17 3:09 PM:
----------------------------------------------------------------
These crashes appear to be occurring when sending a ping request to ports that have Jetty listening. I ran an nmap scan on my local machine looking for open ports and then ran the accumulo shell ping command against the open ports (closed ports return connection refused).
Note that all these tests were run on the 2.0.0-SNAPSHOT.
My results are listed below:
{{TServer port on local instance:
9997/tcp open palace-6?
>>> localhost:9997:OK
Following ports all returned same response:
2181/tcp open eforward?
4560/tcp open unknown
5355/tcp open llmnr?
8030/tcp open hadoop-ipc Hadoop IPC
8031/tcp open hadoop-ipc Hadoop IPC
8032/tcp open hadoop-ipc Hadoop IPC
8033/tcp open hadoop-ipc Hadoop IPC
8040/tcp open hadoop-ipc Hadoop IPC
9000/tcp open hadoop-ipc Hadoop IPC
34737/tcp open unknown
39473/tcp open hadoop-ipc Hadoop IPC
50010/tcp open unknown
50020/tcp open hadoop-ipc Hadoop IPC
>>> localhost:8031 ERROR org.apache.thrift.transport.TTransportException
9998/tcp open distinct32?
9999/tcp open abyss?
10001/tcp open scp-config?
>>> localhost:9998 ERROR org.apache.thrift.TApplicationException: Invalid method name: 'getTabletServerStatus'
13562/tcp open unknown
>>> localhost:13562 ERROR org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: 120000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/127.0.0.1:36716 remote=localhost/127.0.0.1:13562]
Jetty ports:
8042/tcp open http Jetty 6.1.26
8088/tcp open http Jetty 6.1.26
9995/tcp open http Jetty 9.3.21.v20170918
44263/tcp open http Jetty 6.1.26
50070/tcp open http Jetty 6.1.26
50090/tcp open http Jetty 6.1.26
>>> #
>>> # java.lang.OutOfMemoryError: Java heap space
>>> # -XX:OnOutOfMemoryError="kill -9 %p"
>>> # Executing /bin/sh -c "kill -9 7693"...
>>> Killed
This port returned a different response after a timeout:
50075/tcp open http Jetty 6.1.26
>>> localhost:50075 ERROR org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: 120000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/127.0.0.1:37190 remote=localhost/127.0.0.1:50075]}}
I have no feel for how often the 'ping -ts <port>' command is run and how often it would be provided an invalid port? I would assume a user would only supply a port if they suspected it to be a tserver. Given that case this situation would not happen very often, I suspect.
I also noticed that if I stop the tablet servers after I'm in the shell and then run the ping command, the shell never returns the prompt to the user. Ctrl-C'ing at that points exits the shell as well. I would think that should be fixed since the purpose of the ping is to retrieve the status of a tablet server. Has that behavior been documented and/or verified previously?
was (Author: jmark99):
These crashes appear to be occurring when sending a ping request to ports that have Jetty listening. I ran an nmap scan on my local machine looking for open ports and then ran the accumulo shell ping command against the open ports (closed ports return connection refused).
Note that all these tests were run on the 2.0.0-SNAPSHOT.
My results are listed below:
{{
TServer port on local instance:
9997/tcp open palace-6?
>>> localhost:9997:OK
Following ports all returned same response:
2181/tcp open eforward?
4560/tcp open unknown
5355/tcp open llmnr?
8030/tcp open hadoop-ipc Hadoop IPC
8031/tcp open hadoop-ipc Hadoop IPC
8032/tcp open hadoop-ipc Hadoop IPC
8033/tcp open hadoop-ipc Hadoop IPC
8040/tcp open hadoop-ipc Hadoop IPC
9000/tcp open hadoop-ipc Hadoop IPC
34737/tcp open unknown
39473/tcp open hadoop-ipc Hadoop IPC
50010/tcp open unknown
50020/tcp open hadoop-ipc Hadoop IPC
>>> localhost:8031 ERROR org.apache.thrift.transport.TTransportException
9998/tcp open distinct32?
9999/tcp open abyss?
10001/tcp open scp-config?
>>> localhost:9998 ERROR org.apache.thrift.TApplicationException: Invalid method name: 'getTabletServerStatus'
13562/tcp open unknown
>>> localhost:13562 ERROR org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: 120000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/127.0.0.1:36716 remote=localhost/127.0.0.1:13562]
Jetty ports:
8042/tcp open http Jetty 6.1.26
8088/tcp open http Jetty 6.1.26
9995/tcp open http Jetty 9.3.21.v20170918
44263/tcp open http Jetty 6.1.26
50070/tcp open http Jetty 6.1.26
50090/tcp open http Jetty 6.1.26
>>> #
>>> # java.lang.OutOfMemoryError: Java heap space
>>> # -XX:OnOutOfMemoryError="kill -9 %p"
>>> # Executing /bin/sh -c "kill -9 7693"...
>>> Killed
This port returned a different response after a timeout:
50075/tcp open http Jetty 6.1.26
>>> localhost:50075 ERROR org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: 120000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/127.0.0.1:37190 remote=localhost/127.0.0.1:50075]
}}
I have no feel for how often the 'ping -ts <port>' command is run and how often it would be provided an invalid port? I would assume a user would only supply a port if they suspected it to be a tserver. Given that case this situation would not happen very often, I suspect.
I also noticed that if I stop the tablet servers after I'm in the shell and then run the ping command, the shell never returns the prompt to the user. Ctrl-C'ing at that points exits the shell as well. I would think that should be fixed since the purpose of the ping is to retrieve the status of a tablet server. Has that behavior been documented and/or verified previously?
> Crash when using ping on a non-existing server
> ----------------------------------------------
>
> Key: ACCUMULO-4561
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4561
> Project: Accumulo
> Issue Type: Bug
> Components: shell
> Affects Versions: 2.0.0
> Reporter: Luis Tavarez
>
> While working on ACCUMULO-4558, I tried running
> {code}ping -ts localhost:9995{code} (localhost:9995 does not have a a tserver on my setup.)
> And it caused the shell to exit (crashed) and show the following message.
> {code}#
> # java.lang.OutOfMemoryError: Java heap space
> # -XX:OnOutOfMemoryError="kill -9 %p"
> # Executing /bin/sh -c "kill -9 25561"...
> /home/lmtavar/git/uno/bin/uno: line 44: 25561 Killed "$ACCUMULO_HOME"/bin/accumulo shell -u "$ACCUMULO_USER" -p "$ACCUMULO_PASSWORD" "${@:2}"
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)