You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Kiran Kumar Maturi (Jira)" <ji...@apache.org> on 2021/05/24 11:34:00 UTC
[jira] [Updated] (HDFS-16036) Datanode client is stuck, waiting for
connections map to be empty
[ https://issues.apache.org/jira/browse/HDFS-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kiran Kumar Maturi updated HDFS-16036:
--------------------------------------
Description:
In our cluster we have observed when close command is issued to Datanode, It is stuck indefinitely waiting for connections map to be empty for some reason the map is not empty even after the connections have closed. [code link|https://github.com/apache/hadoop/blob/branch-2.10.1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java#L1360].
Instead of infinite wait a timed wait (based on retries or timeout) would enable to shutdown the datanode in timely manner.
{code:java}
for (Connection conn : connections.values()) {
conn.interrupt();
conn.interruptConnectingThread();
}
// wait until all connections are closed
while (!connections.isEmpty()) {
try {
Thread.sleep(STOP_SLEEP_TIME_MS);
} catch (InterruptedException e) {
}
}
{code}
was:
In our cluster we have observed when close command is issued to Datanode, It is stuck indefinitely waiting for connections map to be empty. [code link|https://github.com/apache/hadoop/blob/branch-2.10.1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java#L1360].
Instead of infinite wait a timed wait (based on retries or timeout) would enable to shutdown the datanode in timely manner.
{code:java}
for (Connection conn : connections.values()) {
conn.interrupt();
conn.interruptConnectingThread();
}
// wait until all connections are closed
while (!connections.isEmpty()) {
try {
Thread.sleep(STOP_SLEEP_TIME_MS);
} catch (InterruptedException e) {
}
}
{code}
We have observed even after all the connections are closed the map still some connections. Will create a separate JIRA for the issue
> Datanode client is stuck, waiting for connections map to be empty
> -----------------------------------------------------------------
>
> Key: HDFS-16036
> URL: https://issues.apache.org/jira/browse/HDFS-16036
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.10.1
> Reporter: Kiran Kumar Maturi
> Priority: Major
>
> In our cluster we have observed when close command is issued to Datanode, It is stuck indefinitely waiting for connections map to be empty for some reason the map is not empty even after the connections have closed. [code link|https://github.com/apache/hadoop/blob/branch-2.10.1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java#L1360].
> Instead of infinite wait a timed wait (based on retries or timeout) would enable to shutdown the datanode in timely manner.
> {code:java}
> for (Connection conn : connections.values()) {
> conn.interrupt();
> conn.interruptConnectingThread();
> }
>
> // wait until all connections are closed
> while (!connections.isEmpty()) {
> try {
> Thread.sleep(STOP_SLEEP_TIME_MS);
> } catch (InterruptedException e) {
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org