You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "ZanderXu (Jira)" <ji...@apache.org> on 2024/04/11 02:36:00 UTC
[jira] [Commented] (HDFS-17434) Selector.select in SocketIOWithTimeout.java has significant overhead
[ https://issues.apache.org/jira/browse/HDFS-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835950#comment-17835950 ]
ZanderXu commented on HDFS-17434:
---------------------------------
[~qinyuren] Thanks for involving me. Can you share the screenshot about the sendbuffer of the connection on DN side? We need to confirm that if the connection is writable.
> Selector.select in SocketIOWithTimeout.java has significant overhead
> --------------------------------------------------------------------
>
> Key: HDFS-17434
> URL: https://issues.apache.org/jira/browse/HDFS-17434
> Project: Hadoop HDFS
> Issue Type: Test
> Reporter: qinyuren
> Priority: Major
> Attachments: image-2024-03-20-19-10-13-016.png, image-2024-03-20-19-22-29-829.png, image-2024-03-20-19-24-02-233.png, image-2024-03-20-19-55-18-378.png
>
>
> In our cluster, the SendDataPacketBlockedOnNetworkNanosAvgTime metric ranges from 5ms to 10ms, exceeding the usual disk reading overhead. Our machine network card bandwidth is 20000Mb/s.
> !image-2024-03-20-19-10-13-016.png|width=662,height=135!
> !image-2024-03-20-19-55-18-378.png!
> By adding log printing, it turns out that the Selector.select function has significant overhead.
> !image-2024-03-20-19-22-29-829.png|width=474,height=262!
> !image-2024-03-20-19-24-02-233.png|width=445,height=181!
> I would like to know if this falls within the normal range or how we can improve it.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org