You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/07/14 03:57:00 UTC
[jira] [Work logged] (HDFS-16446) Consider ioutils of disk when choosing volume
[ https://issues.apache.org/jira/browse/HDFS-16446?focusedWorklogId=790695&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-790695 ]
ASF GitHub Bot logged work on HDFS-16446:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 14/Jul/22 03:56
Start Date: 14/Jul/22 03:56
Worklog Time Spent: 10m
Work Description: whbing commented on PR #3960:
URL: https://github.com/apache/hadoop/pull/3960#issuecomment-1183957526
Hi, @tomscut , we also face slow nodes caused by io, but in most cases it is slow to read data because the frequency of reading in the cluster far exceeds that of writing. We have counted many cases, when reading slowly, the **iowait** is basically high, but only one or a few disks are full of **ioutil** (12 disks per dn). Therefore, I have the following points to discuss.
1. Is there any further consideration for slow reading?
2. Have you considered reporting iowait to nn, like XmitsInProgress, ActiveTransferThreadCount, etc, so that it is taken into account when choosing a DN?
Issue Time Tracking
-------------------
Worklog Id: (was: 790695)
Time Spent: 3h (was: 2h 50m)
> Consider ioutils of disk when choosing volume
> ---------------------------------------------
>
> Key: HDFS-16446
> URL: https://issues.apache.org/jira/browse/HDFS-16446
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Tao Li
> Assignee: Tao Li
> Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-02-05-09-50-12-241.png
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Consider ioutils of disk when choosing volume to avoid busy disks.
> Document: [https://docs.google.com/document/d/1Ko1J7shz8hVLnNACT6PKVQ_leIHf_YaIFA2s3yJMZHQ/edit?usp=sharing]
> Principle is as follows:
> !https://user-images.githubusercontent.com/55134131/159827737-f4ca4d66-c2f2-4bef-901b-6d2bc7bdda9a.png|width=440,height=192!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org