You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hadoop.apache.org by Daegyu Han <hd...@gmail.com> on 2019/08/19 08:46:44 UTC

What is the best way to analyze io latency in hdfs?

Hi all,

I'm currently studying HDFS, and I want to analyze HDFS io latency.

I know that C / C ++ programs can use perf and ftrace under Linux to
analyze user level and kernel level latency measurements and overhead.

I would like to analyze the read io latency in HDFS to user level (HDFS)
and system level (kernel I / O stack).

Which way is the best?

Thank you.


ᐧ

Re: What is the best way to analyze io latency in hdfs?

Posted by Julien Laurenceau <ju...@pepitedata.com>.

Hi,
On Linux you can monitor système call of any process using:

strace -p PIDofHDFSdatanode

It can be very verbose but the information will be there.

Did you try metrics available in ambari or cloudera manager ?

Regards

Le mar. 20 août 2019 à 02:47, Daegyu Han <hd...@gmail.com> a écrit :

> Hi all,
>
> I'm currently studying HDFS, and I want to analyze HDFS io latency.
>
> I know that C / C ++ programs can use perf and ftrace under Linux to
> analyze user level and kernel level latency measurements and overhead.
>
> I would like to analyze the read io latency in HDFS to user level (HDFS)
> and system level (kernel I / O stack).
>
> Which way is the best?
>
> Thank you.
>
>
> ᐧ
>