You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Dmitry Pushkarev <um...@stanford.edu> on 2009/02/03 18:53:04 UTC
HDD benchmark/checking tool
Dear hadoop users,
Recently I have had a number of drive failures that slowed down processes a
lot until they were discovered. It is there any easy way or tool, to check
HDD performance and see if there any IO errors?
Currently I wrote a simple script that looks at /var/log/messages and greps
everything abnormal for /dev/sdaX. But if you have better solution I'd
appreciate if you share it.
---
Dmitry Pushkarev
+1-650-644-8988
Re: HDD benchmark/checking tool
Posted by Brian Bockelman <bb...@cse.unl.edu>.
Also, you want to look at combining SMART hard drive monitoring (most
drives support SMART at this point) and combine it with Nagios.
It often lets us known when a hard drive is about to fail *and* when
the drive is under-performing.
Brian
On Feb 3, 2009, at 6:18 PM, Aaron Kimball wrote:
> Dmitry,
>
> Look into cluster/system monitoring tools: nagios and ganglia are
> two to
> start with.
> - Aaron
>
> On Tue, Feb 3, 2009 at 9:53 AM, Dmitry Pushkarev <um...@stanford.edu>
> wrote:
>
>> Dear hadoop users,
>>
>>
>>
>> Recently I have had a number of drive failures that slowed down
>> processes a
>> lot until they were discovered. It is there any easy way or tool,
>> to check
>> HDD performance and see if there any IO errors?
>>
>> Currently I wrote a simple script that looks at /var/log/messages
>> and greps
>> everything abnormal for /dev/sdaX. But if you have better solution
>> I'd
>> appreciate if you share it.
>>
>>
>>
>> ---
>>
>> Dmitry Pushkarev
>>
>> +1-650-644-8988
>>
>>
>>
>>
Re: HDD benchmark/checking tool
Posted by Aaron Kimball <aa...@cloudera.com>.
Dmitry,
Look into cluster/system monitoring tools: nagios and ganglia are two to
start with.
- Aaron
On Tue, Feb 3, 2009 at 9:53 AM, Dmitry Pushkarev <um...@stanford.edu> wrote:
> Dear hadoop users,
>
>
>
> Recently I have had a number of drive failures that slowed down processes a
> lot until they were discovered. It is there any easy way or tool, to check
> HDD performance and see if there any IO errors?
>
> Currently I wrote a simple script that looks at /var/log/messages and greps
> everything abnormal for /dev/sdaX. But if you have better solution I'd
> appreciate if you share it.
>
>
>
> ---
>
> Dmitry Pushkarev
>
> +1-650-644-8988
>
>
>
>
Re: HDD benchmark/checking tool
Posted by Mikhail Yakshin <gr...@gmail.com>.
On Tue, Feb 3, 2009 at 8:53 PM, Dmitry Pushkarev wrote:
> Recently I have had a number of drive failures that slowed down processes a
> lot until they were discovered. It is there any easy way or tool, to check
> HDD performance and see if there any IO errors?
>
> Currently I wrote a simple script that looks at /var/log/messages and greps
> everything abnormal for /dev/sdaX. But if you have better solution I'd
> appreciate if you share it.
If you have any hardware RAIDs you'd like to monitor/manage, good
chances that you'd want to use Einarc to access them:
http://www.inquisitor.ru/doc/einarc/ - in fact, it won't hurt even if
you use just a bunch of HDDs or software RAIDs :)
--
WBR, Mikhail Yakshin