You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@hadoop.apache.org by Anshuman Sachdeva <as...@attributor.com> on 2009/05/01 22:13:46 UTC
du command hangs the machine
Hi All,
My engineers have seen this issue. After every few days the node hangs. we are using xfs file system and when ever hadoop runs "du" command and the data on the file system is really high and it some how locks the directory and we end up rebooting the machine.
A help or guidance from any one will be helpful
Thanks
Re: du command hangs the machine
Posted by jason hadoop <ja...@gmail.com>.
There is a particular problem with XFS that I have only seen on PAE kernels
that can cause XFS deadlocks, on top of the long duration du issue.
We never had a solution but it only seemed to happen on the machines with
the PAE kernels (32 bit kernels with larger than 4gig of physical ram in the
machine).
On Fri, May 1, 2009 at 1:33 PM, Owen O'Malley <om...@apache.org> wrote:
>
> On May 1, 2009, at 1:13 PM, Anshuman Sachdeva wrote:
>
> Hi All,
>> My engineers have seen this issue. After every few days the node
>> hangs. we are using xfs file system and when ever hadoop runs "du" command
>> and the data on the file system is really high and it some how locks the
>> directory and we end up rebooting the machine.
>>
>> A help or guidance from any one will be helpful
>>
>
> Probably the best way to solve the problem is:
>
> https://issues.apache.org/jira/browse/HADOOP-4998
>
> It would be great to have native libraries that provide the functionality
> we need without forking at all.
>
> -- Owen
>
--
Alpha Chapters of my book on Hadoop are available
http://www.apress.com/book/view/9781430219422
Re: du command hangs the machine
Posted by Owen O'Malley <om...@apache.org>.
On May 1, 2009, at 1:13 PM, Anshuman Sachdeva wrote:
> Hi All,
> My engineers have seen this issue. After every few days the
> node hangs. we are using xfs file system and when ever hadoop runs
> "du" command and the data on the file system is really high and it
> some how locks the directory and we end up rebooting the machine.
>
> A help or guidance from any one will be helpful
Probably the best way to solve the problem is:
https://issues.apache.org/jira/browse/HADOOP-4998
It would be great to have native libraries that provide the
functionality we need without forking at all.
-- Owen