You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-user@hadoop.apache.org by manoj <ma...@gmail.com> on 2015/08/20 19:30:29 UTC

How to disable disk capacity check?

Hello all,

I'm on Apache2.6.0.

Look like the Hadoop cluster won't start if the disk occupancy is greater
than 90%.
My hard disk capacity is 1TB and with 90% occupancy I still have 100GB left.
I know for sure that my job won't consume more than 2GB. How can I disable
this check?

Following is from the logs:

WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection:
Directory /tmp/hadoop/logs/userlogs error, used space above threshold of
90.0%, removing from list of valid directories--

Thanks,
--Manoj Kumar M

Re: How to disable disk capacity check?

Posted by manoj <ma...@gmail.com>.

I got the solution :

yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage

sorry for the noise.

Thanks,
-Manoj


On Thu, Aug 20, 2015 at 10:30 AM, manoj <ma...@gmail.com> wrote:

> Hello all,
>
> I'm on Apache2.6.0.
>
> Look like the Hadoop cluster won't start if the disk occupancy is greater
> than 90%.
> My hard disk capacity is 1TB and with 90% occupancy I still have 100GB
> left.
> I know for sure that my job won't consume more than 2GB. How can I disable
> this check?
>
> Following is from the logs:
>
> WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection:
> Directory /tmp/hadoop/logs/userlogs error, used space above threshold of
> 90.0%, removing from list of valid directories--
>
> Thanks,
> --Manoj Kumar M
>



-- 
--Manoj Kumar M

Re: How to disable disk capacity check?

Posted by manoj <ma...@gmail.com>.

I got the solution :

yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage

sorry for the noise.

Thanks,
-Manoj


On Thu, Aug 20, 2015 at 10:30 AM, manoj <ma...@gmail.com> wrote:

> Hello all,
>
> I'm on Apache2.6.0.
>
> Look like the Hadoop cluster won't start if the disk occupancy is greater
> than 90%.
> My hard disk capacity is 1TB and with 90% occupancy I still have 100GB
> left.
> I know for sure that my job won't consume more than 2GB. How can I disable
> this check?
>
> Following is from the logs:
>
> WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection:
> Directory /tmp/hadoop/logs/userlogs error, used space above threshold of
> 90.0%, removing from list of valid directories--
>
> Thanks,
> --Manoj Kumar M
>



-- 
--Manoj Kumar M

Re: How to disable disk capacity check?

Posted by manoj <ma...@gmail.com>.

I got the solution :

yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage

sorry for the noise.

Thanks,
-Manoj


On Thu, Aug 20, 2015 at 10:30 AM, manoj <ma...@gmail.com> wrote:

> Hello all,
>
> I'm on Apache2.6.0.
>
> Look like the Hadoop cluster won't start if the disk occupancy is greater
> than 90%.
> My hard disk capacity is 1TB and with 90% occupancy I still have 100GB
> left.
> I know for sure that my job won't consume more than 2GB. How can I disable
> this check?
>
> Following is from the logs:
>
> WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection:
> Directory /tmp/hadoop/logs/userlogs error, used space above threshold of
> 90.0%, removing from list of valid directories--
>
> Thanks,
> --Manoj Kumar M
>



-- 
--Manoj Kumar M

Re: How to disable disk capacity check?

Posted by manoj <ma...@gmail.com>.

I got the solution :

yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage

sorry for the noise.

Thanks,
-Manoj


On Thu, Aug 20, 2015 at 10:30 AM, manoj <ma...@gmail.com> wrote:

> Hello all,
>
> I'm on Apache2.6.0.
>
> Look like the Hadoop cluster won't start if the disk occupancy is greater
> than 90%.
> My hard disk capacity is 1TB and with 90% occupancy I still have 100GB
> left.
> I know for sure that my job won't consume more than 2GB. How can I disable
> this check?
>
> Following is from the logs:
>
> WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection:
> Directory /tmp/hadoop/logs/userlogs error, used space above threshold of
> 90.0%, removing from list of valid directories--
>
> Thanks,
> --Manoj Kumar M
>



-- 
--Manoj Kumar M