You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Vimal Jain <vk...@gmail.com> on 2013/10/17 14:45:54 UTC
Sudden increase in disk utilization
Hi,
I have configured Hbase in pseudo distributed mode.
I have only one table in hbase (event_data).
When i run following command it shows that this table has utilized 8.38 GB.
Command :-
*[hadoop@woody logs]$ $HBASE_HOME/bin/hbase org.jruby.Main ~/dus.rb
/hbase/event_data*
O/P :-
*Warning: $HADOOP_HOME is deprecated.
hdfs://192.168.20.30:9000/hbase/event_data 8.38 GB
*
where dus.rb is
*include Java
import org.apache.hadoop.util.StringUtils
path = ARGV[0]
dus = %x[$HADOOP_HOME/bin/hadoop fs -dus #{path}]
splited = dus.split
byteDesc = StringUtils.byteDesc(splited[1].to_i)
puts splited[0] + "\t" + byteDesc*
In short hbase command shows that disk utilized is around 9 GB but actual
file system usage is 30 GB.
However this difference vanishes if i restart my Hbase.
I am not able to understand this.
Does Hbase keeps lot of files ( in GBs ) open while working ?
--
Thanks and Regards,
Vimal Jain
Re: Sudden increase in disk utilization
Posted by Vimal Jain <vk...@gmail.com>.
Can anyone please throw some light on this ?
On Fri, Oct 18, 2013 at 7:57 AM, Vimal Jain <vk...@gmail.com> wrote:
> Nopes..i am not using any compression.its disabled by default right?
> Btw i am using hbase 0.94.7
>
> Thanks and Regards,
> Vimal Jain
> On Oct 17, 2013 11:19 PM, "lars hofhansl" <la...@apache.org> wrote:
>
>> Are you using any compression?
>>
>>
>>
>> ________________________________
>> From: Vimal Jain <vk...@gmail.com>
>> To: user@hbase.apache.org
>> Sent: Thursday, October 17, 2013 6:15 AM
>> Subject: Sudden increase in disk utilization
>>
>>
>> Hi,
>> I have configured Hbase in pseudo distributed mode.
>> I have only one table in hbase (event_data).
>> When i run following command it shows that this table has utilized 8.38
>> GB.
>>
>> Command :-
>> *[hadoop@woody logs]$ $HBASE_HOME/bin/hbase org.jruby.Main ~/dus.rb
>> /hbase/event_data*
>>
>> O/P :-
>> *Warning: $HADOOP_HOME is deprecated.
>>
>> hdfs://192.168.20.30:9000/hbase/event_data 8.38 GB
>> *
>>
>> where dus.rb is
>>
>> *include Java
>> import org.apache.hadoop.util.StringUtils
>> path = ARGV[0]
>> dus = %x[$HADOOP_HOME/bin/hadoop fs -dus #{path}]
>> splited = dus.split
>> byteDesc = StringUtils.byteDesc(splited[1].to_i)
>> puts splited[0] + "\t" + byteDesc*
>>
>>
>>
>> In short hbase command shows that disk utilized is around 9 GB but actual
>> file system usage is 30 GB.
>> However this difference vanishes if i restart my Hbase.
>> I am not able to understand this.
>> Does Hbase keeps lot of files ( in GBs ) open while working ?
>> --
>> Thanks and Regards,
>> Vimal Jain
>
>
--
Thanks and Regards,
Vimal Jain
Re: Sudden increase in disk utilization
Posted by Vimal Jain <vk...@gmail.com>.
Nopes..i am not using any compression.its disabled by default right?
Btw i am using hbase 0.94.7
Thanks and Regards,
Vimal Jain
On Oct 17, 2013 11:19 PM, "lars hofhansl" <la...@apache.org> wrote:
> Are you using any compression?
>
>
>
> ________________________________
> From: Vimal Jain <vk...@gmail.com>
> To: user@hbase.apache.org
> Sent: Thursday, October 17, 2013 6:15 AM
> Subject: Sudden increase in disk utilization
>
>
> Hi,
> I have configured Hbase in pseudo distributed mode.
> I have only one table in hbase (event_data).
> When i run following command it shows that this table has utilized 8.38
> GB.
>
> Command :-
> *[hadoop@woody logs]$ $HBASE_HOME/bin/hbase org.jruby.Main ~/dus.rb
> /hbase/event_data*
>
> O/P :-
> *Warning: $HADOOP_HOME is deprecated.
>
> hdfs://192.168.20.30:9000/hbase/event_data 8.38 GB
> *
>
> where dus.rb is
>
> *include Java
> import org.apache.hadoop.util.StringUtils
> path = ARGV[0]
> dus = %x[$HADOOP_HOME/bin/hadoop fs -dus #{path}]
> splited = dus.split
> byteDesc = StringUtils.byteDesc(splited[1].to_i)
> puts splited[0] + "\t" + byteDesc*
>
>
>
> In short hbase command shows that disk utilized is around 9 GB but actual
> file system usage is 30 GB.
> However this difference vanishes if i restart my Hbase.
> I am not able to understand this.
> Does Hbase keeps lot of files ( in GBs ) open while working ?
> --
> Thanks and Regards,
> Vimal Jain
Re: Sudden increase in disk utilization
Posted by lars hofhansl <la...@apache.org>.
Are you using any compression?
________________________________
From: Vimal Jain <vk...@gmail.com>
To: user@hbase.apache.org
Sent: Thursday, October 17, 2013 6:15 AM
Subject: Sudden increase in disk utilization
Hi,
I have configured Hbase in pseudo distributed mode.
I have only one table in hbase (event_data).
When i run following command it shows that this table has utilized 8.38 GB.
Command :-
*[hadoop@woody logs]$ $HBASE_HOME/bin/hbase org.jruby.Main ~/dus.rb
/hbase/event_data*
O/P :-
*Warning: $HADOOP_HOME is deprecated.
hdfs://192.168.20.30:9000/hbase/event_data 8.38 GB
*
where dus.rb is
*include Java
import org.apache.hadoop.util.StringUtils
path = ARGV[0]
dus = %x[$HADOOP_HOME/bin/hadoop fs -dus #{path}]
splited = dus.split
byteDesc = StringUtils.byteDesc(splited[1].to_i)
puts splited[0] + "\t" + byteDesc*
In short hbase command shows that disk utilized is around 9 GB but actual
file system usage is 30 GB.
However this difference vanishes if i restart my Hbase.
I am not able to understand this.
Does Hbase keeps lot of files ( in GBs ) open while working ?
--
Thanks and Regards,
Vimal Jain