Need help, Can I disable datanode block report?

[Xibin Liu <xi...@gmail.com>]

hi all,
My data nodes have very large disks(20T+), and when data node reports block
info, the hdfs will not be able to read or write. Can I disable this
feature or give it a very long interval without risk? My hadoop version is
0.20.2.

-- 
Thanks & Regards
Benjamin Liu

Re: Need help, Can I disable datanode block report?

[Xibin Liu <xi...@gmail.com>]

I'v set block report interval to 1 day, it works fine, shouldn't hadoop
report error if namenode consider the blocks are missing?



2013/4/10 Nitin Pawar <ni...@gmail.com>

> disabling the block report will cause the namenode to consider the blocks
> are missing.
>
> if you think it is slowing down the cluster then you can increase the time
> for heartbeats . Also if possible re-balance the blocks again if you added
> new nodes
>
>
> On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com> wrote:
>
>> hi all,
>> My data nodes have very large disks(20T+), and when data node reports
>> block info, the hdfs will not be able to read or write. Can I disable this
>> feature or give it a very long interval without risk? My hadoop version is
>> 0.20.2.
>>
>> --
>> Thanks & Regards
>> Benjamin Liu
>>
>>
>
>
> --
> Nitin Pawar
>



-- 
Thanks & Regards
Benjamin Liu

Re: Need help, Can I disable datanode block report?

[Xibin Liu <xi...@gmail.com>]

I'v set block report interval to 1 day, it works fine, shouldn't hadoop
report error if namenode consider the blocks are missing?



2013/4/10 Nitin Pawar <ni...@gmail.com>

> disabling the block report will cause the namenode to consider the blocks
> are missing.
>
> if you think it is slowing down the cluster then you can increase the time
> for heartbeats . Also if possible re-balance the blocks again if you added
> new nodes
>
>
> On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com> wrote:
>
>> hi all,
>> My data nodes have very large disks(20T+), and when data node reports
>> block info, the hdfs will not be able to read or write. Can I disable this
>> feature or give it a very long interval without risk? My hadoop version is
>> 0.20.2.
>>
>> --
>> Thanks & Regards
>> Benjamin Liu
>>
>>
>
>
> --
> Nitin Pawar
>



-- 
Thanks & Regards
Benjamin Liu

Re: Need help, Can I disable datanode block report?

[Xibin Liu <xi...@gmail.com>]

I'v set block report interval to 1 day, it works fine, shouldn't hadoop
report error if namenode consider the blocks are missing?



2013/4/10 Nitin Pawar <ni...@gmail.com>

> disabling the block report will cause the namenode to consider the blocks
> are missing.
>
> if you think it is slowing down the cluster then you can increase the time
> for heartbeats . Also if possible re-balance the blocks again if you added
> new nodes
>
>
> On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com> wrote:
>
>> hi all,
>> My data nodes have very large disks(20T+), and when data node reports
>> block info, the hdfs will not be able to read or write. Can I disable this
>> feature or give it a very long interval without risk? My hadoop version is
>> 0.20.2.
>>
>> --
>> Thanks & Regards
>> Benjamin Liu
>>
>>
>
>
> --
> Nitin Pawar
>



-- 
Thanks & Regards
Benjamin Liu

Re: Need help, Can I disable datanode block report?

[Xibin Liu <xi...@gmail.com>]

I'v set block report interval to 1 day, it works fine, shouldn't hadoop
report error if namenode consider the blocks are missing?



2013/4/10 Nitin Pawar <ni...@gmail.com>

> disabling the block report will cause the namenode to consider the blocks
> are missing.
>
> if you think it is slowing down the cluster then you can increase the time
> for heartbeats . Also if possible re-balance the blocks again if you added
> new nodes
>
>
> On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com> wrote:
>
>> hi all,
>> My data nodes have very large disks(20T+), and when data node reports
>> block info, the hdfs will not be able to read or write. Can I disable this
>> feature or give it a very long interval without risk? My hadoop version is
>> 0.20.2.
>>
>> --
>> Thanks & Regards
>> Benjamin Liu
>>
>>
>
>
> --
> Nitin Pawar
>



-- 
Thanks & Regards
Benjamin Liu

Re: Need help, Can I disable datanode block report?

[Nitin Pawar <ni...@gmail.com>]

disabling the block report will cause the namenode to consider the blocks
are missing.

if you think it is slowing down the cluster then you can increase the time
for heartbeats . Also if possible re-balance the blocks again if you added
new nodes


On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com> wrote:

> hi all,
> My data nodes have very large disks(20T+), and when data node reports
> block info, the hdfs will not be able to read or write. Can I disable this
> feature or give it a very long interval without risk? My hadoop version is
> 0.20.2.
>
> --
> Thanks & Regards
> Benjamin Liu
>
>


-- 
Nitin Pawar

Re: Need help, Can I disable datanode block report?

[Nitin Pawar <ni...@gmail.com>]

disabling the block report will cause the namenode to consider the blocks
are missing.

if you think it is slowing down the cluster then you can increase the time
for heartbeats . Also if possible re-balance the blocks again if you added
new nodes


On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com> wrote:

> hi all,
> My data nodes have very large disks(20T+), and when data node reports
> block info, the hdfs will not be able to read or write. Can I disable this
> feature or give it a very long interval without risk? My hadoop version is
> 0.20.2.
>
> --
> Thanks & Regards
> Benjamin Liu
>
>


-- 
Nitin Pawar

Re: Need help, Can I disable datanode block report?

[Nitin Pawar <ni...@gmail.com>]

disabling the block report will cause the namenode to consider the blocks
are missing.

if you think it is slowing down the cluster then you can increase the time
for heartbeats . Also if possible re-balance the blocks again if you added
new nodes


On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com> wrote:

> hi all,
> My data nodes have very large disks(20T+), and when data node reports
> block info, the hdfs will not be able to read or write. Can I disable this
> feature or give it a very long interval without risk? My hadoop version is
> 0.20.2.
>
> --
> Thanks & Regards
> Benjamin Liu
>
>


-- 
Nitin Pawar

Re: Need help, Can I disable datanode block report?

[Nitin Pawar <ni...@gmail.com>]

disabling the block report will cause the namenode to consider the blocks
are missing.

if you think it is slowing down the cluster then you can increase the time
for heartbeats . Also if possible re-balance the blocks again if you added
new nodes


On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com> wrote:

> hi all,
> My data nodes have very large disks(20T+), and when data node reports
> block info, the hdfs will not be able to read or write. Can I disable this
> feature or give it a very long interval without risk? My hadoop version is
> 0.20.2.
>
> --
> Thanks & Regards
> Benjamin Liu
>
>


-- 
Nitin Pawar

Re: Need help, Can I disable datanode block report?

[Harsh J <ha...@cloudera.com>]

While upgrading is safe and works, note that 0.20.2 -> 1.0.4 is not
backwards compatible. That is, existing 0.20.2 clients may fail to work
with 1.0.4 services, if left un-upgraded as well.

Regarding specifying a great length of delay in block reports, it may
affect a few operations that depend on most current block info at the
NameNode, but I can't imagine a very critical issue being caused as a
result of that. You should be OK (afaik) but certainly either apply
HDFS-2379, or upgrade, soon.


On Wed, Apr 10, 2013 at 1:55 PM, Xibin Liu <xi...@gmail.com> wrote:

> Is upgrade from 0.20.2 to 1.0.4 safe? I have nearly 100T data and the data
> are very important.
>
>
> 2013/4/10 Harsh J <ha...@cloudera.com>
>
>> Xibin,
>>
>> I'd recommend upgrading. This was addressed quite a while ago with
>> HDFS-2379.
>>
>>
>> On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com>wrote:
>>
>>> hi all,
>>> My data nodes have very large disks(20T+), and when data node reports
>>> block info, the hdfs will not be able to read or write. Can I disable this
>>> feature or give it a very long interval without risk? My hadoop version is
>>> 0.20.2.
>>>
>>> --
>>> Thanks & Regards
>>> Benjamin Liu
>>>
>>>
>>
>>
>> --
>> Harsh J
>>
>
>
>
> --
> Thanks & Regards
> Benjamin Liu
>
>


-- 
Harsh J

Re: Need help, Can I disable datanode block report?

[Harsh J <ha...@cloudera.com>]

While upgrading is safe and works, note that 0.20.2 -> 1.0.4 is not
backwards compatible. That is, existing 0.20.2 clients may fail to work
with 1.0.4 services, if left un-upgraded as well.

Regarding specifying a great length of delay in block reports, it may
affect a few operations that depend on most current block info at the
NameNode, but I can't imagine a very critical issue being caused as a
result of that. You should be OK (afaik) but certainly either apply
HDFS-2379, or upgrade, soon.


On Wed, Apr 10, 2013 at 1:55 PM, Xibin Liu <xi...@gmail.com> wrote:

> Is upgrade from 0.20.2 to 1.0.4 safe? I have nearly 100T data and the data
> are very important.
>
>
> 2013/4/10 Harsh J <ha...@cloudera.com>
>
>> Xibin,
>>
>> I'd recommend upgrading. This was addressed quite a while ago with
>> HDFS-2379.
>>
>>
>> On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com>wrote:
>>
>>> hi all,
>>> My data nodes have very large disks(20T+), and when data node reports
>>> block info, the hdfs will not be able to read or write. Can I disable this
>>> feature or give it a very long interval without risk? My hadoop version is
>>> 0.20.2.
>>>
>>> --
>>> Thanks & Regards
>>> Benjamin Liu
>>>
>>>
>>
>>
>> --
>> Harsh J
>>
>
>
>
> --
> Thanks & Regards
> Benjamin Liu
>
>


-- 
Harsh J

Re: Need help, Can I disable datanode block report?

[Harsh J <ha...@cloudera.com>]

While upgrading is safe and works, note that 0.20.2 -> 1.0.4 is not
backwards compatible. That is, existing 0.20.2 clients may fail to work
with 1.0.4 services, if left un-upgraded as well.

Regarding specifying a great length of delay in block reports, it may
affect a few operations that depend on most current block info at the
NameNode, but I can't imagine a very critical issue being caused as a
result of that. You should be OK (afaik) but certainly either apply
HDFS-2379, or upgrade, soon.


On Wed, Apr 10, 2013 at 1:55 PM, Xibin Liu <xi...@gmail.com> wrote:

> Is upgrade from 0.20.2 to 1.0.4 safe? I have nearly 100T data and the data
> are very important.
>
>
> 2013/4/10 Harsh J <ha...@cloudera.com>
>
>> Xibin,
>>
>> I'd recommend upgrading. This was addressed quite a while ago with
>> HDFS-2379.
>>
>>
>> On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com>wrote:
>>
>>> hi all,
>>> My data nodes have very large disks(20T+), and when data node reports
>>> block info, the hdfs will not be able to read or write. Can I disable this
>>> feature or give it a very long interval without risk? My hadoop version is
>>> 0.20.2.
>>>
>>> --
>>> Thanks & Regards
>>> Benjamin Liu
>>>
>>>
>>
>>
>> --
>> Harsh J
>>
>
>
>
> --
> Thanks & Regards
> Benjamin Liu
>
>


-- 
Harsh J

Re: Need help, Can I disable datanode block report?

[Harsh J <ha...@cloudera.com>]

While upgrading is safe and works, note that 0.20.2 -> 1.0.4 is not
backwards compatible. That is, existing 0.20.2 clients may fail to work
with 1.0.4 services, if left un-upgraded as well.

Regarding specifying a great length of delay in block reports, it may
affect a few operations that depend on most current block info at the
NameNode, but I can't imagine a very critical issue being caused as a
result of that. You should be OK (afaik) but certainly either apply
HDFS-2379, or upgrade, soon.


On Wed, Apr 10, 2013 at 1:55 PM, Xibin Liu <xi...@gmail.com> wrote:

> Is upgrade from 0.20.2 to 1.0.4 safe? I have nearly 100T data and the data
> are very important.
>
>
> 2013/4/10 Harsh J <ha...@cloudera.com>
>
>> Xibin,
>>
>> I'd recommend upgrading. This was addressed quite a while ago with
>> HDFS-2379.
>>
>>
>> On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com>wrote:
>>
>>> hi all,
>>> My data nodes have very large disks(20T+), and when data node reports
>>> block info, the hdfs will not be able to read or write. Can I disable this
>>> feature or give it a very long interval without risk? My hadoop version is
>>> 0.20.2.
>>>
>>> --
>>> Thanks & Regards
>>> Benjamin Liu
>>>
>>>
>>
>>
>> --
>> Harsh J
>>
>
>
>
> --
> Thanks & Regards
> Benjamin Liu
>
>


-- 
Harsh J

Re: Need help, Can I disable datanode block report?

[Xibin Liu <xi...@gmail.com>]

Is upgrade from 0.20.2 to 1.0.4 safe? I have nearly 100T data and the data
are very important.


2013/4/10 Harsh J <ha...@cloudera.com>

> Xibin,
>
> I'd recommend upgrading. This was addressed quite a while ago with
> HDFS-2379.
>
>
> On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com> wrote:
>
>> hi all,
>> My data nodes have very large disks(20T+), and when data node reports
>> block info, the hdfs will not be able to read or write. Can I disable this
>> feature or give it a very long interval without risk? My hadoop version is
>> 0.20.2.
>>
>> --
>> Thanks & Regards
>> Benjamin Liu
>>
>>
>
>
> --
> Harsh J
>



-- 
Thanks & Regards
Benjamin Liu

Re: Need help, Can I disable datanode block report?

[Xibin Liu <xi...@gmail.com>]

Is upgrade from 0.20.2 to 1.0.4 safe? I have nearly 100T data and the data
are very important.


2013/4/10 Harsh J <ha...@cloudera.com>

> Xibin,
>
> I'd recommend upgrading. This was addressed quite a while ago with
> HDFS-2379.
>
>
> On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com> wrote:
>
>> hi all,
>> My data nodes have very large disks(20T+), and when data node reports
>> block info, the hdfs will not be able to read or write. Can I disable this
>> feature or give it a very long interval without risk? My hadoop version is
>> 0.20.2.
>>
>> --
>> Thanks & Regards
>> Benjamin Liu
>>
>>
>
>
> --
> Harsh J
>



-- 
Thanks & Regards
Benjamin Liu

Re: Need help, Can I disable datanode block report?

[Xibin Liu <xi...@gmail.com>]

Is upgrade from 0.20.2 to 1.0.4 safe? I have nearly 100T data and the data
are very important.


2013/4/10 Harsh J <ha...@cloudera.com>

> Xibin,
>
> I'd recommend upgrading. This was addressed quite a while ago with
> HDFS-2379.
>
>
> On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com> wrote:
>
>> hi all,
>> My data nodes have very large disks(20T+), and when data node reports
>> block info, the hdfs will not be able to read or write. Can I disable this
>> feature or give it a very long interval without risk? My hadoop version is
>> 0.20.2.
>>
>> --
>> Thanks & Regards
>> Benjamin Liu
>>
>>
>
>
> --
> Harsh J
>



-- 
Thanks & Regards
Benjamin Liu

Re: Need help, Can I disable datanode block report?

[Xibin Liu <xi...@gmail.com>]

Is upgrade from 0.20.2 to 1.0.4 safe? I have nearly 100T data and the data
are very important.


2013/4/10 Harsh J <ha...@cloudera.com>

> Xibin,
>
> I'd recommend upgrading. This was addressed quite a while ago with
> HDFS-2379.
>
>
> On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com> wrote:
>
>> hi all,
>> My data nodes have very large disks(20T+), and when data node reports
>> block info, the hdfs will not be able to read or write. Can I disable this
>> feature or give it a very long interval without risk? My hadoop version is
>> 0.20.2.
>>
>> --
>> Thanks & Regards
>> Benjamin Liu
>>
>>
>
>
> --
> Harsh J
>



-- 
Thanks & Regards
Benjamin Liu

Re: Need help, Can I disable datanode block report?

[Harsh J <ha...@cloudera.com>]

Xibin,

I'd recommend upgrading. This was addressed quite a while ago with
HDFS-2379.


On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com> wrote:

> hi all,
> My data nodes have very large disks(20T+), and when data node reports
> block info, the hdfs will not be able to read or write. Can I disable this
> feature or give it a very long interval without risk? My hadoop version is
> 0.20.2.
>
> --
> Thanks & Regards
> Benjamin Liu
>
>


-- 
Harsh J

Re: Need help, Can I disable datanode block report?

[Harsh J <ha...@cloudera.com>]

Xibin,

I'd recommend upgrading. This was addressed quite a while ago with
HDFS-2379.


On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com> wrote:

> hi all,
> My data nodes have very large disks(20T+), and when data node reports
> block info, the hdfs will not be able to read or write. Can I disable this
> feature or give it a very long interval without risk? My hadoop version is
> 0.20.2.
>
> --
> Thanks & Regards
> Benjamin Liu
>
>


-- 
Harsh J

Re: Need help, Can I disable datanode block report?

[Harsh J <ha...@cloudera.com>]

Xibin,

I'd recommend upgrading. This was addressed quite a while ago with
HDFS-2379.


On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com> wrote:

> hi all,
> My data nodes have very large disks(20T+), and when data node reports
> block info, the hdfs will not be able to read or write. Can I disable this
> feature or give it a very long interval without risk? My hadoop version is
> 0.20.2.
>
> --
> Thanks & Regards
> Benjamin Liu
>
>


-- 
Harsh J

Re: Need help, Can I disable datanode block report?

[Harsh J <ha...@cloudera.com>]

Xibin,

I'd recommend upgrading. This was addressed quite a while ago with
HDFS-2379.


On Wed, Apr 10, 2013 at 1:35 PM, Xibin Liu <xi...@gmail.com> wrote:

> hi all,
> My data nodes have very large disks(20T+), and when data node reports
> block info, the hdfs will not be able to read or write. Can I disable this
> feature or give it a very long interval without risk? My hadoop version is
> 0.20.2.
>
> --
> Thanks & Regards
> Benjamin Liu
>
>


-- 
Harsh J