xcievers

[Patai Sangbutsarakum <si...@gmail.com>]

Hello Hadoopers,

How's your cluster behave today ?? hope they run well and strong.

In the past or some bad days i saw 'Too many fetch-failure'; it was
fixed by adjusting dfs.datanode.max.xcievers to 6k.

I am still curious, how do we monitor the consumption of this value in
each datanode. I am looking it like other resource in system that
should be monitored and see the trend; probably can do a heatmap on
this metric. So we can possibly pinpoint the potential problem too.

Is this metric exposed in someway ?


Thanks
Patai

Re: xcievers

[Patai Sangbutsarakum <si...@gmail.com>]

Thanks George

On Thu, Feb 7, 2013 at 4:07 PM, George Datskos
<ge...@jp.fujitsu.com> wrote:
> Patai,
>
>
>> I am still curious, how do we monitor the consumption of this value in
>> each datanode.
>
>
> You can use the getDataNodeStats() method of your your DistributedFileSystem
> instance.  It returns an array of DatanodeInfo which contains, among other
> things, the xceiver count that you are looking for.
>
>
> George

Re: xcievers

[Patai Sangbutsarakum <si...@gmail.com>]

Thanks George

On Thu, Feb 7, 2013 at 4:07 PM, George Datskos
<ge...@jp.fujitsu.com> wrote:
> Patai,
>
>
>> I am still curious, how do we monitor the consumption of this value in
>> each datanode.
>
>
> You can use the getDataNodeStats() method of your your DistributedFileSystem
> instance.  It returns an array of DatanodeInfo which contains, among other
> things, the xceiver count that you are looking for.
>
>
> George

Re: xcievers

[Patai Sangbutsarakum <si...@gmail.com>]

Thanks George

On Thu, Feb 7, 2013 at 4:07 PM, George Datskos
<ge...@jp.fujitsu.com> wrote:
> Patai,
>
>
>> I am still curious, how do we monitor the consumption of this value in
>> each datanode.
>
>
> You can use the getDataNodeStats() method of your your DistributedFileSystem
> instance.  It returns an array of DatanodeInfo which contains, among other
> things, the xceiver count that you are looking for.
>
>
> George

Re: xcievers

[Patai Sangbutsarakum <si...@gmail.com>]

Thanks George

On Thu, Feb 7, 2013 at 4:07 PM, George Datskos
<ge...@jp.fujitsu.com> wrote:
> Patai,
>
>
>> I am still curious, how do we monitor the consumption of this value in
>> each datanode.
>
>
> You can use the getDataNodeStats() method of your your DistributedFileSystem
> instance.  It returns an array of DatanodeInfo which contains, among other
> things, the xceiver count that you are looking for.
>
>
> George

Re: xcievers

[George Datskos <ge...@jp.fujitsu.com>]

Patai,

> I am still curious, how do we monitor the consumption of this value in
> each datanode.

You can use the getDataNodeStats() method of your your 
DistributedFileSystem instance.  It returns an array of DatanodeInfo 
which contains, among other things, the xceiver count that you are 
looking for.


George

Re: xcievers

[George Datskos <ge...@jp.fujitsu.com>]

Patai,

> I am still curious, how do we monitor the consumption of this value in
> each datanode.

You can use the getDataNodeStats() method of your your 
DistributedFileSystem instance.  It returns an array of DatanodeInfo 
which contains, among other things, the xceiver count that you are 
looking for.


George

Re: xcievers

[George Datskos <ge...@jp.fujitsu.com>]

Patai,

> I am still curious, how do we monitor the consumption of this value in
> each datanode.

You can use the getDataNodeStats() method of your your 
DistributedFileSystem instance.  It returns an array of DatanodeInfo 
which contains, among other things, the xceiver count that you are 
looking for.


George

Re: xcievers

[George Datskos <ge...@jp.fujitsu.com>]

Patai,

> I am still curious, how do we monitor the consumption of this value in
> each datanode.

You can use the getDataNodeStats() method of your your 
DistributedFileSystem instance.  It returns an array of DatanodeInfo 
which contains, among other things, the xceiver count that you are 
looking for.


George