You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Wayne <wa...@gmail.com> on 2012/01/18 17:40:07 UTC
pdflush 100% cpu
We have set up a new cluster and are now seeing pdflush processes pegged at
100% cpu during compaction. Is this normal? It sometimes "looks" like
pdflush is pegged at 100% for quite a while and the java cpu usage goes
down as if it is waiting on it to finish. We are using 0.90.4 on centos 5.5
with no swap. Is this a linux kernal issue due to no swap? This cluster is
the first time we have ever seen this but we are running a slightly newer
kernal than other cluster where we did not see this.
Any ideas? Should we be worried about this? Is/can pdflush really be a
bottleneck to disk writes?
Thanks in advance for any help or suggestions.
Re: pdflush 100% cpu
Posted by Wayne <wa...@gmail.com>.
It was the kernel...thanks for the help. We had considered but could not
really believe a slight kernel version difference would cause this.
Thanks
On Wed, Jan 18, 2012 at 11:47 PM, Stack <st...@duboce.net> wrote:
> On Wed, Jan 18, 2012 at 8:40 AM, Wayne <wa...@gmail.com> wrote:
> > We have set up a new cluster and are now seeing pdflush processes pegged
> at
> > 100% cpu during compaction. Is this normal? It sometimes "looks" like
> > pdflush is pegged at 100% for quite a while and the java cpu usage goes
> > down as if it is waiting on it to finish. We are using 0.90.4 on centos
> 5.5
> > with no swap. Is this a linux kernal issue due to no swap? This cluster
> is
> > the first time we have ever seen this but we are running a slightly newer
> > kernal than other cluster where we did not see this.
> >
>
> I've not noticed it Wayne (can't say I was particularly on the lookout
> for it but I suppose I'd have seen a 100%pdflush if it was going on).
>
> > Any ideas? Should we be worried about this? Is/can pdflush really be a
> > bottleneck to disk writes?
> >
>
> Could you try older kernel on one of the cluster members and see if it
> still 100%s?
>
> What about the pdflush tunings on the two clusters? Are they the
> same? (I was looking at this page:
> http://www.westnet.com/~gsmith/content/linux-pdflush.htm)
>
> St.Ack
>
Re: pdflush 100% cpu
Posted by Stack <st...@duboce.net>.
On Wed, Jan 18, 2012 at 8:40 AM, Wayne <wa...@gmail.com> wrote:
> We have set up a new cluster and are now seeing pdflush processes pegged at
> 100% cpu during compaction. Is this normal? It sometimes "looks" like
> pdflush is pegged at 100% for quite a while and the java cpu usage goes
> down as if it is waiting on it to finish. We are using 0.90.4 on centos 5.5
> with no swap. Is this a linux kernal issue due to no swap? This cluster is
> the first time we have ever seen this but we are running a slightly newer
> kernal than other cluster where we did not see this.
>
I've not noticed it Wayne (can't say I was particularly on the lookout
for it but I suppose I'd have seen a 100%pdflush if it was going on).
> Any ideas? Should we be worried about this? Is/can pdflush really be a
> bottleneck to disk writes?
>
Could you try older kernel on one of the cluster members and see if it
still 100%s?
What about the pdflush tunings on the two clusters? Are they the
same? (I was looking at this page:
http://www.westnet.com/~gsmith/content/linux-pdflush.htm)
St.Ack