You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Kang Minwoo <mi...@outlook.com> on 2019/03/11 04:22:13 UTC
Region flush delayed even memstore size above MEMSTORE_FLUSH config.
Hello Users.
---
HBase version is 1.2.9
---
I wonder this region operation is intended.
I set "hbase.regionserver.optionalcacheflushinterval" slightly shorter than the default setting.
So cf has old edit, they flush after a random delay.
If the flush queue has a flush request by old edit, a flush request by memstore size above MEMSTORE_FLUSH config in the same region is ignored.
Because region already has in regionsInQueue.
As a result, memstore size increase until random delay.
I think a flush request by memstore size above MEMSTORE_FLUSH config is a higher priority than a flush request by old edit.
Here are related codes.
---
@Override
public void requestFlush(Region r, boolean forceFlushAllStores) {
synchronized (regionsInQueue) {
if (!regionsInQueue.containsKey(r)) { // <- Here
FlushRegionEntry fqe = new FlushRegionEntry(r, forceFlushAllStores);
this.regionsInQueue.put(r, fqe);
this.flushQueue.add(fqe);
}
}
}
@Override
public void requestDelayedFlush(Region r, long delay, boolean forceFlushAllStores) {
synchronized (regionsInQueue) {
if (!regionsInQueue.containsKey(r)) { // <- Here
FlushRegionEntry fqe = new FlushRegionEntry(r, forceFlushAllStores);
fqe.requeue(delay);
this.regionsInQueue.put(r, fqe);
this.flushQueue.add(fqe);
}
}
}
---
Best regards,
Minwoo Kang
Re: Region flush delayed even memstore size above MEMSTORE_FLUSH
config.
Posted by Kang Minwoo <mi...@outlook.com>.
If region flush request stays in the queue for 5 minutes and there is no global memstore pressure, the region can grow memstore size.
But it seems that adjusting the settings is right.
________________________________________
보낸 사람: Wellington Chevreuil <we...@gmail.com>
보낸 날짜: 2019년 3월 11일 월요일 21:11
받는 사람: user@hbase.apache.org
제목: Re: Region flush delayed even memstore size above MEMSTORE_FLUSH config.
Well, there would be already a flush request for the region in the queue
anyway, and this would force flush of all stores, so doesn't look adding an
additional request would have any benefit, does it?
Em seg, 11 de mar de 2019 às 04:22, Kang Minwoo <mi...@outlook.com>
escreveu:
> Hello Users.
>
> ---
> HBase version is 1.2.9
> ---
>
> I wonder this region operation is intended.
>
> I set "hbase.regionserver.optionalcacheflushinterval" slightly shorter
> than the default setting.
> So cf has old edit, they flush after a random delay.
>
> If the flush queue has a flush request by old edit, a flush request by
> memstore size above MEMSTORE_FLUSH config in the same region is ignored.
> Because region already has in regionsInQueue.
>
> As a result, memstore size increase until random delay.
>
> I think a flush request by memstore size above MEMSTORE_FLUSH config is a
> higher priority than a flush request by old edit.
>
>
> Here are related codes.
>
> ---
> @Override
> public void requestFlush(Region r, boolean forceFlushAllStores) {
> synchronized (regionsInQueue) {
> if (!regionsInQueue.containsKey(r)) { // <- Here
> FlushRegionEntry fqe = new FlushRegionEntry(r,
> forceFlushAllStores);
> this.regionsInQueue.put(r, fqe);
> this.flushQueue.add(fqe);
> }
> }
> }
>
> @Override
> public void requestDelayedFlush(Region r, long delay, boolean
> forceFlushAllStores) {
> synchronized (regionsInQueue) {
> if (!regionsInQueue.containsKey(r)) { // <- Here
> FlushRegionEntry fqe = new FlushRegionEntry(r,
> forceFlushAllStores);
> fqe.requeue(delay);
> this.regionsInQueue.put(r, fqe);
> this.flushQueue.add(fqe);
> }
> }
> }
> ---
>
> Best regards,
> Minwoo Kang
>
Re: Region flush delayed even memstore size above MEMSTORE_FLUSH config.
Posted by Wellington Chevreuil <we...@gmail.com>.
Well, there would be already a flush request for the region in the queue
anyway, and this would force flush of all stores, so doesn't look adding an
additional request would have any benefit, does it?
Em seg, 11 de mar de 2019 às 04:22, Kang Minwoo <mi...@outlook.com>
escreveu:
> Hello Users.
>
> ---
> HBase version is 1.2.9
> ---
>
> I wonder this region operation is intended.
>
> I set "hbase.regionserver.optionalcacheflushinterval" slightly shorter
> than the default setting.
> So cf has old edit, they flush after a random delay.
>
> If the flush queue has a flush request by old edit, a flush request by
> memstore size above MEMSTORE_FLUSH config in the same region is ignored.
> Because region already has in regionsInQueue.
>
> As a result, memstore size increase until random delay.
>
> I think a flush request by memstore size above MEMSTORE_FLUSH config is a
> higher priority than a flush request by old edit.
>
>
> Here are related codes.
>
> ---
> @Override
> public void requestFlush(Region r, boolean forceFlushAllStores) {
> synchronized (regionsInQueue) {
> if (!regionsInQueue.containsKey(r)) { // <- Here
> FlushRegionEntry fqe = new FlushRegionEntry(r,
> forceFlushAllStores);
> this.regionsInQueue.put(r, fqe);
> this.flushQueue.add(fqe);
> }
> }
> }
>
> @Override
> public void requestDelayedFlush(Region r, long delay, boolean
> forceFlushAllStores) {
> synchronized (regionsInQueue) {
> if (!regionsInQueue.containsKey(r)) { // <- Here
> FlushRegionEntry fqe = new FlushRegionEntry(r,
> forceFlushAllStores);
> fqe.requeue(delay);
> this.regionsInQueue.put(r, fqe);
> this.flushQueue.add(fqe);
> }
> }
> }
> ---
>
> Best regards,
> Minwoo Kang
>