You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hbase.apache.org by Kang Minwoo <mi...@outlook.com> on 2020/02/25 04:19:48 UTC

Is there any way to check the system stop is requested in performCompaction over time?

Hello Users.

Is there any way to check the system stop is requested in performCompaction over time?

When the region got a close request, the region should wait there is no compaction and flush.
However, in performCompaction method checked periodically only by write bytes.
If write bytes is too small or scanner returns empty cells, the region status persists pending close.

Best regards,
Minwoo Kang

Re: Is there any way to check the system stop is requested in performCompaction over time?

Posted by Kang Minwoo <mi...@outlook.com>.

Additional information)
I am currently using hbase-1, but I am preparing to upgrade to hbase-2.

2020. 3. 17. 11:51, Kang Minwoo <mi...@outlook.com>> 작성:

Thank you for kindly reply.

I think the solution you gave me is really good.

I didn't know that before, so I took a different approach.
I made a new configuration to check that a system is stopped by time.
And I create a issue[1] and attach a patch[2].

I am not sure this is a proper approach.
I'd appreciate your advice.

[1]: https://issues.apache.org/jira/browse/HBASE-23968
[2]: https://github.com/apache/hbase/pull/1274/files

2020. 3. 17. 01:55, Stack <st...@duboce.net>> 작성:

Thanks for the detail below. Helped.

In your custom scanner, can you check RegionCoprocessorEnvironment on each next invocation? RCE gives you access to the hosting RegionServer via

/** @return reference to the region server services */
RegionServerServices getRegionServerServices();

You can ask RegionServerServices... if it is stopping [2].

(This facility is removed in hbase2 it seems but is available to you in hbase1).

2. https://github.com/saintstack/hbase/blob/branch-1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java#L51

On Sun, Mar 1, 2020 at 8:49 PM Kang Minwoo <mi...@outlook.com>> wrote:
HBase version is 1.2.9
The region was closed after a few minutes of about 1min.

I use a coprocessor that implements BaseRegionObserver.
The purpose is for deleting data with an expired retention period.

My coprocessor return Custom scanner in BaseRegionObserver.preCompact and when scan type is COMPACT_DROP_DELETES.
The custom scanner does only check row key (row key contain time) and if row key range is expired retention period, scanner returns empty List<Cell> result.

While the region does performCompaction[1], the region got a request that is a close region. writesEnabled is changed false. and then performCompaction is stopped only if writing bytes is over hbase.hstore.close.check.interval config[2].

In my case, there are too many expired retention period data. therefore the scanner returns nothing. so writing bytes is zero. It occurs pending close.

The reason why I do not use a TTL. Because the retention period changed dynamically by users.

[1]: https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java#L367
[2]: https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java#L420-L429

Best regards,
Minwoo Kang

________________________________________
보낸 사람: Stack <st...@duboce.net>>
보낸 날짜: 2020년 2월 29일 토요일 02:07
받는 사람: Hbase-User
제목: Re: Is there any way to check the system stop is requested in performCompaction over time?