You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@phoenix.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2019/04/19 06:22:00 UTC
[jira] [Commented] (PHOENIX-5137) Index Rebuilder scan increases
data table region split time
[ https://issues.apache.org/jira/browse/PHOENIX-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16821692#comment-16821692 ]
Hudson commented on PHOENIX-5137:
---------------------------------
FAILURE: Integrated in Jenkins build PreCommit-PHOENIX-Build #2516 (See [https://builds.apache.org/job/PreCommit-PHOENIX-Build/2516/])
PHOENIX-5137 check region close before commiting a batch for index (tdsilva: rev d9f6e969e66b99604a654a3ba51084b622723a74)
* (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/UngroupedAggregateRegionObserver.java
> Index Rebuilder scan increases data table region split time
> -----------------------------------------------------------
>
> Key: PHOENIX-5137
> URL: https://issues.apache.org/jira/browse/PHOENIX-5137
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.14.1
> Reporter: Kiran Kumar Maturi
> Assignee: Kiran Kumar Maturi
> Priority: Major
> Fix For: 4.15.0, 5.1.0, 4.14.2
>
> Attachments: PHOENIX-5137-4.14-HBase-1.3.02.patch, PHOENIX-5137-4.14-Hbase-1.3.01.patch, PHOENIX-5137-4.14-Hbase-1.3.01.patch, PHOENIX-5137-4.x-HBase-1.3.01.patch, PHOENIX-5137-master.01.patch
>
>
> [~lhofhansl] [~vincentpoon] [~tdsilva] please review
> In order to differentiate between the index rebuilder retries (UngroupedAggregateRegionObserver.rebuildIndices()) and commits that happen in the loop of UngroupedAggregateRegionObserver.doPostScannerOpen() as part of PHOENIX-4600 blockingMemstoreSize was set to -1 for rebuildIndices;
> {code:java}
> commitBatchWithRetries(region, mutations, -1);{code}
> blocks the region split as the check for region closing does not happen blockingMemstoreSize > 0
> {code:java}
> for (int i = 0; blockingMemstoreSize > 0 && region.getMemstoreSize() > blockingMemstoreSize && i < 30; i++) {
> try{
> checkForRegionClosing();
> ....
> {code}
> Plan is to have the check for region closing at least once before committing the batch
> {code:java}
> checkForRegionClosing();
> for (int i = 0; blockingMemstoreSize > 0 && region.getMemstoreSize() > blockingMemstoreSize && i < 30; i++) {
> try{
> checkForRegionClosing();
> ....
> {code}
> Steps to reproduce
> 1. Create a table with one index (startime)
> 2. Add 1-2 million rows
> 3. Wait till the index is active
> 4. Disable the index with start time (noted in step 1)
> 5. Once the rebuilder starts split data table region
> Repeat the steps again after applying the patch to check the difference.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)