You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@carbondata.apache.org by xu...@apache.org on 2020/06/23 07:18:08 UTC
[carbondata] branch master updated: [CARBONDATA-3860] Fix
IndexServer keeps loading some segments index repeatly
This is an automated email from the ASF dual-hosted git repository.
xubo245 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/carbondata.git
The following commit(s) were added to refs/heads/master by this push:
new 4145f6f [CARBONDATA-3860] Fix IndexServer keeps loading some segments index repeatly
4145f6f is described below
commit 4145f6f4d573c0da07d39fda4edd6704621824cc
Author: haomarch <ma...@126.com>
AuthorDate: Sat Jun 20 13:04:58 2020 +0800
[CARBONDATA-3860] Fix IndexServer keeps loading some segments index repeatly
Why is this PR needed?
In current getTableBlockIndexUniqueIdentifiers function. if the segmentBlockIndexInfo.getSegmentMetaDataInfo() is null, the IndexServer will keeps loading the index of this segment repeatly. We shall avoid to let it affect query performance, considering MetaDataInfo doesn't matter with the query processing.
What changes were proposed in this PR?
Return the cached index directly if segmentBlockIndexInfo.getTableBlockIndexUniqueIdentifiers() is not empty
Does this PR introduce any user interface change?
NO
Is any new testcase added?
NO
This closes #3795
---
.../core/indexstore/blockletindex/BlockletIndexFactory.java | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/core/src/main/java/org/apache/carbondata/core/indexstore/blockletindex/BlockletIndexFactory.java b/core/src/main/java/org/apache/carbondata/core/indexstore/blockletindex/BlockletIndexFactory.java
index c62b644..ef665c5 100644
--- a/core/src/main/java/org/apache/carbondata/core/indexstore/blockletindex/BlockletIndexFactory.java
+++ b/core/src/main/java/org/apache/carbondata/core/indexstore/blockletindex/BlockletIndexFactory.java
@@ -72,6 +72,7 @@ import org.apache.carbondata.core.util.CarbonProperties;
import org.apache.carbondata.core.util.path.CarbonTablePath;
import org.apache.carbondata.events.Event;
+import org.apache.commons.collections.CollectionUtils;
import org.apache.hadoop.fs.Path;
/**
@@ -352,9 +353,13 @@ public class BlockletIndexFactory extends CoarseGrainIndexFactory
throws IOException {
SegmentBlockIndexInfo segmentBlockIndexInfo = segmentMap.get(segment.getSegmentNo());
Set<TableBlockIndexUniqueIdentifier> tableBlockIndexUniqueIdentifiers = null;
- if (null != segmentBlockIndexInfo && null != segmentBlockIndexInfo.getSegmentMetaDataInfo()) {
- segment.setSegmentMetaDataInfo(
- segmentMap.get(segment.getSegmentNo()).getSegmentMetaDataInfo());
+ if (null != segmentBlockIndexInfo &&
+ CollectionUtils.isNotEmpty(
+ segmentBlockIndexInfo.getTableBlockIndexUniqueIdentifiers())) {
+ if (null != segmentBlockIndexInfo.getSegmentMetaDataInfo()) {
+ segment.setSegmentMetaDataInfo(
+ segmentMap.get(segment.getSegmentNo()).getSegmentMetaDataInfo());
+ }
return segmentBlockIndexInfo.getTableBlockIndexUniqueIdentifiers();
} else {
tableBlockIndexUniqueIdentifiers =