[GitHub] [carbondata] ravipesala commented on a change in pull request #3216: [CARBONDATA-3387] Support Partition with MV datamap & Show DataMap Status

[GitBox <gi...@apache.org>]

ravipesala commented on a change in pull request #3216: [CARBONDATA-3387] Support Partition with MV datamap & Show DataMap Status
URL: https://github.com/apache/carbondata/pull/3216#discussion_r285649323
 
 

 ##########
 File path: core/src/main/java/org/apache/carbondata/core/datamap/DataMapProvider.java
 ##########
 @@ -264,23 +264,52 @@ private boolean getSpecificSegmentsTobeLoaded(Map<String, List<String>> segmentM
     } else {
       for (RelationIdentifier relationIdentifier : relationIdentifiers) {
         List<String> dataMapTableSegmentList = new ArrayList<>();
+        // Get all segments for parent relationIdentifier
+        List<String> mainTableSegmentList =
+            DataMapUtil.getMainTableValidSegmentList(relationIdentifier);
+        boolean ifTableStatusUpdateRequired = false;
         for (LoadMetadataDetails loadMetaDetail : listOfLoadFolderDetails) {
           if (loadMetaDetail.getSegmentStatus() == SegmentStatus.SUCCESS
               || loadMetaDetail.getSegmentStatus() == SegmentStatus.INSERT_IN_PROGRESS) {
             Map<String, List<String>> segmentMaps =
                 DataMapSegmentStatusUtil.getSegmentMap(loadMetaDetail.getExtraInfo());
+            for (String segmentId : mainTableSegmentList) {
+              // In case if dataMap segment(0) is mapped to mainTable segments{0,1,2} and if
+              // {0,1,2} segments of mainTable are compacted to 0.1. Then,
+              // on next rebuild/load to dataMap, no need to load segment(0.1) again. Update the
+              // segmentMapping of dataMap segment from {0,1,2} to {0.1}
+              if (!checkIfSegmentsToBeReloaded(relationIdentifier, segmentMaps.get(
 
 Review comment:
   Reading the tablestatus every segment inside `checkIfSegmentsToBeReloaded`, please read once and pass from outside.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services