[GitHub] [carbondata] MarvinLitt commented on a change in pull request #4189: [CARBONDATA-4263]support query with latestSegment

[GitBox <gi...@apache.org>]

MarvinLitt commented on a change in pull request #4189:
URL: https://github.com/apache/carbondata/pull/4189#discussion_r687310576



##########
File path: hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonTableInputFormat.java
##########
@@ -601,4 +599,60 @@ public String getSegmentIdFromFilePath(String filePath) {
     }
     return CarbonCommonConstants.INVALID_SEGMENT_ID;
   }
+
+  /**
+   * return valid segment to access
+   * first check for mapreduce.input.carboninputformat.segmentnumbers"
+   * second check for table property of latest_segment for query
+   */
+  public Segment[] getSegmentsToAccess(JobContext job, ReadCommittedScope readCommittedScope,
+                                       List<Segment> validSegments) {
+    String segmentString = job.getConfiguration().get(INPUT_SEGMENT_NUMBERS, "");

Review comment:
       the old getSegmentsToAccess fun just use INPUT_SEGMENT_NUMBERS for input to get the segment List.
   But now we need get segment not just INPUT_SEGMENT_NUMBERS  but alse latest segment. the validSegments is need to use.
   if use  getSegmentsToAccess(JobContext job, ReadCommittedScope readCommittedScope) we need to analysis readCommittedScope to validSegments  that the external functions have been implemented.
   so i choose func overload to do this function.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org