[GitHub] [incubator-pinot] yupeng9 commented on a change in pull request #6400: Add real-time H3 index reader

[GitBox <gi...@apache.org>]

yupeng9 commented on a change in pull request #6400:
URL: https://github.com/apache/incubator-pinot/pull/6400#discussion_r550962098



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/realtime/impl/geospatial/RealtimeH3IndexReader.java
##########
@@ -0,0 +1,63 @@
+package org.apache.pinot.core.realtime.impl.geospatial;
+
+import com.uber.h3core.H3Core;
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+import org.apache.pinot.core.realtime.impl.ThreadSafeMutableRoaringBitmap;
+import org.apache.pinot.core.segment.creator.GeoSpatialIndexCreator;
+import org.apache.pinot.core.segment.creator.impl.geospatial.H3IndexResolution;
+import org.apache.pinot.core.segment.index.readers.H3IndexReader;
+import org.roaringbitmap.buffer.ImmutableRoaringBitmap;
+import org.roaringbitmap.buffer.MutableRoaringBitmap;
+
+
+/**
+ * A H3 index reader for the real-time H3 index values on the fly.
+ * <p>This class is thread-safe for single writer multiple readers.
+ */
+public class RealtimeH3IndexReader implements GeoSpatialIndexCreator, H3IndexReader {
+  private final H3Core _h3Core;
+  private final Map<Long, ThreadSafeMutableRoaringBitmap> _h3IndexMap = new ConcurrentHashMap<>();

Review comment:
       there is a tradeoff of memory usage?

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/realtime/impl/geospatial/RealtimeH3IndexReader.java
##########
@@ -0,0 +1,63 @@
+package org.apache.pinot.core.realtime.impl.geospatial;
+
+import com.uber.h3core.H3Core;
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+import org.apache.pinot.core.realtime.impl.ThreadSafeMutableRoaringBitmap;
+import org.apache.pinot.core.segment.creator.GeoSpatialIndexCreator;
+import org.apache.pinot.core.segment.creator.impl.geospatial.H3IndexResolution;
+import org.apache.pinot.core.segment.index.readers.H3IndexReader;
+import org.roaringbitmap.buffer.ImmutableRoaringBitmap;
+import org.roaringbitmap.buffer.MutableRoaringBitmap;
+
+
+/**
+ * A H3 index reader for the real-time H3 index values on the fly.
+ * <p>This class is thread-safe for single writer multiple readers.
+ */
+public class RealtimeH3IndexReader implements GeoSpatialIndexCreator, H3IndexReader {
+  private final H3Core _h3Core;
+  private final Map<Long, ThreadSafeMutableRoaringBitmap> _h3IndexMap = new ConcurrentHashMap<>();
+  private final H3IndexResolution _resolution;
+  private int _lowestResolution;
+
+  public RealtimeH3IndexReader(List<Integer> resolutions)

Review comment:
       done

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/index/column/ColumnIndexContainer.java
##########
@@ -51,7 +51,7 @@
   /**
    * Returns the H3 index for the column, or {@code null} if it does not exist.
    */
-  H3IndexReader getH3Index();
+  ImmutableH3IndexReader getH3Index();

Review comment:
       H3IndexReader is changed to interface




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org