[GitHub] [incubator-pinot] pabrahamusa commented on issue #6431: Text Index lag time for realtime consuming segments

[GitBox <gi...@apache.org>]

pabrahamusa commented on issue #6431:
URL: https://github.com/apache/incubator-pinot/issues/6431#issuecomment-766467559


   @siddharthteotia 
   I dont know much abt Pinot code , However it seems like
   MutableSegmentImpl.addNewRow() calls PhysicalColumnIndexContainer._xxxxxIndex to fetch the Index for all type of indices.
   PhysicalColumnIndexContainer._invertedIndex is pulled from PinotDataBuffer (segmentReader.getIndexFor()) which I assume is memory or Consuming segments and it is same for _fstIndex, _rangeIndex, _h3Index, _jsonIndex etc..
   But for _textIndex it is loading from segmentIndexDir which is Physical file or I guess Completed Segments.
   So we might have to create a new constructor for LuceneTextIndexReader
    new LuceneTextIndexReader(segmentReader.getIndexFor(columnName, ColumnIndexType.TEXT_INDEX));
    to read from Buffer ? Do you think this make any sense? or am I completely talking gibberish?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org