[GitHub] [incubator-pinot] siddharthteotia edited a comment on issue #6431: Text Index lag time for realtime consuming segments

[GitBox <gi...@apache.org>]

siddharthteotia edited a comment on issue #6431:
URL: https://github.com/apache/incubator-pinot/issues/6431#issuecomment-770535834


   We can go through the logs tomorrow based on the new build that I gave with additional logging and debug further. I thought you wanted to reduce the lag from 3 mins to 30secs or so which the new build has done.
   
   Also, your comparison is not fair. If your functionality can be achieved with inverted index, then text index should not be used at all. Inverted index should always be the first choice..
   
   But on the log column in your schema, you can't really use inverted index for the kind of search you want to perform.
   
   Secondly, arbitrarily increasing the number of rows in consuming segment isn't a good idea either. If 100k is giving the optimal performance, my suggestion would be to stick with it. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org