[GitHub] [pinot] mapshen edited a comment on issue #7192: Flush Segments on a Schedule

[GitBox <gi...@apache.org>]

mapshen edited a comment on issue #7192:
URL: https://github.com/apache/pinot/issues/7192#issuecomment-885879389


   > IIUC the problem here is during segment commit, there's no consumption. Instead of having a time scheduled for segment commit, we can find a way to start consuming for the new segment while the current segment is being committed.
   
   @sajjad-moradi It would be great!
   
   @mcvsubbu I did before but didn’t find it very helpful. I can try redoing it later. That said, how fast could it be after tuning it to the bone in your experience? Like Jackie put it earlier,
   
   > With different columns and indexes, the segment creation time could be very different. Normally it might take ~3 minutes to build the segment (of 5m rows).
   
   An ideal state for us would be a few seconds in the 99th percentile.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org