[GitHub] [incubator-pinot] kishoreg commented on issue #4218: Add RealtimeConsumptionCatchupServiceCallback

[GitBox <gi...@apache.org>]

kishoreg commented on issue #4218: Add RealtimeConsumptionCatchupServiceCallback
URL: https://github.com/apache/incubator-pinot/pull/4218#issuecomment-494096953
 
 
   @npawar  I have a few questions. 
   
   What is the impact on the latency and when does it kick in and for what kind of use cases? 
   How will the user know whats the right wait-time? We might still see the same issue if the server has not caught up within the wait time right?
   
   One of the concerns here is that we stop the entire server from serving any other segment.
   
   Looks like we need two things
   - limit the number of threads for consumption so that it does not hog all the cpu during bootstrap/server startup
   -  Have the logic in the state transition for OFFLINE -> CONSUMING to not ack Helix until it's caught up. This has the added benefit that query results are up to date across all replicas.
   
   I am ok with the temporary fix but was wondering what would be the right fix that requires minimal configuration from the user.
   
   
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org