[GitHub] [pinot] mcvsubbu commented on pull request #7780: Make split-commit the default segment commit protocol

[GitBox <gi...@apache.org>]

mcvsubbu commented on pull request #7780:
URL: https://github.com/apache/pinot/pull/7780#issuecomment-982141873


   > > Why do we have this config on both controller and server?
   > > Seems the controller one is for allowing split commit from servers. Should we consider removing this controller config? Controller should always allow splitting commit, unless there are some overhead enabling it
   > > @mcvsubbu wdyt?
   > 
   > That makes sense. Controller should properly handles calls from split committers as well as non-split committers. It's up to server configuration to enable/disable split commit. Maybe there's a legacy reason for it. Let's wait for @mcvsubbu to shed light on this.
   
   I am not sure why we decided to have config both ways when it started. It may have been due to:
   - Upgrade order
   - We wanted finer control
   - Implementation order (I think we were implementing it in time for Uber to create the split commit handler for deep-store bypass)
   - Configuration of the vip address on the controller side. In LinkedIn, we don't use deep store, but the pushes go via the controller. The idea was to use a vip to do the segment push. So, when a realtime segment push comes along, the controller has no context of who is pushing the segment (at the time, we had no ACLs I think). The controller would then store an orphaned file. If the controller is configured to disable split commit, then it can reject the segment.
   - Other factors that I don't remember from 4 years ago.
   
   Now that we move it to be default, let us see if anyone needs to change it back (say, due to vip config). 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org