[GitHub] [incubator-pinot] npawar opened a new issue #4192: Replica groups support for realtime

[GitBox <gi...@apache.org>]

npawar opened a new issue #4192: Replica groups support for realtime
URL: https://github.com/apache/incubator-pinot/issues/4192
 
 
   Background, motivation and details in: https://cwiki.apache.org/confluence/display/PINOT/Replica+groups+support+in+realtime
   
   **Phase 1** 
   Support for realtime
   
   Steps:
   1) Replica group sets to be created when table created/updated with replica group assignment strategy - use same path as for offline
   
   2)StreamPartitionAssignmentGenerator to start reading znode and constructing ReplicaGroup compliant assignment. We have a similar class for offline - ReplicaGroupSegmentAssignmentStrategy. Extract logic for creating replica group segment assignment out of both these classes. 
   
   3) Check RealtimeSegmentsValidationManager handles replica groups correctly. Currently we do a lot of recalculation of partition assignments and rebalance on the fly during segment completion and partitions change
   
   4) Rebalance - make sure ReplicaGroupRebalance can handle both offline as well as realtime tables
   
   5) Routing - combine PartitionAwareOffline and PartitionAwareRealtime to a common code for PartitionAwareRouting. Add config to control the partition pruning for realtime consuming segments.
   
   
   **Phase 2**
   Support for relocation and completed segments
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org