You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "priyen (via GitHub)" <gi...@apache.org> on 2023/04/13 20:56:06 UTC

[GitHub] [pinot] priyen opened a new issue, #10608: Clean up replicaGroupStrategyConfig / replicaGroupPartitionConfig

priyen opened a new issue, #10608:
URL: https://github.com/apache/pinot/issues/10608

   replicaGroupStrategyConfig is the old one and replicaGroupPartitionConfig is the new, but right now you need to maintain both because the new one doesn't have partitioning column but the old one does.
   It can easily lead to confusion why there is 2 configs.
   
   We should fully clean up and remove the old one


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang closed issue #10608: Clean up replicaGroupStrategyConfig / replicaGroupPartitionConfig

Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang closed issue #10608: Clean up replicaGroupStrategyConfig / replicaGroupPartitionConfig
URL: https://github.com/apache/pinot/issues/10608


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on issue #10608: Clean up replicaGroupStrategyConfig / replicaGroupPartitionConfig

Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #10608:
URL: https://github.com/apache/pinot/issues/10608#issuecomment-1518534630

   @abhioncbr The overall goal is to keep the existing config work if new config is not provided, and use the new config if provided. So if the `partitionColumn` is provided in both places, we can either throw an exception to let the user know the config is invalid (preferred, can be done within the `TableConfigUtils.validate()`), or just use the one from the new config.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on issue #10608: Clean up replicaGroupStrategyConfig / replicaGroupPartitionConfig

Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #10608:
URL: https://github.com/apache/pinot/issues/10608#issuecomment-1518338849

   We want to do it in a backward compatible way:
   - Add a new field `_partitionColumn` into the `InstanceReplicaGroupPartitionConfig`
   - If the field is not found, fall back to the old `ReplicaGroupStrategyConfig`
   - Keep the `ReplicaGroupStrategyConfig` and use it when the new config doesn't exist


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] abhioncbr commented on issue #10608: Clean up replicaGroupStrategyConfig / replicaGroupPartitionConfig

Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on issue #10608:
URL: https://github.com/apache/pinot/issues/10608#issuecomment-1518924124

   Please review the changes. Thanks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] GSharayu commented on issue #10608: Clean up replicaGroupStrategyConfig / replicaGroupPartitionConfig

Posted by "GSharayu (via GitHub)" <gi...@apache.org>.
GSharayu commented on issue #10608:
URL: https://github.com/apache/pinot/issues/10608#issuecomment-1518251952

   This is a good issue, but we should not deprecate the old flow since this will impact production systems. We should bring a plan  for migrations / compatibility  along these cleanups to align all the stakeholders before deprecating this fully.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] abhioncbr commented on issue #10608: Clean up replicaGroupStrategyConfig / replicaGroupPartitionConfig

Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on issue #10608:
URL: https://github.com/apache/pinot/issues/10608#issuecomment-1509936302

   I would like to try if no one is working on this. Thanks. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] abhioncbr commented on issue #10608: Clean up replicaGroupStrategyConfig / replicaGroupPartitionConfig

Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on issue #10608:
URL: https://github.com/apache/pinot/issues/10608#issuecomment-1518368717

   Thanks, @siddharthteotia,  @GSharayu and @Jackie-Jiang, for the comments.
   
   I'm just asking the question here that I have asked on the draft PR. 
   What if a user provides a different `partitionColumn` value in both configs? As per @Jackie-Jiang's comment, we will use the value of `partitionColumn` provided in `InstanceReplicaGroupPartitionConfig`. Is it correct? Also, for `numInstancesPerPartition`, it's going to be the same?
   
   I was going over the [documentation](https://docs.pinot.apache.org/operators/operating-pinot/instance-assignment), and I think we need to update the documentation by mentioning `ReplicaGroupStrategyConfig ` as deprecated.
   
   Please let me know if anything else needs to be updated. Thanks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] siddharthteotia commented on issue #10608: Clean up replicaGroupStrategyConfig / replicaGroupPartitionConfig

Posted by "siddharthteotia (via GitHub)" <gi...@apache.org>.
siddharthteotia commented on issue #10608:
URL: https://github.com/apache/pinot/issues/10608#issuecomment-1516978241

   This cleanup is backwards incompatible. I fully agree that we need to start using the new one and there is confusion / maintenance overhead etc but simply removing it right away is going to break things for people in production.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org