You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "Jackie-Jiang (via GitHub)" <gi...@apache.org> on 2023/06/09 06:08:31 UTC
[GitHub] [pinot] Jackie-Jiang opened a new pull request, #10877: Remove the validation of ReplicaGroupStrategyConfig and ReplicaGroupPartitionConfig cannot co-exist
Jackie-Jiang opened a new pull request, #10877:
URL: https://github.com/apache/pinot/pull/10877
This validation is backward incompatible. We used to get the partition column name from the `ReplicaGroupStrategyConfig`, which can co-exist with `ReplicaGroupPartitionConfig`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang closed pull request #10877: Remove the validation of ReplicaGroupStrategyConfig and ReplicaGroupPartitionConfig cannot co-exist
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang closed pull request #10877: Remove the validation of ReplicaGroupStrategyConfig and ReplicaGroupPartitionConfig cannot co-exist
URL: https://github.com/apache/pinot/pull/10877
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10877: Remove the validation of ReplicaGroupStrategyConfig and ReplicaGroupPartitionConfig cannot co-exist
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #10877:
URL: https://github.com/apache/pinot/pull/10877#discussion_r1224630996
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/TableConfigUtils.java:
##########
@@ -607,24 +606,6 @@ static void validateInstancePartitionsTypeMapConfig(TableConfig tableConfig) {
}
}
- /**
- * Detects whether both replicaGroupStrategyConfig and replicaGroupPartitionConfig are set for a given
- * table. Validation fails because the table would ignore replicaGroupStrategyConfig
- * when the replicaGroupPartitionConfig is already set.
- */
- @VisibleForTesting
- static void validatePartitionedReplicaGroupInstance(TableConfig tableConfig) {
- if (tableConfig.getValidationConfig().getReplicaGroupStrategyConfig() == null
- || MapUtils.isEmpty(tableConfig.getInstanceAssignmentConfigMap())) {
- return;
- }
- for (Map.Entry<String, InstanceAssignmentConfig> entry: tableConfig.getInstanceAssignmentConfigMap().entrySet()) {
- boolean isNullReplicaGroupPartitionConfig = entry.getValue().getReplicaGroupPartitionConfig() == null;
- Preconditions.checkState(isNullReplicaGroupPartitionConfig,
Review Comment:
After a second thought, I think we can probably keep it since the validation is applied only when a new table config is uploaded. I found this issue because one table config in the quick start fails this validation, but I can modify the table config to use the new config instead
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on pull request #10877: Remove the validation of ReplicaGroupStrategyConfig and ReplicaGroupPartitionConfig cannot co-exist
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on PR #10877:
URL: https://github.com/apache/pinot/pull/10877#issuecomment-1584026792
cc @abhioncbr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] abhioncbr commented on a diff in pull request #10877: Remove the validation of ReplicaGroupStrategyConfig and ReplicaGroupPartitionConfig cannot co-exist
Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on code in PR #10877:
URL: https://github.com/apache/pinot/pull/10877#discussion_r1224148162
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/TableConfigUtils.java:
##########
@@ -607,24 +606,6 @@ static void validateInstancePartitionsTypeMapConfig(TableConfig tableConfig) {
}
}
- /**
- * Detects whether both replicaGroupStrategyConfig and replicaGroupPartitionConfig are set for a given
- * table. Validation fails because the table would ignore replicaGroupStrategyConfig
- * when the replicaGroupPartitionConfig is already set.
- */
- @VisibleForTesting
- static void validatePartitionedReplicaGroupInstance(TableConfig tableConfig) {
- if (tableConfig.getValidationConfig().getReplicaGroupStrategyConfig() == null
- || MapUtils.isEmpty(tableConfig.getInstanceAssignmentConfigMap())) {
- return;
- }
- for (Map.Entry<String, InstanceAssignmentConfig> entry: tableConfig.getInstanceAssignmentConfigMap().entrySet()) {
- boolean isNullReplicaGroupPartitionConfig = entry.getValue().getReplicaGroupPartitionConfig() == null;
- Preconditions.checkState(isNullReplicaGroupPartitionConfig,
Review Comment:
I think we introduced the validation to handle the scenario like what would happen in-case provided `partitionColumn` value is different in `ReplicaGroupStrategyConfig` and `ReplicaGroupPartitionConfig`?
Looking at the validation logic I can see it's backward incompatible. Can we instead of removing it make sure that if both config has `partitionColumn` value then it should be same otherwise fail?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] abhioncbr commented on a diff in pull request #10877: Remove the validation of ReplicaGroupStrategyConfig and ReplicaGroupPartitionConfig cannot co-exist
Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on code in PR #10877:
URL: https://github.com/apache/pinot/pull/10877#discussion_r1224646834
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/TableConfigUtils.java:
##########
@@ -607,24 +606,6 @@ static void validateInstancePartitionsTypeMapConfig(TableConfig tableConfig) {
}
}
- /**
- * Detects whether both replicaGroupStrategyConfig and replicaGroupPartitionConfig are set for a given
- * table. Validation fails because the table would ignore replicaGroupStrategyConfig
- * when the replicaGroupPartitionConfig is already set.
- */
- @VisibleForTesting
- static void validatePartitionedReplicaGroupInstance(TableConfig tableConfig) {
- if (tableConfig.getValidationConfig().getReplicaGroupStrategyConfig() == null
- || MapUtils.isEmpty(tableConfig.getInstanceAssignmentConfigMap())) {
- return;
- }
- for (Map.Entry<String, InstanceAssignmentConfig> entry: tableConfig.getInstanceAssignmentConfigMap().entrySet()) {
- boolean isNullReplicaGroupPartitionConfig = entry.getValue().getReplicaGroupPartitionConfig() == null;
- Preconditions.checkState(isNullReplicaGroupPartitionConfig,
Review Comment:
IMO, validation failure is a legitimate backward incompatible and can possible happen in production system. I think we should fix it. If you want, I can make a change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org