You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/10/10 03:17:39 UTC
[GitHub] [hudi] danny0405 commented on a diff in pull request #6737: [HUDI-4373] Flink Consistent hashing bucket index write path code
danny0405 commented on code in PR #6737:
URL: https://github.com/apache/hudi/pull/6737#discussion_r990891454
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieClusteringConfig.java:
##########
@@ -624,23 +615,28 @@ private void validate() {
+ "schedule inline clustering (%s) can be enabled. Both can't be set to true at the same time. %s,%s", HoodieClusteringConfig.INLINE_CLUSTERING.key(),
HoodieClusteringConfig.SCHEDULE_INLINE_CLUSTERING.key(), inlineCluster, inlineClusterSchedule));
- // Consistent hashing bucket index
- if (clusteringConfig.contains(HoodieIndexConfig.INDEX_TYPE.key())
- && clusteringConfig.contains(HoodieIndexConfig.BUCKET_INDEX_ENGINE_TYPE.key())
- && clusteringConfig.getString(HoodieIndexConfig.INDEX_TYPE.key()).equalsIgnoreCase(HoodieIndex.IndexType.BUCKET.name())
- && clusteringConfig.getString(HoodieIndexConfig.BUCKET_INDEX_ENGINE_TYPE.key()).equalsIgnoreCase(HoodieIndex.BucketIndexEngineType.CONSISTENT_HASHING.name())) {
- ValidationUtils.checkArgument(clusteringConfig.getString(PLAN_STRATEGY_CLASS_NAME).equals(SPARK_CONSISTENT_BUCKET_CLUSTERING_PLAN_STRATEGY),
- "Consistent hashing bucket index only supports clustering plan strategy : " + SPARK_CONSISTENT_BUCKET_CLUSTERING_PLAN_STRATEGY);
- ValidationUtils.checkArgument(clusteringConfig.getString(EXECUTION_STRATEGY_CLASS_NAME).equals(SPARK_CONSISTENT_BUCKET_EXECUTION_STRATEGY),
+ if (isConsistentHashingBucketIndex()) {
+ String planStrategy = clusteringConfig.getString(PLAN_STRATEGY_CLASS_NAME);
+ ValidationUtils.checkArgument(planStrategy.equals(SPARK_CONSISTENT_BUCKET_CLUSTERING_PLAN_STRATEGY) || planStrategy.equals(FLINK_CONSISTENT_BUCKET_CLUSTERING_PLAN_STRATEGY),
+ "Consistent hashing bucket index only supports clustering plan strategy : " + SPARK_CONSISTENT_BUCKET_CLUSTERING_PLAN_STRATEGY + " | " + FLINK_CONSISTENT_BUCKET_CLUSTERING_PLAN_STRATEGY);
+ ValidationUtils.checkArgument(engineType == EngineType.FLINK || clusteringConfig.getString(EXECUTION_STRATEGY_CLASS_NAME).equals(SPARK_CONSISTENT_BUCKET_EXECUTION_STRATEGY),
"Consistent hashing bucket index only supports clustering execution strategy : " + SPARK_CONSISTENT_BUCKET_EXECUTION_STRATEGY);
Review Comment:
Is there any special logic for `EngineType.FLINK` type here ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org