You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "yihua (via GitHub)" <gi...@apache.org> on 2024/10/15 08:17:24 UTC
Re: [PR] [HUDI-8328] fix fileId of INSERT with NBCC [hudi]
yihua commented on code in PR #12101:
URL: https://github.com/apache/hudi/pull/12101#discussion_r1800675336
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/SparkBucketIndexPartitioner.java:
##########
@@ -130,6 +131,9 @@ public BucketInfo getBucketInfo(int bucketNumber) {
// Always write into log file instead of base file if using NB-CC
BucketType bucketType = isNonBlockingConcurrencyControl ? BucketType.UPDATE : BucketType.INSERT;
String fileIdPrefix = BucketIdentifier.newBucketFileIdPrefix(bucketId, isNonBlockingConcurrencyControl);
+ if (isNonBlockingConcurrencyControl) {
+ fileIdPrefix = FSUtils.createNewFileId(fileIdPrefix, 0);
+ }
Review Comment:
@danny0405 I have another question. Does this mean that the file ID change between single writer and NBCC? Would that cause issues on the same table if the user switches the concurrency control mechanism?
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/SparkBucketIndexPartitioner.java:
##########
@@ -130,6 +131,9 @@ public BucketInfo getBucketInfo(int bucketNumber) {
// Always write into log file instead of base file if using NB-CC
BucketType bucketType = isNonBlockingConcurrencyControl ? BucketType.UPDATE : BucketType.INSERT;
String fileIdPrefix = BucketIdentifier.newBucketFileIdPrefix(bucketId, isNonBlockingConcurrencyControl);
+ if (isNonBlockingConcurrencyControl) {
+ fileIdPrefix = FSUtils.createNewFileId(fileIdPrefix, 0);
+ }
Review Comment:
Could this be combined with L133 since L133 also uses the flag `isNonBlockingConcurrencyControl`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org