[GitHub] [iceberg] rdblue commented on a change in pull request #2691: fix: add and remove partition transform on same column failed when use v1 metadata

[GitBox <gi...@apache.org>]

rdblue commented on a change in pull request #2691:
URL: https://github.com/apache/iceberg/pull/2691#discussion_r650442088



##########
File path: core/src/main/java/org/apache/iceberg/BaseUpdatePartitionSpec.java
##########
@@ -223,7 +223,9 @@ public PartitionSpec apply() {
         // field IDs were not required for v1 and were assigned sequentially in each partition spec starting at 1,000.
         // to maintain consistent field ids across partition specs in v1 tables, any partition field that is removed
         // must be replaced with a null transform. null values are always allowed in partition data.
-        builder.add(field.sourceId(), field.fieldId(), field.name(), Transforms.alwaysNull());
+        // To avoid name conflict when add and remove same partition transform multiple times, field name will be
+        // replaced by field name append with field id.
+        builder.add(field.sourceId(), field.fieldId(), field.name() + "_" + field.fieldId(), Transforms.alwaysNull());

Review comment:
       Why do this instead of just renaming the field when a conflict is detected later? That seems like a better way to fix it to me.

##########
File path: core/src/main/java/org/apache/iceberg/BaseUpdatePartitionSpec.java
##########
@@ -290,7 +292,9 @@ private void checkForRedundantAddedPartitions(PartitionField field) {
     ImmutableMap.Builder<Pair<Integer, String>, PartitionField> builder = ImmutableMap.builder();
     List<PartitionField> fields = spec.fields();
     for (PartitionField field : fields) {
-      builder.put(Pair.of(field.sourceId(), field.transform().toString()), field);
+      if (!field.transform().isVoid()) {

Review comment:
       Is this needed?
   
   The map created by this method is only used in `addField` and `removeField`. My guess is that your intent is to avoid blocking multiple void transforms with the same source id because of multiple calls to remove. But the `apply` method works with the builder directly and so this map doesn't need to change if I'm reading the existing code correctly. What is the case where this causes a failure?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org