[GitHub] [iceberg] rdblue commented on pull request #2365: Spark: SQL extention to update partition field atomically

[GitBox <gi...@apache.org>]

rdblue commented on pull request #2365:
URL: https://github.com/apache/iceberg/pull/2365#issuecomment-806103048


   I considered adding similar syntax when I added the ADD and DROP DDL. I think the problem with this is that it implies that the original field is changed, rather than dropped and added. In format v2, we can sort of do that by replacing the field and using the previous field's name, but even then the values from the old field will no longer appear. In v1, a drop actually replaces the field with a `void` or `alwaysNull` transform and adds a new field to the end of the spec. It seems to me like that would be confusing behavior for `CHANGE`.
   
   One option to fix this is to use `REPLACE` instead, which I think doesn't imply that the field itself is still there, but updated or changed. But then the syntax is further away from column DDL so I'm not sure if that's a good idea. What do you think, @jackye1995, @aokolnychyi, @yyanyy?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org