[GitHub] [iceberg] abmo-x commented on pull request #6301: Update Schema - should check if field is optional/required

[GitBox <gi...@apache.org>]

abmo-x commented on PR #6301:
URL: https://github.com/apache/iceberg/pull/6301#issuecomment-1329935940

   > @abmo-x, the behavior you're describing doesn't sound correct. If you union two schemas together, any field not in both schemas should be optional. The only case where a field in a union result would not be optional is if both schemas have it as required, right?
   
   I agree that any field not in both should be optional. what's your suggestion when a user makes a incompatible change to their Avro schema `v2` by adding a required field and uses `table.updateSchema().unionByNameWith(v2)` to updated the table schema. In this case the field gets added as optional to the table where as the Avro schema has it as required. 
   
   - should the update fail instead of adding the new required field as optional? 
   - should `unionByNameWith` not be used by and instead user should directly call `table.updateSchema().allowIncompatibleChanges().addRequiredColumn` by diffing the v1 vs v2 schema? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org