[GitHub] [incubator-iceberg] rdblue commented on issue #745: schema evolution support

[GitBox <gi...@apache.org>]

rdblue commented on issue #745: schema evolution support
URL: https://github.com/apache/incubator-iceberg/pull/745#issuecomment-577428854
 
 
   From #741, it looks like the problem this is trying to address is that you can't write to a table with a different column order than the order of the table schema.
   
   The reason for this restriction is that Spark should be responsible for reconciling table columns with the data from a query. Spark has two different modes for doing this: by position for SQL, and by name for DataFrame writes. I think that delegating this to Spark is the right long-term solution.
   
   In the short term, Spark 2.4 has no resolution step for v2 writes. That's why Iceberg has the current checks that it does, so that you don't corrupt a table by writing the wrong thing. I'd rather not extend the current checks if we don't need to, given that this will be handled by Spark in the next version. Is that reasonable?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org