Re: [PR] [SPARK-23890][SQL] Support CHANGE COLUMN to add nested fields to structs [spark]

["ottomata (via GitHub)" <gi...@apache.org>]

ottomata commented on PR #21012:
URL: https://github.com/apache/spark/pull/21012#issuecomment-1856336189

   Hi, I'd like to reopen this issue.  Back in 2019, @cloud-fan said that v2 datasource:
   > already support changing nested fields with command AlterTable 
   
   However, there is an [explicit check](https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala#L1441) and [test](https://github.com/apache/spark/blob/e3f46ed57dc063566cdb9425b4d5e02c65332df1/sql/core/src/test/scala/org/apache/spark/sql/connector/AlterTableTests.scala#L583) that prevents this from happening.
   
   We need to be able to evolve tables in a backwards compatible way.  This safe and possible with top level fields, and it should be safe and possible with struct column fields too, as long as we are only adding new fields.  
   
   cc @MaxGekk @beliefer @brkyvz 
   
   Thanks!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org