[GitHub] [spark] thejdeep commented on pull request #35969: [SPARK-38651][SQL] Add configuration to support writing out empty schemas in supported filebased datasources

[GitBox <gi...@apache.org>]

thejdeep commented on PR #35969:
URL: https://github.com/apache/spark/pull/35969#issuecomment-1298735509

   @cloud-fan We have had users writing data with empty schemas in production and changing the schema of a non-trivial number of rows seems like a big change. Spark allows creating empty schemas ad supports reading them, it might be fitting to consider writing out empty schemas. This PR adds a legacy configuration so that users can choose to ignore the validation check to restore old behavior. What are your thoughts on this ? Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org