[jira] [Commented] (SPARK-38651) Writing out empty or nested empty schemas in Datasource should be configurable

["Apache Spark (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-38651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17512202#comment-17512202 ] 

Apache Spark commented on SPARK-38651:
--------------------------------------

User 'thejdeep' has created a pull request for this issue:
https://github.com/apache/spark/pull/35969

> Writing out empty or nested empty schemas in Datasource should be configurable
> ------------------------------------------------------------------------------
>
>                 Key: SPARK-38651
>                 URL: https://issues.apache.org/jira/browse/SPARK-38651
>             Project: Spark
>          Issue Type: Task
>          Components: SQL
>    Affects Versions: 2.4.0
>            Reporter: Thejdeep Gudivada
>            Priority: Major
>
> In SPARK-23372, we introduced a backwards incompatible change that would remove support for writing out empty or nested empty schemas in file based datasources. This introduces backward incompatibility for users who have been using a schema that met the above condition since the datasource supported it. Except for Parquet and text, other file based sources support this behavior.
>  
> We should either :
>  * Make it configurable to enable/disable writing out empty schemas
>  * Enable the validation check only for sources that do not support it - Parquet / Text



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org