You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Gengliang Wang (JIRA)" <ji...@apache.org> on 2018/08/13 17:09:00 UTC

[jira] [Created] (SPARK-25104) Validate user specified output schema

Gengliang Wang created SPARK-25104:
--------------------------------------

             Summary: Validate user specified output schema
                 Key: SPARK-25104
                 URL: https://issues.apache.org/jira/browse/SPARK-25104
             Project: Spark
          Issue Type: Sub-task
          Components: SQL
    Affects Versions: 2.4.0
            Reporter: Gengliang Wang


With code changes in [https://github.com/apache/spark/pull/21847|https://github.com/apache/spark/pull/21847,] , Spark can write out data as per user provided output schema.

To make it more robust and user friendly, we should validate the Avro schema before tasks launched.

Also we should support output logical decimal type as BYTES (By default we output as FIXED)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org