[jira] [Created] (SPARK-13953) Support for specifying the field name for corrupted record at JSON datasource.

["Hyukjin Kwon (JIRA)" <ji...@apache.org>]

Hyukjin Kwon created SPARK-13953:
------------------------------------

             Summary: Support for specifying the field name for corrupted record at JSON datasource.
                 Key: SPARK-13953
                 URL: https://issues.apache.org/jira/browse/SPARK-13953
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 2.0.0
            Reporter: Hyukjin Kwon
            Priority: Minor


 It would be great if we maybe make {{spark.sql.columnNameOfCorruptRecord}} as an option.

Wenchen said, https://github.com/apache/spark/pull/11756#issuecomment-197644079
{quote}
This is a good point, if we only use this config while dealing with json data...
{quote}

Reynold said, https://github.com/apache/spark/pull/11756#issuecomment-197645874
{quote}
...I think it should be an option on each read, not a global option. Most global options don't make a lot of sense as global options.
{quote}

{quote}
But since we had it, i'd say we should keep it to avoid breaking compatibility. We can have the per-read option override the global option.
{quote}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org