You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (JIRA)" <ji...@apache.org> on 2019/07/16 16:42:13 UTC
[jira] [Updated] (SPARK-26376) Skip inputs without tokens by JSON
datasource
[ https://issues.apache.org/jira/browse/SPARK-26376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun updated SPARK-26376:
----------------------------------
Affects Version/s: (was: 2.4.0)
3.0.0
> Skip inputs without tokens by JSON datasource
> ---------------------------------------------
>
> Key: SPARK-26376
> URL: https://issues.apache.org/jira/browse/SPARK-26376
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.0.0
> Reporter: Maxim Gekk
> Priority: Minor
>
> The changes https://github.com/apache/spark/commit/38628dd1b8298d2686e5d00de17c461c70db99a8 can potentially break existing application if it doesn't expect a bad record for string without any JSON tokens in the PERMISSIVE mode. This ticket aims to return previous behaviour of JSON datasource and ignore such strings (including empty strings). The from_json function should keep new behaviour and produce bad records for empty strings and strings without any JSON tokens.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org