You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Wenchen Fan (Jira)" <ji...@apache.org> on 2019/10/18 08:08:00 UTC
[jira] [Resolved] (SPARK-29444) Add configuration to support
JacksonGenrator to keep fields with null values
[ https://issues.apache.org/jira/browse/SPARK-29444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wenchen Fan resolved SPARK-29444.
---------------------------------
Fix Version/s: 3.0.0
Resolution: Fixed
Issue resolved by pull request 26098
[https://github.com/apache/spark/pull/26098]
> Add configuration to support JacksonGenrator to keep fields with null values
> ----------------------------------------------------------------------------
>
> Key: SPARK-29444
> URL: https://issues.apache.org/jira/browse/SPARK-29444
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.0.0
> Reporter: Jackey Lee
> Priority: Major
> Fix For: 3.0.0
>
>
> DataSet.toJSON will lost some column when field data is null. Maybe it is better to keep null data in some scenarios.
> Such as sparkmagic, which is widely used in jupyter with livy, we use toJSON to toJSON is used to get sql results. with toJSON sparkmagic may return empty results, which confused users.
> Maybe adding a config is the best choice. This configuration retains the current semantics and will remain fields with null until the configuration is modified.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org