You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Reynold Xin (JIRA)" <ji...@apache.org> on 2015/11/17 04:44:11 UTC
[jira] [Comment Edited] (SPARK-11753) Understand why
allowNonNumericNumbers JSON option doesn't work
[ https://issues.apache.org/jira/browse/SPARK-11753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15007958#comment-15007958 ]
Reynold Xin edited comment on SPARK-11753 at 11/17/15 3:43 AM:
---------------------------------------------------------------
I don't think this is urgent, so take your time. Note that we also have some custom code to handle NaN in the json parser ourselves. Ideally we should remove those code and just let Jackson handle them.
https://github.com/apache/spark/blob/540bf58f18328c68107d6c616ffd70f3a4640054/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JacksonParser.scala#L102
was (Author: rxin):
I don't think this is urgent, so take your time. Note that we also have some custom code to handle NaN in the json parser ourselves. Ideally we should remove those code and just let Jackson handle them.
> Understand why allowNonNumericNumbers JSON option doesn't work
> --------------------------------------------------------------
>
> Key: SPARK-11753
> URL: https://issues.apache.org/jira/browse/SPARK-11753
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.6.0
> Reporter: Reynold Xin
>
> In SPARK-11745, I introduced an undocumented option allowNonNumericNumbers. According to Jackson's documentation, this option should support converting "NaN", "INF", "-INF" etc into floating point numbers. However, the 2 test cases in JsonParsingOptionsSuite.scala were not working and as a result were ignored.
> The goal of this task is to figure out why they were not working, and re-enable them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org