[jira] [Created] (SPARK-35374) Add string-to-number conversion support to JacksonParser

["Kousuke Saruta (Jira)" <ji...@apache.org>]

Kousuke Saruta created SPARK-35374:
--------------------------------------

             Summary: Add string-to-number conversion support to JacksonParser
                 Key: SPARK-35374
                 URL: https://issues.apache.org/jira/browse/SPARK-35374
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.1.1, 2.4.7, 3.2.0
            Reporter: Kousuke Saruta
            Assignee: Kousuke Saruta


In the current implementation, spark.read.json doesn't convert numbers represented as string even though types are specified by schema.
Here is anexample.
{code}
$ cat test.json
{ "value": "foo" }                                                                                                                                
{ "value": "100" }                                                                                                                                
{ "value": "257" }                                                                                                                                
{ "value": "32768" }                                                                                                                              
{ "value": "2147483648" }                                                                                                                         
{ "value": "2.71f" }                                                                                                                              
{ "value": "1.0E100" }  

$ bin/spark-shell
import org.apache.spark.sql.types._
val df = spark.read.
  schema(StructType(StructField("value", DoubleType)::Nil)).json("test.json")
df.show
+-----+
|value|
+-----+
| null|
| null|
| null|
| null|
| null|
| null|
| null|
+-----+
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org