[jira] [Created] (SPARK-25227) Extend functionality of to_json

["Yuriy Davygora (JIRA)" <ji...@apache.org>]

Yuriy Davygora created SPARK-25227:
--------------------------------------

             Summary: Extend functionality of to_json
                 Key: SPARK-25227
                 URL: https://issues.apache.org/jira/browse/SPARK-25227
             Project: Spark
          Issue Type: Improvement
          Components: PySpark, Spark Core
    Affects Versions: 2.3.1
            Reporter: Yuriy Davygora


At the moment, the 'to_json' function only supports a STRUCT or an ARRAY of STRUCTS as input. Support for ARRAY of primitives is, apparently, coming with Spark 2.4, but it will only support arrays of elements of same data type. It will not, for example, support JSON-arrays like

{noformat}
["string_value", 0, true, null]
{noformat}

which is JSON-valid with schema

{noformat}
{"containsNull":true,"elementType":["string","integer","boolean"],"type":"array"}
{noformat}

We would like to kindly ask you to add support for different-typed element arrays in the 'to_json' function. This will necessitate extending the functionality of ArrayType or maybe adding a new type (refer to [[SPARK-25225]])



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org