[jira] [Commented] (SPARK-5823) Reuse mutable rows for inner structures when parsing JSON objects

["Yanbo Liang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14321792#comment-14321792 ] 

Yanbo Liang commented on SPARK-5823:
------------------------------------

Hi [~yhuai]
Actually, I have implemented reusing the mutable row for inner structures at  https://issues.apache.org/jira/browse/SPARK-5738.
However I have found that you have mentioned it will extend Spark SQL's JSON support to handle the case where each object in the dataset might have considerably different schema. In this scenario, the inner nested mutable row strategy will not take much performance improvements, am I right? 

> Reuse mutable rows for inner structures when parsing JSON objects
> -----------------------------------------------------------------
>
>                 Key: SPARK-5823
>                 URL: https://issues.apache.org/jira/browse/SPARK-5823
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>            Reporter: Yin Huai
>
> With SPARK-5738, we will reuse a mutable row for rows when parsing JSON objects. We can do the same thing for inner structures.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org