[jira] [Commented] (SPARK-23043) Upgrade json4s-jackson to 3.5.3

["Sean Owen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-23043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322059#comment-16322059 ] 

Sean Owen commented on SPARK-23043:
-----------------------------------

This is good info, but as you say, there are problems in updating it. You've cataloged several of them. Have you solved them? otherwise I am not clear what's new here.

> Upgrade json4s-jackson to 3.5.3
> -------------------------------
>
>                 Key: SPARK-23043
>                 URL: https://issues.apache.org/jira/browse/SPARK-23043
>             Project: Spark
>          Issue Type: Improvement
>          Components: Build
>    Affects Versions: 2.2.1
>            Reporter: Takako Shimamoto
>
> Spark tried to upgrade json4s several times, but failed and still use a few years old version 3.2.11. I hope the situation will improve soon.
> h3. Migration from 3.2.11
> Changes that affect Spark are as follows.
> h4. scalap dependency
> Since 3.3, json4s has its own scalap fork and doesn't depend on scala-lang's scalap.
> https://github.com/json4s/json4s/issues/108
> Whereas Spark has added a scala-lang's scalap dependency in POM by the following issue.
> https://github.com/apache/spark/pull/480
> We need to remove scala-lang's scalap dependency for json4s 3.5.3 (same solution as in json4s).
> h4. The behavior of \ function has changed
> Please see https://github.com/json4s/json4s/pull/309
> h5. a single element
> {code}
> val json =
>       """[ {
>         |  "jobId" : 0,
>         |  "status" : "SUCCEEDED"
>         |} ]""".stripMargin
> val ast = JsonMethods.parse(json)
> {code}
> * version 3.2.11
> {code}
> ast \ "status"  // => JString(SUCCEEDED)
> ast \\ "status" // => JString(SUCCEEDED)
> {code}
> * version 3.5.3
> {code}
> ast \ "status"  // => JArray(List(JString(SUCCEEDED)))
> ast \\ "status" // => JString(SUCCEEDED)
> {code}
> h5. more than a single elements
> {code}
> val json =
>   """[ {
>     |  "jobId" : 1,
>     |  "status" : "FAILED"
>     |}, {
>     |  "jobId" : 0,
>     |  "status" : "SUCCEEDED"
>     |} ]""".stripMargin
> val ast = JsonMethods.parse(json)
> {code}
> * version 3.2.11
> {code}
> ast \ "status"  // => JArray(List(JString(FAILED), JString(SUCCEEDED)))
> ast \\ "status" // => JObject(List((status,JString(FAILED)), (status,JString(SUCCEEDED))))
> {code}
> * version 3.5.3
> {code}
> ast \ "status"  // => JArray(List(JString(FAILED), JString(SUCCEEDED)))
> ast \\ "status" // => JObject(List((status,JString(FAILED)), (status,JString(SUCCEEDED))))
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org