[jira] [Updated] (LIVY-601) Add support for user defined date format and timezone when transform spark result rows to json

["wyp (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/LIVY-601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

wyp updated LIVY-601:
---------------------
    Summary: Add support for user defined date format and timezone when transform spark result rows to json  (was: Add support for user defined date format and timezone when transformspark result rows to json)

> Add support for user defined date format and timezone when transform spark result rows to json
> ----------------------------------------------------------------------------------------------
>
>                 Key: LIVY-601
>                 URL: https://issues.apache.org/jira/browse/LIVY-601
>             Project: Livy
>          Issue Type: Improvement
>          Components: REPL
>    Affects Versions: 0.6.0
>            Reporter: wyp
>            Priority: Minor
>
> When Livy get the row data from Spark DataFrame and transform those rows to json, we use DefaultFormats to transform Spark timestamp to scala timestamp which use default timezone.
> {code:java}
> private implicit def formats = DefaultFormats
> val result = spark.sql(code)
> val schema = parse(result.schema.json)
> // Get the row data
> val rows = result.take(maxResult)
>   .map {
>     _.toSeq.map {
>       // Convert java BigDecimal type to Scala BigDecimal, because current version of
>       // Json4s doesn't support java BigDecimal as a primitive type (LIVY-455).
>       case i: java.math.BigDecimal => BigDecimal(i)
>       case e => e
>     }
>   }
> val jRows = Extraction.decompose(rows)
> // DefaultFormats defined in org.json4s.DefaultFormats
> /** Default date format is UTC time.
>  */
> object DefaultFormats extends DefaultFormats {
>   val UTC = TimeZone.getTimeZone("UTC")
>   val losslessDate = {
>     def createSdf = {
>       val f = new java.text.SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'")
>       f.setTimeZone(UTC)
>       f
>     }
>     new ThreadLocal(createSdf)
>   }
> }{code}
> It would be useful if we can customize the time zone and time format



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)