[jira] [Created] (LIVY-601) Add support for user defined date format and timezone when transformspark result rows to json

["wyp (JIRA)" <ji...@apache.org>]

wyp created LIVY-601:
------------------------

             Summary: Add support for user defined date format and timezone when transformspark result rows to json
                 Key: LIVY-601
                 URL: https://issues.apache.org/jira/browse/LIVY-601
             Project: Livy
          Issue Type: Improvement
          Components: REPL
    Affects Versions: 0.6.0
            Reporter: wyp


When Livy get the row data from Spark DataFrame and transform those rows to json, we use DefaultFormats to transform Spark timestamp to scala timestamp which use default timezone.
{code:java}
private implicit def formats = DefaultFormats

val result = spark.sql(code)
val schema = parse(result.schema.json)

// Get the row data
val rows = result.take(maxResult)
  .map {
    _.toSeq.map {
      // Convert java BigDecimal type to Scala BigDecimal, because current version of
      // Json4s doesn't support java BigDecimal as a primitive type (LIVY-455).
      case i: java.math.BigDecimal => BigDecimal(i)
      case e => e
    }
  }

val jRows = Extraction.decompose(rows)

// DefaultFormats defined in org.json4s.DefaultFormats
/** Default date format is UTC time.
 */
object DefaultFormats extends DefaultFormats {
  val UTC = TimeZone.getTimeZone("UTC")

  val losslessDate = {
    def createSdf = {
      val f = new java.text.SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'")
      f.setTimeZone(UTC)
      f
    }
    new ThreadLocal(createSdf)
  }


}{code}
It would be useful if we can customize the time zone and time format



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)