[jira] [Created] (SPARK-13241) add long--formatted timestamps to org.apache.spark.status.api.v1.ApplicationAttemptInfo

["Steve Loughran (JIRA)" <ji...@apache.org>]

Steve Loughran created SPARK-13241:
--------------------------------------

             Summary: add long--formatted timestamps to org.apache.spark.status.api.v1.ApplicationAttemptInfo
                 Key: SPARK-13241
                 URL: https://issues.apache.org/jira/browse/SPARK-13241
             Project: Spark
          Issue Type: Improvement
    Affects Versions: 1.6.0
            Reporter: Steve Loughran


While writing tests to parse {{org.apache.spark.status.api.v1.ApplicationAttemptInfo}} coming off the history rest server, I can see that I can't actually unmarshall the timestamps without parsing the strings —and I fear that may be local specific.

The problem here is that jersey is marshalling {{Date}} classes by calling {{Date.toString()}}, leaving an entry like
{code}
  {
    "id": "application_1111_0000",
    "name": "spark-demo",
    "attempts": [
      {
        "attemptId": "1",
        "startTime": "2016-02-08T20:12:20.825GMT",
        "endTime": "1970-01-01T00:00:00.000GMT",
        "lastUpdated": "2016-02-08T20:12:20.848GMT",
        "duration": 0,
        "sparkUser": "hdfs",
        "completed": false
      }
    ]
  }
{code}

This is good for humans and the web UI, bad for any code trying to use the API for its own purposes.

I propose adding, alongside the existing values, a simple {{long}} value for each time, representing the time in millis from the start of the epoch. This is trivial to marshall/unmarshall.

This is easy to add, provided everyone is happy that adding a new field to the returned JSON is considered compatible with the existing REST API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org