You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by Charles Yeh <ch...@eactiv.com> on 2015/11/04 07:24:39 UTC

Getting new metrics into /api/v1

Hello,

I'm trying to get maxCores and memoryPerExecutorMB into /api/v1 for this
ticket: https://issues.apache.org/jira/browse/SPARK-10565

I can't figure out which *getApplicationInfoList *is used by
*ApiRootResource.scala.
*It's attached in SparkUI but SparkUI's doesn't have start / end times
and /api/v1/applications does.

It looks like:

   - *MasterWebUI.scala* has these fields, since it has the applications
   themselves
   - *HistoryServer.scala *doesn't have these fields, since it infers them
   from logs
   - *SparkUI.scala *looks like a mock since it doesn't have end time /
   user / attempt id either

Re: Getting new metrics into /api/v1

Posted by Charles Yeh <ch...@eactiv.com>.

Never mind, I didn't realize building the core module separately doesn't
update the main jar that the scripts use.

Will look into making the history server consistent.

On Tue, Nov 3, 2015 at 10:24 PM, Charles Yeh <ch...@eactiv.com> wrote:

> Hello,
>
> I'm trying to get maxCores and memoryPerExecutorMB into /api/v1 for this
> ticket: https://issues.apache.org/jira/browse/SPARK-10565
>
> I can't figure out which *getApplicationInfoList *is used by *ApiRootResource.scala.
> *It's attached in SparkUI but SparkUI's doesn't have start / end times
> and /api/v1/applications does.
>
> It looks like:
>
>    - *MasterWebUI.scala* has these fields, since it has the applications
>    themselves
>    - *HistoryServer.scala *doesn't have these fields, since it infers
>    them from logs
>    - *SparkUI.scala *looks like a mock since it doesn't have end time /
>    user / attempt id either
>
>
>