Re: Spark metrics when running with YARN?

[Vladimir Tretyakov <vl...@sematext.com>]

Hello, thx information about Spark API.

Could somebody explain one thing to me please?

On Spark UI I see 2 running applications ..0007 and ...0008.

But API returns only 1 (...0007).

Look at screenshots. Maybe I have to add something to API request? Or maybe
missed something here?

Thx.

On Tue, Aug 30, 2016 at 3:52 PM, Vijay Kiran <ma...@vijaykiran.com> wrote:

> Hi Otis,
>
> Did you check the REST API as documented in  http://spark.apache.org/docs/
> latest/monitoring.html
>
> Regards,
> Vijay
>
> > On 30 Aug 2016, at 14:43, Otis Gospodnetić <ot...@gmail.com>
> wrote:
> >
> > Hi Mich and Vijay,
> >
> > Thanks!  I forgot to include an important bit - I'm looking for a
> programmatic way to get Spark metrics when running Spark under YARN - so
> JMX or API of some kind.
> >
> > Thanks,
> > Otis
> > --
> > Monitoring - Log Management - Alerting - Anomaly Detection
> > Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> >
> >
> > On Tue, Aug 30, 2016 at 6:59 AM, Mich Talebzadeh <
> mich.talebzadeh@gmail.com> wrote:
> > Spark UI regardless of deployment mode Standalone, yarn etc runs on port
> 4040 by default that can be accessed directly
> >
> > Otherwise one can specify a specific port with --conf
> "spark.ui.port=55555" for example 55555
> >
> > HTH
> >
> > Dr Mich Talebzadeh
> >
> > LinkedIn  https://www.linkedin.com/profile/view?id=
> AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> >
> > http://talebzadehmich.wordpress.com
> >
> > Disclaimer: Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
> >
> >
> > On 30 August 2016 at 11:48, Vijay Kiran <ma...@vijaykiran.com> wrote:
> >
> > From Yarm RM UI, find the spark application Id, and in the application
> details, you can click on the “Tracking URL” which should give you the
> Spark UI.
> >
> > ./Vijay
> >
> > > On 30 Aug 2016, at 07:53, Otis Gospodnetić <ot...@gmail.com>
> wrote:
> > >
> > > Hi,
> > >
> > > When Spark is run on top of YARN, where/how can one get Spark metrics?
> > >
> > > Thanks,
> > > Otis
> > > --
> > > Monitoring - Log Management - Alerting - Anomaly Detection
> > > Solr & Elasticsearch Consulting Support Training -
> http://sematext.com/
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe e-mail: user-unsubscribe@spark.apache.org
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>