Redirect from yarn to spark history server

[satish saley <sa...@gmail.com>]

Hello,

I am running pyspark job using yarn-cluster mode. I can see spark job in
yarn but I am able to go from any "log history" link from yarn to spark
history server. How would I keep track of yarn log and its corresponding
log in spark history server? Is there any setting in yarn/spark that let me
redirect to spark history server from yarn?

Best,
Satish

Re: Redirect from yarn to spark history server

[Marcelo Vanzin <va...@cloudera.com>]

See http://spark.apache.org/docs/latest/running-on-yarn.html,
especially the parts that talk about
spark.yarn.historyServer.address.

On Mon, May 2, 2016 at 2:14 PM, satish saley <sa...@gmail.com> wrote:
>
>
> Hello,
>
> I am running pyspark job using yarn-cluster mode. I can see spark job in
> yarn but I am able to go from any "log history" link from yarn to spark
> history server. How would I keep track of yarn log and its corresponding log
> in spark history server? Is there any setting in yarn/spark that let me
> redirect to spark history server from yarn?
>
> Best,
> Satish



-- 
Marcelo

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org