Re:

[Dirceu Semighini Filho <di...@gmail.com>]

Hi Anfemee,
Subject in the email sometimes help ;)
Have you seen if the link is sending you to a hostname that is not
accessible by your workstation? Sometimes changing the hostname to the ip
solve this kind of issue.



2015-10-15 13:34 GMT-03:00 Anfernee Xu <an...@gmail.com>:

> Sorry, I have to re-send it again as I did not get the answer.
>
> Here's the problem I'm facing, I have a standalone java application which
> is periodically submit Spark jobs to my yarn cluster, btw I'm not using
> 'spark-submit' or 'org.apache.spark.launcher' to submit my jobs. These jobs
> are successful and I can see them on Yarn RM webUI, but when I want to
> follow the link to the app history on Spark historyserver, I always got
> 404(application is not found) from Spark historyserver.
>
> My code looks likes as below
>
>
>     SparkConf conf = new
> SparkConf().setAppName("testSpak").setMaster("yarn-client")
>         .setJars(new String[]{IOUtil.getJar(MySparkApp.class)});
>
>     conf.set("spark.yarn.historyServer.address", "10.247.44.155:18080");
>     conf.set("spark.history.fs.logDirectory",
>         "
> hdfs://myHdfsNameNode:55310/scratch/tie/spark/applicationHistory");
>
>     JavaSparkContext sc = new JavaSparkContext(conf);
>
>     try {
>
>      ... my application code
>
>     }finally{
>       sc.stop();
>
>     }
>
> Anything I did wrong or missed? Do I need to configure something on Yarn
> side?
>
> Thanks
>
> --
> --Anfernee
>