You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2019/05/21 04:34:57 UTC
[jira] [Resolved] (SPARK-16595) Spark History server Rest Api gives
Application not found error for yarn-cluster mode
[ https://issues.apache.org/jira/browse/SPARK-16595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-16595.
----------------------------------
Resolution: Incomplete
> Spark History server Rest Api gives Application not found error for yarn-cluster mode
> -------------------------------------------------------------------------------------
>
> Key: SPARK-16595
> URL: https://issues.apache.org/jira/browse/SPARK-16595
> Project: Spark
> Issue Type: Bug
> Affects Versions: 1.6.1
> Reporter: Yesha Vora
> Priority: Major
> Labels: bulk-closed
>
> Scenario:
> * Start SparkPi application in Spark1 using yarn-cluster mode (application_1468686376753_0041)
> * After application finishes validate application exists in respective Spark History server.
> {code}
> Error loading url http://xx.xx.xx.xx:18080/api/v1/applications/application_1468686376753_0041/1/executors
> HTTP Code: 404
> HTTP Data: no such app: application_1468686376753_0041{code}
> {code:title=spark HS log}
> 16/07/16 15:55:29 INFO FsHistoryProvider: Replaying log path: hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0049.inprogress
> 16/07/16 15:56:20 INFO FsHistoryProvider: Replaying log path: hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0049
> 16/07/16 16:23:14 INFO FsHistoryProvider: Replaying log path: hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0061.inprogress
> 16/07/16 16:24:14 INFO FsHistoryProvider: Replaying log path: hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0061
> 16/07/16 17:42:32 INFO FsHistoryProvider: Replaying log path: hdfs://xx.xx.xx.xx:8020/spark-history/local-1468690940553.inprogress
> 16/07/16 17:43:22 INFO FsHistoryProvider: Replaying log path: hdfs://xx.xx.xx.xx:8020/spark-history/local-1468690940553
> 16/07/16 17:43:44 INFO FsHistoryProvider: Replaying log path: hdfs://xx.xx.xx.xx:8020/spark-history/local-1468691017376.inprogress
> 16/07/16 17:44:34 INFO FsHistoryProvider: Replaying log path: hdfs://xx.xx.xx.xx:8020/spark-history/local-1468691017376
> 16/07/16 18:53:10 INFO FsHistoryProvider: Replaying log path: hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0041_1.inprogress
> 16/07/16 19:03:26 INFO PackagesResourceConfig: Scanning for root resource and provider classes in the packages:
> org.apache.spark.status.api.v1
> 16/07/16 19:03:35 INFO ScanningResourceConfig: Root resource classes found:
> class org.apache.spark.status.api.v1.ApiRootResource
> 16/07/16 19:03:35 INFO ScanningResourceConfig: Provider classes found:
> class org.apache.spark.status.api.v1.JacksonMessageWriter
> 16/07/16 19:03:35 INFO WebApplicationImpl: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM'
> 16/07/16 19:03:36 INFO SecurityManager: Changing view acls to: spark
> 16/07/16 19:03:36 INFO SecurityManager: Changing modify acls to: spark
> 16/07/16 19:03:36 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spark); users with modify permissions: Set(spark)
> 16/07/16 19:03:36 INFO ApplicationCache: Failed to load application attempt application_1468686376753_0041/Some(1)
> 16/07/16 19:04:21 INFO FsHistoryProvider: Replaying log path: hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043.inprogress
> 16/07/16 19:12:02 INFO FsHistoryProvider: Replaying log path: hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043
> 16/07/16 19:16:11 INFO SecurityManager: Changing view acls to: spark
> 16/07/16 19:16:11 INFO SecurityManager: Changing modify acls to: spark
> 16/07/16 19:16:11 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spark); users with modify permissions: Set(spark)
> 16/07/16 19:16:11 INFO FsHistoryProvider: Replaying log path: hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043
> 16/07/16 19:16:22 INFO SecurityManager: Changing acls enabled to: false
> 16/07/16 19:16:22 INFO SecurityManager: Changing admin acls to:
> 16/07/16 19:16:22 INFO SecurityManager: Changing view acls to: hrt_qa{code}
> {code}
> hdfs@xxx:/var/log/spark$ hdfs dfs -ls /spark-history/
> Found 8 items
> -rwxrwx--- 3 hrt_qa hadoop 28793 2016-07-16 15:56 /spark-history/application_1468678823755_0049
> -rwxrwx--- 3 hrt_qa hadoop 28763 2016-07-16 16:24 /spark-history/application_1468678823755_0061
> -rwxrwx--- 3 hrt_qa hadoop 58868885 2016-07-16 18:59 /spark-history/application_1468686376753_0041_1
> -rwxrwx--- 3 hrt_qa hadoop 58841982 2016-07-16 19:11 /spark-history/application_1468686376753_0043
> -rwxrwx--- 3 hive hadoop 5823 2016-07-16 11:38 /spark-history/local-1468666932940
> -rwxrwx--- 3 hive hadoop 5757 2016-07-16 22:44 /spark-history/local-1468669677840.inprogress
> -rwxrwx--- 3 hrt_qa hadoop 484113 2016-07-16 17:43 /spark-history/local-1468690940553
> -rwxrwx--- 3 hrt_qa hadoop 57747 2016-07-16 17:44 /spark-history/local-1468691017376
> hdfs@xxx:/var/log/spark$ hdfs dfs -ls /spark-history/application_1468686376753_0041_1
> -rwxrwx--- 3 hrt_qa hadoop 58868885 2016-07-16 18:59 /spark-history/application_1468686376753_0041_1{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org