You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by 남윤민 <ro...@dgist.ac.kr> on 2014/09/04 04:15:56 UTC
Why spark on yarn applicationmaster cannot get a proper resourcemanager address from yarnconfiguration?
Hello, I tried to submit a spark job to yarn cluster, there is an error occured with those messages: [root@saturn00 bin]# ./spark-submit --class SparkHiveJoin --master yarn-cluster --num-executors 10 --executor-memory 12g --executor-cores 1 spark.jarSpark assembly has been built with Hive, including Datanucleus jars on classpathWarning: Ignoring non-spark config property: yarn.resourcemanager.address=10.150.20.22:8032Warning: Ignoring non-spark config property: yarn.resourcemanager.address=10.150.20.22:803214/09/04 11:01:27 INFO client.RMProxy: Connecting to ResourceManager at /10.150.20.22:803214/09/04 11:01:27 INFO yarn.Client: Got Cluster metric info from ApplicationsManager (ASM), number of NodeManagers: 1014/09/04 11:01:27 INFO yarn.Client: Queue info ... queueName: default, queueCurrentCapacity: 0.0, queueMaxCapacity: 1.0, queueApplicationCount = 0, queueChildQueueCount = 014/09/04 11:01:27 INFO yarn.Client: Max mem capabililty of a single resource in this cluster 1228814
/09/04 11:01:27 INFO yarn.Client: Preparing Local resources14/09/04 11:01:27 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable14/09/04 11:01:27 INFO yarn.Client: Uploading file:/opt/spark-1.0.2-bin-hadoop2/bin/spark.jar to hdfs://10.150.20.22:8020/user/root/.sparkStaging/application_1409759471992_0007/spark.jar14/09/04 11:01:28 INFO yarn.Client: Uploading file:/opt/spark-1.0.2-bin-hadoop2/lib/spark-assembly-1.0.2-hadoop2.4.0.jar to hdfs://10.150.20.22:8020/user/root/.sparkStaging/application_1409759471992_0007/spark-assembly-1.0.2-hadoop2.4.0.jar14/09/04 11:01:30 INFO yarn.Client: Setting up the launch environment14/09/04 11:01:30 INFO yarn.Client: Setting up container launch context, , /stderr)14/09/04 11:01:30 INFO yarn.Client: Submitting application to ASM14/09/04 11:01:30 INFO impl.YarnClientImpl: Submitted application application_1409759471992_000714/09/04 11:01:31 INFO yarn.Client: Application repo
rt from ASM: application identifier: application_1409759471992_0007 appId: 7 clientToAMToken: null appDiagnostics: appMasterHost: N/A appQueue: default appMasterRpcPort: -1 appStartTime: 1409796090132 yarnAppState: ACCEPTED distributedFinalState: UNDEFINED appTrackingUrl: http://saturn00:8088/proxy/application_1409759471992_0007/ appUser: root14/09/04 11:01:32 INFO yarn.Client: Application report from ASM: application identifier: application_1409759471992_0007 appId: 7 clientToAMToken: null appDiagnostics: appMasterHost: N/A appQueue: default appMasterRpcPort: -1 appStartTime: 1409796090132 yarnAppState: ACCEPTED distributedFinalState: UNDEFINED appTrackingUrl: http://saturn00:8088/proxy/application_1409759471992_0007/ appUser: root So, I found the log of applicationmaster of spark on yarn job, here is the logs: 14/09/04 11:01:33 INFO ApplicationMaster: ApplicationAttemptId: appattempt_1409759471992_0007_000001
14/09/04 11:01:33 INFO RMProxy: Connecting to ResourceManager at /0.0.0.0:8030
14/09/04 11:01:33 INFO SecurityManager: Changing view acls to: root
14/09/04 11:01:33 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root)
14/09/04 11:01:33 INFO ApplicationMaster: Starting the user JAR in a separate Thread
14/09/04 11:01:33 INFO ApplicationMaster: Waiting for Spark context initialization
14/09/04 11:01:33 INFO ApplicationMaster: Waiting for Spark context initialization ... 0
14/09/04 11:01:33 WARN SparkConf: In Spark 1.0 and later spark.local.dir will be overridden by the value set by the cluster manager (via SPARK_LOCAL_DIRS in mesos/standalone and LOCAL_DIRS in YARN).
14/09/04 11:01:33 INFO SecurityManager: Changing view acls to: root
14/09/04 11:01:33 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root)
14/09/04 11:01:33 INFO Slf4jLogger: Slf4jLogger started
14/09/04 11:01:33 INFO Remoting: Starting remoting
14/09/04 11:01:33 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://spark@saturn02:58444]
14/09/04 11:01:33 INFO Remoting: Remoting now listens on addresses: [akka.tcp://spark@saturn02:58444]
14/09/04 11:01:33 INFO SparkEnv: Registering MapOutputTracker
14/09/04 11:01:33 INFO SparkEnv: Registering BlockManagerMaster
14/09/04 11:01:33 INFO DiskBlockManager: Created local directory at /home/hadoop/logs/hadoop_tmp/usercache/root/appcache/application_1409759471992_0007/spark-local-20140904110133-a55e
14/09/04 11:01:33 INFO MemoryStore: MemoryStore started with capacity 273.1 MB.
14/09/04 11:01:33 INFO ConnectionManager: Bound socket to port 43903 with id = ConnectionManagerId(saturn02,43903)
14/09/04 11:01:33 INFO BlockManagerMaster: Trying to register BlockManager
14/09/04 11:01:33 INFO BlockManagerInfo: Registering block manager saturn02:43903 with 273.1 MB RAM
14/09/04 11:01:33 INFO BlockManagerMaster: Registered BlockManager
14/09/04 11:01:33 INFO HttpServer: Starting HTTP Server
14/09/04 11:01:33 INFO HttpBroadcast: Broadcast server started at http://10.150.20.51:38630
14/09/04 11:01:33 INFO HttpFileServer: HTTP File server directory is /home/hadoop/logs/hadoop_tmp/usercache/root/appcache/application_1409759471992_0007/container_1409759471992_0007_01_000001/tmp/spark-932fbeb0-a408-44ae-a963-2c14e7dc9e21
14/09/04 11:01:33 INFO HttpServer: Starting HTTP Server
14/09/04 11:01:33 INFO JettyUtils: Adding filter: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
14/09/04 11:01:33 INFO SparkUI: Started SparkUI at http://saturn02:33972
14/09/04 11:01:34 INFO EventLoggingListener: Logging events to hdfs://10.150.20.22:8020/sparkLog/sparkhivejoin-1409796093920
14/09/04 11:01:34 INFO YarnClusterScheduler: Created YarnClusterScheduler
14/09/04 11:01:34 INFO ApplicationMaster$$anon$1: Adding shutdown hook for context org.apache.spark.SparkContext@573cf2a9
14/09/04 11:01:34 INFO ApplicationMaster: Registering the ApplicationMaster
14/09/04 11:01:35 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:01:36 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:01:37 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:01:38 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:01:39 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:01:40 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:01:41 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:01:42 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:01:43 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:01:44 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:02:15 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:02:16 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:02:17 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:02:18 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:02:19 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:02:20 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:02:21 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:02:22 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:02:23 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:02:24 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:02:55 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:02:56 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:02:57 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:02:58 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:02:59 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:03:00 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:03:01 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:03:02 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:03:03 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:03:04 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:03:35 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:03:36 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:03:37 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:03:38 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:03:39 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:03:40 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:03:41 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:03:42 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:03:43 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:03:44 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:04:15 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:04:16 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:04:17 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:04:18 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:04:19 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:04:20 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:04:21 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:04:22 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:04:23 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:04:24 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:04:55 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:04:56 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:04:57 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:04:58 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:04:59 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:05:00 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:05:01 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:05:02 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:05:03 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:05:04 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:05:35 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:05:36 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:05:37 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:05:38 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:05:39 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:05:40 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:05:41 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:05:42 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:05:43 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:05:44 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:06:15 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:06:16 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:06:17 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:06:18 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:06:19 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:06:20 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:06:21 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:06:22 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:06:23 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
14/09/04 11:06:24 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS) In the code of YarnRMClientImpl..scala at org.apache.spark.deploy.yarn, there is a function called "registerWithResourceManager"This function tries to get the yarn conf using this code: private def registerWithResourceManager(conf: YarnConfiguration): AMRMProtocol = {val rmAddress = NetUtils.createSocketAddr(conf.get(YarnConfiguration.RM_SCHEDULER_ADDRESS,YarnConfiguration.DEFAULT_RM_SCHEDULER_ADDRESS)) logInfo("Connecting to ResourceManager at " + rmAddress)
rpc.getProxy(classOf[AMRMProtocol], rmAddress, conf).asInstanceOf[AMRMProtocol]}
In my yarn-site.xml in conf folder of yarn, I set the yarn.resourcemanager.scheduler.address as 10.150.20.22:8030, but why this function uses YarnConfiguration.DEFAULT_RM_SCHEDULER_ADDRESS? (0.0.0.0:8030) and is there any solution for my issue?Everything except spark runs well over the yarn in my cluster. Thanks
// Yoonmin Nam
Re: Why spark on yarn applicationmaster cannot get a proper
resourcemanager address from yarnconfiguration?
Posted by Guodong Wang <wa...@gmail.com>.
Did you follow the exact step in this page
https://spark.apache.org/docs/1.0.2/running-on-yarn.html ?
Please be sure that HADOOP_CONF_DIR or YARN_CONF_DIR points to the
directory which contains the (client side) configuration files for the
Hadoop cluster.
Guodong
On Thu, Sep 4, 2014 at 10:15 AM, 남윤민 <ro...@dgist.ac.kr> wrote:
> Hello, I tried to submit a spark job to yarn cluster, there is an error
> occured with those messages:
>
>
>
>
> [root@saturn00 bin]# ./spark-submit --class SparkHiveJoin --master
> yarn-cluster --num-executors 10 --executor-memory 12g --executor-cores 1
> spark.jar
> Spark assembly has been built with Hive, including Datanucleus jars on
> classpath
> Warning: Ignoring non-spark config property: yarn.resourcemanager.address=
> 10.150.20.22:8032
> Warning: Ignoring non-spark config property: yarn.resourcemanager.address=
> 10.150.20.22:8032
> 14/09/04 11:01:27 INFO client.RMProxy: Connecting to ResourceManager at /
> 10.150.20.22:8032
> 14/09/04 11:01:27 INFO yarn.Client: Got Cluster metric info from
> ApplicationsManager (ASM), number of NodeManagers: 10
> 14/09/04 11:01:27 INFO yarn.Client: Queue info ... queueName: default,
> queueCurrentCapacity: 0.0, queueMaxCapacity: 1.0,
> queueApplicationCount = 0, queueChildQueueCount = 0
> 14/09/04 11:01:27 INFO yarn.Client: Max mem capabililty of a single
> resource in this cluster 12288
> 14/09/04 11:01:27 INFO yarn.Client: Preparing Local resources
> 14/09/04 11:01:27 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 14/09/04 11:01:27 INFO yarn.Client: Uploading
> file:/opt/spark-1.0.2-bin-hadoop2/bin/spark.jar to hdfs://
> 10.150.20.22:8020/user/root/.sparkStaging/application_1409759471992_0007/spark.jar
> 14/09/04 11:01:28 INFO yarn.Client: Uploading
> file:/opt/spark-1.0.2-bin-hadoop2/lib/spark-assembly-1.0.2-hadoop2.4.0.jar
> to hdfs://
> 10.150.20.22:8020/user/root/.sparkStaging/application_1409759471992_000
> 7/spark-assembly-1.0.2-hadoop2.4.0.jar
> 14/09/04 11:01:30 INFO yarn.Client: Setting up the launch environment
> 14/09/04 11:01:30 INFO yarn.Client: Setting up container launch context
> 14/09/04 11:01:30 INFO yarn.Client: Command for starting the Spark
> ApplicationMaster: List($JAVA_HOME/bin/java, -server, -Xmx512m,
> -Djava.io.tmpdir=$PWD/tmp, -Dspark.local.dir=\"/hadoop/spark\",
> -Dspark.shuffle.consolidateFiles=\"true\", -Dspark.executor.memory=\"12g\",
> -Dspark.master=\"spark://10.150.20.22:8081\", -Dspark.app.name=\"SparkHiveJoin\",
> -Dspark.eventLog.enabled=\"true\", -Dspark.spill=\"true\",
> -Dspark.serializer=\"org.apache.spark.serializer.KryoSerializer\",
> -Dspark.eventLog.dir=\"hdfs://10.150.20.22:8020/sparkLog\
> <http://10.150.20.22:8020/sparkLog%5C>",
> -Dlog4j.configuration=log4j-spark-container.properties,
> org.apache.spark.deploy.yarn.ApplicationMaster, --class, SparkHiveJoin,
> --jar , file:/opt/spark-1.0.2-bin-hadoop2/bin/spark.jar, ,
> --executor-memory, 12288, --executor-cores, 1, --num-e xecutors , 10, 1>,
> <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr)
> 14/09/04 11:01:30 INFO yarn.Client: Submitting application to ASM
> 14/09/04 11:01:30 INFO impl.YarnClientImpl: Submitted application
> application_1409759471992_0007
> 14/09/04 11:01:31 INFO yarn.Client: Application report from ASM:
> application identifier: application_1409759471992_0007
> appId: 7
> clientToAMToken: null
> appDiagnostics:
> appMasterHost: N/A
> appQueue: default
> appMast erRpcPort: -1
> appStartTime: 1409796090132
> yarnAppState: ACCEPTED
> distributedFinalState: UNDEFINED
> appTrackingUrl:
> http://saturn00:8088/proxy/application_1409759471992_0007/
> appUser: root
> 14/09/04 11:01:32 INFO yarn.Client: Application report from ASM:
> application identifier: application_1409759471992_0007
> appId: 7
> clientToAMToken: null
> appDiagnostics:
> appMasterHost: N/A
> appQueue: default
> appMasterRpcPort: -1
> appStartTime: 1409796090132
> yarnAppState: ACCEPTED
> distributedFinalState: UNDEFINED
> appTrackingUrl:
> http://saturn00:8088/proxy/application_1409759471992_0007/
> appUser: root
>
>
>
>
>
>
>
> So, I found the log of applicationmaster of spark on yarn job, here is the
> logs:
>
>
>
>
>
>
> 14/09/04 11:01:33 INFO ApplicationMaster: ApplicationAttemptId: appattempt_1409759471992_0007_000001
> 14/09/04 11:01:33 INFO RMProxy: Connecting to ResourceManager at /0.0.0.0:8030
> 14/09/04 11:01:33 INFO SecurityManager: Changing view acls to: root
> 14/09/04 11:01:33 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root)
> 14/09/04 11:01:33 INFO ApplicationMaster: Starting the user JAR in a separate Thread
> 14/09/04 11:01:33 INFO ApplicationMaster: Waiting for Spark context initialization
> 14/09/04 11:01:33 INFO ApplicationMaster: Waiting for Spark context initialization ... 0
> 14/09/04 11:01:33 WARN SparkConf: In Spark 1.0 and later spark.local.dir will be overridden by the value set by the cluster manager (via SPARK_LOCAL_DIRS in mesos/standalone and LOCAL_DIRS in YARN).
> 14/09/04 11:01:33 INFO SecurityManager: Changing view acls to: root
> 14/09/04 11:01:33 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root)
> 14/09/04 11:01:33 INFO Slf4jLogger: Slf4jLogger started
> 14/09/04 11:01:33 INFO Remoting: Starting remoting
> 14/09/04 11:01:33 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://spark@saturn02:58444]
> 14/09/04 11:01:33 INFO Remoting: Remoting now listens on addresses: [akka.tcp://spark@saturn02:58444]
> 14/09/04 11:01:33 INFO SparkEnv: Registering MapOutputTracker
> 14/09/04 11:01:33 INFO SparkEnv: Registering BlockManagerMaster
> 14/09/04 11:01:33 INFO DiskBlockManager: Created local directory at /home/hadoop/logs/hadoop_tmp/usercache/root/appcache/application_1409759471992_0007/spark-local-20140904110133-a55e
> 14/09/04 11:01:33 INFO MemoryStore: MemoryStore started with capacity 273.1 MB.
> 14/09/04 11:01:33 INFO ConnectionManager: Bound socket to port 43903 with id = ConnectionManagerId(saturn02,43903)
> 14/09/04 11:01:33 INFO BlockManagerMaster: Trying to register BlockManager
> 14/09/04 11:01:33 INFO BlockManagerInfo: Registering block manager saturn02:43903 with 273.1 MB RAM
> 14/09/04 11:01:33 INFO BlockManagerMaster: Registered BlockManager
> 14/09/04 11:01:33 INFO HttpServer: Starting HTTP Server
> 14/09/04 11:01:33 INFO HttpBroadcast: Broadcast server started at http://10.150.20.51:38630
> 14/09/04 11:01:33 INFO HttpFileServer: HTTP File server directory is /home/hadoop/logs/hadoop_tmp/usercache/root/appcache/application_1409759471992_0007/container_1409759471992_0007_01_000001/tmp/spark-932fbeb0-a408-44ae-a963-2c14e7dc9e21
> 14/09/04 11:01:33 INFO HttpServer: Starting HTTP Server
> 14/09/04 11:01:33 INFO JettyUtils: Adding filter: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
> 14/09/04 11:01:33 INFO SparkUI: Started SparkUI at http://saturn02:33972
> 14/09/04 11:01:34 INFO EventLoggingListener: Logging events to hdfs://10.150.20.22:8020/sparkLog/sparkhivejoin-1409796093920
> 14/09/04 <http://10.150.20.22:8020/sparkLog/sparkhivejoin-140979609392014/09/04> 11:01:34 INFO YarnClusterScheduler: Created YarnClusterScheduler
> 14/09/04 11:01:34 INFO ApplicationMaster$$anon$1: Adding shutdown hook for context org.apache.spark.SparkContext@573cf2a9
> 14/09/04 11:01:34 INFO ApplicationMaster: Registering the ApplicationMaster
> 14/09/04 11:01:35 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:01:36 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:01:37 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:01:38 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:01:39 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:01:40 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:01:41 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:01:42 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:01:43 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:01:44 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:02:15 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:02:16 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:02:17 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:02:18 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:02:19 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:02:20 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:02:21 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:02:22 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:02:23 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:02:24 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:02:55 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:02:56 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:02:57 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:02:58 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:02:59 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:03:00 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:03:01 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:03:02 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:03:03 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:03:04 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:03:35 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:03:36 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:03:37 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:03:38 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:03:39 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:03:40 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:03:41 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:03:42 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:03:43 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:03:44 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:04:15 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:04:16 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:04:17 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:04:18 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:04:19 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:04:20 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:04:21 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:04:22 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:04:23 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:04:24 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:04:55 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:04:56 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:04:57 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:04:58 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:04:59 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:05:00 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:05:01 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:05:02 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:05:03 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:05:04 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:05:35 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:05:36 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:05:37 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:05:38 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:05:39 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:05:40 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:05:41 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:05:42 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:05:43 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:05:44 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:06:15 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:06:16 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:06:17 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:06:18 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:06:19 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:06:20 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:06:21 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:06:22 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:06:23 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
> 14/09/04 11:06:24 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
>
>
>
>
>
> In the code of YarnRMClientImpl..scala at org.apache.spark.deploy.yarn,
> there is a function called "registerWithResourceManager"
>
> This function tries to get the yarn conf using this code:
>
>
>
> private def registerWithResourceManager(conf: YarnConfiguration):
> AMRMProtocol = {
>
> val rmAddress = NetUtils.createSocketAddr(conf.get(YarnConfiguration.
> RM_SCHEDULER_ADDRESS,YarnConfiguration.DEFAULT_RM_SCHEDULER_ADDRESS))
>
> logInfo("Connecting to ResourceManager at " + rmAddress)
>
> rpc.getProxy(classOf[AMRMProtocol], rmAddress, conf).asInstanceOf[
> AMRMProtocol]
>
> }
>
>
> In my yarn-site.xml in conf folder of yarn, I set the
> yarn.resourcemanager.scheduler.address as 10.150.20.22:8030, but
>
> *why this function uses YarnConfiguration.DEFAULT_RM_SCHEDULER_ADDRESS?
> (0.0.0.0:8030 <http://0.0.0.0:8030>)*
>
>
>
> and is there any solution for my issue?
>
> Everything except spark runs well over the yarn in my cluster.
>
>
>
> Thanks
>
>
>
>
>
>
>
> // *Yoonmin Nam*
> < br />
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>