You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Margusja <ma...@roo.ee> on 2014/08/28 19:40:44 UTC
ApplicationMaster link on cluster web page does not work
Hi
<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.app.mapreduce.am.staging-dir</name>
<value>/user</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
<name>yarn.application.classpath</name>
<value>
/etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*,/home/hduser/mahout-1.0-snapshot/math/target/*
</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>0.0.0.0:8030</value>
</property>
<property>
<name>yarn.web-proxy.address</name>
<value>vm38.dbweb.ee:8089</value>
</property>
</configuration>
when i click job's ApplicationMaster link
(http://vm38.dbweb.ee:8089/proxy/application_1409246753441_0001/) on
claster all applications list I will get:
HTTP ERROR 404
Problem accessing /proxy/application_1409246753441_0001/mapreduce. Reason:
NOT_FOUND
------------------------------------------------------------------------
/Powered by Jetty://
/I use yarn.web-proxy because without yarn.web-proxy when I click
ApplicationMaster link I will have loads of tcp connections with
established statuses.
In yarn-proxy log there is nothing interesting
--
Best regards, Margus (Margusja) Roo
+372 51 48 780
http://margus.roo.ee
http://ee.linkedin.com/in/margusroo
skype: margusja
ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)"
Re: ApplicationMaster link on cluster web page does not work
Posted by Margusja <ma...@roo.ee>.
Moved resourcemanager to another server and it works. I guess I have
some network miss routing there :)
Best regards, Margus (Margusja) Roo
+372 51 48 780
http://margus.roo.ee
http://ee.linkedin.com/in/margusroo
skype: margusja
ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)"
On 28/08/14 21:39, Margusja wrote:
> More information
>
> after I started resourcemanager
> [root@vm38 ~]# /etc/init.d/hadoop-yarn-resourcemanager start
> Starting Hadoop resourcemanager: [ OK ]
>
> and I open cluster web interface there is some tcp connections to 8088:
>
> [root@vm38 ~]# netstat -np | grep 8088
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:61120 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:64412 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:50139 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:64407 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:58817 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:52250 ESTABLISHED 15723/java
>
> this is ok
>
> Now I started new map reduce job and got tracking url
> http://server:8088/proxy/application_1409250808355_0001/
>
> now there are loads of connections and tracking url does not response:
> tcp 0 0 ::ffff:90.190.106.33:46910
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:35984
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:37559 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:44154
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:45417
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:57294
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47949 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55330
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 432 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45467 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:58405
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55580
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:51992 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:44578
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:38686 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:39242
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55916
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:48064
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:35148
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:42638
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:50836
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44789 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55051
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:39207 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:53781
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:54180
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:56344
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:52707
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:52274
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45417 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:46627 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44583 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:40928
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:47014
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:60939
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:52274 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:38391
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47321 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:36186 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:39160
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:54352
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:59425 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:46627
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:57140 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:48788 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:37559
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:55052 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47104 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:47515
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45655 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:48064 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:45467
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47343 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:42678
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:33116 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:40077
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:49100
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44547 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:40050 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:43073
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:38686
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:41480
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:56344 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:42933
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45400 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:33023
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:42678 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:43331
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:48524 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:51789
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44447 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:35984 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:50472
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55326
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:44356
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:48788
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44560 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:38690 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:57434 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:45282
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:38284
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:44411
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55712
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:48929
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:49288
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:42505
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:38391 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:54537 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:43331 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47141 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:60823
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:55712 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:42313 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45879 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:33412
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:51992
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:37513 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:39260
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:46269 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:46087
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:47321
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> ...
> ...
> ...
>
> wtf?
>
> Best regards, Margus (Margusja) Roo
> +372 51 48 780
> http://margus.roo.ee
> http://ee.linkedin.com/in/margusroo
> skype: margusja
> ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)"
>
> On 28/08/14 20:40, Margusja wrote:
>> Hi
>>
>> <configuration>
>>
>> <!-- Site specific YARN configuration properties -->
>> <property>
>> <name>yarn.app.mapreduce.am.staging-dir</name>
>> <value>/user</value>
>> </property>
>>
>> <property>
>> <name>yarn.nodemanager.aux-services</name>
>> <value>mapreduce_shuffle</value>
>> </property>
>>
>> <property>
>> <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
>> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
>> </property>
>>
>> <property>
>> <name>yarn.application.classpath</name>
>> <value>
>> /etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*,/home/hduser/mahout-1.0-snapshot/math/target/*
>>
>> </value>
>> </property>
>>
>> <property>
>> <name>yarn.resourcemanager.scheduler.address</name>
>> <value>0.0.0.0:8030</value>
>> </property>
>>
>> <property>
>> <name>yarn.web-proxy.address</name>
>> <value>vm38.dbweb.ee:8089</value>
>> </property>
>>
>> </configuration>
>>
>> when i click job's ApplicationMaster link
>> (http://vm38.dbweb.ee:8089/proxy/application_1409246753441_0001/) on
>> claster all applications list I will get:
>>
>>
>> HTTP ERROR 404
>>
>> Problem accessing /proxy/application_1409246753441_0001/mapreduce.
>> Reason:
>>
>> NOT_FOUND
>>
>> ------------------------------------------------------------------------
>> /Powered by Jetty://
>>
>> /I use yarn.web-proxy because without yarn.web-proxy when I click
>> ApplicationMaster link I will have loads of tcp connections with
>> established statuses.
>> In yarn-proxy log there is nothing interesting
>>
>
Re: ApplicationMaster link on cluster web page does not work
Posted by Margusja <ma...@roo.ee>.
Moved resourcemanager to another server and it works. I guess I have
some network miss routing there :)
Best regards, Margus (Margusja) Roo
+372 51 48 780
http://margus.roo.ee
http://ee.linkedin.com/in/margusroo
skype: margusja
ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)"
On 28/08/14 21:39, Margusja wrote:
> More information
>
> after I started resourcemanager
> [root@vm38 ~]# /etc/init.d/hadoop-yarn-resourcemanager start
> Starting Hadoop resourcemanager: [ OK ]
>
> and I open cluster web interface there is some tcp connections to 8088:
>
> [root@vm38 ~]# netstat -np | grep 8088
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:61120 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:64412 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:50139 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:64407 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:58817 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:52250 ESTABLISHED 15723/java
>
> this is ok
>
> Now I started new map reduce job and got tracking url
> http://server:8088/proxy/application_1409250808355_0001/
>
> now there are loads of connections and tracking url does not response:
> tcp 0 0 ::ffff:90.190.106.33:46910
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:35984
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:37559 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:44154
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:45417
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:57294
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47949 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55330
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 432 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45467 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:58405
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55580
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:51992 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:44578
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:38686 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:39242
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55916
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:48064
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:35148
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:42638
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:50836
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44789 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55051
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:39207 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:53781
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:54180
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:56344
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:52707
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:52274
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45417 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:46627 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44583 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:40928
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:47014
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:60939
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:52274 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:38391
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47321 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:36186 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:39160
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:54352
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:59425 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:46627
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:57140 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:48788 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:37559
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:55052 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47104 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:47515
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45655 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:48064 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:45467
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47343 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:42678
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:33116 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:40077
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:49100
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44547 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:40050 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:43073
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:38686
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:41480
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:56344 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:42933
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45400 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:33023
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:42678 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:43331
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:48524 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:51789
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44447 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:35984 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:50472
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55326
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:44356
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:48788
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44560 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:38690 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:57434 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:45282
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:38284
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:44411
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55712
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:48929
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:49288
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:42505
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:38391 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:54537 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:43331 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47141 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:60823
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:55712 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:42313 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45879 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:33412
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:51992
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:37513 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:39260
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:46269 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:46087
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:47321
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> ...
> ...
> ...
>
> wtf?
>
> Best regards, Margus (Margusja) Roo
> +372 51 48 780
> http://margus.roo.ee
> http://ee.linkedin.com/in/margusroo
> skype: margusja
> ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)"
>
> On 28/08/14 20:40, Margusja wrote:
>> Hi
>>
>> <configuration>
>>
>> <!-- Site specific YARN configuration properties -->
>> <property>
>> <name>yarn.app.mapreduce.am.staging-dir</name>
>> <value>/user</value>
>> </property>
>>
>> <property>
>> <name>yarn.nodemanager.aux-services</name>
>> <value>mapreduce_shuffle</value>
>> </property>
>>
>> <property>
>> <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
>> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
>> </property>
>>
>> <property>
>> <name>yarn.application.classpath</name>
>> <value>
>> /etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*,/home/hduser/mahout-1.0-snapshot/math/target/*
>>
>> </value>
>> </property>
>>
>> <property>
>> <name>yarn.resourcemanager.scheduler.address</name>
>> <value>0.0.0.0:8030</value>
>> </property>
>>
>> <property>
>> <name>yarn.web-proxy.address</name>
>> <value>vm38.dbweb.ee:8089</value>
>> </property>
>>
>> </configuration>
>>
>> when i click job's ApplicationMaster link
>> (http://vm38.dbweb.ee:8089/proxy/application_1409246753441_0001/) on
>> claster all applications list I will get:
>>
>>
>> HTTP ERROR 404
>>
>> Problem accessing /proxy/application_1409246753441_0001/mapreduce.
>> Reason:
>>
>> NOT_FOUND
>>
>> ------------------------------------------------------------------------
>> /Powered by Jetty://
>>
>> /I use yarn.web-proxy because without yarn.web-proxy when I click
>> ApplicationMaster link I will have loads of tcp connections with
>> established statuses.
>> In yarn-proxy log there is nothing interesting
>>
>
Re: ApplicationMaster link on cluster web page does not work
Posted by Margusja <ma...@roo.ee>.
Moved resourcemanager to another server and it works. I guess I have
some network miss routing there :)
Best regards, Margus (Margusja) Roo
+372 51 48 780
http://margus.roo.ee
http://ee.linkedin.com/in/margusroo
skype: margusja
ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)"
On 28/08/14 21:39, Margusja wrote:
> More information
>
> after I started resourcemanager
> [root@vm38 ~]# /etc/init.d/hadoop-yarn-resourcemanager start
> Starting Hadoop resourcemanager: [ OK ]
>
> and I open cluster web interface there is some tcp connections to 8088:
>
> [root@vm38 ~]# netstat -np | grep 8088
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:61120 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:64412 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:50139 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:64407 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:58817 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:52250 ESTABLISHED 15723/java
>
> this is ok
>
> Now I started new map reduce job and got tracking url
> http://server:8088/proxy/application_1409250808355_0001/
>
> now there are loads of connections and tracking url does not response:
> tcp 0 0 ::ffff:90.190.106.33:46910
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:35984
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:37559 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:44154
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:45417
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:57294
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47949 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55330
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 432 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45467 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:58405
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55580
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:51992 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:44578
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:38686 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:39242
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55916
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:48064
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:35148
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:42638
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:50836
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44789 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55051
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:39207 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:53781
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:54180
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:56344
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:52707
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:52274
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45417 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:46627 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44583 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:40928
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:47014
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:60939
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:52274 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:38391
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47321 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:36186 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:39160
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:54352
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:59425 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:46627
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:57140 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:48788 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:37559
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:55052 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47104 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:47515
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45655 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:48064 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:45467
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47343 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:42678
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:33116 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:40077
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:49100
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44547 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:40050 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:43073
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:38686
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:41480
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:56344 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:42933
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45400 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:33023
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:42678 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:43331
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:48524 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:51789
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44447 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:35984 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:50472
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55326
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:44356
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:48788
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44560 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:38690 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:57434 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:45282
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:38284
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:44411
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55712
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:48929
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:49288
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:42505
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:38391 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:54537 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:43331 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47141 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:60823
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:55712 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:42313 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45879 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:33412
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:51992
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:37513 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:39260
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:46269 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:46087
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:47321
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> ...
> ...
> ...
>
> wtf?
>
> Best regards, Margus (Margusja) Roo
> +372 51 48 780
> http://margus.roo.ee
> http://ee.linkedin.com/in/margusroo
> skype: margusja
> ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)"
>
> On 28/08/14 20:40, Margusja wrote:
>> Hi
>>
>> <configuration>
>>
>> <!-- Site specific YARN configuration properties -->
>> <property>
>> <name>yarn.app.mapreduce.am.staging-dir</name>
>> <value>/user</value>
>> </property>
>>
>> <property>
>> <name>yarn.nodemanager.aux-services</name>
>> <value>mapreduce_shuffle</value>
>> </property>
>>
>> <property>
>> <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
>> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
>> </property>
>>
>> <property>
>> <name>yarn.application.classpath</name>
>> <value>
>> /etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*,/home/hduser/mahout-1.0-snapshot/math/target/*
>>
>> </value>
>> </property>
>>
>> <property>
>> <name>yarn.resourcemanager.scheduler.address</name>
>> <value>0.0.0.0:8030</value>
>> </property>
>>
>> <property>
>> <name>yarn.web-proxy.address</name>
>> <value>vm38.dbweb.ee:8089</value>
>> </property>
>>
>> </configuration>
>>
>> when i click job's ApplicationMaster link
>> (http://vm38.dbweb.ee:8089/proxy/application_1409246753441_0001/) on
>> claster all applications list I will get:
>>
>>
>> HTTP ERROR 404
>>
>> Problem accessing /proxy/application_1409246753441_0001/mapreduce.
>> Reason:
>>
>> NOT_FOUND
>>
>> ------------------------------------------------------------------------
>> /Powered by Jetty://
>>
>> /I use yarn.web-proxy because without yarn.web-proxy when I click
>> ApplicationMaster link I will have loads of tcp connections with
>> established statuses.
>> In yarn-proxy log there is nothing interesting
>>
>
Re: ApplicationMaster link on cluster web page does not work
Posted by Margusja <ma...@roo.ee>.
Moved resourcemanager to another server and it works. I guess I have
some network miss routing there :)
Best regards, Margus (Margusja) Roo
+372 51 48 780
http://margus.roo.ee
http://ee.linkedin.com/in/margusroo
skype: margusja
ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)"
On 28/08/14 21:39, Margusja wrote:
> More information
>
> after I started resourcemanager
> [root@vm38 ~]# /etc/init.d/hadoop-yarn-resourcemanager start
> Starting Hadoop resourcemanager: [ OK ]
>
> and I open cluster web interface there is some tcp connections to 8088:
>
> [root@vm38 ~]# netstat -np | grep 8088
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:61120 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:64412 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:50139 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:64407 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:58817 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:84.50.21.39:52250 ESTABLISHED 15723/java
>
> this is ok
>
> Now I started new map reduce job and got tracking url
> http://server:8088/proxy/application_1409250808355_0001/
>
> now there are loads of connections and tracking url does not response:
> tcp 0 0 ::ffff:90.190.106.33:46910
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:35984
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:37559 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:44154
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:45417
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:57294
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47949 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55330
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 432 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45467 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:58405
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55580
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:51992 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:44578
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:38686 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:39242
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55916
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:48064
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:35148
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:42638
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:50836
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44789 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55051
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:39207 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:53781
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:54180
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:56344
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:52707
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:52274
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45417 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:46627 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44583 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:40928
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:47014
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:60939
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:52274 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:38391
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47321 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:36186 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:39160
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:54352
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:59425 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:46627
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:57140 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:48788 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:37559
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:55052 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47104 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:47515
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45655 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:48064 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:45467
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47343 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:42678
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:33116 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:40077
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:49100
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44547 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:40050 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:43073
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:38686
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:41480
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:56344 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:42933
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45400 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:33023
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:42678 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:43331
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:48524 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:51789
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44447 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:35984 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:50472
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55326
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:44356
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:48788
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:44560 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:38690 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:57434 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:45282
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:38284
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:44411
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:55712
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:48929
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:49288
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:42505
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:38391 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:54537 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:43331 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:47141 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:60823
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:55712 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:42313 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:45879 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:33412
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:51992
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:37513 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:39260
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:8088
> ::ffff:90.190.106.33:46269 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:46087
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> tcp 0 0 ::ffff:90.190.106.33:47321
> ::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
> ...
> ...
> ...
>
> wtf?
>
> Best regards, Margus (Margusja) Roo
> +372 51 48 780
> http://margus.roo.ee
> http://ee.linkedin.com/in/margusroo
> skype: margusja
> ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)"
>
> On 28/08/14 20:40, Margusja wrote:
>> Hi
>>
>> <configuration>
>>
>> <!-- Site specific YARN configuration properties -->
>> <property>
>> <name>yarn.app.mapreduce.am.staging-dir</name>
>> <value>/user</value>
>> </property>
>>
>> <property>
>> <name>yarn.nodemanager.aux-services</name>
>> <value>mapreduce_shuffle</value>
>> </property>
>>
>> <property>
>> <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
>> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
>> </property>
>>
>> <property>
>> <name>yarn.application.classpath</name>
>> <value>
>> /etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*,/home/hduser/mahout-1.0-snapshot/math/target/*
>>
>> </value>
>> </property>
>>
>> <property>
>> <name>yarn.resourcemanager.scheduler.address</name>
>> <value>0.0.0.0:8030</value>
>> </property>
>>
>> <property>
>> <name>yarn.web-proxy.address</name>
>> <value>vm38.dbweb.ee:8089</value>
>> </property>
>>
>> </configuration>
>>
>> when i click job's ApplicationMaster link
>> (http://vm38.dbweb.ee:8089/proxy/application_1409246753441_0001/) on
>> claster all applications list I will get:
>>
>>
>> HTTP ERROR 404
>>
>> Problem accessing /proxy/application_1409246753441_0001/mapreduce.
>> Reason:
>>
>> NOT_FOUND
>>
>> ------------------------------------------------------------------------
>> /Powered by Jetty://
>>
>> /I use yarn.web-proxy because without yarn.web-proxy when I click
>> ApplicationMaster link I will have loads of tcp connections with
>> established statuses.
>> In yarn-proxy log there is nothing interesting
>>
>
Re: ApplicationMaster link on cluster web page does not work
Posted by Margusja <ma...@roo.ee>.
More information
after I started resourcemanager
[root@vm38 ~]# /etc/init.d/hadoop-yarn-resourcemanager start
Starting Hadoop resourcemanager: [ OK ]
and I open cluster web interface there is some tcp connections to 8088:
[root@vm38 ~]# netstat -np | grep 8088
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:61120 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:64412 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:50139 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:64407 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:58817 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:52250 ESTABLISHED 15723/java
this is ok
Now I started new map reduce job and got tracking url
http://server:8088/proxy/application_1409250808355_0001/
now there are loads of connections and tracking url does not response:
tcp 0 0 ::ffff:90.190.106.33:46910
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:35984
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:37559 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:44154
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:45417
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:57294
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47949 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55330
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 432 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45467 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:58405
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55580
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:51992 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:44578
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:38686 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:39242
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55916
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:48064
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:35148
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:42638
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:50836
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44789 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55051
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:39207 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:53781
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:54180
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:56344
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:52707
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:52274
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45417 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:46627 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44583 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:40928
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:47014
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:60939
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:52274 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:38391
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47321 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:36186 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:39160
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:54352
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:59425 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:46627
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:57140 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:48788 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:37559
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:55052 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47104 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:47515
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45655 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:48064 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:45467
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47343 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:42678
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:33116 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:40077
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:49100
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44547 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:40050 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:43073
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:38686
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:41480
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:56344 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:42933
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45400 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:33023
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:42678 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:43331
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:48524 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:51789
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44447 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:35984 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:50472
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55326
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:44356
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:48788
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44560 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:38690 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:57434 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:45282
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:38284
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:44411
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55712
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:48929
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:49288
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:42505
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:38391 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:54537 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:43331 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47141 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:60823
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:55712 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:42313 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45879 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:33412
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:51992
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:37513 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:39260
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:46269 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:46087
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:47321
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
...
...
...
wtf?
Best regards, Margus (Margusja) Roo
+372 51 48 780
http://margus.roo.ee
http://ee.linkedin.com/in/margusroo
skype: margusja
ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)"
On 28/08/14 20:40, Margusja wrote:
> Hi
>
> <configuration>
>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.app.mapreduce.am.staging-dir</name>
> <value>/user</value>
> </property>
>
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce_shuffle</value>
> </property>
>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
>
> <property>
> <name>yarn.application.classpath</name>
> <value>
> /etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*,/home/hduser/mahout-1.0-snapshot/math/target/*
>
> </value>
> </property>
>
> <property>
> <name>yarn.resourcemanager.scheduler.address</name>
> <value>0.0.0.0:8030</value>
> </property>
>
> <property>
> <name>yarn.web-proxy.address</name>
> <value>vm38.dbweb.ee:8089</value>
> </property>
>
> </configuration>
>
> when i click job's ApplicationMaster link
> (http://vm38.dbweb.ee:8089/proxy/application_1409246753441_0001/) on
> claster all applications list I will get:
>
>
> HTTP ERROR 404
>
> Problem accessing /proxy/application_1409246753441_0001/mapreduce.
> Reason:
>
> NOT_FOUND
>
> ------------------------------------------------------------------------
> /Powered by Jetty://
>
> /I use yarn.web-proxy because without yarn.web-proxy when I click
> ApplicationMaster link I will have loads of tcp connections with
> established statuses.
> In yarn-proxy log there is nothing interesting
>
Re: ApplicationMaster link on cluster web page does not work
Posted by Margusja <ma...@roo.ee>.
More information
after I started resourcemanager
[root@vm38 ~]# /etc/init.d/hadoop-yarn-resourcemanager start
Starting Hadoop resourcemanager: [ OK ]
and I open cluster web interface there is some tcp connections to 8088:
[root@vm38 ~]# netstat -np | grep 8088
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:61120 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:64412 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:50139 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:64407 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:58817 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:52250 ESTABLISHED 15723/java
this is ok
Now I started new map reduce job and got tracking url
http://server:8088/proxy/application_1409250808355_0001/
now there are loads of connections and tracking url does not response:
tcp 0 0 ::ffff:90.190.106.33:46910
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:35984
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:37559 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:44154
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:45417
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:57294
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47949 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55330
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 432 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45467 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:58405
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55580
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:51992 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:44578
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:38686 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:39242
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55916
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:48064
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:35148
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:42638
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:50836
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44789 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55051
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:39207 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:53781
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:54180
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:56344
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:52707
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:52274
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45417 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:46627 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44583 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:40928
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:47014
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:60939
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:52274 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:38391
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47321 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:36186 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:39160
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:54352
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:59425 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:46627
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:57140 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:48788 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:37559
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:55052 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47104 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:47515
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45655 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:48064 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:45467
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47343 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:42678
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:33116 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:40077
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:49100
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44547 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:40050 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:43073
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:38686
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:41480
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:56344 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:42933
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45400 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:33023
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:42678 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:43331
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:48524 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:51789
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44447 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:35984 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:50472
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55326
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:44356
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:48788
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44560 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:38690 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:57434 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:45282
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:38284
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:44411
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55712
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:48929
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:49288
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:42505
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:38391 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:54537 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:43331 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47141 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:60823
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:55712 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:42313 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45879 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:33412
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:51992
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:37513 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:39260
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:46269 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:46087
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:47321
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
...
...
...
wtf?
Best regards, Margus (Margusja) Roo
+372 51 48 780
http://margus.roo.ee
http://ee.linkedin.com/in/margusroo
skype: margusja
ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)"
On 28/08/14 20:40, Margusja wrote:
> Hi
>
> <configuration>
>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.app.mapreduce.am.staging-dir</name>
> <value>/user</value>
> </property>
>
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce_shuffle</value>
> </property>
>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
>
> <property>
> <name>yarn.application.classpath</name>
> <value>
> /etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*,/home/hduser/mahout-1.0-snapshot/math/target/*
>
> </value>
> </property>
>
> <property>
> <name>yarn.resourcemanager.scheduler.address</name>
> <value>0.0.0.0:8030</value>
> </property>
>
> <property>
> <name>yarn.web-proxy.address</name>
> <value>vm38.dbweb.ee:8089</value>
> </property>
>
> </configuration>
>
> when i click job's ApplicationMaster link
> (http://vm38.dbweb.ee:8089/proxy/application_1409246753441_0001/) on
> claster all applications list I will get:
>
>
> HTTP ERROR 404
>
> Problem accessing /proxy/application_1409246753441_0001/mapreduce.
> Reason:
>
> NOT_FOUND
>
> ------------------------------------------------------------------------
> /Powered by Jetty://
>
> /I use yarn.web-proxy because without yarn.web-proxy when I click
> ApplicationMaster link I will have loads of tcp connections with
> established statuses.
> In yarn-proxy log there is nothing interesting
>
Re: ApplicationMaster link on cluster web page does not work
Posted by Margusja <ma...@roo.ee>.
More information
after I started resourcemanager
[root@vm38 ~]# /etc/init.d/hadoop-yarn-resourcemanager start
Starting Hadoop resourcemanager: [ OK ]
and I open cluster web interface there is some tcp connections to 8088:
[root@vm38 ~]# netstat -np | grep 8088
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:61120 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:64412 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:50139 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:64407 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:58817 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:52250 ESTABLISHED 15723/java
this is ok
Now I started new map reduce job and got tracking url
http://server:8088/proxy/application_1409250808355_0001/
now there are loads of connections and tracking url does not response:
tcp 0 0 ::ffff:90.190.106.33:46910
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:35984
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:37559 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:44154
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:45417
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:57294
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47949 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55330
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 432 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45467 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:58405
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55580
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:51992 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:44578
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:38686 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:39242
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55916
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:48064
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:35148
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:42638
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:50836
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44789 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55051
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:39207 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:53781
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:54180
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:56344
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:52707
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:52274
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45417 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:46627 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44583 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:40928
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:47014
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:60939
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:52274 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:38391
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47321 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:36186 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:39160
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:54352
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:59425 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:46627
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:57140 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:48788 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:37559
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:55052 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47104 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:47515
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45655 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:48064 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:45467
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47343 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:42678
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:33116 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:40077
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:49100
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44547 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:40050 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:43073
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:38686
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:41480
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:56344 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:42933
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45400 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:33023
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:42678 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:43331
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:48524 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:51789
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44447 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:35984 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:50472
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55326
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:44356
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:48788
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44560 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:38690 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:57434 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:45282
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:38284
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:44411
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55712
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:48929
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:49288
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:42505
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:38391 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:54537 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:43331 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47141 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:60823
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:55712 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:42313 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45879 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:33412
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:51992
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:37513 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:39260
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:46269 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:46087
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:47321
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
...
...
...
wtf?
Best regards, Margus (Margusja) Roo
+372 51 48 780
http://margus.roo.ee
http://ee.linkedin.com/in/margusroo
skype: margusja
ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)"
On 28/08/14 20:40, Margusja wrote:
> Hi
>
> <configuration>
>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.app.mapreduce.am.staging-dir</name>
> <value>/user</value>
> </property>
>
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce_shuffle</value>
> </property>
>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
>
> <property>
> <name>yarn.application.classpath</name>
> <value>
> /etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*,/home/hduser/mahout-1.0-snapshot/math/target/*
>
> </value>
> </property>
>
> <property>
> <name>yarn.resourcemanager.scheduler.address</name>
> <value>0.0.0.0:8030</value>
> </property>
>
> <property>
> <name>yarn.web-proxy.address</name>
> <value>vm38.dbweb.ee:8089</value>
> </property>
>
> </configuration>
>
> when i click job's ApplicationMaster link
> (http://vm38.dbweb.ee:8089/proxy/application_1409246753441_0001/) on
> claster all applications list I will get:
>
>
> HTTP ERROR 404
>
> Problem accessing /proxy/application_1409246753441_0001/mapreduce.
> Reason:
>
> NOT_FOUND
>
> ------------------------------------------------------------------------
> /Powered by Jetty://
>
> /I use yarn.web-proxy because without yarn.web-proxy when I click
> ApplicationMaster link I will have loads of tcp connections with
> established statuses.
> In yarn-proxy log there is nothing interesting
>
Re: ApplicationMaster link on cluster web page does not work
Posted by Margusja <ma...@roo.ee>.
More information
after I started resourcemanager
[root@vm38 ~]# /etc/init.d/hadoop-yarn-resourcemanager start
Starting Hadoop resourcemanager: [ OK ]
and I open cluster web interface there is some tcp connections to 8088:
[root@vm38 ~]# netstat -np | grep 8088
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:61120 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:64412 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:50139 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:64407 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:58817 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:84.50.21.39:52250 ESTABLISHED 15723/java
this is ok
Now I started new map reduce job and got tracking url
http://server:8088/proxy/application_1409250808355_0001/
now there are loads of connections and tracking url does not response:
tcp 0 0 ::ffff:90.190.106.33:46910
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:35984
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:37559 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:44154
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:45417
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:57294
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47949 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55330
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 432 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45467 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:58405
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55580
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:51992 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:44578
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:38686 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:39242
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55916
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:48064
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:35148
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:42638
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:50836
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44789 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55051
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:39207 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:53781
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:54180
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:56344
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:52707
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:52274
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45417 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:46627 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44583 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:40928
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:47014
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:60939
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:52274 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:38391
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47321 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:36186 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:39160
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:54352
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:59425 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:46627
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:57140 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:48788 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:37559
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:55052 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47104 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:47515
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45655 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:48064 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:45467
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47343 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:42678
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:33116 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:40077
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:49100
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44547 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:40050 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:43073
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:38686
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:41480
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:56344 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:42933
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45400 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:33023
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:42678 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:43331
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:48524 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:51789
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44447 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:35984 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:50472
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55326
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:44356
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:48788
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:44560 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:38690 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:57434 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:45282
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:38284
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:44411
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:55712
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:48929
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:49288
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:42505
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:38391 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:54537 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:43331 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:47141 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:60823
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:55712 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:42313 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:45879 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:33412
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:51992
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:37513 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:39260
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:8088
::ffff:90.190.106.33:46269 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:46087
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
tcp 0 0 ::ffff:90.190.106.33:47321
::ffff:90.190.106.33:8088 ESTABLISHED 15723/java
...
...
...
wtf?
Best regards, Margus (Margusja) Roo
+372 51 48 780
http://margus.roo.ee
http://ee.linkedin.com/in/margusroo
skype: margusja
ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)"
On 28/08/14 20:40, Margusja wrote:
> Hi
>
> <configuration>
>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.app.mapreduce.am.staging-dir</name>
> <value>/user</value>
> </property>
>
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce_shuffle</value>
> </property>
>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
>
> <property>
> <name>yarn.application.classpath</name>
> <value>
> /etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*,/home/hduser/mahout-1.0-snapshot/math/target/*
>
> </value>
> </property>
>
> <property>
> <name>yarn.resourcemanager.scheduler.address</name>
> <value>0.0.0.0:8030</value>
> </property>
>
> <property>
> <name>yarn.web-proxy.address</name>
> <value>vm38.dbweb.ee:8089</value>
> </property>
>
> </configuration>
>
> when i click job's ApplicationMaster link
> (http://vm38.dbweb.ee:8089/proxy/application_1409246753441_0001/) on
> claster all applications list I will get:
>
>
> HTTP ERROR 404
>
> Problem accessing /proxy/application_1409246753441_0001/mapreduce.
> Reason:
>
> NOT_FOUND
>
> ------------------------------------------------------------------------
> /Powered by Jetty://
>
> /I use yarn.web-proxy because without yarn.web-proxy when I click
> ApplicationMaster link I will have loads of tcp connections with
> established statuses.
> In yarn-proxy log there is nothing interesting
>