You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Bruno Mahé (Created JIRA)" <ji...@apache.org> on 2011/11/19 03:30:52 UTC
[jira] [Created] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
jobhistory link may be broken depending on the interface it is listening on
---------------------------------------------------------------------------
Key: MAPREDUCE-3436
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: webapps
Affects Versions: 0.23.0, 0.23.1
Reporter: Bruno Mahé
On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
Am I missing something?
[root@bigtop-fedora-15 ~]# jps
9968 ResourceManager
1495 NameNode
1645 DataNode
12935 Jps
11140 -- process information unavailable
5309 JobHistoryServer
10237 NodeManager
[root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
tcp 0 0 :::8088 :::* LISTEN 9968/java
For reference, here is my configuration:
root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
<?xml version="1.0"?>
<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce.shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
<name>mapreduce.admin.user.env</name>
<value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
</property>
</configuration>
[root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
<?xml version="1.0"?>
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.permissions</name>
<value>false</value>
</property>
<property>
<!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
<name>dfs.name.dir</name>
<value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
</property>
</configuration>
[root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
<?xml version="1.0"?>
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:8020</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/var/lib/hadoop/cache/${user.name}</value>
</property>
<!-- OOZIE proxy user setting -->
<property>
<name>hadoop.proxyuser.oozie.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.oozie.groups</name>
<value>*</value>
</property>
</configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Robert Joseph Evans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13188712#comment-13188712 ]
Robert Joseph Evans commented on MAPREDUCE-3436:
------------------------------------------------
The patch looks good to me, but I would like to see a unit test for it, before I give a +1. It might even be nice to make the code common between this and the resource manager. IE provide an API that will take two configs, do the split and merge between them. That is minor though. Just a nice to have.
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Siddharth Seth (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166742#comment-13166742 ]
Siddharth Seth commented on MAPREDUCE-3436:
-------------------------------------------
Yep. Have it explicitly set to "hostname:port".
While the Job is running, the tracking URL (AM) is: http://0.0.0.0:8088/proxy/application_1323481362374_0001/
After the Job completes, this changes to : http://0.0.0.0:8088/proxy/application_1323481362374_0001/jobhistory/job/job_1323481362374_1_1
These links are from the http://<RESOURCE_MANAGER>:8088/cluster/apps page (Tracking UI column).
If i explicitly set the proxy address (yarn.web-proxy.address) - then the host:port changes to the value of yarn.web-proxy.address
Bruno, since you initially reported the issue, could you please verify if the patch fixes it. Thanks.
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Ahmed Radwan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189392#comment-13189392 ]
Ahmed Radwan commented on MAPREDUCE-3436:
-----------------------------------------
Thanks Siddarth, I am afraid I am misunderstanding something. To make sure we are talking about the same issue, and since this issue doesn't exist on a single node setup, will you be able to test the patch on a multinode setup (2 nodes for instance)? For example: running all services except HS on one node and HS on the second node. And see if you are seeing the same reported behavior with and without the patch (i.e. the JobHistory link is broken without the patch). You'll need to explicitly set mapreduce.jobhistory.address in both cases.
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) JobHistory webapp address
should use the host from the jobhistory address
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202829#comment-13202829 ]
Hudson commented on MAPREDUCE-3436:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1694 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1694/])
MAPREDUCE-3436. JobHistory webapp address should use the host configured in the jobhistory address. (Contributed by Ahmed Radwan)
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241620
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
> JobHistory webapp address should use the host from the jobhistory address
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Eli Collins (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated MAPREDUCE-3436:
-----------------------------------
Target Version/s: 0.23.1
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Ahmed Radwan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201053#comment-13201053 ]
Ahmed Radwan commented on MAPREDUCE-3436:
-----------------------------------------
@Mahadev,
There are two issues discussed here (please see my previous comment for description of the two issues). The current patch addresses issue#1.
I am planning also on looking into how to resolve issue#2 which Siddharth highlighted. I was busy so didn't have time to look into it, but planning to do so in the coming few days.
So if there is urgency in pushing the current patch (which seems to be accepted by Siddharth and Bruno), I can create another ticket to follow issue#2. If there is no urgency then I'll investigate issue#2 in the coming days and update the current patch. Thanks!
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Bruno Mahé (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162985#comment-13162985 ]
Bruno Mahé commented on MAPREDUCE-3436:
---------------------------------------
Does that mean a user cannot have this service listen on all interfaces without breaking the ui?
But that would also mean the jobhistory link is broken by default, unless its config is updated by the user to point to the correct ip/hostname.
I also don't understand why other web ui links are able to figure out the right hostname/ip by themselves without any extra configuration. I haven't had time to figure out how it's done exactly, but the jobhistory is the only place where such configuration is needed.
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) JobHistory webapp address
should use the host from the jobhistory address
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202787#comment-13202787 ]
Hudson commented on MAPREDUCE-3436:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1755 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1755/])
MAPREDUCE-3436. JobHistory webapp address should use the host configured in the jobhistory address. (Contributed by Ahmed Radwan)
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241620
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
> JobHistory webapp address should use the host from the jobhistory address
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Bruno Mahé (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13188200#comment-13188200 ]
Bruno Mahé commented on MAPREDUCE-3436:
---------------------------------------
Sorry for the delay but I can finally confirm Ahmed patch fixes the issue.
I tested this with a cluster of 2 machines where {namenode, datanode, resourcemanager, nodemanager} run on one machine and {historyserver} run on the other.
Without Ahmed's patch the history server link from a finished job does not point to the correct historyserver and point to the first machine where no historyserver run.
With Ahmed's patch, the history server link from a finished job do point to the correct historyserver and works like a charm.
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Ahmed Radwan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166721#comment-13166721 ]
Ahmed Radwan commented on MAPREDUCE-3436:
-----------------------------------------
Siddarth, when you were still seeing 0.0.0.0, did you have mapreduce.jobhistory.address explicitly set?
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) JobHistory webapp address
should use the host from the jobhistory address
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203604#comment-13203604 ]
Hudson commented on MAPREDUCE-3436:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #983 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/983/])
MAPREDUCE-3436. JobHistory webapp address should use the host configured in the jobhistory address. (Contributed by Ahmed Radwan)
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241620
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
> JobHistory webapp address should use the host from the jobhistory address
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Siddharth Seth (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189351#comment-13189351 ]
Siddharth Seth commented on MAPREDUCE-3436:
-------------------------------------------
Bruno, on the node running the RM - were either of "yarn.resourcemanager.address" or "yarn.web-proxy.address" explicitly set ?
I assume the history server address (mapreduce.jobhistory.address) was set to point to the second node, and "mapreduce.jobhistory.webapp.address" was not set.
The changes to have the history webapp address pick the host from "mapreduce.jobhistory.address" look good.
Other than that - I still can't see how this patch fixes the link being 0.0.0.0 .
Looks like YarnConfiguration.getProxyHostAndPort() will return the default (0.0.0.0), unless the RM address / web-proxy address are configured. That call is used to construct the actual proxy url - which is what causes the link to be http://0.0.0.0:8088/proxy/* .
Setup that I'm using. Single node - "yarn.resourcemanager.address", "yarn.web-proxy.address" not set.
If that's not the behaviour others are seing - i can create that as a separate jira. This one is really just making sure the history webapp address host is being up form "mapreduce.jobhistory.address" instead of "mapreduce.jobhistory.webapp.address"
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3436:
-------------------------------------
Status: Open (was: Patch Available)
@Ahmed,
Looks good. Minor nit - Though the properties names are good enough explanation in themselves, can we add a description for these properties?
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Ahmed Radwan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162567#comment-13162567 ]
Ahmed Radwan commented on MAPREDUCE-3436:
-----------------------------------------
Bruno, I checked this issue, the default mapreduce jobhistory webapp address is set to 0.0.0.0:19888, and this is why you are hitting this link problem. Can you try setting the property: mapreduce.jobhistory.webapp.address in your cluster conf files <hsAddress>:<port>, hsAddress is where you started your history server, and choose any port of your choice (doesn't need to be the default 19888). Can you try that?
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Ahmed Radwan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Radwan updated MAPREDUCE-3436:
------------------------------------
Status: Patch Available (was: Open)
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) JobHistory webapp address
should use the host from the jobhistory address
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202794#comment-13202794 ]
Hudson commented on MAPREDUCE-3436:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #1682 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1682/])
MAPREDUCE-3436. JobHistory webapp address should use the host configured in the jobhistory address. (Contributed by Ahmed Radwan)
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241620
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
> JobHistory webapp address should use the host from the jobhistory address
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3436) JobHistory webapp address should
use the host from the jobhistory address
Posted by "Siddharth Seth (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3436:
--------------------------------------
Resolution: Fixed
Fix Version/s: 0.23.1
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to trunk and branch-0.23. Thanks Ahmed.
Will try posting a patch to MAPREDUCE-3828 later today.
> JobHistory webapp address should use the host from the jobhistory address
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) JobHistory webapp address
should use the host from the jobhistory address
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203578#comment-13203578 ]
Hudson commented on MAPREDUCE-3436:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #185 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/185/])
merge MAPREDUCE-3436 from trunk
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241621
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
> JobHistory webapp address should use the host from the jobhistory address
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Ahmed Radwan (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Radwan reassigned MAPREDUCE-3436:
---------------------------------------
Assignee: Ahmed Radwan
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) JobHistory webapp address
should use the host from the jobhistory address
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203534#comment-13203534 ]
Hudson commented on MAPREDUCE-3436:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #950 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/950/])
MAPREDUCE-3436. JobHistory webapp address should use the host configured in the jobhistory address. (Contributed by Ahmed Radwan)
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241620
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
> JobHistory webapp address should use the host from the jobhistory address
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Ahmed Radwan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13190021#comment-13190021 ]
Ahmed Radwan commented on MAPREDUCE-3436:
-----------------------------------------
@Siddarth Thanks for our offline chat yesterday. I'll summarize my understanding and please correct me if I am wrong. I think we are speaking about two separate issues:
Issue1: When the user explicitly set yarn.resourcemanager.address and mapreduce.jobhistory.address, the jobhistory link is broken (without the patch), the patch basically avoids the need to explicitly set mapreduce.jobhistory.webapp.address. This issue doesn't exist in the single node case because all services run on the single node.
Issue2: If the user doesn't explicitly set yarn.resourcemanager.address, then all links (AM tracking url and jobhistory url) are broken. This issue even exists on a single node setup as you mentioned (if the user tries to connect from another node).
However, I am not sure if Issue2 is a bug or not: For a single node setup, if yarn.resourcemanager.address is not set, the user can still play with the web ui if connecting from the same single node, and all links will work. On a real cluster (multinode), it is expected/required to set the yarn.resourcemanager.address and mapreduce.jobhistory.address for the cluster to function, and so, with the patch, all links will also work.
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) JobHistory webapp address
should use the host from the jobhistory address
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202793#comment-13202793 ]
Hudson commented on MAPREDUCE-3436:
-----------------------------------
Integrated in Hadoop-Common-0.23-Commit #508 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/508/])
merge MAPREDUCE-3436 from trunk
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241621
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
> JobHistory webapp address should use the host from the jobhistory address
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3436) JobHistory webapp address should
use the host from the jobhistory address
Posted by "Siddharth Seth (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3436:
--------------------------------------
Summary: JobHistory webapp address should use the host from the jobhistory address (was: jobhistory link may be broken depending on the interface it is listening on)
Changing the description to reflect the change. Will commit this shortly.
> JobHistory webapp address should use the host from the jobhistory address
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162731#comment-13162731 ]
Hadoop QA commented on MAPREDUCE-3436:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12506104/MAPREDUCE-3436.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1391//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1391//console
This message is automatically generated.
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Bruno Mahé (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189361#comment-13189361 ]
Bruno Mahé commented on MAPREDUCE-3436:
---------------------------------------
Siddarth> "yarn.resourcemanager.address" was set and "mapreduce.jobhistory.address" was indeed set to pointed to the second host. "mapreduce.jobhistory.webapp.address" was not set.
And also during my tests, the link was not set to 0.0.0.0 anymore, but to the first host.
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Siddharth Seth (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191354#comment-13191354 ]
Siddharth Seth commented on MAPREDUCE-3436:
-------------------------------------------
Yep, two separate issues.
bq. Issue1:
The patch fixes this. (avoiding explicitly setting the history webapp address)
bq. Issue2: If the user doesn't explicitly set yarn.resourcemanager.address, then all links (AM tracking url and jobhistory url) are broken. This issue even exists on a single node setup as you mentioned (if the user tries to connect from another node).
>From the jira description and yarn behaviour - I believe this was what the jira was created for. This gets annoying when running a single node cluster on a remote node. Could you please update the patch for this as well - it's a pretty simple change. Otherwise I can do that. Thanks.
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) JobHistory webapp address
should use the host from the jobhistory address
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203554#comment-13203554 ]
Hudson commented on MAPREDUCE-3436:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #163 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/163/])
merge MAPREDUCE-3436 from trunk
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241621
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
> JobHistory webapp address should use the host from the jobhistory address
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) JobHistory webapp address
should use the host from the jobhistory address
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202830#comment-13202830 ]
Hudson commented on MAPREDUCE-3436:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Commit #515 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/515/])
merge MAPREDUCE-3436 from trunk
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241621
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
> JobHistory webapp address should use the host from the jobhistory address
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200494#comment-13200494 ]
Mahadev konar commented on MAPREDUCE-3436:
------------------------------------------
@Ahmed/Sid,
Any update on this one?
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Siddharth Seth (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164142#comment-13164142 ]
Siddharth Seth commented on MAPREDUCE-3436:
-------------------------------------------
Like Tom mentioned, explicitly setting yarn.resourcemanager.webapp.address should fix this. (yarn.resourcemanager.address also needs to be set - YarnConfiguration.getRMWebAppHostAndPort for some reason uses the host from this and the port from the webapp address).
This isn't limited to history urls only - you should see the same issue with the AM address while it's running.
Another option may be to have RM startup set the address correctly - if it's a local address. JobHistory.getHistoryUrl tries to do this.
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Ahmed Radwan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166623#comment-13166623 ]
Ahmed Radwan commented on MAPREDUCE-3436:
-----------------------------------------
Siddarth, Here is a description of what I am seeing with and without the patch:
- Without the patch, the tracking ui history link points (for example) to: http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2, and this link is broken if you try to follow it. (this complies with Bruno's observation above).
- With the patch, the tracking ui history link points to: http://<hostname>:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
and if you follow this link, it is not broken, and it displays the page: http://<hostname>:19888/jobhistory/job/job_1323304600508_6_6/jobhistory/job/job_1323304600508_6_6
<hostname> above is the node running both the resource manager and history server.
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Ahmed Radwan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Radwan updated MAPREDUCE-3436:
------------------------------------
Attachment: MAPREDUCE-3436.patch
This patch exposes the default values for the conf properties: "mapreduce.jobhistory.address" and "mapreduce.jobhistory.webapp.address" as part of the mapred-default.xml.
This will make it easier to identify these properties, their default values, and - if needed - override these values in your *-site.xml conf files.
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3436:
-------------------------------------
Component/s: mrv2
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Ahmed Radwan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Radwan updated MAPREDUCE-3436:
------------------------------------
Attachment: MAPREDUCE-3436_rev2.patch
Thanks Mahadev, Bruno and Siddharth.
Explicitly setting yarn.resourcemanager.webapp.address and yarn.resourcemanager.address doesn't fix the broken JobHistory link. Only explicitly setting mapreduce.jobhistory.webapp.address seems to fix this issue.
Looking more into this, I think the problem is in JobHistoryUtils.getHistoryUrl(..) which uses the default 0.0.0.0 jh webapp address value to return the localhost address, while it needs to do something similar to what YarnConfiguration.getRMWebAppHostAndPort is doing; using the hostname from mapreduce.jobhistory.address and port from mapreduce.jobhistory.webapp.address.
I have made these changes in the attached patch. It is now working without the need to explicitly set mapreduce.jobhistory.webapp.address.
Per Mahadev comment, I have also added brief descriptions to the properties in mapred-default.xml.
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Ahmed Radwan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Radwan updated MAPREDUCE-3436:
------------------------------------
Status: Patch Available (was: Open)
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Ahmed Radwan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202033#comment-13202033 ]
Ahmed Radwan commented on MAPREDUCE-3436:
-----------------------------------------
Thanks Roman, I have created MAPREDUCE-3828 to track issue#2.
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Ahmed Radwan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Radwan updated MAPREDUCE-3436:
------------------------------------
Status: Patch Available (was: Open)
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Siddharth Seth (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189435#comment-13189435 ]
Siddharth Seth commented on MAPREDUCE-3436:
-------------------------------------------
@Bruno - "the link was not 0.0.0.0" - that's with and without the patch, correct ? With the patch - it goes to the correct history server (second node). Without the patch, the history url is the same as the ResourceManager host ?
bq.
There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
@Ahmed, that's from the jira description - and I'm assuming that's what is being fixed. This exists in a single node cluster.
On the 2 node deployment that you're using - could you try not setting "yarn.resourcemanager.address", and "yarn.web-proxy.address" to see if the tracker url is something other than 0.0.0.0.
For the scenario Bruno has described in his previous comment - rm address set, history address set, history webapp address not set - the patch is good.
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) JobHistory webapp address
should use the host from the jobhistory address
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202784#comment-13202784 ]
Hudson commented on MAPREDUCE-3436:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Commit #498 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/498/])
merge MAPREDUCE-3436 from trunk
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241621
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
> JobHistory webapp address should use the host from the jobhistory address
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164763#comment-13164763 ]
Hadoop QA commented on MAPREDUCE-3436:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12506450/MAPREDUCE-3436_rev2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1408//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1408//console
This message is automatically generated.
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Siddharth Seth (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164795#comment-13164795 ]
Siddharth Seth commented on MAPREDUCE-3436:
-------------------------------------------
Ahmed, I tried the patch. The Tracking URL (AM or History) for the app on the RM UI http://<RESOURCE_MANAGER>:8088/cluster/apps - is still using 0.0.0.0.
Env: No proxy address, no rm/rm webapp address, with and without mapreduce.jobhistory.address configured.
The change I do see with the patch is for the non proxied history url - if 'mapreduce.jobhistory.address' is set , the non proxied history url uses this instead of the host's IP address.
Am I missing something, maybe different issues ?
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Roman Shaposhnik (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201851#comment-13201851 ]
Roman Shaposhnik commented on MAPREDUCE-3436:
---------------------------------------------
@Ahmed
I think there's a value in pushing this patch as-is and having a separate issue tracking #2. Can you, please, do that?
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Siddharth Seth (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3436:
--------------------------------------
Status: Open (was: Patch Available)
Canceling patch, till there's additional information on whether it fixes this issue.
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken
depending on the interface it is listening on
Posted by "Tom White (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161069#comment-13161069 ]
Tom White commented on MAPREDUCE-3436:
--------------------------------------
You could try setting yarn.resourcemanager.webapp.address to a host-port pair.
> jobhistory link may be broken depending on the interface it is listening on
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: webapps
> Affects Versions: 0.23.0, 0.23.1
> Reporter: Bruno Mahé
> Labels: bigtop
>
> On the following page : http://<RESOURCE_MANAGER>:8088/cluster/apps
> There are links to the history for each application. None of them can be reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp 0 0 :::8088 :::* LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
> <?xml version="1.0"?>
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.permissions</name>
> <value>false</value>
> </property>
> <property>
> <!-- specify this so that running 'hadoop namenode -format' formats the right dir -->
> <name>dfs.name.dir</name>
> <value>/var/lib/hadoop/cache/hadoop/dfs/name</value>
> </property>
> </configuration>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:8020</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/var/lib/hadoop/cache/${user.name}</value>
> </property>
> <!-- OOZIE proxy user setting -->
> <property>
> <name>hadoop.proxyuser.oozie.hosts</name>
> <value>*</value>
> </property>
> <property>
> <name>hadoop.proxyuser.oozie.groups</name>
> <value>*</value>
> </property>
> </configuration>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira