You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by "Kartashov, Andy" <An...@mpac.ca> on 2012/11/29 15:46:22 UTC
RE: hadoop current properties
Many thanks.
Both suggestions worked well.
<ip-address>:port/conf - displayed all of the conf
Running $hadoop getcong -confKey <property> would tell me if changed property was picked up by the system.
Thanks!
p.s. where is the below property set, in the mapred-site.xml on the JT?
I currently have it as:
<name>mapred.task.tracker.report.address</name>
<value>127.0.0.1:0</value>
and wondering whether it is responsible for some of my broken webUI links.
AK
-----Original Message-----
From: Harsh J [mailto:harsh@cloudera.com]
Sent: Thursday, October 18, 2012 2:43 AM
To: user@hadoop.apache.org
Subject: Re: hadoop current properties
Hi Andy,
Sorry but I do not have a straight answer to this. Mostly, the division is between clients and servers. Which one were you asking for?
Higher versions of Hadoop have, for HDFS alone, a getconf method:
$ hdfs getconf <KEY>
All versions of Hadoop let you load and dump a Configuration object XML, which is helpful in seeing what form of core-site.xml values has it loaded from the classpath (or not):
$ hadoop org.apache.hadoop.conf.Configuration
Almost all versions of hadoop let you visit a daemon's loaded config page, that represents the config object the daemon is running with,
currently:
Visit http://DAEMONHOST:PORT/conf (http://namenode:50070/conf for example at the NN)
For job set information, the info Serge has passed earlier here is correct.
On Thu, Oct 18, 2012 at 2:29 AM, Kartashov, Andy <An...@mpac.ca> wrote:
> Is there a command-linein hadoop or Java methd to display all (if not individual) hadoop's current properties are set to?
>
> Rgds,
> AK
> NOTICE: This e-mail message and any attachments are confidential,
> subject to copyright and may be privileged. Any unauthorized use,
> copying or disclosure is prohibited. If you are not the intended
> recipient, please delete and contact the sender immediately. Please
> consider the environment before printing this e-mail. AVIS : le
> présent courriel et toute pièce jointe qui l'accompagne sont
> confidentiels, protégés par le droit d'auteur et peuvent être couverts
> par le secret professionnel. Toute utilisation, copie ou divulgation
> non autorisée est interdite. Si vous n'êtes pas le destinataire prévu
> de ce courriel, supprimez-le et contactez immédiatement l'expéditeur.
> Veuillez penser à l'environnement avant d'imprimer le présent courriel
--
Harsh J
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
Re: hadoop current properties
Posted by Hemanth Yamijala <yh...@thoughtworks.com>.
It is coming from the default properties file - mapred-default.xml. The
order of loading configuration in Hadoop is default.xml > site.xml >
job.xml.
<property>
<name>mapred.task.tracker.report.address</name>
<value>127.0.0.1:0</value>
<description>The interface and port that task tracker server listens on.
Since it is only connected to by the tasks, it uses the local interface.
EXPERT ONLY. Should only be changed if your host does not have the
loopback
interface.</description>
</property>
>From the description, I feel it should not cause any broken links for you.
Can you tell us which links are broken ?
Thanks
hemanth
On Thu, Nov 29, 2012 at 8:16 PM, Kartashov, Andy <An...@mpac.ca>wrote:
> Many thanks.
>
> Both suggestions worked well.
>
> <ip-address>:port/conf - displayed all of the conf
>
> Running $hadoop getcong -confKey <property> would tell me if changed
> property was picked up by the system.
>
> Thanks!
>
> p.s. where is the below property set, in the mapred-site.xml on the JT?
> I currently have it as:
> <name>mapred.task.tracker.report.address</name>
> <value>127.0.0.1:0</value>
> and wondering whether it is responsible for some of my broken webUI links.
>
> AK
> -----Original Message-----
> From: Harsh J [mailto:harsh@cloudera.com]
> Sent: Thursday, October 18, 2012 2:43 AM
> To: user@hadoop.apache.org
> Subject: Re: hadoop current properties
>
> Hi Andy,
>
> Sorry but I do not have a straight answer to this. Mostly, the division is
> between clients and servers. Which one were you asking for?
>
> Higher versions of Hadoop have, for HDFS alone, a getconf method:
>
> $ hdfs getconf <KEY>
>
> All versions of Hadoop let you load and dump a Configuration object XML,
> which is helpful in seeing what form of core-site.xml values has it loaded
> from the classpath (or not):
>
> $ hadoop org.apache.hadoop.conf.Configuration
>
> Almost all versions of hadoop let you visit a daemon's loaded config page,
> that represents the config object the daemon is running with,
> currently:
>
> Visit http://DAEMONHOST:PORT/conf (http://namenode:50070/conf for example
> at the NN)
>
> For job set information, the info Serge has passed earlier here is correct.
>
> On Thu, Oct 18, 2012 at 2:29 AM, Kartashov, Andy <An...@mpac.ca>
> wrote:
> > Is there a command-linein hadoop or Java methd to display all (if not
> individual) hadoop's current properties are set to?
> >
> > Rgds,
> > AK
> > NOTICE: This e-mail message and any attachments are confidential,
> > subject to copyright and may be privileged. Any unauthorized use,
> > copying or disclosure is prohibited. If you are not the intended
> > recipient, please delete and contact the sender immediately. Please
> > consider the environment before printing this e-mail. AVIS : le
> > présent courriel et toute pièce jointe qui l'accompagne sont
> > confidentiels, protégés par le droit d'auteur et peuvent être couverts
> > par le secret professionnel. Toute utilisation, copie ou divulgation
> > non autorisée est interdite. Si vous n'êtes pas le destinataire prévu
> > de ce courriel, supprimez-le et contactez immédiatement l'expéditeur.
> > Veuillez penser à l'environnement avant d'imprimer le présent courriel
>
>
>
> --
> Harsh J
> NOTICE: This e-mail message and any attachments are confidential, subject
> to copyright and may be privileged. Any unauthorized use, copying or
> disclosure is prohibited. If you are not the intended recipient, please
> delete and contact the sender immediately. Please consider the environment
> before printing this e-mail. AVIS : le présent courriel et toute pièce
> jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur
> et peuvent être couverts par le secret professionnel. Toute utilisation,
> copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le
> destinataire prévu de ce courriel, supprimez-le et contactez immédiatement
> l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent
> courriel
>
Re: hadoop current properties
Posted by Hemanth Yamijala <yh...@thoughtworks.com>.
It is coming from the default properties file - mapred-default.xml. The
order of loading configuration in Hadoop is default.xml > site.xml >
job.xml.
<property>
<name>mapred.task.tracker.report.address</name>
<value>127.0.0.1:0</value>
<description>The interface and port that task tracker server listens on.
Since it is only connected to by the tasks, it uses the local interface.
EXPERT ONLY. Should only be changed if your host does not have the
loopback
interface.</description>
</property>
>From the description, I feel it should not cause any broken links for you.
Can you tell us which links are broken ?
Thanks
hemanth
On Thu, Nov 29, 2012 at 8:16 PM, Kartashov, Andy <An...@mpac.ca>wrote:
> Many thanks.
>
> Both suggestions worked well.
>
> <ip-address>:port/conf - displayed all of the conf
>
> Running $hadoop getcong -confKey <property> would tell me if changed
> property was picked up by the system.
>
> Thanks!
>
> p.s. where is the below property set, in the mapred-site.xml on the JT?
> I currently have it as:
> <name>mapred.task.tracker.report.address</name>
> <value>127.0.0.1:0</value>
> and wondering whether it is responsible for some of my broken webUI links.
>
> AK
> -----Original Message-----
> From: Harsh J [mailto:harsh@cloudera.com]
> Sent: Thursday, October 18, 2012 2:43 AM
> To: user@hadoop.apache.org
> Subject: Re: hadoop current properties
>
> Hi Andy,
>
> Sorry but I do not have a straight answer to this. Mostly, the division is
> between clients and servers. Which one were you asking for?
>
> Higher versions of Hadoop have, for HDFS alone, a getconf method:
>
> $ hdfs getconf <KEY>
>
> All versions of Hadoop let you load and dump a Configuration object XML,
> which is helpful in seeing what form of core-site.xml values has it loaded
> from the classpath (or not):
>
> $ hadoop org.apache.hadoop.conf.Configuration
>
> Almost all versions of hadoop let you visit a daemon's loaded config page,
> that represents the config object the daemon is running with,
> currently:
>
> Visit http://DAEMONHOST:PORT/conf (http://namenode:50070/conf for example
> at the NN)
>
> For job set information, the info Serge has passed earlier here is correct.
>
> On Thu, Oct 18, 2012 at 2:29 AM, Kartashov, Andy <An...@mpac.ca>
> wrote:
> > Is there a command-linein hadoop or Java methd to display all (if not
> individual) hadoop's current properties are set to?
> >
> > Rgds,
> > AK
> > NOTICE: This e-mail message and any attachments are confidential,
> > subject to copyright and may be privileged. Any unauthorized use,
> > copying or disclosure is prohibited. If you are not the intended
> > recipient, please delete and contact the sender immediately. Please
> > consider the environment before printing this e-mail. AVIS : le
> > présent courriel et toute pièce jointe qui l'accompagne sont
> > confidentiels, protégés par le droit d'auteur et peuvent être couverts
> > par le secret professionnel. Toute utilisation, copie ou divulgation
> > non autorisée est interdite. Si vous n'êtes pas le destinataire prévu
> > de ce courriel, supprimez-le et contactez immédiatement l'expéditeur.
> > Veuillez penser à l'environnement avant d'imprimer le présent courriel
>
>
>
> --
> Harsh J
> NOTICE: This e-mail message and any attachments are confidential, subject
> to copyright and may be privileged. Any unauthorized use, copying or
> disclosure is prohibited. If you are not the intended recipient, please
> delete and contact the sender immediately. Please consider the environment
> before printing this e-mail. AVIS : le présent courriel et toute pièce
> jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur
> et peuvent être couverts par le secret professionnel. Toute utilisation,
> copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le
> destinataire prévu de ce courriel, supprimez-le et contactez immédiatement
> l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent
> courriel
>
Re: hadoop current properties
Posted by Hemanth Yamijala <yh...@thoughtworks.com>.
It is coming from the default properties file - mapred-default.xml. The
order of loading configuration in Hadoop is default.xml > site.xml >
job.xml.
<property>
<name>mapred.task.tracker.report.address</name>
<value>127.0.0.1:0</value>
<description>The interface and port that task tracker server listens on.
Since it is only connected to by the tasks, it uses the local interface.
EXPERT ONLY. Should only be changed if your host does not have the
loopback
interface.</description>
</property>
>From the description, I feel it should not cause any broken links for you.
Can you tell us which links are broken ?
Thanks
hemanth
On Thu, Nov 29, 2012 at 8:16 PM, Kartashov, Andy <An...@mpac.ca>wrote:
> Many thanks.
>
> Both suggestions worked well.
>
> <ip-address>:port/conf - displayed all of the conf
>
> Running $hadoop getcong -confKey <property> would tell me if changed
> property was picked up by the system.
>
> Thanks!
>
> p.s. where is the below property set, in the mapred-site.xml on the JT?
> I currently have it as:
> <name>mapred.task.tracker.report.address</name>
> <value>127.0.0.1:0</value>
> and wondering whether it is responsible for some of my broken webUI links.
>
> AK
> -----Original Message-----
> From: Harsh J [mailto:harsh@cloudera.com]
> Sent: Thursday, October 18, 2012 2:43 AM
> To: user@hadoop.apache.org
> Subject: Re: hadoop current properties
>
> Hi Andy,
>
> Sorry but I do not have a straight answer to this. Mostly, the division is
> between clients and servers. Which one were you asking for?
>
> Higher versions of Hadoop have, for HDFS alone, a getconf method:
>
> $ hdfs getconf <KEY>
>
> All versions of Hadoop let you load and dump a Configuration object XML,
> which is helpful in seeing what form of core-site.xml values has it loaded
> from the classpath (or not):
>
> $ hadoop org.apache.hadoop.conf.Configuration
>
> Almost all versions of hadoop let you visit a daemon's loaded config page,
> that represents the config object the daemon is running with,
> currently:
>
> Visit http://DAEMONHOST:PORT/conf (http://namenode:50070/conf for example
> at the NN)
>
> For job set information, the info Serge has passed earlier here is correct.
>
> On Thu, Oct 18, 2012 at 2:29 AM, Kartashov, Andy <An...@mpac.ca>
> wrote:
> > Is there a command-linein hadoop or Java methd to display all (if not
> individual) hadoop's current properties are set to?
> >
> > Rgds,
> > AK
> > NOTICE: This e-mail message and any attachments are confidential,
> > subject to copyright and may be privileged. Any unauthorized use,
> > copying or disclosure is prohibited. If you are not the intended
> > recipient, please delete and contact the sender immediately. Please
> > consider the environment before printing this e-mail. AVIS : le
> > présent courriel et toute pièce jointe qui l'accompagne sont
> > confidentiels, protégés par le droit d'auteur et peuvent être couverts
> > par le secret professionnel. Toute utilisation, copie ou divulgation
> > non autorisée est interdite. Si vous n'êtes pas le destinataire prévu
> > de ce courriel, supprimez-le et contactez immédiatement l'expéditeur.
> > Veuillez penser à l'environnement avant d'imprimer le présent courriel
>
>
>
> --
> Harsh J
> NOTICE: This e-mail message and any attachments are confidential, subject
> to copyright and may be privileged. Any unauthorized use, copying or
> disclosure is prohibited. If you are not the intended recipient, please
> delete and contact the sender immediately. Please consider the environment
> before printing this e-mail. AVIS : le présent courriel et toute pièce
> jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur
> et peuvent être couverts par le secret professionnel. Toute utilisation,
> copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le
> destinataire prévu de ce courriel, supprimez-le et contactez immédiatement
> l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent
> courriel
>
Re: hadoop current properties
Posted by Hemanth Yamijala <yh...@thoughtworks.com>.
It is coming from the default properties file - mapred-default.xml. The
order of loading configuration in Hadoop is default.xml > site.xml >
job.xml.
<property>
<name>mapred.task.tracker.report.address</name>
<value>127.0.0.1:0</value>
<description>The interface and port that task tracker server listens on.
Since it is only connected to by the tasks, it uses the local interface.
EXPERT ONLY. Should only be changed if your host does not have the
loopback
interface.</description>
</property>
>From the description, I feel it should not cause any broken links for you.
Can you tell us which links are broken ?
Thanks
hemanth
On Thu, Nov 29, 2012 at 8:16 PM, Kartashov, Andy <An...@mpac.ca>wrote:
> Many thanks.
>
> Both suggestions worked well.
>
> <ip-address>:port/conf - displayed all of the conf
>
> Running $hadoop getcong -confKey <property> would tell me if changed
> property was picked up by the system.
>
> Thanks!
>
> p.s. where is the below property set, in the mapred-site.xml on the JT?
> I currently have it as:
> <name>mapred.task.tracker.report.address</name>
> <value>127.0.0.1:0</value>
> and wondering whether it is responsible for some of my broken webUI links.
>
> AK
> -----Original Message-----
> From: Harsh J [mailto:harsh@cloudera.com]
> Sent: Thursday, October 18, 2012 2:43 AM
> To: user@hadoop.apache.org
> Subject: Re: hadoop current properties
>
> Hi Andy,
>
> Sorry but I do not have a straight answer to this. Mostly, the division is
> between clients and servers. Which one were you asking for?
>
> Higher versions of Hadoop have, for HDFS alone, a getconf method:
>
> $ hdfs getconf <KEY>
>
> All versions of Hadoop let you load and dump a Configuration object XML,
> which is helpful in seeing what form of core-site.xml values has it loaded
> from the classpath (or not):
>
> $ hadoop org.apache.hadoop.conf.Configuration
>
> Almost all versions of hadoop let you visit a daemon's loaded config page,
> that represents the config object the daemon is running with,
> currently:
>
> Visit http://DAEMONHOST:PORT/conf (http://namenode:50070/conf for example
> at the NN)
>
> For job set information, the info Serge has passed earlier here is correct.
>
> On Thu, Oct 18, 2012 at 2:29 AM, Kartashov, Andy <An...@mpac.ca>
> wrote:
> > Is there a command-linein hadoop or Java methd to display all (if not
> individual) hadoop's current properties are set to?
> >
> > Rgds,
> > AK
> > NOTICE: This e-mail message and any attachments are confidential,
> > subject to copyright and may be privileged. Any unauthorized use,
> > copying or disclosure is prohibited. If you are not the intended
> > recipient, please delete and contact the sender immediately. Please
> > consider the environment before printing this e-mail. AVIS : le
> > présent courriel et toute pièce jointe qui l'accompagne sont
> > confidentiels, protégés par le droit d'auteur et peuvent être couverts
> > par le secret professionnel. Toute utilisation, copie ou divulgation
> > non autorisée est interdite. Si vous n'êtes pas le destinataire prévu
> > de ce courriel, supprimez-le et contactez immédiatement l'expéditeur.
> > Veuillez penser à l'environnement avant d'imprimer le présent courriel
>
>
>
> --
> Harsh J
> NOTICE: This e-mail message and any attachments are confidential, subject
> to copyright and may be privileged. Any unauthorized use, copying or
> disclosure is prohibited. If you are not the intended recipient, please
> delete and contact the sender immediately. Please consider the environment
> before printing this e-mail. AVIS : le présent courriel et toute pièce
> jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur
> et peuvent être couverts par le secret professionnel. Toute utilisation,
> copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le
> destinataire prévu de ce courriel, supprimez-le et contactez immédiatement
> l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent
> courriel
>