You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Federico Baldo <ba...@gmail.com> on 2014/04/05 13:17:09 UTC
customize containter-log4j.properties
Hi there,
I am trying to customize the log format of containers, as a first step I
have tried using a containter-log4j.properties with a different value for
log4j.appender.CLA.layout.ConversionPattern located in the container's
classpath. Tthe result is that the syslog file is written to:
/mnt/var/lib/hadoop/tmp/nm-local-dir/usercache/hadoop/appcache/application_1396678720329_0009/container_1396678720329_0009_01_000009/null/syslog
instead of the configured yarn.app.mapreduce.container.log.dir that is
specified in the cmdline as :
-Dyarn.app.container.log.dir=/mnt/var/log/hadoop/userlogs/application_1396678720329_0009/container_1396678720329_0009_01_000009
Additionally I can see that in Hadoop 2.2.0 the
log4j.configuration=container-log4j.properties is set in
MRApps.addLog4jSystemProperties() and cannot be modified:
http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-mapreduce-client-common/2.2.0/org/apache/hadoop/mapreduce/v2/util/MRApps.java#457
Do you have any idea on how to possibly customize this configuration file?
Thanks
--
Federico Baldo
Re: customize containter-log4j.properties
Posted by Federico Baldo <ba...@gmail.com>.
Thank you for the feedback on that. Though I don't understand why the
container-log4j.properties should be included in the
hadoop-yarn-server-nodemanager and not as a standard configuration file in
the configuration directory, is there any specific requirements for this
config file that I'm missing?
Thanks,
Federico
2014-04-05 18:13 GMT+01:00 Harsh J <ha...@cloudera.com>:
> This is intentional, and only within MR2's scope - we do not support
> configuring the formatting, but do support switching the levels (via
> job properties). You should still be able to customise your own YARN
> apps' logging configuration though.
>
> On Sat, Apr 5, 2014 at 7:09 PM, Federico Baldo <ba...@gmail.com>
> wrote:
> > Hi there,
> > I am trying to customize the log format of containers, as a first step I
> > have tried using a containter-log4j.properties with a different value
> for
> > log4j.appender.CLA.layout.ConversionPattern located in the container's
> > classpath. Tthe result is that the syslog file is written to:
> >
> >
> /mnt/var/lib/hadoop/tmp/nm-local-dir/usercache/hadoop/appcache/application_1396678720329_0009/container_1396678720329_0009_01_000009/null/syslog
> >
> > instead of the configured yarn.app.mapreduce.container.log.dir that is
> > specified in the cmdline as :
> >
> >
> -Dyarn.app.container.log.dir=/mnt/var/log/hadoop/userlogs/application_1396678720329_0009/container_1396678720329_0009_01_000009
> >
> > Additionally I can see that in Hadoop 2.2.0 the
> > log4j.configuration=container-log4j.properties is set in
> > MRApps.addLog4jSystemProperties() and cannot be modified:
> >
> http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-mapreduce-client-common/2.2.0/org/apache/hadoop/mapreduce/v2/util/MRApps.java#457
> >
> > Do you have any idea on how to possibly customize this configuration
> file?
> >
> > Thanks
> > --
> > Federico Baldo
>
>
>
> --
> Harsh J
>
--
Federico Baldo
Re: customize containter-log4j.properties
Posted by Federico Baldo <ba...@gmail.com>.
Thank you for the feedback on that. Though I don't understand why the
container-log4j.properties should be included in the
hadoop-yarn-server-nodemanager and not as a standard configuration file in
the configuration directory, is there any specific requirements for this
config file that I'm missing?
Thanks,
Federico
2014-04-05 18:13 GMT+01:00 Harsh J <ha...@cloudera.com>:
> This is intentional, and only within MR2's scope - we do not support
> configuring the formatting, but do support switching the levels (via
> job properties). You should still be able to customise your own YARN
> apps' logging configuration though.
>
> On Sat, Apr 5, 2014 at 7:09 PM, Federico Baldo <ba...@gmail.com>
> wrote:
> > Hi there,
> > I am trying to customize the log format of containers, as a first step I
> > have tried using a containter-log4j.properties with a different value
> for
> > log4j.appender.CLA.layout.ConversionPattern located in the container's
> > classpath. Tthe result is that the syslog file is written to:
> >
> >
> /mnt/var/lib/hadoop/tmp/nm-local-dir/usercache/hadoop/appcache/application_1396678720329_0009/container_1396678720329_0009_01_000009/null/syslog
> >
> > instead of the configured yarn.app.mapreduce.container.log.dir that is
> > specified in the cmdline as :
> >
> >
> -Dyarn.app.container.log.dir=/mnt/var/log/hadoop/userlogs/application_1396678720329_0009/container_1396678720329_0009_01_000009
> >
> > Additionally I can see that in Hadoop 2.2.0 the
> > log4j.configuration=container-log4j.properties is set in
> > MRApps.addLog4jSystemProperties() and cannot be modified:
> >
> http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-mapreduce-client-common/2.2.0/org/apache/hadoop/mapreduce/v2/util/MRApps.java#457
> >
> > Do you have any idea on how to possibly customize this configuration
> file?
> >
> > Thanks
> > --
> > Federico Baldo
>
>
>
> --
> Harsh J
>
--
Federico Baldo
Re: customize containter-log4j.properties
Posted by Federico Baldo <ba...@gmail.com>.
Thank you for the feedback on that. Though I don't understand why the
container-log4j.properties should be included in the
hadoop-yarn-server-nodemanager and not as a standard configuration file in
the configuration directory, is there any specific requirements for this
config file that I'm missing?
Thanks,
Federico
2014-04-05 18:13 GMT+01:00 Harsh J <ha...@cloudera.com>:
> This is intentional, and only within MR2's scope - we do not support
> configuring the formatting, but do support switching the levels (via
> job properties). You should still be able to customise your own YARN
> apps' logging configuration though.
>
> On Sat, Apr 5, 2014 at 7:09 PM, Federico Baldo <ba...@gmail.com>
> wrote:
> > Hi there,
> > I am trying to customize the log format of containers, as a first step I
> > have tried using a containter-log4j.properties with a different value
> for
> > log4j.appender.CLA.layout.ConversionPattern located in the container's
> > classpath. Tthe result is that the syslog file is written to:
> >
> >
> /mnt/var/lib/hadoop/tmp/nm-local-dir/usercache/hadoop/appcache/application_1396678720329_0009/container_1396678720329_0009_01_000009/null/syslog
> >
> > instead of the configured yarn.app.mapreduce.container.log.dir that is
> > specified in the cmdline as :
> >
> >
> -Dyarn.app.container.log.dir=/mnt/var/log/hadoop/userlogs/application_1396678720329_0009/container_1396678720329_0009_01_000009
> >
> > Additionally I can see that in Hadoop 2.2.0 the
> > log4j.configuration=container-log4j.properties is set in
> > MRApps.addLog4jSystemProperties() and cannot be modified:
> >
> http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-mapreduce-client-common/2.2.0/org/apache/hadoop/mapreduce/v2/util/MRApps.java#457
> >
> > Do you have any idea on how to possibly customize this configuration
> file?
> >
> > Thanks
> > --
> > Federico Baldo
>
>
>
> --
> Harsh J
>
--
Federico Baldo
Re: customize containter-log4j.properties
Posted by Federico Baldo <ba...@gmail.com>.
Thank you for the feedback on that. Though I don't understand why the
container-log4j.properties should be included in the
hadoop-yarn-server-nodemanager and not as a standard configuration file in
the configuration directory, is there any specific requirements for this
config file that I'm missing?
Thanks,
Federico
2014-04-05 18:13 GMT+01:00 Harsh J <ha...@cloudera.com>:
> This is intentional, and only within MR2's scope - we do not support
> configuring the formatting, but do support switching the levels (via
> job properties). You should still be able to customise your own YARN
> apps' logging configuration though.
>
> On Sat, Apr 5, 2014 at 7:09 PM, Federico Baldo <ba...@gmail.com>
> wrote:
> > Hi there,
> > I am trying to customize the log format of containers, as a first step I
> > have tried using a containter-log4j.properties with a different value
> for
> > log4j.appender.CLA.layout.ConversionPattern located in the container's
> > classpath. Tthe result is that the syslog file is written to:
> >
> >
> /mnt/var/lib/hadoop/tmp/nm-local-dir/usercache/hadoop/appcache/application_1396678720329_0009/container_1396678720329_0009_01_000009/null/syslog
> >
> > instead of the configured yarn.app.mapreduce.container.log.dir that is
> > specified in the cmdline as :
> >
> >
> -Dyarn.app.container.log.dir=/mnt/var/log/hadoop/userlogs/application_1396678720329_0009/container_1396678720329_0009_01_000009
> >
> > Additionally I can see that in Hadoop 2.2.0 the
> > log4j.configuration=container-log4j.properties is set in
> > MRApps.addLog4jSystemProperties() and cannot be modified:
> >
> http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-mapreduce-client-common/2.2.0/org/apache/hadoop/mapreduce/v2/util/MRApps.java#457
> >
> > Do you have any idea on how to possibly customize this configuration
> file?
> >
> > Thanks
> > --
> > Federico Baldo
>
>
>
> --
> Harsh J
>
--
Federico Baldo
Re: customize containter-log4j.properties
Posted by Harsh J <ha...@cloudera.com>.
This is intentional, and only within MR2's scope - we do not support
configuring the formatting, but do support switching the levels (via
job properties). You should still be able to customise your own YARN
apps' logging configuration though.
On Sat, Apr 5, 2014 at 7:09 PM, Federico Baldo <ba...@gmail.com> wrote:
> Hi there,
> I am trying to customize the log format of containers, as a first step I
> have tried using a containter-log4j.properties with a different value for
> log4j.appender.CLA.layout.ConversionPattern located in the container's
> classpath. Tthe result is that the syslog file is written to:
>
> /mnt/var/lib/hadoop/tmp/nm-local-dir/usercache/hadoop/appcache/application_1396678720329_0009/container_1396678720329_0009_01_000009/null/syslog
>
> instead of the configured yarn.app.mapreduce.container.log.dir that is
> specified in the cmdline as :
>
> -Dyarn.app.container.log.dir=/mnt/var/log/hadoop/userlogs/application_1396678720329_0009/container_1396678720329_0009_01_000009
>
> Additionally I can see that in Hadoop 2.2.0 the
> log4j.configuration=container-log4j.properties is set in
> MRApps.addLog4jSystemProperties() and cannot be modified:
> http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-mapreduce-client-common/2.2.0/org/apache/hadoop/mapreduce/v2/util/MRApps.java#457
>
> Do you have any idea on how to possibly customize this configuration file?
>
> Thanks
> --
> Federico Baldo
--
Harsh J
Re: customize containter-log4j.properties
Posted by Harsh J <ha...@cloudera.com>.
This is intentional, and only within MR2's scope - we do not support
configuring the formatting, but do support switching the levels (via
job properties). You should still be able to customise your own YARN
apps' logging configuration though.
On Sat, Apr 5, 2014 at 7:09 PM, Federico Baldo <ba...@gmail.com> wrote:
> Hi there,
> I am trying to customize the log format of containers, as a first step I
> have tried using a containter-log4j.properties with a different value for
> log4j.appender.CLA.layout.ConversionPattern located in the container's
> classpath. Tthe result is that the syslog file is written to:
>
> /mnt/var/lib/hadoop/tmp/nm-local-dir/usercache/hadoop/appcache/application_1396678720329_0009/container_1396678720329_0009_01_000009/null/syslog
>
> instead of the configured yarn.app.mapreduce.container.log.dir that is
> specified in the cmdline as :
>
> -Dyarn.app.container.log.dir=/mnt/var/log/hadoop/userlogs/application_1396678720329_0009/container_1396678720329_0009_01_000009
>
> Additionally I can see that in Hadoop 2.2.0 the
> log4j.configuration=container-log4j.properties is set in
> MRApps.addLog4jSystemProperties() and cannot be modified:
> http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-mapreduce-client-common/2.2.0/org/apache/hadoop/mapreduce/v2/util/MRApps.java#457
>
> Do you have any idea on how to possibly customize this configuration file?
>
> Thanks
> --
> Federico Baldo
--
Harsh J
Re: customize containter-log4j.properties
Posted by Harsh J <ha...@cloudera.com>.
This is intentional, and only within MR2's scope - we do not support
configuring the formatting, but do support switching the levels (via
job properties). You should still be able to customise your own YARN
apps' logging configuration though.
On Sat, Apr 5, 2014 at 7:09 PM, Federico Baldo <ba...@gmail.com> wrote:
> Hi there,
> I am trying to customize the log format of containers, as a first step I
> have tried using a containter-log4j.properties with a different value for
> log4j.appender.CLA.layout.ConversionPattern located in the container's
> classpath. Tthe result is that the syslog file is written to:
>
> /mnt/var/lib/hadoop/tmp/nm-local-dir/usercache/hadoop/appcache/application_1396678720329_0009/container_1396678720329_0009_01_000009/null/syslog
>
> instead of the configured yarn.app.mapreduce.container.log.dir that is
> specified in the cmdline as :
>
> -Dyarn.app.container.log.dir=/mnt/var/log/hadoop/userlogs/application_1396678720329_0009/container_1396678720329_0009_01_000009
>
> Additionally I can see that in Hadoop 2.2.0 the
> log4j.configuration=container-log4j.properties is set in
> MRApps.addLog4jSystemProperties() and cannot be modified:
> http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-mapreduce-client-common/2.2.0/org/apache/hadoop/mapreduce/v2/util/MRApps.java#457
>
> Do you have any idea on how to possibly customize this configuration file?
>
> Thanks
> --
> Federico Baldo
--
Harsh J
Re: customize containter-log4j.properties
Posted by Harsh J <ha...@cloudera.com>.
This is intentional, and only within MR2's scope - we do not support
configuring the formatting, but do support switching the levels (via
job properties). You should still be able to customise your own YARN
apps' logging configuration though.
On Sat, Apr 5, 2014 at 7:09 PM, Federico Baldo <ba...@gmail.com> wrote:
> Hi there,
> I am trying to customize the log format of containers, as a first step I
> have tried using a containter-log4j.properties with a different value for
> log4j.appender.CLA.layout.ConversionPattern located in the container's
> classpath. Tthe result is that the syslog file is written to:
>
> /mnt/var/lib/hadoop/tmp/nm-local-dir/usercache/hadoop/appcache/application_1396678720329_0009/container_1396678720329_0009_01_000009/null/syslog
>
> instead of the configured yarn.app.mapreduce.container.log.dir that is
> specified in the cmdline as :
>
> -Dyarn.app.container.log.dir=/mnt/var/log/hadoop/userlogs/application_1396678720329_0009/container_1396678720329_0009_01_000009
>
> Additionally I can see that in Hadoop 2.2.0 the
> log4j.configuration=container-log4j.properties is set in
> MRApps.addLog4jSystemProperties() and cannot be modified:
> http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-mapreduce-client-common/2.2.0/org/apache/hadoop/mapreduce/v2/util/MRApps.java#457
>
> Do you have any idea on how to possibly customize this configuration file?
>
> Thanks
> --
> Federico Baldo
--
Harsh J
Fwd: customize containter-log4j.properties
Posted by Federico Baldo <ba...@gmail.com>.
Hi there,
I am trying to customize the log format of containers, as a first step I
have tried using a containter-log4j.properties with a different value for
log4j.appender.CLA.layout.ConversionPattern located in the container's
classpath. Tthe result is that the syslog file is written to:
/mnt/var/lib/hadoop/tmp/nm-local-dir/usercache/hadoop/appcache/application_1396678720329_0009/container_1396678720329_0009_01_000009/null/syslog
instead of the configured yarn.app.mapreduce.container.log.dir that is
specified in the cmdline as :
-Dyarn.app.container.log.dir=/mnt/var/log/hadoop/userlogs/application_1396678720329_0009/container_1396678720329_0009_01_000009
Additionally I can see that in Hadoop 2.2.0 the
log4j.configuration=container-log4j.properties is set in
MRApps.addLog4jSystemProperties() and cannot be modified:
http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-mapreduce-client-common/2.2.0/org/apache/hadoop/mapreduce/v2/util/MRApps.java#457
Do you have any idea on how to possibly customize this configuration file?
Thanks
--
Federico Baldo
hadoop eclipse plugin compile path
Posted by Alex Lee <el...@hotmail.com>.
Trying to use the below command to generate hadoop eclipse plugin, but seem the directory =/usr/local/hadoop-2.2.0 not correct. I just used the ambari to installed the hadoop.
$ant jar -Dversion=2.2.0 -Declipse.home=/usr/local/eclipse -Dhadoop.home=/usr/local/hadoop-2.2.0
error log
BUILD FAILED
/usr/local/hadoop2x-eclipse-plugin/src/contrib/eclipse-plugin/build.xml:76: /usr/local/hadoop-2.2.0/share/hadoop/mapreduce does not exist.
May I know where is the install path for hadoop.
Any suggestion, thanks.
hadoop eclipse plugin compile path
Posted by Alex Lee <el...@hotmail.com>.
Trying to use the below command to generate hadoop eclipse plugin, but seem the directory =/usr/local/hadoop-2.2.0 not correct. I just used the ambari to installed the hadoop.
$ant jar -Dversion=2.2.0 -Declipse.home=/usr/local/eclipse -Dhadoop.home=/usr/local/hadoop-2.2.0
error log
BUILD FAILED
/usr/local/hadoop2x-eclipse-plugin/src/contrib/eclipse-plugin/build.xml:76: /usr/local/hadoop-2.2.0/share/hadoop/mapreduce does not exist.
May I know where is the install path for hadoop.
Any suggestion, thanks.
hadoop eclipse plugin compile path
Posted by Alex Lee <el...@hotmail.com>.
Trying to use the below command to generate hadoop eclipse plugin, but seem the directory =/usr/local/hadoop-2.2.0 not correct. I just used the ambari to installed the hadoop.
$ant jar -Dversion=2.2.0 -Declipse.home=/usr/local/eclipse -Dhadoop.home=/usr/local/hadoop-2.2.0
error log
BUILD FAILED
/usr/local/hadoop2x-eclipse-plugin/src/contrib/eclipse-plugin/build.xml:76: /usr/local/hadoop-2.2.0/share/hadoop/mapreduce does not exist.
May I know where is the install path for hadoop.
Any suggestion, thanks.
hadoop eclipse plugin compile path
Posted by Alex Lee <el...@hotmail.com>.
Trying to use the below command to generate hadoop eclipse plugin, but seem the directory =/usr/local/hadoop-2.2.0 not correct. I just used the ambari to installed the hadoop.
$ant jar -Dversion=2.2.0 -Declipse.home=/usr/local/eclipse -Dhadoop.home=/usr/local/hadoop-2.2.0
error log
BUILD FAILED
/usr/local/hadoop2x-eclipse-plugin/src/contrib/eclipse-plugin/build.xml:76: /usr/local/hadoop-2.2.0/share/hadoop/mapreduce does not exist.
May I know where is the install path for hadoop.
Any suggestion, thanks.
Re: how to install and distribute master services in 4 nodes.
Posted by Olivier Renault <or...@hortonworks.com>.
Hi Alex,
You may to have a look at the Hortonworks sandbox. It is a self contained
VM which also include some tutorials. The tutorials will walk you through
ingesting data, processing it and visualising the results.
Good luck
Olivier
On 10 April 2014 10:13, Alex Lee <el...@hotmail.com> wrote:
> May want to quickly pickup and we have 2 projects on it.
>
> ------------------------------
> From: eliyart@hotmail.com
> To: orenault@hortonworks.com
> CC: ambari-user@incubator.apache.org
>
> Subject: RE: how to install and distribute master services in 4 nodes.
> Date: Wed, 9 Apr 2014 13:52:34 +0800
>
>
> There is still a advanced Hbase config Hbase local directory still under
> root.
>
> After change it to under hadoop, all been runup.[image: Emoji]
>
> I cannot make it without the help, thanks.
>
> Is there any entry level hello application case. I tried to get some
> weather data and gequest.com/c/flight2-final/data They may over entry
> level.
>
> Ned to learn how to store and manage big data. Is there any doc or
> example, thanks,
>
>
> ------------------------------
> Date: Tue, 8 Apr 2014 18:59:55 +0200
> Subject: Re: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: eliyart@hotmail.com
> CC: ambari-user@incubator.apache.org
>
> Could you share the logs when starting HBase master from Ambari
>
> Thanks,
> Olivier
>
>
> On 8 April 2014 17:36, Alex Lee <el...@hotmail.com> wrote:
>
> After restart the PC of node 1, now can login as admin/admin. Most
> services not started, tried to started one by one. Finally only the Hbase
> cannot start as below.
> http://postimg.org/image/movef7ti1/
>
> ------------------------------
> Date: Tue, 8 Apr 2014 15:02:59 +0100
>
> Subject: Re: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: eliyart@hotmail.com
> CC: ambari-user@incubator.apache.org
>
> That;s strange it should be admin/admin. YOu might want to clear the cache
> of your webbrowser.
>
> Kind regards,
> Olivier
>
>
> On 8 April 2014 14:42, Alex Lee <el...@hotmail.com> wrote:
>
> Tried to start one service in webui and it is successful.
>
> But when trying to login ambari web server again using admin/admin,
> it said invalid username/password combination. I never changed the password.
>
> ------------------------------
> Date: Tue, 8 Apr 2014 10:21:16 +0100
>
> Subject: RE: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: eliyart@hotmail.com
> CC: ambari-user@incubator.apache.org
>
>
> Your installation has been completed, you can simply start the services
> from the service tab in the webui.
>
> Kind regards
> Olivier
> On 8 Apr 2014 10:17, "Alex Lee" <el...@hotmail.com> wrote:
>
> After changing the install folder to hadoop, it is almost done. [image:
> Emoji]
>
> Now 3 nodes completed the installation and the 3rd node stopped at a
> checking of the start of a service as below.
> http://postimg.org/image/73ihyuonp/
>
> As the log attached for this warning, not very sure if the service
> started. Trying to re-install, but seems no entry available.
>
> Any suggestion to complete the checking or re-install or can directly
> carry on, thanks.
>
>
>
> ------------------------------
> Date: Mon, 7 Apr 2014 15:52:22 +0100
> Subject: Re: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: eliyart@hotmail.com
> CC: ambari-user@incubator.apache.org
>
> It looks like your error is coming from not being able to do do a chown.
> TO be honest, you should not installed the namenode inside your /boot
> folder. Could you create a folder called /hadoop ( ideally on a dedicated
> disk ) and reinstalled using this folder as the begining.
>
> You will probably end up with
> /hadoop/hdfs/{namenode,datanode,secondarynamenode}
> /hadoop/{yarn,zookeeper,hbase, ..}/..
>
> Thanks,
> Olivier
>
> err:
> /Stage[2]/Hdp-hadoop::Namenode/Hdp-hadoop::Namenode::Create_name_dirs[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory_recursive_create[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory[/boot/efi/hadoop/hdfs/namenode]/File[/boot/efi/hadoop/hdfs/namenode]/owner:
> change from root to hdfs failed: Failed to set owner to '1004': Operation
> not permitted - /boot/efi/hadoop/hdfs/namenode
> err:
> /Stage[2]/Hdp-hadoop::Namenode/Hdp-hadoop::Namenode::Create_name_dirs[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory_recursive_create[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory[/boot/efi/hadoop/hdfs/namenode]/File[/boot/efi/hadoop/hdfs/namenode]/group:
> change from root to hadoop failed: Failed to set group to '502': Operation
> not permitted - /boot/efi/hadoop/hdfs/namenode
> notice:
> /Stage[2]/Hdp-hadoop::Namenode/Hdp-hadoop::Namenode::Create_name_dirs[/boot/efi/hadoop/hdfs/namenode
>
>
>
>
> On 7 April 2014 15:00, Alex Lee <el...@hotmail.com> wrote:
>
> Now seems all the node start install client first as below. Then, node 1
> hang at NameNode install, then all the other node will also stop.
> http://postimg.org/image/uesy5kp7d/
>
> There is one time node 4th complete. But now I tried to re-assign masters
> :( , not sure any conflict.
>
> The error log for node 1 is as attached text file.
>
> Any suggestion, thanks.
>
> ------------------------------
> Date: Mon, 7 Apr 2014 10:39:29 +0100
>
> Subject: RE: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: ambari-user@incubator.apache.org
>
>
> Moving thread to Ambari user ML.
>
> What are the errors?
>
> FYI, there should be no conflicts in between services. You can even
> installed a single node cluster with all services running on one node.
>
> Kind regards
> Olivier
> On 7 Apr 2014 10:20, "Alex Lee" <el...@hotmail.com> wrote:
>
> hi Olivier,
>
>
> With totally 4 nodes(PC), I tried to put all master services into one
> node. And with all the 4 nodes, tried to put as datanode and installed
> client as the attached screen.
>
> http://postimg.org/image/fflx6tpmz/
>
>
>
> In installation, only the 4th node is completely installed. others all
> have problem. And now cannot roll back. Seems some client and server cannot
> running on same node. I tried to re-assign the master services, but some
> already blocked and also cannot proceed next any more.
>
>
>
> Any suggestion, thank.
>
>
> ------------------------------
> From: eliyart@hotmail.com
> To: user@hadoop.apache.org
> Subject: RE: how to install and distribute master services in 4 nodes.
> Date: Mon, 7 Apr 2014 10:38:06 +0800
>
> Really appreciate the answers, I am trying to follow it, only the
> zookeeper need to distribute into 3 nodes. And for slave and client,
> beside checking all the nodes with datanode and node manager, should I also
> checkbox with the Client or Reginal Server in the web setting.
>
>
> ------------------------------
> Date: Sun, 6 Apr 2014 13:17:44 +0100
> Subject: Re: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: user@hadoop.apache.org
>
> You can run all master services on a single node or if was you I will run
> the datanode / nodemananger on every nodes.
>
> Kind regards
> Olivier
> On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
>
> Trying to install hadoop by horton, there some many master services, such
> as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image
> below.
> http://postimg.org/image/pwbv6go33/
>
> Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master
> services, and only 1 node left for datanode.
> http://postimg.org/image/gsdr8h719/
>
> Does all the master services neccessary. After searching, seem all
> neccessary. Can all the master services put into one node, if they are not
> cost much resources. Or even master services are distrbuted for 3 nodes, I
> still can distribute data services into 3 or 4 nodes.
>
> Any suggestion, thanks.
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
>
>
>
> --
> * Olivier Renault * Solution Engineer
> ------------------------------
>
> Phone: +44 7500 933 036
> Email: orenault@hortonworks.com
> Website: http://www.hortonworks.com/
>
> * Follow Us: *
> <http://facebook.com/hortonworks/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
> <http://twitter.com/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
> <http://www.linkedin.com/company/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
>
> [image: photo]
>
> Latest From Our Blog: Introduction to Apache Falcon: Data Governance
> for Hadoop <http://hortonworks.com/blog/introduction-apache-falcon-hadoop/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
>
>
>
> --
> * Olivier Renault * Solution Engineer
> ------------------------------
>
> Phone: +44 7500 933 036
> Email: orenault@hortonworks.com
> Website: http://www.hortonworks.com/
>
> * Follow Us: *
> <http://facebook.com/hortonworks/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
> <http://twitter.com/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
> <http://www.linkedin.com/company/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
>
> [image: photo]
>
> Latest From Our Blog: Introduction to Apache Falcon: Data Governance
> for Hadoop <http://hortonworks.com/blog/introduction-apache-falcon-hadoop/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
>
>
>
> --
> * Olivier Renault * Solution Engineer
> ------------------------------
>
> Phone: +44 7500 933 036
> Email: orenault@hortonworks.com
> Website: http://www.hortonworks.com/
>
> * Follow Us: *
> <http://facebook.com/hortonworks/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
> <http://twitter.com/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
> <http://www.linkedin.com/company/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
>
> [image: photo]
>
> Latest From Our Blog: Bringing Enterprise Search to Enterprise Hadoop <http://hortonworks.com/blog/bringing-enterprise-search-enterprise-hadoop/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
* Olivier Renault * Solution Engineer
------------------------------
Phone: +44 7500 933 036
Email: orenault@hortonworks.com
Website: http://www.hortonworks.com/
* Follow Us: *
<http://facebook.com/hortonworks/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
<http://twitter.com/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
<http://www.linkedin.com/company/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
[image: photo]
Latest From Our Blog: HDFS ACLs: Fine-Grained Permissions for HDFS Files
in Hadoop <http://hortonworks.com/blog/hdfs-acls-fine-grained-permissions-hdfs-files-hadoop/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: how to install and distribute master services in 4 nodes.
Posted by Olivier Renault <or...@hortonworks.com>.
Could you share the logs when starting HBase master from Ambari
Thanks,
Olivier
On 8 April 2014 17:36, Alex Lee <el...@hotmail.com> wrote:
> After restart the PC of node 1, now can login as admin/admin. Most
> services not started, tried to started one by one. Finally only the Hbase
> cannot start as below.
> http://postimg.org/image/movef7ti1/
>
> ------------------------------
> Date: Tue, 8 Apr 2014 15:02:59 +0100
>
> Subject: Re: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: eliyart@hotmail.com
> CC: ambari-user@incubator.apache.org
>
> That;s strange it should be admin/admin. YOu might want to clear the cache
> of your webbrowser.
>
> Kind regards,
> Olivier
>
>
> On 8 April 2014 14:42, Alex Lee <el...@hotmail.com> wrote:
>
> Tried to start one service in webui and it is successful.
>
> But when trying to login ambari web server again using admin/admin,
> it said invalid username/password combination. I never changed the password.
>
> ------------------------------
> Date: Tue, 8 Apr 2014 10:21:16 +0100
>
> Subject: RE: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: eliyart@hotmail.com
> CC: ambari-user@incubator.apache.org
>
>
> Your installation has been completed, you can simply start the services
> from the service tab in the webui.
>
> Kind regards
> Olivier
> On 8 Apr 2014 10:17, "Alex Lee" <el...@hotmail.com> wrote:
>
> After changing the install folder to hadoop, it is almost done. [image:
> Emoji]
>
> Now 3 nodes completed the installation and the 3rd node stopped at a
> checking of the start of a service as below.
> http://postimg.org/image/73ihyuonp/
>
> As the log attached for this warning, not very sure if the service
> started. Trying to re-install, but seems no entry available.
>
> Any suggestion to complete the checking or re-install or can directly
> carry on, thanks.
>
>
>
> ------------------------------
> Date: Mon, 7 Apr 2014 15:52:22 +0100
> Subject: Re: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: eliyart@hotmail.com
> CC: ambari-user@incubator.apache.org
>
> It looks like your error is coming from not being able to do do a chown.
> TO be honest, you should not installed the namenode inside your /boot
> folder. Could you create a folder called /hadoop ( ideally on a dedicated
> disk ) and reinstalled using this folder as the begining.
>
> You will probably end up with
> /hadoop/hdfs/{namenode,datanode,secondarynamenode}
> /hadoop/{yarn,zookeeper,hbase, ..}/..
>
> Thanks,
> Olivier
>
> err:
> /Stage[2]/Hdp-hadoop::Namenode/Hdp-hadoop::Namenode::Create_name_dirs[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory_recursive_create[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory[/boot/efi/hadoop/hdfs/namenode]/File[/boot/efi/hadoop/hdfs/namenode]/owner:
> change from root to hdfs failed: Failed to set owner to '1004': Operation
> not permitted - /boot/efi/hadoop/hdfs/namenode
> err:
> /Stage[2]/Hdp-hadoop::Namenode/Hdp-hadoop::Namenode::Create_name_dirs[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory_recursive_create[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory[/boot/efi/hadoop/hdfs/namenode]/File[/boot/efi/hadoop/hdfs/namenode]/group:
> change from root to hadoop failed: Failed to set group to '502': Operation
> not permitted - /boot/efi/hadoop/hdfs/namenode
> notice:
> /Stage[2]/Hdp-hadoop::Namenode/Hdp-hadoop::Namenode::Create_name_dirs[/boot/efi/hadoop/hdfs/namenode
>
>
>
>
> On 7 April 2014 15:00, Alex Lee <el...@hotmail.com> wrote:
>
> Now seems all the node start install client first as below. Then, node 1
> hang at NameNode install, then all the other node will also stop.
> http://postimg.org/image/uesy5kp7d/
>
> There is one time node 4th complete. But now I tried to re-assign masters
> :( , not sure any conflict.
>
> The error log for node 1 is as attached text file.
>
> Any suggestion, thanks.
>
> ------------------------------
> Date: Mon, 7 Apr 2014 10:39:29 +0100
>
> Subject: RE: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: ambari-user@incubator.apache.org
>
>
> Moving thread to Ambari user ML.
>
> What are the errors?
>
> FYI, there should be no conflicts in between services. You can even
> installed a single node cluster with all services running on one node.
>
> Kind regards
> Olivier
> On 7 Apr 2014 10:20, "Alex Lee" <el...@hotmail.com> wrote:
>
> hi Olivier,
>
>
> With totally 4 nodes(PC), I tried to put all master services into one
> node. And with all the 4 nodes, tried to put as datanode and installed
> client as the attached screen.
>
> http://postimg.org/image/fflx6tpmz/
>
>
>
> In installation, only the 4th node is completely installed. others all
> have problem. And now cannot roll back. Seems some client and server cannot
> running on same node. I tried to re-assign the master services, but some
> already blocked and also cannot proceed next any more.
>
>
>
> Any suggestion, thank.
>
>
> ------------------------------
> From: eliyart@hotmail.com
> To: user@hadoop.apache.org
> Subject: RE: how to install and distribute master services in 4 nodes.
> Date: Mon, 7 Apr 2014 10:38:06 +0800
>
> Really appreciate the answers, I am trying to follow it, only the
> zookeeper need to distribute into 3 nodes. And for slave and client,
> beside checking all the nodes with datanode and node manager, should I also
> checkbox with the Client or Reginal Server in the web setting.
>
>
> ------------------------------
> Date: Sun, 6 Apr 2014 13:17:44 +0100
> Subject: Re: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: user@hadoop.apache.org
>
> You can run all master services on a single node or if was you I will run
> the datanode / nodemananger on every nodes.
>
> Kind regards
> Olivier
> On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
>
> Trying to install hadoop by horton, there some many master services, such
> as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image
> below.
> http://postimg.org/image/pwbv6go33/
>
> Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master
> services, and only 1 node left for datanode.
> http://postimg.org/image/gsdr8h719/
>
> Does all the master services neccessary. After searching, seem all
> neccessary. Can all the master services put into one node, if they are not
> cost much resources. Or even master services are distrbuted for 3 nodes, I
> still can distribute data services into 3 or 4 nodes.
>
> Any suggestion, thanks.
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
>
>
>
> --
> * Olivier Renault * Solution Engineer
> ------------------------------
>
> Phone: +44 7500 933 036
> Email: orenault@hortonworks.com
> Website: http://www.hortonworks.com/
>
> * Follow Us: *
> <http://facebook.com/hortonworks/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
> <http://twitter.com/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
> <http://www.linkedin.com/company/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
>
> [image: photo]
>
> Latest From Our Blog: Introduction to Apache Falcon: Data Governance
> for Hadoop <http://hortonworks.com/blog/introduction-apache-falcon-hadoop/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
>
>
>
> --
> * Olivier Renault * Solution Engineer
> ------------------------------
>
> Phone: +44 7500 933 036
> Email: orenault@hortonworks.com
> Website: http://www.hortonworks.com/
>
> * Follow Us: *
> <http://facebook.com/hortonworks/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
> <http://twitter.com/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
> <http://www.linkedin.com/company/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
>
> [image: photo]
>
> Latest From Our Blog: Introduction to Apache Falcon: Data Governance
> for Hadoop <http://hortonworks.com/blog/introduction-apache-falcon-hadoop/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
* Olivier Renault * Solution Engineer
------------------------------
Phone: +44 7500 933 036
Email: orenault@hortonworks.com
Website: http://www.hortonworks.com/
* Follow Us: *
<http://facebook.com/hortonworks/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
<http://twitter.com/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
<http://www.linkedin.com/company/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
[image: photo]
Latest From Our Blog: Bringing Enterprise Search to Enterprise
Hadoop <http://hortonworks.com/blog/bringing-enterprise-search-enterprise-hadoop/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: how to install and distribute master services in 4 nodes.
Posted by Olivier Renault <or...@hortonworks.com>.
That;s strange it should be admin/admin. YOu might want to clear the cache
of your webbrowser.
Kind regards,
Olivier
On 8 April 2014 14:42, Alex Lee <el...@hotmail.com> wrote:
> Tried to start one service in webui and it is successful.
>
> But when trying to login ambari web server again using admin/admin,
> it said invalid username/password combination. I never changed the password.
>
> ------------------------------
> Date: Tue, 8 Apr 2014 10:21:16 +0100
>
> Subject: RE: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: eliyart@hotmail.com
> CC: ambari-user@incubator.apache.org
>
>
> Your installation has been completed, you can simply start the services
> from the service tab in the webui.
>
> Kind regards
> Olivier
> On 8 Apr 2014 10:17, "Alex Lee" <el...@hotmail.com> wrote:
>
> After changing the install folder to hadoop, it is almost done. [image:
> Emoji]
>
> Now 3 nodes completed the installation and the 3rd node stopped at a
> checking of the start of a service as below.
> http://postimg.org/image/73ihyuonp/
>
> As the log attached for this warning, not very sure if the service
> started. Trying to re-install, but seems no entry available.
>
> Any suggestion to complete the checking or re-install or can directly
> carry on, thanks.
>
>
>
> ------------------------------
> Date: Mon, 7 Apr 2014 15:52:22 +0100
> Subject: Re: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: eliyart@hotmail.com
> CC: ambari-user@incubator.apache.org
>
> It looks like your error is coming from not being able to do do a chown.
> TO be honest, you should not installed the namenode inside your /boot
> folder. Could you create a folder called /hadoop ( ideally on a dedicated
> disk ) and reinstalled using this folder as the begining.
>
> You will probably end up with
> /hadoop/hdfs/{namenode,datanode,secondarynamenode}
> /hadoop/{yarn,zookeeper,hbase, ..}/..
>
> Thanks,
> Olivier
>
> err:
> /Stage[2]/Hdp-hadoop::Namenode/Hdp-hadoop::Namenode::Create_name_dirs[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory_recursive_create[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory[/boot/efi/hadoop/hdfs/namenode]/File[/boot/efi/hadoop/hdfs/namenode]/owner:
> change from root to hdfs failed: Failed to set owner to '1004': Operation
> not permitted - /boot/efi/hadoop/hdfs/namenode
> err:
> /Stage[2]/Hdp-hadoop::Namenode/Hdp-hadoop::Namenode::Create_name_dirs[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory_recursive_create[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory[/boot/efi/hadoop/hdfs/namenode]/File[/boot/efi/hadoop/hdfs/namenode]/group:
> change from root to hadoop failed: Failed to set group to '502': Operation
> not permitted - /boot/efi/hadoop/hdfs/namenode
> notice:
> /Stage[2]/Hdp-hadoop::Namenode/Hdp-hadoop::Namenode::Create_name_dirs[/boot/efi/hadoop/hdfs/namenode
>
>
>
>
> On 7 April 2014 15:00, Alex Lee <el...@hotmail.com> wrote:
>
> Now seems all the node start install client first as below. Then, node 1
> hang at NameNode install, then all the other node will also stop.
> http://postimg.org/image/uesy5kp7d/
>
> There is one time node 4th complete. But now I tried to re-assign masters
> :( , not sure any conflict.
>
> The error log for node 1 is as attached text file.
>
> Any suggestion, thanks.
>
> ------------------------------
> Date: Mon, 7 Apr 2014 10:39:29 +0100
>
> Subject: RE: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: ambari-user@incubator.apache.org
>
>
> Moving thread to Ambari user ML.
>
> What are the errors?
>
> FYI, there should be no conflicts in between services. You can even
> installed a single node cluster with all services running on one node.
>
> Kind regards
> Olivier
> On 7 Apr 2014 10:20, "Alex Lee" <el...@hotmail.com> wrote:
>
> hi Olivier,
>
>
> With totally 4 nodes(PC), I tried to put all master services into one
> node. And with all the 4 nodes, tried to put as datanode and installed
> client as the attached screen.
>
> http://postimg.org/image/fflx6tpmz/
>
>
>
> In installation, only the 4th node is completely installed. others all
> have problem. And now cannot roll back. Seems some client and server cannot
> running on same node. I tried to re-assign the master services, but some
> already blocked and also cannot proceed next any more.
>
>
>
> Any suggestion, thank.
>
>
> ------------------------------
> From: eliyart@hotmail.com
> To: user@hadoop.apache.org
> Subject: RE: how to install and distribute master services in 4 nodes.
> Date: Mon, 7 Apr 2014 10:38:06 +0800
>
> Really appreciate the answers, I am trying to follow it, only the
> zookeeper need to distribute into 3 nodes. And for slave and client,
> beside checking all the nodes with datanode and node manager, should I also
> checkbox with the Client or Reginal Server in the web setting.
>
>
> ------------------------------
> Date: Sun, 6 Apr 2014 13:17:44 +0100
> Subject: Re: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: user@hadoop.apache.org
>
> You can run all master services on a single node or if was you I will run
> the datanode / nodemananger on every nodes.
>
> Kind regards
> Olivier
> On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
>
> Trying to install hadoop by horton, there some many master services, such
> as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image
> below.
> http://postimg.org/image/pwbv6go33/
>
> Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master
> services, and only 1 node left for datanode.
> http://postimg.org/image/gsdr8h719/
>
> Does all the master services neccessary. After searching, seem all
> neccessary. Can all the master services put into one node, if they are not
> cost much resources. Or even master services are distrbuted for 3 nodes, I
> still can distribute data services into 3 or 4 nodes.
>
> Any suggestion, thanks.
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
>
>
>
> --
> * Olivier Renault * Solution Engineer
> ------------------------------
>
> Phone: +44 7500 933 036
> Email: orenault@hortonworks.com
> Website: http://www.hortonworks.com/
>
> * Follow Us: *
> <http://facebook.com/hortonworks/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
> <http://twitter.com/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
> <http://www.linkedin.com/company/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
>
> [image: photo]
>
> Latest From Our Blog: Introduction to Apache Falcon: Data Governance
> for Hadoop <http://hortonworks.com/blog/introduction-apache-falcon-hadoop/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
* Olivier Renault * Solution Engineer
------------------------------
Phone: +44 7500 933 036
Email: orenault@hortonworks.com
Website: http://www.hortonworks.com/
* Follow Us: *
<http://facebook.com/hortonworks/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
<http://twitter.com/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
<http://www.linkedin.com/company/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
[image: photo]
Latest From Our Blog: Introduction to Apache Falcon: Data Governance for
Hadoop <http://hortonworks.com/blog/introduction-apache-falcon-hadoop/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
RE: how to install and distribute master services in 4 nodes.
Posted by Olivier Renault <or...@hortonworks.com>.
Your installation has been completed, you can simply start the services
from the service tab in the webui.
Kind regards
Olivier
On 8 Apr 2014 10:17, "Alex Lee" <el...@hotmail.com> wrote:
> After changing the install folder to hadoop, it is almost done. [image:
> Emoji]
>
> Now 3 nodes completed the installation and the 3rd node stopped at a
> checking of the start of a service as below.
> http://postimg.org/image/73ihyuonp/
>
> As the log attached for this warning, not very sure if the service
> started. Trying to re-install, but seems no entry available.
>
> Any suggestion to complete the checking or re-install or can directly
> carry on, thanks.
>
>
>
> ------------------------------
> Date: Mon, 7 Apr 2014 15:52:22 +0100
> Subject: Re: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: eliyart@hotmail.com
> CC: ambari-user@incubator.apache.org
>
> It looks like your error is coming from not being able to do do a chown.
> TO be honest, you should not installed the namenode inside your /boot
> folder. Could you create a folder called /hadoop ( ideally on a dedicated
> disk ) and reinstalled using this folder as the begining.
>
> You will probably end up with
> /hadoop/hdfs/{namenode,datanode,secondarynamenode}
> /hadoop/{yarn,zookeeper,hbase, ..}/..
>
> Thanks,
> Olivier
>
> err:
> /Stage[2]/Hdp-hadoop::Namenode/Hdp-hadoop::Namenode::Create_name_dirs[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory_recursive_create[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory[/boot/efi/hadoop/hdfs/namenode]/File[/boot/efi/hadoop/hdfs/namenode]/owner:
> change from root to hdfs failed: Failed to set owner to '1004': Operation
> not permitted - /boot/efi/hadoop/hdfs/namenode
> err:
> /Stage[2]/Hdp-hadoop::Namenode/Hdp-hadoop::Namenode::Create_name_dirs[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory_recursive_create[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory[/boot/efi/hadoop/hdfs/namenode]/File[/boot/efi/hadoop/hdfs/namenode]/group:
> change from root to hadoop failed: Failed to set group to '502': Operation
> not permitted - /boot/efi/hadoop/hdfs/namenode
> notice:
> /Stage[2]/Hdp-hadoop::Namenode/Hdp-hadoop::Namenode::Create_name_dirs[/boot/efi/hadoop/hdfs/namenode
>
>
>
>
> On 7 April 2014 15:00, Alex Lee <el...@hotmail.com> wrote:
>
> Now seems all the node start install client first as below. Then, node 1
> hang at NameNode install, then all the other node will also stop.
> http://postimg.org/image/uesy5kp7d/
>
> There is one time node 4th complete. But now I tried to re-assign masters
> :( , not sure any conflict.
>
> The error log for node 1 is as attached text file.
>
> Any suggestion, thanks.
>
> ------------------------------
> Date: Mon, 7 Apr 2014 10:39:29 +0100
>
> Subject: RE: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: ambari-user@incubator.apache.org
>
>
> Moving thread to Ambari user ML.
>
> What are the errors?
>
> FYI, there should be no conflicts in between services. You can even
> installed a single node cluster with all services running on one node.
>
> Kind regards
> Olivier
> On 7 Apr 2014 10:20, "Alex Lee" <el...@hotmail.com> wrote:
>
> hi Olivier,
>
>
> With totally 4 nodes(PC), I tried to put all master services into one
> node. And with all the 4 nodes, tried to put as datanode and installed
> client as the attached screen.
>
> http://postimg.org/image/fflx6tpmz/
>
>
>
> In installation, only the 4th node is completely installed. others all
> have problem. And now cannot roll back. Seems some client and server cannot
> running on same node. I tried to re-assign the master services, but some
> already blocked and also cannot proceed next any more.
>
>
>
> Any suggestion, thank.
>
>
> ------------------------------
> From: eliyart@hotmail.com
> To: user@hadoop.apache.org
> Subject: RE: how to install and distribute master services in 4 nodes.
> Date: Mon, 7 Apr 2014 10:38:06 +0800
>
> Really appreciate the answers, I am trying to follow it, only the
> zookeeper need to distribute into 3 nodes. And for slave and client,
> beside checking all the nodes with datanode and node manager, should I also
> checkbox with the Client or Reginal Server in the web setting.
>
>
> ------------------------------
> Date: Sun, 6 Apr 2014 13:17:44 +0100
> Subject: Re: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: user@hadoop.apache.org
>
> You can run all master services on a single node or if was you I will run
> the datanode / nodemananger on every nodes.
>
> Kind regards
> Olivier
> On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
>
> Trying to install hadoop by horton, there some many master services, such
> as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image
> below.
> http://postimg.org/image/pwbv6go33/
>
> Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master
> services, and only 1 node left for datanode.
> http://postimg.org/image/gsdr8h719/
>
> Does all the master services neccessary. After searching, seem all
> neccessary. Can all the master services put into one node, if they are not
> cost much resources. Or even master services are distrbuted for 3 nodes, I
> still can distribute data services into 3 or 4 nodes.
>
> Any suggestion, thanks.
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
>
>
>
> --
> * Olivier Renault * Solution Engineer
> ------------------------------
>
> Phone: +44 7500 933 036
> Email: orenault@hortonworks.com
> Website: http://www.hortonworks.com/
>
> * Follow Us: *
> <http://facebook.com/hortonworks/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
> <http://twitter.com/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
> <http://www.linkedin.com/company/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
>
> [image: photo]
>
> Latest From Our Blog: Introduction to Apache Falcon: Data Governance
> for Hadoop <http://hortonworks.com/blog/introduction-apache-falcon-hadoop/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: how to install and distribute master services in 4 nodes.
Posted by Olivier Renault <or...@hortonworks.com>.
It looks like your error is coming from not being able to do do a chown. TO
be honest, you should not installed the namenode inside your /boot folder.
Could you create a folder called /hadoop ( ideally on a dedicated disk )
and reinstalled using this folder as the begining.
You will probably end up with
/hadoop/hdfs/{namenode,datanode,secondarynamenode}
/hadoop/{yarn,zookeeper,hbase, ..}/..
Thanks,
Olivier
err:
/Stage[2]/Hdp-hadoop::Namenode/Hdp-hadoop::Namenode::Create_name_dirs[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory_recursive_create[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory[/boot/efi/hadoop/hdfs/namenode]/File[/boot/efi/hadoop/hdfs/namenode]/owner:
change from root to hdfs failed: Failed to set owner to '1004': Operation
not permitted - /boot/efi/hadoop/hdfs/namenode
err:
/Stage[2]/Hdp-hadoop::Namenode/Hdp-hadoop::Namenode::Create_name_dirs[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory_recursive_create[/boot/efi/hadoop/hdfs/namenode]/Hdp::Directory[/boot/efi/hadoop/hdfs/namenode]/File[/boot/efi/hadoop/hdfs/namenode]/group:
change from root to hadoop failed: Failed to set group to '502': Operation
not permitted - /boot/efi/hadoop/hdfs/namenode
notice:
/Stage[2]/Hdp-hadoop::Namenode/Hdp-hadoop::Namenode::Create_name_dirs[/boot/efi/hadoop/hdfs/namenode
On 7 April 2014 15:00, Alex Lee <el...@hotmail.com> wrote:
> Now seems all the node start install client first as below. Then, node 1
> hang at NameNode install, then all the other node will also stop.
> http://postimg.org/image/uesy5kp7d/
>
> There is one time node 4th complete. But now I tried to re-assign masters
> :( , not sure any conflict.
>
> The error log for node 1 is as attached text file.
>
> Any suggestion, thanks.
>
> ------------------------------
> Date: Mon, 7 Apr 2014 10:39:29 +0100
>
> Subject: RE: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: ambari-user@incubator.apache.org
>
>
> Moving thread to Ambari user ML.
>
> What are the errors?
>
> FYI, there should be no conflicts in between services. You can even
> installed a single node cluster with all services running on one node.
>
> Kind regards
> Olivier
> On 7 Apr 2014 10:20, "Alex Lee" <el...@hotmail.com> wrote:
>
> hi Olivier,
>
>
> With totally 4 nodes(PC), I tried to put all master services into one
> node. And with all the 4 nodes, tried to put as datanode and installed
> client as the attached screen.
>
> http://postimg.org/image/fflx6tpmz/
>
>
>
> In installation, only the 4th node is completely installed. others all
> have problem. And now cannot roll back. Seems some client and server cannot
> running on same node. I tried to re-assign the master services, but some
> already blocked and also cannot proceed next any more.
>
>
>
> Any suggestion, thank.
>
>
> ------------------------------
> From: eliyart@hotmail.com
> To: user@hadoop.apache.org
> Subject: RE: how to install and distribute master services in 4 nodes.
> Date: Mon, 7 Apr 2014 10:38:06 +0800
>
> Really appreciate the answers, I am trying to follow it, only the
> zookeeper need to distribute into 3 nodes. And for slave and client,
> beside checking all the nodes with datanode and node manager, should I also
> checkbox with the Client or Reginal Server in the web setting.
>
>
> ------------------------------
> Date: Sun, 6 Apr 2014 13:17:44 +0100
> Subject: Re: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: user@hadoop.apache.org
>
> You can run all master services on a single node or if was you I will run
> the datanode / nodemananger on every nodes.
>
> Kind regards
> Olivier
> On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
>
> Trying to install hadoop by horton, there some many master services, such
> as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image
> below.
> http://postimg.org/image/pwbv6go33/
>
> Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master
> services, and only 1 node left for datanode.
> http://postimg.org/image/gsdr8h719/
>
> Does all the master services neccessary. After searching, seem all
> neccessary. Can all the master services put into one node, if they are not
> cost much resources. Or even master services are distrbuted for 3 nodes, I
> still can distribute data services into 3 or 4 nodes.
>
> Any suggestion, thanks.
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
* Olivier Renault * Solution Engineer
------------------------------
Phone: +44 7500 933 036
Email: orenault@hortonworks.com
Website: http://www.hortonworks.com/
* Follow Us: *
<http://facebook.com/hortonworks/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
<http://twitter.com/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
<http://www.linkedin.com/company/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
[image: photo]
Latest From Our Blog: Introduction to Apache Falcon: Data Governance for
Hadoop <http://hortonworks.com/blog/introduction-apache-falcon-hadoop/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
RE: how to install and distribute master services in 4 nodes.
Posted by Olivier Renault <or...@hortonworks.com>.
Moving thread to Ambari user ML.
What are the errors?
FYI, there should be no conflicts in between services. You can even
installed a single node cluster with all services running on one node.
Kind regards
Olivier
On 7 Apr 2014 10:20, "Alex Lee" <el...@hotmail.com> wrote:
> hi Olivier,
>
>
> With totally 4 nodes(PC), I tried to put all master services into one
> node. And with all the 4 nodes, tried to put as datanode and installed
> client as the attached screen.
>
> http://postimg.org/image/fflx6tpmz/
>
>
>
> In installation, only the 4th node is completely installed. others all
> have problem. And now cannot roll back. Seems some client and server cannot
> running on same node. I tried to re-assign the master services, but some
> already blocked and also cannot proceed next any more.
>
>
>
> Any suggestion, thank.
>
>
> ------------------------------
> From: eliyart@hotmail.com
> To: user@hadoop.apache.org
> Subject: RE: how to install and distribute master services in 4 nodes.
> Date: Mon, 7 Apr 2014 10:38:06 +0800
>
> Really appreciate the answers, I am trying to follow it, only the
> zookeeper need to distribute into 3 nodes. And for slave and client,
> beside checking all the nodes with datanode and node manager, should I also
> checkbox with the Client or Reginal Server in the web setting.
>
>
> ------------------------------
> Date: Sun, 6 Apr 2014 13:17:44 +0100
> Subject: Re: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: user@hadoop.apache.org
>
> You can run all master services on a single node or if was you I will run
> the datanode / nodemananger on every nodes.
>
> Kind regards
> Olivier
> On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
>
> Trying to install hadoop by horton, there some many master services, such
> as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image
> below.
> http://postimg.org/image/pwbv6go33/
>
> Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master
> services, and only 1 node left for datanode.
> http://postimg.org/image/gsdr8h719/
>
> Does all the master services neccessary. After searching, seem all
> neccessary. Can all the master services put into one node, if they are not
> cost much resources. Or even master services are distrbuted for 3 nodes, I
> still can distribute data services into 3 or 4 nodes.
>
> Any suggestion, thanks.
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
RE: how to install and distribute master services in 4 nodes.
Posted by Olivier Renault <or...@hortonworks.com>.
Moving thread to Ambari user ML.
What are the errors?
FYI, there should be no conflicts in between services. You can even
installed a single node cluster with all services running on one node.
Kind regards
Olivier
On 7 Apr 2014 10:20, "Alex Lee" <el...@hotmail.com> wrote:
> hi Olivier,
>
>
> With totally 4 nodes(PC), I tried to put all master services into one
> node. And with all the 4 nodes, tried to put as datanode and installed
> client as the attached screen.
>
> http://postimg.org/image/fflx6tpmz/
>
>
>
> In installation, only the 4th node is completely installed. others all
> have problem. And now cannot roll back. Seems some client and server cannot
> running on same node. I tried to re-assign the master services, but some
> already blocked and also cannot proceed next any more.
>
>
>
> Any suggestion, thank.
>
>
> ------------------------------
> From: eliyart@hotmail.com
> To: user@hadoop.apache.org
> Subject: RE: how to install and distribute master services in 4 nodes.
> Date: Mon, 7 Apr 2014 10:38:06 +0800
>
> Really appreciate the answers, I am trying to follow it, only the
> zookeeper need to distribute into 3 nodes. And for slave and client,
> beside checking all the nodes with datanode and node manager, should I also
> checkbox with the Client or Reginal Server in the web setting.
>
>
> ------------------------------
> Date: Sun, 6 Apr 2014 13:17:44 +0100
> Subject: Re: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: user@hadoop.apache.org
>
> You can run all master services on a single node or if was you I will run
> the datanode / nodemananger on every nodes.
>
> Kind regards
> Olivier
> On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
>
> Trying to install hadoop by horton, there some many master services, such
> as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image
> below.
> http://postimg.org/image/pwbv6go33/
>
> Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master
> services, and only 1 node left for datanode.
> http://postimg.org/image/gsdr8h719/
>
> Does all the master services neccessary. After searching, seem all
> neccessary. Can all the master services put into one node, if they are not
> cost much resources. Or even master services are distrbuted for 3 nodes, I
> still can distribute data services into 3 or 4 nodes.
>
> Any suggestion, thanks.
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
RE: how to install and distribute master services in 4 nodes.
Posted by Olivier Renault <or...@hortonworks.com>.
Moving thread to Ambari user ML.
What are the errors?
FYI, there should be no conflicts in between services. You can even
installed a single node cluster with all services running on one node.
Kind regards
Olivier
On 7 Apr 2014 10:20, "Alex Lee" <el...@hotmail.com> wrote:
> hi Olivier,
>
>
> With totally 4 nodes(PC), I tried to put all master services into one
> node. And with all the 4 nodes, tried to put as datanode and installed
> client as the attached screen.
>
> http://postimg.org/image/fflx6tpmz/
>
>
>
> In installation, only the 4th node is completely installed. others all
> have problem. And now cannot roll back. Seems some client and server cannot
> running on same node. I tried to re-assign the master services, but some
> already blocked and also cannot proceed next any more.
>
>
>
> Any suggestion, thank.
>
>
> ------------------------------
> From: eliyart@hotmail.com
> To: user@hadoop.apache.org
> Subject: RE: how to install and distribute master services in 4 nodes.
> Date: Mon, 7 Apr 2014 10:38:06 +0800
>
> Really appreciate the answers, I am trying to follow it, only the
> zookeeper need to distribute into 3 nodes. And for slave and client,
> beside checking all the nodes with datanode and node manager, should I also
> checkbox with the Client or Reginal Server in the web setting.
>
>
> ------------------------------
> Date: Sun, 6 Apr 2014 13:17:44 +0100
> Subject: Re: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: user@hadoop.apache.org
>
> You can run all master services on a single node or if was you I will run
> the datanode / nodemananger on every nodes.
>
> Kind regards
> Olivier
> On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
>
> Trying to install hadoop by horton, there some many master services, such
> as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image
> below.
> http://postimg.org/image/pwbv6go33/
>
> Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master
> services, and only 1 node left for datanode.
> http://postimg.org/image/gsdr8h719/
>
> Does all the master services neccessary. After searching, seem all
> neccessary. Can all the master services put into one node, if they are not
> cost much resources. Or even master services are distrbuted for 3 nodes, I
> still can distribute data services into 3 or 4 nodes.
>
> Any suggestion, thanks.
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
RE: how to install and distribute master services in 4 nodes.
Posted by Olivier Renault <or...@hortonworks.com>.
Moving thread to Ambari user ML.
What are the errors?
FYI, there should be no conflicts in between services. You can even
installed a single node cluster with all services running on one node.
Kind regards
Olivier
On 7 Apr 2014 10:20, "Alex Lee" <el...@hotmail.com> wrote:
> hi Olivier,
>
>
> With totally 4 nodes(PC), I tried to put all master services into one
> node. And with all the 4 nodes, tried to put as datanode and installed
> client as the attached screen.
>
> http://postimg.org/image/fflx6tpmz/
>
>
>
> In installation, only the 4th node is completely installed. others all
> have problem. And now cannot roll back. Seems some client and server cannot
> running on same node. I tried to re-assign the master services, but some
> already blocked and also cannot proceed next any more.
>
>
>
> Any suggestion, thank.
>
>
> ------------------------------
> From: eliyart@hotmail.com
> To: user@hadoop.apache.org
> Subject: RE: how to install and distribute master services in 4 nodes.
> Date: Mon, 7 Apr 2014 10:38:06 +0800
>
> Really appreciate the answers, I am trying to follow it, only the
> zookeeper need to distribute into 3 nodes. And for slave and client,
> beside checking all the nodes with datanode and node manager, should I also
> checkbox with the Client or Reginal Server in the web setting.
>
>
> ------------------------------
> Date: Sun, 6 Apr 2014 13:17:44 +0100
> Subject: Re: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: user@hadoop.apache.org
>
> You can run all master services on a single node or if was you I will run
> the datanode / nodemananger on every nodes.
>
> Kind regards
> Olivier
> On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
>
> Trying to install hadoop by horton, there some many master services, such
> as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image
> below.
> http://postimg.org/image/pwbv6go33/
>
> Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master
> services, and only 1 node left for datanode.
> http://postimg.org/image/gsdr8h719/
>
> Does all the master services neccessary. After searching, seem all
> neccessary. Can all the master services put into one node, if they are not
> cost much resources. Or even master services are distrbuted for 3 nodes, I
> still can distribute data services into 3 or 4 nodes.
>
> Any suggestion, thanks.
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
RE: how to install and distribute master services in 4 nodes.
Posted by Olivier Renault <or...@hortonworks.com>.
Moving thread to Ambari user ML.
What are the errors?
FYI, there should be no conflicts in between services. You can even
installed a single node cluster with all services running on one node.
Kind regards
Olivier
On 7 Apr 2014 10:20, "Alex Lee" <el...@hotmail.com> wrote:
> hi Olivier,
>
>
> With totally 4 nodes(PC), I tried to put all master services into one
> node. And with all the 4 nodes, tried to put as datanode and installed
> client as the attached screen.
>
> http://postimg.org/image/fflx6tpmz/
>
>
>
> In installation, only the 4th node is completely installed. others all
> have problem. And now cannot roll back. Seems some client and server cannot
> running on same node. I tried to re-assign the master services, but some
> already blocked and also cannot proceed next any more.
>
>
>
> Any suggestion, thank.
>
>
> ------------------------------
> From: eliyart@hotmail.com
> To: user@hadoop.apache.org
> Subject: RE: how to install and distribute master services in 4 nodes.
> Date: Mon, 7 Apr 2014 10:38:06 +0800
>
> Really appreciate the answers, I am trying to follow it, only the
> zookeeper need to distribute into 3 nodes. And for slave and client,
> beside checking all the nodes with datanode and node manager, should I also
> checkbox with the Client or Reginal Server in the web setting.
>
>
> ------------------------------
> Date: Sun, 6 Apr 2014 13:17:44 +0100
> Subject: Re: how to install and distribute master services in 4 nodes.
> From: orenault@hortonworks.com
> To: user@hadoop.apache.org
>
> You can run all master services on a single node or if was you I will run
> the datanode / nodemananger on every nodes.
>
> Kind regards
> Olivier
> On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
>
> Trying to install hadoop by horton, there some many master services, such
> as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image
> below.
> http://postimg.org/image/pwbv6go33/
>
> Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master
> services, and only 1 node left for datanode.
> http://postimg.org/image/gsdr8h719/
>
> Does all the master services neccessary. After searching, seem all
> neccessary. Can all the master services put into one node, if they are not
> cost much resources. Or even master services are distrbuted for 3 nodes, I
> still can distribute data services into 3 or 4 nodes.
>
> Any suggestion, thanks.
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
RE: how to install and distribute master services in 4 nodes.
Posted by Alex Lee <el...@hotmail.com>.
hi Olivier,
With totally 4 nodes(PC), I tried to put all master services into one node. And with all the 4 nodes, tried to put as datanode and installed client as the attached screen.http://postimg.org/image/fflx6tpmz/ In installation, only the 4th node is completely installed. others all have problem. And now cannot roll back. Seems some client and server cannot running on same node. I tried to re-assign the master services, but some already blocked and also cannot proceed next any more. Any suggestion, thank. From: eliyart@hotmail.com
To: user@hadoop.apache.org
Subject: RE: how to install and distribute master services in 4 nodes.
Date: Mon, 7 Apr 2014 10:38:06 +0800
Really appreciate the answers, I am trying to follow it, only the zookeeper need to distribute into 3 nodes. And for slave and client, beside checking all the nodes with datanode and node manager, should I also checkbox with the Client or Reginal Server in the web setting.
Date: Sun, 6 Apr 2014 13:17:44 +0100
Subject: Re: how to install and distribute master services in 4 nodes.
From: orenault@hortonworks.com
To: user@hadoop.apache.org
You can run all master services on a single node or if was you I will run the datanode / nodemananger on every nodes.
Kind regards
Olivier
On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
Trying to install hadoop by horton, there some many master services, such as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image below.
http://postimg.org/image/pwbv6go33/
Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master services, and only 1 node left for datanode.
http://postimg.org/image/gsdr8h719/
Does all the master services neccessary. After searching, seem all neccessary. Can all the master services put into one node, if they are not cost much resources. Or even master services are distrbuted for 3 nodes, I still can distribute data services into 3 or 4 nodes.
Any suggestion, thanks.
CONFIDENTIALITY NOTICENOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
RE: how to install and distribute master services in 4 nodes.
Posted by Alex Lee <el...@hotmail.com>.
hi Olivier,
With totally 4 nodes(PC), I tried to put all master services into one node. And with all the 4 nodes, tried to put as datanode and installed client as the attached screen.http://postimg.org/image/fflx6tpmz/ In installation, only the 4th node is completely installed. others all have problem. And now cannot roll back. Seems some client and server cannot running on same node. I tried to re-assign the master services, but some already blocked and also cannot proceed next any more. Any suggestion, thank. From: eliyart@hotmail.com
To: user@hadoop.apache.org
Subject: RE: how to install and distribute master services in 4 nodes.
Date: Mon, 7 Apr 2014 10:38:06 +0800
Really appreciate the answers, I am trying to follow it, only the zookeeper need to distribute into 3 nodes. And for slave and client, beside checking all the nodes with datanode and node manager, should I also checkbox with the Client or Reginal Server in the web setting.
Date: Sun, 6 Apr 2014 13:17:44 +0100
Subject: Re: how to install and distribute master services in 4 nodes.
From: orenault@hortonworks.com
To: user@hadoop.apache.org
You can run all master services on a single node or if was you I will run the datanode / nodemananger on every nodes.
Kind regards
Olivier
On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
Trying to install hadoop by horton, there some many master services, such as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image below.
http://postimg.org/image/pwbv6go33/
Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master services, and only 1 node left for datanode.
http://postimg.org/image/gsdr8h719/
Does all the master services neccessary. After searching, seem all neccessary. Can all the master services put into one node, if they are not cost much resources. Or even master services are distrbuted for 3 nodes, I still can distribute data services into 3 or 4 nodes.
Any suggestion, thanks.
CONFIDENTIALITY NOTICENOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
RE: how to install and distribute master services in 4 nodes.
Posted by Alex Lee <el...@hotmail.com>.
hi Olivier,
With totally 4 nodes(PC), I tried to put all master services into one node. And with all the 4 nodes, tried to put as datanode and installed client as the attached screen.http://postimg.org/image/fflx6tpmz/ In installation, only the 4th node is completely installed. others all have problem. And now cannot roll back. Seems some client and server cannot running on same node. I tried to re-assign the master services, but some already blocked and also cannot proceed next any more. Any suggestion, thank. From: eliyart@hotmail.com
To: user@hadoop.apache.org
Subject: RE: how to install and distribute master services in 4 nodes.
Date: Mon, 7 Apr 2014 10:38:06 +0800
Really appreciate the answers, I am trying to follow it, only the zookeeper need to distribute into 3 nodes. And for slave and client, beside checking all the nodes with datanode and node manager, should I also checkbox with the Client or Reginal Server in the web setting.
Date: Sun, 6 Apr 2014 13:17:44 +0100
Subject: Re: how to install and distribute master services in 4 nodes.
From: orenault@hortonworks.com
To: user@hadoop.apache.org
You can run all master services on a single node or if was you I will run the datanode / nodemananger on every nodes.
Kind regards
Olivier
On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
Trying to install hadoop by horton, there some many master services, such as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image below.
http://postimg.org/image/pwbv6go33/
Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master services, and only 1 node left for datanode.
http://postimg.org/image/gsdr8h719/
Does all the master services neccessary. After searching, seem all neccessary. Can all the master services put into one node, if they are not cost much resources. Or even master services are distrbuted for 3 nodes, I still can distribute data services into 3 or 4 nodes.
Any suggestion, thanks.
CONFIDENTIALITY NOTICENOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
RE: how to install and distribute master services in 4 nodes.
Posted by Alex Lee <el...@hotmail.com>.
hi Olivier,
With totally 4 nodes(PC), I tried to put all master services into one node. And with all the 4 nodes, tried to put as datanode and installed client as the attached screen.http://postimg.org/image/fflx6tpmz/ In installation, only the 4th node is completely installed. others all have problem. And now cannot roll back. Seems some client and server cannot running on same node. I tried to re-assign the master services, but some already blocked and also cannot proceed next any more. Any suggestion, thank. From: eliyart@hotmail.com
To: user@hadoop.apache.org
Subject: RE: how to install and distribute master services in 4 nodes.
Date: Mon, 7 Apr 2014 10:38:06 +0800
Really appreciate the answers, I am trying to follow it, only the zookeeper need to distribute into 3 nodes. And for slave and client, beside checking all the nodes with datanode and node manager, should I also checkbox with the Client or Reginal Server in the web setting.
Date: Sun, 6 Apr 2014 13:17:44 +0100
Subject: Re: how to install and distribute master services in 4 nodes.
From: orenault@hortonworks.com
To: user@hadoop.apache.org
You can run all master services on a single node or if was you I will run the datanode / nodemananger on every nodes.
Kind regards
Olivier
On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
Trying to install hadoop by horton, there some many master services, such as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image below.
http://postimg.org/image/pwbv6go33/
Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master services, and only 1 node left for datanode.
http://postimg.org/image/gsdr8h719/
Does all the master services neccessary. After searching, seem all neccessary. Can all the master services put into one node, if they are not cost much resources. Or even master services are distrbuted for 3 nodes, I still can distribute data services into 3 or 4 nodes.
Any suggestion, thanks.
CONFIDENTIALITY NOTICENOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
RE: how to install and distribute master services in 4 nodes.
Posted by Alex Lee <el...@hotmail.com>.
Really appreciate the answers, I am trying to follow it, only the zookeeper need to distribute into 3 nodes. And for slave and client, beside checking all the nodes with datanode and node manager, should I also checkbox with the Client or Reginal Server in the web setting.
Date: Sun, 6 Apr 2014 13:17:44 +0100
Subject: Re: how to install and distribute master services in 4 nodes.
From: orenault@hortonworks.com
To: user@hadoop.apache.org
You can run all master services on a single node or if was you I will run the datanode / nodemananger on every nodes.
Kind regards
Olivier
On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
Trying to install hadoop by horton, there some many master services, such as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image below.
http://postimg.org/image/pwbv6go33/
Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master services, and only 1 node left for datanode.
http://postimg.org/image/gsdr8h719/
Does all the master services neccessary. After searching, seem all neccessary. Can all the master services put into one node, if they are not cost much resources. Or even master services are distrbuted for 3 nodes, I still can distribute data services into 3 or 4 nodes.
Any suggestion, thanks.
CONFIDENTIALITY NOTICENOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
RE: how to install and distribute master services in 4 nodes.
Posted by Alex Lee <el...@hotmail.com>.
Really appreciate the answers, I am trying to follow it, only the zookeeper need to distribute into 3 nodes. And for slave and client, beside checking all the nodes with datanode and node manager, should I also checkbox with the Client or Reginal Server in the web setting.
Date: Sun, 6 Apr 2014 13:17:44 +0100
Subject: Re: how to install and distribute master services in 4 nodes.
From: orenault@hortonworks.com
To: user@hadoop.apache.org
You can run all master services on a single node or if was you I will run the datanode / nodemananger on every nodes.
Kind regards
Olivier
On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
Trying to install hadoop by horton, there some many master services, such as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image below.
http://postimg.org/image/pwbv6go33/
Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master services, and only 1 node left for datanode.
http://postimg.org/image/gsdr8h719/
Does all the master services neccessary. After searching, seem all neccessary. Can all the master services put into one node, if they are not cost much resources. Or even master services are distrbuted for 3 nodes, I still can distribute data services into 3 or 4 nodes.
Any suggestion, thanks.
CONFIDENTIALITY NOTICENOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
RE: how to install and distribute master services in 4 nodes.
Posted by Alex Lee <el...@hotmail.com>.
Really appreciate the answers, I am trying to follow it, only the zookeeper need to distribute into 3 nodes. And for slave and client, beside checking all the nodes with datanode and node manager, should I also checkbox with the Client or Reginal Server in the web setting.
Date: Sun, 6 Apr 2014 13:17:44 +0100
Subject: Re: how to install and distribute master services in 4 nodes.
From: orenault@hortonworks.com
To: user@hadoop.apache.org
You can run all master services on a single node or if was you I will run the datanode / nodemananger on every nodes.
Kind regards
Olivier
On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
Trying to install hadoop by horton, there some many master services, such as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image below.
http://postimg.org/image/pwbv6go33/
Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master services, and only 1 node left for datanode.
http://postimg.org/image/gsdr8h719/
Does all the master services neccessary. After searching, seem all neccessary. Can all the master services put into one node, if they are not cost much resources. Or even master services are distrbuted for 3 nodes, I still can distribute data services into 3 or 4 nodes.
Any suggestion, thanks.
CONFIDENTIALITY NOTICENOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
RE: how to install and distribute master services in 4 nodes.
Posted by Alex Lee <el...@hotmail.com>.
Really appreciate the answers, I am trying to follow it, only the zookeeper need to distribute into 3 nodes. And for slave and client, beside checking all the nodes with datanode and node manager, should I also checkbox with the Client or Reginal Server in the web setting.
Date: Sun, 6 Apr 2014 13:17:44 +0100
Subject: Re: how to install and distribute master services in 4 nodes.
From: orenault@hortonworks.com
To: user@hadoop.apache.org
You can run all master services on a single node or if was you I will run the datanode / nodemananger on every nodes.
Kind regards
Olivier
On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
Trying to install hadoop by horton, there some many master services, such as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image below.
http://postimg.org/image/pwbv6go33/
Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master services, and only 1 node left for datanode.
http://postimg.org/image/gsdr8h719/
Does all the master services neccessary. After searching, seem all neccessary. Can all the master services put into one node, if they are not cost much resources. Or even master services are distrbuted for 3 nodes, I still can distribute data services into 3 or 4 nodes.
Any suggestion, thanks.
CONFIDENTIALITY NOTICENOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: how to install and distribute master services in 4 nodes.
Posted by Olivier Renault <or...@hortonworks.com>.
You can run all master services on a single node or if was you I will run
the datanode / nodemananger on every nodes.
Kind regards
Olivier
On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
> Trying to install hadoop by horton, there some many master services, such
> as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image
> below.
> http://postimg.org/image/pwbv6go33/
>
> Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master
> services, and only 1 node left for datanode.
> http://postimg.org/image/gsdr8h719/
>
> Does all the master services neccessary. After searching, seem all
> neccessary. Can all the master services put into one node, if they are not
> cost much resources. Or even master services are distrbuted for 3 nodes, I
> still can distribute data services into 3 or 4 nodes.
>
> Any suggestion, thanks.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: how to install and distribute master services in 4 nodes.
Posted by Olivier Renault <or...@hortonworks.com>.
You can run all master services on a single node or if was you I will run
the datanode / nodemananger on every nodes.
Kind regards
Olivier
On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
> Trying to install hadoop by horton, there some many master services, such
> as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image
> below.
> http://postimg.org/image/pwbv6go33/
>
> Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master
> services, and only 1 node left for datanode.
> http://postimg.org/image/gsdr8h719/
>
> Does all the master services neccessary. After searching, seem all
> neccessary. Can all the master services put into one node, if they are not
> cost much resources. Or even master services are distrbuted for 3 nodes, I
> still can distribute data services into 3 or 4 nodes.
>
> Any suggestion, thanks.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: how to install and distribute master services in 4 nodes.
Posted by Olivier Renault <or...@hortonworks.com>.
You can run all master services on a single node or if was you I will run
the datanode / nodemananger on every nodes.
Kind regards
Olivier
On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
> Trying to install hadoop by horton, there some many master services, such
> as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image
> below.
> http://postimg.org/image/pwbv6go33/
>
> Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master
> services, and only 1 node left for datanode.
> http://postimg.org/image/gsdr8h719/
>
> Does all the master services neccessary. After searching, seem all
> neccessary. Can all the master services put into one node, if they are not
> cost much resources. Or even master services are distrbuted for 3 nodes, I
> still can distribute data services into 3 or 4 nodes.
>
> Any suggestion, thanks.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: how to install and distribute master services in 4 nodes.
Posted by Olivier Renault <or...@hortonworks.com>.
You can run all master services on a single node or if was you I will run
the datanode / nodemananger on every nodes.
Kind regards
Olivier
On 6 Apr 2014 05:26, "Alex Lee" <el...@hotmail.com> wrote:
> Trying to install hadoop by horton, there some many master services, such
> as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image
> below.
> http://postimg.org/image/pwbv6go33/
>
> Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master
> services, and only 1 node left for datanode.
> http://postimg.org/image/gsdr8h719/
>
> Does all the master services neccessary. After searching, seem all
> neccessary. Can all the master services put into one node, if they are not
> cost much resources. Or even master services are distrbuted for 3 nodes, I
> still can distribute data services into 3 or 4 nodes.
>
> Any suggestion, thanks.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
how to install and distribute master services in 4 nodes.
Posted by Alex Lee <el...@hotmail.com>.
Trying to install hadoop by horton, there some many master services, such as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image below.
http://postimg.org/image/pwbv6go33/
Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master services, and only 1 node left for datanode.
http://postimg.org/image/gsdr8h719/
Does all the master services neccessary. After searching, seem all neccessary. Can all the master services put into one node, if they are not cost much resources. Or even master services are distrbuted for 3 nodes, I still can distribute data services into 3 or 4 nodes.
Any suggestion, thanks.
Fwd: customize containter-log4j.properties
Posted by Federico Baldo <ba...@gmail.com>.
Hi there,
I am trying to customize the log format of containers, as a first step I
have tried using a containter-log4j.properties with a different value for
log4j.appender.CLA.layout.ConversionPattern located in the container's
classpath. Tthe result is that the syslog file is written to:
/mnt/var/lib/hadoop/tmp/nm-local-dir/usercache/hadoop/appcache/application_1396678720329_0009/container_1396678720329_0009_01_000009/null/syslog
instead of the configured yarn.app.mapreduce.container.log.dir that is
specified in the cmdline as :
-Dyarn.app.container.log.dir=/mnt/var/log/hadoop/userlogs/application_1396678720329_0009/container_1396678720329_0009_01_000009
Additionally I can see that in Hadoop 2.2.0 the
log4j.configuration=container-log4j.properties is set in
MRApps.addLog4jSystemProperties() and cannot be modified:
http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-mapreduce-client-common/2.2.0/org/apache/hadoop/mapreduce/v2/util/MRApps.java#457
Do you have any idea on how to possibly customize this configuration file?
Thanks
--
Federico Baldo
Fwd: customize containter-log4j.properties
Posted by Federico Baldo <ba...@gmail.com>.
Hi there,
I am trying to customize the log format of containers, as a first step I
have tried using a containter-log4j.properties with a different value for
log4j.appender.CLA.layout.ConversionPattern located in the container's
classpath. Tthe result is that the syslog file is written to:
/mnt/var/lib/hadoop/tmp/nm-local-dir/usercache/hadoop/appcache/application_1396678720329_0009/container_1396678720329_0009_01_000009/null/syslog
instead of the configured yarn.app.mapreduce.container.log.dir that is
specified in the cmdline as :
-Dyarn.app.container.log.dir=/mnt/var/log/hadoop/userlogs/application_1396678720329_0009/container_1396678720329_0009_01_000009
Additionally I can see that in Hadoop 2.2.0 the
log4j.configuration=container-log4j.properties is set in
MRApps.addLog4jSystemProperties() and cannot be modified:
http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-mapreduce-client-common/2.2.0/org/apache/hadoop/mapreduce/v2/util/MRApps.java#457
Do you have any idea on how to possibly customize this configuration file?
Thanks
--
Federico Baldo
how to install and distribute master services in 4 nodes.
Posted by Alex Lee <el...@hotmail.com>.
Trying to install hadoop by horton, there some many master services, such as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image below.
http://postimg.org/image/pwbv6go33/
Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master services, and only 1 node left for datanode.
http://postimg.org/image/gsdr8h719/
Does all the master services neccessary. After searching, seem all neccessary. Can all the master services put into one node, if they are not cost much resources. Or even master services are distrbuted for 3 nodes, I still can distribute data services into 3 or 4 nodes.
Any suggestion, thanks.
how to install and distribute master services in 4 nodes.
Posted by Alex Lee <el...@hotmail.com>.
Trying to install hadoop by horton, there some many master services, such as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image below.
http://postimg.org/image/pwbv6go33/
Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master services, and only 1 node left for datanode.
http://postimg.org/image/gsdr8h719/
Does all the master services neccessary. After searching, seem all neccessary. Can all the master services put into one node, if they are not cost much resources. Or even master services are distrbuted for 3 nodes, I still can distribute data services into 3 or 4 nodes.
Any suggestion, thanks.
how to install and distribute master services in 4 nodes.
Posted by Alex Lee <el...@hotmail.com>.
Trying to install hadoop by horton, there some many master services, such as HDFS, Yarn+mapreduce, Nagios, Ganglia, Hive, Hbase and Pig as the image below.
http://postimg.org/image/pwbv6go33/
Currently, only have 4 nodes, and as below, 3 nodes may ocuppy by master services, and only 1 node left for datanode.
http://postimg.org/image/gsdr8h719/
Does all the master services neccessary. After searching, seem all neccessary. Can all the master services put into one node, if they are not cost much resources. Or even master services are distrbuted for 3 nodes, I still can distribute data services into 3 or 4 nodes.
Any suggestion, thanks.
Fwd: customize containter-log4j.properties
Posted by Federico Baldo <ba...@gmail.com>.
Hi there,
I am trying to customize the log format of containers, as a first step I
have tried using a containter-log4j.properties with a different value for
log4j.appender.CLA.layout.ConversionPattern located in the container's
classpath. Tthe result is that the syslog file is written to:
/mnt/var/lib/hadoop/tmp/nm-local-dir/usercache/hadoop/appcache/application_1396678720329_0009/container_1396678720329_0009_01_000009/null/syslog
instead of the configured yarn.app.mapreduce.container.log.dir that is
specified in the cmdline as :
-Dyarn.app.container.log.dir=/mnt/var/log/hadoop/userlogs/application_1396678720329_0009/container_1396678720329_0009_01_000009
Additionally I can see that in Hadoop 2.2.0 the
log4j.configuration=container-log4j.properties is set in
MRApps.addLog4jSystemProperties() and cannot be modified:
http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-mapreduce-client-common/2.2.0/org/apache/hadoop/mapreduce/v2/util/MRApps.java#457
Do you have any idea on how to possibly customize this configuration file?
Thanks
--
Federico Baldo