You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Patai Sangbutsarakum <si...@gmail.com> on 2013/10/16 23:24:46 UTC
even possible?
Question is on cdh3u4, the cluster was setup before I owned this cluster,
and somehow the namenode/jobtracker/datanode/tasktracker every server
process is run by a user named foo, and all job are launch and run by foo
user include HDFS directories/files structure owenership, basically foo is
everywhere.
Today i start to think of trying to correct this by having
has namenode + datanode run by hdfs user
has jobtracker + tasktracker run by mapred user
So far, i have a very short list that need to be changed, and i will try
out in the test cluster.
eg.
create use hdfs, mapred every where
ownership of dfs.name.dir, dfs.data.dir, fs.checkpoint.dir will change to
hdfs
ownership of mapred.local.dir, will change to mapred
restart the cluster with hdfs for HDFS side, and mapred for MapRed side.
i am 100% sure that i missed certain things that have to take care, I will
really appreciate all the input.
However, the original question i would love to ask is this even feasible or
make sense trying to change this.
Thanks
P
Re: even possible?
Posted by Pradeep Gollakota <pr...@gmail.com>.
Don't fix it if it ain't broken =P
There shouldn't be any reason why you couldn't change it (back) to the
standard way that cloudera distributions are set up. Off the top of my
head, I can't think of anything that you're missing. But at the same time,
if your cluster is working as is, why change it?
On Wed, Oct 16, 2013 at 2:24 PM, Patai Sangbutsarakum <
silvianhadoop@gmail.com> wrote:
> Question is on cdh3u4, the cluster was setup before I owned this cluster,
> and somehow the namenode/jobtracker/datanode/tasktracker every server
> process is run by a user named foo, and all job are launch and run by foo
> user include HDFS directories/files structure owenership, basically foo is
> everywhere.
>
> Today i start to think of trying to correct this by having
> has namenode + datanode run by hdfs user
> has jobtracker + tasktracker run by mapred user
>
> So far, i have a very short list that need to be changed, and i will try
> out in the test cluster.
> eg.
> create use hdfs, mapred every where
> ownership of dfs.name.dir, dfs.data.dir, fs.checkpoint.dir will change to
> hdfs
> ownership of mapred.local.dir, will change to mapred
> restart the cluster with hdfs for HDFS side, and mapred for MapRed side.
>
> i am 100% sure that i missed certain things that have to take care, I will
> really appreciate all the input.
>
> However, the original question i would love to ask is this even feasible
> or make sense trying to change this.
>
>
> Thanks
> P
>
Re: even possible?
Posted by Pradeep Gollakota <pr...@gmail.com>.
Don't fix it if it ain't broken =P
There shouldn't be any reason why you couldn't change it (back) to the
standard way that cloudera distributions are set up. Off the top of my
head, I can't think of anything that you're missing. But at the same time,
if your cluster is working as is, why change it?
On Wed, Oct 16, 2013 at 2:24 PM, Patai Sangbutsarakum <
silvianhadoop@gmail.com> wrote:
> Question is on cdh3u4, the cluster was setup before I owned this cluster,
> and somehow the namenode/jobtracker/datanode/tasktracker every server
> process is run by a user named foo, and all job are launch and run by foo
> user include HDFS directories/files structure owenership, basically foo is
> everywhere.
>
> Today i start to think of trying to correct this by having
> has namenode + datanode run by hdfs user
> has jobtracker + tasktracker run by mapred user
>
> So far, i have a very short list that need to be changed, and i will try
> out in the test cluster.
> eg.
> create use hdfs, mapred every where
> ownership of dfs.name.dir, dfs.data.dir, fs.checkpoint.dir will change to
> hdfs
> ownership of mapred.local.dir, will change to mapred
> restart the cluster with hdfs for HDFS side, and mapred for MapRed side.
>
> i am 100% sure that i missed certain things that have to take care, I will
> really appreciate all the input.
>
> However, the original question i would love to ask is this even feasible
> or make sense trying to change this.
>
>
> Thanks
> P
>
Re: even possible?
Posted by Pradeep Gollakota <pr...@gmail.com>.
Don't fix it if it ain't broken =P
There shouldn't be any reason why you couldn't change it (back) to the
standard way that cloudera distributions are set up. Off the top of my
head, I can't think of anything that you're missing. But at the same time,
if your cluster is working as is, why change it?
On Wed, Oct 16, 2013 at 2:24 PM, Patai Sangbutsarakum <
silvianhadoop@gmail.com> wrote:
> Question is on cdh3u4, the cluster was setup before I owned this cluster,
> and somehow the namenode/jobtracker/datanode/tasktracker every server
> process is run by a user named foo, and all job are launch and run by foo
> user include HDFS directories/files structure owenership, basically foo is
> everywhere.
>
> Today i start to think of trying to correct this by having
> has namenode + datanode run by hdfs user
> has jobtracker + tasktracker run by mapred user
>
> So far, i have a very short list that need to be changed, and i will try
> out in the test cluster.
> eg.
> create use hdfs, mapred every where
> ownership of dfs.name.dir, dfs.data.dir, fs.checkpoint.dir will change to
> hdfs
> ownership of mapred.local.dir, will change to mapred
> restart the cluster with hdfs for HDFS side, and mapred for MapRed side.
>
> i am 100% sure that i missed certain things that have to take care, I will
> really appreciate all the input.
>
> However, the original question i would love to ask is this even feasible
> or make sense trying to change this.
>
>
> Thanks
> P
>
Re: even possible?
Posted by Pradeep Gollakota <pr...@gmail.com>.
Don't fix it if it ain't broken =P
There shouldn't be any reason why you couldn't change it (back) to the
standard way that cloudera distributions are set up. Off the top of my
head, I can't think of anything that you're missing. But at the same time,
if your cluster is working as is, why change it?
On Wed, Oct 16, 2013 at 2:24 PM, Patai Sangbutsarakum <
silvianhadoop@gmail.com> wrote:
> Question is on cdh3u4, the cluster was setup before I owned this cluster,
> and somehow the namenode/jobtracker/datanode/tasktracker every server
> process is run by a user named foo, and all job are launch and run by foo
> user include HDFS directories/files structure owenership, basically foo is
> everywhere.
>
> Today i start to think of trying to correct this by having
> has namenode + datanode run by hdfs user
> has jobtracker + tasktracker run by mapred user
>
> So far, i have a very short list that need to be changed, and i will try
> out in the test cluster.
> eg.
> create use hdfs, mapred every where
> ownership of dfs.name.dir, dfs.data.dir, fs.checkpoint.dir will change to
> hdfs
> ownership of mapred.local.dir, will change to mapred
> restart the cluster with hdfs for HDFS side, and mapred for MapRed side.
>
> i am 100% sure that i missed certain things that have to take care, I will
> really appreciate all the input.
>
> However, the original question i would love to ask is this even feasible
> or make sense trying to change this.
>
>
> Thanks
> P
>