You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by 王学超 <wa...@gmail.com> on 2007/12/12 02:41:09 UTC
some questions about hadoop
hi, everyone!
I'm new,I am not familiar with hadoop，so I have some following
questions,please help me. thanks
1、Are there 3rd party technologies and included in hadoop or dependencies?
2、How is hadoop's high-availability handled? Built-in or requires
separate HA manager?
3、How is security handled?
4、How is error logging handled?
Re: some questions about hadoop
Posted by Allen Wittenauer <aw...@yahoo-inc.com>.
On 12/12/07 9:16 AM, "Ted Dunning" <td...@veoh.com> wrote:
> 3. There is currently no security. Weak user level security will appear
> soon (but you will still be able to lie about who you are). Stronger
> security is in the works, but you should expect to protect a Hadoop cluster
> from the outside.
It is probably worth pointing out that the approach we took to
protecting the HDFS was to firewall off the fs.default.name machine/port and
50075 (datanode.something.or.other) while also having users use another port
that pointed to an apache instance that wrapped the dfs browser stuff. Hosts
that can connect to the HDFS is then strictly controlled, including limited
root access, etc.
[Yes, this is painful.]
Re: some questions about hadoop
Posted by Ted Dunning <td...@veoh.com>.
3. There is currently no security. Weak user level security will appear
soon (but you will still be able to lie about who you are). Stronger
security is in the works, but you should expect to protect a Hadoop cluster
from the outside.
2. High availability is inherent in hadoop's map-reduce system and storage
system. You have to use external tools to achieve full HA capabilities for
the central name node.
On 12/12/07 12:21 AM, "王学超" <wa...@gmail.com> wrote:
> thank you ,Khalil Honsali
Can others can give me some other suggestion?
I am
> eagerly waiting for your reply.
2007/12/12, Khalil Honsali
> <k....@gmail.com>:
>
> As far as my basic understanding goes:
> 1 - 3rd
> party - or so technologies are located in the folder hadoop-xx/lib
> ,
> for
> example apache's commons-logging technologie ...
> 4 - ... which is used by
> hadoop for logging into the log files specified
> in
> the configuration
> files. log files are usually in the hadoop-xx/logs/
> directory and there are
> two types of logging: INFO (information messages)
> and ERROR. you can tune
> logging schemes via hadoop-xx/conf/commons-
> logging.properties or
> hadoop-xx/conf/log4j.properties
>
> On 12/12/2007, 王学超 <wa...@gmail.com>
> wrote:
> >
> > hi, everyone!
> > I'm new,I am not familiar with hadoop，so I
> have some following
> > questions,please help me. thanks
> > 1、Are there 3rd
> party technologies and included in hadoop or
> dependencies?
> > 2、How is
> hadoop's high-availability handled? Built-in or requires
> > separate HA
> manager?
> > 3、How is security handled?
> > 4、How is error logging handled?
>
> >
>
>
>
> --
> ---------------------------------------------------------
> شهر
> مبارك كريم
> كل عام و أنتم بخير
>
> ---------------------------------------------------------
> Honsali Khalil − 本
> 査理 カリル
> Academic>Japan>NIT>Grad. Sc. Eng.>Dept. CS>Matsuo&Tsumura Lab.
>
> http://www.matlab.nitech.ac.jp/~k-hon/<
>
> http://www.matlab.nitech.ac.jp/%7Ek-hon/>
> +81 (zero-)eight-zero 5134 8119
>
> k.honsali@ezweb.ne.jp (instant reply mail)
>
--
此致
敬
> 礼
Re: some questions about hadoop
Posted by 王学超 <wa...@gmail.com>.
thank you ,Khalil Honsali
Can others can give me some other suggestion?
I am eagerly waiting for your reply.
2007/12/12, Khalil Honsali <k....@gmail.com>:
>
> As far as my basic understanding goes:
> 1 - 3rd party - or so technologies are located in the folder hadoop-xx/lib
> ,
> for example apache's commons-logging technologie ...
> 4 - ... which is used by hadoop for logging into the log files specified
> in
> the configuration files. log files are usually in the hadoop-xx/logs/
> directory and there are two types of logging: INFO (information messages)
> and ERROR. you can tune logging schemes via hadoop-xx/conf/commons-
> logging.properties or hadoop-xx/conf/log4j.properties
>
> On 12/12/2007, 王学超 <wa...@gmail.com> wrote:
> >
> > hi, everyone!
> > I'm new,I am not familiar with hadoop，so I have some following
> > questions,please help me. thanks
> > 1、Are there 3rd party technologies and included in hadoop or
> dependencies?
> > 2、How is hadoop's high-availability handled? Built-in or requires
> > separate HA manager?
> > 3、How is security handled?
> > 4、How is error logging handled?
> >
>
>
>
> --
> ---------------------------------------------------------
> شهر مبارك كريم
> كل عام و أنتم بخير
> ---------------------------------------------------------
> Honsali Khalil − 本査理 カリル
> Academic>Japan>NIT>Grad. Sc. Eng.>Dept. CS>Matsuo&Tsumura Lab.
> http://www.matlab.nitech.ac.jp/~k-hon/<
> http://www.matlab.nitech.ac.jp/%7Ek-hon/>
> +81 (zero-)eight-zero 5134 8119
> k.honsali@ezweb.ne.jp (instant reply mail)
>
--
此致
敬礼
Re: some questions about hadoop
Posted by Khalil Honsali <k....@gmail.com>.
As far as my basic understanding goes:
1 - 3rd party - or so technologies are located in the folder hadoop-xx/lib ,
for example apache's commons-logging technologie ...
4 - ... which is used by hadoop for logging into the log files specified in
the configuration files. log files are usually in the hadoop-xx/logs/
directory and there are two types of logging: INFO (information messages)
and ERROR. you can tune logging schemes via hadoop-xx/conf/commons-
logging.properties or hadoop-xx/conf/log4j.properties
On 12/12/2007, 王学超 <wa...@gmail.com> wrote:
>
> hi, everyone!
> I'm new,I am not familiar with hadoop，so I have some following
> questions,please help me. thanks
> 1、Are there 3rd party technologies and included in hadoop or dependencies?
> 2、How is hadoop's high-availability handled? Built-in or requires
> separate HA manager?
> 3、How is security handled?
> 4、How is error logging handled?
>
--
---------------------------------------------------------
شهر مبارك كريم
كل عام و أنتم بخير
---------------------------------------------------------
Honsali Khalil − 本査理 カリル
Academic>Japan>NIT>Grad. Sc. Eng.>Dept. CS>Matsuo&Tsumura Lab.
http://www.matlab.nitech.ac.jp/~k-hon/<http://www.matlab.nitech.ac.jp/%7Ek-hon/>
+81 (zero-)eight-zero 5134 8119
k.honsali@ezweb.ne.jp (instant reply mail)