You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Raj Hadoop <ha...@yahoo.com> on 2013/08/30 00:11:40 UTC
Hadoop Clients (Hive,Pig) and Hadoop Cluster
Hi,
I am trying to setup a multi node hadoop cluster. I am trying to understand where hadoop clients like (Hive,Pig,Sqoop) would be installed in the Hadoop Cluster.
Say - I have three Linux machines-
Node 1 - Master - (Name Node , Job Tracker and Secondary Name Node)
Node 2 - Slave (Task Tracker,Data Node)
Node 3 - Slave (Task Tracker,Data Node)
On which machines should I install Hive? Should it be installed or Can it be installed on a separate machine? What user and privileges are required ?
On which machines should I install Pig? Should it be installed or Can it be installed on a separate machine? What user and privileges are required ?
On which machines should I install Sqoop? Should it be installed or Can it be installed on a separate machine? What user and privileges are required ?
Thanks,
Raj
Re: Hadoop Clients (Hive,Pig) and Hadoop Cluster
Posted by Peyman Mohajerian <mo...@gmail.com>.
Regarding Sqoop, you can install it wherever you would have access to your
database and HDFS cluster, you could e.g. install it on the namenode if you
want it as long as it has access to the database that is the source or
target of your data transfer.
On Thu, Aug 29, 2013 at 3:11 PM, Raj Hadoop <ha...@yahoo.com> wrote:
> Hi,
>
> I am trying to setup a multi node hadoop cluster. I am trying to
> understand where hadoop clients like (Hive,Pig,Sqoop) would be installed in
> the Hadoop Cluster.
>
> Say - I have three Linux machines-
>
> Node 1 - Master - (Name Node , Job Tracker and Secondary Name Node)
> Node 2 - Slave (Task Tracker,Data Node)
> Node 3 - Slave (Task Tracker,Data Node)
>
> On which machines should I install Hive? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
> On which machines should I install Pig? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
> On which machines should I install Sqoop? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
>
> Thanks,
> Raj
>
Re: Hadoop Clients (Hive,Pig) and Hadoop Cluster
Posted by Xuri Nagarin <se...@gmail.com>.
Yes, ideally you want to setup a 4th gateway node to run clients.
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Security-Guide/AppxG-Setting-Up-Gateway.html
On Thu, Aug 29, 2013 at 3:11 PM, Raj Hadoop <ha...@yahoo.com> wrote:
> Hi,
>
> I am trying to setup a multi node hadoop cluster. I am trying to
> understand where hadoop clients like (Hive,Pig,Sqoop) would be installed in
> the Hadoop Cluster.
>
> Say - I have three Linux machines-
>
> Node 1 - Master - (Name Node , Job Tracker and Secondary Name Node)
> Node 2 - Slave (Task Tracker,Data Node)
> Node 3 - Slave (Task Tracker,Data Node)
>
> On which machines should I install Hive? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
> On which machines should I install Pig? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
> On which machines should I install Sqoop? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
>
> Thanks,
> Raj
>
Re: Hadoop Clients (Hive,Pig) and Hadoop Cluster
Posted by Peyman Mohajerian <mo...@gmail.com>.
Regarding Sqoop, you can install it wherever you would have access to your
database and HDFS cluster, you could e.g. install it on the namenode if you
want it as long as it has access to the database that is the source or
target of your data transfer.
On Thu, Aug 29, 2013 at 3:11 PM, Raj Hadoop <ha...@yahoo.com> wrote:
> Hi,
>
> I am trying to setup a multi node hadoop cluster. I am trying to
> understand where hadoop clients like (Hive,Pig,Sqoop) would be installed in
> the Hadoop Cluster.
>
> Say - I have three Linux machines-
>
> Node 1 - Master - (Name Node , Job Tracker and Secondary Name Node)
> Node 2 - Slave (Task Tracker,Data Node)
> Node 3 - Slave (Task Tracker,Data Node)
>
> On which machines should I install Hive? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
> On which machines should I install Pig? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
> On which machines should I install Sqoop? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
>
> Thanks,
> Raj
>
Re: Hadoop Clients (Hive,Pig) and Hadoop Cluster
Posted by Xuri Nagarin <se...@gmail.com>.
Yes, ideally you want to setup a 4th gateway node to run clients.
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Security-Guide/AppxG-Setting-Up-Gateway.html
On Thu, Aug 29, 2013 at 3:11 PM, Raj Hadoop <ha...@yahoo.com> wrote:
> Hi,
>
> I am trying to setup a multi node hadoop cluster. I am trying to
> understand where hadoop clients like (Hive,Pig,Sqoop) would be installed in
> the Hadoop Cluster.
>
> Say - I have three Linux machines-
>
> Node 1 - Master - (Name Node , Job Tracker and Secondary Name Node)
> Node 2 - Slave (Task Tracker,Data Node)
> Node 3 - Slave (Task Tracker,Data Node)
>
> On which machines should I install Hive? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
> On which machines should I install Pig? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
> On which machines should I install Sqoop? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
>
> Thanks,
> Raj
>
Re: Hadoop Clients (Hive,Pig) and Hadoop Cluster
Posted by Peyman Mohajerian <mo...@gmail.com>.
Regarding Sqoop, you can install it wherever you would have access to your
database and HDFS cluster, you could e.g. install it on the namenode if you
want it as long as it has access to the database that is the source or
target of your data transfer.
On Thu, Aug 29, 2013 at 3:11 PM, Raj Hadoop <ha...@yahoo.com> wrote:
> Hi,
>
> I am trying to setup a multi node hadoop cluster. I am trying to
> understand where hadoop clients like (Hive,Pig,Sqoop) would be installed in
> the Hadoop Cluster.
>
> Say - I have three Linux machines-
>
> Node 1 - Master - (Name Node , Job Tracker and Secondary Name Node)
> Node 2 - Slave (Task Tracker,Data Node)
> Node 3 - Slave (Task Tracker,Data Node)
>
> On which machines should I install Hive? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
> On which machines should I install Pig? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
> On which machines should I install Sqoop? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
>
> Thanks,
> Raj
>
Re: Hadoop Clients (Hive,Pig) and Hadoop Cluster
Posted by Peyman Mohajerian <mo...@gmail.com>.
Regarding Sqoop, you can install it wherever you would have access to your
database and HDFS cluster, you could e.g. install it on the namenode if you
want it as long as it has access to the database that is the source or
target of your data transfer.
On Thu, Aug 29, 2013 at 3:11 PM, Raj Hadoop <ha...@yahoo.com> wrote:
> Hi,
>
> I am trying to setup a multi node hadoop cluster. I am trying to
> understand where hadoop clients like (Hive,Pig,Sqoop) would be installed in
> the Hadoop Cluster.
>
> Say - I have three Linux machines-
>
> Node 1 - Master - (Name Node , Job Tracker and Secondary Name Node)
> Node 2 - Slave (Task Tracker,Data Node)
> Node 3 - Slave (Task Tracker,Data Node)
>
> On which machines should I install Hive? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
> On which machines should I install Pig? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
> On which machines should I install Sqoop? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
>
> Thanks,
> Raj
>
Re: Hadoop Clients (Hive,Pig) and Hadoop Cluster
Posted by Xuri Nagarin <se...@gmail.com>.
Yes, ideally you want to setup a 4th gateway node to run clients.
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Security-Guide/AppxG-Setting-Up-Gateway.html
On Thu, Aug 29, 2013 at 3:11 PM, Raj Hadoop <ha...@yahoo.com> wrote:
> Hi,
>
> I am trying to setup a multi node hadoop cluster. I am trying to
> understand where hadoop clients like (Hive,Pig,Sqoop) would be installed in
> the Hadoop Cluster.
>
> Say - I have three Linux machines-
>
> Node 1 - Master - (Name Node , Job Tracker and Secondary Name Node)
> Node 2 - Slave (Task Tracker,Data Node)
> Node 3 - Slave (Task Tracker,Data Node)
>
> On which machines should I install Hive? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
> On which machines should I install Pig? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
> On which machines should I install Sqoop? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
>
> Thanks,
> Raj
>
Re: Hadoop Clients (Hive,Pig) and Hadoop Cluster
Posted by Xuri Nagarin <se...@gmail.com>.
Yes, ideally you want to setup a 4th gateway node to run clients.
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Security-Guide/AppxG-Setting-Up-Gateway.html
On Thu, Aug 29, 2013 at 3:11 PM, Raj Hadoop <ha...@yahoo.com> wrote:
> Hi,
>
> I am trying to setup a multi node hadoop cluster. I am trying to
> understand where hadoop clients like (Hive,Pig,Sqoop) would be installed in
> the Hadoop Cluster.
>
> Say - I have three Linux machines-
>
> Node 1 - Master - (Name Node , Job Tracker and Secondary Name Node)
> Node 2 - Slave (Task Tracker,Data Node)
> Node 3 - Slave (Task Tracker,Data Node)
>
> On which machines should I install Hive? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
> On which machines should I install Pig? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
> On which machines should I install Sqoop? Should it be installed or Can it
> be installed on a separate machine? What user and privileges are required ?
>
> Thanks,
> Raj
>