You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by Matthew Johnson <ma...@algomi.com> on 2015/03/23 15:51:33 UTC
Hadoop Yarn with Phoenix?
Hi all,
Currently when I start up my cluster, I use the packaged scripts for
starting up Hadoop (v2.6.0) and HBase (0.98.9-hadoop2). This results in the
following processes running:
HADOOP:
*hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_namenode ---
org.apache.hadoop.hdfs.server.namenode.NameNode*
*hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_datanode ---
org.apache.hadoop.hdfs.server.datanode.DataNode*
*hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_secondarynamenode ---
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode*
*hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_resourcemanager ---
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager*
*hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_nodemanager ---
org.apache.hadoop.yarn.server.nodemanager.NodeManager*
HBASE:
*hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_master ---
org.apache.hadoop.hbase.master.HMaster *
*hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_regionserver ---
org.apache.hadoop.hbase.regionserver.HRegionServer*
My understanding is that the Hadoop processes “Resource Manager” and “Node
Manager” are part of Hadoop YARN (correct me if I’m wrong). If all I want
to do with my cluster is put/get/scan directly with HBase and
upsert/select/join with Phoenix, do I need to have YARN running? Does
Phoenix use the YARN framework for distributing parts of queries, for
example if I count the number of rows in a table? Or will it just scan
through and count rows directly in HBase?
Thanks!
Matt
RE: Hadoop Yarn with Phoenix?
Posted by Matthew Johnson <ma...@algomi.com>.
Perfect, thanks for confirming! Saves a little bit of complexity on the
scripts / cluster monitoring etc.
Cheers,
Matt
*From:* Ravi Kiran [mailto:maghamravikiran@gmail.com]
*Sent:* 23 March 2015 16:21
*To:* user@phoenix.apache.org
*Subject:* Re: Hadoop Yarn with Phoenix?
Hi Matt,
Your understanding is right. You don't need YARN services running as
long as you don't run any Map Reduce jobs(CSV Bulk Loading, Map Reduce ,
phoenix-pig) .
Regards
Ravi
On Mon, Mar 23, 2015 at 7:51 AM, Matthew Johnson <ma...@algomi.com>
wrote:
Hi all,
Currently when I start up my cluster, I use the packaged scripts for
starting up Hadoop (v2.6.0) and HBase (0.98.9-hadoop2). This results in the
following processes running:
HADOOP:
*hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_namenode ---
org.apache.hadoop.hdfs.server.namenode.NameNode*
*hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_datanode ---
org.apache.hadoop.hdfs.server.datanode.DataNode*
*hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_secondarynamenode ---
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode*
*hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_resourcemanager ---
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager*
*hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_nodemanager ---
org.apache.hadoop.yarn.server.nodemanager.NodeManager*
HBASE:
*hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_master ---
org.apache.hadoop.hbase.master.HMaster *
*hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_regionserver ---
org.apache.hadoop.hbase.regionserver.HRegionServer*
My understanding is that the Hadoop processes “Resource Manager” and “Node
Manager” are part of Hadoop YARN (correct me if I’m wrong). If all I want
to do with my cluster is put/get/scan directly with HBase and
upsert/select/join with Phoenix, do I need to have YARN running? Does
Phoenix use the YARN framework for distributing parts of queries, for
example if I count the number of rows in a table? Or will it just scan
through and count rows directly in HBase?
Thanks!
Matt
Re: Hadoop Yarn with Phoenix?
Posted by Ravi Kiran <ma...@gmail.com>.
Hi Matt,
Your understanding is right. You don't need YARN services running as
long as you don't run any Map Reduce jobs(CSV Bulk Loading, Map Reduce ,
phoenix-pig) .
Regards
Ravi
On Mon, Mar 23, 2015 at 7:51 AM, Matthew Johnson <ma...@algomi.com>
wrote:
> Hi all,
>
>
>
> Currently when I start up my cluster, I use the packaged scripts for
> starting up Hadoop (v2.6.0) and HBase (0.98.9-hadoop2). This results in the
> following processes running:
>
>
>
> HADOOP:
>
> *hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_namenode ---
> org.apache.hadoop.hdfs.server.namenode.NameNode*
>
> *hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_datanode ---
> org.apache.hadoop.hdfs.server.datanode.DataNode*
>
> *hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_secondarynamenode ---
> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode*
>
> *hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_resourcemanager ---
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager*
>
> *hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_nodemanager ---
> org.apache.hadoop.yarn.server.nodemanager.NodeManager*
>
>
>
> HBASE:
>
> *hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_master ---
> org.apache.hadoop.hbase.master.HMaster *
>
> *hadoop --- /usr/java/jre1.7.0_65/bin/java -Dproc_regionserver ---
> org.apache.hadoop.hbase.regionserver.HRegionServer*
>
>
>
> My understanding is that the Hadoop processes “Resource Manager” and “Node
> Manager” are part of Hadoop YARN (correct me if I’m wrong). If all I want
> to do with my cluster is put/get/scan directly with HBase and
> upsert/select/join with Phoenix, do I need to have YARN running? Does
> Phoenix use the YARN framework for distributing parts of queries, for
> example if I count the number of rows in a table? Or will it just scan
> through and count rows directly in HBase?
>
>
>
> Thanks!
>
> Matt
>
>
>
>
>
>
>