You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Venkateswara Rao Dokku <dv...@gmail.com> on 2012/09/27 08:56:32 UTC
Hbase clustering
Hi,
I am completely new to Hbase. I want to cluster the Hbase on two
nodes.I installed hadoop,hbase on the two nodes & my conf files are as
given below.
*cat conf/regionservers *
hbase-regionserver1
hbase-master
*cat conf/masters *
hadoop-namenode
* cat conf/slaves *
hadoop-datanode1
*vim conf/hdfs-site.xml *
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>dfs.replication</name>
<value>2</value>
<description>Default block replication.The actual number of
replications can be specified when the file is created. The default is used
if replication is not specified in create time.
</description>
</property>
<property>
<name>dfs.support.append</name>
<value>true</value>
<description>Default block replication.The actual number of
replications can be specified when the file is created. The default is used
if replication is not specified in create time.
</description>
</property>
</configuration>
*& finally my /etc/hosts file is *
127.0.0.1 localhost
127.0.0.1 oc-PowerEdge-R610
10.2.32.48 hbase-master hadoop-namenode
10.240.13.35 hbase-regionserver1 hadoop-datanode1
The above files are identical on both of the machines. The following are
the processes that are running on my m/c's when I ran start scripts in
hadoop as well as hbase
*hadoop-namenode:*
HQuorumPeer
HMaster
Main
HRegionServer
SecondaryNameNode
Jps
NameNode
JobTracker
*hadoop-datanode1:*
TaskTracker
Jps
DataNode
-- process information unavailable
Main
NC
HRegionServer
I can able to create,list & scan tables on the *hadoop-namenode* machine
using Hbase shell. But while trying to run the same on the *
hadoop-datanode1 *machine I couldn't able to do it as I am getting
following error.
hbase(main):001:0> list
TABLE
ERROR: org.apache.hadoop.hbase.MasterNotRunningException: Retried 7 times
Here is some help for this command:
List all tables in hbase. Optional regular expression parameter could
be used to filter the output. Examples:
hbase> list
hbase> list 'abc.*'
How can I list,scan the tables that are created by the *hadoop-namenode *
from the *hadoop-datanode1* machine. Similarly Can I create some tables on
*hadoop-datanode1 *& can I access them from the *hadoop-namenode * &
vice-versa as the data is distributed as this is a cluster.
--
Thanks & Regards,
Venkateswara Rao Dokku,
Software Engineer,One Convergence Devices Pvt Ltd.,
Jubille Hills,Hyderabad.
Re: Hbase clustering
Posted by Venkateswara Rao Dokku <dv...@gmail.com>.
On Thu, Sep 27, 2012 at 1:09 PM, Venkateswara Rao Dokku <dvrao.584@gmail.com
> wrote:
> I can see that HMaster is not started on the data-node machine when the
> start scripts in hadoop & hbase ran on the hadoop-namenode. My doubt is
> that,Shall we have to start that master on the hadoop-datanode1 too or the
> hadoop-datanode1 will access the Hmaster that is running on the
> hadoop-namenode to create,list,scan tables as the two nodes are in the
> cluster as namenode & datanode.
>
>
> On Thu, Sep 27, 2012 at 1:02 PM, n keywal <nk...@gmail.com> wrote:
>
>> Hi,
>>
>> I would like to direct you to the reference guide, but I must acknowledge
>> that, well, it's a reference guide, hence not really easy for a plain new
>> start.
>> You should have a look at Lars' blog (and may be buy his book), and
>> especially this entry:
>> http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
>>
>> Some hints however:
>> - the replication occurs at the hdfs level, not the hbase level: hbase
>> writes files that are split in hdfs blocks that are replicated accross the
>> datanodes. If you want to check the replications, you must look at what
>> files are written by hbase and how they have been split in blocks by hdfs
>> and how these blocks have been replicated. That will be in the hdfs
>> interface. As a side note, it's not the easiest thing to learn when you
>> start :-)
>> - The error > ERROR: org.apache.hadoop.hbase.MasterNotRunningException:
>> Retried 7 times
>> this is not linked to replication or whatever. It means that second
>> machine cannot find the master. You need to fix this first. (googling &
>> checking the logs).
>>
>>
>> Good luck,
>>
>> Nicolas
>>
>>
>>
>>
>> On Thu, Sep 27, 2012 at 9:07 AM, Venkateswara Rao Dokku <
>> dvrao.584@gmail.com
>> > wrote:
>>
>> > How can we verify that the data(tables) is distributed across the
>> cluster??
>> > Is there a way to confirm it that the data is distributed across all the
>> > nodes in the cluster.?
>> >
>> > On Thu, Sep 27, 2012 at 12:26 PM, Venkateswara Rao Dokku <
>> > dvrao.584@gmail.com> wrote:
>> >
>> > > Hi,
>> > > I am completely new to Hbase. I want to cluster the Hbase on two
>> > > nodes.I installed hadoop,hbase on the two nodes & my conf files are as
>> > > given below.
>> > > *cat conf/regionservers *
>> > > hbase-regionserver1
>> > > hbase-master
>> > > *cat conf/masters *
>> > > hadoop-namenode
>> > > * cat conf/slaves *
>> > > hadoop-datanode1
>> > > *vim conf/hdfs-site.xml *
>> > > <?xml version="1.0"?>
>> > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>> > >
>> > > <!-- Put site-specific property overrides in this file. -->
>> > >
>> > > <configuration>
>> > > <property>
>> > > <name>dfs.replication</name>
>> > > <value>2</value>
>> > > <description>Default block replication.The actual number of
>> > > replications can be specified when the file is created. The default is
>> > used
>> > > if replication is not specified in create time.
>> > > </description>
>> > > </property>
>> > > <property>
>> > > <name>dfs.support.append</name>
>> > > <value>true</value>
>> > > <description>Default block replication.The actual number of
>> > > replications can be specified when the file is created. The default is
>> > used
>> > > if replication is not specified in create time.
>> > > </description>
>> > > </property>
>> > > </configuration>
>> > > *& finally my /etc/hosts file is *
>> > > 127.0.0.1 localhost
>> > > 127.0.0.1 oc-PowerEdge-R610
>> > > 10.2.32.48 hbase-master hadoop-namenode
>> > > 10.240.13.35 hbase-regionserver1 hadoop-datanode1
>> > > The above files are identical on both of the machines. The following
>> are
>> > > the processes that are running on my m/c's when I ran start scripts in
>> > > hadoop as well as hbase
>> > > *hadoop-namenode:*
>> > > HQuorumPeer
>> > > HMaster
>> > > Main
>> > > HRegionServer
>> > > SecondaryNameNode
>> > > Jps
>> > > NameNode
>> > > JobTracker
>> > > *hadoop-datanode1:*
>> > >
>> > > TaskTracker
>> > > Jps
>> > > DataNode
>> > > -- process information unavailable
>> > > Main
>> > > NC
>> > > HRegionServer
>> > >
>> > > I can able to create,list & scan tables on the *hadoop-namenode*
>> machine
>> > > using Hbase shell. But while trying to run the same on the *
>> > > hadoop-datanode1 *machine I couldn't able to do it as I am getting
>> > > following error.
>> > > hbase(main):001:0> list
>> > > TABLE
>> > >
>> > >
>> > > ERROR: org.apache.hadoop.hbase.MasterNotRunningException: Retried 7
>> times
>> > >
>> > > Here is some help for this command:
>> > > List all tables in hbase. Optional regular expression parameter could
>> > > be used to filter the output. Examples:
>> > >
>> > > hbase> list
>> > > hbase> list 'abc.*'
>> > > How can I list,scan the tables that are created by the
>> *hadoop-namenode *
>> > > from the *hadoop-datanode1* machine. Similarly Can I create some
>> tables
>> > > on *hadoop-datanode1 *& can I access them from the *hadoop-namenode
>> * &
>> > > vice-versa as the data is distributed as this is a cluster.
>> > >
>> > >
>> > >
>> > > --
>> > > Thanks & Regards,
>> > > Venkateswara Rao Dokku,
>> > > Software Engineer,One Convergence Devices Pvt Ltd.,
>> > > Jubille Hills,Hyderabad.
>> > >
>> > >
>> >
>> >
>> > --
>> > Thanks & Regards,
>> > Venkateswara Rao Dokku,
>> > Software Engineer,One Convergence Devices Pvt Ltd.,
>> > Jubille Hills,Hyderabad.
>> >
>>
>
>
>
> --
> Thanks & Regards,
> Venkateswara Rao Dokku,
> Software Engineer,One Convergence Devices Pvt Ltd.,
> Jubille Hills,Hyderabad.
>
>
--
Thanks & Regards,
Venkateswara Rao Dokku,
Software Engineer,One Convergence Devices Pvt Ltd.,
Jubille Hills,Hyderabad.
Re: Hbase clustering
Posted by Venkateswara Rao Dokku <dv...@gmail.com>.
Yes, I can see the region server running. The output of the jps command is
given below
*Hadoop-namenode:*
* *HQuorumPeer
Main
HMaster
HRegionServer
SecondaryNameNode
Jps
NameNode
JobTracker
hadoop-datanode1:
TaskTracker
DataNode
Jps
Main
NC
HRegionServer
The complete error is given below.
hbase(main):003:0> scan 't1'
ROW COLUMN+CELL
12/09/27 17:54:42 INFO ipc.HBaseRPC: Problem connecting to server:
localhost/127.0.0.1:60020
12/09/27 17:55:44 INFO ipc.HBaseRPC: Problem connecting to server:
localhost/127.0.0.1:60020
12/09/27 17:56:46 INFO ipc.HBaseRPC: Problem connecting to server:
localhost/127.0.0.1:60020
12/09/27 17:57:48 INFO ipc.HBaseRPC: Problem connecting to server:
localhost/127.0.0.1:60020
12/09/27 17:58:52 INFO ipc.HBaseRPC: Problem connecting to server:
localhost/127.0.0.1:60020
12/09/27 17:59:55 INFO ipc.HBaseRPC: Problem connecting to server:
localhost/127.0.0.1:60020
12/09/27 18:01:00 INFO ipc.HBaseRPC: Problem connecting to server:
localhost/127.0.0.1:60020
12/09/27 18:02:01 INFO ipc.HBaseRPC: Problem connecting to server:
localhost/127.0.0.1:60020
12/09/27 18:03:03 INFO ipc.HBaseRPC: Problem connecting to server:
localhost/127.0.0.1:60020
12/09/27 18:04:05 INFO ipc.HBaseRPC: Problem connecting to server:
localhost/127.0.0.1:60020
12/09/27 18:05:07 INFO ipc.HBaseRPC: Problem connecting to server:
localhost/127.0.0.1:60020
12/09/27 18:06:10 INFO ipc.HBaseRPC: Problem connecting to server:
localhost/127.0.0.1:60020
12/09/27 18:07:13 INFO ipc.HBaseRPC: Problem connecting to server:
localhost/127.0.0.1:60020
12/09/27 18:08:19 INFO ipc.HBaseRPC: Problem connecting to server:
localhost/127.0.0.1:60020
ERROR: java.net.SocketTimeoutException: Call to
localhost/127.0.0.1:60020failed on socket timeout exception:
java.net.SocketTimeoutException: 60000
millis timeout while waiting for channel to be ready for read. ch :
java.nio.channels.SocketChannel[connected
local=/127.0.0.1:33970remote=localhost/
127.0.0.1:60020]
Here is some help for this command:
Scan a table; pass table name and optionally a dictionary of scanner
specifications. Scanner specifications may include one or more of:
TIMERANGE, FILTER, LIMIT, STARTROW, STOPROW, TIMESTAMP, MAXLENGTH,
or COLUMNS.
If no columns are specified, all columns will be scanned.
To scan all members of a column family, leave the qualifier empty as in
'col_family:'.
The filter can be specified in two ways:
1. Using a filterString - more information on this is available in the
Filter Language document attached to the HBASE-4176 JIRA
2. Using the entire package name of the filter.
Some examples:
hbase> scan '.META.'
hbase> scan '.META.', {COLUMNS => 'info:regioninfo'}
hbase> scan 't1', {COLUMNS => ['c1', 'c2'], LIMIT => 10, STARTROW =>
'xyz'}
hbase> scan 't1', {COLUMNS => 'c1', TIMERANGE => [1303668804, 1303668904]}
hbase> scan 't1', {FILTER => "(PrefixFilter ('row2') AND (QualifierFilter
(>=, 'binary:xyz'))) AND (TimestampsFilter ( 123, 456))"}
hbase> scan 't1', {FILTER =>
org.apache.hadoop.hbase.filter.ColumnPaginationFilter.new(1, 0)}
For experts, there is an additional option -- CACHE_BLOCKS -- which
switches block caching for the scanner on (true) or off (false). By
default it is enabled. Examples:
hbase> scan 't1', {COLUMNS => ['c1', 'c2'], CACHE_BLOCKS => false}
On Thu, Sep 27, 2012 at 6:11 PM, Stas Maksimov <ma...@gmail.com> wrote:
> Rao,
>
> Can you make sure your region server is actually running? You can use jps
> command to see Java processes, or a "ps ax |grep region".
>
> Thanks,
> Stas
>
> On Thu, Sep 27, 2012 at 12:25 PM, Venkateswara Rao Dokku <
> dvrao.584@gmail.com> wrote:
>
> > When I try to scan the table that is created by hadoop-namenode in the
> > hadoop-datanode, I am getting the following error
> > 12/09/27 16:47:55 INFO ipc.HBaseRPC: Problem connecting to server:
> > localhost/127.0.0.1:60020
> >
> > Could you please help me out in overcoming this problem.
> > Thanks for replying.
> >
> > On Thu, Sep 27, 2012 at 4:02 PM, Venkateswara Rao Dokku <
> > dvrao.584@gmail.com
> > > wrote:
> >
> > > I started the Hmaster on the hadoop-namenode. But I was not able to
> > access
> > > it from the hadoop-datanode. Could you please help me solving this
> > problem
> > > by sharing what are the possibilities for this to happen.
> > >
> > >
> > > On Thu, Sep 27, 2012 at 1:21 PM, n keywal <nk...@gmail.com> wrote:
> > >
> > >> You should launch the master only once, on whatever machine you like.
> > Then
> > >> you will be able to access it from any other machine.
> > >> Please have a look at the blog I mentioned in my previous mail.
> > >>
> > >> On Thu, Sep 27, 2012 at 9:39 AM, Venkateswara Rao Dokku <
> > >> dvrao.584@gmail.com
> > >> > wrote:
> > >>
> > >> > I can see that HMaster is not started on the data-node machine when
> > the
> > >> > start scripts in hadoop & hbase ran on the hadoop-namenode. My doubt
> > is
> > >> > that,Shall we have to start that master on the hadoop-datanode1 too
> or
> > >> the
> > >> > hadoop-datanode1 will access the Hmaster that is running on the
> > >> > hadoop-namenode to create,list,scan tables as the two nodes are in
> the
> > >> > cluster as namenode & datanode.
> > >> >
> > >> > On Thu, Sep 27, 2012 at 1:02 PM, n keywal <nk...@gmail.com>
> wrote:
> > >> >
> > >> > > Hi,
> > >> > >
> > >> > > I would like to direct you to the reference guide, but I must
> > >> acknowledge
> > >> > > that, well, it's a reference guide, hence not really easy for a
> > plain
> > >> new
> > >> > > start.
> > >> > > You should have a look at Lars' blog (and may be buy his book),
> and
> > >> > > especially this entry:
> > >> > >
> > http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
> > >> > >
> > >> > > Some hints however:
> > >> > > - the replication occurs at the hdfs level, not the hbase level:
> > hbase
> > >> > > writes files that are split in hdfs blocks that are replicated
> > accross
> > >> > the
> > >> > > datanodes. If you want to check the replications, you must look at
> > >> what
> > >> > > files are written by hbase and how they have been split in blocks
> by
> > >> hdfs
> > >> > > and how these blocks have been replicated. That will be in the
> hdfs
> > >> > > interface. As a side note, it's not the easiest thing to learn
> when
> > >> you
> > >> > > start :-)
> > >> > > - The error > ERROR:
> > >> org.apache.hadoop.hbase.MasterNotRunningException:
> > >> > > Retried 7 times
> > >> > > this is not linked to replication or whatever. It means that
> > second
> > >> > > machine cannot find the master. You need to fix this first.
> > (googling
> > >> &
> > >> > > checking the logs).
> > >> > >
> > >> > >
> > >> > > Good luck,
> > >> > >
> > >> > > Nicolas
> > >> > >
> > >> > >
> > >> > >
> > >> > >
> > >> > > On Thu, Sep 27, 2012 at 9:07 AM, Venkateswara Rao Dokku <
> > >> > > dvrao.584@gmail.com
> > >> > > > wrote:
> > >> > >
> > >> > > > How can we verify that the data(tables) is distributed across
> the
> > >> > > cluster??
> > >> > > > Is there a way to confirm it that the data is distributed across
> > all
> > >> > the
> > >> > > > nodes in the cluster.?
> > >> > > >
> > >> > > > On Thu, Sep 27, 2012 at 12:26 PM, Venkateswara Rao Dokku <
> > >> > > > dvrao.584@gmail.com> wrote:
> > >> > > >
> > >> > > > > Hi,
> > >> > > > > I am completely new to Hbase. I want to cluster the Hbase
> on
> > >> two
> > >> > > > > nodes.I installed hadoop,hbase on the two nodes & my conf
> files
> > >> are
> > >> > as
> > >> > > > > given below.
> > >> > > > > *cat conf/regionservers *
> > >> > > > > hbase-regionserver1
> > >> > > > > hbase-master
> > >> > > > > *cat conf/masters *
> > >> > > > > hadoop-namenode
> > >> > > > > * cat conf/slaves *
> > >> > > > > hadoop-datanode1
> > >> > > > > *vim conf/hdfs-site.xml *
> > >> > > > > <?xml version="1.0"?>
> > >> > > > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> > >> > > > >
> > >> > > > > <!-- Put site-specific property overrides in this file. -->
> > >> > > > >
> > >> > > > > <configuration>
> > >> > > > > <property>
> > >> > > > > <name>dfs.replication</name>
> > >> > > > > <value>2</value>
> > >> > > > > <description>Default block replication.The actual
> number
> > >> of
> > >> > > > > replications can be specified when the file is created. The
> > >> default
> > >> > is
> > >> > > > used
> > >> > > > > if replication is not specified in create time.
> > >> > > > > </description>
> > >> > > > > </property>
> > >> > > > > <property>
> > >> > > > > <name>dfs.support.append</name>
> > >> > > > > <value>true</value>
> > >> > > > > <description>Default block replication.The actual
> number
> > >> of
> > >> > > > > replications can be specified when the file is created. The
> > >> default
> > >> > is
> > >> > > > used
> > >> > > > > if replication is not specified in create time.
> > >> > > > > </description>
> > >> > > > > </property>
> > >> > > > > </configuration>
> > >> > > > > *& finally my /etc/hosts file is *
> > >> > > > > 127.0.0.1 localhost
> > >> > > > > 127.0.0.1 oc-PowerEdge-R610
> > >> > > > > 10.2.32.48 hbase-master hadoop-namenode
> > >> > > > > 10.240.13.35 hbase-regionserver1 hadoop-datanode1
> > >> > > > > The above files are identical on both of the machines. The
> > >> following
> > >> > > are
> > >> > > > > the processes that are running on my m/c's when I ran start
> > >> scripts
> > >> > in
> > >> > > > > hadoop as well as hbase
> > >> > > > > *hadoop-namenode:*
> > >> > > > > HQuorumPeer
> > >> > > > > HMaster
> > >> > > > > Main
> > >> > > > > HRegionServer
> > >> > > > > SecondaryNameNode
> > >> > > > > Jps
> > >> > > > > NameNode
> > >> > > > > JobTracker
> > >> > > > > *hadoop-datanode1:*
> > >> > > > >
> > >> > > > > TaskTracker
> > >> > > > > Jps
> > >> > > > > DataNode
> > >> > > > > -- process information unavailable
> > >> > > > > Main
> > >> > > > > NC
> > >> > > > > HRegionServer
> > >> > > > >
> > >> > > > > I can able to create,list & scan tables on the
> *hadoop-namenode*
> > >> > > machine
> > >> > > > > using Hbase shell. But while trying to run the same on the *
> > >> > > > > hadoop-datanode1 *machine I couldn't able to do it as I am
> > getting
> > >> > > > > following error.
> > >> > > > > hbase(main):001:0> list
> > >> > > > > TABLE
> > >> > > > >
> > >> > > > >
> > >> > > > > ERROR: org.apache.hadoop.hbase.MasterNotRunningException:
> > Retried
> > >> 7
> > >> > > times
> > >> > > > >
> > >> > > > > Here is some help for this command:
> > >> > > > > List all tables in hbase. Optional regular expression
> parameter
> > >> could
> > >> > > > > be used to filter the output. Examples:
> > >> > > > >
> > >> > > > > hbase> list
> > >> > > > > hbase> list 'abc.*'
> > >> > > > > How can I list,scan the tables that are created by the
> > >> > > *hadoop-namenode *
> > >> > > > > from the *hadoop-datanode1* machine. Similarly Can I create
> some
> > >> > tables
> > >> > > > > on *hadoop-datanode1 *& can I access them from the
> > >> *hadoop-namenode
> > >> > *
> > >> > > &
> > >> > > > > vice-versa as the data is distributed as this is a cluster.
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > --
> > >> > > > > Thanks & Regards,
> > >> > > > > Venkateswara Rao Dokku,
> > >> > > > > Software Engineer,One Convergence Devices Pvt Ltd.,
> > >> > > > > Jubille Hills,Hyderabad.
> > >> > > > >
> > >> > > > >
> > >> > > >
> > >> > > >
> > >> > > > --
> > >> > > > Thanks & Regards,
> > >> > > > Venkateswara Rao Dokku,
> > >> > > > Software Engineer,One Convergence Devices Pvt Ltd.,
> > >> > > > Jubille Hills,Hyderabad.
> > >> > > >
> > >> > >
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Thanks & Regards,
> > >> > Venkateswara Rao Dokku,
> > >> > Software Engineer,One Convergence Devices Pvt Ltd.,
> > >> > Jubille Hills,Hyderabad.
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > Thanks & Regards,
> > > Venkateswara Rao Dokku,
> > > Software Engineer,One Convergence Devices Pvt Ltd.,
> > > Jubille Hills,Hyderabad.
> > >
> > >
> >
> >
> > --
> > Thanks & Regards,
> > Venkateswara Rao Dokku,
> > Software Engineer,One Convergence Devices Pvt Ltd.,
> > Jubille Hills,Hyderabad.
> >
>
--
Thanks & Regards,
Venkateswara Rao Dokku,
Software Engineer,One Convergence Devices Pvt Ltd.,
Jubille Hills,Hyderabad.
Re: Hbase clustering
Posted by Stas Maksimov <ma...@gmail.com>.
Rao,
Can you make sure your region server is actually running? You can use jps
command to see Java processes, or a "ps ax |grep region".
Thanks,
Stas
On Thu, Sep 27, 2012 at 12:25 PM, Venkateswara Rao Dokku <
dvrao.584@gmail.com> wrote:
> When I try to scan the table that is created by hadoop-namenode in the
> hadoop-datanode, I am getting the following error
> 12/09/27 16:47:55 INFO ipc.HBaseRPC: Problem connecting to server:
> localhost/127.0.0.1:60020
>
> Could you please help me out in overcoming this problem.
> Thanks for replying.
>
> On Thu, Sep 27, 2012 at 4:02 PM, Venkateswara Rao Dokku <
> dvrao.584@gmail.com
> > wrote:
>
> > I started the Hmaster on the hadoop-namenode. But I was not able to
> access
> > it from the hadoop-datanode. Could you please help me solving this
> problem
> > by sharing what are the possibilities for this to happen.
> >
> >
> > On Thu, Sep 27, 2012 at 1:21 PM, n keywal <nk...@gmail.com> wrote:
> >
> >> You should launch the master only once, on whatever machine you like.
> Then
> >> you will be able to access it from any other machine.
> >> Please have a look at the blog I mentioned in my previous mail.
> >>
> >> On Thu, Sep 27, 2012 at 9:39 AM, Venkateswara Rao Dokku <
> >> dvrao.584@gmail.com
> >> > wrote:
> >>
> >> > I can see that HMaster is not started on the data-node machine when
> the
> >> > start scripts in hadoop & hbase ran on the hadoop-namenode. My doubt
> is
> >> > that,Shall we have to start that master on the hadoop-datanode1 too or
> >> the
> >> > hadoop-datanode1 will access the Hmaster that is running on the
> >> > hadoop-namenode to create,list,scan tables as the two nodes are in the
> >> > cluster as namenode & datanode.
> >> >
> >> > On Thu, Sep 27, 2012 at 1:02 PM, n keywal <nk...@gmail.com> wrote:
> >> >
> >> > > Hi,
> >> > >
> >> > > I would like to direct you to the reference guide, but I must
> >> acknowledge
> >> > > that, well, it's a reference guide, hence not really easy for a
> plain
> >> new
> >> > > start.
> >> > > You should have a look at Lars' blog (and may be buy his book), and
> >> > > especially this entry:
> >> > >
> http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
> >> > >
> >> > > Some hints however:
> >> > > - the replication occurs at the hdfs level, not the hbase level:
> hbase
> >> > > writes files that are split in hdfs blocks that are replicated
> accross
> >> > the
> >> > > datanodes. If you want to check the replications, you must look at
> >> what
> >> > > files are written by hbase and how they have been split in blocks by
> >> hdfs
> >> > > and how these blocks have been replicated. That will be in the hdfs
> >> > > interface. As a side note, it's not the easiest thing to learn when
> >> you
> >> > > start :-)
> >> > > - The error > ERROR:
> >> org.apache.hadoop.hbase.MasterNotRunningException:
> >> > > Retried 7 times
> >> > > this is not linked to replication or whatever. It means that
> second
> >> > > machine cannot find the master. You need to fix this first.
> (googling
> >> &
> >> > > checking the logs).
> >> > >
> >> > >
> >> > > Good luck,
> >> > >
> >> > > Nicolas
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > On Thu, Sep 27, 2012 at 9:07 AM, Venkateswara Rao Dokku <
> >> > > dvrao.584@gmail.com
> >> > > > wrote:
> >> > >
> >> > > > How can we verify that the data(tables) is distributed across the
> >> > > cluster??
> >> > > > Is there a way to confirm it that the data is distributed across
> all
> >> > the
> >> > > > nodes in the cluster.?
> >> > > >
> >> > > > On Thu, Sep 27, 2012 at 12:26 PM, Venkateswara Rao Dokku <
> >> > > > dvrao.584@gmail.com> wrote:
> >> > > >
> >> > > > > Hi,
> >> > > > > I am completely new to Hbase. I want to cluster the Hbase on
> >> two
> >> > > > > nodes.I installed hadoop,hbase on the two nodes & my conf files
> >> are
> >> > as
> >> > > > > given below.
> >> > > > > *cat conf/regionservers *
> >> > > > > hbase-regionserver1
> >> > > > > hbase-master
> >> > > > > *cat conf/masters *
> >> > > > > hadoop-namenode
> >> > > > > * cat conf/slaves *
> >> > > > > hadoop-datanode1
> >> > > > > *vim conf/hdfs-site.xml *
> >> > > > > <?xml version="1.0"?>
> >> > > > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> >> > > > >
> >> > > > > <!-- Put site-specific property overrides in this file. -->
> >> > > > >
> >> > > > > <configuration>
> >> > > > > <property>
> >> > > > > <name>dfs.replication</name>
> >> > > > > <value>2</value>
> >> > > > > <description>Default block replication.The actual number
> >> of
> >> > > > > replications can be specified when the file is created. The
> >> default
> >> > is
> >> > > > used
> >> > > > > if replication is not specified in create time.
> >> > > > > </description>
> >> > > > > </property>
> >> > > > > <property>
> >> > > > > <name>dfs.support.append</name>
> >> > > > > <value>true</value>
> >> > > > > <description>Default block replication.The actual number
> >> of
> >> > > > > replications can be specified when the file is created. The
> >> default
> >> > is
> >> > > > used
> >> > > > > if replication is not specified in create time.
> >> > > > > </description>
> >> > > > > </property>
> >> > > > > </configuration>
> >> > > > > *& finally my /etc/hosts file is *
> >> > > > > 127.0.0.1 localhost
> >> > > > > 127.0.0.1 oc-PowerEdge-R610
> >> > > > > 10.2.32.48 hbase-master hadoop-namenode
> >> > > > > 10.240.13.35 hbase-regionserver1 hadoop-datanode1
> >> > > > > The above files are identical on both of the machines. The
> >> following
> >> > > are
> >> > > > > the processes that are running on my m/c's when I ran start
> >> scripts
> >> > in
> >> > > > > hadoop as well as hbase
> >> > > > > *hadoop-namenode:*
> >> > > > > HQuorumPeer
> >> > > > > HMaster
> >> > > > > Main
> >> > > > > HRegionServer
> >> > > > > SecondaryNameNode
> >> > > > > Jps
> >> > > > > NameNode
> >> > > > > JobTracker
> >> > > > > *hadoop-datanode1:*
> >> > > > >
> >> > > > > TaskTracker
> >> > > > > Jps
> >> > > > > DataNode
> >> > > > > -- process information unavailable
> >> > > > > Main
> >> > > > > NC
> >> > > > > HRegionServer
> >> > > > >
> >> > > > > I can able to create,list & scan tables on the *hadoop-namenode*
> >> > > machine
> >> > > > > using Hbase shell. But while trying to run the same on the *
> >> > > > > hadoop-datanode1 *machine I couldn't able to do it as I am
> getting
> >> > > > > following error.
> >> > > > > hbase(main):001:0> list
> >> > > > > TABLE
> >> > > > >
> >> > > > >
> >> > > > > ERROR: org.apache.hadoop.hbase.MasterNotRunningException:
> Retried
> >> 7
> >> > > times
> >> > > > >
> >> > > > > Here is some help for this command:
> >> > > > > List all tables in hbase. Optional regular expression parameter
> >> could
> >> > > > > be used to filter the output. Examples:
> >> > > > >
> >> > > > > hbase> list
> >> > > > > hbase> list 'abc.*'
> >> > > > > How can I list,scan the tables that are created by the
> >> > > *hadoop-namenode *
> >> > > > > from the *hadoop-datanode1* machine. Similarly Can I create some
> >> > tables
> >> > > > > on *hadoop-datanode1 *& can I access them from the
> >> *hadoop-namenode
> >> > *
> >> > > &
> >> > > > > vice-versa as the data is distributed as this is a cluster.
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > Thanks & Regards,
> >> > > > > Venkateswara Rao Dokku,
> >> > > > > Software Engineer,One Convergence Devices Pvt Ltd.,
> >> > > > > Jubille Hills,Hyderabad.
> >> > > > >
> >> > > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > Thanks & Regards,
> >> > > > Venkateswara Rao Dokku,
> >> > > > Software Engineer,One Convergence Devices Pvt Ltd.,
> >> > > > Jubille Hills,Hyderabad.
> >> > > >
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > Thanks & Regards,
> >> > Venkateswara Rao Dokku,
> >> > Software Engineer,One Convergence Devices Pvt Ltd.,
> >> > Jubille Hills,Hyderabad.
> >> >
> >>
> >
> >
> >
> > --
> > Thanks & Regards,
> > Venkateswara Rao Dokku,
> > Software Engineer,One Convergence Devices Pvt Ltd.,
> > Jubille Hills,Hyderabad.
> >
> >
>
>
> --
> Thanks & Regards,
> Venkateswara Rao Dokku,
> Software Engineer,One Convergence Devices Pvt Ltd.,
> Jubille Hills,Hyderabad.
>
Re: Hbase clustering
Posted by Venkateswara Rao Dokku <dv...@gmail.com>.
When I try to scan the table that is created by hadoop-namenode in the
hadoop-datanode, I am getting the following error
12/09/27 16:47:55 INFO ipc.HBaseRPC: Problem connecting to server:
localhost/127.0.0.1:60020
Could you please help me out in overcoming this problem.
Thanks for replying.
On Thu, Sep 27, 2012 at 4:02 PM, Venkateswara Rao Dokku <dvrao.584@gmail.com
> wrote:
> I started the Hmaster on the hadoop-namenode. But I was not able to access
> it from the hadoop-datanode. Could you please help me solving this problem
> by sharing what are the possibilities for this to happen.
>
>
> On Thu, Sep 27, 2012 at 1:21 PM, n keywal <nk...@gmail.com> wrote:
>
>> You should launch the master only once, on whatever machine you like. Then
>> you will be able to access it from any other machine.
>> Please have a look at the blog I mentioned in my previous mail.
>>
>> On Thu, Sep 27, 2012 at 9:39 AM, Venkateswara Rao Dokku <
>> dvrao.584@gmail.com
>> > wrote:
>>
>> > I can see that HMaster is not started on the data-node machine when the
>> > start scripts in hadoop & hbase ran on the hadoop-namenode. My doubt is
>> > that,Shall we have to start that master on the hadoop-datanode1 too or
>> the
>> > hadoop-datanode1 will access the Hmaster that is running on the
>> > hadoop-namenode to create,list,scan tables as the two nodes are in the
>> > cluster as namenode & datanode.
>> >
>> > On Thu, Sep 27, 2012 at 1:02 PM, n keywal <nk...@gmail.com> wrote:
>> >
>> > > Hi,
>> > >
>> > > I would like to direct you to the reference guide, but I must
>> acknowledge
>> > > that, well, it's a reference guide, hence not really easy for a plain
>> new
>> > > start.
>> > > You should have a look at Lars' blog (and may be buy his book), and
>> > > especially this entry:
>> > > http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
>> > >
>> > > Some hints however:
>> > > - the replication occurs at the hdfs level, not the hbase level: hbase
>> > > writes files that are split in hdfs blocks that are replicated accross
>> > the
>> > > datanodes. If you want to check the replications, you must look at
>> what
>> > > files are written by hbase and how they have been split in blocks by
>> hdfs
>> > > and how these blocks have been replicated. That will be in the hdfs
>> > > interface. As a side note, it's not the easiest thing to learn when
>> you
>> > > start :-)
>> > > - The error > ERROR:
>> org.apache.hadoop.hbase.MasterNotRunningException:
>> > > Retried 7 times
>> > > this is not linked to replication or whatever. It means that second
>> > > machine cannot find the master. You need to fix this first. (googling
>> &
>> > > checking the logs).
>> > >
>> > >
>> > > Good luck,
>> > >
>> > > Nicolas
>> > >
>> > >
>> > >
>> > >
>> > > On Thu, Sep 27, 2012 at 9:07 AM, Venkateswara Rao Dokku <
>> > > dvrao.584@gmail.com
>> > > > wrote:
>> > >
>> > > > How can we verify that the data(tables) is distributed across the
>> > > cluster??
>> > > > Is there a way to confirm it that the data is distributed across all
>> > the
>> > > > nodes in the cluster.?
>> > > >
>> > > > On Thu, Sep 27, 2012 at 12:26 PM, Venkateswara Rao Dokku <
>> > > > dvrao.584@gmail.com> wrote:
>> > > >
>> > > > > Hi,
>> > > > > I am completely new to Hbase. I want to cluster the Hbase on
>> two
>> > > > > nodes.I installed hadoop,hbase on the two nodes & my conf files
>> are
>> > as
>> > > > > given below.
>> > > > > *cat conf/regionservers *
>> > > > > hbase-regionserver1
>> > > > > hbase-master
>> > > > > *cat conf/masters *
>> > > > > hadoop-namenode
>> > > > > * cat conf/slaves *
>> > > > > hadoop-datanode1
>> > > > > *vim conf/hdfs-site.xml *
>> > > > > <?xml version="1.0"?>
>> > > > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>> > > > >
>> > > > > <!-- Put site-specific property overrides in this file. -->
>> > > > >
>> > > > > <configuration>
>> > > > > <property>
>> > > > > <name>dfs.replication</name>
>> > > > > <value>2</value>
>> > > > > <description>Default block replication.The actual number
>> of
>> > > > > replications can be specified when the file is created. The
>> default
>> > is
>> > > > used
>> > > > > if replication is not specified in create time.
>> > > > > </description>
>> > > > > </property>
>> > > > > <property>
>> > > > > <name>dfs.support.append</name>
>> > > > > <value>true</value>
>> > > > > <description>Default block replication.The actual number
>> of
>> > > > > replications can be specified when the file is created. The
>> default
>> > is
>> > > > used
>> > > > > if replication is not specified in create time.
>> > > > > </description>
>> > > > > </property>
>> > > > > </configuration>
>> > > > > *& finally my /etc/hosts file is *
>> > > > > 127.0.0.1 localhost
>> > > > > 127.0.0.1 oc-PowerEdge-R610
>> > > > > 10.2.32.48 hbase-master hadoop-namenode
>> > > > > 10.240.13.35 hbase-regionserver1 hadoop-datanode1
>> > > > > The above files are identical on both of the machines. The
>> following
>> > > are
>> > > > > the processes that are running on my m/c's when I ran start
>> scripts
>> > in
>> > > > > hadoop as well as hbase
>> > > > > *hadoop-namenode:*
>> > > > > HQuorumPeer
>> > > > > HMaster
>> > > > > Main
>> > > > > HRegionServer
>> > > > > SecondaryNameNode
>> > > > > Jps
>> > > > > NameNode
>> > > > > JobTracker
>> > > > > *hadoop-datanode1:*
>> > > > >
>> > > > > TaskTracker
>> > > > > Jps
>> > > > > DataNode
>> > > > > -- process information unavailable
>> > > > > Main
>> > > > > NC
>> > > > > HRegionServer
>> > > > >
>> > > > > I can able to create,list & scan tables on the *hadoop-namenode*
>> > > machine
>> > > > > using Hbase shell. But while trying to run the same on the *
>> > > > > hadoop-datanode1 *machine I couldn't able to do it as I am getting
>> > > > > following error.
>> > > > > hbase(main):001:0> list
>> > > > > TABLE
>> > > > >
>> > > > >
>> > > > > ERROR: org.apache.hadoop.hbase.MasterNotRunningException: Retried
>> 7
>> > > times
>> > > > >
>> > > > > Here is some help for this command:
>> > > > > List all tables in hbase. Optional regular expression parameter
>> could
>> > > > > be used to filter the output. Examples:
>> > > > >
>> > > > > hbase> list
>> > > > > hbase> list 'abc.*'
>> > > > > How can I list,scan the tables that are created by the
>> > > *hadoop-namenode *
>> > > > > from the *hadoop-datanode1* machine. Similarly Can I create some
>> > tables
>> > > > > on *hadoop-datanode1 *& can I access them from the
>> *hadoop-namenode
>> > *
>> > > &
>> > > > > vice-versa as the data is distributed as this is a cluster.
>> > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Thanks & Regards,
>> > > > > Venkateswara Rao Dokku,
>> > > > > Software Engineer,One Convergence Devices Pvt Ltd.,
>> > > > > Jubille Hills,Hyderabad.
>> > > > >
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > Thanks & Regards,
>> > > > Venkateswara Rao Dokku,
>> > > > Software Engineer,One Convergence Devices Pvt Ltd.,
>> > > > Jubille Hills,Hyderabad.
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Thanks & Regards,
>> > Venkateswara Rao Dokku,
>> > Software Engineer,One Convergence Devices Pvt Ltd.,
>> > Jubille Hills,Hyderabad.
>> >
>>
>
>
>
> --
> Thanks & Regards,
> Venkateswara Rao Dokku,
> Software Engineer,One Convergence Devices Pvt Ltd.,
> Jubille Hills,Hyderabad.
>
>
--
Thanks & Regards,
Venkateswara Rao Dokku,
Software Engineer,One Convergence Devices Pvt Ltd.,
Jubille Hills,Hyderabad.
Re: Hbase clustering
Posted by Venkateswara Rao Dokku <dv...@gmail.com>.
I started the Hmaster on the hadoop-namenode. But I was not able to access
it from the hadoop-datanode. Could you please help me solving this problem
by sharing what are the possibilities for this to happen.
On Thu, Sep 27, 2012 at 1:21 PM, n keywal <nk...@gmail.com> wrote:
> You should launch the master only once, on whatever machine you like. Then
> you will be able to access it from any other machine.
> Please have a look at the blog I mentioned in my previous mail.
>
> On Thu, Sep 27, 2012 at 9:39 AM, Venkateswara Rao Dokku <
> dvrao.584@gmail.com
> > wrote:
>
> > I can see that HMaster is not started on the data-node machine when the
> > start scripts in hadoop & hbase ran on the hadoop-namenode. My doubt is
> > that,Shall we have to start that master on the hadoop-datanode1 too or
> the
> > hadoop-datanode1 will access the Hmaster that is running on the
> > hadoop-namenode to create,list,scan tables as the two nodes are in the
> > cluster as namenode & datanode.
> >
> > On Thu, Sep 27, 2012 at 1:02 PM, n keywal <nk...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I would like to direct you to the reference guide, but I must
> acknowledge
> > > that, well, it's a reference guide, hence not really easy for a plain
> new
> > > start.
> > > You should have a look at Lars' blog (and may be buy his book), and
> > > especially this entry:
> > > http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
> > >
> > > Some hints however:
> > > - the replication occurs at the hdfs level, not the hbase level: hbase
> > > writes files that are split in hdfs blocks that are replicated accross
> > the
> > > datanodes. If you want to check the replications, you must look at what
> > > files are written by hbase and how they have been split in blocks by
> hdfs
> > > and how these blocks have been replicated. That will be in the hdfs
> > > interface. As a side note, it's not the easiest thing to learn when you
> > > start :-)
> > > - The error > ERROR: org.apache.hadoop.hbase.MasterNotRunningException:
> > > Retried 7 times
> > > this is not linked to replication or whatever. It means that second
> > > machine cannot find the master. You need to fix this first. (googling &
> > > checking the logs).
> > >
> > >
> > > Good luck,
> > >
> > > Nicolas
> > >
> > >
> > >
> > >
> > > On Thu, Sep 27, 2012 at 9:07 AM, Venkateswara Rao Dokku <
> > > dvrao.584@gmail.com
> > > > wrote:
> > >
> > > > How can we verify that the data(tables) is distributed across the
> > > cluster??
> > > > Is there a way to confirm it that the data is distributed across all
> > the
> > > > nodes in the cluster.?
> > > >
> > > > On Thu, Sep 27, 2012 at 12:26 PM, Venkateswara Rao Dokku <
> > > > dvrao.584@gmail.com> wrote:
> > > >
> > > > > Hi,
> > > > > I am completely new to Hbase. I want to cluster the Hbase on
> two
> > > > > nodes.I installed hadoop,hbase on the two nodes & my conf files are
> > as
> > > > > given below.
> > > > > *cat conf/regionservers *
> > > > > hbase-regionserver1
> > > > > hbase-master
> > > > > *cat conf/masters *
> > > > > hadoop-namenode
> > > > > * cat conf/slaves *
> > > > > hadoop-datanode1
> > > > > *vim conf/hdfs-site.xml *
> > > > > <?xml version="1.0"?>
> > > > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> > > > >
> > > > > <!-- Put site-specific property overrides in this file. -->
> > > > >
> > > > > <configuration>
> > > > > <property>
> > > > > <name>dfs.replication</name>
> > > > > <value>2</value>
> > > > > <description>Default block replication.The actual number of
> > > > > replications can be specified when the file is created. The default
> > is
> > > > used
> > > > > if replication is not specified in create time.
> > > > > </description>
> > > > > </property>
> > > > > <property>
> > > > > <name>dfs.support.append</name>
> > > > > <value>true</value>
> > > > > <description>Default block replication.The actual number of
> > > > > replications can be specified when the file is created. The default
> > is
> > > > used
> > > > > if replication is not specified in create time.
> > > > > </description>
> > > > > </property>
> > > > > </configuration>
> > > > > *& finally my /etc/hosts file is *
> > > > > 127.0.0.1 localhost
> > > > > 127.0.0.1 oc-PowerEdge-R610
> > > > > 10.2.32.48 hbase-master hadoop-namenode
> > > > > 10.240.13.35 hbase-regionserver1 hadoop-datanode1
> > > > > The above files are identical on both of the machines. The
> following
> > > are
> > > > > the processes that are running on my m/c's when I ran start scripts
> > in
> > > > > hadoop as well as hbase
> > > > > *hadoop-namenode:*
> > > > > HQuorumPeer
> > > > > HMaster
> > > > > Main
> > > > > HRegionServer
> > > > > SecondaryNameNode
> > > > > Jps
> > > > > NameNode
> > > > > JobTracker
> > > > > *hadoop-datanode1:*
> > > > >
> > > > > TaskTracker
> > > > > Jps
> > > > > DataNode
> > > > > -- process information unavailable
> > > > > Main
> > > > > NC
> > > > > HRegionServer
> > > > >
> > > > > I can able to create,list & scan tables on the *hadoop-namenode*
> > > machine
> > > > > using Hbase shell. But while trying to run the same on the *
> > > > > hadoop-datanode1 *machine I couldn't able to do it as I am getting
> > > > > following error.
> > > > > hbase(main):001:0> list
> > > > > TABLE
> > > > >
> > > > >
> > > > > ERROR: org.apache.hadoop.hbase.MasterNotRunningException: Retried 7
> > > times
> > > > >
> > > > > Here is some help for this command:
> > > > > List all tables in hbase. Optional regular expression parameter
> could
> > > > > be used to filter the output. Examples:
> > > > >
> > > > > hbase> list
> > > > > hbase> list 'abc.*'
> > > > > How can I list,scan the tables that are created by the
> > > *hadoop-namenode *
> > > > > from the *hadoop-datanode1* machine. Similarly Can I create some
> > tables
> > > > > on *hadoop-datanode1 *& can I access them from the
> *hadoop-namenode
> > *
> > > &
> > > > > vice-versa as the data is distributed as this is a cluster.
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Thanks & Regards,
> > > > > Venkateswara Rao Dokku,
> > > > > Software Engineer,One Convergence Devices Pvt Ltd.,
> > > > > Jubille Hills,Hyderabad.
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Thanks & Regards,
> > > > Venkateswara Rao Dokku,
> > > > Software Engineer,One Convergence Devices Pvt Ltd.,
> > > > Jubille Hills,Hyderabad.
> > > >
> > >
> >
> >
> >
> > --
> > Thanks & Regards,
> > Venkateswara Rao Dokku,
> > Software Engineer,One Convergence Devices Pvt Ltd.,
> > Jubille Hills,Hyderabad.
> >
>
--
Thanks & Regards,
Venkateswara Rao Dokku,
Software Engineer,One Convergence Devices Pvt Ltd.,
Jubille Hills,Hyderabad.
Re: Hbase clustering
Posted by n keywal <nk...@gmail.com>.
You should launch the master only once, on whatever machine you like. Then
you will be able to access it from any other machine.
Please have a look at the blog I mentioned in my previous mail.
On Thu, Sep 27, 2012 at 9:39 AM, Venkateswara Rao Dokku <dvrao.584@gmail.com
> wrote:
> I can see that HMaster is not started on the data-node machine when the
> start scripts in hadoop & hbase ran on the hadoop-namenode. My doubt is
> that,Shall we have to start that master on the hadoop-datanode1 too or the
> hadoop-datanode1 will access the Hmaster that is running on the
> hadoop-namenode to create,list,scan tables as the two nodes are in the
> cluster as namenode & datanode.
>
> On Thu, Sep 27, 2012 at 1:02 PM, n keywal <nk...@gmail.com> wrote:
>
> > Hi,
> >
> > I would like to direct you to the reference guide, but I must acknowledge
> > that, well, it's a reference guide, hence not really easy for a plain new
> > start.
> > You should have a look at Lars' blog (and may be buy his book), and
> > especially this entry:
> > http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
> >
> > Some hints however:
> > - the replication occurs at the hdfs level, not the hbase level: hbase
> > writes files that are split in hdfs blocks that are replicated accross
> the
> > datanodes. If you want to check the replications, you must look at what
> > files are written by hbase and how they have been split in blocks by hdfs
> > and how these blocks have been replicated. That will be in the hdfs
> > interface. As a side note, it's not the easiest thing to learn when you
> > start :-)
> > - The error > ERROR: org.apache.hadoop.hbase.MasterNotRunningException:
> > Retried 7 times
> > this is not linked to replication or whatever. It means that second
> > machine cannot find the master. You need to fix this first. (googling &
> > checking the logs).
> >
> >
> > Good luck,
> >
> > Nicolas
> >
> >
> >
> >
> > On Thu, Sep 27, 2012 at 9:07 AM, Venkateswara Rao Dokku <
> > dvrao.584@gmail.com
> > > wrote:
> >
> > > How can we verify that the data(tables) is distributed across the
> > cluster??
> > > Is there a way to confirm it that the data is distributed across all
> the
> > > nodes in the cluster.?
> > >
> > > On Thu, Sep 27, 2012 at 12:26 PM, Venkateswara Rao Dokku <
> > > dvrao.584@gmail.com> wrote:
> > >
> > > > Hi,
> > > > I am completely new to Hbase. I want to cluster the Hbase on two
> > > > nodes.I installed hadoop,hbase on the two nodes & my conf files are
> as
> > > > given below.
> > > > *cat conf/regionservers *
> > > > hbase-regionserver1
> > > > hbase-master
> > > > *cat conf/masters *
> > > > hadoop-namenode
> > > > * cat conf/slaves *
> > > > hadoop-datanode1
> > > > *vim conf/hdfs-site.xml *
> > > > <?xml version="1.0"?>
> > > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> > > >
> > > > <!-- Put site-specific property overrides in this file. -->
> > > >
> > > > <configuration>
> > > > <property>
> > > > <name>dfs.replication</name>
> > > > <value>2</value>
> > > > <description>Default block replication.The actual number of
> > > > replications can be specified when the file is created. The default
> is
> > > used
> > > > if replication is not specified in create time.
> > > > </description>
> > > > </property>
> > > > <property>
> > > > <name>dfs.support.append</name>
> > > > <value>true</value>
> > > > <description>Default block replication.The actual number of
> > > > replications can be specified when the file is created. The default
> is
> > > used
> > > > if replication is not specified in create time.
> > > > </description>
> > > > </property>
> > > > </configuration>
> > > > *& finally my /etc/hosts file is *
> > > > 127.0.0.1 localhost
> > > > 127.0.0.1 oc-PowerEdge-R610
> > > > 10.2.32.48 hbase-master hadoop-namenode
> > > > 10.240.13.35 hbase-regionserver1 hadoop-datanode1
> > > > The above files are identical on both of the machines. The following
> > are
> > > > the processes that are running on my m/c's when I ran start scripts
> in
> > > > hadoop as well as hbase
> > > > *hadoop-namenode:*
> > > > HQuorumPeer
> > > > HMaster
> > > > Main
> > > > HRegionServer
> > > > SecondaryNameNode
> > > > Jps
> > > > NameNode
> > > > JobTracker
> > > > *hadoop-datanode1:*
> > > >
> > > > TaskTracker
> > > > Jps
> > > > DataNode
> > > > -- process information unavailable
> > > > Main
> > > > NC
> > > > HRegionServer
> > > >
> > > > I can able to create,list & scan tables on the *hadoop-namenode*
> > machine
> > > > using Hbase shell. But while trying to run the same on the *
> > > > hadoop-datanode1 *machine I couldn't able to do it as I am getting
> > > > following error.
> > > > hbase(main):001:0> list
> > > > TABLE
> > > >
> > > >
> > > > ERROR: org.apache.hadoop.hbase.MasterNotRunningException: Retried 7
> > times
> > > >
> > > > Here is some help for this command:
> > > > List all tables in hbase. Optional regular expression parameter could
> > > > be used to filter the output. Examples:
> > > >
> > > > hbase> list
> > > > hbase> list 'abc.*'
> > > > How can I list,scan the tables that are created by the
> > *hadoop-namenode *
> > > > from the *hadoop-datanode1* machine. Similarly Can I create some
> tables
> > > > on *hadoop-datanode1 *& can I access them from the *hadoop-namenode
> *
> > &
> > > > vice-versa as the data is distributed as this is a cluster.
> > > >
> > > >
> > > >
> > > > --
> > > > Thanks & Regards,
> > > > Venkateswara Rao Dokku,
> > > > Software Engineer,One Convergence Devices Pvt Ltd.,
> > > > Jubille Hills,Hyderabad.
> > > >
> > > >
> > >
> > >
> > > --
> > > Thanks & Regards,
> > > Venkateswara Rao Dokku,
> > > Software Engineer,One Convergence Devices Pvt Ltd.,
> > > Jubille Hills,Hyderabad.
> > >
> >
>
>
>
> --
> Thanks & Regards,
> Venkateswara Rao Dokku,
> Software Engineer,One Convergence Devices Pvt Ltd.,
> Jubille Hills,Hyderabad.
>
Re: Hbase clustering
Posted by Venkateswara Rao Dokku <dv...@gmail.com>.
I can see that HMaster is not started on the data-node machine when the
start scripts in hadoop & hbase ran on the hadoop-namenode. My doubt is
that,Shall we have to start that master on the hadoop-datanode1 too or the
hadoop-datanode1 will access the Hmaster that is running on the
hadoop-namenode to create,list,scan tables as the two nodes are in the
cluster as namenode & datanode.
On Thu, Sep 27, 2012 at 1:02 PM, n keywal <nk...@gmail.com> wrote:
> Hi,
>
> I would like to direct you to the reference guide, but I must acknowledge
> that, well, it's a reference guide, hence not really easy for a plain new
> start.
> You should have a look at Lars' blog (and may be buy his book), and
> especially this entry:
> http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
>
> Some hints however:
> - the replication occurs at the hdfs level, not the hbase level: hbase
> writes files that are split in hdfs blocks that are replicated accross the
> datanodes. If you want to check the replications, you must look at what
> files are written by hbase and how they have been split in blocks by hdfs
> and how these blocks have been replicated. That will be in the hdfs
> interface. As a side note, it's not the easiest thing to learn when you
> start :-)
> - The error > ERROR: org.apache.hadoop.hbase.MasterNotRunningException:
> Retried 7 times
> this is not linked to replication or whatever. It means that second
> machine cannot find the master. You need to fix this first. (googling &
> checking the logs).
>
>
> Good luck,
>
> Nicolas
>
>
>
>
> On Thu, Sep 27, 2012 at 9:07 AM, Venkateswara Rao Dokku <
> dvrao.584@gmail.com
> > wrote:
>
> > How can we verify that the data(tables) is distributed across the
> cluster??
> > Is there a way to confirm it that the data is distributed across all the
> > nodes in the cluster.?
> >
> > On Thu, Sep 27, 2012 at 12:26 PM, Venkateswara Rao Dokku <
> > dvrao.584@gmail.com> wrote:
> >
> > > Hi,
> > > I am completely new to Hbase. I want to cluster the Hbase on two
> > > nodes.I installed hadoop,hbase on the two nodes & my conf files are as
> > > given below.
> > > *cat conf/regionservers *
> > > hbase-regionserver1
> > > hbase-master
> > > *cat conf/masters *
> > > hadoop-namenode
> > > * cat conf/slaves *
> > > hadoop-datanode1
> > > *vim conf/hdfs-site.xml *
> > > <?xml version="1.0"?>
> > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> > >
> > > <!-- Put site-specific property overrides in this file. -->
> > >
> > > <configuration>
> > > <property>
> > > <name>dfs.replication</name>
> > > <value>2</value>
> > > <description>Default block replication.The actual number of
> > > replications can be specified when the file is created. The default is
> > used
> > > if replication is not specified in create time.
> > > </description>
> > > </property>
> > > <property>
> > > <name>dfs.support.append</name>
> > > <value>true</value>
> > > <description>Default block replication.The actual number of
> > > replications can be specified when the file is created. The default is
> > used
> > > if replication is not specified in create time.
> > > </description>
> > > </property>
> > > </configuration>
> > > *& finally my /etc/hosts file is *
> > > 127.0.0.1 localhost
> > > 127.0.0.1 oc-PowerEdge-R610
> > > 10.2.32.48 hbase-master hadoop-namenode
> > > 10.240.13.35 hbase-regionserver1 hadoop-datanode1
> > > The above files are identical on both of the machines. The following
> are
> > > the processes that are running on my m/c's when I ran start scripts in
> > > hadoop as well as hbase
> > > *hadoop-namenode:*
> > > HQuorumPeer
> > > HMaster
> > > Main
> > > HRegionServer
> > > SecondaryNameNode
> > > Jps
> > > NameNode
> > > JobTracker
> > > *hadoop-datanode1:*
> > >
> > > TaskTracker
> > > Jps
> > > DataNode
> > > -- process information unavailable
> > > Main
> > > NC
> > > HRegionServer
> > >
> > > I can able to create,list & scan tables on the *hadoop-namenode*
> machine
> > > using Hbase shell. But while trying to run the same on the *
> > > hadoop-datanode1 *machine I couldn't able to do it as I am getting
> > > following error.
> > > hbase(main):001:0> list
> > > TABLE
> > >
> > >
> > > ERROR: org.apache.hadoop.hbase.MasterNotRunningException: Retried 7
> times
> > >
> > > Here is some help for this command:
> > > List all tables in hbase. Optional regular expression parameter could
> > > be used to filter the output. Examples:
> > >
> > > hbase> list
> > > hbase> list 'abc.*'
> > > How can I list,scan the tables that are created by the
> *hadoop-namenode *
> > > from the *hadoop-datanode1* machine. Similarly Can I create some tables
> > > on *hadoop-datanode1 *& can I access them from the *hadoop-namenode *
> &
> > > vice-versa as the data is distributed as this is a cluster.
> > >
> > >
> > >
> > > --
> > > Thanks & Regards,
> > > Venkateswara Rao Dokku,
> > > Software Engineer,One Convergence Devices Pvt Ltd.,
> > > Jubille Hills,Hyderabad.
> > >
> > >
> >
> >
> > --
> > Thanks & Regards,
> > Venkateswara Rao Dokku,
> > Software Engineer,One Convergence Devices Pvt Ltd.,
> > Jubille Hills,Hyderabad.
> >
>
--
Thanks & Regards,
Venkateswara Rao Dokku,
Software Engineer,One Convergence Devices Pvt Ltd.,
Jubille Hills,Hyderabad.
Re: Hbase clustering
Posted by n keywal <nk...@gmail.com>.
Hi,
I would like to direct you to the reference guide, but I must acknowledge
that, well, it's a reference guide, hence not really easy for a plain new
start.
You should have a look at Lars' blog (and may be buy his book), and
especially this entry:
http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
Some hints however:
- the replication occurs at the hdfs level, not the hbase level: hbase
writes files that are split in hdfs blocks that are replicated accross the
datanodes. If you want to check the replications, you must look at what
files are written by hbase and how they have been split in blocks by hdfs
and how these blocks have been replicated. That will be in the hdfs
interface. As a side note, it's not the easiest thing to learn when you
start :-)
- The error > ERROR: org.apache.hadoop.hbase.MasterNotRunningException:
Retried 7 times
this is not linked to replication or whatever. It means that second
machine cannot find the master. You need to fix this first. (googling &
checking the logs).
Good luck,
Nicolas
On Thu, Sep 27, 2012 at 9:07 AM, Venkateswara Rao Dokku <dvrao.584@gmail.com
> wrote:
> How can we verify that the data(tables) is distributed across the cluster??
> Is there a way to confirm it that the data is distributed across all the
> nodes in the cluster.?
>
> On Thu, Sep 27, 2012 at 12:26 PM, Venkateswara Rao Dokku <
> dvrao.584@gmail.com> wrote:
>
> > Hi,
> > I am completely new to Hbase. I want to cluster the Hbase on two
> > nodes.I installed hadoop,hbase on the two nodes & my conf files are as
> > given below.
> > *cat conf/regionservers *
> > hbase-regionserver1
> > hbase-master
> > *cat conf/masters *
> > hadoop-namenode
> > * cat conf/slaves *
> > hadoop-datanode1
> > *vim conf/hdfs-site.xml *
> > <?xml version="1.0"?>
> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> >
> > <!-- Put site-specific property overrides in this file. -->
> >
> > <configuration>
> > <property>
> > <name>dfs.replication</name>
> > <value>2</value>
> > <description>Default block replication.The actual number of
> > replications can be specified when the file is created. The default is
> used
> > if replication is not specified in create time.
> > </description>
> > </property>
> > <property>
> > <name>dfs.support.append</name>
> > <value>true</value>
> > <description>Default block replication.The actual number of
> > replications can be specified when the file is created. The default is
> used
> > if replication is not specified in create time.
> > </description>
> > </property>
> > </configuration>
> > *& finally my /etc/hosts file is *
> > 127.0.0.1 localhost
> > 127.0.0.1 oc-PowerEdge-R610
> > 10.2.32.48 hbase-master hadoop-namenode
> > 10.240.13.35 hbase-regionserver1 hadoop-datanode1
> > The above files are identical on both of the machines. The following are
> > the processes that are running on my m/c's when I ran start scripts in
> > hadoop as well as hbase
> > *hadoop-namenode:*
> > HQuorumPeer
> > HMaster
> > Main
> > HRegionServer
> > SecondaryNameNode
> > Jps
> > NameNode
> > JobTracker
> > *hadoop-datanode1:*
> >
> > TaskTracker
> > Jps
> > DataNode
> > -- process information unavailable
> > Main
> > NC
> > HRegionServer
> >
> > I can able to create,list & scan tables on the *hadoop-namenode* machine
> > using Hbase shell. But while trying to run the same on the *
> > hadoop-datanode1 *machine I couldn't able to do it as I am getting
> > following error.
> > hbase(main):001:0> list
> > TABLE
> >
> >
> > ERROR: org.apache.hadoop.hbase.MasterNotRunningException: Retried 7 times
> >
> > Here is some help for this command:
> > List all tables in hbase. Optional regular expression parameter could
> > be used to filter the output. Examples:
> >
> > hbase> list
> > hbase> list 'abc.*'
> > How can I list,scan the tables that are created by the *hadoop-namenode *
> > from the *hadoop-datanode1* machine. Similarly Can I create some tables
> > on *hadoop-datanode1 *& can I access them from the *hadoop-namenode * &
> > vice-versa as the data is distributed as this is a cluster.
> >
> >
> >
> > --
> > Thanks & Regards,
> > Venkateswara Rao Dokku,
> > Software Engineer,One Convergence Devices Pvt Ltd.,
> > Jubille Hills,Hyderabad.
> >
> >
>
>
> --
> Thanks & Regards,
> Venkateswara Rao Dokku,
> Software Engineer,One Convergence Devices Pvt Ltd.,
> Jubille Hills,Hyderabad.
>
Re: Hbase clustering
Posted by Venkateswara Rao Dokku <dv...@gmail.com>.
How can we verify that the data(tables) is distributed across the cluster??
Is there a way to confirm it that the data is distributed across all the
nodes in the cluster.?
On Thu, Sep 27, 2012 at 12:26 PM, Venkateswara Rao Dokku <
dvrao.584@gmail.com> wrote:
> Hi,
> I am completely new to Hbase. I want to cluster the Hbase on two
> nodes.I installed hadoop,hbase on the two nodes & my conf files are as
> given below.
> *cat conf/regionservers *
> hbase-regionserver1
> hbase-master
> *cat conf/masters *
> hadoop-namenode
> * cat conf/slaves *
> hadoop-datanode1
> *vim conf/hdfs-site.xml *
> <?xml version="1.0"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>
> <!-- Put site-specific property overrides in this file. -->
>
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>2</value>
> <description>Default block replication.The actual number of
> replications can be specified when the file is created. The default is used
> if replication is not specified in create time.
> </description>
> </property>
> <property>
> <name>dfs.support.append</name>
> <value>true</value>
> <description>Default block replication.The actual number of
> replications can be specified when the file is created. The default is used
> if replication is not specified in create time.
> </description>
> </property>
> </configuration>
> *& finally my /etc/hosts file is *
> 127.0.0.1 localhost
> 127.0.0.1 oc-PowerEdge-R610
> 10.2.32.48 hbase-master hadoop-namenode
> 10.240.13.35 hbase-regionserver1 hadoop-datanode1
> The above files are identical on both of the machines. The following are
> the processes that are running on my m/c's when I ran start scripts in
> hadoop as well as hbase
> *hadoop-namenode:*
> HQuorumPeer
> HMaster
> Main
> HRegionServer
> SecondaryNameNode
> Jps
> NameNode
> JobTracker
> *hadoop-datanode1:*
>
> TaskTracker
> Jps
> DataNode
> -- process information unavailable
> Main
> NC
> HRegionServer
>
> I can able to create,list & scan tables on the *hadoop-namenode* machine
> using Hbase shell. But while trying to run the same on the *
> hadoop-datanode1 *machine I couldn't able to do it as I am getting
> following error.
> hbase(main):001:0> list
> TABLE
>
>
> ERROR: org.apache.hadoop.hbase.MasterNotRunningException: Retried 7 times
>
> Here is some help for this command:
> List all tables in hbase. Optional regular expression parameter could
> be used to filter the output. Examples:
>
> hbase> list
> hbase> list 'abc.*'
> How can I list,scan the tables that are created by the *hadoop-namenode *
> from the *hadoop-datanode1* machine. Similarly Can I create some tables
> on *hadoop-datanode1 *& can I access them from the *hadoop-namenode * &
> vice-versa as the data is distributed as this is a cluster.
>
>
>
> --
> Thanks & Regards,
> Venkateswara Rao Dokku,
> Software Engineer,One Convergence Devices Pvt Ltd.,
> Jubille Hills,Hyderabad.
>
>
--
Thanks & Regards,
Venkateswara Rao Dokku,
Software Engineer,One Convergence Devices Pvt Ltd.,
Jubille Hills,Hyderabad.