You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Pankil Doshi <fo...@gmail.com> on 2015/10/22 23:17:26 UTC
org.apache.hadoop.hbase.exceptions.DeserializationException: Missing
pb magic PBUF prefix
Hi,
I am using hbase-0.98.15-hadoop2 and hbase-indexer from lily (
http://ngdata.github.io/hbase-indexer/).
I am seeing below error when I add my indexer:
2015-10-22 14:08:27,468 INFO [regionserver60020-EventThread]
replication.ReplicationTrackerZKImpl: /hbase/replication/peers znode
expired, triggering peerListChanged event
2015-10-22 14:08:27,473 ERROR [regionserver60020-EventThread]
regionserver.ReplicationSourceManager: Error while adding a new peer
org.apache.hadoop.hbase.replication.ReplicationException: Error adding peer
with id=Indexer_newtest2
at
org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:386)
at
org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.peerAdded(ReplicationPeersZKImpl.java:358)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.peerListChanged(ReplicationSourceManager.java:514)
at
org.apache.hadoop.hbase.replication.ReplicationTrackerZKImpl$PeersWatcher.nodeChildrenChanged(ReplicationTrackerZKImpl.java:189)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:468)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
Caused by: org.apache.hadoop.hbase.replication.ReplicationException: Error
starting the peer state tracker for peerId=Indexer_newtest2
at
org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:454)
at
org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:384)
... 6 more
Caused by: org.apache.zookeeper.KeeperException$DataInconsistencyException:
KeeperErrorCode = DataInconsistency
at org.apache.hadoop.hbase.zookeeper.ZKUtil.convert(ZKUtil.java:2063)
at
org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:85)
at
org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:452)
... 7 more
Caused by: org.apache.hadoop.hbase.exceptions.DeserializationException:
Missing pb magic PBUF prefix
at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.expectPBMagicPrefix(ProtobufUtil.java:270)
at
org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.parseStateFrom(ReplicationPeerZKImpl.java:243)
at
org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.isStateEnabled(ReplicationPeerZKImpl.java:232)
at
org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.readPeerStateZnode(ReplicationPeerZKImpl.java:90)
at
org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:83)
... 8 more
My Hbase-site.xml:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
/**
* Copyright 2010 The Apache Software Foundation
*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-->
<configuration>
<!--<property>
<name>zookeeper.znode.parent</name>
<value>/hbase-unsecure</value>
</property>-->
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
//Here you have to set the path where you want HBase to store its files.
<property>
<name>hbase.rootdir</name>
<value>file:/tmp/HBase/HFiles</value>
</property>
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2181</value>
<description>Property from ZooKeeper's config zoo.cfg.
The port at which the clients will connect.
</description>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>localhost</value>
<description>Comma separated list of servers in the ZooKeeper Quorum.
For example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com
".
By default this is set to localhost for local and pseudo-distributed
modes
of operation. For a fully-distributed setup, this should be set to a
full
list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
hbase-env.sh
this is the list of servers which we will start/stop ZooKeeper on.
</description>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/tmp/zookeeper</value>
<description>Property from ZooKeeper config zoo.cfg.
The directory where the snapshot is stored.
</description>
</property>
// FOR HBASE INDEXER
<!-- SEP is basically replication, so enable it -->
<property>
<name>hbase.replication</name>
<value>true</value>
</property>
<!-- Source ratio of 100% makes sure that each SEP consumer is actually
used (otherwise, some can sit idle, especially with small clusters)
-->
<property>
<name>replication.source.ratio</name>
<value>1.0</value>
</property>
<!-- Maximum number of hlog entries to replicate in one go. If this is
large, and a consumer takes a while to process the events, the
HBase rpc call will time out. -->
<property>
<name>replication.source.nb.capacity</name>
<value>1000</value>
</property>
<!-- A custom replication source that fixes a few things and adds
some functionality (doesn't interfere with normal replication
usage). -->
<property>
<name>replication.replicationsource.implementation</name>
<value>com.ngdata.sep.impl.SepReplicationSource</value>
</property>
</configuration>
All research so far shows me one can face problem while upgrading hbase.
But in my case, I am not upgrading.
Any ideas/suggestions will be very helpful.
Thanks,
Pankil
Re: org.apache.hadoop.hbase.exceptions.DeserializationException:
Missing pb magic PBUF prefix
Posted by Pankil Doshi <fo...@gmail.com>.
Hi Ashish,
Yes that is due to hbase-indexer code. Still trying to find out soln.
No I didnt upgrade hbase from 0.94.
I directly installed hbase-0.98.15-hadoop2.
Thanks,
Pankil
On Fri, Oct 23, 2015 at 9:55 AM, Ashish Singhi <
ashish.singhi.apache@gmail.com> wrote:
> Hi Pankil.
>
> A similar issue was reported few days back (
>
> http://search-hadoop.com/m/YGbbknQt52rKBDS1&subj=HRegionServer+failed+due+to+replication
> ).
>
> May be this is due to hbase-indexer code ?
> One more Q, did you upgrade hbase from 0.94 and you see this issue ?
>
> Regards,
> Ashish Singhi
>
> On Fri, Oct 23, 2015 at 2:47 AM, Pankil Doshi <fo...@gmail.com> wrote:
>
> > Hi,
> >
> > I am using hbase-0.98.15-hadoop2 and hbase-indexer from lily (
> > http://ngdata.github.io/hbase-indexer/).
> >
> > I am seeing below error when I add my indexer:
> >
> >
> > 2015-10-22 14:08:27,468 INFO [regionserver60020-EventThread]
> > replication.ReplicationTrackerZKImpl: /hbase/replication/peers znode
> > expired, triggering peerListChanged event
> >
> > 2015-10-22 14:08:27,473 ERROR [regionserver60020-EventThread]
> > regionserver.ReplicationSourceManager: Error while adding a new peer
> >
> > org.apache.hadoop.hbase.replication.ReplicationException: Error adding
> peer
> > with id=Indexer_newtest2
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:386)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.peerAdded(ReplicationPeersZKImpl.java:358)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.peerListChanged(ReplicationSourceManager.java:514)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationTrackerZKImpl$PeersWatcher.nodeChildrenChanged(ReplicationTrackerZKImpl.java:189)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:468)
> >
> > at
> >
> >
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
> >
> > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
> >
> > Caused by: org.apache.hadoop.hbase.replication.ReplicationException:
> Error
> > starting the peer state tracker for peerId=Indexer_newtest2
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:454)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:384)
> >
> > ... 6 more
> >
> > Caused by:
> org.apache.zookeeper.KeeperException$DataInconsistencyException:
> > KeeperErrorCode = DataInconsistency
> >
> > at org.apache.hadoop.hbase.zookeeper.ZKUtil.convert(ZKUtil.java:2063)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:85)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:452)
> >
> > ... 7 more
> >
> > Caused by: org.apache.hadoop.hbase.exceptions.DeserializationException:
> > Missing pb magic PBUF prefix
> >
> > at
> >
> >
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.expectPBMagicPrefix(ProtobufUtil.java:270)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.parseStateFrom(ReplicationPeerZKImpl.java:243)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.isStateEnabled(ReplicationPeerZKImpl.java:232)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.readPeerStateZnode(ReplicationPeerZKImpl.java:90)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:83)
> >
> > ... 8 more
> >
> >
> >
> > My Hbase-site.xml:
> >
> >
> > <?xml version="1.0"?>
> >
> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> >
> > <!--
> >
> > /**
> >
> > * Copyright 2010 The Apache Software Foundation
> >
> > *
> >
> > * Licensed to the Apache Software Foundation (ASF) under one
> >
> > * or more contributor license agreements. See the NOTICE file
> >
> > * distributed with this work for additional information
> >
> > * regarding copyright ownership. The ASF licenses this file
> >
> > * to you under the Apache License, Version 2.0 (the
> >
> > * "License"); you may not use this file except in compliance
> >
> > * with the License. You may obtain a copy of the License at
> >
> > *
> >
> > * http://www.apache.org/licenses/LICENSE-2.0
> >
> > *
> >
> > * Unless required by applicable law or agreed to in writing, software
> >
> > * distributed under the License is distributed on an "AS IS" BASIS,
> >
> > * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> >
> > * See the License for the specific language governing permissions and
> >
> > * limitations under the License.
> >
> > */
> >
> > -->
> >
> > <configuration>
> >
> > <!--<property>
> >
> > <name>zookeeper.znode.parent</name>
> >
> > <value>/hbase-unsecure</value>
> >
> > </property>-->
> >
> >
> > <property>
> >
> > <name>hbase.cluster.distributed</name>
> >
> > <value>true</value>
> >
> > </property>
> >
> > //Here you have to set the path where you want HBase to store its files.
> >
> > <property>
> >
> > <name>hbase.rootdir</name>
> >
> > <value>file:/tmp/HBase/HFiles</value>
> >
> > </property>
> >
> > <property>
> >
> > <name>hbase.zookeeper.property.clientPort</name>
> >
> > <value>2181</value>
> >
> > <description>Property from ZooKeeper's config zoo.cfg.
> >
> > The port at which the clients will connect.
> >
> > </description>
> >
> > </property>
> >
> > <property>
> >
> > <name>hbase.zookeeper.quorum</name>
> >
> > <value>localhost</value>
> >
> > <description>Comma separated list of servers in the ZooKeeper
> Quorum.
> >
> > For example, "host1.mydomain.com,host2.mydomain.com,
> > host3.mydomain.com
> > ".
> >
> > By default this is set to localhost for local and
> pseudo-distributed
> > modes
> >
> > of operation. For a fully-distributed setup, this should be set to
> a
> > full
> >
> > list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
> > hbase-env.sh
> >
> > this is the list of servers which we will start/stop ZooKeeper on.
> >
> > </description>
> >
> > </property>
> >
> > <property>
> >
> > <name>hbase.zookeeper.property.dataDir</name>
> >
> > <value>/tmp/zookeeper</value>
> >
> > <description>Property from ZooKeeper config zoo.cfg.
> >
> > The directory where the snapshot is stored.
> >
> > </description>
> >
> > </property>
> >
> > // FOR HBASE INDEXER
> >
> >
> > <!-- SEP is basically replication, so enable it -->
> >
> > <property>
> >
> > <name>hbase.replication</name>
> >
> > <value>true</value>
> >
> > </property>
> >
> > <!-- Source ratio of 100% makes sure that each SEP consumer is actually
> >
> > used (otherwise, some can sit idle, especially with small
> clusters)
> > -->
> >
> > <property>
> >
> > <name>replication.source.ratio</name>
> >
> > <value>1.0</value>
> >
> > </property>
> >
> > <!-- Maximum number of hlog entries to replicate in one go. If this is
> >
> > large, and a consumer takes a while to process the events, the
> >
> > HBase rpc call will time out. -->
> >
> > <property>
> >
> > <name>replication.source.nb.capacity</name>
> >
> > <value>1000</value>
> >
> > </property>
> >
> > <!-- A custom replication source that fixes a few things and adds
> >
> > some functionality (doesn't interfere with normal replication
> >
> > usage). -->
> >
> > <property>
> >
> > <name>replication.replicationsource.implementation</name>
> >
> > <value>com.ngdata.sep.impl.SepReplicationSource</value>
> >
> > </property>
> >
> > </configuration>
> >
> >
> > All research so far shows me one can face problem while upgrading hbase.
> > But in my case, I am not upgrading.
> >
> > Any ideas/suggestions will be very helpful.
> >
> > Thanks,
> > Pankil
> >
>
Re: org.apache.hadoop.hbase.exceptions.DeserializationException:
Missing pb magic PBUF prefix
Posted by Ashish Singhi <as...@gmail.com>.
As you mentioned in one of the previous mail that this issue is due to
hbase-indexer code.
Can you post your problem on ngdata forum.
Regards,
Ashish Singhi
On Sat, Oct 24, 2015 at 2:31 AM, Pankil Doshi <fo...@gmail.com> wrote:
> I tried setting up using hdfs, still I have the same issue.
>
> On Fri, Oct 23, 2015 at 11:28 AM, Pankil Doshi <fo...@gmail.com>
> wrote:
>
> > One other thing which is different in my setup is I am using filesystem
> > for hbase rather hbase-indexer needs hdfs setup to be fully functional.
> So
> > I need to change that.
> >
> > Also, do you have local hbase setup or hbase cluster mode ?
> >
> > Pankil
> >
> > On Fri, Oct 23, 2015 at 11:23 AM, Pankil Doshi <fo...@gmail.com>
> > wrote:
> >
> >> Hi Beeshma,
> >>
> >> Thanks for your response.
> >>
> >> I am running zookeeper locally. but I am not managing it with hbase i.e
> I
> >> have this set:
> >> export HBASE_MANAGES_ZK=false
> >>
> >> and also:
> >> <property>
> >> <name>hbase.cluster.distributed</name>
> >> <value>true</value>
> >> </property>
> >>
> >> even though I have everything running locally in standalone mode.
> >>
> >> If I dont set "hbase.cluster.distributed" I am seeing zookeeper being
> >> started with start of my hbase. I am not sure if there is any other good
> >> way not to start or stop zookeeper with hbase as by only setting
> >> (HBASE_MANAGES_ZK=false) it doesnt work.
> >>
> >> Were you able to setup hbase-indexer at all ?
> >>
> >> Pankil
> >>
> >>
> >> On Fri, Oct 23, 2015 at 10:51 AM, beeshma r <be...@gmail.com>
> wrote:
> >>
> >>> Hi Pankil,
> >>>
> >>> Are you sure your hbase is running with external zookeeper ensemble ?
> >>>
> >>> As per documentation on Hbase Replication
> >>>
> >>>
> >>>
> http://www.cloudera.com/content/www/en-us/documentation/archive/cdh/4-x/4-2-0/CDH4-Installation-Guide/cdh4ig_topic_20_11.html
> >>>
> >>> zookeeper must not be managed by HBase,.But i havent tried this
> >>>
> >>> On Fri, Oct 23, 2015 at 9:55 AM, Ashish Singhi <
> >>> ashish.singhi.apache@gmail.com> wrote:
> >>>
> >>> > Hi Pankil.
> >>> >
> >>> > A similar issue was reported few days back (
> >>> >
> >>> >
> >>>
> http://search-hadoop.com/m/YGbbknQt52rKBDS1&subj=HRegionServer+failed+due+to+replication
> >>> > ).
> >>> >
> >>> > May be this is due to hbase-indexer code ?
> >>> > One more Q, did you upgrade hbase from 0.94 and you see this issue ?
> >>> >
> >>> > Regards,
> >>> > Ashish Singhi
> >>> >
> >>> > On Fri, Oct 23, 2015 at 2:47 AM, Pankil Doshi <fo...@gmail.com>
> >>> wrote:
> >>> >
> >>> > > Hi,
> >>> > >
> >>> > > I am using hbase-0.98.15-hadoop2 and hbase-indexer from lily (
> >>> > > http://ngdata.github.io/hbase-indexer/).
> >>> > >
> >>> > > I am seeing below error when I add my indexer:
> >>> > >
> >>> > >
> >>> > > 2015-10-22 14:08:27,468 INFO [regionserver60020-EventThread]
> >>> > > replication.ReplicationTrackerZKImpl: /hbase/replication/peers
> znode
> >>> > > expired, triggering peerListChanged event
> >>> > >
> >>> > > 2015-10-22 14:08:27,473 ERROR [regionserver60020-EventThread]
> >>> > > regionserver.ReplicationSourceManager: Error while adding a new
> peer
> >>> > >
> >>> > > org.apache.hadoop.hbase.replication.ReplicationException: Error
> >>> adding
> >>> > peer
> >>> > > with id=Indexer_newtest2
> >>> > >
> >>> > > at
> >>> > >
> >>> > >
> >>> >
> >>>
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:386)
> >>> > >
> >>> > > at
> >>> > >
> >>> > >
> >>> >
> >>>
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.peerAdded(ReplicationPeersZKImpl.java:358)
> >>> > >
> >>> > > at
> >>> > >
> >>> > >
> >>> >
> >>>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.peerListChanged(ReplicationSourceManager.java:514)
> >>> > >
> >>> > > at
> >>> > >
> >>> > >
> >>> >
> >>>
> org.apache.hadoop.hbase.replication.ReplicationTrackerZKImpl$PeersWatcher.nodeChildrenChanged(ReplicationTrackerZKImpl.java:189)
> >>> > >
> >>> > > at
> >>> > >
> >>> > >
> >>> >
> >>>
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:468)
> >>> > >
> >>> > > at
> >>> > >
> >>> > >
> >>> >
> >>>
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
> >>> > >
> >>> > > at
> >>> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
> >>> > >
> >>> > > Caused by:
> org.apache.hadoop.hbase.replication.ReplicationException:
> >>> > Error
> >>> > > starting the peer state tracker for peerId=Indexer_newtest2
> >>> > >
> >>> > > at
> >>> > >
> >>> > >
> >>> >
> >>>
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:454)
> >>> > >
> >>> > > at
> >>> > >
> >>> > >
> >>> >
> >>>
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:384)
> >>> > >
> >>> > > ... 6 more
> >>> > >
> >>> > > Caused by:
> >>> > org.apache.zookeeper.KeeperException$DataInconsistencyException:
> >>> > > KeeperErrorCode = DataInconsistency
> >>> > >
> >>> > > at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.convert(ZKUtil.java:2063)
> >>> > >
> >>> > > at
> >>> > >
> >>> > >
> >>> >
> >>>
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:85)
> >>> > >
> >>> > > at
> >>> > >
> >>> > >
> >>> >
> >>>
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:452)
> >>> > >
> >>> > > ... 7 more
> >>> > >
> >>> > > Caused by:
> >>> org.apache.hadoop.hbase.exceptions.DeserializationException:
> >>> > > Missing pb magic PBUF prefix
> >>> > >
> >>> > > at
> >>> > >
> >>> > >
> >>> >
> >>>
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.expectPBMagicPrefix(ProtobufUtil.java:270)
> >>> > >
> >>> > > at
> >>> > >
> >>> > >
> >>> >
> >>>
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.parseStateFrom(ReplicationPeerZKImpl.java:243)
> >>> > >
> >>> > > at
> >>> > >
> >>> > >
> >>> >
> >>>
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.isStateEnabled(ReplicationPeerZKImpl.java:232)
> >>> > >
> >>> > > at
> >>> > >
> >>> > >
> >>> >
> >>>
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.readPeerStateZnode(ReplicationPeerZKImpl.java:90)
> >>> > >
> >>> > > at
> >>> > >
> >>> > >
> >>> >
> >>>
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:83)
> >>> > >
> >>> > > ... 8 more
> >>> > >
> >>> > >
> >>> > >
> >>> > > My Hbase-site.xml:
> >>> > >
> >>> > >
> >>> > > <?xml version="1.0"?>
> >>> > >
> >>> > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> >>> > >
> >>> > > <!--
> >>> > >
> >>> > > /**
> >>> > >
> >>> > > * Copyright 2010 The Apache Software Foundation
> >>> > >
> >>> > > *
> >>> > >
> >>> > > * Licensed to the Apache Software Foundation (ASF) under one
> >>> > >
> >>> > > * or more contributor license agreements. See the NOTICE file
> >>> > >
> >>> > > * distributed with this work for additional information
> >>> > >
> >>> > > * regarding copyright ownership. The ASF licenses this file
> >>> > >
> >>> > > * to you under the Apache License, Version 2.0 (the
> >>> > >
> >>> > > * "License"); you may not use this file except in compliance
> >>> > >
> >>> > > * with the License. You may obtain a copy of the License at
> >>> > >
> >>> > > *
> >>> > >
> >>> > > * http://www.apache.org/licenses/LICENSE-2.0
> >>> > >
> >>> > > *
> >>> > >
> >>> > > * Unless required by applicable law or agreed to in writing,
> >>> software
> >>> > >
> >>> > > * distributed under the License is distributed on an "AS IS"
> BASIS,
> >>> > >
> >>> > > * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> >>> > implied.
> >>> > >
> >>> > > * See the License for the specific language governing permissions
> >>> and
> >>> > >
> >>> > > * limitations under the License.
> >>> > >
> >>> > > */
> >>> > >
> >>> > > -->
> >>> > >
> >>> > > <configuration>
> >>> > >
> >>> > > <!--<property>
> >>> > >
> >>> > > <name>zookeeper.znode.parent</name>
> >>> > >
> >>> > > <value>/hbase-unsecure</value>
> >>> > >
> >>> > > </property>-->
> >>> > >
> >>> > >
> >>> > > <property>
> >>> > >
> >>> > > <name>hbase.cluster.distributed</name>
> >>> > >
> >>> > > <value>true</value>
> >>> > >
> >>> > > </property>
> >>> > >
> >>> > > //Here you have to set the path where you want HBase to store its
> >>> files.
> >>> > >
> >>> > > <property>
> >>> > >
> >>> > > <name>hbase.rootdir</name>
> >>> > >
> >>> > > <value>file:/tmp/HBase/HFiles</value>
> >>> > >
> >>> > > </property>
> >>> > >
> >>> > > <property>
> >>> > >
> >>> > > <name>hbase.zookeeper.property.clientPort</name>
> >>> > >
> >>> > > <value>2181</value>
> >>> > >
> >>> > > <description>Property from ZooKeeper's config zoo.cfg.
> >>> > >
> >>> > > The port at which the clients will connect.
> >>> > >
> >>> > > </description>
> >>> > >
> >>> > > </property>
> >>> > >
> >>> > > <property>
> >>> > >
> >>> > > <name>hbase.zookeeper.quorum</name>
> >>> > >
> >>> > > <value>localhost</value>
> >>> > >
> >>> > > <description>Comma separated list of servers in the ZooKeeper
> >>> > Quorum.
> >>> > >
> >>> > > For example, "host1.mydomain.com,host2.mydomain.com,
> >>> > > host3.mydomain.com
> >>> > > ".
> >>> > >
> >>> > > By default this is set to localhost for local and
> >>> > pseudo-distributed
> >>> > > modes
> >>> > >
> >>> > > of operation. For a fully-distributed setup, this should be
> >>> set to
> >>> > a
> >>> > > full
> >>> > >
> >>> > > list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set
> in
> >>> > > hbase-env.sh
> >>> > >
> >>> > > this is the list of servers which we will start/stop
> ZooKeeper
> >>> on.
> >>> > >
> >>> > > </description>
> >>> > >
> >>> > > </property>
> >>> > >
> >>> > > <property>
> >>> > >
> >>> > > <name>hbase.zookeeper.property.dataDir</name>
> >>> > >
> >>> > > <value>/tmp/zookeeper</value>
> >>> > >
> >>> > > <description>Property from ZooKeeper config zoo.cfg.
> >>> > >
> >>> > > The directory where the snapshot is stored.
> >>> > >
> >>> > > </description>
> >>> > >
> >>> > > </property>
> >>> > >
> >>> > > // FOR HBASE INDEXER
> >>> > >
> >>> > >
> >>> > > <!-- SEP is basically replication, so enable it -->
> >>> > >
> >>> > > <property>
> >>> > >
> >>> > > <name>hbase.replication</name>
> >>> > >
> >>> > > <value>true</value>
> >>> > >
> >>> > > </property>
> >>> > >
> >>> > > <!-- Source ratio of 100% makes sure that each SEP consumer is
> >>> actually
> >>> > >
> >>> > > used (otherwise, some can sit idle, especially with small
> >>> > clusters)
> >>> > > -->
> >>> > >
> >>> > > <property>
> >>> > >
> >>> > > <name>replication.source.ratio</name>
> >>> > >
> >>> > > <value>1.0</value>
> >>> > >
> >>> > > </property>
> >>> > >
> >>> > > <!-- Maximum number of hlog entries to replicate in one go. If
> >>> this is
> >>> > >
> >>> > > large, and a consumer takes a while to process the events,
> the
> >>> > >
> >>> > > HBase rpc call will time out. -->
> >>> > >
> >>> > > <property>
> >>> > >
> >>> > > <name>replication.source.nb.capacity</name>
> >>> > >
> >>> > > <value>1000</value>
> >>> > >
> >>> > > </property>
> >>> > >
> >>> > > <!-- A custom replication source that fixes a few things and adds
> >>> > >
> >>> > > some functionality (doesn't interfere with normal
> replication
> >>> > >
> >>> > > usage). -->
> >>> > >
> >>> > > <property>
> >>> > >
> >>> > > <name>replication.replicationsource.implementation</name>
> >>> > >
> >>> > > <value>com.ngdata.sep.impl.SepReplicationSource</value>
> >>> > >
> >>> > > </property>
> >>> > >
> >>> > > </configuration>
> >>> > >
> >>> > >
> >>> > > All research so far shows me one can face problem while upgrading
> >>> hbase.
> >>> > > But in my case, I am not upgrading.
> >>> > >
> >>> > > Any ideas/suggestions will be very helpful.
> >>> > >
> >>> > > Thanks,
> >>> > > Pankil
> >>> > >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>>
> >>
> >>
> >
>
Re: org.apache.hadoop.hbase.exceptions.DeserializationException:
Missing pb magic PBUF prefix
Posted by Pankil Doshi <fo...@gmail.com>.
I tried setting up using hdfs, still I have the same issue.
On Fri, Oct 23, 2015 at 11:28 AM, Pankil Doshi <fo...@gmail.com> wrote:
> One other thing which is different in my setup is I am using filesystem
> for hbase rather hbase-indexer needs hdfs setup to be fully functional. So
> I need to change that.
>
> Also, do you have local hbase setup or hbase cluster mode ?
>
> Pankil
>
> On Fri, Oct 23, 2015 at 11:23 AM, Pankil Doshi <fo...@gmail.com>
> wrote:
>
>> Hi Beeshma,
>>
>> Thanks for your response.
>>
>> I am running zookeeper locally. but I am not managing it with hbase i.e I
>> have this set:
>> export HBASE_MANAGES_ZK=false
>>
>> and also:
>> <property>
>> <name>hbase.cluster.distributed</name>
>> <value>true</value>
>> </property>
>>
>> even though I have everything running locally in standalone mode.
>>
>> If I dont set "hbase.cluster.distributed" I am seeing zookeeper being
>> started with start of my hbase. I am not sure if there is any other good
>> way not to start or stop zookeeper with hbase as by only setting
>> (HBASE_MANAGES_ZK=false) it doesnt work.
>>
>> Were you able to setup hbase-indexer at all ?
>>
>> Pankil
>>
>>
>> On Fri, Oct 23, 2015 at 10:51 AM, beeshma r <be...@gmail.com> wrote:
>>
>>> Hi Pankil,
>>>
>>> Are you sure your hbase is running with external zookeeper ensemble ?
>>>
>>> As per documentation on Hbase Replication
>>>
>>>
>>> http://www.cloudera.com/content/www/en-us/documentation/archive/cdh/4-x/4-2-0/CDH4-Installation-Guide/cdh4ig_topic_20_11.html
>>>
>>> zookeeper must not be managed by HBase,.But i havent tried this
>>>
>>> On Fri, Oct 23, 2015 at 9:55 AM, Ashish Singhi <
>>> ashish.singhi.apache@gmail.com> wrote:
>>>
>>> > Hi Pankil.
>>> >
>>> > A similar issue was reported few days back (
>>> >
>>> >
>>> http://search-hadoop.com/m/YGbbknQt52rKBDS1&subj=HRegionServer+failed+due+to+replication
>>> > ).
>>> >
>>> > May be this is due to hbase-indexer code ?
>>> > One more Q, did you upgrade hbase from 0.94 and you see this issue ?
>>> >
>>> > Regards,
>>> > Ashish Singhi
>>> >
>>> > On Fri, Oct 23, 2015 at 2:47 AM, Pankil Doshi <fo...@gmail.com>
>>> wrote:
>>> >
>>> > > Hi,
>>> > >
>>> > > I am using hbase-0.98.15-hadoop2 and hbase-indexer from lily (
>>> > > http://ngdata.github.io/hbase-indexer/).
>>> > >
>>> > > I am seeing below error when I add my indexer:
>>> > >
>>> > >
>>> > > 2015-10-22 14:08:27,468 INFO [regionserver60020-EventThread]
>>> > > replication.ReplicationTrackerZKImpl: /hbase/replication/peers znode
>>> > > expired, triggering peerListChanged event
>>> > >
>>> > > 2015-10-22 14:08:27,473 ERROR [regionserver60020-EventThread]
>>> > > regionserver.ReplicationSourceManager: Error while adding a new peer
>>> > >
>>> > > org.apache.hadoop.hbase.replication.ReplicationException: Error
>>> adding
>>> > peer
>>> > > with id=Indexer_newtest2
>>> > >
>>> > > at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:386)
>>> > >
>>> > > at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.peerAdded(ReplicationPeersZKImpl.java:358)
>>> > >
>>> > > at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.peerListChanged(ReplicationSourceManager.java:514)
>>> > >
>>> > > at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.replication.ReplicationTrackerZKImpl$PeersWatcher.nodeChildrenChanged(ReplicationTrackerZKImpl.java:189)
>>> > >
>>> > > at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:468)
>>> > >
>>> > > at
>>> > >
>>> > >
>>> >
>>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
>>> > >
>>> > > at
>>> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
>>> > >
>>> > > Caused by: org.apache.hadoop.hbase.replication.ReplicationException:
>>> > Error
>>> > > starting the peer state tracker for peerId=Indexer_newtest2
>>> > >
>>> > > at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:454)
>>> > >
>>> > > at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:384)
>>> > >
>>> > > ... 6 more
>>> > >
>>> > > Caused by:
>>> > org.apache.zookeeper.KeeperException$DataInconsistencyException:
>>> > > KeeperErrorCode = DataInconsistency
>>> > >
>>> > > at org.apache.hadoop.hbase.zookeeper.ZKUtil.convert(ZKUtil.java:2063)
>>> > >
>>> > > at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:85)
>>> > >
>>> > > at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:452)
>>> > >
>>> > > ... 7 more
>>> > >
>>> > > Caused by:
>>> org.apache.hadoop.hbase.exceptions.DeserializationException:
>>> > > Missing pb magic PBUF prefix
>>> > >
>>> > > at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.protobuf.ProtobufUtil.expectPBMagicPrefix(ProtobufUtil.java:270)
>>> > >
>>> > > at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.parseStateFrom(ReplicationPeerZKImpl.java:243)
>>> > >
>>> > > at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.isStateEnabled(ReplicationPeerZKImpl.java:232)
>>> > >
>>> > > at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.readPeerStateZnode(ReplicationPeerZKImpl.java:90)
>>> > >
>>> > > at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:83)
>>> > >
>>> > > ... 8 more
>>> > >
>>> > >
>>> > >
>>> > > My Hbase-site.xml:
>>> > >
>>> > >
>>> > > <?xml version="1.0"?>
>>> > >
>>> > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>>> > >
>>> > > <!--
>>> > >
>>> > > /**
>>> > >
>>> > > * Copyright 2010 The Apache Software Foundation
>>> > >
>>> > > *
>>> > >
>>> > > * Licensed to the Apache Software Foundation (ASF) under one
>>> > >
>>> > > * or more contributor license agreements. See the NOTICE file
>>> > >
>>> > > * distributed with this work for additional information
>>> > >
>>> > > * regarding copyright ownership. The ASF licenses this file
>>> > >
>>> > > * to you under the Apache License, Version 2.0 (the
>>> > >
>>> > > * "License"); you may not use this file except in compliance
>>> > >
>>> > > * with the License. You may obtain a copy of the License at
>>> > >
>>> > > *
>>> > >
>>> > > * http://www.apache.org/licenses/LICENSE-2.0
>>> > >
>>> > > *
>>> > >
>>> > > * Unless required by applicable law or agreed to in writing,
>>> software
>>> > >
>>> > > * distributed under the License is distributed on an "AS IS" BASIS,
>>> > >
>>> > > * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>>> > implied.
>>> > >
>>> > > * See the License for the specific language governing permissions
>>> and
>>> > >
>>> > > * limitations under the License.
>>> > >
>>> > > */
>>> > >
>>> > > -->
>>> > >
>>> > > <configuration>
>>> > >
>>> > > <!--<property>
>>> > >
>>> > > <name>zookeeper.znode.parent</name>
>>> > >
>>> > > <value>/hbase-unsecure</value>
>>> > >
>>> > > </property>-->
>>> > >
>>> > >
>>> > > <property>
>>> > >
>>> > > <name>hbase.cluster.distributed</name>
>>> > >
>>> > > <value>true</value>
>>> > >
>>> > > </property>
>>> > >
>>> > > //Here you have to set the path where you want HBase to store its
>>> files.
>>> > >
>>> > > <property>
>>> > >
>>> > > <name>hbase.rootdir</name>
>>> > >
>>> > > <value>file:/tmp/HBase/HFiles</value>
>>> > >
>>> > > </property>
>>> > >
>>> > > <property>
>>> > >
>>> > > <name>hbase.zookeeper.property.clientPort</name>
>>> > >
>>> > > <value>2181</value>
>>> > >
>>> > > <description>Property from ZooKeeper's config zoo.cfg.
>>> > >
>>> > > The port at which the clients will connect.
>>> > >
>>> > > </description>
>>> > >
>>> > > </property>
>>> > >
>>> > > <property>
>>> > >
>>> > > <name>hbase.zookeeper.quorum</name>
>>> > >
>>> > > <value>localhost</value>
>>> > >
>>> > > <description>Comma separated list of servers in the ZooKeeper
>>> > Quorum.
>>> > >
>>> > > For example, "host1.mydomain.com,host2.mydomain.com,
>>> > > host3.mydomain.com
>>> > > ".
>>> > >
>>> > > By default this is set to localhost for local and
>>> > pseudo-distributed
>>> > > modes
>>> > >
>>> > > of operation. For a fully-distributed setup, this should be
>>> set to
>>> > a
>>> > > full
>>> > >
>>> > > list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
>>> > > hbase-env.sh
>>> > >
>>> > > this is the list of servers which we will start/stop ZooKeeper
>>> on.
>>> > >
>>> > > </description>
>>> > >
>>> > > </property>
>>> > >
>>> > > <property>
>>> > >
>>> > > <name>hbase.zookeeper.property.dataDir</name>
>>> > >
>>> > > <value>/tmp/zookeeper</value>
>>> > >
>>> > > <description>Property from ZooKeeper config zoo.cfg.
>>> > >
>>> > > The directory where the snapshot is stored.
>>> > >
>>> > > </description>
>>> > >
>>> > > </property>
>>> > >
>>> > > // FOR HBASE INDEXER
>>> > >
>>> > >
>>> > > <!-- SEP is basically replication, so enable it -->
>>> > >
>>> > > <property>
>>> > >
>>> > > <name>hbase.replication</name>
>>> > >
>>> > > <value>true</value>
>>> > >
>>> > > </property>
>>> > >
>>> > > <!-- Source ratio of 100% makes sure that each SEP consumer is
>>> actually
>>> > >
>>> > > used (otherwise, some can sit idle, especially with small
>>> > clusters)
>>> > > -->
>>> > >
>>> > > <property>
>>> > >
>>> > > <name>replication.source.ratio</name>
>>> > >
>>> > > <value>1.0</value>
>>> > >
>>> > > </property>
>>> > >
>>> > > <!-- Maximum number of hlog entries to replicate in one go. If
>>> this is
>>> > >
>>> > > large, and a consumer takes a while to process the events, the
>>> > >
>>> > > HBase rpc call will time out. -->
>>> > >
>>> > > <property>
>>> > >
>>> > > <name>replication.source.nb.capacity</name>
>>> > >
>>> > > <value>1000</value>
>>> > >
>>> > > </property>
>>> > >
>>> > > <!-- A custom replication source that fixes a few things and adds
>>> > >
>>> > > some functionality (doesn't interfere with normal replication
>>> > >
>>> > > usage). -->
>>> > >
>>> > > <property>
>>> > >
>>> > > <name>replication.replicationsource.implementation</name>
>>> > >
>>> > > <value>com.ngdata.sep.impl.SepReplicationSource</value>
>>> > >
>>> > > </property>
>>> > >
>>> > > </configuration>
>>> > >
>>> > >
>>> > > All research so far shows me one can face problem while upgrading
>>> hbase.
>>> > > But in my case, I am not upgrading.
>>> > >
>>> > > Any ideas/suggestions will be very helpful.
>>> > >
>>> > > Thanks,
>>> > > Pankil
>>> > >
>>> >
>>>
>>>
>>>
>>> --
>>>
>>
>>
>
Re: org.apache.hadoop.hbase.exceptions.DeserializationException:
Missing pb magic PBUF prefix
Posted by Pankil Doshi <fo...@gmail.com>.
One other thing which is different in my setup is I am using filesystem for
hbase rather hbase-indexer needs hdfs setup to be fully functional. So I
need to change that.
Also, do you have local hbase setup or hbase cluster mode ?
Pankil
On Fri, Oct 23, 2015 at 11:23 AM, Pankil Doshi <fo...@gmail.com> wrote:
> Hi Beeshma,
>
> Thanks for your response.
>
> I am running zookeeper locally. but I am not managing it with hbase i.e I
> have this set:
> export HBASE_MANAGES_ZK=false
>
> and also:
> <property>
> <name>hbase.cluster.distributed</name>
> <value>true</value>
> </property>
>
> even though I have everything running locally in standalone mode.
>
> If I dont set "hbase.cluster.distributed" I am seeing zookeeper being
> started with start of my hbase. I am not sure if there is any other good
> way not to start or stop zookeeper with hbase as by only setting
> (HBASE_MANAGES_ZK=false) it doesnt work.
>
> Were you able to setup hbase-indexer at all ?
>
> Pankil
>
>
> On Fri, Oct 23, 2015 at 10:51 AM, beeshma r <be...@gmail.com> wrote:
>
>> Hi Pankil,
>>
>> Are you sure your hbase is running with external zookeeper ensemble ?
>>
>> As per documentation on Hbase Replication
>>
>>
>> http://www.cloudera.com/content/www/en-us/documentation/archive/cdh/4-x/4-2-0/CDH4-Installation-Guide/cdh4ig_topic_20_11.html
>>
>> zookeeper must not be managed by HBase,.But i havent tried this
>>
>> On Fri, Oct 23, 2015 at 9:55 AM, Ashish Singhi <
>> ashish.singhi.apache@gmail.com> wrote:
>>
>> > Hi Pankil.
>> >
>> > A similar issue was reported few days back (
>> >
>> >
>> http://search-hadoop.com/m/YGbbknQt52rKBDS1&subj=HRegionServer+failed+due+to+replication
>> > ).
>> >
>> > May be this is due to hbase-indexer code ?
>> > One more Q, did you upgrade hbase from 0.94 and you see this issue ?
>> >
>> > Regards,
>> > Ashish Singhi
>> >
>> > On Fri, Oct 23, 2015 at 2:47 AM, Pankil Doshi <fo...@gmail.com>
>> wrote:
>> >
>> > > Hi,
>> > >
>> > > I am using hbase-0.98.15-hadoop2 and hbase-indexer from lily (
>> > > http://ngdata.github.io/hbase-indexer/).
>> > >
>> > > I am seeing below error when I add my indexer:
>> > >
>> > >
>> > > 2015-10-22 14:08:27,468 INFO [regionserver60020-EventThread]
>> > > replication.ReplicationTrackerZKImpl: /hbase/replication/peers znode
>> > > expired, triggering peerListChanged event
>> > >
>> > > 2015-10-22 14:08:27,473 ERROR [regionserver60020-EventThread]
>> > > regionserver.ReplicationSourceManager: Error while adding a new peer
>> > >
>> > > org.apache.hadoop.hbase.replication.ReplicationException: Error adding
>> > peer
>> > > with id=Indexer_newtest2
>> > >
>> > > at
>> > >
>> > >
>> >
>> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:386)
>> > >
>> > > at
>> > >
>> > >
>> >
>> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.peerAdded(ReplicationPeersZKImpl.java:358)
>> > >
>> > > at
>> > >
>> > >
>> >
>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.peerListChanged(ReplicationSourceManager.java:514)
>> > >
>> > > at
>> > >
>> > >
>> >
>> org.apache.hadoop.hbase.replication.ReplicationTrackerZKImpl$PeersWatcher.nodeChildrenChanged(ReplicationTrackerZKImpl.java:189)
>> > >
>> > > at
>> > >
>> > >
>> >
>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:468)
>> > >
>> > > at
>> > >
>> > >
>> >
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
>> > >
>> > > at
>> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
>> > >
>> > > Caused by: org.apache.hadoop.hbase.replication.ReplicationException:
>> > Error
>> > > starting the peer state tracker for peerId=Indexer_newtest2
>> > >
>> > > at
>> > >
>> > >
>> >
>> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:454)
>> > >
>> > > at
>> > >
>> > >
>> >
>> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:384)
>> > >
>> > > ... 6 more
>> > >
>> > > Caused by:
>> > org.apache.zookeeper.KeeperException$DataInconsistencyException:
>> > > KeeperErrorCode = DataInconsistency
>> > >
>> > > at org.apache.hadoop.hbase.zookeeper.ZKUtil.convert(ZKUtil.java:2063)
>> > >
>> > > at
>> > >
>> > >
>> >
>> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:85)
>> > >
>> > > at
>> > >
>> > >
>> >
>> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:452)
>> > >
>> > > ... 7 more
>> > >
>> > > Caused by:
>> org.apache.hadoop.hbase.exceptions.DeserializationException:
>> > > Missing pb magic PBUF prefix
>> > >
>> > > at
>> > >
>> > >
>> >
>> org.apache.hadoop.hbase.protobuf.ProtobufUtil.expectPBMagicPrefix(ProtobufUtil.java:270)
>> > >
>> > > at
>> > >
>> > >
>> >
>> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.parseStateFrom(ReplicationPeerZKImpl.java:243)
>> > >
>> > > at
>> > >
>> > >
>> >
>> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.isStateEnabled(ReplicationPeerZKImpl.java:232)
>> > >
>> > > at
>> > >
>> > >
>> >
>> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.readPeerStateZnode(ReplicationPeerZKImpl.java:90)
>> > >
>> > > at
>> > >
>> > >
>> >
>> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:83)
>> > >
>> > > ... 8 more
>> > >
>> > >
>> > >
>> > > My Hbase-site.xml:
>> > >
>> > >
>> > > <?xml version="1.0"?>
>> > >
>> > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>> > >
>> > > <!--
>> > >
>> > > /**
>> > >
>> > > * Copyright 2010 The Apache Software Foundation
>> > >
>> > > *
>> > >
>> > > * Licensed to the Apache Software Foundation (ASF) under one
>> > >
>> > > * or more contributor license agreements. See the NOTICE file
>> > >
>> > > * distributed with this work for additional information
>> > >
>> > > * regarding copyright ownership. The ASF licenses this file
>> > >
>> > > * to you under the Apache License, Version 2.0 (the
>> > >
>> > > * "License"); you may not use this file except in compliance
>> > >
>> > > * with the License. You may obtain a copy of the License at
>> > >
>> > > *
>> > >
>> > > * http://www.apache.org/licenses/LICENSE-2.0
>> > >
>> > > *
>> > >
>> > > * Unless required by applicable law or agreed to in writing, software
>> > >
>> > > * distributed under the License is distributed on an "AS IS" BASIS,
>> > >
>> > > * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>> > implied.
>> > >
>> > > * See the License for the specific language governing permissions and
>> > >
>> > > * limitations under the License.
>> > >
>> > > */
>> > >
>> > > -->
>> > >
>> > > <configuration>
>> > >
>> > > <!--<property>
>> > >
>> > > <name>zookeeper.znode.parent</name>
>> > >
>> > > <value>/hbase-unsecure</value>
>> > >
>> > > </property>-->
>> > >
>> > >
>> > > <property>
>> > >
>> > > <name>hbase.cluster.distributed</name>
>> > >
>> > > <value>true</value>
>> > >
>> > > </property>
>> > >
>> > > //Here you have to set the path where you want HBase to store its
>> files.
>> > >
>> > > <property>
>> > >
>> > > <name>hbase.rootdir</name>
>> > >
>> > > <value>file:/tmp/HBase/HFiles</value>
>> > >
>> > > </property>
>> > >
>> > > <property>
>> > >
>> > > <name>hbase.zookeeper.property.clientPort</name>
>> > >
>> > > <value>2181</value>
>> > >
>> > > <description>Property from ZooKeeper's config zoo.cfg.
>> > >
>> > > The port at which the clients will connect.
>> > >
>> > > </description>
>> > >
>> > > </property>
>> > >
>> > > <property>
>> > >
>> > > <name>hbase.zookeeper.quorum</name>
>> > >
>> > > <value>localhost</value>
>> > >
>> > > <description>Comma separated list of servers in the ZooKeeper
>> > Quorum.
>> > >
>> > > For example, "host1.mydomain.com,host2.mydomain.com,
>> > > host3.mydomain.com
>> > > ".
>> > >
>> > > By default this is set to localhost for local and
>> > pseudo-distributed
>> > > modes
>> > >
>> > > of operation. For a fully-distributed setup, this should be set
>> to
>> > a
>> > > full
>> > >
>> > > list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
>> > > hbase-env.sh
>> > >
>> > > this is the list of servers which we will start/stop ZooKeeper
>> on.
>> > >
>> > > </description>
>> > >
>> > > </property>
>> > >
>> > > <property>
>> > >
>> > > <name>hbase.zookeeper.property.dataDir</name>
>> > >
>> > > <value>/tmp/zookeeper</value>
>> > >
>> > > <description>Property from ZooKeeper config zoo.cfg.
>> > >
>> > > The directory where the snapshot is stored.
>> > >
>> > > </description>
>> > >
>> > > </property>
>> > >
>> > > // FOR HBASE INDEXER
>> > >
>> > >
>> > > <!-- SEP is basically replication, so enable it -->
>> > >
>> > > <property>
>> > >
>> > > <name>hbase.replication</name>
>> > >
>> > > <value>true</value>
>> > >
>> > > </property>
>> > >
>> > > <!-- Source ratio of 100% makes sure that each SEP consumer is
>> actually
>> > >
>> > > used (otherwise, some can sit idle, especially with small
>> > clusters)
>> > > -->
>> > >
>> > > <property>
>> > >
>> > > <name>replication.source.ratio</name>
>> > >
>> > > <value>1.0</value>
>> > >
>> > > </property>
>> > >
>> > > <!-- Maximum number of hlog entries to replicate in one go. If this
>> is
>> > >
>> > > large, and a consumer takes a while to process the events, the
>> > >
>> > > HBase rpc call will time out. -->
>> > >
>> > > <property>
>> > >
>> > > <name>replication.source.nb.capacity</name>
>> > >
>> > > <value>1000</value>
>> > >
>> > > </property>
>> > >
>> > > <!-- A custom replication source that fixes a few things and adds
>> > >
>> > > some functionality (doesn't interfere with normal replication
>> > >
>> > > usage). -->
>> > >
>> > > <property>
>> > >
>> > > <name>replication.replicationsource.implementation</name>
>> > >
>> > > <value>com.ngdata.sep.impl.SepReplicationSource</value>
>> > >
>> > > </property>
>> > >
>> > > </configuration>
>> > >
>> > >
>> > > All research so far shows me one can face problem while upgrading
>> hbase.
>> > > But in my case, I am not upgrading.
>> > >
>> > > Any ideas/suggestions will be very helpful.
>> > >
>> > > Thanks,
>> > > Pankil
>> > >
>> >
>>
>>
>>
>> --
>>
>
>
Re: org.apache.hadoop.hbase.exceptions.DeserializationException:
Missing pb magic PBUF prefix
Posted by Pankil Doshi <fo...@gmail.com>.
Hi Beeshma,
Thanks for your response.
I am running zookeeper locally. but I am not managing it with hbase i.e I
have this set:
export HBASE_MANAGES_ZK=false
and also:
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
even though I have everything running locally in standalone mode.
If I dont set "hbase.cluster.distributed" I am seeing zookeeper being
started with start of my hbase. I am not sure if there is any other good
way not to start or stop zookeeper with hbase as by only setting
(HBASE_MANAGES_ZK=false) it doesnt work.
Were you able to setup hbase-indexer at all ?
Pankil
On Fri, Oct 23, 2015 at 10:51 AM, beeshma r <be...@gmail.com> wrote:
> Hi Pankil,
>
> Are you sure your hbase is running with external zookeeper ensemble ?
>
> As per documentation on Hbase Replication
>
>
> http://www.cloudera.com/content/www/en-us/documentation/archive/cdh/4-x/4-2-0/CDH4-Installation-Guide/cdh4ig_topic_20_11.html
>
> zookeeper must not be managed by HBase,.But i havent tried this
>
> On Fri, Oct 23, 2015 at 9:55 AM, Ashish Singhi <
> ashish.singhi.apache@gmail.com> wrote:
>
> > Hi Pankil.
> >
> > A similar issue was reported few days back (
> >
> >
> http://search-hadoop.com/m/YGbbknQt52rKBDS1&subj=HRegionServer+failed+due+to+replication
> > ).
> >
> > May be this is due to hbase-indexer code ?
> > One more Q, did you upgrade hbase from 0.94 and you see this issue ?
> >
> > Regards,
> > Ashish Singhi
> >
> > On Fri, Oct 23, 2015 at 2:47 AM, Pankil Doshi <fo...@gmail.com>
> wrote:
> >
> > > Hi,
> > >
> > > I am using hbase-0.98.15-hadoop2 and hbase-indexer from lily (
> > > http://ngdata.github.io/hbase-indexer/).
> > >
> > > I am seeing below error when I add my indexer:
> > >
> > >
> > > 2015-10-22 14:08:27,468 INFO [regionserver60020-EventThread]
> > > replication.ReplicationTrackerZKImpl: /hbase/replication/peers znode
> > > expired, triggering peerListChanged event
> > >
> > > 2015-10-22 14:08:27,473 ERROR [regionserver60020-EventThread]
> > > regionserver.ReplicationSourceManager: Error while adding a new peer
> > >
> > > org.apache.hadoop.hbase.replication.ReplicationException: Error adding
> > peer
> > > with id=Indexer_newtest2
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:386)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.peerAdded(ReplicationPeersZKImpl.java:358)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.peerListChanged(ReplicationSourceManager.java:514)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationTrackerZKImpl$PeersWatcher.nodeChildrenChanged(ReplicationTrackerZKImpl.java:189)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:468)
> > >
> > > at
> > >
> > >
> >
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
> > >
> > > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
> > >
> > > Caused by: org.apache.hadoop.hbase.replication.ReplicationException:
> > Error
> > > starting the peer state tracker for peerId=Indexer_newtest2
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:454)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:384)
> > >
> > > ... 6 more
> > >
> > > Caused by:
> > org.apache.zookeeper.KeeperException$DataInconsistencyException:
> > > KeeperErrorCode = DataInconsistency
> > >
> > > at org.apache.hadoop.hbase.zookeeper.ZKUtil.convert(ZKUtil.java:2063)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:85)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:452)
> > >
> > > ... 7 more
> > >
> > > Caused by: org.apache.hadoop.hbase.exceptions.DeserializationException:
> > > Missing pb magic PBUF prefix
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.expectPBMagicPrefix(ProtobufUtil.java:270)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.parseStateFrom(ReplicationPeerZKImpl.java:243)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.isStateEnabled(ReplicationPeerZKImpl.java:232)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.readPeerStateZnode(ReplicationPeerZKImpl.java:90)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:83)
> > >
> > > ... 8 more
> > >
> > >
> > >
> > > My Hbase-site.xml:
> > >
> > >
> > > <?xml version="1.0"?>
> > >
> > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> > >
> > > <!--
> > >
> > > /**
> > >
> > > * Copyright 2010 The Apache Software Foundation
> > >
> > > *
> > >
> > > * Licensed to the Apache Software Foundation (ASF) under one
> > >
> > > * or more contributor license agreements. See the NOTICE file
> > >
> > > * distributed with this work for additional information
> > >
> > > * regarding copyright ownership. The ASF licenses this file
> > >
> > > * to you under the Apache License, Version 2.0 (the
> > >
> > > * "License"); you may not use this file except in compliance
> > >
> > > * with the License. You may obtain a copy of the License at
> > >
> > > *
> > >
> > > * http://www.apache.org/licenses/LICENSE-2.0
> > >
> > > *
> > >
> > > * Unless required by applicable law or agreed to in writing, software
> > >
> > > * distributed under the License is distributed on an "AS IS" BASIS,
> > >
> > > * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> > implied.
> > >
> > > * See the License for the specific language governing permissions and
> > >
> > > * limitations under the License.
> > >
> > > */
> > >
> > > -->
> > >
> > > <configuration>
> > >
> > > <!--<property>
> > >
> > > <name>zookeeper.znode.parent</name>
> > >
> > > <value>/hbase-unsecure</value>
> > >
> > > </property>-->
> > >
> > >
> > > <property>
> > >
> > > <name>hbase.cluster.distributed</name>
> > >
> > > <value>true</value>
> > >
> > > </property>
> > >
> > > //Here you have to set the path where you want HBase to store its
> files.
> > >
> > > <property>
> > >
> > > <name>hbase.rootdir</name>
> > >
> > > <value>file:/tmp/HBase/HFiles</value>
> > >
> > > </property>
> > >
> > > <property>
> > >
> > > <name>hbase.zookeeper.property.clientPort</name>
> > >
> > > <value>2181</value>
> > >
> > > <description>Property from ZooKeeper's config zoo.cfg.
> > >
> > > The port at which the clients will connect.
> > >
> > > </description>
> > >
> > > </property>
> > >
> > > <property>
> > >
> > > <name>hbase.zookeeper.quorum</name>
> > >
> > > <value>localhost</value>
> > >
> > > <description>Comma separated list of servers in the ZooKeeper
> > Quorum.
> > >
> > > For example, "host1.mydomain.com,host2.mydomain.com,
> > > host3.mydomain.com
> > > ".
> > >
> > > By default this is set to localhost for local and
> > pseudo-distributed
> > > modes
> > >
> > > of operation. For a fully-distributed setup, this should be set
> to
> > a
> > > full
> > >
> > > list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
> > > hbase-env.sh
> > >
> > > this is the list of servers which we will start/stop ZooKeeper
> on.
> > >
> > > </description>
> > >
> > > </property>
> > >
> > > <property>
> > >
> > > <name>hbase.zookeeper.property.dataDir</name>
> > >
> > > <value>/tmp/zookeeper</value>
> > >
> > > <description>Property from ZooKeeper config zoo.cfg.
> > >
> > > The directory where the snapshot is stored.
> > >
> > > </description>
> > >
> > > </property>
> > >
> > > // FOR HBASE INDEXER
> > >
> > >
> > > <!-- SEP is basically replication, so enable it -->
> > >
> > > <property>
> > >
> > > <name>hbase.replication</name>
> > >
> > > <value>true</value>
> > >
> > > </property>
> > >
> > > <!-- Source ratio of 100% makes sure that each SEP consumer is
> actually
> > >
> > > used (otherwise, some can sit idle, especially with small
> > clusters)
> > > -->
> > >
> > > <property>
> > >
> > > <name>replication.source.ratio</name>
> > >
> > > <value>1.0</value>
> > >
> > > </property>
> > >
> > > <!-- Maximum number of hlog entries to replicate in one go. If this
> is
> > >
> > > large, and a consumer takes a while to process the events, the
> > >
> > > HBase rpc call will time out. -->
> > >
> > > <property>
> > >
> > > <name>replication.source.nb.capacity</name>
> > >
> > > <value>1000</value>
> > >
> > > </property>
> > >
> > > <!-- A custom replication source that fixes a few things and adds
> > >
> > > some functionality (doesn't interfere with normal replication
> > >
> > > usage). -->
> > >
> > > <property>
> > >
> > > <name>replication.replicationsource.implementation</name>
> > >
> > > <value>com.ngdata.sep.impl.SepReplicationSource</value>
> > >
> > > </property>
> > >
> > > </configuration>
> > >
> > >
> > > All research so far shows me one can face problem while upgrading
> hbase.
> > > But in my case, I am not upgrading.
> > >
> > > Any ideas/suggestions will be very helpful.
> > >
> > > Thanks,
> > > Pankil
> > >
> >
>
>
>
> --
>
Re: org.apache.hadoop.hbase.exceptions.DeserializationException:
Missing pb magic PBUF prefix
Posted by beeshma r <be...@gmail.com>.
Hi Pankil,
Are you sure your hbase is running with external zookeeper ensemble ?
As per documentation on Hbase Replication
http://www.cloudera.com/content/www/en-us/documentation/archive/cdh/4-x/4-2-0/CDH4-Installation-Guide/cdh4ig_topic_20_11.html
zookeeper must not be managed by HBase,.But i havent tried this
On Fri, Oct 23, 2015 at 9:55 AM, Ashish Singhi <
ashish.singhi.apache@gmail.com> wrote:
> Hi Pankil.
>
> A similar issue was reported few days back (
>
> http://search-hadoop.com/m/YGbbknQt52rKBDS1&subj=HRegionServer+failed+due+to+replication
> ).
>
> May be this is due to hbase-indexer code ?
> One more Q, did you upgrade hbase from 0.94 and you see this issue ?
>
> Regards,
> Ashish Singhi
>
> On Fri, Oct 23, 2015 at 2:47 AM, Pankil Doshi <fo...@gmail.com> wrote:
>
> > Hi,
> >
> > I am using hbase-0.98.15-hadoop2 and hbase-indexer from lily (
> > http://ngdata.github.io/hbase-indexer/).
> >
> > I am seeing below error when I add my indexer:
> >
> >
> > 2015-10-22 14:08:27,468 INFO [regionserver60020-EventThread]
> > replication.ReplicationTrackerZKImpl: /hbase/replication/peers znode
> > expired, triggering peerListChanged event
> >
> > 2015-10-22 14:08:27,473 ERROR [regionserver60020-EventThread]
> > regionserver.ReplicationSourceManager: Error while adding a new peer
> >
> > org.apache.hadoop.hbase.replication.ReplicationException: Error adding
> peer
> > with id=Indexer_newtest2
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:386)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.peerAdded(ReplicationPeersZKImpl.java:358)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.peerListChanged(ReplicationSourceManager.java:514)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationTrackerZKImpl$PeersWatcher.nodeChildrenChanged(ReplicationTrackerZKImpl.java:189)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:468)
> >
> > at
> >
> >
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
> >
> > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
> >
> > Caused by: org.apache.hadoop.hbase.replication.ReplicationException:
> Error
> > starting the peer state tracker for peerId=Indexer_newtest2
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:454)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:384)
> >
> > ... 6 more
> >
> > Caused by:
> org.apache.zookeeper.KeeperException$DataInconsistencyException:
> > KeeperErrorCode = DataInconsistency
> >
> > at org.apache.hadoop.hbase.zookeeper.ZKUtil.convert(ZKUtil.java:2063)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:85)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:452)
> >
> > ... 7 more
> >
> > Caused by: org.apache.hadoop.hbase.exceptions.DeserializationException:
> > Missing pb magic PBUF prefix
> >
> > at
> >
> >
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.expectPBMagicPrefix(ProtobufUtil.java:270)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.parseStateFrom(ReplicationPeerZKImpl.java:243)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.isStateEnabled(ReplicationPeerZKImpl.java:232)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.readPeerStateZnode(ReplicationPeerZKImpl.java:90)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:83)
> >
> > ... 8 more
> >
> >
> >
> > My Hbase-site.xml:
> >
> >
> > <?xml version="1.0"?>
> >
> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> >
> > <!--
> >
> > /**
> >
> > * Copyright 2010 The Apache Software Foundation
> >
> > *
> >
> > * Licensed to the Apache Software Foundation (ASF) under one
> >
> > * or more contributor license agreements. See the NOTICE file
> >
> > * distributed with this work for additional information
> >
> > * regarding copyright ownership. The ASF licenses this file
> >
> > * to you under the Apache License, Version 2.0 (the
> >
> > * "License"); you may not use this file except in compliance
> >
> > * with the License. You may obtain a copy of the License at
> >
> > *
> >
> > * http://www.apache.org/licenses/LICENSE-2.0
> >
> > *
> >
> > * Unless required by applicable law or agreed to in writing, software
> >
> > * distributed under the License is distributed on an "AS IS" BASIS,
> >
> > * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> >
> > * See the License for the specific language governing permissions and
> >
> > * limitations under the License.
> >
> > */
> >
> > -->
> >
> > <configuration>
> >
> > <!--<property>
> >
> > <name>zookeeper.znode.parent</name>
> >
> > <value>/hbase-unsecure</value>
> >
> > </property>-->
> >
> >
> > <property>
> >
> > <name>hbase.cluster.distributed</name>
> >
> > <value>true</value>
> >
> > </property>
> >
> > //Here you have to set the path where you want HBase to store its files.
> >
> > <property>
> >
> > <name>hbase.rootdir</name>
> >
> > <value>file:/tmp/HBase/HFiles</value>
> >
> > </property>
> >
> > <property>
> >
> > <name>hbase.zookeeper.property.clientPort</name>
> >
> > <value>2181</value>
> >
> > <description>Property from ZooKeeper's config zoo.cfg.
> >
> > The port at which the clients will connect.
> >
> > </description>
> >
> > </property>
> >
> > <property>
> >
> > <name>hbase.zookeeper.quorum</name>
> >
> > <value>localhost</value>
> >
> > <description>Comma separated list of servers in the ZooKeeper
> Quorum.
> >
> > For example, "host1.mydomain.com,host2.mydomain.com,
> > host3.mydomain.com
> > ".
> >
> > By default this is set to localhost for local and
> pseudo-distributed
> > modes
> >
> > of operation. For a fully-distributed setup, this should be set to
> a
> > full
> >
> > list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
> > hbase-env.sh
> >
> > this is the list of servers which we will start/stop ZooKeeper on.
> >
> > </description>
> >
> > </property>
> >
> > <property>
> >
> > <name>hbase.zookeeper.property.dataDir</name>
> >
> > <value>/tmp/zookeeper</value>
> >
> > <description>Property from ZooKeeper config zoo.cfg.
> >
> > The directory where the snapshot is stored.
> >
> > </description>
> >
> > </property>
> >
> > // FOR HBASE INDEXER
> >
> >
> > <!-- SEP is basically replication, so enable it -->
> >
> > <property>
> >
> > <name>hbase.replication</name>
> >
> > <value>true</value>
> >
> > </property>
> >
> > <!-- Source ratio of 100% makes sure that each SEP consumer is actually
> >
> > used (otherwise, some can sit idle, especially with small
> clusters)
> > -->
> >
> > <property>
> >
> > <name>replication.source.ratio</name>
> >
> > <value>1.0</value>
> >
> > </property>
> >
> > <!-- Maximum number of hlog entries to replicate in one go. If this is
> >
> > large, and a consumer takes a while to process the events, the
> >
> > HBase rpc call will time out. -->
> >
> > <property>
> >
> > <name>replication.source.nb.capacity</name>
> >
> > <value>1000</value>
> >
> > </property>
> >
> > <!-- A custom replication source that fixes a few things and adds
> >
> > some functionality (doesn't interfere with normal replication
> >
> > usage). -->
> >
> > <property>
> >
> > <name>replication.replicationsource.implementation</name>
> >
> > <value>com.ngdata.sep.impl.SepReplicationSource</value>
> >
> > </property>
> >
> > </configuration>
> >
> >
> > All research so far shows me one can face problem while upgrading hbase.
> > But in my case, I am not upgrading.
> >
> > Any ideas/suggestions will be very helpful.
> >
> > Thanks,
> > Pankil
> >
>
--
Re: org.apache.hadoop.hbase.exceptions.DeserializationException:
Missing pb magic PBUF prefix
Posted by Ashish Singhi <as...@gmail.com>.
Hi Pankil.
A similar issue was reported few days back (
http://search-hadoop.com/m/YGbbknQt52rKBDS1&subj=HRegionServer+failed+due+to+replication
).
May be this is due to hbase-indexer code ?
One more Q, did you upgrade hbase from 0.94 and you see this issue ?
Regards,
Ashish Singhi
On Fri, Oct 23, 2015 at 2:47 AM, Pankil Doshi <fo...@gmail.com> wrote:
> Hi,
>
> I am using hbase-0.98.15-hadoop2 and hbase-indexer from lily (
> http://ngdata.github.io/hbase-indexer/).
>
> I am seeing below error when I add my indexer:
>
>
> 2015-10-22 14:08:27,468 INFO [regionserver60020-EventThread]
> replication.ReplicationTrackerZKImpl: /hbase/replication/peers znode
> expired, triggering peerListChanged event
>
> 2015-10-22 14:08:27,473 ERROR [regionserver60020-EventThread]
> regionserver.ReplicationSourceManager: Error while adding a new peer
>
> org.apache.hadoop.hbase.replication.ReplicationException: Error adding peer
> with id=Indexer_newtest2
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:386)
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.peerAdded(ReplicationPeersZKImpl.java:358)
>
> at
>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.peerListChanged(ReplicationSourceManager.java:514)
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationTrackerZKImpl$PeersWatcher.nodeChildrenChanged(ReplicationTrackerZKImpl.java:189)
>
> at
>
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:468)
>
> at
>
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
>
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
>
> Caused by: org.apache.hadoop.hbase.replication.ReplicationException: Error
> starting the peer state tracker for peerId=Indexer_newtest2
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:454)
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:384)
>
> ... 6 more
>
> Caused by: org.apache.zookeeper.KeeperException$DataInconsistencyException:
> KeeperErrorCode = DataInconsistency
>
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.convert(ZKUtil.java:2063)
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:85)
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:452)
>
> ... 7 more
>
> Caused by: org.apache.hadoop.hbase.exceptions.DeserializationException:
> Missing pb magic PBUF prefix
>
> at
>
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.expectPBMagicPrefix(ProtobufUtil.java:270)
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.parseStateFrom(ReplicationPeerZKImpl.java:243)
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.isStateEnabled(ReplicationPeerZKImpl.java:232)
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.readPeerStateZnode(ReplicationPeerZKImpl.java:90)
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:83)
>
> ... 8 more
>
>
>
> My Hbase-site.xml:
>
>
> <?xml version="1.0"?>
>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>
> <!--
>
> /**
>
> * Copyright 2010 The Apache Software Foundation
>
> *
>
> * Licensed to the Apache Software Foundation (ASF) under one
>
> * or more contributor license agreements. See the NOTICE file
>
> * distributed with this work for additional information
>
> * regarding copyright ownership. The ASF licenses this file
>
> * to you under the Apache License, Version 2.0 (the
>
> * "License"); you may not use this file except in compliance
>
> * with the License. You may obtain a copy of the License at
>
> *
>
> * http://www.apache.org/licenses/LICENSE-2.0
>
> *
>
> * Unless required by applicable law or agreed to in writing, software
>
> * distributed under the License is distributed on an "AS IS" BASIS,
>
> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>
> * See the License for the specific language governing permissions and
>
> * limitations under the License.
>
> */
>
> -->
>
> <configuration>
>
> <!--<property>
>
> <name>zookeeper.znode.parent</name>
>
> <value>/hbase-unsecure</value>
>
> </property>-->
>
>
> <property>
>
> <name>hbase.cluster.distributed</name>
>
> <value>true</value>
>
> </property>
>
> //Here you have to set the path where you want HBase to store its files.
>
> <property>
>
> <name>hbase.rootdir</name>
>
> <value>file:/tmp/HBase/HFiles</value>
>
> </property>
>
> <property>
>
> <name>hbase.zookeeper.property.clientPort</name>
>
> <value>2181</value>
>
> <description>Property from ZooKeeper's config zoo.cfg.
>
> The port at which the clients will connect.
>
> </description>
>
> </property>
>
> <property>
>
> <name>hbase.zookeeper.quorum</name>
>
> <value>localhost</value>
>
> <description>Comma separated list of servers in the ZooKeeper Quorum.
>
> For example, "host1.mydomain.com,host2.mydomain.com,
> host3.mydomain.com
> ".
>
> By default this is set to localhost for local and pseudo-distributed
> modes
>
> of operation. For a fully-distributed setup, this should be set to a
> full
>
> list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
> hbase-env.sh
>
> this is the list of servers which we will start/stop ZooKeeper on.
>
> </description>
>
> </property>
>
> <property>
>
> <name>hbase.zookeeper.property.dataDir</name>
>
> <value>/tmp/zookeeper</value>
>
> <description>Property from ZooKeeper config zoo.cfg.
>
> The directory where the snapshot is stored.
>
> </description>
>
> </property>
>
> // FOR HBASE INDEXER
>
>
> <!-- SEP is basically replication, so enable it -->
>
> <property>
>
> <name>hbase.replication</name>
>
> <value>true</value>
>
> </property>
>
> <!-- Source ratio of 100% makes sure that each SEP consumer is actually
>
> used (otherwise, some can sit idle, especially with small clusters)
> -->
>
> <property>
>
> <name>replication.source.ratio</name>
>
> <value>1.0</value>
>
> </property>
>
> <!-- Maximum number of hlog entries to replicate in one go. If this is
>
> large, and a consumer takes a while to process the events, the
>
> HBase rpc call will time out. -->
>
> <property>
>
> <name>replication.source.nb.capacity</name>
>
> <value>1000</value>
>
> </property>
>
> <!-- A custom replication source that fixes a few things and adds
>
> some functionality (doesn't interfere with normal replication
>
> usage). -->
>
> <property>
>
> <name>replication.replicationsource.implementation</name>
>
> <value>com.ngdata.sep.impl.SepReplicationSource</value>
>
> </property>
>
> </configuration>
>
>
> All research so far shows me one can face problem while upgrading hbase.
> But in my case, I am not upgrading.
>
> Any ideas/suggestions will be very helpful.
>
> Thanks,
> Pankil
>