You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by "Brahma Reddy Battula (JIRA)" <ji...@apache.org> on 2012/07/11 16:10:34 UTC
[jira] [Commented] (BOOKKEEPER-337) Add entry fails with
MetadataVersionException when last ensemble has morethan one bookie
failures
[ https://issues.apache.org/jira/browse/BOOKKEEPER-337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411538#comment-13411538 ]
Brahma Reddy Battula commented on BOOKKEEPER-337:
-------------------------------------------------
I had faced this in HDFS-HA,where ANN got shutdown by throwing following..Here availoble bookies are seven even then addentry failed because continuous fail bookie fail which are in ensemble
{noformat}
2012-07-11 04:02:50,445 ERROR hidden.bkjournal.org.apache.bookkeeper.proto.PerChannelBookieClient: Invoked callback method: 1058
2012-07-11 04:02:50,445 ERROR hidden.bkjournal.org.apache.bookkeeper.proto.PerChannelBookieClient: Could not write request for adding entry: 1058 ledger-id: 364 bookie: /HOST****.168:3183
2012-07-11 04:02:50,445 WARN hidden.bkjournal.org.apache.bookkeeper.client.PendingAddOp: Write did not succeed: 364, 1058
2012-07-11 04:02:50,445 ERROR hidden.bkjournal.org.apache.bookkeeper.proto.PerChannelBookieClient: Invoked callback method: 1058
2012-07-11 04:02:50,445 ERROR hidden.bkjournal.org.apache.bookkeeper.proto.PerChannelBookieClient: Unexpected add response received from bookie: /HOST****.168:3183 for ledger: 364, entry: 1058 , ignoring
2012-07-11 04:02:50,445 ERROR hidden.bkjournal.org.apache.bookkeeper.proto.PerChannelBookieClient: Unexpected add response received from bookie: /HOST****.168:3182 for ledger: 364, entry: 1058 , ignoring
2012-07-11 04:02:50,981 ERROR hidden.bkjournal.org.apache.bookkeeper.client.LedgerHandle: Could not resolve ledger metadata conflict while changing ensemble to: [/HOST****.20:3181, /HOST****.168:3181, /10.18.52.55:3187], old meta data is
BookieMetadataFormatVersion 1
2
3
0
0 HOST****.168:3183 HOST****.168:3181 HOST****.168:3182
, new meta data is
BookieMetadataFormatVersion 1
2
3
0
0 HOST****.168:3183 HOST****.168:3181 HOST****.168:3182
1058 HOST****.168:3183 HOST****.168:3181 10.18.52.55:3187
,closing ledger
2012-07-11 04:02:51,366 FATAL org.apache.hadoop.hdfs.server.namenode.FSEditLog: Error: flush failed for required journal (JournalAndStream(mgr=org.apache.hadoop.contrib.bkjournal.BookKeeperJournalManager@f73f2c5, stream=org.apache.hadoop.contrib.bkjournal.BookKeeperEditLogOutputStream@5476b70))
java.io.IOException: Failed to write to bookkeeper; Error is (-9) Error while using ZooKeeper
at org.apache.hadoop.contrib.bkjournal.BookKeeperEditLogOutputStream.flushAndSync(BookKeeperEditLogOutputStream.java:141)
at org.apache.hadoop.hdfs.server.namenode.EditLogOutputStream.flush(EditLogOutputStream.java:105)
at org.apache.hadoop.hdfs.server.namenode.JournalSet$JournalSetOutputStream$8.apply(JournalSet.java:460)
at org.apache.hadoop.hdfs.server.namenode.JournalSet.mapJournalsAndReportErrors(JournalSet.java:319)
at org.apache.hadoop.hdfs.server.namenode.JournalSet.access$200(JournalSet.java:50)
at org.apache.hadoop.hdfs.server.namenode.JournalSet$JournalSetOutputStream.flush(JournalSet.java:456)
at org.apache.hadoop.hdfs.server.namenode.FSEditLog.logSync(FSEditLog.java:550)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2007)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:471)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:292)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:42676)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:427)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:916)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1692)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1688)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1686)
{noformat}
> Add entry fails with MetadataVersionException when last ensemble has morethan one bookie failures
> -------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-337
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-337
> Project: Bookkeeper
> Issue Type: Bug
> Reporter: Brahma Reddy Battula
> Assignee: Rakesh R
>
> Scenario:
> ========
> Start Five BK's
> Write ledger's with ensemble three and quroum size=2
> while write inprogress down two bookies(Bookies should be in ensemble)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira