You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "JunHo Cho (JIRA)" <ji...@apache.org> on 2016/03/09 06:31:40 UTC
[jira] [Commented] (HBASE-15430) Failed taking snapshot - Manifest
proto-message too large
[ https://issues.apache.org/jira/browse/HBASE-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15186547#comment-15186547 ]
JunHo Cho commented on HBASE-15430:
-----------------------------------
i think that it is not easy to create very big manifest file (over 64MB). it there any way to make manifest easily? if not, I attached big file for test.
> Failed taking snapshot - Manifest proto-message too large
> ---------------------------------------------------------
>
> Key: HBASE-15430
> URL: https://issues.apache.org/jira/browse/HBASE-15430
> Project: HBase
> Issue Type: Bug
> Components: snapshots
> Affects Versions: 0.98.11
> Reporter: JunHo Cho
> Priority: Critical
> Attachments: hbase-15430.patch
>
>
> the size of a protobuf message is 64MB (default). but the size of snapshot meta is over 64MB.
> Caused by: com.google.protobuf.InvalidProtocolBufferException via Failed taking snapshot { ss=snapshot_xxx table=xxx type=FLUSH } due to exception:Protocol message was too large. May be malicious. Use CodedInputStream.setSizeLimit() to increase the size limit.:com.google.protobuf.InvalidProtocolBufferException: Protocol message was too large. May be malicious. Use CodedInputStream.setSizeLimit() to increase the size limit.
> at org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher.rethrowException(ForeignExceptionDispatcher.java:83)
> at org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.rethrowExceptionIfFailed(TakeSnapshotHandler.java:307)
> at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:341)
> ... 10 more
> Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol message was too large. May be malicious. Use CodedInputStream.setSizeLimit() to increase the size limit.
> at com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
> at com.google.protobuf.CodedInputStream.refillBuffer(CodedInputStream.java:755)
> at com.google.protobuf.CodedInputStream.readRawBytes(CodedInputStream.java:811)
> at com.google.protobuf.CodedInputStream.readBytes(CodedInputStream.java:329)
> at org.apache.hadoop.hbase.protobuf.generated.HBaseProtos$RegionInfo.<init>(HBaseProtos.java:3767)
> at org.apache.hadoop.hbase.protobuf.generated.HBaseProtos$RegionInfo.<init>(HBaseProtos.java:3699)
> at org.apache.hadoop.hbase.protobuf.generated.HBaseProtos$RegionInfo$1.parsePartialFrom(HBaseProtos.java:3815)
> at org.apache.hadoop.hbase.protobuf.generated.HBaseProtos$RegionInfo$1.parsePartialFrom(HBaseProtos.java:3810)
> at com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java:309)
> at org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotRegionManifest.<init>(SnapshotProtos.java:1152)
> at org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotRegionManifest.<init>(SnapshotProtos.java:1094)
> at org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotRegionManifest$1.parsePartialFrom(SnapshotProtos.java:1201)
> at org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotRegionManifest$1.parsePartialFrom(SnapshotProtos.java:1196)
> at com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java:309)
> at org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest.<init>(SnapshotProtos.java:3858)
> at org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest.<init>(SnapshotProtos.java:3792)
> at org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest$1.parsePartialFrom(SnapshotProtos.java:3894)
> at org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest$1.parsePartialFrom(SnapshotProtos.java:3889)
> at com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:200)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:217)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:223)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:49)
> at org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest.parseFrom(SnapshotProtos.java:4094)
> at org.apache.hadoop.hbase.snapshot.SnapshotManifest.readDataManifest(SnapshotManifest.java:433)
> at org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:273)
> at org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119)
> at org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier.verifySnapshot(MasterSnapshotVerifier.java:106
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)