You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-issues@jackrabbit.apache.org by "Stefan Egli (Jira)" <ji...@apache.org> on 2021/08/16 16:18:00 UTC
[jira] [Commented] (OAK-9535) Support recovery of large branch
merge
[ https://issues.apache.org/jira/browse/OAK-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17399825#comment-17399825 ]
Stefan Egli commented on OAK-9535:
----------------------------------
* working on this in [this branch|https://github.com/stefan-egli/jackrabbit-oak/tree/OAK-9535]
* added a [test case that reproduces this|https://github.com/stefan-egli/jackrabbit-oak/commit/3a5fbaf705851cd58c04be7cd5d6649072627cc8]
> Support recovery of large branch merge
> --------------------------------------
>
> Key: OAK-9535
> URL: https://issues.apache.org/jira/browse/OAK-9535
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: documentmk
> Affects Versions: 1.40.0
> Reporter: Stefan Egli
> Assignee: Stefan Egli
> Priority: Major
>
> When branch changes get too large, they are committed and a journal entry is created as a result. This way, journal entries - in terms of their byte size they use up - cannot grow large.
> When a recovery happenes the mechanism is slightly different: The recovery attempts to store a single big journal entry. In case of MongoDB, this can result in the document to be larger than 16MB and below exception.
> The recovery must create multiple journal entries as well to avoid running into the MongoDB document size limit.
> {noformat}
> Payload document size is larger than maximum of 16777216.
> org.bson.BsonMaximumSizeExceededException: Payload document size is larger than maximum of 16777216.
> at com.mongodb.internal.connection.BsonWriterHelper.writePayload(BsonWriterHelper.java:68) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.internal.connection.CommandMessage.encodeMessageBodyWithMetadata(CommandMessage.java:147) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.internal.connection.RequestMessage.encode(RequestMessage.java:138) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.internal.connection.CommandMessage.encode(CommandMessage.java:61) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:248) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:99) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:450) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:72) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:226) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:269) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:131) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.operation.MixedBulkWriteOperation.executeCommand(MixedBulkWriteOperation.java:435) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.operation.MixedBulkWriteOperation.executeBulkWriteBatch(MixedBulkWriteOperation.java:261) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.operation.MixedBulkWriteOperation.access$700(MixedBulkWriteOperation.java:72) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.operation.MixedBulkWriteOperation$1.call(MixedBulkWriteOperation.java:205) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.operation.MixedBulkWriteOperation$1.call(MixedBulkWriteOperation.java:196) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.operation.OperationHelper.withReleasableConnection(OperationHelper.java:501) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:196) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:71) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:211) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.client.internal.MongoCollectionImpl.executeInsertMany(MongoCollectionImpl.java:529) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.client.internal.MongoCollectionImpl.insertMany(MongoCollectionImpl.java:524) [org.mongodb.mongo-java-driver:3.12.7]
> at com.mongodb.client.internal.MongoCollectionImpl.insertMany(MongoCollectionImpl.java:518) [org.mongodb.mongo-java-driver:3.12.7]
> at org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.lambda$create$11(MongoDocumentStore.java:1354) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
> at org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.execute(MongoDocumentStore.java:1902) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
> at org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.create(MongoDocumentStore.java:1352) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
> at org.apache.jackrabbit.oak.plugins.document.util.LeaseCheckDocumentStoreWrapper.create(LeaseCheckDocumentStoreWrapper.java:120) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
> at org.apache.jackrabbit.oak.plugins.document.LastRevRecoveryAgent$3.acquiring(LastRevRecoveryAgent.java:468) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
> at org.apache.jackrabbit.oak.plugins.document.UnsavedModifications.persist(UnsavedModifications.java:166) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
> at org.apache.jackrabbit.oak.plugins.document.LastRevRecoveryAgent.recover(LastRevRecoveryAgent.java:435) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
> at org.apache.jackrabbit.oak.plugins.document.LastRevRecoveryAgent.recover(LastRevRecoveryAgent.java:206) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
> at org.apache.jackrabbit.oak.plugins.document.LastRevRecoveryAgent.recoverCandidates(LastRevRecoveryAgent.java:613) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
> at org.apache.jackrabbit.oak.plugins.document.LastRevRecoveryAgent.recover(LastRevRecoveryAgent.java:172) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
> at org.apache.jackrabbit.oak.plugins.document.LastRevRecoveryAgent.recover(LastRevRecoveryAgent.java:191) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
> at org.apache.jackrabbit.oak.plugins.document.LastRevRecoveryAgent.performRecoveryIfNeeded(LastRevRecoveryAgent.java:674) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
> at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreService$LastRevRecoveryJob.run(DocumentNodeStoreService.java:992) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)