[jira] [Created] (OAK-9535) Support recovery of large branch merge

["Stefan Egli (Jira)" <ji...@apache.org>]

Stefan Egli created OAK-9535:
--------------------------------

             Summary: Support recovery of large branch merge
                 Key: OAK-9535
                 URL: https://issues.apache.org/jira/browse/OAK-9535
             Project: Jackrabbit Oak
          Issue Type: Bug
          Components: documentmk
    Affects Versions: 1.40.0
            Reporter: Stefan Egli
            Assignee: Stefan Egli


When branch changes get too large, they are committed and a journal entry is created as a result. This way, journal entries - in terms of their byte size they use up - cannot grow large.

When a recovery happenes the mechanism is slightly different: The recovery attempts to store a single big journal entry. In case of MongoDB, this can result in the document to be larger than 16MB and below exception.

The recovery must create multiple journal entries as well to avoid running into the MongoDB document size limit.

{noformat}
Payload document size is larger than maximum of 16777216.
org.bson.BsonMaximumSizeExceededException: Payload document size is larger than maximum of 16777216.
	at com.mongodb.internal.connection.BsonWriterHelper.writePayload(BsonWriterHelper.java:68) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.internal.connection.CommandMessage.encodeMessageBodyWithMetadata(CommandMessage.java:147) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.internal.connection.RequestMessage.encode(RequestMessage.java:138) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.internal.connection.CommandMessage.encode(CommandMessage.java:61) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:248) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:99) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:450) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:72) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:226) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:269) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:131) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.operation.MixedBulkWriteOperation.executeCommand(MixedBulkWriteOperation.java:435) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.operation.MixedBulkWriteOperation.executeBulkWriteBatch(MixedBulkWriteOperation.java:261) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.operation.MixedBulkWriteOperation.access$700(MixedBulkWriteOperation.java:72) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.operation.MixedBulkWriteOperation$1.call(MixedBulkWriteOperation.java:205) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.operation.MixedBulkWriteOperation$1.call(MixedBulkWriteOperation.java:196) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.operation.OperationHelper.withReleasableConnection(OperationHelper.java:501) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:196) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:71) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:211) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.client.internal.MongoCollectionImpl.executeInsertMany(MongoCollectionImpl.java:529) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.client.internal.MongoCollectionImpl.insertMany(MongoCollectionImpl.java:524) [org.mongodb.mongo-java-driver:3.12.7]
	at com.mongodb.client.internal.MongoCollectionImpl.insertMany(MongoCollectionImpl.java:518) [org.mongodb.mongo-java-driver:3.12.7]
	at org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.lambda$create$11(MongoDocumentStore.java:1354) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
	at org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.execute(MongoDocumentStore.java:1902) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
	at org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.create(MongoDocumentStore.java:1352) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
	at org.apache.jackrabbit.oak.plugins.document.util.LeaseCheckDocumentStoreWrapper.create(LeaseCheckDocumentStoreWrapper.java:120) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
	at org.apache.jackrabbit.oak.plugins.document.LastRevRecoveryAgent$3.acquiring(LastRevRecoveryAgent.java:468) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
	at org.apache.jackrabbit.oak.plugins.document.UnsavedModifications.persist(UnsavedModifications.java:166) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
	at org.apache.jackrabbit.oak.plugins.document.LastRevRecoveryAgent.recover(LastRevRecoveryAgent.java:435) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
	at org.apache.jackrabbit.oak.plugins.document.LastRevRecoveryAgent.recover(LastRevRecoveryAgent.java:206) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
	at org.apache.jackrabbit.oak.plugins.document.LastRevRecoveryAgent.recoverCandidates(LastRevRecoveryAgent.java:613) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
	at org.apache.jackrabbit.oak.plugins.document.LastRevRecoveryAgent.recover(LastRevRecoveryAgent.java:172) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
	at org.apache.jackrabbit.oak.plugins.document.LastRevRecoveryAgent.recover(LastRevRecoveryAgent.java:191) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
	at org.apache.jackrabbit.oak.plugins.document.LastRevRecoveryAgent.performRecoveryIfNeeded(LastRevRecoveryAgent.java:674) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreService$LastRevRecoveryJob.run(DocumentNodeStoreService.java:992) [org.apache.jackrabbit.oak-store-document:1.39.0.R1888564]
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)