You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-issues@jackrabbit.apache.org by "Michael Dürig (JIRA)" <ji...@apache.org> on 2014/11/26 22:37:12 UTC
[jira] [Comment Edited] (OAK-2294) Corrupt repository after
concurrent version operations
[ https://issues.apache.org/jira/browse/OAK-2294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14226472#comment-14226472 ]
Michael Dürig edited comment on OAK-2294 at 11/26/14 9:36 PM:
--------------------------------------------------------------
The root cause for this issue is a limitation of the SegmentMK that we are hitting here. Apparently the SegmentMK can't handle more than 255 property names in a template when those are spread over separate segments. So this isn't a concurrency issue after all. I was able to reproduce this within a single thread by just adding *many* properties to the same node. The reason this is more expressed in a concurrent setting is that there the chances are higher that the property names get spread across various segments.
The reason we are seeing this issue with the versioning code (while adding labels) is that labels are maintained as reference properties on the version history node.
[~tmueller] your [comment | https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/segment/SegmentWriter.java#L955] in the {{SegmentWriter}} class was a great help in finding this. --Could you summarise here why spreading property names across more than 255 segments doesn't work as you are stating in that comment?-- Found it out myself: this would cause the segment containing the property names to have more than 255 references to other segments, which overflows it header.
was (Author: mduerig):
The root cause for this issue is a limitation of the SegmentMK that we are hitting here. Apparently the SegmentMK can't handle more than 255 property names in a template when those are spread over separate segments. So this isn't a concurrency issue after all. I was able to reproduce this within a single thread by just adding *many* properties to the same node. The reason this is more expressed in a concurrent setting is that there the chances are higher that the property names get spread across various segments.
The reason we are seeing this issue with the versioning code (while adding labels) is that labels are maintained as reference properties on the version history node.
[~tmueller] your [comment | https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/segment/SegmentWriter.java#L955] in the {{SegmentWriter}} class was a great help in finding this. Could you summarise here why spreading property names across more than 255 segments doesn't work as you are stating in that comment?
> Corrupt repository after concurrent version operations
> ------------------------------------------------------
>
> Key: OAK-2294
> URL: https://issues.apache.org/jira/browse/OAK-2294
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segmentmk
> Reporter: Michael Dürig
> Assignee: Michael Dürig
> Priority: Critical
> Attachments: OAK-2294-2.patch, OAK-2294.patch
>
>
> Performing version operations (checkin / checkout / addVersionLabel) concurrently can corrupt the repository.
> Executing the following code in parallel from multiple threads demonstrates this:
> {code}
> Version version = versionManager.checkin(vPath);
> versionManager.checkout(vPath);
> String label = version.getName() + " " + Thread.currentThread().getName();
> version.getContainingHistory()
> .addVersionLabel(version.getName(), label, true);
> {code}
> In my tests this eventually lead to all sorts of exceptions:
> {noformat}
> java.lang.IllegalStateException: RefId '85' doesn't exist in data segment 0c5c0814-902c-429c-ad41-cd82aea276a2
> at org.apache.jackrabbit.oak.plugins.segment.Segment.getRefId(Segment.java:196)
> at org.apache.jackrabbit.oak.plugins.segment.Segment.internalReadRecordId(Segment.java:307)
> at org.apache.jackrabbit.oak.plugins.segment.Segment.readRecordId(Segment.java:303)
> at org.apache.jackrabbit.oak.plugins.segment.MapRecord.getBucketList(MapRecord.java:134)
> at org.apache.jackrabbit.oak.plugins.segment.MapRecord.getEntries(MapRecord.java:347)
> at org.apache.jackrabbit.oak.plugins.segment.MapRecord.getEntries(MapRecord.java:325)
> at org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:474)
> at org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:394)
> at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:544)
> ...
> {noformat}
> {noformat}
> java.lang.IllegalStateException: String is too long: 2159501163930351661
> at org.apache.jackrabbit.oak.plugins.segment.Segment.loadString(Segment.java:352)
> at org.apache.jackrabbit.oak.plugins.segment.Segment.readString(Segment.java:319)
> at org.apache.jackrabbit.oak.plugins.segment.Segment.readString(Segment.java:313)
> at org.apache.jackrabbit.oak.plugins.segment.Segment.loadTemplate(Segment.java:418)
> at org.apache.jackrabbit.oak.plugins.segment.Segment.readTemplate(Segment.java:367)
> at org.apache.jackrabbit.oak.plugins.segment.Segment.readTemplate(Segment.java:361)
> at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.getTemplate(SegmentNodeState.java:78)
> at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:408)
> ...
> {noformat}
> {noformat}
> java.lang.IllegalStateException
> at com.google.common.base.Preconditions.checkState(Preconditions.java:134)
> at org.apache.jackrabbit.oak.plugins.segment.file.TarWriter.writeEntry(TarWriter.java:206)
> at org.apache.jackrabbit.oak.plugins.segment.file.TarWriter.writeEntry(TarWriter.java:200)
> at org.apache.jackrabbit.oak.plugins.segment.file.FileStore.writeSegment(FileStore.java:682)
> at org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.flush(SegmentWriter.java:228)
> at org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.prepare(SegmentWriter.java:329)
> at org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeTemplate(SegmentWriter.java:969)
> at org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1039)
> at org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1062)
> at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:395)
> ...
> {noformat}
> {noformat}
> Caused by: java.lang.IllegalArgumentException: Invalid type tag: 81
> at org.apache.jackrabbit.oak.api.Type.fromTag(Type.java:202)
> at org.apache.jackrabbit.oak.plugins.segment.Segment.loadTemplate(Segment.java:418)
> at org.apache.jackrabbit.oak.plugins.segment.Segment.readTemplate(Segment.java:367)
> at org.apache.jackrabbit.oak.plugins.segment.Segment.readTemplate(Segment.java:361)
> at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.getTemplate(SegmentNodeState.java:78)
> at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.getProperty(SegmentNodeState.java:122)
> ...
> {noformat}
> {noformat}
> Caused by: java.lang.IllegalStateException
> at com.google.common.base.Preconditions.checkState(Preconditions.java:134)
> at org.apache.jackrabbit.oak.plugins.segment.Segment.pos(Segment.java:178)
> at org.apache.jackrabbit.oak.plugins.segment.Segment.loadString(Segment.java:326)
> at org.apache.jackrabbit.oak.plugins.segment.Segment.readString(Segment.java:319)
> at org.apache.jackrabbit.oak.plugins.segment.Segment.readString(Segment.java:313)
> at org.apache.jackrabbit.oak.plugins.segment.SegmentPropertyState.getValue(SegmentPropertyState.java:174)
> at org.apache.jackrabbit.oak.plugins.segment.SegmentPropertyState.getValue(SegmentPropertyState.java:147)
> at org.apache.jackrabbit.oak.plugins.memory.AbstractPropertyState.equal(AbstractPropertyState.java:53)
> ...
> {noformat}
> Will attach a patch with a test case shortly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)