You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-issues@jackrabbit.apache.org by "Michael Dürig (JIRA)" <ji...@apache.org> on 2016/09/07 12:58:23 UTC
[jira] [Updated] (OAK-4675) SNFE thrown while testing
FileStore.cleanup() running concurrently with writes
[ https://issues.apache.org/jira/browse/OAK-4675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Dürig updated OAK-4675:
-------------------------------
Fix Version/s: 1.0.34
> SNFE thrown while testing FileStore.cleanup() running concurrently with writes
> ------------------------------------------------------------------------------
>
> Key: OAK-4675
> URL: https://issues.apache.org/jira/browse/OAK-4675
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar, segmentmk
> Affects Versions: 1.0.32, 1.4.6, 1.2.18, Segment Tar 0.0.8
> Reporter: Andrei Dulceanu
> Assignee: Andrei Dulceanu
> Labels: cleanup, gc
> Fix For: Segment Tar 0.0.10, 1.0.34
>
> Attachments: OAK-4675-01.patch, OAK-4675-02.patch, OAK-4675-oak-segment-03.patch, test-case.patch
>
>
> {{SegmentNotFoundException}} is thrown from time to time in the following scenario: plenty of concurrent writes (each creating a {{625 bytes}} blob) interrupted by a cleanup.
> Stack trace (including some debugging statements added by me):
> {code:java}
> Pre cleanup readers: []
> Before cleanup readers: [/Users/dulceanu/work/test-repo/data00000a.tar]
> Initial size: 357.4 kB
> After cleanup readers: [/Users/dulceanu/work/test-repo/data00000a.tar]
> After cleanup size: 357.4 kB
> Final size: 361.0 kB
> Exception in thread "pool-5-thread-74" org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Cannot copy record from a generation that has been gc'ed already
> at org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.isOldGeneration(SegmentWriter.java:1207)
> at org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNodeUncached(SegmentWriter.java:1096)
> at org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNode(SegmentWriter.java:1013)
> at org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNodeUncached(SegmentWriter.java:1074)
> at org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNode(SegmentWriter.java:1013)
> at org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNode(SegmentWriter.java:987)
> at org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.access$700(SegmentWriter.java:379)
> at org.apache.jackrabbit.oak.segment.SegmentWriter$8.execute(SegmentWriter.java:337)
> at org.apache.jackrabbit.oak.segment.SegmentBufferWriterPool.execute(SegmentBufferWriterPool.java:105)
> at org.apache.jackrabbit.oak.segment.SegmentWriter.writeNode(SegmentWriter.java:334)
> at org.apache.jackrabbit.oak.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:111)
> at org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.prepare(SegmentNodeStore.java:550)
> at org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.optimisticMerge(SegmentNodeStore.java:571)
> at org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.execute(SegmentNodeStore.java:627)
> at org.apache.jackrabbit.oak.segment.SegmentNodeStore.merge(SegmentNodeStore.java:287)
> at org.apache.jackrabbit.oak.segment.CompactionAndCleanupIT$1.run(CompactionAndCleanupIT.java:961)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment 4fb637cc-5013-4925-ab13-0629c4406481 not found
> at org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:1341)
> at org.apache.jackrabbit.oak.segment.SegmentId.getSegment(SegmentId.java:123)
> at org.apache.jackrabbit.oak.segment.RecordId.getSegment(RecordId.java:94)
> at org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.isOldGeneration(SegmentWriter.java:1199)
> ... 18 more
> Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Invalid segment format. Dumping segment 4fb637cc-5013-4925-ab13-0629c4406481
> 00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000000A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000000B0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000000C0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000000D0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000000E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000000F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000130 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000140 39 37 39 31 31 36 30 38 2D 63 31 63 65 2D 34 62 97911608-c1ce-4b
> 00000150 35 63 2D 61 36 33 37 2D 39 36 61 65 39 34 38 38 5c-a637-96ae9488
> 00000160 61 37 65 38 2E 30 61 62 34 30 36 38 36 00 00 00 a7e8.0ab40686...
> 00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000180 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000190 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000001A0 00 00 00 00 30 30 30 30 34 30 30 00 30 30 30 30 ....0000400.0000
> 000001B0 30 30 30 00 30 30 30 30 30 30 30 00 30 30 30 30 000.0000000.0000
> 000001C0 30 30 30 31 33 30 30 00 31 32 37 35 34 36 30 33 0001300.12754603
> 000001D0 37 32 32 00 30 31 32 33 30 37 00 20 30 00 00 00 722.012307. 0...
> 000001E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000001F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000210 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000220 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000230 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000240 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000260 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000270 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000280 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000290 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000002A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000002B0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> at org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1015)
> at org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.get(CacheLIRS.java:972)
> at org.apache.jackrabbit.oak.cache.CacheLIRS.get(CacheLIRS.java:283)
> at org.apache.jackrabbit.oak.segment.SegmentCache.getSegment(SegmentCache.java:92)
> at org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:1275)
> ... 21 more
> Caused by: java.lang.IllegalStateException: Invalid segment format. Dumping segment 4fb637cc-5013-4925-ab13-0629c4406481
> 00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000000A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000000B0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000000C0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000000D0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000000E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000000F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000130 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000140 39 37 39 31 31 36 30 38 2D 63 31 63 65 2D 34 62 97911608-c1ce-4b
> 00000150 35 63 2D 61 36 33 37 2D 39 36 61 65 39 34 38 38 5c-a637-96ae9488
> 00000160 61 37 65 38 2E 30 61 62 34 30 36 38 36 00 00 00 a7e8.0ab40686...
> 00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000180 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000190 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000001A0 00 00 00 00 30 30 30 30 34 30 30 00 30 30 30 30 ....0000400.0000
> 000001B0 30 30 30 00 30 30 30 30 30 30 30 00 30 30 30 30 000.0000000.0000
> 000001C0 30 30 30 31 33 30 30 00 31 32 37 35 34 36 30 33 0001300.12754603
> 000001D0 37 32 32 00 30 31 32 33 30 37 00 20 30 00 00 00 722.012307. 0...
> 000001E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000001F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000210 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000220 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000230 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000240 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000260 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000270 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000280 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00000290 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000002A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 000002B0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> at com.google.common.base.Preconditions.checkState(Preconditions.java:150)
> at org.apache.jackrabbit.oak.segment.Segment.<init>(Segment.java:185)
> at org.apache.jackrabbit.oak.segment.file.FileStore$15.call(FileStore.java:1292)
> at org.apache.jackrabbit.oak.segment.file.FileStore$15.call(FileStore.java:1)
> at org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1011)
> ... 25 more
> 0
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)