You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-issues@jackrabbit.apache.org by "Michael Dürig (JIRA)" <ji...@apache.org> on 2016/05/13 11:00:17 UTC
[jira] [Commented] (OAK-4099) Lucene index appear to be corrupted
with compaction enabled
[ https://issues.apache.org/jira/browse/OAK-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15282617#comment-15282617 ]
Michael Dürig commented on OAK-4099:
------------------------------------
This is just a heads up that this issue might not be fully solved. On one of my test instance I have seen the same stack trace again:
{code}
13.05.2016 02:23:45.886 *ERROR* [127.0.0.1 [1463106225851] GET /system/sling/monitoring/mbeans/org/apache/jackrabbit/oak/LuceneIndex.tidy.2.json HTTP/1.1] org.apache.jackrabbit.oak.segment.SegmentTracker Segment not found: b0c0bbc2-930b-4
org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment b0c0bbc2-930b-4054-a24b-d33e5ffbd075 not found
at org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:1364)
at org.apache.jackrabbit.oak.segment.SegmentTracker.readSegment(SegmentTracker.java:232)
at org.apache.jackrabbit.oak.segment.SegmentId.getSegment(SegmentId.java:141)
at org.apache.jackrabbit.oak.segment.Record.getSegment(Record.java:79)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.getChildNodeMap(SegmentNodeState.java:89)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.getChildNode(SegmentNodeState.java:376)
at org.apache.jackrabbit.oak.spi.state.ReadOnlyBuilder.getChildNode(ReadOnlyBuilder.java:196)
at org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory.fileLength(OakDirectory.java:143)
at org.apache.lucene.store.FilterDirectory.fileLength(FilterDirectory.java:63)
at org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexMBeanImpl.dirSize(LuceneIndexMBeanImpl.java:409)
at org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexMBeanImpl.access$400(LuceneIndexMBeanImpl.java:70)
at org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexMBeanImpl$IndexStats.<init>(LuceneIndexMBeanImpl.java:365)
at org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexMBeanImpl.getIndexStats(LuceneIndexMBeanImpl.java:92)
{code}
For the implemented fix to take effect I would have expected to find the message {{Refreshed the opened indexes}} logged at info level. But there was nothing.
[~chetanm], as this occurred on an instance that has other problems this is FYI for now unless you can derive something actionable from it.
> Lucene index appear to be corrupted with compaction enabled
> -----------------------------------------------------------
>
> Key: OAK-4099
> URL: https://issues.apache.org/jira/browse/OAK-4099
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
> Reporter: Chetan Mehrotra
> Assignee: Chetan Mehrotra
> Priority: Blocker
> Labels: resilience
> Fix For: 1.6, 1.4.2, 1.2.15
>
> Attachments: OAK-4099-v1.patch
>
>
> While running on SegmentNodStore and online compaction enabled it can happen that access to Lucene index start failing with SegmentNotFoundException
> {noformat}
> Caused by: org.apache.jackrabbit.oak.plugins.segment.SegmentNotFoundException: Segment a949519a-8903-44f9-a17e-b6d83fb32186 not found
> at org.apache.jackrabbit.oak.plugins.segment.file.FileStore.readSegment(FileStore.java:870)
> at org.apache.jackrabbit.oak.plugins.segment.SegmentTracker.getSegment(SegmentTracker.java:136)
> at org.apache.jackrabbit.oak.plugins.segment.SegmentId.getSegment(SegmentId.java:108)
> at org.apache.jackrabbit.oak.plugins.segment.Record.getSegment(Record.java:82)
> at org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.getNewStream(SegmentBlob.java:64)
> at org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory$OakIndexFile.loadBlob(OakDirectory.java:259)
> at org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory$OakIndexFile.readBytes(OakDirectory.java:307)
> at org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory$OakIndexInput.readBytes(OakDirectory.java:404)
> at org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory$OakIndexInput.readByte(OakDirectory.java:411)
> at org.apache.lucene.store.DataInput.readVInt(DataInput.java:108)
> at org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.loadBlock(BlockTreeTermsReader.java:2397)
> at org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum.seekCeil(BlockTreeTermsReader.java:1973)
> at org.apache.lucene.index.FilteredTermsEnum.next(FilteredTermsEnum.java:225)
> at org.apache.lucene.search.TermCollectingRewrite.collectTerms(TermCollectingRewrite.java:78)
> at org.apache.lucene.search.ConstantScoreAutoRewrite.rewrite(ConstantScoreAutoRewrite.java:95)
> at org.apache.lucene.search.MultiTermQuery$ConstantScoreAutoRewrite.rewrite(MultiTermQuery.java:220)
> at org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:288)
> at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:418)
> at org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:636)
> at org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:683)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:378)
> {noformat}
> The above segmentId was mentioned in the compaction log
> {noformat}
> 06.03.2016 02:03:30.706 *INFO* [TarMK flush thread [/app/repository/segmentstore], active since Sun Mar 06 02:03:29 GMT 2016, previous max duration 8218ms] org.apache.jackrabbit.oak.plugins.segment.file.TarReader-GC Cleaned segments from data00233a.tar:
> 37ec786e-a9f7-46eb-a3b5-ce5d4777ea01, f36051fe-d8c4-46d1-ac1d-081946389eb6, fae91ff2-8ca6-4ac1-a8d8-d4bd09b7f6a6, 16d87f09-721b-4155-a9c8-b8ecf471bfc3,
> e641f1a3-b323-44e6-aad0-7b894a1efb69, edc9d141-6c05-42c9-a2a2-d7130fd9c826, b602372c-b17a-448a-a8e9-8bdccc64fb82, acc2f032-07ba-46ed-a9c7-d3a05ab53d7a,
> a7323ed2-b2de-4006-ae51-e4f84165a0e4, cb320c70-5ca9-4ed1-a972-e87a6bba9f9b, f45afd7e-5417-42dd-a2f7-4624f74b6c6e, c66f66ef-cdd0-4327-abc6-bf910cb5768d,
> 7f925a07-ff56-4613-ac8f-272a0e481926, 4ad044ec-3b2d-4c3e-aeb0-d5f5a04bc23e, 82f1c3aa-2e0c-421c-a033-e4ffcb6002c7, 1387655b-f633-4011-a55c-d9580e40929b,
> c50c94fc-2e8b-4904-a37f-0a33cc001312, 7915e9ce-bb9d-4628-ad6f-e7f2844b2399, e7cd013b-a147-426a-af29-fa025058a08a, f16d43b0-2113-4808-aea6-5910102e5c7d,
> ...
> *31edad2e-e14b-463d-a6af-540bac6009f1*,
> ...,
> *a949519a-8903-44f9-a17e-b6d83fb32186*,
> ...
> {noformat}
> *Note that system recovered after a restart so the corruption was transient*
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)