You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by feedly team <fe...@gmail.com> on 2014/05/09 18:14:06 UTC
ConcurrentModificationException in ICU analyzer
I am using the 4.7.0 ICU analyzer (via elastic search) and noticed this
exception in the logs. It's sporadic. Any ideas what is going on or if this
is already fixed:
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:894)
at java.util.HashMap$KeyIterator.next(HashMap.java:928)
at
com.ibm.icu.text.RuleBasedBreakIterator.getEngineFor(RuleBasedBreakIterator.java:1011)
at
com.ibm.icu.text.RuleBasedBreakIterator.handleNext(RuleBasedBreakIterator.java:1085)
at
com.ibm.icu.text.RuleBasedBreakIterator.next(RuleBasedBreakIterator.java:449)
at
org.apache.lucene.analysis.icu.segmentation.BreakIteratorWrapper$RBBIWrapper.next(BreakIteratorWrapper.java:96)
at
org.apache.lucene.analysis.icu.segmentation.CompositeBreakIterator.next(CompositeBreakIterator.java:60)
at
org.apache.lucene.analysis.icu.segmentation.ICUTokenizer.incrementTokenBuffer(ICUTokenizer.java:212)
at
org.apache.lucene.analysis.icu.segmentation.ICUTokenizer.incrementToken(ICUTokenizer.java:106)
at
org.apache.lucene.analysis.icu.ICUNormalizer2Filter.incrementToken(ICUNormalizer2Filter.java:80)
at
org.apache.lucene.analysis.miscellaneous.KeywordMarkerFilter.incrementToken(KeywordMarkerFilter.java:45)
at
org.apache.lucene.analysis.util.FilteringTokenFilter.incrementToken(FilteringTokenFilter.java:82)
at
org.apache.lucene.analysis.en.EnglishPossessiveFilter.incrementToken(EnglishPossessiveFilter.java:57)
at
org.apache.lucene.analysis.util.ElisionFilter.incrementToken(ElisionFilter.java:52)
at
org.apache.lucene.analysis.icu.ICUNormalizer2Filter.incrementToken(ICUNormalizer2Filter.java:80)
at
org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:182)
at
org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcessor.java:248)
at
org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:253)
at
org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:455)
at
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1534)
at
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1204)
at
org.elasticsearch.index.engine.internal.InternalEngine.innerCreate(InternalEngine.java:451)
at
org.elasticsearch.index.engine.internal.InternalEngine.create(InternalEngine.java:382)
at
org.elasticsearch.index.shard.service.InternalIndexShard.create(InternalIndexShard.java:374)
at
org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:431)
at
org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:160)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:556)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:426)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Re: ConcurrentModificationException in ICU analyzer
Posted by Robert Muir <rc...@gmail.com>.
I opened https://issues.apache.org/jira/browse/LUCENE-5671
for now, if you are able to use the latest release of ICU, it should
prevent the bug.
On Wed, May 14, 2014 at 11:47 AM, Robert Muir <rc...@gmail.com> wrote:
> fyi: this bug was already found and fixed in ICU's trunk:
> http://bugs.icu-project.org/trac/ticket/10767
>
>
> On Wed, May 14, 2014 at 4:32 AM, Robert Muir <rc...@gmail.com> wrote:
>> This looks like a bug in ICU? I'll try to reproduce it. We are also a
>> little out of date, maybe they've already fixed it.
>>
>> Thank you for reporting this.
>>
>> On Fri, May 9, 2014 at 12:14 PM, feedly team <fe...@gmail.com> wrote:
>>> I am using the 4.7.0 ICU analyzer (via elastic search) and noticed this
>>> exception in the logs. It's sporadic. Any ideas what is going on or if this
>>> is already fixed:
>>>
>>> java.util.ConcurrentModificationException
>>>
>>> at java.util.HashMap$HashIterator.nextEntry(HashMap.java:894)
>>>
>>> at java.util.HashMap$KeyIterator.next(HashMap.java:928)
>>>
>>> at
>>> com.ibm.icu.text.RuleBasedBreakIterator.getEngineFor(RuleBasedBreakIterator.java:1011)
>>>
>>> at
>>> com.ibm.icu.text.RuleBasedBreakIterator.handleNext(RuleBasedBreakIterator.java:1085)
>>>
>>> at
>>> com.ibm.icu.text.RuleBasedBreakIterator.next(RuleBasedBreakIterator.java:449)
>>>
>>> at
>>> org.apache.lucene.analysis.icu.segmentation.BreakIteratorWrapper$RBBIWrapper.next(BreakIteratorWrapper.java:96)
>>>
>>> at
>>> org.apache.lucene.analysis.icu.segmentation.CompositeBreakIterator.next(CompositeBreakIterator.java:60)
>>>
>>> at
>>> org.apache.lucene.analysis.icu.segmentation.ICUTokenizer.incrementTokenBuffer(ICUTokenizer.java:212)
>>>
>>> at
>>> org.apache.lucene.analysis.icu.segmentation.ICUTokenizer.incrementToken(ICUTokenizer.java:106)
>>>
>>> at
>>> org.apache.lucene.analysis.icu.ICUNormalizer2Filter.incrementToken(ICUNormalizer2Filter.java:80)
>>>
>>> at
>>> org.apache.lucene.analysis.miscellaneous.KeywordMarkerFilter.incrementToken(KeywordMarkerFilter.java:45)
>>>
>>> at
>>> org.apache.lucene.analysis.util.FilteringTokenFilter.incrementToken(FilteringTokenFilter.java:82)
>>>
>>> at
>>> org.apache.lucene.analysis.en.EnglishPossessiveFilter.incrementToken(EnglishPossessiveFilter.java:57)
>>>
>>> at
>>> org.apache.lucene.analysis.util.ElisionFilter.incrementToken(ElisionFilter.java:52)
>>>
>>> at
>>> org.apache.lucene.analysis.icu.ICUNormalizer2Filter.incrementToken(ICUNormalizer2Filter.java:80)
>>>
>>> at
>>> org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:182)
>>>
>>> at
>>> org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcessor.java:248)
>>>
>>> at
>>> org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:253)
>>>
>>> at
>>> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:455)
>>>
>>> at
>>> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1534)
>>>
>>> at
>>> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1204)
>>>
>>> at
>>> org.elasticsearch.index.engine.internal.InternalEngine.innerCreate(InternalEngine.java:451)
>>>
>>> at
>>> org.elasticsearch.index.engine.internal.InternalEngine.create(InternalEngine.java:382)
>>>
>>> at
>>> org.elasticsearch.index.shard.service.InternalIndexShard.create(InternalIndexShard.java:374)
>>>
>>> at
>>> org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:431)
>>>
>>> at
>>> org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:160)
>>>
>>> at
>>> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:556)
>>>
>>> at
>>> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:426)
>>>
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>
>>> at java.lang.Thread.run(Thread.java:722)
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: ConcurrentModificationException in ICU analyzer
Posted by Robert Muir <rc...@gmail.com>.
fyi: this bug was already found and fixed in ICU's trunk:
http://bugs.icu-project.org/trac/ticket/10767
On Wed, May 14, 2014 at 4:32 AM, Robert Muir <rc...@gmail.com> wrote:
> This looks like a bug in ICU? I'll try to reproduce it. We are also a
> little out of date, maybe they've already fixed it.
>
> Thank you for reporting this.
>
> On Fri, May 9, 2014 at 12:14 PM, feedly team <fe...@gmail.com> wrote:
>> I am using the 4.7.0 ICU analyzer (via elastic search) and noticed this
>> exception in the logs. It's sporadic. Any ideas what is going on or if this
>> is already fixed:
>>
>> java.util.ConcurrentModificationException
>>
>> at java.util.HashMap$HashIterator.nextEntry(HashMap.java:894)
>>
>> at java.util.HashMap$KeyIterator.next(HashMap.java:928)
>>
>> at
>> com.ibm.icu.text.RuleBasedBreakIterator.getEngineFor(RuleBasedBreakIterator.java:1011)
>>
>> at
>> com.ibm.icu.text.RuleBasedBreakIterator.handleNext(RuleBasedBreakIterator.java:1085)
>>
>> at
>> com.ibm.icu.text.RuleBasedBreakIterator.next(RuleBasedBreakIterator.java:449)
>>
>> at
>> org.apache.lucene.analysis.icu.segmentation.BreakIteratorWrapper$RBBIWrapper.next(BreakIteratorWrapper.java:96)
>>
>> at
>> org.apache.lucene.analysis.icu.segmentation.CompositeBreakIterator.next(CompositeBreakIterator.java:60)
>>
>> at
>> org.apache.lucene.analysis.icu.segmentation.ICUTokenizer.incrementTokenBuffer(ICUTokenizer.java:212)
>>
>> at
>> org.apache.lucene.analysis.icu.segmentation.ICUTokenizer.incrementToken(ICUTokenizer.java:106)
>>
>> at
>> org.apache.lucene.analysis.icu.ICUNormalizer2Filter.incrementToken(ICUNormalizer2Filter.java:80)
>>
>> at
>> org.apache.lucene.analysis.miscellaneous.KeywordMarkerFilter.incrementToken(KeywordMarkerFilter.java:45)
>>
>> at
>> org.apache.lucene.analysis.util.FilteringTokenFilter.incrementToken(FilteringTokenFilter.java:82)
>>
>> at
>> org.apache.lucene.analysis.en.EnglishPossessiveFilter.incrementToken(EnglishPossessiveFilter.java:57)
>>
>> at
>> org.apache.lucene.analysis.util.ElisionFilter.incrementToken(ElisionFilter.java:52)
>>
>> at
>> org.apache.lucene.analysis.icu.ICUNormalizer2Filter.incrementToken(ICUNormalizer2Filter.java:80)
>>
>> at
>> org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:182)
>>
>> at
>> org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcessor.java:248)
>>
>> at
>> org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:253)
>>
>> at
>> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:455)
>>
>> at
>> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1534)
>>
>> at
>> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1204)
>>
>> at
>> org.elasticsearch.index.engine.internal.InternalEngine.innerCreate(InternalEngine.java:451)
>>
>> at
>> org.elasticsearch.index.engine.internal.InternalEngine.create(InternalEngine.java:382)
>>
>> at
>> org.elasticsearch.index.shard.service.InternalIndexShard.create(InternalIndexShard.java:374)
>>
>> at
>> org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:431)
>>
>> at
>> org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:160)
>>
>> at
>> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:556)
>>
>> at
>> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:426)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>
>> at java.lang.Thread.run(Thread.java:722)
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: ConcurrentModificationException in ICU analyzer
Posted by Robert Muir <rc...@gmail.com>.
This looks like a bug in ICU? I'll try to reproduce it. We are also a
little out of date, maybe they've already fixed it.
Thank you for reporting this.
On Fri, May 9, 2014 at 12:14 PM, feedly team <fe...@gmail.com> wrote:
> I am using the 4.7.0 ICU analyzer (via elastic search) and noticed this
> exception in the logs. It's sporadic. Any ideas what is going on or if this
> is already fixed:
>
> java.util.ConcurrentModificationException
>
> at java.util.HashMap$HashIterator.nextEntry(HashMap.java:894)
>
> at java.util.HashMap$KeyIterator.next(HashMap.java:928)
>
> at
> com.ibm.icu.text.RuleBasedBreakIterator.getEngineFor(RuleBasedBreakIterator.java:1011)
>
> at
> com.ibm.icu.text.RuleBasedBreakIterator.handleNext(RuleBasedBreakIterator.java:1085)
>
> at
> com.ibm.icu.text.RuleBasedBreakIterator.next(RuleBasedBreakIterator.java:449)
>
> at
> org.apache.lucene.analysis.icu.segmentation.BreakIteratorWrapper$RBBIWrapper.next(BreakIteratorWrapper.java:96)
>
> at
> org.apache.lucene.analysis.icu.segmentation.CompositeBreakIterator.next(CompositeBreakIterator.java:60)
>
> at
> org.apache.lucene.analysis.icu.segmentation.ICUTokenizer.incrementTokenBuffer(ICUTokenizer.java:212)
>
> at
> org.apache.lucene.analysis.icu.segmentation.ICUTokenizer.incrementToken(ICUTokenizer.java:106)
>
> at
> org.apache.lucene.analysis.icu.ICUNormalizer2Filter.incrementToken(ICUNormalizer2Filter.java:80)
>
> at
> org.apache.lucene.analysis.miscellaneous.KeywordMarkerFilter.incrementToken(KeywordMarkerFilter.java:45)
>
> at
> org.apache.lucene.analysis.util.FilteringTokenFilter.incrementToken(FilteringTokenFilter.java:82)
>
> at
> org.apache.lucene.analysis.en.EnglishPossessiveFilter.incrementToken(EnglishPossessiveFilter.java:57)
>
> at
> org.apache.lucene.analysis.util.ElisionFilter.incrementToken(ElisionFilter.java:52)
>
> at
> org.apache.lucene.analysis.icu.ICUNormalizer2Filter.incrementToken(ICUNormalizer2Filter.java:80)
>
> at
> org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:182)
>
> at
> org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcessor.java:248)
>
> at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:253)
>
> at
> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:455)
>
> at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1534)
>
> at
> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1204)
>
> at
> org.elasticsearch.index.engine.internal.InternalEngine.innerCreate(InternalEngine.java:451)
>
> at
> org.elasticsearch.index.engine.internal.InternalEngine.create(InternalEngine.java:382)
>
> at
> org.elasticsearch.index.shard.service.InternalIndexShard.create(InternalIndexShard.java:374)
>
> at
> org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:431)
>
> at
> org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:160)
>
> at
> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:556)
>
> at
> org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:426)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:722)
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org