You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Bernd Fehling <be...@uni-bielefeld.de> on 2016/08/04 11:03:30 UTC
no concurrent merging?
While increasing the indexing load of version 5.5.3 I see
threads where one merging thread is blocking other merging threads.
But is this concurrent merging?
Bernd
"Lucene Merge Thread #6" - Thread t@40280 java.lang.Thread.State: BLOCKED
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4008)
- waiting to lock <6d75db> (a org.apache.solr.update.SolrIndexWriter) owned by "Lucene Merge Thread #8" t@53896
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
Locked ownable synchronizers: - None
"Lucene Merge Thread #7" - Thread t@40281 java.lang.Thread.State: BLOCKED
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4166)
- waiting to lock <6d75db> (a org.apache.solr.update.SolrIndexWriter) owned by "Lucene Merge Thread #8" t@53896
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
Locked ownable synchronizers: - None
"Lucene Merge Thread #8" - Thread t@53896 java.lang.Thread.State: RUNNABLE
at java.lang.System.identityHashCode(Native Method)
at org.apache.lucene.index.IndexReader.hashCode(IndexReader.java:302)
at java.util.WeakHashMap.hash(WeakHashMap.java:298)
at java.util.WeakHashMap.put(WeakHashMap.java:449)
at java.util.Collections$SetFromMap.add(Collections.java:5461)
at java.util.Collections$SynchronizedCollection.add(Collections.java:2035)
- locked <4c8b5399> (a java.util.Collections$SynchronizedSet)
at org.apache.lucene.index.IndexReader.registerParentReader(IndexReader.java:138)
at org.apache.lucene.index.FilterLeafReader.<init>(FilterLeafReader.java:306)
at org.apache.lucene.uninverting.UninvertingReader.<init>(UninvertingReader.java:184)
at org.apache.solr.update.DeleteByQueryWrapper.wrap(DeleteByQueryWrapper.java:52)
at org.apache.solr.update.DeleteByQueryWrapper.createWeight(DeleteByQueryWrapper.java:72)
at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:904)
at org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:887)
at org.apache.lucene.index.BufferedUpdatesStream.applyQueryDeletes(BufferedUpdatesStream.java:713)
at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:246)
- locked <9f8d81c> (a org.apache.lucene.index.BufferedUpdatesStream)
at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:3834)
- locked <6d75db> (a org.apache.solr.update.SolrIndexWriter)
at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:3792)
- locked <6d75db> (a org.apache.solr.update.SolrIndexWriter)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3646)
at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: no concurrent merging?
Posted by Michael McCandless <lu...@mikemccandless.com>.
On Tue, Aug 9, 2016 at 7:14 PM, Yonik Seeley <ys...@gmail.com> wrote:
But other than perhaps changing how long a DBQ takes to execute, it
> should be unrelated to the question of if other merges can proceed in
> parallel.
>
> A quick look at the lucene IndexWriter code says, no... Lucene DBQ
> processing cannot proceed in parallel.
> IndexWriter.mergeInit is synchronized (on IW). The DBQ processing is
> called from there and thus anything else that needs the IW monitor
> will block.
Yeah, the impl Lucene has today is fully sync'd on IW while deletes are
being resolved (from Query or Term to docIDs).
I don't think there's a fundamental reason that it has to be, i.e. "it's
just software" and we could fix this (patches welcome!), but it's hairy.
I do think we would need a lock on each segment that's resolving its
deletes, or maybe even that could be made concurrent e.g. where a thread
can pull out one Query at a time and resolve it.
In any case, fixing it will likely be complex!
Mike McCandless
http://blog.mikemccandless.com
Re: no concurrent merging?
Posted by Yonik Seeley <ys...@gmail.com>.
On Thu, Aug 4, 2016 at 9:35 AM, Michael McCandless
<lu...@mikemccandless.com> wrote:
> Lucene's merging is concurrent, but Solr unfortunately uses
> UninvertingReader on each DBQ ... I'm not sure why.
It looks like DeleteByQueryWrapper was added by
https://issues.apache.org/jira/browse/LUCENE-5666
But other than perhaps changing how long a DBQ takes to execute, it
should be unrelated to the question of if other merges can proceed in
parallel.
A quick look at the lucene IndexWriter code says, no... Lucene DBQ
processing cannot proceed in parallel.
IndexWriter.mergeInit is synchronized (on IW). The DBQ processing is
called from there and thus anything else that needs the IW monitor
will block.
-Yonik
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: no concurrent merging?
Posted by Bernd Fehling <be...@uni-bielefeld.de>.
Yes, excactly, that's it.
But is it a Lucene or a Solr problem?
Should Solr use a different reader from DBQ or can Lucene
do something to solve this because it is reported as a
Lucene issue?
Regards
Bernd
Am 04.08.2016 um 16:02 schrieb Mikhail Khludnev:
> Hello,
> There is https://issues.apache.org/jira/browse/LUCENE-7049
>
>
> On Thu, Aug 4, 2016 at 4:35 PM, Michael McCandless <
> lucene@mikemccandless.com> wrote:
>
>> Lucene's merging is concurrent, but Solr unfortunately uses
>> UninvertingReader on each DBQ ... I'm not sure why. I think you should ask
>> on the solr-user list?
>>
>> Or maybe try to change your deletes to be by Term instead of Query?
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>> On Thu, Aug 4, 2016 at 7:03 AM, Bernd Fehling <
>> bernd.fehling@uni-bielefeld.de> wrote:
>>
>>> While increasing the indexing load of version 5.5.3 I see
>>> threads where one merging thread is blocking other merging threads.
>>> But is this concurrent merging?
>>>
>>> Bernd
>>>
>>> "Lucene Merge Thread #6" - Thread t@40280 java.lang.Thread.State:
>>> BLOCKED
>>> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4008)
>>> - waiting to lock <6d75db> (a org.apache.solr.update.SolrIndexWriter)
>>> owned by "Lucene Merge Thread #8" t@53896
>>> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
>>> at
>>>
>> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
>>> at
>>>
>> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
>>> Locked ownable synchronizers: - None
>>>
>>> "Lucene Merge Thread #7" - Thread t@40281 java.lang.Thread.State:
>>> BLOCKED
>>> at
>> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4166)
>>> - waiting to lock <6d75db> (a org.apache.solr.update.SolrIndexWriter)
>>> owned by "Lucene Merge Thread #8" t@53896
>>> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
>>> at
>>>
>> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
>>> at
>>>
>> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
>>> Locked ownable synchronizers: - None
>>>
>>> "Lucene Merge Thread #8" - Thread t@53896 java.lang.Thread.State:
>>> RUNNABLE
>>> at java.lang.System.identityHashCode(Native Method)
>>> at org.apache.lucene.index.IndexReader.hashCode(IndexReader.java:302)
>>> at java.util.WeakHashMap.hash(WeakHashMap.java:298)
>>> at java.util.WeakHashMap.put(WeakHashMap.java:449)
>>> at java.util.Collections$SetFromMap.add(Collections.java:5461)
>>> at
>> java.util.Collections$SynchronizedCollection.add(Collections.java:2035)
>>> - locked <4c8b5399> (a java.util.Collections$SynchronizedSet)
>>> at
>>>
>> org.apache.lucene.index.IndexReader.registerParentReader(IndexReader.java:138)
>>> at
>>>
>> org.apache.lucene.index.FilterLeafReader.<init>(FilterLeafReader.java:306)
>>> at
>>>
>> org.apache.lucene.uninverting.UninvertingReader.<init>(UninvertingReader.java:184)
>>> at
>>>
>> org.apache.solr.update.DeleteByQueryWrapper.wrap(DeleteByQueryWrapper.java:52)
>>> at
>>>
>> org.apache.solr.update.DeleteByQueryWrapper.createWeight(DeleteByQueryWrapper.java:72)
>>> at
>>>
>> org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:904)
>>> at
>>>
>> org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:887)
>>> at
>>>
>> org.apache.lucene.index.BufferedUpdatesStream.applyQueryDeletes(BufferedUpdatesStream.java:713)
>>> at
>>>
>> org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:246)
>>> - locked <9f8d81c> (a org.apache.lucene.index.BufferedUpdatesStream)
>>> at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:3834)
>>> - locked <6d75db> (a org.apache.solr.update.SolrIndexWriter)
>>> at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:3792)
>>> - locked <6d75db> (a org.apache.solr.update.SolrIndexWriter)
>>> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3646)
>>> at
>>>
>> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
>>> at
>>>
>> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>
>
>
--
*************************************************************
Bernd Fehling Bielefeld University Library
Dipl.-Inform. (FH) LibTec - Library Technology
Universit�tsstr. 25 and Knowledge Management
33615 Bielefeld
Tel. +49 521 106-4060 bernd.fehling(at)uni-bielefeld.de
BASE - Bielefeld Academic Search Engine - www.base-search.net
*************************************************************
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: no concurrent merging?
Posted by Mikhail Khludnev <mk...@apache.org>.
Hello,
There is https://issues.apache.org/jira/browse/LUCENE-7049
On Thu, Aug 4, 2016 at 4:35 PM, Michael McCandless <
lucene@mikemccandless.com> wrote:
> Lucene's merging is concurrent, but Solr unfortunately uses
> UninvertingReader on each DBQ ... I'm not sure why. I think you should ask
> on the solr-user list?
>
> Or maybe try to change your deletes to be by Term instead of Query?
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Thu, Aug 4, 2016 at 7:03 AM, Bernd Fehling <
> bernd.fehling@uni-bielefeld.de> wrote:
>
> > While increasing the indexing load of version 5.5.3 I see
> > threads where one merging thread is blocking other merging threads.
> > But is this concurrent merging?
> >
> > Bernd
> >
> > "Lucene Merge Thread #6" - Thread t@40280 java.lang.Thread.State:
> > BLOCKED
> > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4008)
> > - waiting to lock <6d75db> (a org.apache.solr.update.SolrIndexWriter)
> > owned by "Lucene Merge Thread #8" t@53896
> > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
> > at
> >
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
> > at
> >
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
> > Locked ownable synchronizers: - None
> >
> > "Lucene Merge Thread #7" - Thread t@40281 java.lang.Thread.State:
> > BLOCKED
> > at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4166)
> > - waiting to lock <6d75db> (a org.apache.solr.update.SolrIndexWriter)
> > owned by "Lucene Merge Thread #8" t@53896
> > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
> > at
> >
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
> > at
> >
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
> > Locked ownable synchronizers: - None
> >
> > "Lucene Merge Thread #8" - Thread t@53896 java.lang.Thread.State:
> > RUNNABLE
> > at java.lang.System.identityHashCode(Native Method)
> > at org.apache.lucene.index.IndexReader.hashCode(IndexReader.java:302)
> > at java.util.WeakHashMap.hash(WeakHashMap.java:298)
> > at java.util.WeakHashMap.put(WeakHashMap.java:449)
> > at java.util.Collections$SetFromMap.add(Collections.java:5461)
> > at
> java.util.Collections$SynchronizedCollection.add(Collections.java:2035)
> > - locked <4c8b5399> (a java.util.Collections$SynchronizedSet)
> > at
> >
> org.apache.lucene.index.IndexReader.registerParentReader(IndexReader.java:138)
> > at
> >
> org.apache.lucene.index.FilterLeafReader.<init>(FilterLeafReader.java:306)
> > at
> >
> org.apache.lucene.uninverting.UninvertingReader.<init>(UninvertingReader.java:184)
> > at
> >
> org.apache.solr.update.DeleteByQueryWrapper.wrap(DeleteByQueryWrapper.java:52)
> > at
> >
> org.apache.solr.update.DeleteByQueryWrapper.createWeight(DeleteByQueryWrapper.java:72)
> > at
> >
> org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:904)
> > at
> >
> org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:887)
> > at
> >
> org.apache.lucene.index.BufferedUpdatesStream.applyQueryDeletes(BufferedUpdatesStream.java:713)
> > at
> >
> org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:246)
> > - locked <9f8d81c> (a org.apache.lucene.index.BufferedUpdatesStream)
> > at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:3834)
> > - locked <6d75db> (a org.apache.solr.update.SolrIndexWriter)
> > at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:3792)
> > - locked <6d75db> (a org.apache.solr.update.SolrIndexWriter)
> > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3646)
> > at
> >
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
> > at
> >
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>
--
Sincerely yours
Mikhail Khludnev
Re: no concurrent merging?
Posted by Michael McCandless <lu...@mikemccandless.com>.
Lucene's merging is concurrent, but Solr unfortunately uses
UninvertingReader on each DBQ ... I'm not sure why. I think you should ask
on the solr-user list?
Or maybe try to change your deletes to be by Term instead of Query?
Mike McCandless
http://blog.mikemccandless.com
On Thu, Aug 4, 2016 at 7:03 AM, Bernd Fehling <
bernd.fehling@uni-bielefeld.de> wrote:
> While increasing the indexing load of version 5.5.3 I see
> threads where one merging thread is blocking other merging threads.
> But is this concurrent merging?
>
> Bernd
>
> "Lucene Merge Thread #6" - Thread t@40280 java.lang.Thread.State:
> BLOCKED
> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4008)
> - waiting to lock <6d75db> (a org.apache.solr.update.SolrIndexWriter)
> owned by "Lucene Merge Thread #8" t@53896
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
> Locked ownable synchronizers: - None
>
> "Lucene Merge Thread #7" - Thread t@40281 java.lang.Thread.State:
> BLOCKED
> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4166)
> - waiting to lock <6d75db> (a org.apache.solr.update.SolrIndexWriter)
> owned by "Lucene Merge Thread #8" t@53896
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
> Locked ownable synchronizers: - None
>
> "Lucene Merge Thread #8" - Thread t@53896 java.lang.Thread.State:
> RUNNABLE
> at java.lang.System.identityHashCode(Native Method)
> at org.apache.lucene.index.IndexReader.hashCode(IndexReader.java:302)
> at java.util.WeakHashMap.hash(WeakHashMap.java:298)
> at java.util.WeakHashMap.put(WeakHashMap.java:449)
> at java.util.Collections$SetFromMap.add(Collections.java:5461)
> at java.util.Collections$SynchronizedCollection.add(Collections.java:2035)
> - locked <4c8b5399> (a java.util.Collections$SynchronizedSet)
> at
> org.apache.lucene.index.IndexReader.registerParentReader(IndexReader.java:138)
> at
> org.apache.lucene.index.FilterLeafReader.<init>(FilterLeafReader.java:306)
> at
> org.apache.lucene.uninverting.UninvertingReader.<init>(UninvertingReader.java:184)
> at
> org.apache.solr.update.DeleteByQueryWrapper.wrap(DeleteByQueryWrapper.java:52)
> at
> org.apache.solr.update.DeleteByQueryWrapper.createWeight(DeleteByQueryWrapper.java:72)
> at
> org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:904)
> at
> org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:887)
> at
> org.apache.lucene.index.BufferedUpdatesStream.applyQueryDeletes(BufferedUpdatesStream.java:713)
> at
> org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:246)
> - locked <9f8d81c> (a org.apache.lucene.index.BufferedUpdatesStream)
> at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:3834)
> - locked <6d75db> (a org.apache.solr.update.SolrIndexWriter)
> at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:3792)
> - locked <6d75db> (a org.apache.solr.update.SolrIndexWriter)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3646)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>