You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Matthew Runo <mr...@zappos.com> on 2008/04/30 23:39:25 UTC
Multiple open SegmentReaders?
Hello!
In using the SVN head version of Solr, I've found that recently we
started getting multiple open SegmentReaders, all registered... etc..
Any ideas why this would happen? They don't go away unless the server
is restarted, and don't go away with commits, etc. In fact, commits
seem to cause the issue. They're causing issues since it causes really
stale searchers to be around...
For example, right now...
org.apache.solr.search.SolrIndexSearcher
caching : true
numDocs : 153312
maxDoc : 153324
readerImpl : SegmentReader
readerDir : org.apache.lucene.store.FSDirectory@/opt/solr/data/index
indexVersion : 1205944085143
openedAt : Wed Apr 30 14:04:15 PDT 2008
registeredAt : Wed Apr 30 14:04:15 PDT 2008
(and right below that one...)
org.apache.solr.search.SolrIndexSearcher
caching : true
numDocs : 153312
maxDoc : 153324
readerImpl : SegmentReader
readerDir : org.apache.lucene.store.FSDirectory@/opt/solr/data/index
indexVersion : 1205944085143
openedAt : Wed Apr 30 14:30:02 PDT 2008
registeredAt : Wed Apr 30 14:30:02 PDT 2008
Thanks!
Matthew Runo
Software Developer
Zappos.com
702.943.7833
Re: Multiple open SegmentReaders?
Posted by Yonik Seeley <yo...@apache.org>.
This bug was introduced in SOLR-509 (committed April 17th).
I'm working on a fix now.
-Yonik
On Fri, May 2, 2008 at 2:32 PM, Yonik Seeley <yo...@apache.org> wrote:
> On Fri, May 2, 2008 at 1:08 PM, Matthew Runo <mr...@zappos.com> wrote:
> > Hah, thank you for doing this. Sometimes I see MultiSegmentReaders,
> > sometimes SegmentReaders, so both show up from time to time. Right now we've
> > got two MultiSegmentReaders open..
>
> OK, this implies there's a leak and the initial searcher that is
> opened never gets closed.
> Could you open a JIRA issue for this?
>
> -Yonik
>
>
>
>
> >
> > Thanks!
> >
> > Matthew Runo
> > Software Developer
> > Zappos.com
> > 702.943.7833
> >
> >
> > On May 1, 2008, at 7:19 PM, Koji Sekiguchi wrote:
> >
> > > I can reproduce with solr/example setup.
> > > What I did:
> > >
> > > 1. $ svn co http://svn.apache.org/repos/asf/lucene/solr/trunk TEMP
> > > 2. $ cd TEMP
> > > 3. $ ant clean example
> > > 4. $ cd example
> > > 5. $ java -jar start.jar
> > >
> > > (to post commit)
> > > 6. $ cd $SOLR_HOME/example/exampledocs
> > > 7. $ ./post.sh
> > >
> > > then see admin>statistics. I can see MultiSegmentReader instead of
> > > SegmentReader, though.
> > >
> > > name: Searcher@19b1de main class:
> > org.apache.solr.search.SolrIndexSearcher version: 1.0 description:
> > index searcher stats: caching : true
> > > numDocs : 0
> > > maxDoc : 0
> > > readerImpl : MultiSegmentReader
> > > readerDir :
> > org.apache.lucene.store.FSDirectory@C:\Project\jakarta\lucene\solr\TEMP\example\solr\data\index
> > > indexVersion : 1209693930226
> > > openedAt : Fri May 02 11:05:30 JST 2008
> > > registeredAt : Fri May 02 11:05:30 JST 2008
> > > name: Searcher@7be8c2 main class:
> > org.apache.solr.search.SolrIndexSearcher version: 1.0 description:
> > index searcher stats: caching : true
> > > numDocs : 0
> > > maxDoc : 0
> > > readerImpl : MultiSegmentReader
> > > readerDir :
> > org.apache.lucene.store.FSDirectory@C:\Project\jakarta\lucene\solr\TEMP\example\solr\data\index
> > > indexVersion : 1209693930226
> > > openedAt : Fri May 02 11:06:13 JST 2008
> > > registeredAt : Fri May 02 11:06:13 JST 2008
> > >
> > > Koji
> > >
> > >
> > > Yonik Seeley wrote:
> > >
> > > > Hmmm, if there is a bug, odds are it's due to multicore stuff -
> > > > probably nothing else has touched core stuff like that recently.
> > > > Can you reproduce (or rather help others to reproduce) with the
> > > > solr/example setup?
> > > >
> > > > -Yonik
> > > >
> > > > On Wed, Apr 30, 2008 at 5:39 PM, Matthew Runo <mr...@zappos.com> wrote:
> > > >
> > > >
> > > > > Hello!
> > > > >
> > > > > In using the SVN head version of Solr, I've found that recently we
> > started
> > > > > getting multiple open SegmentReaders, all registered... etc..
> > > > >
> > > > > Any ideas why this would happen? They don't go away unless the server
> > is
> > > > > restarted, and don't go away with commits, etc. In fact, commits seem
> > to
> > > > > cause the issue. They're causing issues since it causes really stale
> > > > > searchers to be around...
> > > > >
> > > > > For example, right now...
> > > > > org.apache.solr.search.SolrIndexSearcher
> > > > > caching : true
> > > > > numDocs : 153312
> > > > > maxDoc : 153324
> > > > > readerImpl : SegmentReader
> > > > > readerDir : org.apache.lucene.store.FSDirectory@/opt/solr/data/index
> > > > > indexVersion : 1205944085143
> > > > > openedAt : Wed Apr 30 14:04:15 PDT 2008
> > > > > registeredAt : Wed Apr 30 14:04:15 PDT 2008
> > > > >
> > > > > (and right below that one...)
> > > > > org.apache.solr.search.SolrIndexSearcher
> > > > > caching : true
> > > > > numDocs : 153312
> > > > > maxDoc : 153324
> > > > > readerImpl : SegmentReader
> > > > > readerDir : org.apache.lucene.store.FSDirectory@/opt/solr/data/index
> > > > > indexVersion : 1205944085143
> > > > > openedAt : Wed Apr 30 14:30:02 PDT 2008
> > > > > registeredAt : Wed Apr 30 14:30:02 PDT 2008
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Matthew Runo
> > > > > Software Developer
> > > > > Zappos.com
> > > > > 702.943.7833
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
Re: Multiple open SegmentReaders?
Posted by Yonik Seeley <yo...@apache.org>.
On Fri, May 2, 2008 at 1:08 PM, Matthew Runo <mr...@zappos.com> wrote:
> Hah, thank you for doing this. Sometimes I see MultiSegmentReaders,
> sometimes SegmentReaders, so both show up from time to time. Right now we've
> got two MultiSegmentReaders open..
OK, this implies there's a leak and the initial searcher that is
opened never gets closed.
Could you open a JIRA issue for this?
-Yonik
>
> Thanks!
>
> Matthew Runo
> Software Developer
> Zappos.com
> 702.943.7833
>
>
> On May 1, 2008, at 7:19 PM, Koji Sekiguchi wrote:
>
> > I can reproduce with solr/example setup.
> > What I did:
> >
> > 1. $ svn co http://svn.apache.org/repos/asf/lucene/solr/trunk TEMP
> > 2. $ cd TEMP
> > 3. $ ant clean example
> > 4. $ cd example
> > 5. $ java -jar start.jar
> >
> > (to post commit)
> > 6. $ cd $SOLR_HOME/example/exampledocs
> > 7. $ ./post.sh
> >
> > then see admin>statistics. I can see MultiSegmentReader instead of
> > SegmentReader, though.
> >
> > name: Searcher@19b1de main class:
> org.apache.solr.search.SolrIndexSearcher version: 1.0 description:
> index searcher stats: caching : true
> > numDocs : 0
> > maxDoc : 0
> > readerImpl : MultiSegmentReader
> > readerDir :
> org.apache.lucene.store.FSDirectory@C:\Project\jakarta\lucene\solr\TEMP\example\solr\data\index
> > indexVersion : 1209693930226
> > openedAt : Fri May 02 11:05:30 JST 2008
> > registeredAt : Fri May 02 11:05:30 JST 2008
> > name: Searcher@7be8c2 main class:
> org.apache.solr.search.SolrIndexSearcher version: 1.0 description:
> index searcher stats: caching : true
> > numDocs : 0
> > maxDoc : 0
> > readerImpl : MultiSegmentReader
> > readerDir :
> org.apache.lucene.store.FSDirectory@C:\Project\jakarta\lucene\solr\TEMP\example\solr\data\index
> > indexVersion : 1209693930226
> > openedAt : Fri May 02 11:06:13 JST 2008
> > registeredAt : Fri May 02 11:06:13 JST 2008
> >
> > Koji
> >
> >
> > Yonik Seeley wrote:
> >
> > > Hmmm, if there is a bug, odds are it's due to multicore stuff -
> > > probably nothing else has touched core stuff like that recently.
> > > Can you reproduce (or rather help others to reproduce) with the
> > > solr/example setup?
> > >
> > > -Yonik
> > >
> > > On Wed, Apr 30, 2008 at 5:39 PM, Matthew Runo <mr...@zappos.com> wrote:
> > >
> > >
> > > > Hello!
> > > >
> > > > In using the SVN head version of Solr, I've found that recently we
> started
> > > > getting multiple open SegmentReaders, all registered... etc..
> > > >
> > > > Any ideas why this would happen? They don't go away unless the server
> is
> > > > restarted, and don't go away with commits, etc. In fact, commits seem
> to
> > > > cause the issue. They're causing issues since it causes really stale
> > > > searchers to be around...
> > > >
> > > > For example, right now...
> > > > org.apache.solr.search.SolrIndexSearcher
> > > > caching : true
> > > > numDocs : 153312
> > > > maxDoc : 153324
> > > > readerImpl : SegmentReader
> > > > readerDir : org.apache.lucene.store.FSDirectory@/opt/solr/data/index
> > > > indexVersion : 1205944085143
> > > > openedAt : Wed Apr 30 14:04:15 PDT 2008
> > > > registeredAt : Wed Apr 30 14:04:15 PDT 2008
> > > >
> > > > (and right below that one...)
> > > > org.apache.solr.search.SolrIndexSearcher
> > > > caching : true
> > > > numDocs : 153312
> > > > maxDoc : 153324
> > > > readerImpl : SegmentReader
> > > > readerDir : org.apache.lucene.store.FSDirectory@/opt/solr/data/index
> > > > indexVersion : 1205944085143
> > > > openedAt : Wed Apr 30 14:30:02 PDT 2008
> > > > registeredAt : Wed Apr 30 14:30:02 PDT 2008
> > > >
> > > > Thanks!
> > > >
> > > > Matthew Runo
> > > > Software Developer
> > > > Zappos.com
> > > > 702.943.7833
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
>
>
Re: Multiple open SegmentReaders?
Posted by Matthew Runo <mr...@zappos.com>.
Hah, thank you for doing this. Sometimes I see MultiSegmentReaders,
sometimes SegmentReaders, so both show up from time to time. Right now
we've got two MultiSegmentReaders open..
Thanks!
Matthew Runo
Software Developer
Zappos.com
702.943.7833
On May 1, 2008, at 7:19 PM, Koji Sekiguchi wrote:
> I can reproduce with solr/example setup.
> What I did:
>
> 1. $ svn co http://svn.apache.org/repos/asf/lucene/solr/trunk TEMP
> 2. $ cd TEMP
> 3. $ ant clean example
> 4. $ cd example
> 5. $ java -jar start.jar
>
> (to post commit)
> 6. $ cd $SOLR_HOME/example/exampledocs
> 7. $ ./post.sh
>
> then see admin>statistics. I can see MultiSegmentReader instead of
> SegmentReader, though.
>
> name: Searcher@19b1de main class:
> org.apache.solr.search.SolrIndexSearcher version: 1.0
> description: index searcher stats: caching : true
> numDocs : 0
> maxDoc : 0
> readerImpl : MultiSegmentReader
> readerDir : org.apache.lucene.store.FSDirectory@C:\Project\jakarta
> \lucene\solr\TEMP\example\solr\data\index
> indexVersion : 1209693930226
> openedAt : Fri May 02 11:05:30 JST 2008
> registeredAt : Fri May 02 11:05:30 JST 2008
> name: Searcher@7be8c2 main class:
> org.apache.solr.search.SolrIndexSearcher version: 1.0
> description: index searcher stats: caching : true
> numDocs : 0
> maxDoc : 0
> readerImpl : MultiSegmentReader
> readerDir : org.apache.lucene.store.FSDirectory@C:\Project\jakarta
> \lucene\solr\TEMP\example\solr\data\index
> indexVersion : 1209693930226
> openedAt : Fri May 02 11:06:13 JST 2008
> registeredAt : Fri May 02 11:06:13 JST 2008
>
> Koji
>
>
> Yonik Seeley wrote:
>> Hmmm, if there is a bug, odds are it's due to multicore stuff -
>> probably nothing else has touched core stuff like that recently.
>> Can you reproduce (or rather help others to reproduce) with the
>> solr/example setup?
>>
>> -Yonik
>>
>> On Wed, Apr 30, 2008 at 5:39 PM, Matthew Runo <mr...@zappos.com>
>> wrote:
>>
>>> Hello!
>>>
>>> In using the SVN head version of Solr, I've found that recently we
>>> started
>>> getting multiple open SegmentReaders, all registered... etc..
>>>
>>> Any ideas why this would happen? They don't go away unless the
>>> server is
>>> restarted, and don't go away with commits, etc. In fact, commits
>>> seem to
>>> cause the issue. They're causing issues since it causes really stale
>>> searchers to be around...
>>>
>>> For example, right now...
>>> org.apache.solr.search.SolrIndexSearcher
>>> caching : true
>>> numDocs : 153312
>>> maxDoc : 153324
>>> readerImpl : SegmentReader
>>> readerDir : org.apache.lucene.store.FSDirectory@/opt/solr/data/index
>>> indexVersion : 1205944085143
>>> openedAt : Wed Apr 30 14:04:15 PDT 2008
>>> registeredAt : Wed Apr 30 14:04:15 PDT 2008
>>>
>>> (and right below that one...)
>>> org.apache.solr.search.SolrIndexSearcher
>>> caching : true
>>> numDocs : 153312
>>> maxDoc : 153324
>>> readerImpl : SegmentReader
>>> readerDir : org.apache.lucene.store.FSDirectory@/opt/solr/data/index
>>> indexVersion : 1205944085143
>>> openedAt : Wed Apr 30 14:30:02 PDT 2008
>>> registeredAt : Wed Apr 30 14:30:02 PDT 2008
>>>
>>> Thanks!
>>>
>>> Matthew Runo
>>> Software Developer
>>> Zappos.com
>>> 702.943.7833
>>>
>>>
>>>
>>
>>
>
Re: Multiple open SegmentReaders?
Posted by Koji Sekiguchi <ko...@r.email.ne.jp>.
I can reproduce with solr/example setup.
What I did:
1. $ svn co http://svn.apache.org/repos/asf/lucene/solr/trunk TEMP
2. $ cd TEMP
3. $ ant clean example
4. $ cd example
5. $ java -jar start.jar
(to post commit)
6. $ cd $SOLR_HOME/example/exampledocs
7. $ ./post.sh
then see admin>statistics. I can see MultiSegmentReader instead of
SegmentReader, though.
name: Searcher@19b1de main
class: org.apache.solr.search.SolrIndexSearcher
version: 1.0
description: index searcher
stats: caching : true
numDocs : 0
maxDoc : 0
readerImpl : MultiSegmentReader
readerDir :
org.apache.lucene.store.FSDirectory@C:\Project\jakarta\lucene\solr\TEMP\example\solr\data\index
indexVersion : 1209693930226
openedAt : Fri May 02 11:05:30 JST 2008
registeredAt : Fri May 02 11:05:30 JST 2008
name: Searcher@7be8c2 main
class: org.apache.solr.search.SolrIndexSearcher
version: 1.0
description: index searcher
stats: caching : true
numDocs : 0
maxDoc : 0
readerImpl : MultiSegmentReader
readerDir :
org.apache.lucene.store.FSDirectory@C:\Project\jakarta\lucene\solr\TEMP\example\solr\data\index
indexVersion : 1209693930226
openedAt : Fri May 02 11:06:13 JST 2008
registeredAt : Fri May 02 11:06:13 JST 2008
Koji
Yonik Seeley wrote:
> Hmmm, if there is a bug, odds are it's due to multicore stuff -
> probably nothing else has touched core stuff like that recently.
> Can you reproduce (or rather help others to reproduce) with the
> solr/example setup?
>
> -Yonik
>
> On Wed, Apr 30, 2008 at 5:39 PM, Matthew Runo <mr...@zappos.com> wrote:
>
>> Hello!
>>
>> In using the SVN head version of Solr, I've found that recently we started
>> getting multiple open SegmentReaders, all registered... etc..
>>
>> Any ideas why this would happen? They don't go away unless the server is
>> restarted, and don't go away with commits, etc. In fact, commits seem to
>> cause the issue. They're causing issues since it causes really stale
>> searchers to be around...
>>
>> For example, right now...
>> org.apache.solr.search.SolrIndexSearcher
>> caching : true
>> numDocs : 153312
>> maxDoc : 153324
>> readerImpl : SegmentReader
>> readerDir : org.apache.lucene.store.FSDirectory@/opt/solr/data/index
>> indexVersion : 1205944085143
>> openedAt : Wed Apr 30 14:04:15 PDT 2008
>> registeredAt : Wed Apr 30 14:04:15 PDT 2008
>>
>> (and right below that one...)
>> org.apache.solr.search.SolrIndexSearcher
>> caching : true
>> numDocs : 153312
>> maxDoc : 153324
>> readerImpl : SegmentReader
>> readerDir : org.apache.lucene.store.FSDirectory@/opt/solr/data/index
>> indexVersion : 1205944085143
>> openedAt : Wed Apr 30 14:30:02 PDT 2008
>> registeredAt : Wed Apr 30 14:30:02 PDT 2008
>>
>> Thanks!
>>
>> Matthew Runo
>> Software Developer
>> Zappos.com
>> 702.943.7833
>>
>>
>>
>
>
Re: Multiple open SegmentReaders?
Posted by Yonik Seeley <yo...@apache.org>.
Hmmm, if there is a bug, odds are it's due to multicore stuff -
probably nothing else has touched core stuff like that recently.
Can you reproduce (or rather help others to reproduce) with the
solr/example setup?
-Yonik
On Wed, Apr 30, 2008 at 5:39 PM, Matthew Runo <mr...@zappos.com> wrote:
> Hello!
>
> In using the SVN head version of Solr, I've found that recently we started
> getting multiple open SegmentReaders, all registered... etc..
>
> Any ideas why this would happen? They don't go away unless the server is
> restarted, and don't go away with commits, etc. In fact, commits seem to
> cause the issue. They're causing issues since it causes really stale
> searchers to be around...
>
> For example, right now...
> org.apache.solr.search.SolrIndexSearcher
> caching : true
> numDocs : 153312
> maxDoc : 153324
> readerImpl : SegmentReader
> readerDir : org.apache.lucene.store.FSDirectory@/opt/solr/data/index
> indexVersion : 1205944085143
> openedAt : Wed Apr 30 14:04:15 PDT 2008
> registeredAt : Wed Apr 30 14:04:15 PDT 2008
>
> (and right below that one...)
> org.apache.solr.search.SolrIndexSearcher
> caching : true
> numDocs : 153312
> maxDoc : 153324
> readerImpl : SegmentReader
> readerDir : org.apache.lucene.store.FSDirectory@/opt/solr/data/index
> indexVersion : 1205944085143
> openedAt : Wed Apr 30 14:30:02 PDT 2008
> registeredAt : Wed Apr 30 14:30:02 PDT 2008
>
> Thanks!
>
> Matthew Runo
> Software Developer
> Zappos.com
> 702.943.7833
>
>