[jira] Commented: (PYLUCENE-2) Memory leak when searching in real time reader

["Andi Vajda (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PYLUCENE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852916#action_12852916 ] 

Andi Vajda commented on PYLUCENE-2:
-----------------------------------

Please verify your code against Lucene Java, first. Thanks.

> Memory leak when searching in real time reader
> ----------------------------------------------
>
>                 Key: PYLUCENE-2
>                 URL: https://issues.apache.org/jira/browse/PYLUCENE-2
>             Project: PyLucene
>          Issue Type: Bug
>         Environment: ubuntu 9.10, python 2.6, pylucene 3.0
>            Reporter: feng xiaojie
>            Priority: Critical
>   Original Estimate: 336h
>  Remaining Estimate: 336h
>
> Our codes is following:
> We have 31 index dirs in /tmp (there are about 5 million records in our indexs), we want to real time search, so we use the writer.getReader() to get the real time reader.
> Then we did the search repeatly, finally  java 'out of memory' issue will happen(about 10 minites).
>         initVM(CLASSPATH,initialheap='100m',maxheap='100m')
>         keywordQuery = QueryParser(Version.LUCENE_CURRENT,"content", StandardAnalyzer(Version.LUCENE_CURRENT)).parse("when AND you")
>         writers = []
>         for i in range(1,32):
>             dir = os.path.join("/tmp",str(i))
>         
>             luceneDir = SimpleFSDirectory(File(dir))
>         
>             writer = IndexWriter(luceneDir, StandardAnalyzer(Version.LUCENE_CURRENT), False,IndexWriter.MaxFieldLength.LIMITED)
>             writer.setRAMBufferSizeMB(32.0)
>             writer.setUseCompoundFile(True)
>             writer.setMergeFactor(10)
>             writers.append(writer)
>             
>         while True:
>             searchersList = []
>             readers = []
>             for writer in writers:
>                 reader = writer.getReader()
>                 searcher = IndexSearcher(reader)
>                 searchersList.append(searcher)
>                 readers.append(reader)
>             multiSearcherInstance =  MultiSearcher(searchersList)
>             docs = multiSearcherInstance.search(keywordQuery,IndexerCons.TOP_DOC_NUMBER).scoreDocs
>             
>             multiSearcherInstance.close()
>             for searcher in searchersList:
>                 searcher.close()
>             for reader in readers:
>                 reader.close()
> Then we use the normal reader (directly open from the dirs) instead of the real time reader, the test is OK, no 'out of memory' issue.
> The bug maybe come from java lucene, i don't sure. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.