You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by yuexiang zhang <ea...@gmail.com> on 2006/07/08 09:50:30 UTC
java.lang.OutOfMemoryError when search in large index
I have finished importing 10,000,000 docments into Lucene 2.0 .
Every document includes journalTitle and articleTitle both of which should
be sortable, so I
add other two fields : journalTitle_sort and articleTitle_sort both of
which use un_tokenized option for
indexer.
When I query by "journaltitle:advertising" and sort by journaltitle_sort,
it 's ok for return "total result num is 28684"
But when I use the same query and change sort field from journaltitle_sort
to articleTitle_sort, java.lang.OutOfMemoryError happens.
Then I try set JVM options to "-Xmx2048m", it also do take affect,
java.lang.OutOfMemoryError happens too.
I think maybe lucene preload all unique sortable field vlaue into memory,
but my articleTitle_sort has too large different values because of
un_tokenized option.
Who can help me? Thanks!
Re: java.lang.OutOfMemoryError when search in large index
Posted by Chris Hostetter <ho...@fucit.org>.
These two threads may be educational...
http://www.nabble.com/OutOfMemory-error-while-sorting-tf1785358.html#a4862471
http://www.nabble.com/MemoryUsage-of-sorting-tf1861157.html#a5083259
: Date: Sat, 8 Jul 2006 15:50:30 +0800
: From: yuexiang zhang <ea...@gmail.com>
: Reply-To: java-user@lucene.apache.org
: To: java-user@lucene.apache.org
: Subject: java.lang.OutOfMemoryError when search in large index
:
: I have finished importing 10,000,000 docments into Lucene 2.0 .
:
: Every document includes journalTitle and articleTitle both of which should
: be sortable, so I
: add other two fields : journalTitle_sort and articleTitle_sort both of
: which use un_tokenized option for
: indexer.
:
: When I query by "journaltitle:advertising" and sort by journaltitle_sort,
: it 's ok for return "total result num is 28684"
: But when I use the same query and change sort field from journaltitle_sort
: to articleTitle_sort, java.lang.OutOfMemoryError happens.
: Then I try set JVM options to "-Xmx2048m", it also do take affect,
: java.lang.OutOfMemoryError happens too.
:
: I think maybe lucene preload all unique sortable field vlaue into memory,
: but my articleTitle_sort has too large different values because of
: un_tokenized option.
:
: Who can help me? Thanks!
:
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org