You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by 齐保元 <qi...@126.com> on 2012/07/16 08:57:32 UTC
many index reader problem
Hi,buddy:
I have a problem concerning index readers:there are many small index/searcher instances in my application which are hold by a map.when new index request or seach request comes,I process them and return the result.The problem is,when the number of small index becomes large,the JVM throws OOV error:not enough native thread.So,any idea to solve this problem?
--
____________________________________________
齐保元(Qi Baoyuan)
中科院计算技术研究所 智能信息处理重点实验室(ICT,NKI,GUCAS,Beijing,China)
手机：15120084688
E-Mail：qibaoyuan@126.com
Re: many index reader problem
Posted by Danil ŢORIN <to...@gmail.com>.
Do you really HAVE to keep all those indexes opened?
You could use a LRU or LFU cache of reasonable size with opened
indexes, and open new searcher if it's not in the cache.
If your indexes are quite small, the open call shouldn't be too expensive.
On Mon, Jul 16, 2012 at 11:51 AM, Ian Lea <ia...@gmail.com> wrote:
> OOV or OOM? Always best to post a full stack trace, and version of
> lucene, and OS.
>
> Anyway - give your app more memory? Close searchers after use or some
> period of inactivity?
>
> Best long term solution is probably to merge the many small indexes
> into one, or a few, larger indexes and restrict queries by an indexed
> field identifying the index you are interested in. A cached filter
> can work well for that.
>
>
> --
> Ian.
>
>
> On Mon, Jul 16, 2012 at 7:57 AM, 齐保元 <qi...@126.com> wrote:
>>
>> Hi,buddy:
>> I have a problem concerning index readers:there are many small index/searcher instances in my application which are hold by a map.when new index request or seach request comes,I process them and return the result.The problem is,when the number of small index becomes large,the JVM throws OOV error:not enough native thread.So,any idea to solve this problem?
>>
>> --
>>
>> ____________________________________________
>> 齐保元(Qi Baoyuan)
>> 中科院计算技术研究所 智能信息处理重点实验室(ICT,NKI,GUCAS,Beijing,China)
>> 手机：15120084688
>> E-Mail：qibaoyuan@126.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: many index reader problem
Posted by Ian Lea <ia...@gmail.com>.
OOV or OOM? Always best to post a full stack trace, and version of
lucene, and OS.
Anyway - give your app more memory? Close searchers after use or some
period of inactivity?
Best long term solution is probably to merge the many small indexes
into one, or a few, larger indexes and restrict queries by an indexed
field identifying the index you are interested in. A cached filter
can work well for that.
--
Ian.
On Mon, Jul 16, 2012 at 7:57 AM, 齐保元 <qi...@126.com> wrote:
>
> Hi,buddy:
> I have a problem concerning index readers:there are many small index/searcher instances in my application which are hold by a map.when new index request or seach request comes,I process them and return the result.The problem is,when the number of small index becomes large,the JVM throws OOV error:not enough native thread.So,any idea to solve this problem?
>
> --
>
> ____________________________________________
> 齐保元(Qi Baoyuan)
> 中科院计算技术研究所 智能信息处理重点实验室(ICT,NKI,GUCAS,Beijing,China)
> 手机：15120084688
> E-Mail：qibaoyuan@126.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org