You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Maduranga Kannangara <mk...@infomedia.com.au> on 2010/05/14 06:58:56 UTC
Solr Deployment Question
Hi
We use separate JVMs to Index and Query.
(Client applications will query only slaves,
while master does only indexing)
Recently we moved a two master indexes to
a single JVM. Our memory allocation was for
each index was 512Mb and 1Gb.
Once we moved both indexes to a single VM,
we thought it would still Index using 1Gb as we
use only one index at a time. But for our surprise
it needed more than that (1.2Gb) even though
only one index was used at a time.
Can I know why, or can I know how to find
why this is?
Solr 1.4
Java 1.6.0_20
We use a VPS for deployment.
Thanks in advance
Madu
RE: Solr Deployment Question
Posted by Maduranga Kannangara <mk...@infomedia.com.au>.
They are two web applications running on a single Tomcat instance.
Thanks
Madu
-----Original Message-----
From: findbestopensource [mailto:findbestopensource@gmail.com]
Sent: Friday, 14 May 2010 4:38 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr Deployment Question
Please explain how you have handled two indexes in a single VM. Is it multi
core?
To identify memory consumption, You need to calculate usedmemory before and
after loading the indexes, basically calculate usedmemory before and after
any check point you want to analyse. Their difference will give you the
actual memory consumption.
Regards
Aditya
http://www.findbestopensource.com
On Fri, May 14, 2010 at 11:14 AM, Maduranga Kannangara <
mkannangara@infomedia.com.au> wrote:
> But even we used a single index, we were running out of memory.
> What do you mean by "active"? No queries on the masters.
> Only one index is being processed/optimized.
>
> Also, if I may add to my same question, how can I find the
> amount of memory that an index would use, theoretically?
> i.e.: Is there a formulae etc?
>
> Thanks
> Madu
>
>
>
> -----Original Message-----
> From: findbestopensource [mailto:findbestopensource@gmail.com]
> Sent: Friday, 14 May 2010 3:34 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr Deployment Question
>
> You may use one index at a time, but both indexes are active and loaded all
> its terms in memory. Memory consumption will be certainly more.
>
> Regards
> Aditya
> http://www.findbestopensource.com
>
> On Fri, May 14, 2010 at 10:28 AM, Maduranga Kannangara <
> mkannangara@infomedia.com.au> wrote:
>
> > Hi
> >
> > We use separate JVMs to Index and Query.
> > (Client applications will query only slaves,
> > while master does only indexing)
> >
> > Recently we moved a two master indexes to
> > a single JVM. Our memory allocation was for
> > each index was 512Mb and 1Gb.
> >
> > Once we moved both indexes to a single VM,
> > we thought it would still Index using 1Gb as we
> > use only one index at a time. But for our surprise
> > it needed more than that (1.2Gb) even though
> > only one index was used at a time.
> >
> > Can I know why, or can I know how to find
> > why this is?
> >
> > Solr 1.4
> > Java 1.6.0_20
> >
> > We use a VPS for deployment.
> >
> > Thanks in advance
> > Madu
> >
> >
> >
>
Re: Solr Deployment Question
Posted by findbestopensource <fi...@gmail.com>.
Please explain how you have handled two indexes in a single VM. Is it multi
core?
To identify memory consumption, You need to calculate usedmemory before and
after loading the indexes, basically calculate usedmemory before and after
any check point you want to analyse. Their difference will give you the
actual memory consumption.
Regards
Aditya
http://www.findbestopensource.com
On Fri, May 14, 2010 at 11:14 AM, Maduranga Kannangara <
mkannangara@infomedia.com.au> wrote:
> But even we used a single index, we were running out of memory.
> What do you mean by "active"? No queries on the masters.
> Only one index is being processed/optimized.
>
> Also, if I may add to my same question, how can I find the
> amount of memory that an index would use, theoretically?
> i.e.: Is there a formulae etc?
>
> Thanks
> Madu
>
>
>
> -----Original Message-----
> From: findbestopensource [mailto:findbestopensource@gmail.com]
> Sent: Friday, 14 May 2010 3:34 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr Deployment Question
>
> You may use one index at a time, but both indexes are active and loaded all
> its terms in memory. Memory consumption will be certainly more.
>
> Regards
> Aditya
> http://www.findbestopensource.com
>
> On Fri, May 14, 2010 at 10:28 AM, Maduranga Kannangara <
> mkannangara@infomedia.com.au> wrote:
>
> > Hi
> >
> > We use separate JVMs to Index and Query.
> > (Client applications will query only slaves,
> > while master does only indexing)
> >
> > Recently we moved a two master indexes to
> > a single JVM. Our memory allocation was for
> > each index was 512Mb and 1Gb.
> >
> > Once we moved both indexes to a single VM,
> > we thought it would still Index using 1Gb as we
> > use only one index at a time. But for our surprise
> > it needed more than that (1.2Gb) even though
> > only one index was used at a time.
> >
> > Can I know why, or can I know how to find
> > why this is?
> >
> > Solr 1.4
> > Java 1.6.0_20
> >
> > We use a VPS for deployment.
> >
> > Thanks in advance
> > Madu
> >
> >
> >
>
RE: Solr Deployment Question
Posted by Maduranga Kannangara <mk...@infomedia.com.au>.
But even we used a single index, we were running out of memory.
What do you mean by "active"? No queries on the masters.
Only one index is being processed/optimized.
Also, if I may add to my same question, how can I find the
amount of memory that an index would use, theoretically?
i.e.: Is there a formulae etc?
Thanks
Madu
-----Original Message-----
From: findbestopensource [mailto:findbestopensource@gmail.com]
Sent: Friday, 14 May 2010 3:34 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr Deployment Question
You may use one index at a time, but both indexes are active and loaded all
its terms in memory. Memory consumption will be certainly more.
Regards
Aditya
http://www.findbestopensource.com
On Fri, May 14, 2010 at 10:28 AM, Maduranga Kannangara <
mkannangara@infomedia.com.au> wrote:
> Hi
>
> We use separate JVMs to Index and Query.
> (Client applications will query only slaves,
> while master does only indexing)
>
> Recently we moved a two master indexes to
> a single JVM. Our memory allocation was for
> each index was 512Mb and 1Gb.
>
> Once we moved both indexes to a single VM,
> we thought it would still Index using 1Gb as we
> use only one index at a time. But for our surprise
> it needed more than that (1.2Gb) even though
> only one index was used at a time.
>
> Can I know why, or can I know how to find
> why this is?
>
> Solr 1.4
> Java 1.6.0_20
>
> We use a VPS for deployment.
>
> Thanks in advance
> Madu
>
>
>
Re: Solr Deployment Question
Posted by findbestopensource <fi...@gmail.com>.
You may use one index at a time, but both indexes are active and loaded all
its terms in memory. Memory consumption will be certainly more.
Regards
Aditya
http://www.findbestopensource.com
On Fri, May 14, 2010 at 10:28 AM, Maduranga Kannangara <
mkannangara@infomedia.com.au> wrote:
> Hi
>
> We use separate JVMs to Index and Query.
> (Client applications will query only slaves,
> while master does only indexing)
>
> Recently we moved a two master indexes to
> a single JVM. Our memory allocation was for
> each index was 512Mb and 1Gb.
>
> Once we moved both indexes to a single VM,
> we thought it would still Index using 1Gb as we
> use only one index at a time. But for our surprise
> it needed more than that (1.2Gb) even though
> only one index was used at a time.
>
> Can I know why, or can I know how to find
> why this is?
>
> Solr 1.4
> Java 1.6.0_20
>
> We use a VPS for deployment.
>
> Thanks in advance
> Madu
>
>
>