You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by sunnyfr <jo...@gmail.com> on 2009/04/02 14:04:22 UTC
JVM best tune? help ... solr1.4
Hi,
Just applied replication by requestHandler.
And since this the Qtime went mad and can reach long time <int
name="QTime">9068</int>
Without this replication Qtime can be around 1sec.
I've 14Mdocs stores for 11G. so not a lot of data stores.
I've servers with 8G and tomcat use 7G.
I'm updating every 30mn which is about 50 000docs.
Have a look as well at my cpu which are aswell quite full ?
Have you an idea? Do I miss a patch ?
Thanks a lot,
Solr Specification Version: 1.3.0.2009.01.22.13.51.22
Solr Implementation Version: 1.4-dev exported - root - 2009-01-22 13:51:22
http://www.nabble.com/file/p22846336/CPU.jpg CPU.jpg
--
View this message in context: http://www.nabble.com/JVM-best-tune--help-...-solr1.4-tp22846336p22846336.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: JVM best tune? help ... solr1.4
Posted by Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>.
http://people.apache.org/~hossman/#threadhijack
On Thu, Apr 2, 2009 at 5:47 PM, sunnyfr <jo...@gmail.com> wrote:
>
> I think about the slave.
> When I start in multi thread 20 request second my cpu is very bad.
> I'm sure I don't manage properly my gc. I've 8G per slave it should be fine.
>
> I wonder, I shouldn't put 7G to xmx jvm, I don't know,
> but slave is as well a little problem during replication from the master.
>
>
>
> Noble Paul നോബിള് नोब्ळ् wrote:
>>
>> If you are looking at the QTime on the master it is likely to be
>> skewed by ReplicationHandler becaus ethe files are downloaded using a
>> request. On a slave it should not be a problem.
>>
>> I guess we must not add the qtimes of ReplicationHandler
>> --Noble
>>
>> On Thu, Apr 2, 2009 at 5:34 PM, sunnyfr <jo...@gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> Just applied replication by requestHandler.
>>> And since this the Qtime went mad and can reach long time <int
>>> name="QTime">9068</int>
>>> Without this replication Qtime can be around 1sec.
>>>
>>> I've 14Mdocs stores for 11G. so not a lot of data stores.
>>> I've servers with 8G and tomcat use 7G.
>>> I'm updating every 30mn which is about 50 000docs.
>>> Have a look as well at my cpu which are aswell quite full ?
>>>
>>> Have you an idea? Do I miss a patch ?
>>> Thanks a lot,
>>>
>>> Solr Specification Version: 1.3.0.2009.01.22.13.51.22
>>> Solr Implementation Version: 1.4-dev exported - root - 2009-01-22
>>> 13:51:22
>>>
>>> http://www.nabble.com/file/p22846336/CPU.jpg CPU.jpg
>>> --
>>> View this message in context:
>>> http://www.nabble.com/JVM-best-tune--help-...-solr1.4-tp22846336p22846336.html
>>> Sent from the Solr - User mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> --Noble Paul
>>
>>
>
> --
> View this message in context: http://www.nabble.com/JVM-best-tune--help-...-solr1.4-tp22846336p22846546.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
--
--Noble Paul
Re: JVM best tune? help ... solr1.4
Posted by sunnyfr <jo...@gmail.com>.
I think about the slave.
When I start in multi thread 20 request second my cpu is very bad.
I'm sure I don't manage properly my gc. I've 8G per slave it should be fine.
I wonder, I shouldn't put 7G to xmx jvm, I don't know,
but slave is as well a little problem during replication from the master.
Noble Paul നോബിള് नोब्ळ् wrote:
>
> If you are looking at the QTime on the master it is likely to be
> skewed by ReplicationHandler becaus ethe files are downloaded using a
> request. On a slave it should not be a problem.
>
> I guess we must not add the qtimes of ReplicationHandler
> --Noble
>
> On Thu, Apr 2, 2009 at 5:34 PM, sunnyfr <jo...@gmail.com> wrote:
>>
>> Hi,
>>
>> Just applied replication by requestHandler.
>> And since this the Qtime went mad and can reach long time <int
>> name="QTime">9068</int>
>> Without this replication Qtime can be around 1sec.
>>
>> I've 14Mdocs stores for 11G. so not a lot of data stores.
>> I've servers with 8G and tomcat use 7G.
>> I'm updating every 30mn which is about 50 000docs.
>> Have a look as well at my cpu which are aswell quite full ?
>>
>> Have you an idea? Do I miss a patch ?
>> Thanks a lot,
>>
>> Solr Specification Version: 1.3.0.2009.01.22.13.51.22
>> Solr Implementation Version: 1.4-dev exported - root - 2009-01-22
>> 13:51:22
>>
>> http://www.nabble.com/file/p22846336/CPU.jpg CPU.jpg
>> --
>> View this message in context:
>> http://www.nabble.com/JVM-best-tune--help-...-solr1.4-tp22846336p22846336.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> --Noble Paul
>
>
--
View this message in context: http://www.nabble.com/JVM-best-tune--help-...-solr1.4-tp22846336p22846546.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: JVM best tune? help ... solr1.4
Posted by Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>.
If you are looking at the QTime on the master it is likely to be
skewed by ReplicationHandler becaus ethe files are downloaded using a
request. On a slave it should not be a problem.
I guess we must not add the qtimes of ReplicationHandler
--Noble
On Thu, Apr 2, 2009 at 5:34 PM, sunnyfr <jo...@gmail.com> wrote:
>
> Hi,
>
> Just applied replication by requestHandler.
> And since this the Qtime went mad and can reach long time <int
> name="QTime">9068</int>
> Without this replication Qtime can be around 1sec.
>
> I've 14Mdocs stores for 11G. so not a lot of data stores.
> I've servers with 8G and tomcat use 7G.
> I'm updating every 30mn which is about 50 000docs.
> Have a look as well at my cpu which are aswell quite full ?
>
> Have you an idea? Do I miss a patch ?
> Thanks a lot,
>
> Solr Specification Version: 1.3.0.2009.01.22.13.51.22
> Solr Implementation Version: 1.4-dev exported - root - 2009-01-22 13:51:22
>
> http://www.nabble.com/file/p22846336/CPU.jpg CPU.jpg
> --
> View this message in context: http://www.nabble.com/JVM-best-tune--help-...-solr1.4-tp22846336p22846336.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
--
--Noble Paul
Re: JVM best tune? help ... solr1.4
Posted by sunnyfr <jo...@gmail.com>.
This is my conf :
http://www.nabble.com/file/p22847570/solrconfig.xml solrconfig.xml
And this is my delta import:
*/20 * * * * /usr/bin/wget -q --output-document=/home/video_import.txt
http:/master.com:8180/solr/video/dataimport?command=delta-import&optimize=false
--
View this message in context: http://www.nabble.com/JVM-best-tune--help-...-solr1.4-tp22846336p22847570.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: JVM best tune? help ... solr1.4
Posted by sunnyfr <jo...@gmail.com>.
I don't optimize at all.
my delta-import&optimize=false
I didn't turnd on optimize, I think it merges segment alone, because size
increase too quickly?
Noble Paul നോബിള് नोब्ळ् wrote:
>
> it is not a good idea to optimize the index everytime you commit. That
> is why your downloads are taking so long
>
> 2009/4/2 Johanna Mortemousque <jo...@gmail.com>:
>> I've so many update, almost 2 000 every 20mn, that lucene merge my index
>> folder,
>> so everytime my slave replicate its a new index folder merged so every
>> time
>> it brings back 10G datas.
>>
>> And during this time my repond time of my request are very slow.
>> What can I check?
>>
>> Thanks Paul
>>
>> 2009/4/2 Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>
>>>
>>> slave would not show increased request times because of replication.
>>> If it does should be some bug
>>>
>>> On Thu, Apr 2, 2009 at 6:00 PM, <jo...@gmail.com> wrote:
>>> > I think its the same problem, tune jvm for multi thread ... 20request
>>> > seconde.
>>> > no??
>>> >
>>> >
>>> >
>>> > Noble Paul നോബിള് नोब्ळ् wrote:
>>> >>
>>> >> http://people.apache.org/~hossman/#threadhijack
>>> >>
>>> >> On Thu, Apr 2, 2009 at 5:47 PM, sunnyfr <jo...@gmail.com> wrote:
>>> >>>
>>> >>> I think about the slave.
>>> >>> When I start in multi thread 20 request second my cpu is very bad.
>>> >>> I'm sure I don't manage properly my gc. I've 8G per slave it should
>>> be
>>> >>> fine.
>>> >>>
>>> >>> I wonder, I shouldn't put 7G to xmx jvm, I don't know,
>>> >>> but slave is as well a little problem during replication from the
>>> >>> master.
>>> >>>
>>> >>>
>>> >>>
>>> >>> Noble Paul നോബിള് नोब्ळ् wrote:
>>> >>>>
>>> >>>> If you are looking at the QTime on the master it is likely to be
>>> >>>> skewed by ReplicationHandler becaus ethe files are downloaded using
>>> a
>>> >>>> request. On a slave it should not be a problem.
>>> >>>>
>>> >>>> I guess we must not add the qtimes of ReplicationHandler
>>> >>>> --Noble
>>> >>>>
>>> >>>> On Thu, Apr 2, 2009 at 5:34 PM, sunnyfr <jo...@gmail.com>
>>> wrote:
>>> >>>>>
>>> >>>>> Hi,
>>> >>>>>
>>> >>>>> Just applied replication by requestHandler.
>>> >>>>> And since this the Qtime went mad and can reach long time <int
>>> >>>>> name="QTime">9068</int>
>>> >>>>> Without this replication Qtime can be around 1sec.
>>> >>>>>
>>> >>>>> I've 14Mdocs stores for 11G. so not a lot of data stores.
>>> >>>>> I've servers with 8G and tomcat use 7G.
>>> >>>>> I'm updating every 30mn which is about 50 000docs.
>>> >>>>> Have a look as well at my cpu which are aswell quite full ?
>>> >>>>>
>>> >>>>> Have you an idea? Do I miss a patch ?
>>> >>>>> Thanks a lot,
>>> >>>>>
>>> >>>>> Solr Specification Version: 1.3.0.2009.01.22.13.51.22
>>> >>>>> Solr Implementation Version: 1.4-dev exported - root - 2009-01-22
>>> >>>>> 13:51:22
>>> >>>>>
>>> >>>>> http://www.nabble.com/file/p22846336/CPU.jpg CPU.jpg
>>> >>>>> --
>>> >>>>> View this message in context:
>>> >>>>>
>>> >>>>>
>>> http://www.nabble.com/JVM-best-tune--help-...-solr1.4-tp22846336p22846336.html
>>> >>>>> Sent from the Solr - User mailing list archive at Nabble.com.
>>> >>>>>
>>> >>>>>
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> --
>>> >>>> --Noble Paul
>>> >>>>
>>> >>>>
>>> >>>
>>> >>> --
>>> >>> View this message in context:
>>> >>>
>>> >>>
>>> http://www.nabble.com/JVM-best-tune--help-...-solr1.4-tp22846336p22846546.html
>>> >>> Sent from the Solr - User mailing list archive at Nabble.com.
>>> >>>
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> --Noble Paul
>>> >>
>>> >>
>>> > Quoted from:
>>> >
>>> >
>>> http://www.nabble.com/JVM-best-tune--help-...-solr1.4-tp22846336p22846716.html
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> --Noble Paul
>>
>>
>
>
>
> --
> --Noble Paul
>
>
--
View this message in context: http://www.nabble.com/JVM-best-tune--help-...-solr1.4-tp22846336p22847455.html
Sent from the Solr - User mailing list archive at Nabble.com.