You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Robert Gründler <ro...@dubture.com> on 2011/03/14 15:44:11 UTC
MySQL queries high when using delta-import
Hi,
we have 3 solr cores, each of them is running a delta-import every 2
minutes on
a MySQL database.
We've noticed a significant increase of MySQL queries per second, since
we've
started the delta updates.
Before that, the database server received between 50 and 100 queries per
second,
since the Delta-Imports the query count has rised up to 100 to 200
queries per second.
I've temporarily disabled the delta imports for 2 hours, and the queries
per second immediately
decreased again to 50-100 per second.
I followed the Wiki entry which only uses one Query for the delta import:
http://wiki.apache.org/solr/DataImportHandlerDeltaQueryViaFullImport
I did not expect the queries per seconds to the database increase that
high, so i'm wondering
if others experienced similar issues.
cheers
-robert
Re: MySQL queries high when using delta-import
Posted by Bill Bell <bi...@gmail.com>.
You could use clean= false parameter trick then just use query. Thus would reduce the queries by half for deltas.
Bill Bell
Sent from mobile
On Mar 14, 2011, at 8:57 AM, Stefan Matheis <ma...@googlemail.com> wrote:
> Robert,
>
> that may extremly depend on your (sub-)entities, and how you built your queries.
>
> perhaps http://wiki.apache.org/solr/DataImportHandler#CachedSqlEntityProcessor
> would help you - like said, depending on your config
>
> Regards
> Stefan
>
> 2011/3/14 Robert Gründler <ro...@dubture.com>:
>> Hi,
>>
>> we have 3 solr cores, each of them is running a delta-import every 2 minutes
>> on
>> a MySQL database.
>>
>> We've noticed a significant increase of MySQL queries per second, since
>> we've
>> started the delta updates.
>>
>> Before that, the database server received between 50 and 100 queries per
>> second,
>> since the Delta-Imports the query count has rised up to 100 to 200 queries
>> per second.
>>
>> I've temporarily disabled the delta imports for 2 hours, and the queries per
>> second immediately
>> decreased again to 50-100 per second.
>>
>> I followed the Wiki entry which only uses one Query for the delta import:
>>
>> http://wiki.apache.org/solr/DataImportHandlerDeltaQueryViaFullImport
>>
>> I did not expect the queries per seconds to the database increase that high,
>> so i'm wondering
>> if others experienced similar issues.
>>
>>
>> cheers
>>
>> -robert
>>
>>
>>
Re: MySQL queries high when using delta-import
Posted by Stefan Matheis <ma...@googlemail.com>.
Robert,
that may extremly depend on your (sub-)entities, and how you built your queries.
perhaps http://wiki.apache.org/solr/DataImportHandler#CachedSqlEntityProcessor
would help you - like said, depending on your config
Regards
Stefan
2011/3/14 Robert Gründler <ro...@dubture.com>:
> Hi,
>
> we have 3 solr cores, each of them is running a delta-import every 2 minutes
> on
> a MySQL database.
>
> We've noticed a significant increase of MySQL queries per second, since
> we've
> started the delta updates.
>
> Before that, the database server received between 50 and 100 queries per
> second,
> since the Delta-Imports the query count has rised up to 100 to 200 queries
> per second.
>
> I've temporarily disabled the delta imports for 2 hours, and the queries per
> second immediately
> decreased again to 50-100 per second.
>
> I followed the Wiki entry which only uses one Query for the delta import:
>
> http://wiki.apache.org/solr/DataImportHandlerDeltaQueryViaFullImport
>
> I did not expect the queries per seconds to the database increase that high,
> so i'm wondering
> if others experienced similar issues.
>
>
> cheers
>
> -robert
>
>
>