You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Shawn Heisey <so...@elyograg.org> on 2013/08/20 22:15:29 UTC
Distributed MLT is slow
Before I file an issue on this, I wanted to bring it up here, so I can
see if there's something I'm overlooking.
Distributed MLT is very very slow for me. I can make it work, but a
QTime of one to two minutes in production isn't acceptable. Sending a
non-distributed MLT request directly to a large shard takes about 1.5
seconds. There are six large cold shards and one tiny hot shard.
I used my dev server to gather some logs. This server is considerably
less powerful than my production servers, but has exactly the same data.
It's running a 4.5 snapshot with the patch from SOLR-5125. Unlike my
production servers, the dev server takes over four minutes for the
distributed MLT request. Slightly redacted logfile at this URL:
https://dl.dropboxusercontent.com/u/97770508/slow-mlt.log
After I ran the query that you can see in the logfile, I restarted Solr
on my dev server and ran one of the slow subrequests directly to a
shard. Here's the debugQuery timing section from that request. QTime
on it was 56506:
"QParser":"LuceneQParser",
"timing":{
"time":56504.0,
"prepare":{
"time":29.0,
"query":{
"time":29.0},
"facet":{
"time":0.0},
"mlt":{
"time":0.0},
"highlight":{
"time":0.0},
"stats":{
"time":0.0},
"spellcheck":{
"time":0.0},
"debug":{
"time":0.0}},
"process":{
"time":56475.0,
"query":{
"time":935.0},
"facet":{
"time":0.0},
"mlt":{
"time":55442.0},
"highlight":{
"time":0.0},
"stats":{
"time":0.0},
"spellcheck":{
"time":0.0},
"debug":{
"time":98.0}}}}}
Is there anything for me to do other than file an issue?
Thanks,
Shawn
Re: Distributed MLT is slow
Posted by Shawn Heisey <so...@elyograg.org>.
On 8/20/2013 2:21 PM, Luis Cappa Banda wrote:
> Is distributed MLT officially released or you are using a patch?
It was added by SOLR-788 to version 4.1.0. It has not received very
much scrutiny yet.
In addition to SOLR-5125, I have discovered SOLR-5161. Both issues have
workarounds, and 5125 has a patch that has not yet been committed. I
have that patch applied to the 4.5-SNAPSHOT that I'm running.
Thanks,
Shawn
Re: Distributed MLT is slow
Posted by Luis Cappa Banda <lu...@gmail.com>.
Is distributed MLT officially released or you are using a patch?
El martes, 20 de agosto de 2013, Shawn Heisey escribió:
> Before I file an issue on this, I wanted to bring it up here, so I can see
> if there's something I'm overlooking.
>
> Distributed MLT is very very slow for me. I can make it work, but a QTime
> of one to two minutes in production isn't acceptable. Sending a
> non-distributed MLT request directly to a large shard takes about 1.5
> seconds. There are six large cold shards and one tiny hot shard.
>
> I used my dev server to gather some logs. This server is considerably
> less powerful than my production servers, but has exactly the same data.
> It's running a 4.5 snapshot with the patch from SOLR-5125. Unlike my
> production servers, the dev server takes over four minutes for the
> distributed MLT request. Slightly redacted logfile at this URL:
>
> https://dl.dropboxusercontent.**com/u/97770508/slow-mlt.log<https://dl.dropboxusercontent.com/u/97770508/slow-mlt.log>
>
> After I ran the query that you can see in the logfile, I restarted Solr on
> my dev server and ran one of the slow subrequests directly to a shard.
> Here's the debugQuery timing section from that request. QTime on it was
> 56506:
>
> "QParser":"LuceneQParser",
> "timing":{
> "time":56504.0,
> "prepare":{
> "time":29.0,
> "query":{
> "time":29.0},
> "facet":{
> "time":0.0},
> "mlt":{
> "time":0.0},
> "highlight":{
> "time":0.0},
> "stats":{
> "time":0.0},
> "spellcheck":{
> "time":0.0},
> "debug":{
> "time":0.0}},
> "process":{
> "time":56475.0,
> "query":{
> "time":935.0},
> "facet":{
> "time":0.0},
> "mlt":{
> "time":55442.0},
> "highlight":{
> "time":0.0},
> "stats":{
> "time":0.0},
> "spellcheck":{
> "time":0.0},
> "debug":{
> "time":98.0}}}}}
>
> Is there anything for me to do other than file an issue?
>
> Thanks,
> Shawn
>
--
- Luis Cappa