High QTime when wildcards in hl.fl are used

[Karol Sikora <ka...@laboratorium.ee>]

Hi all,

I'm currently stumbling with following case:
I have indexed documents with fileds named like fulltext_[some id].
I'm testing highlighting on document which have only one that field, 
fulltext_1234.
When 'fulltext_*' is provided as hl.fl, QTime is horribly big (> 10s), 
when explicit 'fulltext_1234' is provided, QTime is acceptable (~30ms).
I've found that using widlcards in hl.fl can increase QTime ( 
http://stackoverflow.com/questions/11774508/optimize-solr-highlighter), 
but it definitly should not cost so much.

I'm using fastVectorHighliter in both cases.
Any ideas why using wildcards cause such big QTimes? Maybe there is 
workaround?

-- 
  
Karol Sikora
+48 781 493 788

Laboratorium EE
ul. Mokotowska 46A/23 | 00-543 Warszawa |
www.laboratorium.ee | www.laboratorium.ee/facebook

Re: High QTime when wildcards in hl.fl are used

[Karol Sikora <ka...@laboratorium.ee>]

I've found more interesting informations about using 
fastVectorHighlighting combined with highlighted fields with wildcards 
after testing on isolated group of documents with text content.
fvh + fulltext_*: QTime ~4s (!)
fvh + fulltext_1234: QTime ~50ms
no fvh + fulltext_*: QTime ~600ms
no fvh + fulltext_1234: QTime ~500ms.

As we can see very long query times as associated with using fvh 
combined with wildcarded hl.fl.
In source code I found that fields to highlight when we using wildcards 
are computed by regex in loop through fields returned by query in 
document, so I this case, when I have only one fileld that is matching 
given pattern it should be no diference between using wildcards and not.

Any ideas?


W dniu 08.03.2013 13:49, Karol Sikora pisze:
> Hi all,
>
> I'm currently stumbling with following case:
> I have indexed documents with fileds named like fulltext_[some id].
> I'm testing highlighting on document which have only one that field, 
> fulltext_1234.
> When 'fulltext_*' is provided as hl.fl, QTime is horribly big (> 10s), 
> when explicit 'fulltext_1234' is provided, QTime is acceptable (~30ms).
> I've found that using widlcards in hl.fl can increase QTime ( 
> http://stackoverflow.com/questions/11774508/optimize-solr-highlighter), but 
> it definitly should not cost so much.
>
> I'm using fastVectorHighliter in both cases.
> Any ideas why using wildcards cause such big QTimes? Maybe there is 
> workaround?
> -- 
>   
> Karol Sikora
> +48 781 493 788
>
> Laboratorium EE
> ul. Mokotowska 46A/23 | 00-543 Warszawa |
> www.laboratorium.ee  |www.laboratorium.ee/facebook

-- 
Karol Sikora
+48 781 493 788

Laboratorium EE
ul. Mokotowska 46A/23 | 00-543 Warszawa |
www.laboratorium.ee | www.laboratorium.ee/facebook