You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by jo...@aol.com on 2012/11/12 23:33:46 UTC
Is leading wildcard search turned on by default in Solr 3.6.1?
Hi,
I'm migrating from Solr 1.2 to 3.6.1. I used the same analyzer as I was, and re-indexed my data. I did not add
solr.ReversedWildcardFilterFactory to my index analyzer, but yet leading wild cards are working!! Does this mean it's turned on by default? If so, how do I turn it off, and what are the implication of leaving ON? Won't my searches be slower and consume more memory?
Thanks,
--MJ
Re: Is leading wildcard search turned on by default in Solr 3.6.1?
Posted by François Schiettecatte <fs...@gmail.com>.
I suspect it is just part of the wildcard handling, maybe someone can chime in here, you may need to catch this before it gets to SOLR.
François
On Nov 12, 2012, at 5:44 PM, johnmunir@aol.com wrote:
> Thanks for the quick response.
>
>
> So, I do not want to use ReversedWildcardFilterFactory, but leading wildcard is working and thus is ON by default. How do I disable it to prevent the use of it and the issues that come with it?
>
>
> -- MJ
>
>
>
> -----Original Message-----
> From: François Schiettecat
> te <fs...@gmail.com>
> To: solr-user <so...@lucene.apache.org>
> Sent: Mon, Nov 12, 2012 5:39 pm
> Subject: Re: Is leading wildcard search turned on by default in Solr 3.6.1?
>
>
> John
>
> You can still use leading wildcards even if you dont have the
> ReversedWildcardFilterFactory in your analysis but it means you will be scanning
> the entire dictionary when the search is run which can be a performance issue.
> If you do use ReversedWildcardFilterFactory you wont have that performance issue
> but you will increase the overall size of your index. Its a tradeoff.
>
> When I looked into it for a site I built I decided that the tradeoff was not
> worth it (after benchmarking) given how few leading wildcards searches it was
> getting.
>
> Best regards
>
> François
>
>
> On Nov 12, 2012, at 5:33 PM, johnmunir@aol.com wrote:
>
>>
>>
>> Hi,
>>
>>
>> I'm migrating from Solr 1.2 to 3.6.1. I used the same analyzer as I was, and
> re-indexed my data. I did not add
>> solr.ReversedWildcardFilterFactory to my index analyzer, but yet leading wild
> cards are working!! Does this mean it's turned on by default? If so, how do I
> turn it off, and what are the implication of leaving ON? Won't my searches be
> slower and consume more memory?
>>
>>
>> Thanks,
>>
>>
>> --MJ
>>
>
>
>
>
Re: Is leading wildcard search turned on by default in Solr 3.6.1?
Posted by jo...@aol.com.
Thanks for the quick response.
So, I do not want to use ReversedWildcardFilterFactory, but leading wildcard is working and thus is ON by default. How do I disable it to prevent the use of it and the issues that come with it?
-- MJ
-----Original Message-----
From: François Schiettecat
te <fs...@gmail.com>
To: solr-user <so...@lucene.apache.org>
Sent: Mon, Nov 12, 2012 5:39 pm
Subject: Re: Is leading wildcard search turned on by default in Solr 3.6.1?
John
You can still use leading wildcards even if you dont have the
ReversedWildcardFilterFactory in your analysis but it means you will be scanning
the entire dictionary when the search is run which can be a performance issue.
If you do use ReversedWildcardFilterFactory you wont have that performance issue
but you will increase the overall size of your index. Its a tradeoff.
When I looked into it for a site I built I decided that the tradeoff was not
worth it (after benchmarking) given how few leading wildcards searches it was
getting.
Best regards
François
On Nov 12, 2012, at 5:33 PM, johnmunir@aol.com wrote:
>
>
> Hi,
>
>
> I'm migrating from Solr 1.2 to 3.6.1. I used the same analyzer as I was, and
re-indexed my data. I did not add
> solr.ReversedWildcardFilterFactory to my index analyzer, but yet leading wild
cards are working!! Does this mean it's turned on by default? If so, how do I
turn it off, and what are the implication of leaving ON? Won't my searches be
slower and consume more memory?
>
>
> Thanks,
>
>
> --MJ
>
Re: Is leading wildcard search turned on by default in Solr 3.6.1?
Posted by Dmitry Kan <dm...@gmail.com>.
Just a quick comment from our experience: since we have quite a lot of data
indexed in our Solr, we take some extra measures to ensure, no bogus
wild-card queries are accepted by the system (for instance *, **, *** etc).
And that is done in the QueryParser. Wanted to mention this approach as one
way of handling simple query security checks.
-- Dmitry
On Tue, Nov 13, 2012 at 6:22 AM, Jack Krupansky <ja...@basetechnology.com>wrote:
> Be sure to realize that even with reverse wildcard support, the user can
> add a trailing wildcard as well (double-ended wildcard) and then you are
> back in the same boat.
>
> The overall idea is that: 1) Hardware is much faster than just 3 or 4
> years ago, and 2) even though document counts are getting much larger, the
> number of unique terms (which is all that matters for wildcard performance)
> does not tend to grow as fast as document count grows. And, some fields
> have a much more limited vocabulary (unique terms), so a leading wildcard
> is not necessarily a big performance hit.
>
> Technology advances. We should permit our mindsets to advance as well.
>
> -- Jack Krupansky
>
>
> -----Original Message----- From: François Schiettecatte
> Sent: Monday, November 12, 2012 2:38 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Is leading wildcard search turned on by default in Solr 3.6.1?
>
>
> John
>
> You can still use leading wildcards even if you dont have the
> ReversedWildcardFilterFactory in your analysis but it means you will be
> scanning the entire dictionary when the search is run which can be a
> performance issue. If you do use ReversedWildcardFilterFactory you wont
> have that performance issue but you will increase the overall size of your
> index. Its a tradeoff.
>
> When I looked into it for a site I built I decided that the tradeoff was
> not worth it (after benchmarking) given how few leading wildcards searches
> it was getting.
>
> Best regards
>
> François
>
>
> On Nov 12, 2012, at 5:33 PM, johnmunir@aol.com wrote:
>
>
>>
>> Hi,
>>
>>
>> I'm migrating from Solr 1.2 to 3.6.1. I used the same analyzer as I was,
>> and re-indexed my data. I did not add
>> solr.**ReversedWildcardFilterFactory to my index analyzer, but yet
>> leading wild cards are working!! Does this mean it's turned on by default?
>> If so, how do I turn it off, and what are the implication of leaving ON?
>> Won't my searches be slower and consume more memory?
>>
>>
>> Thanks,
>>
>>
>> --MJ
>>
>>
--
Regards,
Dmitry Kan
Re: Is leading wildcard search turned on by default in Solr 3.6.1?
Posted by Jack Krupansky <ja...@basetechnology.com>.
Be sure to realize that even with reverse wildcard support, the user can add
a trailing wildcard as well (double-ended wildcard) and then you are back in
the same boat.
The overall idea is that: 1) Hardware is much faster than just 3 or 4 years
ago, and 2) even though document counts are getting much larger, the number
of unique terms (which is all that matters for wildcard performance) does
not tend to grow as fast as document count grows. And, some fields have a
much more limited vocabulary (unique terms), so a leading wildcard is not
necessarily a big performance hit.
Technology advances. We should permit our mindsets to advance as well.
-- Jack Krupansky
-----Original Message-----
From: François Schiettecatte
Sent: Monday, November 12, 2012 2:38 PM
To: solr-user@lucene.apache.org
Subject: Re: Is leading wildcard search turned on by default in Solr 3.6.1?
John
You can still use leading wildcards even if you dont have the
ReversedWildcardFilterFactory in your analysis but it means you will be
scanning the entire dictionary when the search is run which can be a
performance issue. If you do use ReversedWildcardFilterFactory you wont have
that performance issue but you will increase the overall size of your index.
Its a tradeoff.
When I looked into it for a site I built I decided that the tradeoff was not
worth it (after benchmarking) given how few leading wildcards searches it
was getting.
Best regards
François
On Nov 12, 2012, at 5:33 PM, johnmunir@aol.com wrote:
>
>
> Hi,
>
>
> I'm migrating from Solr 1.2 to 3.6.1. I used the same analyzer as I was,
> and re-indexed my data. I did not add
> solr.ReversedWildcardFilterFactory to my index analyzer, but yet leading
> wild cards are working!! Does this mean it's turned on by default? If
> so, how do I turn it off, and what are the implication of leaving ON?
> Won't my searches be slower and consume more memory?
>
>
> Thanks,
>
>
> --MJ
>
Re: Is leading wildcard search turned on by default in Solr 3.6.1?
Posted by François Schiettecatte <fs...@gmail.com>.
John
You can still use leading wildcards even if you dont have the ReversedWildcardFilterFactory in your analysis but it means you will be scanning the entire dictionary when the search is run which can be a performance issue. If you do use ReversedWildcardFilterFactory you wont have that performance issue but you will increase the overall size of your index. Its a tradeoff.
When I looked into it for a site I built I decided that the tradeoff was not worth it (after benchmarking) given how few leading wildcards searches it was getting.
Best regards
François
On Nov 12, 2012, at 5:33 PM, johnmunir@aol.com wrote:
>
>
> Hi,
>
>
> I'm migrating from Solr 1.2 to 3.6.1. I used the same analyzer as I was, and re-indexed my data. I did not add
> solr.ReversedWildcardFilterFactory to my index analyzer, but yet leading wild cards are working!! Does this mean it's turned on by default? If so, how do I turn it off, and what are the implication of leaving ON? Won't my searches be slower and consume more memory?
>
>
> Thanks,
>
>
> --MJ
>