You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by kirsty <ki...@gmail.com> on 2010/05/27 09:11:16 UTC
Help with PatternReplaceFilterFactory
Hi,
I have a field that is a text field eg: R500,000-550,000 Per Annum,
R350,000-550,000 Per Annum Cost To Company etc.
I would like to facet on the salary range.
I have created a new field type
<fieldType name="remuneration" class="solr.TextField" sortMissingLast="true"
omitNorms="true">
<analyzer>
<tokenizer class="solr.KeywordTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.TrimFilterFactory" />
<filter class="solr.PatternReplaceFilterFactory" pattern="([a-z])"
replacement="" replace="all" />
</analyzer>
</fieldType>
to remove all the letters.
I have to two fields like this:
<field name="Remuneration" type="string" indexed="true" stored="true"/>
<field name="Remuneration_strip" type="remuneration" indexed="true"
stored="true"/>
And then the copy field:
<copyField source="Remuneration" dest="Remuneration_strip"/>
But my index still has all the text. Am I misunderstanding? Where have I
gone wrong? Any help would be greatly appreciated!
Kirsty
--
View this message in context: http://lucene.472066.n3.nabble.com/Help-with-PatternReplaceFilterFactory-tp847408p847408.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Help with PatternReplaceFilterFactory
Posted by Lance Norskog <go...@gmail.com>.
You probably want to store these as numbers instead of text. The
DataImportHandler allows you to take apart text blocks and save the
parts in number fields. The text analysis stack inside basic indexing
will not do this.
On Thu, May 27, 2010 at 7:24 AM, Koji Sekiguchi <ko...@r.email.ne.jp> wrote:
>
>> Yes you are right, I get that type of result. I guess my wording was
>> wrong.
>> My field looks like this in the index:
>> <str name="Remuneration">R500,000-550,000 Per Annum</str>
>> <str name="Remuneration_strip">R500,000-550,000 Per Annum</str>
>>
>> How would I search for say salaries in the range of 500,000 - 550,000?
>> Trying fq=Rumeration_strip:500,000-550,00 doesn't bring back anything. I
>> must have something wrong.
>>
>>
>
> So you are not asking facet...
>
> For query syntax of range search, take a look at:
>
> http://lucene.apache.org/java/2_9_1/queryparsersyntax.html#Range%20Searches
>
> And you need to index the lower and upper salary to separate fields i.e.
>
> low:500000
> up:550000
>
> Then you can search the both of the fields e.g.
>
> q=low:[500000 TO *] AND up:[* TO 550000]
>
> Koji
>
> --
> http://www.rondhuit.com/en/
>
>
--
Lance Norskog
goksron@gmail.com
Re: Help with PatternReplaceFilterFactory
Posted by Koji Sekiguchi <ko...@r.email.ne.jp>.
> Yes you are right, I get that type of result. I guess my wording was wrong.
> My field looks like this in the index:
> <str name="Remuneration">R500,000-550,000 Per Annum</str>
> <str name="Remuneration_strip">R500,000-550,000 Per Annum</str>
>
> How would I search for say salaries in the range of 500,000 - 550,000?
> Trying fq=Rumeration_strip:500,000-550,00 doesn't bring back anything. I
> must have something wrong.
>
>
So you are not asking facet...
For query syntax of range search, take a look at:
http://lucene.apache.org/java/2_9_1/queryparsersyntax.html#Range%20Searches
And you need to index the lower and upper salary to separate fields i.e.
low:500000
up:550000
Then you can search the both of the fields e.g.
q=low:[500000 TO *] AND up:[* TO 550000]
Koji
--
http://www.rondhuit.com/en/
Re: Help with PatternReplaceFilterFactory
Posted by kirsty <ki...@gmail.com>.
Koji Sekiguchi wrote:
>
> (10/05/27 16:11), kirsty wrote:
>> Hi,
>> I have a field that is a text field eg: R500,000-550,000 Per Annum,
>> R350,000-550,000 Per Annum Cost To Company etc.
>> I would like to facet on the salary range.
>> I have created a new field type
>> <fieldType name="remuneration" class="solr.TextField"
>> sortMissingLast="true"
>> omitNorms="true">
>> <analyzer>
>> <tokenizer class="solr.KeywordTokenizerFactory"/>
>> <filter class="solr.LowerCaseFilterFactory" />
>> <filter class="solr.TrimFilterFactory" />
>> <filter class="solr.PatternReplaceFilterFactory" pattern="([a-z])"
>> replacement="" replace="all" />
>> </analyzer>
>> </fieldType>
>> to remove all the letters.
>>
>> I have to two fields like this:
>> <field name="Remuneration" type="string" indexed="true" stored="true"/>
>> <field name="Remuneration_strip" type="remuneration" indexed="true"
>> stored="true"/>
>>
>> And then the copy field:
>> <copyField source="Remuneration" dest="Remuneration_strip"/>
>>
>> But my index still has all the text. Am I misunderstanding? Where have I
>> gone wrong? Any help would be greatly appreciated!
>> Kirsty
>>
>>
> What do you mean by "my index still has all the text "?
> With your schema above, I think you can get a facet result eg:
>
> <lst name="facet_fields">
> <lst name="Remuneration_strip">
> <int name="350,000-550,000 ">1</int>
> <int name="500,000-550,000 ">1</int>
> </lst>
> </lst>
>
> when you request q=*:*&facet=on&facet.field=Remuneration_strip
>
> Koji
>
> --
> http://www.rondhuit.com/en/
>
>
>
Yes you are right, I get that type of result. I guess my wording was wrong.
My field looks like this in the index:
<str name="Remuneration">R500,000-550,000 Per Annum</str>
<str name="Remuneration_strip">R500,000-550,000 Per Annum</str>
How would I search for say salaries in the range of 500,000 - 550,000?
Trying fq=Rumeration_strip:500,000-550,00 doesn't bring back anything. I
must have something wrong.
--
View this message in context: http://lucene.472066.n3.nabble.com/Help-with-PatternReplaceFilterFactory-tp847408p848078.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Help with PatternReplaceFilterFactory
Posted by Koji Sekiguchi <ko...@r.email.ne.jp>.
(10/05/27 16:11), kirsty wrote:
> Hi,
> I have a field that is a text field eg: R500,000-550,000 Per Annum,
> R350,000-550,000 Per Annum Cost To Company etc.
> I would like to facet on the salary range.
> I have created a new field type
> <fieldType name="remuneration" class="solr.TextField" sortMissingLast="true"
> omitNorms="true">
> <analyzer>
> <tokenizer class="solr.KeywordTokenizerFactory"/>
> <filter class="solr.LowerCaseFilterFactory" />
> <filter class="solr.TrimFilterFactory" />
> <filter class="solr.PatternReplaceFilterFactory" pattern="([a-z])"
> replacement="" replace="all" />
> </analyzer>
> </fieldType>
> to remove all the letters.
>
> I have to two fields like this:
> <field name="Remuneration" type="string" indexed="true" stored="true"/>
> <field name="Remuneration_strip" type="remuneration" indexed="true"
> stored="true"/>
>
> And then the copy field:
> <copyField source="Remuneration" dest="Remuneration_strip"/>
>
> But my index still has all the text. Am I misunderstanding? Where have I
> gone wrong? Any help would be greatly appreciated!
> Kirsty
>
>
What do you mean by "my index still has all the text "?
With your schema above, I think you can get a facet result eg:
<lst name="facet_fields">
<lst name="Remuneration_strip">
<int name="350,000-550,000 ">1</int>
<int name="500,000-550,000 ">1</int>
</lst>
</lst>
when you request q=*:*&facet=on&facet.field=Remuneration_strip
Koji
--
http://www.rondhuit.com/en/