You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jean-Sebastien Vachon <je...@wantedanalytics.com> on 2014/04/15 21:57:35 UTC
Transformation on a numeric field
Hi All,
I am looking for a way to index a numeric field and its value divided by 1 000 into another numeric field.
I thought about using a CopyField with a PatternReplaceFilterFactory to keep only the first few digits (cutting the last three).
Solr complains that I can not have an analysis chain on a numeric field:
Core: org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Plugin init failure for [schema.xml] fieldType "truncated_salary": FieldType: TrieIntField (truncated_salary) does not support specifying an analyzer. Schema file is /data/solr/solr-no-cloud/Core1/schema.xml
Is there a way to accomplish this ?
Thanks
Re: Transformation on a numeric field
Posted by Rafał Kuć <r....@solr.pl>.
Hello!
You can achieve that using update processor, for example look here: http://wiki.apache.org/solr/ScriptUpdateProcessor
What you would have to do, in general, is create a script that would
take a value of the field, divide it by the 1000 and put it in another
field - the target numeric field.
--
Regards,
Rafał Kuć
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/
> Hi All,
> I am looking for a way to index a numeric field and its value
> divided by 1 000 into another numeric field.
> I thought about using a CopyField with a
> PatternReplaceFilterFactory to keep only the first few digits (cutting the last three).
> Solr complains that I can not have an analysis chain on a numeric field:
> Core:
> org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> Plugin init failure for [schema.xml] fieldType "truncated_salary":
> FieldType: TrieIntField (truncated_salary) does not support
> specifying an analyzer. Schema file is
> /data/solr/solr-no-cloud/Core1/schema.xml
> Is there a way to accomplish this ?
> Thanks
RE: Transformation on a numeric field
Posted by Chris Hostetter <ho...@fucit.org>.
: Anyone knows why we can`t have an analysis chain on a numeric field ?
: Looks to me like it would be very useful to be able to
: manipulate/transform a value without an external resources.
Analysis only affects *indexed* terms -- it has no impact on the stored
values (or things like docValues). UpdateProcessors are the correct place
to make changes like this.
Analysis Factories have existed a lot longer in SOlr then
UpdateProcessors, so there are a lot more of them -- but the toolkit of
available UpdateProcessors is devent and growing larger each release.
If you really want to "divide by 100" then a script based processor that
does math is the correct way to go -- if you just want to truncate the
last 2 characters of a string, you could use the
RegexReplaceProcessorFactory -- but that might not behave the way you
expect if your client is really sending you in a numeric value (via SolrJ,
or DIH, or the JSON input format)
-Hoss
http://www.lucidworks.com/
RE: Transformation on a numeric field
Posted by Jean-Sebastien Vachon <je...@wantedanalytics.com>.
Thanks for the information. I will look into this but I`m curious to know why something this basic requires an external script...
Anyone knows why we can`t have an analysis chain on a numeric field ? Looks to me like it would be very useful to be able to manipulate/transform a value without an external resources.
Thanks
> -----Original Message-----
> From: Jack Krupansky [mailto:jack@basetechnology.com]
> Sent: April-15-14 4:36 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Transformation on a numeric field
>
> You can use an update processor. The stateless script update processor will
> let you write arbitrary JavaScript code, which can do this calculation.
>
> You should be able to figure it out from the wiki:
> http://wiki.apache.org/solr/ScriptUpdateProcessor
>
> My e-book has plenty of script examples for this processor as well.
>
> We could also write a generic script that takes a source and destination field
> name and then does a specified operation on it, like add an offset or multiple
> by a scale factor.
>
> -- Jack Krupansky
>
> -----Original Message-----
> From: Jean-Sebastien Vachon
> Sent: Tuesday, April 15, 2014 3:57 PM
> To: 'solr-user@lucene.apache.org'
> Subject: Transformation on a numeric field
>
> Hi All,
>
> I am looking for a way to index a numeric field and its value divided by 1
> 000 into another numeric field.
> I thought about using a CopyField with a PatternReplaceFilterFactory to keep
> only the first few digits (cutting the last three).
>
> Solr complains that I can not have an analysis chain on a numeric field:
>
> Core:
> org.apache.solr.common.SolrException:org.apache.solr.common.SolrExcept
> ion:
> Plugin init failure for [schema.xml] fieldType "truncated_salary":
> FieldType: TrieIntField (truncated_salary) does not support specifying an
> analyzer. Schema file is /data/solr/solr-no-cloud/Core1/schema.xml
>
>
> Is there a way to accomplish this ?
>
> Thanks
>
>
> -----
> Aucun virus trouvé dans ce message.
> Analyse effectuée par AVG - www.avg.fr
> Version: 2014.0.4355 / Base de données virale: 3882/7323 - Date:
> 09/04/2014
Re: Transformation on a numeric field
Posted by Jack Krupansky <ja...@basetechnology.com>.
You can use an update processor. The stateless script update processor will
let you write arbitrary JavaScript code, which can do this calculation.
You should be able to figure it out from the wiki:
http://wiki.apache.org/solr/ScriptUpdateProcessor
My e-book has plenty of script examples for this processor as well.
We could also write a generic script that takes a source and destination
field name and then does a specified operation on it, like add an offset or
multiple by a scale factor.
-- Jack Krupansky
-----Original Message-----
From: Jean-Sebastien Vachon
Sent: Tuesday, April 15, 2014 3:57 PM
To: 'solr-user@lucene.apache.org'
Subject: Transformation on a numeric field
Hi All,
I am looking for a way to index a numeric field and its value divided by 1
000 into another numeric field.
I thought about using a CopyField with a PatternReplaceFilterFactory to keep
only the first few digits (cutting the last three).
Solr complains that I can not have an analysis chain on a numeric field:
Core:
org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
Plugin init failure for [schema.xml] fieldType "truncated_salary":
FieldType: TrieIntField (truncated_salary) does not support specifying an
analyzer. Schema file is /data/solr/solr-no-cloud/Core1/schema.xml
Is there a way to accomplish this ?
Thanks