You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-user@lucene.apache.org by varun srivastava <va...@gmail.com> on 2013/03/06 01:48:40 UTC

dropping fields from input data

Hi,
 I have a usecase where i want to index all the fields except some
blacklisted ones ( which I want to simply drop, as if they never existed in
input data) .. So in my schema I have defined 2 rules

   <dynamicField name="*" type="string" indexed="true" stored="true"
multiValued="true"/>

   <dynamicField name="stamp_*" type="string" indexed="false"
stored="false" multiValued="true"/>

Is there any better way of dropping the fields from input data other than
above ?

Thanks
Varun

Re: dropping fields from input data

Posted by Fredrik Rødland <so...@rodland.no>.

6. mars 2013 kl. 02:24 skrev varun srivastava <va...@gmail.com>:

> Thanks Hoss .. Is this available in 4.0 ?
> 
> On Tue, Mar 5, 2013 at 5:14 PM, Chris Hostetter <ho...@fucit.org>wrote:
> 
>> 
>> :    <dynamicField name="stamp_*" type="string" indexed="false"
>> : stored="false" multiValued="true"/>
>> 
>> Take a look at IgnoreFieldUpdateProcessorFactory...
>> 
>> 
>> https://lucene.apache.org/solr/4_1_0/solr-core/org/apache/solr/update/processor/IgnoreFieldUpdateProcessorFactory.html
>> 
>> https://lucene.apache.org/solr/4_1_0/solr-core/org/apache/solr/update/processor/FieldMutatingUpdateProcessorFactory.html


Yes:
https://lucene.apache.org/solr/4_0_0/solr-core/org/apache/solr/update/processor/IgnoreFieldUpdateProcessorFactory.html
https://lucene.apache.org/solr/4_0_0/solr-core/org/apache/solr/update/processor/FieldMutatingUpdateProcessorFactory.html


Fredrik


--
Fredrik Rødland               Mail:    fredrik@rodland.no
                              Cell:    +47 99 21 98 17
                              Twitter: @fredrikr
Maisen Pedersens vei 1        Flickr:  http://www.flickr.com/fmmr/
NO-1363 Høvik, NORWAY         Web:     http://about.me/fmr

Re: dropping fields from input data

Posted by varun srivastava <va...@gmail.com>.

Thanks Hoss .. Is this available in 4.0 ?

On Tue, Mar 5, 2013 at 5:14 PM, Chris Hostetter <ho...@fucit.org>wrote:

>
> :    <dynamicField name="stamp_*" type="string" indexed="false"
> : stored="false" multiValued="true"/>
>
> Take a look at IgnoreFieldUpdateProcessorFactory...
>
>
> https://lucene.apache.org/solr/4_1_0/solr-core/org/apache/solr/update/processor/IgnoreFieldUpdateProcessorFactory.html
>
> https://lucene.apache.org/solr/4_1_0/solr-core/org/apache/solr/update/processor/FieldMutatingUpdateProcessorFactory.html
>
> Using that (instead of setting indexed=false stored=false in schema.xml)
> has the advantage that you can use it to throw away fields early in the
> update processor pipeline, before any distributed logic happens in
> SolrCloud.
>
>
> -Hoss
>

Re: dropping fields from input data

Posted by Chris Hostetter <ho...@fucit.org>.

:    <dynamicField name="stamp_*" type="string" indexed="false"
: stored="false" multiValued="true"/>

Take a look at IgnoreFieldUpdateProcessorFactory...

https://lucene.apache.org/solr/4_1_0/solr-core/org/apache/solr/update/processor/IgnoreFieldUpdateProcessorFactory.html
https://lucene.apache.org/solr/4_1_0/solr-core/org/apache/solr/update/processor/FieldMutatingUpdateProcessorFactory.html

Using that (instead of setting indexed=false stored=false in schema.xml) 
has the advantage that you can use it to throw away fields early in the 
update processor pipeline, before any distributed logic happens in 
SolrCloud.


-Hoss