Re: Generic RemoveDuplicatesTokenFilter

[astubbs <an...@gmail.com>]

That's exactly what I need. I'm using phonetic tokens on ngrams, and there's
lots of dupes. Can you submit it as a patch? What's the easiest way to get
this into my solr?

--
View this message in context: http://lucene.472066.n3.nabble.com/Generic-RemoveDuplicatesTokenFilter-tp3581656p3632499.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Generic RemoveDuplicatesTokenFilter

[Erick Erickson <er...@gmail.com>]

@Pravesh
That look reasonable. Of course you could extend it
to do many things. I'm assuming you've just created
a plugin that you use rather than compile this into
the Solr code, right?

@astubbs
I'd probably use a TokenFilter(Factory) implementation
as a plugin as I think pravesh has.
It would also be possible to use a TokenizerFactory,
depending on where you need this to happen, but FilterFactory
is my first choice, they allow more flexibility.

I rather doubt a patch on this order will make it into the code,
it's rather special-purpose.

Best
Erick

On Wed, Jan 4, 2012 at 12:32 PM, astubbs <an...@gmail.com> wrote:
> That's exactly what I need. I'm using phonetic tokens on ngrams, and there's
> lots of dupes. Can you submit it as a patch? What's the easiest way to get
> this into my solr?
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Generic-RemoveDuplicatesTokenFilter-tp3581656p3632499.html
> Sent from the Solr - User mailing list archive at Nabble.com.