You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Alex Sylka <sy...@gmail.com> on 2015/03/27 13:32:26 UTC
Solr advanced StopFilterFactory
We need advanced stop words filter in Solr.
We need stopwords to be stored in db and ability to change them by users
(each user should have own stopwords). That's why I am thinking about
sending stop words to solr from our app or connect to our db from solr and
use updated stop words in custom StopFilterFactory.
Also each user will have own stopwords list which will be stored in mysql
db "stopwords" table. (id, user_id, stopword).
We have next index structure. This index will store data for all users.
<field name="user_id" type="int" indexed="true" stored="true"
required="true" multiValued="false" />
<field name="tag_name" type="text_general" indexed="true"
stored="true" required="false" multiValued="false"/>
...
<field name="tag_description" type="text_general" indexed="true"
stored="true" required="false" multiValued="false"/>
I am not sure how to achive behaviour described above but I am thinking
about writing own custom StopFilterFactory which will grab stopwords from
db and use different stopwords for users while indexing their documents.
What you can suggest ? Is that possible ? Am I on right way ?
Re: Solr advanced StopFilterFactory
Posted by Timothy Potter <th...@gmail.com>.
Seems like you should be able to use the ManagedStopFilterFactory with
a custom StorageIO impl that pulls from your db:
http://lucene.apache.org/solr/5_1_0/solr-core/index.html?org/apache/solr/rest/ManagedResourceStorage.StorageIO.html
On Thu, May 28, 2015 at 7:03 AM, Alessandro Benedetti
<be...@gmail.com> wrote:
> As Alex initially specified , the custom stop filter factory is the right
> way !
> So is mainly related to the suggester ?
> Anyway with a custom stop filter, it can be possible and actually can be a
> nice contribution as well.
>
> Cheers
>
>
> 2015-05-28 13:01 GMT+01:00 Rupali <ru...@gmail.com>:
>
>> sylkaalex <sylkaalex <at> gmail.com> writes:
>>
>> >
>> > The main goal to allow each user use own stop words list. For example
>> user
>> > type "th"
>> > now he will see next results in his terms search:
>> > the
>> > the one
>> > the then
>> > then
>> > then and
>> >
>> > But user has stop word "the" and he want get next results:
>> > then
>> > then and
>> >
>> > --
>> > View this message in context: http://lucene.472066.n3.nabble.com/Solr-
>> advanced-StopFilterFactory-tp4195797p4195855.html
>> > Sent from the Solr - User mailing list archive at Nabble.com.
>> >
>> >
>>
>> Hi,
>>
>> Is there any way to get the stopwords from database? Checking options to
>> get the list from database and use that list as stopwords into
>> spellcheck component.
>>
>> Thanks in advance.
>>
>>
>>
>>
>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card : http://about.me/alessandro_benedetti
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
Re: Solr advanced StopFilterFactory
Posted by Alessandro Benedetti <be...@gmail.com>.
As Alex initially specified , the custom stop filter factory is the right
way !
So is mainly related to the suggester ?
Anyway with a custom stop filter, it can be possible and actually can be a
nice contribution as well.
Cheers
2015-05-28 13:01 GMT+01:00 Rupali <ru...@gmail.com>:
> sylkaalex <sylkaalex <at> gmail.com> writes:
>
> >
> > The main goal to allow each user use own stop words list. For example
> user
> > type "th"
> > now he will see next results in his terms search:
> > the
> > the one
> > the then
> > then
> > then and
> >
> > But user has stop word "the" and he want get next results:
> > then
> > then and
> >
> > --
> > View this message in context: http://lucene.472066.n3.nabble.com/Solr-
> advanced-StopFilterFactory-tp4195797p4195855.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
> >
> >
>
> Hi,
>
> Is there any way to get the stopwords from database? Checking options to
> get the list from database and use that list as stopwords into
> spellcheck component.
>
> Thanks in advance.
>
>
>
>
--
--------------------------
Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti
"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"
William Blake - Songs of Experience -1794 England
Re: Solr advanced StopFilterFactory
Posted by Rupali <ru...@gmail.com>.
sylkaalex <sylkaalex <at> gmail.com> writes:
>
> The main goal to allow each user use own stop words list. For example
user
> type "th"
> now he will see next results in his terms search:
> the
> the one
> the then
> then
> then and
>
> But user has stop word "the" and he want get next results:
> then
> then and
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Solr-
advanced-StopFilterFactory-tp4195797p4195855.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
Hi,
Is there any way to get the stopwords from database? Checking options to
get the list from database and use that list as stopwords into
spellcheck component.
Thanks in advance.
Re: Solr advanced StopFilterFactory
Posted by sylkaalex <sy...@gmail.com>.
The main goal to allow each user use own stop words list. For example user
type "th"
now he will see next results in his terms search:
the
the one
the then
then
then and
But user has stop word "the" and he want get next results:
then
then and
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-advanced-StopFilterFactory-tp4195797p4195855.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr advanced StopFilterFactory
Posted by Erik Hatcher <er...@gmail.com>.
Alex - that’s definitely possible, with performance being the main consideration here.
But since this is for query time stop words, maybe instead your fronting application could take the users list and remove those words from the query before sending it to Solr?
I’m curious what the ultimate goal / use case is for this feature, which may help us better guide you on ways to do what you need.
—
Erik Hatcher, Senior Solutions Architect
http://www.lucidworks.com <http://www.lucidworks.com/>
> On Mar 27, 2015, at 8:32 AM, Alex Sylka <sy...@gmail.com> wrote:
>
> We need advanced stop words filter in Solr.
>
> We need stopwords to be stored in db and ability to change them by users
> (each user should have own stopwords). That's why I am thinking about
> sending stop words to solr from our app or connect to our db from solr and
> use updated stop words in custom StopFilterFactory.
>
> Also each user will have own stopwords list which will be stored in mysql
> db "stopwords" table. (id, user_id, stopword).
>
> We have next index structure. This index will store data for all users.
>
> <field name="user_id" type="int" indexed="true" stored="true"
> required="true" multiValued="false" />
> <field name="tag_name" type="text_general" indexed="true"
> stored="true" required="false" multiValued="false"/>
> ...
> <field name="tag_description" type="text_general" indexed="true"
> stored="true" required="false" multiValued="false"/>
>
> I am not sure how to achive behaviour described above but I am thinking
> about writing own custom StopFilterFactory which will grab stopwords from
> db and use different stopwords for users while indexing their documents.
>
> What you can suggest ? Is that possible ? Am I on right way ?