You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by revas <re...@gmail.com> on 2009/03/17 12:40:16 UTC
stop word search
Hi,
I have a query like this
content:the AND iuser_id:5
which means return all docs of user id 5 which have the word "the" in
content .Since 'the' is a stop word ,this query executes as just user_id :5
inspite of the "AND" clause ,Whereas the expected result here is since there
is no result for "the " ,no results shloud be returned.
Am i missing anythin here?
Regards
Re: stop word search
Posted by revas <re...@gmail.com>.
Hi Erick,
I still don't get it.The scenario is like this.
Intially i indexed the content with the stop word filter at both index time
and query time.That means
the stop words are not there in the index .
Now i removed the stop filter only at query time so that a query like this
will not fetch results
content:the AND id:8 as due to the stop filter this query becomes just id:8
and returns results.
Why would i have to reindex as there shloud not be any stop words in the
index in the first place.
Thanks for your time .
Regards
On 3/21/09, Erick Erickson <er...@gmail.com> wrote:
>
> Yes, you do need to reindex after removing the stopword filter
> from the configuration. When you indexed the first time using
> the stopword filter, the words were NOT indexed, so they won't
> be found now that they're getting through the query analyzer.
>
> Best
> Erick
>
> On Fri, Mar 20, 2009 at 1:02 PM, revas <re...@gmail.com> wrote:
>
> > Hi Erik,
> >
> > I have now commented the query time stopword analyzer .I restarted the
> > server.But now when i search for a stop word ,i am getting results.
> >
> > We had earlier indexed the content with the stop word analyzer.I dont
> think
> > we need to reindex after commentting the query analyzer,right?
> >
> > This field is a text field with the defaul analyzer.
> >
> > Please let me know if i have missed something here.
> >
> > Regards
> > Sujatha
> >
> >
> > On 3/17/09, Erick Erickson <er...@gmail.com> wrote:
> > >
> > > Well, by definition, using an analyzer that removes stopwords
> > > *should* do this at query time. This assumes that you used
> > > an analyzer that removed stopwords at index and query time.
> > > The stopwords are not in the index.
> > >
> > > You can get the behavior you expect by using an analyzer at
> > > query time that does NOT remove stopwords, and one at
> > > indexing time that *does* remove stopwords. Gut I'm having a
> > > hard time imagining that this would result in a good user experience.
> > >
> > > I mean anytime that you had a stopword in the query where the
> > > stopword was required, no results would be returned. Which would
> > > be hard to explain to a user....
> > >
> > > What is it you're trying to accomplish?
> > >
> > > Best
> > > Erick
> > >
> > >
> > >
> > > On Tue, Mar 17, 2009 at 7:40 AM, revas <re...@gmail.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > I have a query like this
> > > >
> > > > content:the AND iuser_id:5
> > > >
> > > > which means return all docs of user id 5 which have the word "the" in
> > > > content .Since 'the' is a stop word ,this query executes as just
> > user_id
> > > :5
> > > > inspite of the "AND" clause ,Whereas the expected result here is
> since
> > > > there
> > > > is no result for "the " ,no results shloud be returned.
> > > >
> > > > Am i missing anythin here?
> > > >
> > > > Regards
> > > >
> > >
> >
>
Re: stop word search
Posted by Erick Erickson <er...@gmail.com>.
Yes, you do need to reindex after removing the stopword filter
from the configuration. When you indexed the first time using
the stopword filter, the words were NOT indexed, so they won't
be found now that they're getting through the query analyzer.
Best
Erick
On Fri, Mar 20, 2009 at 1:02 PM, revas <re...@gmail.com> wrote:
> Hi Erik,
>
> I have now commented the query time stopword analyzer .I restarted the
> server.But now when i search for a stop word ,i am getting results.
>
> We had earlier indexed the content with the stop word analyzer.I dont think
> we need to reindex after commentting the query analyzer,right?
>
> This field is a text field with the defaul analyzer.
>
> Please let me know if i have missed something here.
>
> Regards
> Sujatha
>
>
> On 3/17/09, Erick Erickson <er...@gmail.com> wrote:
> >
> > Well, by definition, using an analyzer that removes stopwords
> > *should* do this at query time. This assumes that you used
> > an analyzer that removed stopwords at index and query time.
> > The stopwords are not in the index.
> >
> > You can get the behavior you expect by using an analyzer at
> > query time that does NOT remove stopwords, and one at
> > indexing time that *does* remove stopwords. Gut I'm having a
> > hard time imagining that this would result in a good user experience.
> >
> > I mean anytime that you had a stopword in the query where the
> > stopword was required, no results would be returned. Which would
> > be hard to explain to a user....
> >
> > What is it you're trying to accomplish?
> >
> > Best
> > Erick
> >
> >
> >
> > On Tue, Mar 17, 2009 at 7:40 AM, revas <re...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I have a query like this
> > >
> > > content:the AND iuser_id:5
> > >
> > > which means return all docs of user id 5 which have the word "the" in
> > > content .Since 'the' is a stop word ,this query executes as just
> user_id
> > :5
> > > inspite of the "AND" clause ,Whereas the expected result here is since
> > > there
> > > is no result for "the " ,no results shloud be returned.
> > >
> > > Am i missing anythin here?
> > >
> > > Regards
> > >
> >
>
Re: stop word search
Posted by revas <re...@gmail.com>.
Hi Erik,
I have now commented the query time stopword analyzer .I restarted the
server.But now when i search for a stop word ,i am getting results.
We had earlier indexed the content with the stop word analyzer.I dont think
we need to reindex after commentting the query analyzer,right?
This field is a text field with the defaul analyzer.
Please let me know if i have missed something here.
Regards
Sujatha
On 3/17/09, Erick Erickson <er...@gmail.com> wrote:
>
> Well, by definition, using an analyzer that removes stopwords
> *should* do this at query time. This assumes that you used
> an analyzer that removed stopwords at index and query time.
> The stopwords are not in the index.
>
> You can get the behavior you expect by using an analyzer at
> query time that does NOT remove stopwords, and one at
> indexing time that *does* remove stopwords. Gut I'm having a
> hard time imagining that this would result in a good user experience.
>
> I mean anytime that you had a stopword in the query where the
> stopword was required, no results would be returned. Which would
> be hard to explain to a user....
>
> What is it you're trying to accomplish?
>
> Best
> Erick
>
>
>
> On Tue, Mar 17, 2009 at 7:40 AM, revas <re...@gmail.com> wrote:
>
> > Hi,
> >
> > I have a query like this
> >
> > content:the AND iuser_id:5
> >
> > which means return all docs of user id 5 which have the word "the" in
> > content .Since 'the' is a stop word ,this query executes as just user_id
> :5
> > inspite of the "AND" clause ,Whereas the expected result here is since
> > there
> > is no result for "the " ,no results shloud be returned.
> >
> > Am i missing anythin here?
> >
> > Regards
> >
>
Re: stop word search
Posted by Erick Erickson <er...@gmail.com>.
Well, by definition, using an analyzer that removes stopwords
*should* do this at query time. This assumes that you used
an analyzer that removed stopwords at index and query time.
The stopwords are not in the index.
You can get the behavior you expect by using an analyzer at
query time that does NOT remove stopwords, and one at
indexing time that *does* remove stopwords. Gut I'm having a
hard time imagining that this would result in a good user experience.
I mean anytime that you had a stopword in the query where the
stopword was required, no results would be returned. Which would
be hard to explain to a user....
What is it you're trying to accomplish?
Best
Erick
On Tue, Mar 17, 2009 at 7:40 AM, revas <re...@gmail.com> wrote:
> Hi,
>
> I have a query like this
>
> content:the AND iuser_id:5
>
> which means return all docs of user id 5 which have the word "the" in
> content .Since 'the' is a stop word ,this query executes as just user_id :5
> inspite of the "AND" clause ,Whereas the expected result here is since
> there
> is no result for "the " ,no results shloud be returned.
>
> Am i missing anythin here?
>
> Regards
>