You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Akeel <ak...@gmail.com> on 2008/05/21 07:50:49 UTC
What are stopwords and protwords ???
Hi,
I am a beginner to Solr, I have successfully indexed my db in solr. I want
to know that what are the stopwords and protwords ??? and how much they have
effect on my search results ?
Thanks in advance.
--
Akeel
Re: What are stopwords and protwords ???
Posted by gurudev <su...@yahoo.com>.
Hi Akeel
-Stopwords are general words of language, which, as such do not contain any
meaning in searches like; a,an, the, where, who, am etc. The analyzer in
lucene ignores such words and do not index them. You can also specify you
own stopwords in stopwords.txt in SOLR
-Protwords are the words which you do not want to be stemmed (In stemming
case manager/managing/managed/manageable all are indexed as ---> manag. Same
thing goes in case of searching. In case you do not want a particular word
to be stemmed at index/search time just put it in protwords.txt of SOLR.
Akeel wrote:
>
> Hi,
>
> I am a beginner to Solr, I have successfully indexed my db in solr. I want
> to know that what are the stopwords and protwords ??? and how much they
> have
> effect on my search results ?
>
>
>
> Thanks in advance.
>
>
>
> --
>
> Akeel
>
>
>
--
View this message in context: http://www.nabble.com/What-are-stopwords-and-protwords-----tp17356365p17364189.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: What are stopwords and protwords ???
Posted by Akeel <ak...@gmail.com>.
thanks everyone
On Thu, May 22, 2008 at 7:18 AM, Grant Ingersoll <gs...@apache.org>
wrote:
> See http://lucene.apache.org/solr/tutorial.html. You can also see the
> wiki for a whole bunch of docs, including links to tutorials, etc.
>
> Also, just for future reference, please separate out questions so that they
> can be addressed separately, and more easily found by others in the future.
>
> -Grant
>
>
> On May 21, 2008, at 7:45 AM, Akeel wrote:
>
> Thank you very much for such a detailed reply. can you please tell me how
>> can i interact with solr from within my Java/JSP application ? I mean how
>> to
>> query the solr running at localhost and getting results back in the
>> application. Do i have to change something there in solrconfig.xml ?
>> Please
>> help me in this regards
>>
>> Thanks in advance
>>
>> --
>> Akeel
>>
>> On Wed, May 21, 2008 at 4:11 PM, Grant Ingersoll <gs...@apache.org>
>> wrote:
>>
>> Stopwords are commonly occurring words that don't add _much_ value to
>>> search, such as the, an, a and are usually removed during analysis.
>>> Protwords (protected words) are words that would be stemmed by the
>>> English
>>> porter stemmer that you do not want to be stemmed.
>>>
>>> In the end, removing stopwords may keep your index smaller and can keep
>>> some queries from taking a long time, but they also mean you can't query
>>> for
>>> those words. As for protwords, that is something you would do if you
>>> felt
>>> the results for those tokens was "off".
>>>
>>> Many people use stopwords, many don't. Personally, I don't think
>>> removing
>>> them is the right thing to do, as there isn't always a way to recover
>>> them
>>> and they do provide meaning, otherwise why would they be needed in the
>>> language? Often, the best thing to do, is keep stopwords, but handle
>>> them
>>> intelligently on the query side (in phrases, etc.). However, since
>>> you're a
>>> beginner, it probably makes sense to just throw out stopwords for now.
>>>
>>> -Grant
>>>
>>>
>>> On May 21, 2008, at 1:50 AM, Akeel wrote:
>>>
>>> Hi,
>>>
>>>>
>>>> I am a beginner to Solr, I have successfully indexed my db in solr. I
>>>> want
>>>> to know that what are the stopwords and protwords ??? and how much they
>>>> have
>>>> effect on my search results ?
>>>>
>>>>
>>>>
>>>> Thanks in advance.
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Akeel
>>>>
>>>>
>>>>
>>>
>>
>> --
>> Thanks and Regards,
>> Akeel ur Rehman Faridee
>> http://riseofpakistan.blogspot.com
>> cell: 0321-4714151
>> ====================================================
>> When there is injustice in society, then everyone will go to politics
>> Except the two kinds: those who are timid and those who are materialist
>> (Aristotle)
>> ====================================================
>>
>
> --------------------------
> Grant Ingersoll
>
> Lucene Helpful Hints:
> http://wiki.apache.org/lucene-java/BasicsOfPerformance
> http://wiki.apache.org/lucene-java/LuceneFAQ
>
>
>
>
>
>
>
--
Thanks and Regards,
Akeel ur Rehman Faridee
http://riseofpakistan.blogspot.com
cell: 0321-4714151
====================================================
When there is injustice in society, then everyone will go to politics
Except the two kinds: those who are timid and those who are materialist
(Aristotle)
====================================================
Re: What are stopwords and protwords ???
Posted by Grant Ingersoll <gs...@apache.org>.
See http://lucene.apache.org/solr/tutorial.html. You can also see the
wiki for a whole bunch of docs, including links to tutorials, etc.
Also, just for future reference, please separate out questions so that
they can be addressed separately, and more easily found by others in
the future.
-Grant
On May 21, 2008, at 7:45 AM, Akeel wrote:
> Thank you very much for such a detailed reply. can you please tell
> me how
> can i interact with solr from within my Java/JSP application ? I
> mean how to
> query the solr running at localhost and getting results back in the
> application. Do i have to change something there in solrconfig.xml ?
> Please
> help me in this regards
>
> Thanks in advance
>
> --
> Akeel
>
> On Wed, May 21, 2008 at 4:11 PM, Grant Ingersoll <gs...@apache.org>
> wrote:
>
>> Stopwords are commonly occurring words that don't add _much_ value to
>> search, such as the, an, a and are usually removed during analysis.
>> Protwords (protected words) are words that would be stemmed by the
>> English
>> porter stemmer that you do not want to be stemmed.
>>
>> In the end, removing stopwords may keep your index smaller and can
>> keep
>> some queries from taking a long time, but they also mean you can't
>> query for
>> those words. As for protwords, that is something you would do if
>> you felt
>> the results for those tokens was "off".
>>
>> Many people use stopwords, many don't. Personally, I don't think
>> removing
>> them is the right thing to do, as there isn't always a way to
>> recover them
>> and they do provide meaning, otherwise why would they be needed in
>> the
>> language? Often, the best thing to do, is keep stopwords, but
>> handle them
>> intelligently on the query side (in phrases, etc.). However, since
>> you're a
>> beginner, it probably makes sense to just throw out stopwords for
>> now.
>>
>> -Grant
>>
>>
>> On May 21, 2008, at 1:50 AM, Akeel wrote:
>>
>> Hi,
>>>
>>> I am a beginner to Solr, I have successfully indexed my db in
>>> solr. I want
>>> to know that what are the stopwords and protwords ??? and how much
>>> they
>>> have
>>> effect on my search results ?
>>>
>>>
>>>
>>> Thanks in advance.
>>>
>>>
>>>
>>> --
>>>
>>> Akeel
>>>
>>>
>>
>
>
> --
> Thanks and Regards,
> Akeel ur Rehman Faridee
> http://riseofpakistan.blogspot.com
> cell: 0321-4714151
> ====================================================
> When there is injustice in society, then everyone will go to politics
> Except the two kinds: those who are timid and those who are
> materialist
> (Aristotle)
> ====================================================
--------------------------
Grant Ingersoll
Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ
Re: What are stopwords and protwords ???
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
Here's the link to wiki documentation on SolrJ
http://wiki.apache.org/solr/Solrj
On Wed, May 21, 2008 at 11:09 PM, Shalin Shekhar Mangar
<sh...@gmail.com> wrote:
> Hi Akeel,
>
> Take a look at SolrJ which is a Java client library for Solr. It is
> packaged with the Solr nightly binary downloads. This can be used by
> your Java/JSP application to add documents or query Solr. No changes
> to any config files is needed.
>
> On Wed, May 21, 2008 at 5:15 PM, Akeel <ak...@gmail.com> wrote:
>> Thank you very much for such a detailed reply. can you please tell me how
>> can i interact with solr from within my Java/JSP application ? I mean how to
>> query the solr running at localhost and getting results back in the
>> application. Do i have to change something there in solrconfig.xml ? Please
>> help me in this regards
>>
>> Thanks in advance
>>
>> --
>> Akeel
>>
>> On Wed, May 21, 2008 at 4:11 PM, Grant Ingersoll <gs...@apache.org>
>> wrote:
>>
>>> Stopwords are commonly occurring words that don't add _much_ value to
>>> search, such as the, an, a and are usually removed during analysis.
>>> Protwords (protected words) are words that would be stemmed by the English
>>> porter stemmer that you do not want to be stemmed.
>>>
>>> In the end, removing stopwords may keep your index smaller and can keep
>>> some queries from taking a long time, but they also mean you can't query for
>>> those words. As for protwords, that is something you would do if you felt
>>> the results for those tokens was "off".
>>>
>>> Many people use stopwords, many don't. Personally, I don't think removing
>>> them is the right thing to do, as there isn't always a way to recover them
>>> and they do provide meaning, otherwise why would they be needed in the
>>> language? Often, the best thing to do, is keep stopwords, but handle them
>>> intelligently on the query side (in phrases, etc.). However, since you're a
>>> beginner, it probably makes sense to just throw out stopwords for now.
>>>
>>> -Grant
>>>
>>>
>>> On May 21, 2008, at 1:50 AM, Akeel wrote:
>>>
>>> Hi,
>>>>
>>>> I am a beginner to Solr, I have successfully indexed my db in solr. I want
>>>> to know that what are the stopwords and protwords ??? and how much they
>>>> have
>>>> effect on my search results ?
>>>>
>>>>
>>>>
>>>> Thanks in advance.
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Akeel
>>>>
>>>>
>>>
>>
>>
>> --
>> Thanks and Regards,
>> Akeel ur Rehman Faridee
>> http://riseofpakistan.blogspot.com
>> cell: 0321-4714151
>> ====================================================
>> When there is injustice in society, then everyone will go to politics
>> Except the two kinds: those who are timid and those who are materialist
>> (Aristotle)
>> ====================================================
>>
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
--
Regards,
Shalin Shekhar Mangar.
Re: What are stopwords and protwords ???
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
Hi Akeel,
Take a look at SolrJ which is a Java client library for Solr. It is
packaged with the Solr nightly binary downloads. This can be used by
your Java/JSP application to add documents or query Solr. No changes
to any config files is needed.
On Wed, May 21, 2008 at 5:15 PM, Akeel <ak...@gmail.com> wrote:
> Thank you very much for such a detailed reply. can you please tell me how
> can i interact with solr from within my Java/JSP application ? I mean how to
> query the solr running at localhost and getting results back in the
> application. Do i have to change something there in solrconfig.xml ? Please
> help me in this regards
>
> Thanks in advance
>
> --
> Akeel
>
> On Wed, May 21, 2008 at 4:11 PM, Grant Ingersoll <gs...@apache.org>
> wrote:
>
>> Stopwords are commonly occurring words that don't add _much_ value to
>> search, such as the, an, a and are usually removed during analysis.
>> Protwords (protected words) are words that would be stemmed by the English
>> porter stemmer that you do not want to be stemmed.
>>
>> In the end, removing stopwords may keep your index smaller and can keep
>> some queries from taking a long time, but they also mean you can't query for
>> those words. As for protwords, that is something you would do if you felt
>> the results for those tokens was "off".
>>
>> Many people use stopwords, many don't. Personally, I don't think removing
>> them is the right thing to do, as there isn't always a way to recover them
>> and they do provide meaning, otherwise why would they be needed in the
>> language? Often, the best thing to do, is keep stopwords, but handle them
>> intelligently on the query side (in phrases, etc.). However, since you're a
>> beginner, it probably makes sense to just throw out stopwords for now.
>>
>> -Grant
>>
>>
>> On May 21, 2008, at 1:50 AM, Akeel wrote:
>>
>> Hi,
>>>
>>> I am a beginner to Solr, I have successfully indexed my db in solr. I want
>>> to know that what are the stopwords and protwords ??? and how much they
>>> have
>>> effect on my search results ?
>>>
>>>
>>>
>>> Thanks in advance.
>>>
>>>
>>>
>>> --
>>>
>>> Akeel
>>>
>>>
>>
>
>
> --
> Thanks and Regards,
> Akeel ur Rehman Faridee
> http://riseofpakistan.blogspot.com
> cell: 0321-4714151
> ====================================================
> When there is injustice in society, then everyone will go to politics
> Except the two kinds: those who are timid and those who are materialist
> (Aristotle)
> ====================================================
>
--
Regards,
Shalin Shekhar Mangar.
Re: What are stopwords and protwords ???
Posted by Akeel <ak...@gmail.com>.
Thank you very much for such a detailed reply. can you please tell me how
can i interact with solr from within my Java/JSP application ? I mean how to
query the solr running at localhost and getting results back in the
application. Do i have to change something there in solrconfig.xml ? Please
help me in this regards
Thanks in advance
--
Akeel
On Wed, May 21, 2008 at 4:11 PM, Grant Ingersoll <gs...@apache.org>
wrote:
> Stopwords are commonly occurring words that don't add _much_ value to
> search, such as the, an, a and are usually removed during analysis.
> Protwords (protected words) are words that would be stemmed by the English
> porter stemmer that you do not want to be stemmed.
>
> In the end, removing stopwords may keep your index smaller and can keep
> some queries from taking a long time, but they also mean you can't query for
> those words. As for protwords, that is something you would do if you felt
> the results for those tokens was "off".
>
> Many people use stopwords, many don't. Personally, I don't think removing
> them is the right thing to do, as there isn't always a way to recover them
> and they do provide meaning, otherwise why would they be needed in the
> language? Often, the best thing to do, is keep stopwords, but handle them
> intelligently on the query side (in phrases, etc.). However, since you're a
> beginner, it probably makes sense to just throw out stopwords for now.
>
> -Grant
>
>
> On May 21, 2008, at 1:50 AM, Akeel wrote:
>
> Hi,
>>
>> I am a beginner to Solr, I have successfully indexed my db in solr. I want
>> to know that what are the stopwords and protwords ??? and how much they
>> have
>> effect on my search results ?
>>
>>
>>
>> Thanks in advance.
>>
>>
>>
>> --
>>
>> Akeel
>>
>>
>
--
Thanks and Regards,
Akeel ur Rehman Faridee
http://riseofpakistan.blogspot.com
cell: 0321-4714151
====================================================
When there is injustice in society, then everyone will go to politics
Except the two kinds: those who are timid and those who are materialist
(Aristotle)
====================================================
Re: What are stopwords and protwords ???
Posted by Grant Ingersoll <gs...@apache.org>.
Stopwords are commonly occurring words that don't add _much_ value to
search, such as the, an, a and are usually removed during analysis.
Protwords (protected words) are words that would be stemmed by the
English porter stemmer that you do not want to be stemmed.
In the end, removing stopwords may keep your index smaller and can
keep some queries from taking a long time, but they also mean you
can't query for those words. As for protwords, that is something you
would do if you felt the results for those tokens was "off".
Many people use stopwords, many don't. Personally, I don't think
removing them is the right thing to do, as there isn't always a way to
recover them and they do provide meaning, otherwise why would they be
needed in the language? Often, the best thing to do, is keep
stopwords, but handle them intelligently on the query side (in
phrases, etc.). However, since you're a beginner, it probably makes
sense to just throw out stopwords for now.
-Grant
On May 21, 2008, at 1:50 AM, Akeel wrote:
> Hi,
>
> I am a beginner to Solr, I have successfully indexed my db in solr.
> I want
> to know that what are the stopwords and protwords ??? and how much
> they have
> effect on my search results ?
>
>
>
> Thanks in advance.
>
>
>
> --
>
> Akeel
>