You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by aniljayanti <an...@gmail.com> on 2012/08/01 12:04:45 UTC
auto completion search with solr using NGrams in SOLR
I want to implement an auto completion search with solr using NGrams. If the
user is searching for names of employees, then auto completion should be
applied. ie.,
if types "j" then need to show the names starts with "j" if types "ja" then
need to show the names starts with "ja" if types "jac" then need to show the
names starts with "jak" if types "jack" then need to show the names starts
with "jack"
Below is my configuration settings in schema.xml, Please suggest me if
anything wrong.
below is my code in schema.xml
<fieldType name="edgytext" class="solr.TextField"
positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.KeywordTokenizerFactory" />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.EdgeNGramFilterFactory" minGramSize="1"
maxGramSize="15" />
</analyzer>
<analyzer type="query">
<tokenizer class="solr.KeywordTokenizerFactory" />
<filter class="solr.LowerCaseFilterFactory" />
</analyzer>
</fieldType>
<field name="empname" type="edgytext" indexed="true" stored="true" />
<field name="autocomplete_text" type="edgytext" indexed="true" stored="true"
omitNorms="true" omitTermFreqAndPositions="true" />
<copyField source="empname" dest="text" />
when im searching with name "mado" or "madonna" getting employees names.But
when searching with "madon" not getting any data.
Please help me on this.
Thanks in Advance,
Anil.
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: auto completion search with solr using NGrams in SOLR
Posted by Jan Høydahl <ja...@cominvent.com>.
Have a look at my blog post http://www.cominvent.com/2012/01/25/super-flexible-autocomplete-with-solr/ for a walkthrough of how it could be done, as a separate Solr core.
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
Solr Training - www.solrtraining.com
On 1. aug. 2012, at 12:04, aniljayanti wrote:
> I want to implement an auto completion search with solr using NGrams. If the
> user is searching for names of employees, then auto completion should be
> applied. ie.,
>
> if types "j" then need to show the names starts with "j" if types "ja" then
> need to show the names starts with "ja" if types "jac" then need to show the
> names starts with "jak" if types "jack" then need to show the names starts
> with "jack"
>
> Below is my configuration settings in schema.xml, Please suggest me if
> anything wrong.
>
> below is my code in schema.xml
>
> <fieldType name="edgytext" class="solr.TextField"
> positionIncrementGap="100">
> <analyzer type="index">
> <tokenizer class="solr.KeywordTokenizerFactory" />
> <filter class="solr.LowerCaseFilterFactory" />
> <filter class="solr.EdgeNGramFilterFactory" minGramSize="1"
> maxGramSize="15" />
> </analyzer>
> <analyzer type="query">
> <tokenizer class="solr.KeywordTokenizerFactory" />
> <filter class="solr.LowerCaseFilterFactory" />
> </analyzer>
> </fieldType>
> <field name="empname" type="edgytext" indexed="true" stored="true" />
> <field name="autocomplete_text" type="edgytext" indexed="true" stored="true"
> omitNorms="true" omitTermFreqAndPositions="true" />
> <copyField source="empname" dest="text" />
>
> when im searching with name "mado" or "madonna" getting employees names.But
> when searching with "madon" not getting any data.
>
> Please help me on this.
>
>
> Thanks in Advance,
>
> Anil.
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: LateBinding
Posted by Alexey Serba <as...@gmail.com>.
http://searchhub.org/dev/2012/02/22/custom-security-filtering-in-solr/
See section about PostFilter.
On Wed, Aug 29, 2012 at 4:43 PM, <Jo...@blum.com> wrote:
> Hello,
>
> Has anyone ever implementet the security feature called late-binding?
>
> I am trying this but I am very new to solr and I would be very glad if I
> would get some hints to this.
>
> Regards,
> Johannes
Re: LateBinding
Posted by Chris Hostetter <ho...@fucit.org>.
: In-Reply-To: <13...@n3.nabble.com>
: References: <13...@n3.nabble.com>
: <F2...@SHI-Exchange.SHIGruppe.local>
: <13...@n3.nabble.com>
: <F2...@SHI-Exchange.SHIGruppe.local>
: <13...@n3.nabble.com>
: <CA...@mail.gmail.com>
: <13...@n3.nabble.com>
: Subject: LateBinding
https://people.apache.org/~hossman/#threadhijack
Thread Hijacking on Mailing Lists
When starting a new discussion on a mailing list, please do not reply to
an existing message, instead start a fresh email. Even if you change the
subject line of your email, other mail headers still track which thread
you replied to and your question is "hidden" in that thread and gets less
attention. It makes following discussions in the mailing list archives
particularly difficult.
-Hoss
LateBinding
Posted by Jo...@blum.com.
Hello,
Has anyone ever implementet the security feature called late-binding?
I am trying this but I am very new to solr and I would be very glad if I
would get some hints to this.
Regards,
Johannes
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by "anurag.sharma" <an...@gmail.com>.
Hi ... I am trying to build autocomplete functionality using your post. But I
am getting the following error
*2577 [coreLoadExecutor-3-thread-1] WARN
org.apache.solr.spelling.suggest.Suggester – Loading stored lookup data
failed
java.io.FileNotFoundException:
/home/anurag/Downloads/solr-4.4.0/example/solr/collection1/data/suggest/tst.dat
(No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:137)
at org.apache.solr.spelling.suggest.Suggester.init(Suggester.java:116)
at
org.apache.solr.handler.component.SpellCheckComponent.inform(SpellCheckComponent.java:623)
at
org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:601)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:830)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:629)
*
I am using solr 4.4. Is the suggester component still works in this version
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4098032.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by AnnaVak <an...@gmail.com>.
I have added it but still don't have another fields
terms/?fl=*:*&terms=true&terms.fl=autocomplete_text&terms.sort=index&terms.prefix=br&wt=json
Also I can't make it work with spellchecking
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4035968.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by Naresh <nn...@gmail.com>.
Hi,
You can fetch all the stored fields by passing them as part of
*fl*parameter. Go through
http://wiki.apache.org/solr/CommonQueryParameters#fl
On Thu, Jan 24, 2013 at 8:56 PM, AnnaVak <an...@gmail.com> wrote:
> Thanks for your solution it works for me too, I'm new with Solr but how I
> can
> additionally fetch another fields not only field that was used for
> searching? For example I have product title and image fields and I want to
> get the title but also related to this title image ? How can I do this?
>
> Thanks in advance
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4035931.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Regards
Naresh
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by AnnaVak <an...@gmail.com>.
Thanks for your solution it works for me too, I'm new with Solr but how I can
additionally fetch another fields not only field that was used for
searching? For example I have product title and image fields and I want to
get the title but also related to this title image ? How can I do this?
Thanks in advance
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4035931.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by aniljayanti <an...@gmail.com>.
Hi,
Finally got Solved using TermComponents.
schema.xml
----------
<fieldType name="edgytext" class="solr.TextField" >
<analyzer>
<tokenizer class="solr.KeywordTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
<field name="title" type="edgytext" indexed="true" stored="true" />
<field name="name" type="edgytext" indexed="true" stored="true" />
<field name="autocomplete_text" type="edgytext" indexed="true"
stored="false" multiValued="true" omitNorms="true"
omitTermFreqAndPositions="false" />
<copyField source="title" dest="autocomplete_text"/>
<copyField source="name" dest="autocomplete_text"/>
URL :
http://localhost:8080/solr/terms?*terms.fl=autocomplete_text&omitHeader=true&terms.sort=index&terms.prefix=mad*
Response :
<?xml version="1.0" encoding="UTF-8" ?>
<response>
<lst name="terms">
<lst name="autocomplete_text">
*<int name="mad - album version">1</int>
<int name="mad 6">1</int>
<int name="mad about the boy">2</int>
<int name="mad about the boy - 78rpm version">1</int>
<int name="mad about the boy - remastered 2002">1</int>
<int name="mad dog - album version">1</int> *
</lst>
</lst>
</response>
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4010125.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by aniljayanti <an...@gmail.com>.
Hi,
Thanks,
Iam getting the results with below url.
*suggest/?q="michael b"&df=title&defType=lucene&fl=title*
But, i want the results in spellcheck section.
i want to search with title or empname or both.
Aniljayanti
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4005812.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by Ahmet Arslan <io...@yahoo.com>.
> i want to search with title and empname both.
I know, I give that URL just to get the idea here.
If you try
suggest/?q="michael b"&df=title&defType=lucene&fl=title
you will see that your interested will in results section not <lst name="spellcheck"> section.
> or title or song...). Here (*suggest/?q="michael
> b"&df=title&defType=lucene*) we are specifying the
> title type search.
q=title:"michael b" OR empname:"michael b"&fl=title,empname would the trick.
> I removed said configurations in solrconfig.xml file, got
> result like below.
If you removed it, then there shouldn't be spellcheck response. And you are still looking results in the wrong place.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by aniljayanti <an...@gmail.com>.
HI,
Thanks,
i want to search with title and empname both. for example when we use any
search engine like google,yahoo... we donot specify any type that is (name
or title or song...). Here (*suggest/?q="michael
b"&df=title&defType=lucene*) we are specifying the title type search.
I removed said configurations in solrconfig.xml file, got result like below.
<lst name="spellcheck">
<lst name="suggestions">
<lst name="michael">
<int name="numFound">10</int>
<int name="startOffset">1</int>
<int name="endOffset">8</int>
<arr name="suggestion">
<str>michael</str>
<str>michael</str>
<str>michael "</str>
<str>michael j</str>
<str>michael ja</str>
<str>michael jac</str>
<str>michael jack</str>
<str>michael jacks</str>
<str>michael jackso</str>
<str>michael jackson</str>
</arr>
</lst>
<lst name="b">
<int name="numFound">10</int>
<int name="startOffset">9</int>
<int name="endOffset">10</int>
<arr name="suggestion">
<str>b</str>
<str>b</str>
<str>ba</str>
<str>bab</str>
<str>bar</str>
<str>barb</str>
<str>be</str>
<str>ben</str>
<str>bi</str>
<str>bl</str>
</arr>
</lst>
<str name="collation">"michael b"</str>
</lst>
</lst>
I sent my schema and solrconfig xml file configurations. Please check.
Aniljayanti
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4005545.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by Ahmet Arslan <io...@yahoo.com>.
Hi,
You are trying to use two different approaches at the same time.
1) Remove
<arr name="last-components">
<str>suggest</str>
<str>query</str>
</arr>
from your requestHandler.
2) Execute this query URL : suggest/?q="michael b"&df=title&defType=lucene
And you will see my point.
--- On Wed, 9/5/12, aniljayanti <an...@gmail.com> wrote:
> From: aniljayanti <an...@gmail.com>
> Subject: Re: AW: AW: auto completion search with solr using NGrams in SOLR
> To: solr-user@lucene.apache.org
> Date: Wednesday, September 5, 2012, 7:29 AM
> Hi,
>
> thanks,
>
> I m sending my whole configurations in schema and
> solrconfig.xml files.
>
> ************************************************************
> schema.xml
> -----------
>
> <fieldType name="edgytext" class="solr.TextField"
> positionIncrementGap="100"
> omitNorms="true">
> <analyzer type="index">
> <tokenizer
> class="solr.KeywordTokenizerFactory" />
> <filter
> class="solr.LowerCaseFilterFactory" />
> <filter
> class="solr.PatternReplaceFilterFactory" pattern="\s+"
> replacement=" " replace="all"/>
> <filter
> class="solr.EdgeNGramFilterFactory" minGramSize="1"
> maxGramSize="15" side="front" />
> </analyzer>
> <analyzer type="query">
> <tokenizer
> class="solr.KeywordTokenizerFactory" />
> <filter
> class="solr.LowerCaseFilterFactory" />
> <filter
> class="solr.PatternReplaceFilterFactory" pattern="\s+"
> replacement=" " replace="all"/>
> </analyzer>
> </fieldType>
>
>
> <field name="title"
> type="edgytext" indexed="true"
> stored="true" />
> <field name="empname" type="edgytext"
> indexed="true"
> stored="true" />
>
> <field name="autocomplete_text" type="edgytext"
> indexed="true"
> stored="false" multiValued="true" omitNorms="true"
> omitTermFreqAndPositions="false" />
>
> <copyField source="title" dest="autocomplete_text"/>
> <copyField source="empname"
> dest="autocomplete_text"/>
> *********************************************************
> solrconfig.xml
> -------------
> <searchComponent class="solr.SpellCheckComponent"
> name="suggest">
> <lst name="spellchecker">
> <str
> name="name">suggest</str>
> <str
> name="classname">org.apache.solr.spelling.suggest.Suggester</str>
> <str
> name="lookupImpl">org.apache.solr.spelling.suggest.fst.FSTLookup</str>
>
> <str
> name="storeDir">suggest</str>
> <str
> name="field">autocomplete_text</str>
> <bool
> name="exactMatchFirst">true</bool>
> <float
> name="threshold">0.005</float>
> <str
> name="buildOnCommit">true</str>
> <str
> name="buildOnOptimize">true</str>
> </lst>
> <lst name="spellchecker">
> <str
> name="name">jarowinkler</str>
> <str
> name="field">lowerfilt</str>
> <str
> name="distanceMeasure">org.apache.lucene.search.spell.JaroWinklerDistance</str>
>
> <str
> name="spellcheckIndexDir">spellchecker</str>
> </lst>
> <str
> name="queryAnalyzerFieldType">edgytext</str>
> </searchComponent>
>
> <requestHandler
> class="org.apache.solr.handler.component.SearchHandler"
> name="/suggest" startup="lazy">
>
> <lst name="defaults">
> <str
> name="spellcheck">true</str>
> <str
> name="spellcheck.dictionary">suggest</str>
> <str
> name="spellcheck.onlyMorePopular">true</str>
> <str
> name="spellcheck.count">5</str>
> <str
> name="spellcheck.collate">false</str>
> <str
> name="spellcheck.maxCollations">5</str>
> <str
> name="spellcheck.maxCollationTries">1000</str>
> <str
> name="spellcheck.collateExtendedResults">true</str>
> </lst>
> <arr name="last-components">
> <str>suggest</str>
> <str>query</str>
> </arr>
> </requestHandler>
>
> URL : suggest/?q="michael b"
> -------------------------
> Response :
>
> <?xml version="1.0" encoding="UTF-8" ?>
> <response>
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">3</int>
> </lst>
> <result name="response" numFound="0" start="0"
> />
> <lst name="spellcheck">
> <lst name="suggestions">
> <lst name="michael">
> <int name="numFound">10</int>
> <int name="startOffset">1</int>
> <int name="endOffset">8</int>
> <arr name="suggestion">
> <str>michael bully herbig</str>
> <str>michael bolton</str>
> <str>michael bolton: arias</str>
> <str>michael falch</str>
> <str>michael holm</str>
> <str>michael jackson</str>
> <str>michael neale</str>
> <str>michael penn</str>
> <str>michael salgado</str>
> <str>michael w. smith</str>
> </arr>
> </lst>
> <lst name="b">
> <int name="numFound">10</int>
> <int name="startOffset">9</int>
> <int name="endOffset">10</int>
> <arr name="suggestion">
> <str>b in the mix - the remixes</str>
> <str>b2k</str>
> <str>backstreet boys</str>
> <str>backyard babies</str>
> <str>banda maguey</str>
> <str>barbra streisand</str>
> <str>barry manilow</str>
> <str>benny goodman</str>
> <str>beny more</str>
> <str>beyonce</str>
> </arr>
> </lst>
> <str name="collation">"michael bully herbig b
> in the mix - the
> remixes"</str>
> </lst>
> </lst>
> </response>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4005490.html
> Sent from the Solr - User mailing list archive at
> Nabble.com.
>
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by atpatil11 <at...@gmail.com>.
Hi I have done the same changes as you told & changed the code with my fields
name. However I'm getting following error. I even reverted edited code but
still its throwing same error. We're having Solr 4.6. When i restart the
solr it says solr (pid 4610) already running.
SolrCore Initialization Failures
ole-beta:
org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
copyField dest :'autocomplete_text' is not an explicit field and doesn't
match a dynamicField.. Schema file is
/opt/bitnami/apache-solr/solr/collection1/schema.xml
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4129807.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by Erick Erickson <er...@gmail.com>.
Perhaps your ngram filter is set to terminate at 14 (maxGram)?
Best,
Erick
On Wed, Oct 1, 2014 at 3:18 AM, xoku <xo...@anlab.info> wrote:
> help me!
> i can't find all result.
> str name="spellcheck.count">200</str>
> Ex:
> i find: file
> result expected: file name documentabcxyz
> but solr return result (suggest: result term object) :
> -----
>
> [suggestions:protected] => Array
> (
> [0] => file
> [1] => file (whitespace)
> [2] => file n
> [3] => file nam
> [4] =>file name
> [5] => file name (whitespace)
> [6] => file name do
> [7] => file name doc
> [8] => file name docu (always is 14 character)
>
> )
> -------
> when result is 14 character, it stop and show result is file name docu.
>
>
>
>
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4162063.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by xoku <xo...@anlab.info>.
help me!
i can't find all result.
str name="spellcheck.count">200</str>
Ex:
i find: file
result expected: file name documentabcxyz
but solr return result (suggest: result term object) :
-----
[suggestions:protected] => Array
(
[0] => file
[1] => file (whitespace)
[2] => file n
[3] => file nam
[4] =>file name
[5] => file name (whitespace)
[6] => file name do
[7] => file name doc
[8] => file name docu (always is 14 character)
)
-------
when result is 14 character, it stop and show result is file name docu.
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4162063.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by aniljayanti <an...@gmail.com>.
Hi,
thanks,
I m sending my whole configurations in schema and solrconfig.xml files.
************************************************************
schema.xml
-----------
<fieldType name="edgytext" class="solr.TextField" positionIncrementGap="100"
omitNorms="true">
<analyzer type="index">
<tokenizer class="solr.KeywordTokenizerFactory" />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.PatternReplaceFilterFactory" pattern="\s+"
replacement=" " replace="all"/>
<filter class="solr.EdgeNGramFilterFactory" minGramSize="1"
maxGramSize="15" side="front" />
</analyzer>
<analyzer type="query">
<tokenizer class="solr.KeywordTokenizerFactory" />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.PatternReplaceFilterFactory" pattern="\s+"
replacement=" " replace="all"/>
</analyzer>
</fieldType>
<field name="title" type="edgytext" indexed="true" stored="true" />
<field name="empname" type="edgytext" indexed="true" stored="true" />
<field name="autocomplete_text" type="edgytext" indexed="true"
stored="false" multiValued="true" omitNorms="true"
omitTermFreqAndPositions="false" />
<copyField source="title" dest="autocomplete_text"/>
<copyField source="empname" dest="autocomplete_text"/>
*********************************************************
solrconfig.xml
-------------
<searchComponent class="solr.SpellCheckComponent" name="suggest">
<lst name="spellchecker">
<str name="name">suggest</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str
name="lookupImpl">org.apache.solr.spelling.suggest.fst.FSTLookup</str>
<str name="storeDir">suggest</str>
<str name="field">autocomplete_text</str>
<bool name="exactMatchFirst">true</bool>
<float name="threshold">0.005</float>
<str name="buildOnCommit">true</str>
<str name="buildOnOptimize">true</str>
</lst>
<lst name="spellchecker">
<str name="name">jarowinkler</str>
<str name="field">lowerfilt</str>
<str
name="distanceMeasure">org.apache.lucene.search.spell.JaroWinklerDistance</str>
<str name="spellcheckIndexDir">spellchecker</str>
</lst>
<str name="queryAnalyzerFieldType">edgytext</str>
</searchComponent>
<requestHandler class="org.apache.solr.handler.component.SearchHandler"
name="/suggest" startup="lazy">
<lst name="defaults">
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">suggest</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.count">5</str>
<str name="spellcheck.collate">false</str>
<str name="spellcheck.maxCollations">5</str>
<str name="spellcheck.maxCollationTries">1000</str>
<str name="spellcheck.collateExtendedResults">true</str>
</lst>
<arr name="last-components">
<str>suggest</str>
<str>query</str>
</arr>
</requestHandler>
URL : suggest/?q="michael b"
-------------------------
Response :
<?xml version="1.0" encoding="UTF-8" ?>
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">3</int>
</lst>
<result name="response" numFound="0" start="0" />
<lst name="spellcheck">
<lst name="suggestions">
<lst name="michael">
<int name="numFound">10</int>
<int name="startOffset">1</int>
<int name="endOffset">8</int>
<arr name="suggestion">
<str>michael bully herbig</str>
<str>michael bolton</str>
<str>michael bolton: arias</str>
<str>michael falch</str>
<str>michael holm</str>
<str>michael jackson</str>
<str>michael neale</str>
<str>michael penn</str>
<str>michael salgado</str>
<str>michael w. smith</str>
</arr>
</lst>
<lst name="b">
<int name="numFound">10</int>
<int name="startOffset">9</int>
<int name="endOffset">10</int>
<arr name="suggestion">
<str>b in the mix - the remixes</str>
<str>b2k</str>
<str>backstreet boys</str>
<str>backyard babies</str>
<str>banda maguey</str>
<str>barbra streisand</str>
<str>barry manilow</str>
<str>benny goodman</str>
<str>beny more</str>
<str>beyonce</str>
</arr>
</lst>
<str name="collation">"michael bully herbig b in the mix - the
remixes"</str>
</lst>
</lst>
</response>
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4005490.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by Kiran Jayakumar <ki...@gmail.com>.
I wonder why. I had a similar use case & works great for me. If you can
send the snapshot of analysis for a sample string (say "hello world " for
indexing, "hel" - positive case, "wo" - negative case for querying), then
we can see whats going on. Also the debug query output would be helpful.
On Fri, Aug 31, 2012 at 10:28 PM, aniljayanti <an...@gmail.com>wrote:
> Hi,
>
> Thanks,
>
> As i already used "KeywordTokenizerFactory" in my earlier posts.
>
> <fieldType name="edgytext" class="solr.TextField"
> positionIncrementGap="100"
> omitNorms="true">
> <analyzer type="index">
> *<tokenizer class="solr.KeywordTokenizerFactory" />
> <filter class="solr.LowerCaseFilterFactory" />
> <filter class="solr.PatternReplaceFilterFactory" pattern="\s+"
> replacement=" " replace="all"/>
> <filter class="solr.EdgeNGramFilterFactory" minGramSize="1"
> maxGramSize="15" side="front" />
> * </analyzer>
> <analyzer type="query">
> *<tokenizer class="solr.KeywordTokenizerFactory" />
> <filter class="solr.LowerCaseFilterFactory" />
> <filter class="solr.PatternReplaceFilterFactory" pattern="\s+"
> replacement=" " replace="all"/>
> * </analyzer>
> </fieldType>
>
> getting same results.
>
> AnilJayanti
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4004871.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by aniljayanti <an...@gmail.com>.
Hi,
thanks,
Im not able to attach asked xml files here. Can u give me ur Email id, so
that i can send schema and solrconfig.xmls.
Regards,
AnilJayanti
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4005141.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by Ahmet Arslan <io...@yahoo.com>.
Hi Anil,
Your search URL suggest/?q="michael ja" and fieldType (without PatternReplaceFilterFactory) is correct.
However looking in to your response carefully I see that you have numFound="0" no results.
And results you point come form "suggest" (probably suggester component). But in this approach (auto completion with NGrams) suggestions will be simply your search results. Remove suggester component from RequestHander.
I suspect
* you are not querying correct field
* you are not populating that field via copyField
What is your default search field? Can you send/post complete solrconfig.xml and schema.xml?
--- On Sat, 9/1/12, aniljayanti <an...@gmail.com> wrote:
> From: aniljayanti <an...@gmail.com>
> Subject: Re: AW: AW: auto completion search with solr using NGrams in SOLR
> To: solr-user@lucene.apache.org
> Date: Saturday, September 1, 2012, 8:28 AM
> Hi,
>
> Thanks,
>
> As i already used "KeywordTokenizerFactory" in my earlier
> posts.
>
> <fieldType name="edgytext" class="solr.TextField"
> positionIncrementGap="100"
> omitNorms="true">
> <analyzer type="index">
> *<tokenizer
> class="solr.KeywordTokenizerFactory" />
> <filter
> class="solr.LowerCaseFilterFactory" />
> <filter
> class="solr.PatternReplaceFilterFactory" pattern="\s+"
> replacement=" " replace="all"/>
> <filter
> class="solr.EdgeNGramFilterFactory" minGramSize="1"
> maxGramSize="15" side="front" />
> * </analyzer>
> <analyzer type="query">
> *<tokenizer
> class="solr.KeywordTokenizerFactory" />
> <filter
> class="solr.LowerCaseFilterFactory" />
> <filter
> class="solr.PatternReplaceFilterFactory" pattern="\s+"
> replacement=" " replace="all"/>
> * </analyzer>
> </fieldType>
>
> getting same results.
>
> AnilJayanti
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4004871.html
> Sent from the Solr - User mailing list archive at
> Nabble.com.
>
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by aniljayanti <an...@gmail.com>.
Hi,
Thanks,
As i already used "KeywordTokenizerFactory" in my earlier posts.
<fieldType name="edgytext" class="solr.TextField" positionIncrementGap="100"
omitNorms="true">
<analyzer type="index">
*<tokenizer class="solr.KeywordTokenizerFactory" />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.PatternReplaceFilterFactory" pattern="\s+"
replacement=" " replace="all"/>
<filter class="solr.EdgeNGramFilterFactory" minGramSize="1"
maxGramSize="15" side="front" />
* </analyzer>
<analyzer type="query">
*<tokenizer class="solr.KeywordTokenizerFactory" />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.PatternReplaceFilterFactory" pattern="\s+"
replacement=" " replace="all"/>
* </analyzer>
</fieldType>
getting same results.
AnilJayanti
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4004871.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by Kiran Jayakumar <ki...@gmail.com>.
Try this:
<tokenizer class="solr.KeywordTokenizerFactory"/>
http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.KeywordTokenizerFactory
On Thu, Aug 30, 2012 at 9:07 PM, aniljayanti <an...@gmail.com> wrote:
> Hi,
>
> thanks,
>
> I checked with given changes, getting below error saying that SOLR is not
> allowing without tokenizer.
>
> org.apache.solr.common.SolrException: analyzer without class or tokenizer &
> filter list at
> org.apache.solr.schema.IndexSchema.readAnalyzer(IndexSchema.java:914) at
> org.apache.solr.schema.IndexSchema.access$100(IndexSchema.java:62) at
> ...
>
> can u tell me wht to do?
>
> AnilJayanti
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4004605.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by aniljayanti <an...@gmail.com>.
Hi,
thanks,
I checked with given changes, getting below error saying that SOLR is not
allowing without tokenizer.
org.apache.solr.common.SolrException: analyzer without class or tokenizer &
filter list at
org.apache.solr.schema.IndexSchema.readAnalyzer(IndexSchema.java:914) at
org.apache.solr.schema.IndexSchema.access$100(IndexSchema.java:62) at
...
can u tell me wht to do?
AnilJayanti
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4004605.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by Kiran Jayakumar <ki...@gmail.com>.
You wouldnt need any tokenizer, I'm not sure if Solr will allow that. You
also need to move the Edge n gram filter to index from query (I think Ahmet
suggested this). It will look something like this:
<fieldType name="edgytext" class="solr.TextField"
positionIncrementGap="100" omitNorms="true">
<analyzer type="index">
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.PatternReplaceFilterFactory"
pattern="\s+" replacement=" " replace="all"/>
<filter class="solr.EdgeNGramFilterFactory"
minGramSize="1" maxGramSize="15" side="front" />
</analyzer>
<analyzer type="query">
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.PatternReplaceFilterFactory"
pattern="\s+" replacement=" " replace="all"/>
</analyzer>
</fieldType>
note that pattern replace goes before edge, order is very important. I
still highly recommend using the analysis option in Solr admin. It will
show how your text is transformed.
On Wed, Aug 29, 2012 at 9:40 PM, aniljayanti <an...@gmail.com> wrote:
> Hi,
>
> thanks,
>
> I added "PatternReplaceFilterFactory" like below.Getting results
> differently(not like suggester). You suggested to remove
> "KeywordTokenizerFactory" , "PatternReplace" is a FilterFactory, then which
> "TokenizerFactory" need to use ?
>
> <fieldType name="edgytext" class="solr.TextField"
> positionIncrementGap="100" omitNorms="true">
> <analyzer type="index">
> <tokenizer class="solr.KeywordTokenizerFactory" />
> <filter class="solr.LowerCaseFilterFactory" />
>
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> <filter class="solr.PatternReplaceFilterFactory" pattern="\s+"
> replacement=" " replace="all"/>
> </analyzer>
> <analyzer type="query">
> <tokenizer class="solr.KeywordTokenizerFactory" />
> <filter class="solr.LowerCaseFilterFactory" />
> <filter class="solr.EdgeNGramFilterFactory" minGramSize="1"
> maxGramSize="15" side="front" />
> <filter class="solr.PatternReplaceFilterFactory" pattern="\s+"
> replacement=" " replace="all"/>
> </analyzer>
> </fieldType>
>
> Result :
>
> <?xml version="1.0" encoding="UTF-8" ?>
> - <response>
> - <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">2</int>
> </lst>
> <result name="response" numFound="0" start="0" />
> - <lst name="spellcheck">
> - <lst name="suggestions">
> - <lst name="michael">
> <int name="numFound">10</int>
> <int name="startOffset">0</int>
> <int name="endOffset">7</int>
> - <arr name="suggestion">
> *<str>michael</str>
> <str>michael</str>
> <str>michael "</str>
> <str>michael j</str>
> <str>michael ja</str>
> <str>michael jac</str>
> <str>michael jack</str>
> <str>michael jacks</str>
> <str>michael jackso</str>
> <str>michael jackson</str>*
> </arr>
> </lst>
> - <lst name="ja">
> <int name="numFound">10</int>
> <int name="startOffset">8</int>
> <int name="endOffset">10</int>
> - <arr name="suggestion">
> *<str>ja</str>
> <str>ja</str>
> <str>jag</str>
> <str>jam</str>
> <str>jami</str>
> <str>jamp</str>
> <str>jampp</str>
> <str>jamppa</str>
> <str>jamppa</str>
> <str>jamppa t</str>*
> </arr>
> </lst>
> <str name="collation">michael ja</str>
> </lst>
> </lst>
> </response>
>
> Please suggest me if anything missing?
>
> Regards,
>
> AnilJayanti
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4004231.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by aniljayanti <an...@gmail.com>.
Hi,
thanks,
I added "PatternReplaceFilterFactory" like below.Getting results
differently(not like suggester). You suggested to remove
"KeywordTokenizerFactory" , "PatternReplace" is a FilterFactory, then which
"TokenizerFactory" need to use ?
<fieldType name="edgytext" class="solr.TextField"
positionIncrementGap="100" omitNorms="true">
<analyzer type="index">
<tokenizer class="solr.KeywordTokenizerFactory" />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
<filter class="solr.PatternReplaceFilterFactory" pattern="\s+"
replacement=" " replace="all"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.KeywordTokenizerFactory" />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.EdgeNGramFilterFactory" minGramSize="1"
maxGramSize="15" side="front" />
<filter class="solr.PatternReplaceFilterFactory" pattern="\s+"
replacement=" " replace="all"/>
</analyzer>
</fieldType>
Result :
<?xml version="1.0" encoding="UTF-8" ?>
- <response>
- <lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">2</int>
</lst>
<result name="response" numFound="0" start="0" />
- <lst name="spellcheck">
- <lst name="suggestions">
- <lst name="michael">
<int name="numFound">10</int>
<int name="startOffset">0</int>
<int name="endOffset">7</int>
- <arr name="suggestion">
*<str>michael</str>
<str>michael</str>
<str>michael "</str>
<str>michael j</str>
<str>michael ja</str>
<str>michael jac</str>
<str>michael jack</str>
<str>michael jacks</str>
<str>michael jackso</str>
<str>michael jackson</str>*
</arr>
</lst>
- <lst name="ja">
<int name="numFound">10</int>
<int name="startOffset">8</int>
<int name="endOffset">10</int>
- <arr name="suggestion">
*<str>ja</str>
<str>ja</str>
<str>jag</str>
<str>jam</str>
<str>jami</str>
<str>jamp</str>
<str>jampp</str>
<str>jamppa</str>
<str>jamppa</str>
<str>jamppa t</str>*
</arr>
</lst>
<str name="collation">michael ja</str>
</lst>
</lst>
</response>
Please suggest me if anything missing?
Regards,
AnilJayanti
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4004231.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by Kiran Jayakumar <ki...@gmail.com>.
You need this for both index and query:
<filter class="solr.PatternReplaceFilterFactory" pattern="\s+"
replacement=" " replace="all"/>
http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.PatternReplaceFilterFactory
On Wed, Aug 29, 2012 at 4:55 AM, aniljayanti <an...@gmail.com> wrote:
> Hi,
>
> thanks for ur reply,
>
> I donot know how to remove multiple white spaces using regax in the search
> text. Can u share me that one.
>
> Thanks,
>
> AnilJayanti
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4003991.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by aniljayanti <an...@gmail.com>.
Hi,
thanks for ur reply,
I donot know how to remove multiple white spaces using regax in the search
text. Can u share me that one.
Thanks,
AnilJayanti
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4003991.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by Kiran Jayakumar <ki...@gmail.com>.
Since you have <tokenizer class="solr.KeywordTokenizerFactory" />, during
indexing time, it is going split the text on white spaces and then apply
edge n-grams. If you remove this & may be replace it with a simpler regex
which does basic clean up like removing multiple white spaces etc., then it
will not match on the beginning of non-first words. You can use the
analyzer and see how your text is transformed during index/query time.
On Tue, Aug 28, 2012 at 4:57 AM, aniljayanti <an...@gmail.com> wrote:
> Hi ,
>
> Thanks for reply,
>
> Now it's working for me after changing like below.
>
> <fieldType name="edgytext" class="solr.TextField"
> positionIncrementGap="100"
> omitNorms="true">
> <analyzer type="index">
> <tokenizer class="solr.KeywordTokenizerFactory" />
> <filter class="solr.LowerCaseFilterFactory" />
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analyzer>
> <analyzer type="query">
> <tokenizer class="solr.KeywordTokenizerFactory" />
> <filter class="solr.LowerCaseFilterFactory" />
> <filter class="solr.EdgeNGramFilterFactory" minGramSize="1"
> maxGramSize="15" side="front" />
> </analyzer>
> </fieldType>
>
>
> <field name="title" type="edgytext" indexed="true" stored="true"
> omitNorms="true" omitTermFreqAndPositions="true"/>
> <field name="empname" type="edgytext" indexed="true" stored="true"
> omitNorms="true" omitTermFreqAndPositions="true" />
>
> <field name="autocomplete_text" type="edgytext" indexed="true"
> stored="false" multiValued="true" omitNorms="true"
> omitTermFreqAndPositions="false" />
>
> <copyField source="empname" dest="autocomplete_text"/>
> <copyField source="title" dest="autocomplete_text"/>
> **************************************************************
> URL : http://localhost:8080/test/suggest/?q=michael
>
> Result :
> <?xml version="1.0" encoding="UTF-8" ?>
> - <response>
> - <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">1</int>
> </lst>
> <result name="response" numFound="0" start="0" />
> - <lst name="spellcheck">
> - <lst name="suggestions">
> - <lst name="michael">
> <int name="numFound">9</int>
> <int name="startOffset">0</int>
> <int name="endOffset">7</int>
> - <arr name="suggestion">
> <str>michael bolton</str>
> <str>michael foret</str>
> <str>michael houser</str>
> <str>michael o'brien</str>
> <str>michael penn</str>
> <str>michael row your boat ashore</str>
> <str>michael tilson thomas</str>
> <str>michael w. smith</str>
> <str>michael w. smith featuring andrae crouch</str>
> </arr>
> </lst>
> <str name="collation">michael bolton</str>
> </lst>
> </lst>
> </response>
>
> It's working fine for me. When im searching with "michael f", getting
> response like below. (http://localhost:8080/test/suggest/?q=michael f)
>
> Response :
>
> <?xml version="1.0" encoding="UTF-8" ?>
> - <response>
> - <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">1</int>
> </lst>
> <result name="response" numFound="0" start="0" />
> - <lst name="spellcheck">
> - <lst name="suggestions">
> - <lst name="michael">
> <int name="numFound">9</int>
> <int name="startOffset">0</int>
> <int name="endOffset">7</int>
> - <arr name="suggestion">
> <str>michael bolton</str>
> <str>michael foret</str>
> <str>michael houser</str>
> <str>michael o'brien</str>
> <str>michael penn</str>
> <str>michael row your boat ashore</str>
> <str>michael tilson thomas</str>
> <str>michael w. smith</str>
> <str>michael w. smith featuring andrae crouch</str>
> </arr>
> </lst>
> - <lst name="f">
> <int name="numFound">10</int>
> <int name="startOffset">8</int>
> <int name="endOffset">9</int>
> - <arr name="suggestion">
> <str>f**k the facts</str>
> <str>fairest lord jesus</str>
> <str>fatboy slim</str>
> <str>ffh</str>
> <str>fiona apple</str>
> <str>foo fighters</str>
> <str>frank sinatra</str>
> <str>frans bauer</str>
> <str>franz ferdinand</str>
> <str>françois rauber</str>
> </arr>
> </lst>
> <str name="collation">michael bolton f**k the facts</str>
> </lst>
> </lst>
> </response>.
>
> So when i search with "michael f" then, i should get "michael foret" only.
> Data coming starts with "f". Please help me on this.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4003689.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by Ahmet Arslan <io...@yahoo.com>.
> I tried with below said changes, but getting same result as
> earlier.
>
> suggest/?q="michael ja"
> -----------------------
> <fieldType name="edgytext" class="solr.TextField"
> positionIncrementGap="100"
> omitNorms="true">
> <analyzer type="index">
> <tokenizer
> class="solr.KeywordTokenizerFactory" />
> <filter
> class="solr.LowerCaseFilterFactory" />
> <filter
> class="solr.EdgeNGramFilterFactory" minGramSize="1"
> maxGramSize="15" side="front" />
> <filter
> class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analyzer>
> <analyzer type="query">
> <tokenizer
> class="solr.KeywordTokenizerFactory" />
> <filter
> class="solr.LowerCaseFilterFactory" />
>
> </analyzer>
> </fieldType>
>
> Response :
>
> <?xml version="1.0" encoding="UTF-8" ?>
> - <response>
> - <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">1</int>
> </lst>
> <result name="response" numFound="0" start="0"
> />
> - <lst name="spellcheck">
> - <lst name="suggestions">
> - <lst name="michael">
> <int name="numFound">10</int>
> <int name="startOffset">1</int>
> <int name="endOffset">8</int>
> - <arr name="suggestion">
> *<str>michael "bully" herbig</str>
> <str>michael bolton</str>
> <str>michael bolton: arias</str>
> <str>michael falch</str>
> <str>michael holm</str>
> <str>michael jackson</str>
> <str>michael neale</str>
> <str>michael penn</str>
> <str>michael salgado</str>
> <str>michael w. smith</str>*
> </arr>
> </lst>
> - <lst name="ja">
> <int name="numFound">10</int>
> <int name="startOffset">9</int>
> <int name="endOffset">11</int>
> - <arr name="suggestion">
> * <str>ja me tanssimme</str>
> <str>jacob andersen</str>
> <str>jacob haugaard</str>
> <str>jagged edge</str>
> <str>jaguares</str>
> <str>jamiroquai</str>
> <str>jamppa tuominen</str>
> <str>jane olivor</str>
> <str>janis joplin</str>
> <str>janne tulkki</str>*
> </arr>
> </lst>
> <str name="collation">"michael "bully" herbig
> ja me tanssimme"</str>
> </lst>
> </lst>
> </response>
Hi, looking in to your response carefully I see that you have numFound="0"
And results you point come form "suggest". The field type above is correct. But in this approach suggestions will be simply your search results. What is your default search field?
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by aniljayanti <an...@gmail.com>.
Hi
thanks,
I tried with below said changes, but getting same result as earlier.
suggest/?q="michael ja"
-----------------------
<fieldType name="edgytext" class="solr.TextField" positionIncrementGap="100"
omitNorms="true">
<analyzer type="index">
<tokenizer class="solr.KeywordTokenizerFactory" />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.EdgeNGramFilterFactory" minGramSize="1"
maxGramSize="15" side="front" />
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.KeywordTokenizerFactory" />
<filter class="solr.LowerCaseFilterFactory" />
</analyzer>
</fieldType>
Response :
<?xml version="1.0" encoding="UTF-8" ?>
- <response>
- <lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">1</int>
</lst>
<result name="response" numFound="0" start="0" />
- <lst name="spellcheck">
- <lst name="suggestions">
- <lst name="michael">
<int name="numFound">10</int>
<int name="startOffset">1</int>
<int name="endOffset">8</int>
- <arr name="suggestion">
*<str>michael "bully" herbig</str>
<str>michael bolton</str>
<str>michael bolton: arias</str>
<str>michael falch</str>
<str>michael holm</str>
<str>michael jackson</str>
<str>michael neale</str>
<str>michael penn</str>
<str>michael salgado</str>
<str>michael w. smith</str>*
</arr>
</lst>
- <lst name="ja">
<int name="numFound">10</int>
<int name="startOffset">9</int>
<int name="endOffset">11</int>
- <arr name="suggestion">
* <str>ja me tanssimme</str>
<str>jacob andersen</str>
<str>jacob haugaard</str>
<str>jagged edge</str>
<str>jaguares</str>
<str>jamiroquai</str>
<str>jamppa tuominen</str>
<str>jane olivor</str>
<str>janis joplin</str>
<str>janne tulkki</str>*
</arr>
</lst>
<str name="collation">"michael "bully" herbig ja me tanssimme"</str>
</lst>
</lst>
</response>
Please Help,
AnilHayanti
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4004230.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by Ahmet Arslan <io...@yahoo.com>.
> Hi,
>
> thanks,
>
> I tried by adding " marks, but still giving same
> results.
>
> http://localhost:8080/test/suggest/?q="michael f"
Looking back to your field type definition, i saw that you have defined
<filter class="solr.EdgeNGramFilterFactory" minGramSize="1" maxGramSize="15" side="front" /> in query analyzer. Move this into index analyzer. Restart solr, re-index and suggest/?q="michael f" should return expected results.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by aniljayanti <an...@gmail.com>.
Hi,
thanks,
I tried by adding " marks, but still giving same results.
http://localhost:8080/test/suggest/?q="michael f"
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4003959.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by Ahmet Arslan <io...@yahoo.com>.
You need to use quotes in your query :
http://localhost:8080/test/suggest/?q="michael f"
--- On Tue, 8/28/12, aniljayanti <an...@gmail.com> wrote:
> From: aniljayanti <an...@gmail.com>
> Subject: Re: AW: AW: auto completion search with solr using NGrams in SOLR
> To: solr-user@lucene.apache.org
> Date: Tuesday, August 28, 2012, 2:57 PM
> Hi ,
>
> Thanks for reply,
>
> Now it's working for me after changing like below.
>
> <fieldType name="edgytext" class="solr.TextField"
> positionIncrementGap="100"
> omitNorms="true">
> <analyzer type="index">
> <tokenizer
> class="solr.KeywordTokenizerFactory" />
> <filter
> class="solr.LowerCaseFilterFactory" />
> <filter
> class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analyzer>
> <analyzer type="query">
> <tokenizer
> class="solr.KeywordTokenizerFactory" />
> <filter
> class="solr.LowerCaseFilterFactory" />
> <filter
> class="solr.EdgeNGramFilterFactory" minGramSize="1"
> maxGramSize="15" side="front"
> />
> </analyzer>
> </fieldType>
>
>
> <field name="title" type="edgytext" indexed="true"
> stored="true"
> omitNorms="true" omitTermFreqAndPositions="true"/>
> <field name="empname" type="edgytext" indexed="true"
> stored="true"
> omitNorms="true" omitTermFreqAndPositions="true" />
>
> <field name="autocomplete_text" type="edgytext"
> indexed="true"
> stored="false" multiValued="true" omitNorms="true"
> omitTermFreqAndPositions="false" />
>
> <copyField source="empname"
> dest="autocomplete_text"/>
> <copyField source="title" dest="autocomplete_text"/>
> **************************************************************
> URL : http://localhost:8080/test/suggest/?q=michael
>
> Result :
> <?xml version="1.0" encoding="UTF-8" ?>
> - <response>
> - <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">1</int>
> </lst>
> <result name="response" numFound="0" start="0"
> />
> - <lst name="spellcheck">
> - <lst name="suggestions">
> - <lst name="michael">
> <int name="numFound">9</int>
> <int name="startOffset">0</int>
> <int name="endOffset">7</int>
> - <arr name="suggestion">
> <str>michael bolton</str>
> <str>michael foret</str>
> <str>michael houser</str>
> <str>michael o'brien</str>
> <str>michael penn</str>
> <str>michael row your boat ashore</str>
> <str>michael tilson thomas</str>
> <str>michael w. smith</str>
> <str>michael w. smith featuring andrae
> crouch</str>
> </arr>
> </lst>
> <str name="collation">michael
> bolton</str>
> </lst>
> </lst>
> </response>
>
> It's working fine for me. When im searching with "michael
> f", getting
> response like below. (http://localhost:8080/test/suggest/?q=michael f)
>
> Response :
>
> <?xml version="1.0" encoding="UTF-8" ?>
> - <response>
> - <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">1</int>
> </lst>
> <result name="response" numFound="0" start="0"
> />
> - <lst name="spellcheck">
> - <lst name="suggestions">
> - <lst name="michael">
> <int name="numFound">9</int>
> <int name="startOffset">0</int>
> <int name="endOffset">7</int>
> - <arr name="suggestion">
> <str>michael bolton</str>
> <str>michael foret</str>
> <str>michael houser</str>
> <str>michael o'brien</str>
> <str>michael penn</str>
> <str>michael row your boat ashore</str>
> <str>michael tilson thomas</str>
> <str>michael w. smith</str>
> <str>michael w. smith featuring andrae
> crouch</str>
> </arr>
> </lst>
> - <lst name="f">
> <int name="numFound">10</int>
> <int name="startOffset">8</int>
> <int name="endOffset">9</int>
> - <arr name="suggestion">
> <str>f**k the facts</str>
> <str>fairest lord jesus</str>
> <str>fatboy slim</str>
> <str>ffh</str>
> <str>fiona apple</str>
> <str>foo fighters</str>
> <str>frank sinatra</str>
> <str>frans bauer</str>
> <str>franz ferdinand</str>
> <str>françois rauber</str>
> </arr>
> </lst>
> <str name="collation">michael bolton f**k the
> facts</str>
> </lst>
> </lst>
> </response>.
>
> So when i search with "michael f" then, i should get
> "michael foret" only.
> Data coming starts with "f". Please help me on this.
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4003689.html
> Sent from the Solr - User mailing list archive at
> Nabble.com.
>
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by aniljayanti <an...@gmail.com>.
Hi ,
Thanks for reply,
Now it's working for me after changing like below.
<fieldType name="edgytext" class="solr.TextField" positionIncrementGap="100"
omitNorms="true">
<analyzer type="index">
<tokenizer class="solr.KeywordTokenizerFactory" />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.KeywordTokenizerFactory" />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.EdgeNGramFilterFactory" minGramSize="1"
maxGramSize="15" side="front" />
</analyzer>
</fieldType>
<field name="title" type="edgytext" indexed="true" stored="true"
omitNorms="true" omitTermFreqAndPositions="true"/>
<field name="empname" type="edgytext" indexed="true" stored="true"
omitNorms="true" omitTermFreqAndPositions="true" />
<field name="autocomplete_text" type="edgytext" indexed="true"
stored="false" multiValued="true" omitNorms="true"
omitTermFreqAndPositions="false" />
<copyField source="empname" dest="autocomplete_text"/>
<copyField source="title" dest="autocomplete_text"/>
**************************************************************
URL : http://localhost:8080/test/suggest/?q=michael
Result :
<?xml version="1.0" encoding="UTF-8" ?>
- <response>
- <lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">1</int>
</lst>
<result name="response" numFound="0" start="0" />
- <lst name="spellcheck">
- <lst name="suggestions">
- <lst name="michael">
<int name="numFound">9</int>
<int name="startOffset">0</int>
<int name="endOffset">7</int>
- <arr name="suggestion">
<str>michael bolton</str>
<str>michael foret</str>
<str>michael houser</str>
<str>michael o'brien</str>
<str>michael penn</str>
<str>michael row your boat ashore</str>
<str>michael tilson thomas</str>
<str>michael w. smith</str>
<str>michael w. smith featuring andrae crouch</str>
</arr>
</lst>
<str name="collation">michael bolton</str>
</lst>
</lst>
</response>
It's working fine for me. When im searching with "michael f", getting
response like below. (http://localhost:8080/test/suggest/?q=michael f)
Response :
<?xml version="1.0" encoding="UTF-8" ?>
- <response>
- <lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">1</int>
</lst>
<result name="response" numFound="0" start="0" />
- <lst name="spellcheck">
- <lst name="suggestions">
- <lst name="michael">
<int name="numFound">9</int>
<int name="startOffset">0</int>
<int name="endOffset">7</int>
- <arr name="suggestion">
<str>michael bolton</str>
<str>michael foret</str>
<str>michael houser</str>
<str>michael o'brien</str>
<str>michael penn</str>
<str>michael row your boat ashore</str>
<str>michael tilson thomas</str>
<str>michael w. smith</str>
<str>michael w. smith featuring andrae crouch</str>
</arr>
</lst>
- <lst name="f">
<int name="numFound">10</int>
<int name="startOffset">8</int>
<int name="endOffset">9</int>
- <arr name="suggestion">
<str>f**k the facts</str>
<str>fairest lord jesus</str>
<str>fatboy slim</str>
<str>ffh</str>
<str>fiona apple</str>
<str>foo fighters</str>
<str>frank sinatra</str>
<str>frans bauer</str>
<str>franz ferdinand</str>
<str>françois rauber</str>
</arr>
</lst>
<str name="collation">michael bolton f**k the facts</str>
</lst>
</lst>
</response>.
So when i search with "michael f" then, i should get "michael foret" only.
Data coming starts with "f". Please help me on this.
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p4003689.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by Ahmet Arslan <io...@yahoo.com>.
--- On Sat, 8/4/12, aniljayanti <an...@gmail.com> wrote:
> From: aniljayanti <an...@gmail.com>
> Subject: Re: AW: AW: auto completion search with solr using NGrams in SOLR
> To: solr-user@lucene.apache.org
> Date: Saturday, August 4, 2012, 8:57 AM
> Hi
> thanks,
>
> which doing searching i will search either with empname or
> title only. And
> also not using any asterics in the query.
> ex : if i search with "mic" result should come like
>
> michale jackson
> michale border
> michale smith
>
> want the result just like google search.
>
> can us suggest me wht are the configuration need to
> add/change to get the
> result like google search ?.
> for my required result which tokenizers need to use. ?
> can u tell me how to call a query for this??
Only suspicious thing is omitTermFreqAndPositions="true". Try changing this to false.
Also see Chantal's message http://search-lucene.com/m/rPrhO1RIlfQ
Re: AW: AW: auto completion search with solr using NGrams in SOLR
Posted by aniljayanti <an...@gmail.com>.
Hi
thanks,
which doing searching i will search either with empname or title only. And
also not using any asterics in the query.
ex : if i search with "mic" result should come like
michale jackson
michale border
michale smith
want the result just like google search.
can us suggest me wht are the configuration need to add/change to get the
result like google search ?.
for my required result which tokenizers need to use. ?
can u tell me how to call a query for this??
thanks,
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p3999171.html
Sent from the Solr - User mailing list archive at Nabble.com.
AW: AW: auto completion search with solr using NGrams in SOLR
Posted by Markus Klose <mk...@shi-gmbh.com>.
If you want to search in the two fields "title" and "empname" you have to use the query parser (e)dismax
http://wiki.apache.org/solr/ExtendedDisMax
you need to specify the qf param: qf=title empname
check your solrconfig.xml to verifiy which queryparser you are using right now.
In your usecase you do not need the asterics in the query.
"q=am" in combination with the edgengram will find aman, amar, amal
Viele Grüße aus Augsburg
Markus Klose
SHI Elektronische Medien GmbH
-----Ursprüngliche Nachricht-----
Von: aniljayanti [mailto:anil.jayanti@gmail.com]
Gesendet: Donnerstag, 2. August 2012 09:34
An: solr-user@lucene.apache.org
Betreff: Re: AW: auto completion search with solr using NGrams in SOLR
Hi,
thanks,
im searching with empname filed. want to search with both "empname" and "title".
below is my changed code.
<fieldType name="edgytext" class="solr.TextField"
positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.LowerCaseTokenizerFactory"/>
<filter class="solr.EdgeNGramFilterFactory" minGramSize="1" maxGramSize="15"
/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.LowerCaseTokenizerFactory"/>
</analyzer>
</fieldType>
<field name="title" type="edgytext" indexed="true" stored="true"
omitNorms="true" omitTermFreqAndPositions="true"/> <field name="empname" type="edgytext" indexed="true" stored="true"
omitNorms="true" omitTermFreqAndPositions="true" />
<copyField source="empname" dest="text"/> <copyField source="title" dest="text"/>
SOLR Query :
http://localhost:8080/AC/select/?q=(*am*)&rows=500
1) want to search with "title" and "empname" both.
2) am i quering correctly with above url ??
but getting result like .....
p*am*
s*am*
ra*am*a
but i want the result like ....
aman
amar
amal
please help me in this...
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p3998721.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: AW: auto completion search with solr using NGrams in SOLR
Posted by aniljayanti <an...@gmail.com>.
Hi,
thanks,
im searching with empname filed. want to search with both "empname" and
"title".
below is my changed code.
<fieldType name="edgytext" class="solr.TextField"
positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.LowerCaseTokenizerFactory"/>
<filter class="solr.EdgeNGramFilterFactory" minGramSize="1" maxGramSize="15"
/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.LowerCaseTokenizerFactory"/>
</analyzer>
</fieldType>
<field name="title" type="edgytext" indexed="true" stored="true"
omitNorms="true" omitTermFreqAndPositions="true"/>
<field name="empname" type="edgytext" indexed="true" stored="true"
omitNorms="true" omitTermFreqAndPositions="true" />
<copyField source="empname" dest="text"/>
<copyField source="title" dest="text"/>
SOLR Query :
http://localhost:8080/AC/select/?q=(*am*)&rows=500
1) want to search with "title" and "empname" both.
2) am i quering correctly with above url ??
but getting result like .....
p*am*
s*am*
ra*am*a
but i want the result like ....
aman
amar
amal
please help me in this...
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559p3998721.html
Sent from the Solr - User mailing list archive at Nabble.com.
AW: auto completion search with solr using NGrams in SOLR
Posted by Markus Klose <mk...@shi-gmbh.com>.
Your configuration of the fieldtype looks quite ok.
In what field are you searching? "text"? " empname " ? "autocomplete_text"?
If you are searching in "autocomplete_text " how do you add content to it? Is there another copyfield statement? If you are searching in "text" what fieldtype has that field.
You can use the analysis.jsp (linked at the admin console) to check what happens with your content at index time and search time and if there is a match.
Viele Grüße aus Augsburg
Markus Klose
SHI Elektronische Medien GmbH
-----Ursprüngliche Nachricht-----
Von: aniljayanti [mailto:anil.jayanti@gmail.com]
Gesendet: Mittwoch, 1. August 2012 12:05
An: solr-user@lucene.apache.org
Betreff: auto completion search with solr using NGrams in SOLR
I want to implement an auto completion search with solr using NGrams. If the user is searching for names of employees, then auto completion should be applied. ie.,
if types "j" then need to show the names starts with "j" if types "ja" then need to show the names starts with "ja" if types "jac" then need to show the names starts with "jak" if types "jack" then need to show the names starts with "jack"
Below is my configuration settings in schema.xml, Please suggest me if anything wrong.
below is my code in schema.xml
<fieldType name="edgytext" class="solr.TextField"
positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.KeywordTokenizerFactory" />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.EdgeNGramFilterFactory" minGramSize="1"
maxGramSize="15" />
</analyzer>
<analyzer type="query">
<tokenizer class="solr.KeywordTokenizerFactory" />
<filter class="solr.LowerCaseFilterFactory" />
</analyzer>
</fieldType>
<field name="empname" type="edgytext" indexed="true" stored="true" /> <field name="autocomplete_text" type="edgytext" indexed="true" stored="true"
omitNorms="true" omitTermFreqAndPositions="true" /> <copyField source="empname" dest="text" />
when im searching with name "mado" or "madonna" getting employees names.But when searching with "madon" not getting any data.
Please help me on this.
Thanks in Advance,
Anil.
--
View this message in context: http://lucene.472066.n3.nabble.com/auto-completion-search-with-solr-using-NGrams-in-SOLR-tp3998559.html
Sent from the Solr - User mailing list archive at Nabble.com.