You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by in...@joinso.cat on 2024/05/14 08:49:27 UTC
Synonyms and text search
Hi!
I am using Apache Solr 5.3.1, and I setup synonyms text file.
It works as expected.
For example, in my synonyms.txt file I have:
ML,mL,Ml,ml,Machine Learning
AI,Ai,aI,ai,Artificial Intelligence
I have 2 categories in a field with this terms: Machine Learning and
Artificial Intelligence
I tagged 4 documents with Artificial Intelligence and one document without
this tag.
But in this document I added in the description the word AI several times.
When I search for AI it returns the 4 documents but not the one not tagged
that it has the word AI inside the description.
There is anyway to return all the document that match the synonyms and also
that the text is in inside the document?
Atentamente / Atentament / Regards,
JOINSO
_____
IT Consulting and Advice
93 707 90 70 | <http://www.joinso.cat/> www.joinso.cat
Nota Legal: Este correo se rige por la política de privacidad descrita
<http://www.joinso.cat/politica-de-privacidad> aquí.
Nota Legal: Aquest correu es regeix per la política de privacitat descrita
<http://www.joinso.cat/ca/politica-de-privacitat> aquí.
Legal Disclaimer: This email is governed by privacy policy described
<http://www.joinso.cat/en/privacy-policy> here.
RE: Synonyms and text search
Posted by in...@joinso.cat.
The schema is like this:
Field: sm_vid_Category
Field-Type:org.apache.solr.schema.StrField
In this field, we have this value : Artificial Intelligence
And there is 4 documents that has this value.
The description field belongs to:
Field: content
Field-Type:
org.apache.solr.schema.TextField
PI Gap:
100
Docs:
<http://admin:$so23os32@apache-solr-dev.food4rhino.com:8983/solr/#/drupal_dev/query?q=content:[*%20TO%20*]> 171
Here we can found for example, the word “AI” than in synonyms belongs to Artificial Intelligence
If I do the search with AI only returns the 4 documents that belongs to sm_vid_Category
But in the other document that contents this in the description:
AI AI AI AI AI AI AI AI
skjdskjdkjdskdj
AI AI AI AI AI AI AI AI
sdlsldsjdsjdjs
AI AI AI AI AI AI AI AI
ssdjashdkjshd asdhashdakhd
it never return in the search if I search “AI”, but it return the document if I search “skjdskjdkjdskdj” or “sdlsldsjdsjdjs”
Regards,
Jordi
JOINSO
-----Mensaje original-----
De: Mikhail Khludnev <mk...@apache.org>
Enviado el: martes, 14 de mayo de 2024 15:01
Para: users@solr.apache.org
Asunto: Re: Synonyms and text search
It depends on schema and query parser details. You may use Analysis screen, debugQuery=true, and explainOther=id:123 to troubleshoot this case.
On Tue, May 14, 2024 at 11:50 AM < <ma...@joinso.cat> info@joinso.cat> wrote:
> Hi!
> I am using Apache Solr 5.3.1, and I setup synonyms text file.
> It works as expected.
>
> For example, in my synonyms.txt file I have:
>
> ML,mL,Ml,ml,Machine Learning
>
> AI,Ai,aI,ai,Artificial Intelligence
>
>
> I have 2 categories in a field with this terms: Machine Learning and
> Artificial Intelligence I tagged 4 documents with Artificial
> Intelligence and one document without this tag.
> But in this document I added in the description the word “AI” several
> times.
>
> When I search for AI it returns the 4 documents but not the one not
> tagged that it has the word AI inside the description.
> There is anyway to return all the document that match the synonyms and
> also that the text is in inside the document?
>
>
>
> Atentamente / Atentament / Regards,
> JOINSO
> ------------------------------
>
> [image: JOINSO IT CONSULTING AND ADVICE]
>
> IT Consulting and Advice
>
> 93 707 90 70 | <http://www.joinso.cat> www.joinso.cat
>
>
>
>
>
>
> Nota Legal: Este correo se rige por la política de privacidad descrita
> aquí < <http://www.joinso.cat/politica-de-privacidad> http://www.joinso.cat/politica-de-privacidad>.
>
> Nota Legal: Aquest correu es regeix per la política de privacitat
> descrita aquí < <http://www.joinso.cat/ca/politica-de-privacitat> http://www.joinso.cat/ca/politica-de-privacitat>.
>
> Legal Disclaimer: This email is governed by privacy policy described
> here < <http://www.joinso.cat/en/privacy-policy> http://www.joinso.cat/en/privacy-policy>.
>
>
>
--
Sincerely yours
Mikhail Khludnev
RE: Synonyms and text search
Posted by in...@joinso.cat.
Hi!
Now my synonyms.txt file is like this:
ml,Machine Learning
ai,Artificial Intelligence
But when I search I only get results for documents categorized with " Artificial Intelligence" when I search "ai".
But I do not get in the results list a document with the word "ai" inside.
Any idea to get all the documents categorized as " Artificial Intelligence" and also the ones with the word "ai" inside?
Regards,
Jordi
-----Mensaje original-----
De: Charlie Hull <ch...@opensourceconnections.com>
Enviado el: miércoles, 15 de mayo de 2024 14:54
Para: users@solr.apache.org
Asunto: Re: Synonyms and text search
oh and you generally use synonyms for the 'AI = artificial intelligence'
use case, the lower/upper case stuff should be handled by the analysis chain (that turns your source documents into a normalised form for
indexing)
https://solr.apache.org/guide/solr/latest/indexing-guide/document-analysis.html
Charlie
On 14/05/2024 14:01, Mikhail Khludnev wrote:
> It depends on schema and query parser details. You may use Analysis
> screen, debugQuery=true, and explainOther=id:123 to troubleshoot this case.
>
> On Tue, May 14, 2024 at 11:50 AM <in...@joinso.cat> wrote:
>
>> Hi!
>> I am using Apache Solr 5.3.1, and I setup synonyms text file.
>> It works as expected.
>>
>> For example, in my synonyms.txt file I have:
>>
>> ML,mL,Ml,ml,Machine Learning
>>
>> AI,Ai,aI,ai,Artificial Intelligence
>>
>>
>> I have 2 categories in a field with this terms: Machine Learning and
>> Artificial Intelligence I tagged 4 documents with Artificial
>> Intelligence and one document without this tag.
>> But in this document I added in the description the word “AI” several
>> times.
>>
>> When I search for AI it returns the 4 documents but not the one not
>> tagged that it has the word AI inside the description.
>> There is anyway to return all the document that match the synonyms
>> and also that the text is in inside the document?
>>
>>
>>
>> Atentamente / Atentament / Regards,
>> JOINSO
>> ------------------------------
>>
>> [image: JOINSO IT CONSULTING AND ADVICE]
>>
>> IT Consulting and Advice
>>
>> 93 707 90 70 | www.joinso.cat
>>
>>
>>
>>
>>
>>
>> Nota Legal: Este correo se rige por la política de privacidad
>> descrita aquí <http://www.joinso.cat/politica-de-privacidad>.
>>
>> Nota Legal: Aquest correu es regeix per la política de privacitat
>> descrita aquí <http://www.joinso.cat/ca/politica-de-privacitat>.
>>
>> Legal Disclaimer: This email is governed by privacy policy described
>> here <http://www.joinso.cat/en/privacy-policy>.
>>
>>
>>
>
--
Charlie Hull - Managing Consultant at OpenSource Connections Limited Founding member of The Search Network and co-author of Searching the Enterprise
tel/fax: +44 (0)8700 118334
mobile: +44 (0)7767 825828
OpenSource Connections Europe GmbH | Pappelallee 78/79 | 10437 Berlin Amtsgericht Charlottenburg | HRB 230712 B
Geschäftsführer: John M. Woodell | David E. Pugh
Finanzamt: Berlin Finanzamt für Körperschaften II
Re: Synonyms and text search
Posted by Charlie Hull <ch...@opensourceconnections.com>.
oh and you generally use synonyms for the 'AI = artificial intelligence'
use case, the lower/upper case stuff should be handled by the analysis
chain (that turns your source documents into a normalised form for
indexing)
https://solr.apache.org/guide/solr/latest/indexing-guide/document-analysis.html
Charlie
On 14/05/2024 14:01, Mikhail Khludnev wrote:
> It depends on schema and query parser details. You may use Analysis screen,
> debugQuery=true, and explainOther=id:123 to troubleshoot this case.
>
> On Tue, May 14, 2024 at 11:50 AM <in...@joinso.cat> wrote:
>
>> Hi!
>> I am using Apache Solr 5.3.1, and I setup synonyms text file.
>> It works as expected.
>>
>> For example, in my synonyms.txt file I have:
>>
>> ML,mL,Ml,ml,Machine Learning
>>
>> AI,Ai,aI,ai,Artificial Intelligence
>>
>>
>> I have 2 categories in a field with this terms: Machine Learning and
>> Artificial Intelligence
>> I tagged 4 documents with Artificial Intelligence and one document without
>> this tag.
>> But in this document I added in the description the word “AI” several
>> times.
>>
>> When I search for AI it returns the 4 documents but not the one not tagged
>> that it has the word AI inside the description.
>> There is anyway to return all the document that match the synonyms and
>> also that the text is in inside the document?
>>
>>
>>
>> Atentamente / Atentament / Regards,
>> JOINSO
>> ------------------------------
>>
>> [image: JOINSO IT CONSULTING AND ADVICE]
>>
>> IT Consulting and Advice
>>
>> 93 707 90 70 | www.joinso.cat
>>
>>
>>
>>
>>
>>
>> Nota Legal: Este correo se rige por la política de privacidad descrita
>> aquí <http://www.joinso.cat/politica-de-privacidad>.
>>
>> Nota Legal: Aquest correu es regeix per la política de privacitat descrita
>> aquí <http://www.joinso.cat/ca/politica-de-privacitat>.
>>
>> Legal Disclaimer: This email is governed by privacy policy described here
>> <http://www.joinso.cat/en/privacy-policy>.
>>
>>
>>
>
--
Charlie Hull - Managing Consultant at OpenSource Connections Limited
Founding member of The Search Network and co-author of Searching the Enterprise
tel/fax: +44 (0)8700 118334
mobile: +44 (0)7767 825828
OpenSource Connections Europe GmbH | Pappelallee 78/79 | 10437 Berlin
Amtsgericht Charlottenburg | HRB 230712 B
Geschäftsführer: John M. Woodell | David E. Pugh
Finanzamt: Berlin Finanzamt für Körperschaften II
Re: Synonyms and text search
Posted by Mikhail Khludnev <mk...@apache.org>.
It depends on schema and query parser details. You may use Analysis screen,
debugQuery=true, and explainOther=id:123 to troubleshoot this case.
On Tue, May 14, 2024 at 11:50 AM <in...@joinso.cat> wrote:
> Hi!
> I am using Apache Solr 5.3.1, and I setup synonyms text file.
> It works as expected.
>
> For example, in my synonyms.txt file I have:
>
> ML,mL,Ml,ml,Machine Learning
>
> AI,Ai,aI,ai,Artificial Intelligence
>
>
> I have 2 categories in a field with this terms: Machine Learning and
> Artificial Intelligence
> I tagged 4 documents with Artificial Intelligence and one document without
> this tag.
> But in this document I added in the description the word “AI” several
> times.
>
> When I search for AI it returns the 4 documents but not the one not tagged
> that it has the word AI inside the description.
> There is anyway to return all the document that match the synonyms and
> also that the text is in inside the document?
>
>
>
> Atentamente / Atentament / Regards,
> JOINSO
> ------------------------------
>
> [image: JOINSO IT CONSULTING AND ADVICE]
>
> IT Consulting and Advice
>
> 93 707 90 70 | www.joinso.cat
>
>
>
>
>
>
> Nota Legal: Este correo se rige por la política de privacidad descrita
> aquí <http://www.joinso.cat/politica-de-privacidad>.
>
> Nota Legal: Aquest correu es regeix per la política de privacitat descrita
> aquí <http://www.joinso.cat/ca/politica-de-privacitat>.
>
> Legal Disclaimer: This email is governed by privacy policy described here
> <http://www.joinso.cat/en/privacy-policy>.
>
>
>
--
Sincerely yours
Mikhail Khludnev