You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by marcyborg <ma...@gmail.com> on 2016/09/28 08:25:46 UTC
solr hierarchical search hyponyms hypernyms
Hi,
Refering to this post Does Lucene / Solr support hypernyms and hyponyms?, I
want to use Solr as a semantic search engine, in order to do a hierarchical
search (narrower term, broader term), based on a thesaurus (which is in
XML/RDF format).
I've not found any tokenizers or filters being able of doing this
https://cwiki.apache.org/confluence/display/solr/Tokenizers
Have you any suggestions helping me?
Thanks a lot.
--
View this message in context: http://lucene.472066.n3.nabble.com/solr-hierarchical-search-hyponyms-hypernyms-tp4298385.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr hierarchical search hyponyms hypernyms
Posted by Andrea Gazzarini <gx...@gmail.com>.
Hi Francesco,
On 29/09/16 10:47, marcyborg wrote:
> Hi Andrea,
> Thanks very much for your complete reply.
> You're right, I'm new about Solr, so I'm sorry if'm asking trivial
> questions, or I'not exaustive in my questions!
>
> About the scenario, I try to explain it:
> I have to load the thesaurus in Solr core, and the user would be able to
> query that thesaurus, when searching a keyword.
> Getting into the details: I search a keyword T, if this T has BT and/or NT,
> I'd like to retrive that terms, and show that.
If you mean "user enters term T; in the thesaurus it is associated to a
BT and NT1, NT2; I want to expand the search using all those terms" then
I think the most trivial thing you can do is a simple Java standalone
program (or whatever language you prefer) that loads the thesaurus and
convert it in the plain synonyms format. Then, you can see in the Solr
reference guide [1] how to configure that (the default configuration
should already have set up, if I remember well, just a matter of
replacing the default synynoms.txt file included in the example).
I suggest you to start reading the reference guide and then go deeper
into the synonyms topic, which can be very tricky (the "pain in the ass"
Hoss mentioned in his answer).
Best,
Andrea
[1]
https://cwiki.apache.org/confluence/display/solr/Filter+Descriptions#FilterDescriptions-SynonymFilter
> I hope this clarifies the scenario!
>
> Ciao,
> Francesco
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/solr-hierarchical-search-hyponyms-hypernyms-tp4298385p4298569.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr hierarchical search hyponyms hypernyms
Posted by marcyborg <ma...@gmail.com>.
Hi Andrea,
Thanks very much for your complete reply.
You're right, I'm new about Solr, so I'm sorry if'm asking trivial
questions, or I'not exaustive in my questions!
About the scenario, I try to explain it:
I have to load the thesaurus in Solr core, and the user would be able to
query that thesaurus, when searching a keyword.
Getting into the details: I search a keyword T, if this T has BT and/or NT,
I'd like to retrive that terms, and show that.
I hope this clarifies the scenario!
Ciao,
Francesco
--
View this message in context: http://lucene.472066.n3.nabble.com/solr-hierarchical-search-hyponyms-hypernyms-tp4298385p4298569.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr hierarchical search hyponyms hypernyms
Posted by Andrea Gazzarini <gx...@gmail.com>.
Hi Francesco,
I think an information it's missing here: what are you trying to do
concretely? "Using Solr as semantic search engine" means at the same
time everything and nothing :) and (I guess) it involves something more
than a thesaurus.
Keeping things simple, and assuming your only concern is the thesaurus,
what is your search requirement / scenario / expectation? I mean, let's
assume you can index / manage the thesaurus hierarchy, something like
BT
---> T
------> NT1
------> NT2
------> NT3
What do you want to do with such data?
When the user search T you want a query expansion to T,BT?
Or you want to store that information at index time?
Or you want to reduce all T to BT?
What about the narrower terms? How do you imagine their role within the
search context?
In any case, there's nothing (AFAIK) in Solr that understands SKOS /
RDF. In the links you posted it seems someone already went in that
direction but I don't know those add-ons and (see above) I don't know
your needs.
Ciao,
Andrea
On 28/09/16 12:00, marcyborg wrote:
> Hi Andrea,
> Thanks very much for your helpful link!
> Is there a way to contact Hoss?
>
> About that link, I have two doubts:
> -) I'm using Solr (5.4.1) in schemaless mode, and I can't manage the
> schema.xml in order to customize the SynonymFilterFactory;
> -) If I copy my thesaurus in the synonyms.txt file, I lose all the hierarchy
> terms;
>
> According to your experience, these two plugins can be helpful for my
> purpose?
> http://redlink.co/adding-semantic-search-to-apache-solr/
> http://www.opensemanticsearch.org/
>
> Best,
> Francesco
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/solr-hierarchical-search-hyponyms-hypernyms-tp4298385p4298401.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr hierarchical search hyponyms hypernyms
Posted by marcyborg <ma...@gmail.com>.
Hi Shawn,
Thanks for your reply!
Until now, I was using Solr 5.4.1, and there were a lot of difference
between schemaless mode and with schema mode.
Reading your answer, it seems that now there is a better and simpler
management about this feature, right? I'm going to become familiar with Solr
6.2.1 version.
Thanks.
Best,
Francesco
--
View this message in context: http://lucene.472066.n3.nabble.com/solr-hierarchical-search-hyponyms-hypernyms-tp4298385p4298571.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr hierarchical search hyponyms hypernyms
Posted by Shawn Heisey <ap...@elyograg.org>.
On 9/28/2016 5:07 AM, marcyborg wrote:
> How can I edit managed-schema, if I'm using schemaless configuration?
> That file doesn't exist in Solr core.
When you are using the managed schema class, the "conf" directory (or
the zookeeper config location, when running SolrCloud) will contain a
file named "managed-schema" with no extension. If you are not using the
managed schema class, the file will be named "schema.xml".
As of Solr version 5.5, all example configs use the managed schema.
Before that, only the "data_driven_schema_configs" example used the
managed schema. Both before and after 5.5, the data driven example is
the one that is schemaless -- fields can be automatically added to
managed-schema by indexing.
Right at the top of the managed-schema file found in the example
configs, it will tell you to not edit the file manually. There are good
reasons for this comment being in that file, but if manual edits are the
only way the file ever gets changed, you can ignore that warning.
If you're running in schemaless mode, you do need to be careful that you
are not indexing data when you make manual edits, or the schema may
change as you are editing it.
Thanks,
Shawn
Re: solr hierarchical search hyponyms hypernyms
Posted by marcyborg <ma...@gmail.com>.
Hi Alex,
I was just asking, nothing about prayers and crying, because I'm new in this
mailing list.
How can I edit managed-schema, if I'm using schemaless configuration? That
file doesn't exist in Solr core.
Regards.
--
View this message in context: http://lucene.472066.n3.nabble.com/solr-hierarchical-search-hyponyms-hypernyms-tp4298385p4298418.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr hierarchical search hyponyms hypernyms
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
Hoss is like a deity, omniscient and omnipresent. But not answering all
prayers for help. You cry to him on this same mailing list and sometimes he
gives answers.
On the other hand, there is also Stump the Champ if you are going to the
Solr Revolution.
Regards,
Alex
P.s. you can actually still hand edit managed-schema, just reload the core
after and don't send documents or edit in UI until reload.
On 28 Sep 2016 5:01 PM, "marcyborg" <ma...@gmail.com> wrote:
> Hi Andrea,
> Thanks very much for your helpful link!
> Is there a way to contact Hoss?
>
> About that link, I have two doubts:
> -) I'm using Solr (5.4.1) in schemaless mode, and I can't manage the
> schema.xml in order to customize the SynonymFilterFactory;
> -) If I copy my thesaurus in the synonyms.txt file, I lose all the
> hierarchy
> terms;
>
> According to your experience, these two plugins can be helpful for my
> purpose?
> http://redlink.co/adding-semantic-search-to-apache-solr/
> http://www.opensemanticsearch.org/
>
> Best,
> Francesco
>
>
>
> --
> View this message in context: http://lucene.472066.n3.
> nabble.com/solr-hierarchical-search-hyponyms-hypernyms-
> tp4298385p4298401.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: solr hierarchical search hyponyms hypernyms
Posted by marcyborg <ma...@gmail.com>.
Hi Andrea,
Thanks very much for your helpful link!
Is there a way to contact Hoss?
About that link, I have two doubts:
-) I'm using Solr (5.4.1) in schemaless mode, and I can't manage the
schema.xml in order to customize the SynonymFilterFactory;
-) If I copy my thesaurus in the synonyms.txt file, I lose all the hierarchy
terms;
According to your experience, these two plugins can be helpful for my
purpose?
http://redlink.co/adding-semantic-search-to-apache-solr/
http://www.opensemanticsearch.org/
Best,
Francesco
--
View this message in context: http://lucene.472066.n3.nabble.com/solr-hierarchical-search-hyponyms-hypernyms-tp4298385p4298401.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr hierarchical search hyponyms hypernyms
Posted by Andrea Gazzarini <gx...@gmail.com>.
Hi, I don't believe there's something for doing that in Solr and
personally I'm not aware if someone developed such filter.
Please have a look at this exchange [1], where Hoss gave some useful
hints about this topic.
Best,
Andrea
[1] http://osdir.com/ml/solr-user.lucene.apache.org/2010-12/msg00470.html
On 28/09/16 10:25, marcyborg wrote:
> Hi,
> Refering to this post Does Lucene / Solr support hypernyms and hyponyms?, I
> want to use Solr as a semantic search engine, in order to do a hierarchical
> search (narrower term, broader term), based on a thesaurus (which is in
> XML/RDF format).
>
> I've not found any tokenizers or filters being able of doing this
> https://cwiki.apache.org/confluence/display/solr/Tokenizers
>
> Have you any suggestions helping me?
>
> Thanks a lot.
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/solr-hierarchical-search-hyponyms-hypernyms-tp4298385.html
> Sent from the Solr - User mailing list archive at Nabble.com.