You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stanbol.apache.org by Rupert Westenthaler <ru...@salzburgresearch.at> on 2011/10/26 13:10:22 UTC
Updates to LMF/Stanbol integration
Hi Stanbol Community
Sebastian, Jakob Frank and my self had a meeting about the next steps in the integration of the LMF and Apache Stanbol. This mail provides an overview about the discussed points and my opinion how the integration with Stanbol could look like.
We decided to start of the integration with the following two assets: (1) RdfPathLanguage [1] and (2) LMF semantic search component [2]. (*)
In the following I will give an short overview about the necessary steps for integrating this two assets with Stanbol:
(1) make the RdfPathLanguage [1] generic so we can use it within Stanbol. The decision was to define an own Java API for this specification and manage it as an own library (outside of Stanbol). This is mainly because we think this specification is of general interest as as query language/indexing language for the Linked Data and therefore it makes sense to keep the Java API definition in depended of any specific project.
* Sebastian will take the lead in defining the API. Jakob Frank has also done a lot of work on this. So he will also actively contribute to that.
* The current Implementation - based on the Kiwi Triple store will be adapted to the updated API.
* I will work on implementations for Clerezza and the Entityhub.
* It was also discussed to provide an implementation based on SPRAQL. This could be used to retrieve data referenced in paths from Sparql Endpoints.
* In addition I think it would make also sense to provide an implementation based on the CMA adapter to allowing retrieving data from CMS. Suat it would be great if you could check if this would make sense.
* As a first usage of the RdfPathLanguage within Stanbol I will replace the current FieldMapping infrastructure used by the Entityhub with the RdfPathLanguage. This will also the test case for the Clerezza and Entityhub based implementations.
(2) Integrate the LMF semantic search component [2] with Stanbol: The LMF semantic search is based on semantic index configurations that are based on the RdfPathLanguage [1]. Based on this configuration a Solr schema (schema.xml) is generated that is than used to store the data of the semantic index. The actual search interface is the normal Solr RESTful API - just the data stored within the index are smarter. This has two very big advantage: For users - that a lot of programmers do already know how to use Solr and for us - that we do not need to reinvent to rich query interface of Solr (think about features such as facets, /mlt, ranking functions …). (**)
The usage of the LMF semantic search component within Stanbol required the following three features (I already started to work on):
* Enable the RESTful API for SolrCores managed by Stanbol (STANBOL-353): This is needed to allow the usage of the Solr RESTful API to query the semantic index. (nearly finished)
* Improve the Component that manages internal SolrIncexes to better support update and removal of SolrIndexes. This is needed to allow the LMF to update schema.xml files, deleting and reindexing indexes after incompatible changes. However this will than also allow to "replace" a small default index for DBpedia with a bigger version or a version with additional fields, languages … (plan to start work on that next week)
* Implementation of the RdfPathLanguage based on Clerezza (to index EnhancementResults) and the Entityhub (to include additional Information for suggested Entities). Optional: Implementation based on the CMS Adapter to also include additional data from a connected CMS. This is required to allow updates to the Semantic Index when running within Stanbol.
The usage of the LMF semantic search component within Stanbol will also require to integrate is within the "/contentub". With the following points I try to highlight the main topics that need further investigations/discussions:
* The LMF semantic search component overlaps greatly with the recently by Anil contributed "contenthub/search/engines/solr" component. Related to this it would be great if Anil could have a look at [2] and check for similarities/differencies and possible integration paths.
* The Semantic Search Inteface: The Contenthub currently defines it's own query API (supports keyword based search as well as "field -> value" like constraints, supports facets). The LMF directly exposes the RESTful API of the semantic Solr index. I strongly prefer the approach of the LMF, because the two points already described above. But I am also the opinion that a semantic search interface should at least provide the following three additional features:
1. Query preprocessing: e.g. substitute "Paris" in the query with "http://dbpedia.org/resource/Paris";
2. Entity Facets: if a keyword matches a Entity (e.g. "Paris" -> "dbpedia:Paris", "dbpedia:Paris_Texas", "dbpedia:Paris_Hilton") than provide a Facet to the user over such possible matches;
3. Semantic Facets: if a user uses an instance of an ontology type (e.g. a Place, Person, Organization) in a query, that provide facets over semantic relations for such types (e.g. fiends for persons, products/services for Organizations, nearby Points-Of-Interests for Places, Participants for Events, …). To implement features like that we need components that provide query preprocessing capabilities based on data available in the Entityhub, Ontonet … . To me it seams that the contenthub/search/engines/ontologyresource component provides already some functionality related to this so this might be a good starting point.
* How is the relation to the Factstore? As far as I understand the Factstore provides the possibility to first define how facts do look like and than allows to efficiently CRUD over such facts. Related to this one can ask two questions:
1. Could one use the RdfPathLanguage to define Fact schemas and/or to create Fact instances?
2. could it be possible to integrate the LMF semantic search component as a FactStore implementation into Stanbol?
Next Steps:
Work in the coming weeks need to focus on making the IKS developer meeting end of November in Salzburg [3] as productive as possible. Let me also note that all interested people of the Stanbol Community are also very welcome to join.
To prepare this meeting it would be important if we could already start discussion about the main topics - especially the /contenthub before this meeting on this mailing list. Implementation wise we (the LMF developer and me) will try realize a first integration until the start of the meeting. This will not try to integrate the LMF semantic search component with the content hub but rather concentrate on the RdfPathLanguage and the necessary improvements to the management of Solr indexes within stanbol. The should allow us to run LMF demos like [4] within the Stanbol environment. But without the possibility to update the semantic index or to index additional documents - because this would need than a real integration with the Contenthub.
This two things should than allow us to use the face to face meeting in Salzburg to really work on the integration and hopefully to show a demo that allows to
1. send a Document to the Contenthub (optionally by using the CMS adapter)
2. enhancing it with the enhancer
3. storing the enhanced document within the semantic index
4. use the faced search interface to navigate through documents
at the end of the meeting.
best
Rupert Westenthaler
[1] http://code.google.com/p/kiwi/wiki/RdfPathLanguage
[2] http://code.google.com/p/kiwi/source/browse/#hg%2Flmf-search%2Fsrc%2Fmain%2Fjava%2Fat%2Fnewmedialab%2Flmf%2Fsearch
[3] http://wiki.iks-project.eu/index.php/IntegrationHackathonSalzburg
[4] http://labs.newmedialab.at/LMF/sn/search/suche.html
(*) Note that within this we excluded the topic of how to integrate the LMF rule engine. Not because this is not important, because I am do not have the according knowledge about the rule implementation of Stanbol to discuss this.
(**) As a side note: The use of Solr by the Semantic Search component is very different of the one of the Entityhub SolrYard. First creates a dedicated schema based on a configuration - it can only index specific data, but produces a nice schema that is easy to understand and to use by users - very important for the semantic search use case. The later can store any kind of RDF data with a singe schema - very important when managing very diverse entities but the generic schema needs to use complex prefixes, and suffixes in combination with dynamic fields that would it make very hard for users to write Solr queries.
Re: Updates to LMF/Stanbol integration
Posted by Olivier Grisel <ol...@ensta.org>.
2011/10/28 Rupert Westenthaler <ru...@salzburgresearch.at>:
>
> On 27.10.2011, at 16:59, Ali Anil SINACI wrote:
>>> * The Semantic Search Inteface: The Contenthub currently defines it's own query API (supports keyword based search as well as "field -> value" like constraints, supports facets). The LMF directly exposes the RESTful API of the semantic Solr index. I strongly prefer the approach of the LMF, because the two points already described above.
>>
>> We think that we do not have to make a selection here. We can keep a simple wrap-up on the Solr interface (contenthub's own query API) while providing the Solr RESTful API as is. IMO a wrap-up on Solr interface would be beneficial. On the other hand, in this interface we try to make use of an ontology to be used in OntologyResourceSearchEngine. This might help to figure out new keywords based on the subsumption hierarchy inside the ontology. However, I think this may lead to performance issues and may not be useful at all. We can decide on this later.
>
> You forgot to mention one additional advantage for using the Solr RESTful API: If we do that one could create the Semantic Index and than copy it over to some other SolrServer without the need to run Stanbol directly on the production infrastructure.
+1
> In general I would suggest to first focus the discussion on the unique features we would like to provide with the Semantic Search component. I already included three features I would like to have in my first Mail (Query preprocessing, Entity Facets, Semantic Facets). As you now mention the OntologyResourceSearchEngine is very relevant in relation to such features.
> However adding such features must not necessarily mean to create an own query language. One could also try to add such features directly to Solr by implementing some Solr extensions.
+1
By avoiding adding a yet a new HTTP layer for basic Solr queries we
will make it possible for CMS that already have Solr integration (and
they are numerous) to leverage the "smart" stanbol semantic indexing
without having to change a single line of code in their Solr
connector. That would be great for adoption.
Off course we might still want to develop adtional HTTP endpoints for
things that are not covered by the default Solr API and further add a
JSON-LD response serializer but that should prevent the users to
directly call the raw Solr API if they decide to do so.
--
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel
Semantic Query Interface for the Contenthub (was Re: Updates to LMF/Stanbol integration)
Posted by Rupert Westenthaler <ru...@gmail.com>.
Hi Anil, all
Created also an own thread for this topic …
Currently my focus is not (yet) on the implementation, but much more on how features like
* Entity substitution (replacing "Paris" in a query with the entity "http://dbpedia.org/resource/Paris")
* Entity Facets (adding facets for Entities used or substituted in the Users query)
* Semantic Facets (adding Facets and possible special UI elements) based on the type of Entities used in the Query or the type Instances selected by query)
I clearly agree that the
* Entityhub could be used to implement Entity substitution and Entity Facets
* Ontologies managed by the Ontonet component could be used to provide Semantic Facets
and that the current implementations of the Contenthub provides already a lot of functionality related to that. However I think before we start to implement we need first a more clear picture how all such features could help End-Users to interact with the Knowledge-Space we want to manage in the Contenthub.
So if people could share Ideas, existing Examples of cool search interfaces … it would be really helpful!
Some links to existing resources of the IKS project:
* For the first Community Meeting of the IKS project I created an presentation on semantic search [1]. I think some of the concepts presented by this presentation are still highly relevant.
* The UserStories[2]
* "S01" mentions "Entity Facets"
* "S04" could be supported by "Entity substitution" but goes much further than that
* "S20" suggest query and result contextualization based on user specific profiles. Maybe a feature we might also want to consider when defining the API
* "S27" includes "Entity substitution" and "Semantic Facets"
Sebastian/Jakob maybe you could also provide a short overview about related resources/demos based on the work on the LMF
best
Rupert
[1] http://www.interactive-knowledge.org/sites/www.interactive-knowledge.org/files/iks_semantic_search_20090528_rw.pdf
[2] http://wiki.iks-project.eu/index.php/User-stories
On 28.10.2011, at 10:07, Ali Anil SINACI wrote:
>
>
>>>> * The Semantic Search Inteface: The Contenthub currently defines it's own query API (supports keyword based search as well as "field -> value" like constraints, supports facets). The LMF directly exposes the RESTful API of the semantic Solr index. I strongly prefer the approach of the LMF, because the two points already described above.
>>> We think that we do not have to make a selection here. We can keep a simple wrap-up on the Solr interface (contenthub's own query API) while providing the Solr RESTful API as is. IMO a wrap-up on Solr interface would be beneficial. On the other hand, in this interface we try to make use of an ontology to be used in OntologyResourceSearchEngine. This might help to figure out new keywords based on the subsumption hierarchy inside the ontology. However, I think this may lead to performance issues and may not be useful at all. We can decide on this later.
>> You forgot to mention one additional advantage for using the Solr RESTful API: If we do that one could create the Semantic Index and than copy it over to some other SolrServer without the need to run Stanbol directly on the production infrastructure.
>>
>> In general I would suggest to first focus the discussion on the unique features we would like to provide with the Semantic Search component. I already included three features I would like to have in my first Mail (Query preprocessing, Entity Facets, Semantic Facets). As you now mention the OntologyResourceSearchEngine is very relevant in relation to such features.
>> However adding such features must not necessarily mean to create an own query language. One could also try to add such features directly to Solr by implementing some Solr extensions.
>>
>
> Let me briefly comment in your suggestions about the semantic search.
>
>>>> But I am also the opinion that a semantic search interface should at least provide the following three additional features:
>>>> 1. Query preprocessing: e.g. substitute "Paris" in the query with "http://dbpedia.org/resource/Paris";
>>>> 2. Entity Facets: if a keyword matches a Entity (e.g. "Paris" -> "dbpedia:Paris", "dbpedia:Paris_Texas", "dbpedia:Paris_Hilton") than provide a Facet to the user over such possible nnnnnnnnmatches;
>
> As far as we understand, first and second features will be handled by querying the Entityhub with the query keyword (Paris) i.e the first entity obtained from the Entityhub will help us to recognize its type and the other entities will be served as facet values of Paris facet.
>
>>>> 3. Semantic Facets: if a user uses an instance of an ontology type (e.g. a Place, Person, Organization) in a query, that provide facets over semantic relations for such types (e.g. fiends for persons, products/services for Organizations, nearby Points-Of-Interests for Places, Participants for Events, …). To implement features like that we need components that provide query preprocessing capabilities based on data available in the Entityhub, Ontonet … . To me it seams that the contenthub/search/engines/ontologyresource component provides already some functionality related to this so this might be a good starting point.
>
> Currently, we are trying to integrate an exploration mechanism like you said above. It is also based on DBPedia ontology. OntologyResourceEngine can be used for this purpose for the user registered ontologies. Current implementation of this engine only computes closures by exploiting the hierarchy in the ontology. RDFPath Programs can also be an option at this point. With an RDF Path Program user may specify the relations to be used in the exploration process. But I think this means the user decides beforehand which fields should be presented to him as exploration fields. I think this is open to discussion.
>
>> best
>> Rupert
>>
>
> Regards,
> Anil.
Re: RdfPathLanguage: Next steps (was Re: Updates to LMF/Stanbol integration)
Posted by Suat Gonul <su...@gmail.com>.
Hi,
On 10/28/2011 11:26 AM, Rupert Westenthaler wrote:
> Hi Sebastian, Jakob, Stanbol team
>
> Based on the positive feedback of Anil to participate on this I decided to create an own thread to plan the next steps.
>
> Next steps:
>
> The first step will be to define Java API that allows to provide different implementations. I think the Idea was to create an own Project (should we use Github or GoogleCode? MIT/BSD/Apache licensed?) that only focusses on the Specification of the Language [1] and the Java API. Sebastian needs to take the lead of this. If I remember correctly his plan was to start this next week.
>
> As soon as a first version of this specification is available we can start to work on implementations.
>
> * Kiwi TripleStore: I assume Sebastian and Jakob will work on that
> * Clerezza: Anil could you take the lead for that?
Anil says OK for taking the lead for this.
> * Entityhub: This will be my responsibility
> * SPARQL based implementation: I think that would be interesting - someone interested to work on that?
> * CMS Adapter: Suat could you follow this effort and check for possible usage scenarios.
Currently, CMS Adapter can generate RDF from JCR/CMIS content
repositories. This RDF conforms to the simple specification that we
created while implementing bidirectional mapping feature. As a first
attempt, this generated RDF can be queried with RDFPathLanguage.
For the time being, I don't have a concrete use case yet for directly
querying the CMS with RDFPathLanguage. I would be glad hear any use
cases. I think CMS developers might have more ideas at this point.
Best,
Suat
> * Fact Store: This could be also an interesting. But same as for the CMS Adapter we need first to check usage scenarios.
>
> best
> Rupert
>
>
>
> On 28.10.2011, at 10:07, Ali Anil SINACI wrote:
>
>> Dear Rupert,
>>
>> On 10/28/2011 08:47 AM, Rupert Westenthaler wrote:
>>> On 27.10.2011, at 16:59, Ali Anil SINACI wrote:
>>>>> * The LMF semantic search component overlaps greatly with the recently by Anil contributed "contenthub/search/engines/solr" component. Related to this it would be great if Anil could have a look at [2] and check for similarities/differencies and possible integration paths.
>>>>>
>>>> I had a look on the semantic search component of LMF. As you pointed it out, LMF semantic search provides a convenient way to index any part of documents with the help of RDFPath Language. I think that we can make use of this feature in contenthub. As I described in my previous e-mail, currently, contenthub indexes a number of semantic fields based on DBPedia relations. These are hardcoded relations. RDFPath language can be used to indicate specific semantic fields to be indexed along with the content itself. Let me describe the thing in our mind in a scenario:
>>>>
>>>> A user provides a domain ontology (e.g. music domain), submits to Entityhub to be used in the enhancement process. Suppose the domain ontology includes vast of information about artists, their albums etc... I assume that this ontology does not include conceptual definitions (it only includes Abox definitions). User writes an RDF Path Program (in LMF terminology) to indicate the fields to be indexed when a content item has an enhancement related with any path in that program. Suppose user submits a content item along with the RDF Path Program(s) to be used to determine the fields to be indexed. Enhancement engines find an entity (or lots of entities). Now, we execute the selected RDF Path Program(s) and embed the results into the Solr representation of the content item.
>>>>
>>>> If you have any other suggestions, please let me know so that we can discuss in detail (in SRDC) before the meeting.
>>>>
>>> This is exactly what I was thinking about. Let me only add that such additional Knowledge to be included within the Semantic Index might not only come from the Entityhub, but also from other sources (like the CMS via the CMS adapter)
>>>
>>> I you would like to help me with an Implementation of the RdfPathLanguage (e.g. the Clerezza based Implementation, or maybe a Jena bases implementation) please let me know. Help would be greatly welcome, because I have already a lot of things on my TODO list before the Meeting in November (such as defining a Proposal for the Stanbol Enhancement Structure).
>>>
>> We would like to get involved in the implementation of RDFPathLanguage for Stanbol. We plan to work on this starting from next week. I think you& LMF team already have a design in your mind. I will appreciate if you could share your thoughts with us.
>>
>>>>> * The Semantic Search Inteface: The Contenthub currently defines it's own query API (supports keyword based search as well as "field -> value" like constraints, supports facets). The LMF directly exposes the RESTful API of the semantic Solr index. I strongly prefer the approach of the LMF, because the two points already described above.
>>>> We think that we do not have to make a selection here. We can keep a simple wrap-up on the Solr interface (contenthub's own query API) while providing the Solr RESTful API as is. IMO a wrap-up on Solr interface would be beneficial. On the other hand, in this interface we try to make use of an ontology to be used in OntologyResourceSearchEngine. This might help to figure out new keywords based on the subsumption hierarchy inside the ontology. However, I think this may lead to performance issues and may not be useful at all. We can decide on this later.
>>> You forgot to mention one additional advantage for using the Solr RESTful API: If we do that one could create the Semantic Index and than copy it over to some other SolrServer without the need to run Stanbol directly on the production infrastructure.
>>>
>>> In general I would suggest to first focus the discussion on the unique features we would like to provide with the Semantic Search component. I already included three features I would like to have in my first Mail (Query preprocessing, Entity Facets, Semantic Facets). As you now mention the OntologyResourceSearchEngine is very relevant in relation to such features.
>>> However adding such features must not necessarily mean to create an own query language. One could also try to add such features directly to Solr by implementing some Solr extensions.
>>>
>> Let me briefly comment in your suggestions about the semantic search.
>>
>>>>> But I am also the opinion that a semantic search interface should at least provide the following three additional features:
>>>>> 1. Query preprocessing: e.g. substitute "Paris" in the query with "http://dbpedia.org/resource/Paris";
>>>>> 2. Entity Facets: if a keyword matches a Entity (e.g. "Paris" -> "dbpedia:Paris", "dbpedia:Paris_Texas", "dbpedia:Paris_Hilton") than provide a Facet to the user over such possible nnnnnnnnmatches;
>> As far as we understand, first and second features will be handled by querying the Entityhub with the query keyword (Paris) i.e the first entity obtained from the Entityhub will help us to recognize its type and the other entities will be served as facet values of Paris facet.
>>
>>>>> 3. Semantic Facets: if a user uses an instance of an ontology type (e.g. a Place, Person, Organization) in a query, that provide facets over semantic relations for such types (e.g. fiends for persons, products/services for Organizations, nearby Points-Of-Interests for Places, Participants for Events, …). To implement features like that we need components that provide query preprocessing capabilities based on data available in the Entityhub, Ontonet … . To me it seams that the contenthub/search/engines/ontologyresource component provides already some functionality related to this so this might be a good starting point.
>> Currently, we are trying to integrate an exploration mechanism like you said above. It is also based on DBPedia ontology. OntologyResourceEngine can be used for this purpose for the user registered ontologies. Current implementation of this engine only computes closures by exploiting the hierarchy in the ontology. RDFPath Programs can also be an option at this point. With an RDF Path Program user may specify the relations to be used in the exploration process. But I think this means the user decides beforehand which fields should be presented to him as exploration fields. I think this is open to discussion.
>>
>>> best
>>> Rupert
>>>
>> Regards,
>> Anil.
>
Re: RdfPathLanguage: Next steps (was Re: Updates to LMF/Stanbol integration)
Posted by valentina presutti <va...@cnr.it>.
Hi Sebastian,
Andrea will go through the RDFPath in order to better understand how the work we've done can benefit from RDFPath.
He'll be back with more details.
Just to give you an idea, actually our approach is focused on extracting paths by looking at their occurrences in the data.
Maybe what we call paths for you are path-types.
Such path-types are very useful for producing a summary of a dataset and for identifying invariances in the usage of data.
In few words, you can use such information for better accessing the data (e.g., as a support to developers designing queries for a dataset, designing interfaces, etc.)
My intuition (but let me look more in details at RDFPath, I could be wrong) is that we could use an implementation of RDFPath for more efficiently gather such information and extract dataset patterns.
Val
On Nov 2, 2011, at 9:48 AM, Sebastian Schaffert wrote:
> Dear Valentina and all,
>
> This is indeed very interesting. We developed "RDFPath" originally as a configuration language for the semantic search index, but later realised that it is a very natural way of querying over the Linked Data Cloud. In contrast to SPARQL, it is resource-centric like Linked Data, and it is restricted in a way that is more suitable for Linked Data servers. The path language is documented here:
>
> http://code.google.com/p/kiwi/wiki/RdfPathLanguage
>
> Combining this with pattern extraction would be very interesting, e.g. for automatically suggesting a configuration of our search index.
>
>
> Greetings,
>
> Sebastian
>
>
> Am 28.10.2011 um 15:17 schrieb valentina.presutti@cnr.it:
>
>> Actually, we have developed experience on this issue (SPARQL-based implementation for path extraction). I think that Andrea could lead the task and bring in the work already done as a starting point.
>>
>> We have also identified a number of indicators that is worth to compute and associate to paths. Everything is in the two papers (and the vocabulary) referred by Andrea. For example, we compute the occurrences of paths in a dataset, their popularity (we have defined a measure for this), and other ones.
>> They can be optional features offered by the SPARQL-based services.
>>
>> Val
>>
>> Quoting nuzzoles@cs.unibo.it:
>>
>>> Hi Rupert, Stanbol team
>>>
>>> I am interested in the SPARQL based implementation.
>>> We have recently worked on extracting paths from linked data.
>>> The work has been just presented at ISWC [1, 2].
>>> Hence I have already some experience with the issue and some preliminary
>>> results to reuse:
>>>
>>> * a number of scripts for extracting paths
>>> * a vocabulary for representing and storing them [3]
>>>
>>> I am not sure I will be in Salzburg, but we can discuss this possibility
>>> for including these issues at the hackathon.
>>>
>>> --
>>> Andrea Giovanni Nuzzolese
>>> Semantic Technology Laboratory (STLab)
>>> Institute for Cognitive Science and Technology (ISTC)
>>> National Research Council (CNR)
>>> Via Nomentana 56, Roma - Italy
>>>
>>>
>>> [1] http://www.stlab.istc.cnr.it/documents/papers/cold2011.pdf
>>> [2] http://www.stlab.istc.cnr.it/documents/papers/wikilinkpatterns.pdf
>>> [3] http://www.ontologydesignpatterns.org/ont/lod-analysis-properties.owl
>>>
>>>
>>>
>>>> Hi Sebastian, Jakob, Stanbol team
>>>>
>>>> Based on the positive feedback of Anil to participate on this I decided to
>>>> create an own thread to plan the next steps.
>>>>
>>>> Next steps:
>>>>
>>>> The first step will be to define Java API that allows to provide different
>>>> implementations. I think the Idea was to create an own Project (should we
>>>> use Github or GoogleCode? MIT/BSD/Apache licensed?) that only focusses on
>>>> the Specification of the Language [1] and the Java API. Sebastian needs to
>>>> take the lead of this. If I remember correctly his plan was to start this
>>>> next week.
>>>>
>>>> As soon as a first version of this specification is available we can start
>>>> to work on implementations.
>>>>
>>>> * Kiwi TripleStore: I assume Sebastian and Jakob will work on that
>>>> * Clerezza: Anil could you take the lead for that?
>>>> * Entityhub: This will be my responsibility
>>>> * SPARQL based implementation: I think that would be interesting - someone
>>>> interested to work on that?
>>>> * CMS Adapter: Suat could you follow this effort and check for possible
>>>> usage scenarios.
>>>> * Fact Store: This could be also an interesting. But same as for the CMS
>>>> Adapter we need first to check usage scenarios.
>>>>
>>>> best
>>>> Rupert
>>>>
>>>>
>>>>
>>>> On 28.10.2011, at 10:07, Ali Anil SINACI wrote:
>>>>
>>>>> Dear Rupert,
>>>>>
>>>>> On 10/28/2011 08:47 AM, Rupert Westenthaler wrote:
>>>>>> On 27.10.2011, at 16:59, Ali Anil SINACI wrote:
>>>>>>>>
>>>>>>>> * The LMF semantic search component overlaps greatly with the
>>>>>>>> recently by Anil contributed "contenthub/search/engines/solr"
>>>>>>>> component. Related to this it would be great if Anil could have a
>>>>>>>> look at [2] and check for similarities/differencies and possible
>>>>>>>> integration paths.
>>>>>>>>
>>>>>>> I had a look on the semantic search component of LMF. As you pointed
>>>>>>> it out, LMF semantic search provides a convenient way to index any
>>>>>>> part of documents with the help of RDFPath Language. I think that we
>>>>>>> can make use of this feature in contenthub. As I described in my
>>>>>>> previous e-mail, currently, contenthub indexes a number of semantic
>>>>>>> fields based on DBPedia relations. These are hardcoded relations.
>>>>>>> RDFPath language can be used to indicate specific semantic fields to
>>>>>>> be indexed along with the content itself. Let me describe the thing in
>>>>>>> our mind in a scenario:
>>>>>>>
>>>>>>> A user provides a domain ontology (e.g. music domain), submits to
>>>>>>> Entityhub to be used in the enhancement process. Suppose the domain
>>>>>>> ontology includes vast of information about artists, their albums
>>>>>>> etc... I assume that this ontology does not include conceptual
>>>>>>> definitions (it only includes Abox definitions). User writes an RDF
>>>>>>> Path Program (in LMF terminology) to indicate the fields to be indexed
>>>>>>> when a content item has an enhancement related with any path in that
>>>>>>> program. Suppose user submits a content item along with the RDF Path
>>>>>>> Program(s) to be used to determine the fields to be indexed.
>>>>>>> Enhancement engines find an entity (or lots of entities). Now, we
>>>>>>> execute the selected RDF Path Program(s) and embed the results into
>>>>>>> the Solr representation of the content item.
>>>>>>>
>>>>>>> If you have any other suggestions, please let me know so that we can
>>>>>>> discuss in detail (in SRDC) before the meeting.
>>>>>>>
>>>>>> This is exactly what I was thinking about. Let me only add that such
>>>>>> additional Knowledge to be included within the Semantic Index might not
>>>>>> only come from the Entityhub, but also from other sources (like the CMS
>>>>>> via the CMS adapter)
>>>>>>
>>>>>> I you would like to help me with an Implementation of the
>>>>>> RdfPathLanguage (e.g. the Clerezza based Implementation, or maybe a
>>>>>> Jena bases implementation) please let me know. Help would be greatly
>>>>>> welcome, because I have already a lot of things on my TODO list before
>>>>>> the Meeting in November (such as defining a Proposal for the Stanbol
>>>>>> Enhancement Structure).
>>>>>>
>>>>>
>>>>> We would like to get involved in the implementation of RDFPathLanguage
>>>>> for Stanbol. We plan to work on this starting from next week. I think
>>>>> you & LMF team already have a design in your mind. I will appreciate if
>>>>> you could share your thoughts with us.
>>>>>
>>>>>>>> * The Semantic Search Inteface: The Contenthub currently defines it's
>>>>>>>> own query API (supports keyword based search as well as "field ->
>>>>>>>> value" like constraints, supports facets). The LMF directly exposes
>>>>>>>> the RESTful API of the semantic Solr index. I strongly prefer the
>>>>>>>> approach of the LMF, because the two points already described above.
>>>>>>> We think that we do not have to make a selection here. We can keep a
>>>>>>> simple wrap-up on the Solr interface (contenthub's own query API)
>>>>>>> while providing the Solr RESTful API as is. IMO a wrap-up on Solr
>>>>>>> interface would be beneficial. On the other hand, in this interface we
>>>>>>> try to make use of an ontology to be used in
>>>>>>> OntologyResourceSearchEngine. This might help to figure out new
>>>>>>> keywords based on the subsumption hierarchy inside the ontology.
>>>>>>> However, I think this may lead to performance issues and may not be
>>>>>>> useful at all. We can decide on this later.
>>>>>> You forgot to mention one additional advantage for using the Solr
>>>>>> RESTful API: If we do that one could create the Semantic Index and than
>>>>>> copy it over to some other SolrServer without the need to run Stanbol
>>>>>> directly on the production infrastructure.
>>>>>>
>>>>>> In general I would suggest to first focus the discussion on the unique
>>>>>> features we would like to provide with the Semantic Search component. I
>>>>>> already included three features I would like to have in my first Mail
>>>>>> (Query preprocessing, Entity Facets, Semantic Facets). As you now
>>>>>> mention the OntologyResourceSearchEngine is very relevant in relation
>>>>>> to such features.
>>>>>> However adding such features must not necessarily mean to create an own
>>>>>> query language. One could also try to add such features directly to
>>>>>> Solr by implementing some Solr extensions.
>>>>>>
>>>>>
>>>>> Let me briefly comment in your suggestions about the semantic search.
>>>>>
>>>>>>>> But I am also the opinion that a semantic search interface should at
>>>>>>>> least provide the following three additional features:
>>>>>>>> 1. Query preprocessing: e.g. substitute "Paris" in the query
>>>>>>>> with "http://dbpedia.org/resource/Paris";
>>>>>>>> 2. Entity Facets: if a keyword matches a Entity (e.g. "Paris" ->
>>>>>>>> "dbpedia:Paris", "dbpedia:Paris_Texas", "dbpedia:Paris_Hilton")
>>>>>>>> than provide a Facet to the user over such possible
>>>>>>>> nnnnnnnnmatches;
>>>>>
>>>>> As far as we understand, first and second features will be handled by
>>>>> querying the Entityhub with the query keyword (Paris) i.e the first
>>>>> entity obtained from the Entityhub will help us to recognize its type
>>>>> and the other entities will be served as facet values of Paris facet.
>>>>>
>>>>>>>> 3. Semantic Facets: if a user uses an instance of an ontology
>>>>>>>> type (e.g. a Place, Person, Organization) in a query, that
>>>>>>>> provide facets over semantic relations for such types (e.g.
>>>>>>>> fiends for persons, products/services for Organizations, nearby
>>>>>>>> Points-Of-Interests for Places, Participants for Events, …). To
>>>>>>>> implement features like that we need components that provide
>>>>>>>> query preprocessing capabilities based on data available in the
>>>>>>>> Entityhub, Ontonet … . To me it seams that the
>>>>>>>> contenthub/search/engines/ontologyresource component provides
>>>>>>>> already some functionality related to this so this might be a
>>>>>>>> good starting point.
>>>>>
>>>>> Currently, we are trying to integrate an exploration mechanism like you
>>>>> said above. It is also based on DBPedia ontology.
>>>>> OntologyResourceEngine can be used for this purpose for the user
>>>>> registered ontologies. Current implementation of this engine only
>>>>> computes closures by exploiting the hierarchy in the ontology. RDFPath
>>>>> Programs can also be an option at this point. With an RDF Path Program
>>>>> user may specify the relations to be used in the exploration process.
>>>>> But I think this means the user decides beforehand which fields should
>>>>> be presented to him as exploration fields. I think this is open to
>>>>> discussion.
>>>>>
>>>>>> best
>>>>>> Rupert
>>>>>>
>>>>>
>>>>> Regards,
>>>>> Anil.
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>> ----------------------------------------------------------------
>> This message was sent using IMP, the Internet Messaging Program.
>>
>>
>
> Sebastian
> --
> | Dr. Sebastian Schaffert sebastian.schaffert@salzburgresearch.at
> | Salzburg Research Forschungsgesellschaft http://www.salzburgresearch.at
> | Head of Knowledge and Media Technologies Group +43 662 2288 423
> | Jakob-Haringer Strasse 5/II
> | A-5020 Salzburg
>
Re: RdfPathLanguage: Next steps (was Re: Updates to LMF/Stanbol integration)
Posted by Sebastian Schaffert <se...@salzburgresearch.at>.
Dear Valentina and all,
This is indeed very interesting. We developed "RDFPath" originally as a configuration language for the semantic search index, but later realised that it is a very natural way of querying over the Linked Data Cloud. In contrast to SPARQL, it is resource-centric like Linked Data, and it is restricted in a way that is more suitable for Linked Data servers. The path language is documented here:
http://code.google.com/p/kiwi/wiki/RdfPathLanguage
Combining this with pattern extraction would be very interesting, e.g. for automatically suggesting a configuration of our search index.
Greetings,
Sebastian
Am 28.10.2011 um 15:17 schrieb valentina.presutti@cnr.it:
> Actually, we have developed experience on this issue (SPARQL-based implementation for path extraction). I think that Andrea could lead the task and bring in the work already done as a starting point.
>
> We have also identified a number of indicators that is worth to compute and associate to paths. Everything is in the two papers (and the vocabulary) referred by Andrea. For example, we compute the occurrences of paths in a dataset, their popularity (we have defined a measure for this), and other ones.
> They can be optional features offered by the SPARQL-based services.
>
> Val
>
> Quoting nuzzoles@cs.unibo.it:
>
>> Hi Rupert, Stanbol team
>>
>> I am interested in the SPARQL based implementation.
>> We have recently worked on extracting paths from linked data.
>> The work has been just presented at ISWC [1, 2].
>> Hence I have already some experience with the issue and some preliminary
>> results to reuse:
>>
>> * a number of scripts for extracting paths
>> * a vocabulary for representing and storing them [3]
>>
>> I am not sure I will be in Salzburg, but we can discuss this possibility
>> for including these issues at the hackathon.
>>
>> --
>> Andrea Giovanni Nuzzolese
>> Semantic Technology Laboratory (STLab)
>> Institute for Cognitive Science and Technology (ISTC)
>> National Research Council (CNR)
>> Via Nomentana 56, Roma - Italy
>>
>>
>> [1] http://www.stlab.istc.cnr.it/documents/papers/cold2011.pdf
>> [2] http://www.stlab.istc.cnr.it/documents/papers/wikilinkpatterns.pdf
>> [3] http://www.ontologydesignpatterns.org/ont/lod-analysis-properties.owl
>>
>>
>>
>>> Hi Sebastian, Jakob, Stanbol team
>>>
>>> Based on the positive feedback of Anil to participate on this I decided to
>>> create an own thread to plan the next steps.
>>>
>>> Next steps:
>>>
>>> The first step will be to define Java API that allows to provide different
>>> implementations. I think the Idea was to create an own Project (should we
>>> use Github or GoogleCode? MIT/BSD/Apache licensed?) that only focusses on
>>> the Specification of the Language [1] and the Java API. Sebastian needs to
>>> take the lead of this. If I remember correctly his plan was to start this
>>> next week.
>>>
>>> As soon as a first version of this specification is available we can start
>>> to work on implementations.
>>>
>>> * Kiwi TripleStore: I assume Sebastian and Jakob will work on that
>>> * Clerezza: Anil could you take the lead for that?
>>> * Entityhub: This will be my responsibility
>>> * SPARQL based implementation: I think that would be interesting - someone
>>> interested to work on that?
>>> * CMS Adapter: Suat could you follow this effort and check for possible
>>> usage scenarios.
>>> * Fact Store: This could be also an interesting. But same as for the CMS
>>> Adapter we need first to check usage scenarios.
>>>
>>> best
>>> Rupert
>>>
>>>
>>>
>>> On 28.10.2011, at 10:07, Ali Anil SINACI wrote:
>>>
>>>> Dear Rupert,
>>>>
>>>> On 10/28/2011 08:47 AM, Rupert Westenthaler wrote:
>>>>> On 27.10.2011, at 16:59, Ali Anil SINACI wrote:
>>>>>>>
>>>>>>> * The LMF semantic search component overlaps greatly with the
>>>>>>> recently by Anil contributed "contenthub/search/engines/solr"
>>>>>>> component. Related to this it would be great if Anil could have a
>>>>>>> look at [2] and check for similarities/differencies and possible
>>>>>>> integration paths.
>>>>>>>
>>>>>> I had a look on the semantic search component of LMF. As you pointed
>>>>>> it out, LMF semantic search provides a convenient way to index any
>>>>>> part of documents with the help of RDFPath Language. I think that we
>>>>>> can make use of this feature in contenthub. As I described in my
>>>>>> previous e-mail, currently, contenthub indexes a number of semantic
>>>>>> fields based on DBPedia relations. These are hardcoded relations.
>>>>>> RDFPath language can be used to indicate specific semantic fields to
>>>>>> be indexed along with the content itself. Let me describe the thing in
>>>>>> our mind in a scenario:
>>>>>>
>>>>>> A user provides a domain ontology (e.g. music domain), submits to
>>>>>> Entityhub to be used in the enhancement process. Suppose the domain
>>>>>> ontology includes vast of information about artists, their albums
>>>>>> etc... I assume that this ontology does not include conceptual
>>>>>> definitions (it only includes Abox definitions). User writes an RDF
>>>>>> Path Program (in LMF terminology) to indicate the fields to be indexed
>>>>>> when a content item has an enhancement related with any path in that
>>>>>> program. Suppose user submits a content item along with the RDF Path
>>>>>> Program(s) to be used to determine the fields to be indexed.
>>>>>> Enhancement engines find an entity (or lots of entities). Now, we
>>>>>> execute the selected RDF Path Program(s) and embed the results into
>>>>>> the Solr representation of the content item.
>>>>>>
>>>>>> If you have any other suggestions, please let me know so that we can
>>>>>> discuss in detail (in SRDC) before the meeting.
>>>>>>
>>>>> This is exactly what I was thinking about. Let me only add that such
>>>>> additional Knowledge to be included within the Semantic Index might not
>>>>> only come from the Entityhub, but also from other sources (like the CMS
>>>>> via the CMS adapter)
>>>>>
>>>>> I you would like to help me with an Implementation of the
>>>>> RdfPathLanguage (e.g. the Clerezza based Implementation, or maybe a
>>>>> Jena bases implementation) please let me know. Help would be greatly
>>>>> welcome, because I have already a lot of things on my TODO list before
>>>>> the Meeting in November (such as defining a Proposal for the Stanbol
>>>>> Enhancement Structure).
>>>>>
>>>>
>>>> We would like to get involved in the implementation of RDFPathLanguage
>>>> for Stanbol. We plan to work on this starting from next week. I think
>>>> you & LMF team already have a design in your mind. I will appreciate if
>>>> you could share your thoughts with us.
>>>>
>>>>>>> * The Semantic Search Inteface: The Contenthub currently defines it's
>>>>>>> own query API (supports keyword based search as well as "field ->
>>>>>>> value" like constraints, supports facets). The LMF directly exposes
>>>>>>> the RESTful API of the semantic Solr index. I strongly prefer the
>>>>>>> approach of the LMF, because the two points already described above.
>>>>>> We think that we do not have to make a selection here. We can keep a
>>>>>> simple wrap-up on the Solr interface (contenthub's own query API)
>>>>>> while providing the Solr RESTful API as is. IMO a wrap-up on Solr
>>>>>> interface would be beneficial. On the other hand, in this interface we
>>>>>> try to make use of an ontology to be used in
>>>>>> OntologyResourceSearchEngine. This might help to figure out new
>>>>>> keywords based on the subsumption hierarchy inside the ontology.
>>>>>> However, I think this may lead to performance issues and may not be
>>>>>> useful at all. We can decide on this later.
>>>>> You forgot to mention one additional advantage for using the Solr
>>>>> RESTful API: If we do that one could create the Semantic Index and than
>>>>> copy it over to some other SolrServer without the need to run Stanbol
>>>>> directly on the production infrastructure.
>>>>>
>>>>> In general I would suggest to first focus the discussion on the unique
>>>>> features we would like to provide with the Semantic Search component. I
>>>>> already included three features I would like to have in my first Mail
>>>>> (Query preprocessing, Entity Facets, Semantic Facets). As you now
>>>>> mention the OntologyResourceSearchEngine is very relevant in relation
>>>>> to such features.
>>>>> However adding such features must not necessarily mean to create an own
>>>>> query language. One could also try to add such features directly to
>>>>> Solr by implementing some Solr extensions.
>>>>>
>>>>
>>>> Let me briefly comment in your suggestions about the semantic search.
>>>>
>>>>>>> But I am also the opinion that a semantic search interface should at
>>>>>>> least provide the following three additional features:
>>>>>>> 1. Query preprocessing: e.g. substitute "Paris" in the query
>>>>>>> with "http://dbpedia.org/resource/Paris";
>>>>>>> 2. Entity Facets: if a keyword matches a Entity (e.g. "Paris" ->
>>>>>>> "dbpedia:Paris", "dbpedia:Paris_Texas", "dbpedia:Paris_Hilton")
>>>>>>> than provide a Facet to the user over such possible
>>>>>>> nnnnnnnnmatches;
>>>>
>>>> As far as we understand, first and second features will be handled by
>>>> querying the Entityhub with the query keyword (Paris) i.e the first
>>>> entity obtained from the Entityhub will help us to recognize its type
>>>> and the other entities will be served as facet values of Paris facet.
>>>>
>>>>>>> 3. Semantic Facets: if a user uses an instance of an ontology
>>>>>>> type (e.g. a Place, Person, Organization) in a query, that
>>>>>>> provide facets over semantic relations for such types (e.g.
>>>>>>> fiends for persons, products/services for Organizations, nearby
>>>>>>> Points-Of-Interests for Places, Participants for Events, …). To
>>>>>>> implement features like that we need components that provide
>>>>>>> query preprocessing capabilities based on data available in the
>>>>>>> Entityhub, Ontonet … . To me it seams that the
>>>>>>> contenthub/search/engines/ontologyresource component provides
>>>>>>> already some functionality related to this so this might be a
>>>>>>> good starting point.
>>>>
>>>> Currently, we are trying to integrate an exploration mechanism like you
>>>> said above. It is also based on DBPedia ontology.
>>>> OntologyResourceEngine can be used for this purpose for the user
>>>> registered ontologies. Current implementation of this engine only
>>>> computes closures by exploiting the hierarchy in the ontology. RDFPath
>>>> Programs can also be an option at this point. With an RDF Path Program
>>>> user may specify the relations to be used in the exploration process.
>>>> But I think this means the user decides beforehand which fields should
>>>> be presented to him as exploration fields. I think this is open to
>>>> discussion.
>>>>
>>>>> best
>>>>> Rupert
>>>>>
>>>>
>>>> Regards,
>>>> Anil.
>>>
>>>
>>
>>
>>
>
>
>
> ----------------------------------------------------------------
> This message was sent using IMP, the Internet Messaging Program.
>
>
Sebastian
--
| Dr. Sebastian Schaffert sebastian.schaffert@salzburgresearch.at
| Salzburg Research Forschungsgesellschaft http://www.salzburgresearch.at
| Head of Knowledge and Media Technologies Group +43 662 2288 423
| Jakob-Haringer Strasse 5/II
| A-5020 Salzburg
Re: RdfPathLanguage: Next steps (was Re: Updates to LMF/Stanbol
integration)
Posted by va...@cnr.it.
Actually, we have developed experience on this issue (SPARQL-based
implementation for path extraction). I think that Andrea could lead
the task and bring in the work already done as a starting point.
We have also identified a number of indicators that is worth to
compute and associate to paths. Everything is in the two papers (and
the vocabulary) referred by Andrea. For example, we compute the
occurrences of paths in a dataset, their popularity (we have defined a
measure for this), and other ones.
They can be optional features offered by the SPARQL-based services.
Val
Quoting nuzzoles@cs.unibo.it:
> Hi Rupert, Stanbol team
>
> I am interested in the SPARQL based implementation.
> We have recently worked on extracting paths from linked data.
> The work has been just presented at ISWC [1, 2].
> Hence I have already some experience with the issue and some preliminary
> results to reuse:
>
> * a number of scripts for extracting paths
> * a vocabulary for representing and storing them [3]
>
> I am not sure I will be in Salzburg, but we can discuss this possibility
> for including these issues at the hackathon.
>
> --
> Andrea Giovanni Nuzzolese
> Semantic Technology Laboratory (STLab)
> Institute for Cognitive Science and Technology (ISTC)
> National Research Council (CNR)
> Via Nomentana 56, Roma - Italy
>
>
> [1] http://www.stlab.istc.cnr.it/documents/papers/cold2011.pdf
> [2] http://www.stlab.istc.cnr.it/documents/papers/wikilinkpatterns.pdf
> [3] http://www.ontologydesignpatterns.org/ont/lod-analysis-properties.owl
>
>
>
>> Hi Sebastian, Jakob, Stanbol team
>>
>> Based on the positive feedback of Anil to participate on this I decided to
>> create an own thread to plan the next steps.
>>
>> Next steps:
>>
>> The first step will be to define Java API that allows to provide different
>> implementations. I think the Idea was to create an own Project (should we
>> use Github or GoogleCode? MIT/BSD/Apache licensed?) that only focusses on
>> the Specification of the Language [1] and the Java API. Sebastian needs to
>> take the lead of this. If I remember correctly his plan was to start this
>> next week.
>>
>> As soon as a first version of this specification is available we can start
>> to work on implementations.
>>
>> * Kiwi TripleStore: I assume Sebastian and Jakob will work on that
>> * Clerezza: Anil could you take the lead for that?
>> * Entityhub: This will be my responsibility
>> * SPARQL based implementation: I think that would be interesting - someone
>> interested to work on that?
>> * CMS Adapter: Suat could you follow this effort and check for possible
>> usage scenarios.
>> * Fact Store: This could be also an interesting. But same as for the CMS
>> Adapter we need first to check usage scenarios.
>>
>> best
>> Rupert
>>
>>
>>
>> On 28.10.2011, at 10:07, Ali Anil SINACI wrote:
>>
>>> Dear Rupert,
>>>
>>> On 10/28/2011 08:47 AM, Rupert Westenthaler wrote:
>>>> On 27.10.2011, at 16:59, Ali Anil SINACI wrote:
>>>>>>
>>>>>> * The LMF semantic search component overlaps greatly with the
>>>>>> recently by Anil contributed "contenthub/search/engines/solr"
>>>>>> component. Related to this it would be great if Anil could have a
>>>>>> look at [2] and check for similarities/differencies and possible
>>>>>> integration paths.
>>>>>>
>>>>> I had a look on the semantic search component of LMF. As you pointed
>>>>> it out, LMF semantic search provides a convenient way to index any
>>>>> part of documents with the help of RDFPath Language. I think that we
>>>>> can make use of this feature in contenthub. As I described in my
>>>>> previous e-mail, currently, contenthub indexes a number of semantic
>>>>> fields based on DBPedia relations. These are hardcoded relations.
>>>>> RDFPath language can be used to indicate specific semantic fields to
>>>>> be indexed along with the content itself. Let me describe the thing in
>>>>> our mind in a scenario:
>>>>>
>>>>> A user provides a domain ontology (e.g. music domain), submits to
>>>>> Entityhub to be used in the enhancement process. Suppose the domain
>>>>> ontology includes vast of information about artists, their albums
>>>>> etc... I assume that this ontology does not include conceptual
>>>>> definitions (it only includes Abox definitions). User writes an RDF
>>>>> Path Program (in LMF terminology) to indicate the fields to be indexed
>>>>> when a content item has an enhancement related with any path in that
>>>>> program. Suppose user submits a content item along with the RDF Path
>>>>> Program(s) to be used to determine the fields to be indexed.
>>>>> Enhancement engines find an entity (or lots of entities). Now, we
>>>>> execute the selected RDF Path Program(s) and embed the results into
>>>>> the Solr representation of the content item.
>>>>>
>>>>> If you have any other suggestions, please let me know so that we can
>>>>> discuss in detail (in SRDC) before the meeting.
>>>>>
>>>> This is exactly what I was thinking about. Let me only add that such
>>>> additional Knowledge to be included within the Semantic Index might not
>>>> only come from the Entityhub, but also from other sources (like the CMS
>>>> via the CMS adapter)
>>>>
>>>> I you would like to help me with an Implementation of the
>>>> RdfPathLanguage (e.g. the Clerezza based Implementation, or maybe a
>>>> Jena bases implementation) please let me know. Help would be greatly
>>>> welcome, because I have already a lot of things on my TODO list before
>>>> the Meeting in November (such as defining a Proposal for the Stanbol
>>>> Enhancement Structure).
>>>>
>>>
>>> We would like to get involved in the implementation of RDFPathLanguage
>>> for Stanbol. We plan to work on this starting from next week. I think
>>> you & LMF team already have a design in your mind. I will appreciate if
>>> you could share your thoughts with us.
>>>
>>>>>> * The Semantic Search Inteface: The Contenthub currently defines it's
>>>>>> own query API (supports keyword based search as well as "field ->
>>>>>> value" like constraints, supports facets). The LMF directly exposes
>>>>>> the RESTful API of the semantic Solr index. I strongly prefer the
>>>>>> approach of the LMF, because the two points already described above.
>>>>> We think that we do not have to make a selection here. We can keep a
>>>>> simple wrap-up on the Solr interface (contenthub's own query API)
>>>>> while providing the Solr RESTful API as is. IMO a wrap-up on Solr
>>>>> interface would be beneficial. On the other hand, in this interface we
>>>>> try to make use of an ontology to be used in
>>>>> OntologyResourceSearchEngine. This might help to figure out new
>>>>> keywords based on the subsumption hierarchy inside the ontology.
>>>>> However, I think this may lead to performance issues and may not be
>>>>> useful at all. We can decide on this later.
>>>> You forgot to mention one additional advantage for using the Solr
>>>> RESTful API: If we do that one could create the Semantic Index and than
>>>> copy it over to some other SolrServer without the need to run Stanbol
>>>> directly on the production infrastructure.
>>>>
>>>> In general I would suggest to first focus the discussion on the unique
>>>> features we would like to provide with the Semantic Search component. I
>>>> already included three features I would like to have in my first Mail
>>>> (Query preprocessing, Entity Facets, Semantic Facets). As you now
>>>> mention the OntologyResourceSearchEngine is very relevant in relation
>>>> to such features.
>>>> However adding such features must not necessarily mean to create an own
>>>> query language. One could also try to add such features directly to
>>>> Solr by implementing some Solr extensions.
>>>>
>>>
>>> Let me briefly comment in your suggestions about the semantic search.
>>>
>>>>>> But I am also the opinion that a semantic search interface should at
>>>>>> least provide the following three additional features:
>>>>>> 1. Query preprocessing: e.g. substitute "Paris" in the query
>>>>>> with "http://dbpedia.org/resource/Paris";
>>>>>> 2. Entity Facets: if a keyword matches a Entity (e.g. "Paris" ->
>>>>>> "dbpedia:Paris", "dbpedia:Paris_Texas", "dbpedia:Paris_Hilton")
>>>>>> than provide a Facet to the user over such possible
>>>>>> nnnnnnnnmatches;
>>>
>>> As far as we understand, first and second features will be handled by
>>> querying the Entityhub with the query keyword (Paris) i.e the first
>>> entity obtained from the Entityhub will help us to recognize its type
>>> and the other entities will be served as facet values of Paris facet.
>>>
>>>>>> 3. Semantic Facets: if a user uses an instance of an ontology
>>>>>> type (e.g. a Place, Person, Organization) in a query, that
>>>>>> provide facets over semantic relations for such types (e.g.
>>>>>> fiends for persons, products/services for Organizations, nearby
>>>>>> Points-Of-Interests for Places, Participants for Events,
). To
>>>>>> implement features like that we need components that provide
>>>>>> query preprocessing capabilities based on data available in the
>>>>>> Entityhub, Ontonet
. To me it seams that the
>>>>>> contenthub/search/engines/ontologyresource component provides
>>>>>> already some functionality related to this so this might be a
>>>>>> good starting point.
>>>
>>> Currently, we are trying to integrate an exploration mechanism like you
>>> said above. It is also based on DBPedia ontology.
>>> OntologyResourceEngine can be used for this purpose for the user
>>> registered ontologies. Current implementation of this engine only
>>> computes closures by exploiting the hierarchy in the ontology. RDFPath
>>> Programs can also be an option at this point. With an RDF Path Program
>>> user may specify the relations to be used in the exploration process.
>>> But I think this means the user decides beforehand which fields should
>>> be presented to him as exploration fields. I think this is open to
>>> discussion.
>>>
>>>> best
>>>> Rupert
>>>>
>>>
>>> Regards,
>>> Anil.
>>
>>
>
>
>
----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.
Re: RdfPathLanguage: Next steps (was Re: Updates to LMF/Stanbol
integration)
Posted by nu...@cs.unibo.it.
Hi Rupert, Stanbol team
I am interested in the SPARQL based implementation.
We have recently worked on extracting paths from linked data.
The work has been just presented at ISWC [1, 2].
Hence I have already some experience with the issue and some preliminary
results to reuse:
* a number of scripts for extracting paths
* a vocabulary for representing and storing them [3]
I am not sure I will be in Salzburg, but we can discuss this possibility
for including these issues at the hackathon.
--
Andrea Giovanni Nuzzolese
Semantic Technology Laboratory (STLab)
Institute for Cognitive Science and Technology (ISTC)
National Research Council (CNR)
Via Nomentana 56, Roma - Italy
[1] http://www.stlab.istc.cnr.it/documents/papers/cold2011.pdf
[2] http://www.stlab.istc.cnr.it/documents/papers/wikilinkpatterns.pdf
[3] http://www.ontologydesignpatterns.org/ont/lod-analysis-properties.owl
> Hi Sebastian, Jakob, Stanbol team
>
> Based on the positive feedback of Anil to participate on this I decided to
> create an own thread to plan the next steps.
>
> Next steps:
>
> The first step will be to define Java API that allows to provide different
> implementations. I think the Idea was to create an own Project (should we
> use Github or GoogleCode? MIT/BSD/Apache licensed?) that only focusses on
> the Specification of the Language [1] and the Java API. Sebastian needs to
> take the lead of this. If I remember correctly his plan was to start this
> next week.
>
> As soon as a first version of this specification is available we can start
> to work on implementations.
>
> * Kiwi TripleStore: I assume Sebastian and Jakob will work on that
> * Clerezza: Anil could you take the lead for that?
> * Entityhub: This will be my responsibility
> * SPARQL based implementation: I think that would be interesting - someone
> interested to work on that?
> * CMS Adapter: Suat could you follow this effort and check for possible
> usage scenarios.
> * Fact Store: This could be also an interesting. But same as for the CMS
> Adapter we need first to check usage scenarios.
>
> best
> Rupert
>
>
>
> On 28.10.2011, at 10:07, Ali Anil SINACI wrote:
>
>> Dear Rupert,
>>
>> On 10/28/2011 08:47 AM, Rupert Westenthaler wrote:
>>> On 27.10.2011, at 16:59, Ali Anil SINACI wrote:
>>>>>
>>>>> * The LMF semantic search component overlaps greatly with the
>>>>> recently by Anil contributed "contenthub/search/engines/solr"
>>>>> component. Related to this it would be great if Anil could have a
>>>>> look at [2] and check for similarities/differencies and possible
>>>>> integration paths.
>>>>>
>>>> I had a look on the semantic search component of LMF. As you pointed
>>>> it out, LMF semantic search provides a convenient way to index any
>>>> part of documents with the help of RDFPath Language. I think that we
>>>> can make use of this feature in contenthub. As I described in my
>>>> previous e-mail, currently, contenthub indexes a number of semantic
>>>> fields based on DBPedia relations. These are hardcoded relations.
>>>> RDFPath language can be used to indicate specific semantic fields to
>>>> be indexed along with the content itself. Let me describe the thing in
>>>> our mind in a scenario:
>>>>
>>>> A user provides a domain ontology (e.g. music domain), submits to
>>>> Entityhub to be used in the enhancement process. Suppose the domain
>>>> ontology includes vast of information about artists, their albums
>>>> etc... I assume that this ontology does not include conceptual
>>>> definitions (it only includes Abox definitions). User writes an RDF
>>>> Path Program (in LMF terminology) to indicate the fields to be indexed
>>>> when a content item has an enhancement related with any path in that
>>>> program. Suppose user submits a content item along with the RDF Path
>>>> Program(s) to be used to determine the fields to be indexed.
>>>> Enhancement engines find an entity (or lots of entities). Now, we
>>>> execute the selected RDF Path Program(s) and embed the results into
>>>> the Solr representation of the content item.
>>>>
>>>> If you have any other suggestions, please let me know so that we can
>>>> discuss in detail (in SRDC) before the meeting.
>>>>
>>> This is exactly what I was thinking about. Let me only add that such
>>> additional Knowledge to be included within the Semantic Index might not
>>> only come from the Entityhub, but also from other sources (like the CMS
>>> via the CMS adapter)
>>>
>>> I you would like to help me with an Implementation of the
>>> RdfPathLanguage (e.g. the Clerezza based Implementation, or maybe a
>>> Jena bases implementation) please let me know. Help would be greatly
>>> welcome, because I have already a lot of things on my TODO list before
>>> the Meeting in November (such as defining a Proposal for the Stanbol
>>> Enhancement Structure).
>>>
>>
>> We would like to get involved in the implementation of RDFPathLanguage
>> for Stanbol. We plan to work on this starting from next week. I think
>> you & LMF team already have a design in your mind. I will appreciate if
>> you could share your thoughts with us.
>>
>>>>> * The Semantic Search Inteface: The Contenthub currently defines it's
>>>>> own query API (supports keyword based search as well as "field ->
>>>>> value" like constraints, supports facets). The LMF directly exposes
>>>>> the RESTful API of the semantic Solr index. I strongly prefer the
>>>>> approach of the LMF, because the two points already described above.
>>>> We think that we do not have to make a selection here. We can keep a
>>>> simple wrap-up on the Solr interface (contenthub's own query API)
>>>> while providing the Solr RESTful API as is. IMO a wrap-up on Solr
>>>> interface would be beneficial. On the other hand, in this interface we
>>>> try to make use of an ontology to be used in
>>>> OntologyResourceSearchEngine. This might help to figure out new
>>>> keywords based on the subsumption hierarchy inside the ontology.
>>>> However, I think this may lead to performance issues and may not be
>>>> useful at all. We can decide on this later.
>>> You forgot to mention one additional advantage for using the Solr
>>> RESTful API: If we do that one could create the Semantic Index and than
>>> copy it over to some other SolrServer without the need to run Stanbol
>>> directly on the production infrastructure.
>>>
>>> In general I would suggest to first focus the discussion on the unique
>>> features we would like to provide with the Semantic Search component. I
>>> already included three features I would like to have in my first Mail
>>> (Query preprocessing, Entity Facets, Semantic Facets). As you now
>>> mention the OntologyResourceSearchEngine is very relevant in relation
>>> to such features.
>>> However adding such features must not necessarily mean to create an own
>>> query language. One could also try to add such features directly to
>>> Solr by implementing some Solr extensions.
>>>
>>
>> Let me briefly comment in your suggestions about the semantic search.
>>
>>>>> But I am also the opinion that a semantic search interface should at
>>>>> least provide the following three additional features:
>>>>> 1. Query preprocessing: e.g. substitute "Paris" in the query
>>>>> with "http://dbpedia.org/resource/Paris";
>>>>> 2. Entity Facets: if a keyword matches a Entity (e.g. "Paris" ->
>>>>> "dbpedia:Paris", "dbpedia:Paris_Texas", "dbpedia:Paris_Hilton")
>>>>> than provide a Facet to the user over such possible
>>>>> nnnnnnnnmatches;
>>
>> As far as we understand, first and second features will be handled by
>> querying the Entityhub with the query keyword (Paris) i.e the first
>> entity obtained from the Entityhub will help us to recognize its type
>> and the other entities will be served as facet values of Paris facet.
>>
>>>>> 3. Semantic Facets: if a user uses an instance of an ontology
>>>>> type (e.g. a Place, Person, Organization) in a query, that
>>>>> provide facets over semantic relations for such types (e.g.
>>>>> fiends for persons, products/services for Organizations, nearby
>>>>> Points-Of-Interests for Places, Participants for Events,
). To
>>>>> implement features like that we need components that provide
>>>>> query preprocessing capabilities based on data available in the
>>>>> Entityhub, Ontonet
. To me it seams that the
>>>>> contenthub/search/engines/ontologyresource component provides
>>>>> already some functionality related to this so this might be a
>>>>> good starting point.
>>
>> Currently, we are trying to integrate an exploration mechanism like you
>> said above. It is also based on DBPedia ontology.
>> OntologyResourceEngine can be used for this purpose for the user
>> registered ontologies. Current implementation of this engine only
>> computes closures by exploiting the hierarchy in the ontology. RDFPath
>> Programs can also be an option at this point. With an RDF Path Program
>> user may specify the relations to be used in the exploration process.
>> But I think this means the user decides beforehand which fields should
>> be presented to him as exploration fields. I think this is open to
>> discussion.
>>
>>> best
>>> Rupert
>>>
>>
>> Regards,
>> Anil.
>
>
RdfPathLanguage: Next steps (was Re: Updates to LMF/Stanbol integration)
Posted by Rupert Westenthaler <ru...@gmail.com>.
Hi Sebastian, Jakob, Stanbol team
Based on the positive feedback of Anil to participate on this I decided to create an own thread to plan the next steps.
Next steps:
The first step will be to define Java API that allows to provide different implementations. I think the Idea was to create an own Project (should we use Github or GoogleCode? MIT/BSD/Apache licensed?) that only focusses on the Specification of the Language [1] and the Java API. Sebastian needs to take the lead of this. If I remember correctly his plan was to start this next week.
As soon as a first version of this specification is available we can start to work on implementations.
* Kiwi TripleStore: I assume Sebastian and Jakob will work on that
* Clerezza: Anil could you take the lead for that?
* Entityhub: This will be my responsibility
* SPARQL based implementation: I think that would be interesting - someone interested to work on that?
* CMS Adapter: Suat could you follow this effort and check for possible usage scenarios.
* Fact Store: This could be also an interesting. But same as for the CMS Adapter we need first to check usage scenarios.
best
Rupert
On 28.10.2011, at 10:07, Ali Anil SINACI wrote:
> Dear Rupert,
>
> On 10/28/2011 08:47 AM, Rupert Westenthaler wrote:
>> On 27.10.2011, at 16:59, Ali Anil SINACI wrote:
>>>>
>>>> * The LMF semantic search component overlaps greatly with the recently by Anil contributed "contenthub/search/engines/solr" component. Related to this it would be great if Anil could have a look at [2] and check for similarities/differencies and possible integration paths.
>>>>
>>> I had a look on the semantic search component of LMF. As you pointed it out, LMF semantic search provides a convenient way to index any part of documents with the help of RDFPath Language. I think that we can make use of this feature in contenthub. As I described in my previous e-mail, currently, contenthub indexes a number of semantic fields based on DBPedia relations. These are hardcoded relations. RDFPath language can be used to indicate specific semantic fields to be indexed along with the content itself. Let me describe the thing in our mind in a scenario:
>>>
>>> A user provides a domain ontology (e.g. music domain), submits to Entityhub to be used in the enhancement process. Suppose the domain ontology includes vast of information about artists, their albums etc... I assume that this ontology does not include conceptual definitions (it only includes Abox definitions). User writes an RDF Path Program (in LMF terminology) to indicate the fields to be indexed when a content item has an enhancement related with any path in that program. Suppose user submits a content item along with the RDF Path Program(s) to be used to determine the fields to be indexed. Enhancement engines find an entity (or lots of entities). Now, we execute the selected RDF Path Program(s) and embed the results into the Solr representation of the content item.
>>>
>>> If you have any other suggestions, please let me know so that we can discuss in detail (in SRDC) before the meeting.
>>>
>> This is exactly what I was thinking about. Let me only add that such additional Knowledge to be included within the Semantic Index might not only come from the Entityhub, but also from other sources (like the CMS via the CMS adapter)
>>
>> I you would like to help me with an Implementation of the RdfPathLanguage (e.g. the Clerezza based Implementation, or maybe a Jena bases implementation) please let me know. Help would be greatly welcome, because I have already a lot of things on my TODO list before the Meeting in November (such as defining a Proposal for the Stanbol Enhancement Structure).
>>
>
> We would like to get involved in the implementation of RDFPathLanguage for Stanbol. We plan to work on this starting from next week. I think you & LMF team already have a design in your mind. I will appreciate if you could share your thoughts with us.
>
>>>> * The Semantic Search Inteface: The Contenthub currently defines it's own query API (supports keyword based search as well as "field -> value" like constraints, supports facets). The LMF directly exposes the RESTful API of the semantic Solr index. I strongly prefer the approach of the LMF, because the two points already described above.
>>> We think that we do not have to make a selection here. We can keep a simple wrap-up on the Solr interface (contenthub's own query API) while providing the Solr RESTful API as is. IMO a wrap-up on Solr interface would be beneficial. On the other hand, in this interface we try to make use of an ontology to be used in OntologyResourceSearchEngine. This might help to figure out new keywords based on the subsumption hierarchy inside the ontology. However, I think this may lead to performance issues and may not be useful at all. We can decide on this later.
>> You forgot to mention one additional advantage for using the Solr RESTful API: If we do that one could create the Semantic Index and than copy it over to some other SolrServer without the need to run Stanbol directly on the production infrastructure.
>>
>> In general I would suggest to first focus the discussion on the unique features we would like to provide with the Semantic Search component. I already included three features I would like to have in my first Mail (Query preprocessing, Entity Facets, Semantic Facets). As you now mention the OntologyResourceSearchEngine is very relevant in relation to such features.
>> However adding such features must not necessarily mean to create an own query language. One could also try to add such features directly to Solr by implementing some Solr extensions.
>>
>
> Let me briefly comment in your suggestions about the semantic search.
>
>>>> But I am also the opinion that a semantic search interface should at least provide the following three additional features:
>>>> 1. Query preprocessing: e.g. substitute "Paris" in the query with "http://dbpedia.org/resource/Paris";
>>>> 2. Entity Facets: if a keyword matches a Entity (e.g. "Paris" -> "dbpedia:Paris", "dbpedia:Paris_Texas", "dbpedia:Paris_Hilton") than provide a Facet to the user over such possible nnnnnnnnmatches;
>
> As far as we understand, first and second features will be handled by querying the Entityhub with the query keyword (Paris) i.e the first entity obtained from the Entityhub will help us to recognize its type and the other entities will be served as facet values of Paris facet.
>
>>>> 3. Semantic Facets: if a user uses an instance of an ontology type (e.g. a Place, Person, Organization) in a query, that provide facets over semantic relations for such types (e.g. fiends for persons, products/services for Organizations, nearby Points-Of-Interests for Places, Participants for Events, …). To implement features like that we need components that provide query preprocessing capabilities based on data available in the Entityhub, Ontonet … . To me it seams that the contenthub/search/engines/ontologyresource component provides already some functionality related to this so this might be a good starting point.
>
> Currently, we are trying to integrate an exploration mechanism like you said above. It is also based on DBPedia ontology. OntologyResourceEngine can be used for this purpose for the user registered ontologies. Current implementation of this engine only computes closures by exploiting the hierarchy in the ontology. RDFPath Programs can also be an option at this point. With an RDF Path Program user may specify the relations to be used in the exploration process. But I think this means the user decides beforehand which fields should be presented to him as exploration fields. I think this is open to discussion.
>
>> best
>> Rupert
>>
>
> Regards,
> Anil.
Re: Updates to LMF/Stanbol integration
Posted by Ali Anil SINACI <an...@srdc.com.tr>.
Dear Rupert,
On 10/28/2011 08:47 AM, Rupert Westenthaler wrote:
> On 27.10.2011, at 16:59, Ali Anil SINACI wrote:
>>>
>>> * The LMF semantic search component overlaps greatly with the recently by Anil contributed "contenthub/search/engines/solr" component. Related to this it would be great if Anil could have a look at [2] and check for similarities/differencies and possible integration paths.
>>>
>> I had a look on the semantic search component of LMF. As you pointed it out, LMF semantic search provides a convenient way to index any part of documents with the help of RDFPath Language. I think that we can make use of this feature in contenthub. As I described in my previous e-mail, currently, contenthub indexes a number of semantic fields based on DBPedia relations. These are hardcoded relations. RDFPath language can be used to indicate specific semantic fields to be indexed along with the content itself. Let me describe the thing in our mind in a scenario:
>>
>> A user provides a domain ontology (e.g. music domain), submits to Entityhub to be used in the enhancement process. Suppose the domain ontology includes vast of information about artists, their albums etc... I assume that this ontology does not include conceptual definitions (it only includes Abox definitions). User writes an RDF Path Program (in LMF terminology) to indicate the fields to be indexed when a content item has an enhancement related with any path in that program. Suppose user submits a content item along with the RDF Path Program(s) to be used to determine the fields to be indexed. Enhancement engines find an entity (or lots of entities). Now, we execute the selected RDF Path Program(s) and embed the results into the Solr representation of the content item.
>>
>> If you have any other suggestions, please let me know so that we can discuss in detail (in SRDC) before the meeting.
>>
> This is exactly what I was thinking about. Let me only add that such additional Knowledge to be included within the Semantic Index might not only come from the Entityhub, but also from other sources (like the CMS via the CMS adapter)
>
> I you would like to help me with an Implementation of the RdfPathLanguage (e.g. the Clerezza based Implementation, or maybe a Jena bases implementation) please let me know. Help would be greatly welcome, because I have already a lot of things on my TODO list before the Meeting in November (such as defining a Proposal for the Stanbol Enhancement Structure).
>
We would like to get involved in the implementation of RDFPathLanguage
for Stanbol. We plan to work on this starting from next week. I think
you & LMF team already have a design in your mind. I will appreciate if
you could share your thoughts with us.
>>> * The Semantic Search Inteface: The Contenthub currently defines it's own query API (supports keyword based search as well as "field -> value" like constraints, supports facets). The LMF directly exposes the RESTful API of the semantic Solr index. I strongly prefer the approach of the LMF, because the two points already described above.
>> We think that we do not have to make a selection here. We can keep a simple wrap-up on the Solr interface (contenthub's own query API) while providing the Solr RESTful API as is. IMO a wrap-up on Solr interface would be beneficial. On the other hand, in this interface we try to make use of an ontology to be used in OntologyResourceSearchEngine. This might help to figure out new keywords based on the subsumption hierarchy inside the ontology. However, I think this may lead to performance issues and may not be useful at all. We can decide on this later.
> You forgot to mention one additional advantage for using the Solr RESTful API: If we do that one could create the Semantic Index and than copy it over to some other SolrServer without the need to run Stanbol directly on the production infrastructure.
>
> In general I would suggest to first focus the discussion on the unique features we would like to provide with the Semantic Search component. I already included three features I would like to have in my first Mail (Query preprocessing, Entity Facets, Semantic Facets). As you now mention the OntologyResourceSearchEngine is very relevant in relation to such features.
> However adding such features must not necessarily mean to create an own query language. One could also try to add such features directly to Solr by implementing some Solr extensions.
>
Let me briefly comment in your suggestions about the semantic search.
>>> But I am also the opinion that a semantic search interface should at least provide the following three additional features:
>>> 1. Query preprocessing: e.g. substitute "Paris" in the query with "http://dbpedia.org/resource/Paris";
>>> 2. Entity Facets: if a keyword matches a Entity (e.g. "Paris" -> "dbpedia:Paris", "dbpedia:Paris_Texas", "dbpedia:Paris_Hilton") than provide a Facet to the user over such possible nnnnnnnnmatches;
As far as we understand, first and second features will be handled by
querying the Entityhub with the query keyword (Paris) i.e the first
entity obtained from the Entityhub will help us to recognize its type
and the other entities will be served as facet values of Paris facet.
>>> 3. Semantic Facets: if a user uses an instance of an ontology type (e.g. a Place, Person, Organization) in a query, that provide facets over semantic relations for such types (e.g. fiends for persons, products/services for Organizations, nearby Points-Of-Interests for Places, Participants for Events, …). To implement features like that we need components that provide query preprocessing capabilities based on data available in the Entityhub, Ontonet … . To me it seams that the contenthub/search/engines/ontologyresource component provides already some functionality related to this so this might be a good starting point.
Currently, we are trying to integrate an exploration mechanism like you
said above. It is also based on DBPedia ontology.
OntologyResourceEngine can be used for this purpose for the user
registered ontologies. Current implementation of this engine only
computes closures by exploiting the hierarchy in the ontology. RDFPath
Programs can also be an option at this point. With an RDF Path Program
user may specify the relations to be used in the exploration process.
But I think this means the user decides beforehand which fields should
be presented to him as exploration fields. I think this is open to
discussion.
> best
> Rupert
>
Regards,
Anil.
Re: Updates to LMF/Stanbol integration
Posted by Rupert Westenthaler <ru...@salzburgresearch.at>.
On 27.10.2011, at 16:59, Ali Anil SINACI wrote:
>>
>>
>> * The LMF semantic search component overlaps greatly with the recently by Anil contributed "contenthub/search/engines/solr" component. Related to this it would be great if Anil could have a look at [2] and check for similarities/differencies and possible integration paths.
>>
>
> I had a look on the semantic search component of LMF. As you pointed it out, LMF semantic search provides a convenient way to index any part of documents with the help of RDFPath Language. I think that we can make use of this feature in contenthub. As I described in my previous e-mail, currently, contenthub indexes a number of semantic fields based on DBPedia relations. These are hardcoded relations. RDFPath language can be used to indicate specific semantic fields to be indexed along with the content itself. Let me describe the thing in our mind in a scenario:
>
> A user provides a domain ontology (e.g. music domain), submits to Entityhub to be used in the enhancement process. Suppose the domain ontology includes vast of information about artists, their albums etc... I assume that this ontology does not include conceptual definitions (it only includes Abox definitions). User writes an RDF Path Program (in LMF terminology) to indicate the fields to be indexed when a content item has an enhancement related with any path in that program. Suppose user submits a content item along with the RDF Path Program(s) to be used to determine the fields to be indexed. Enhancement engines find an entity (or lots of entities). Now, we execute the selected RDF Path Program(s) and embed the results into the Solr representation of the content item.
>
> If you have any other suggestions, please let me know so that we can discuss in detail (in SRDC) before the meeting.
>
This is exactly what I was thinking about. Let me only add that such additional Knowledge to be included within the Semantic Index might not only come from the Entityhub, but also from other sources (like the CMS via the CMS adapter)
I you would like to help me with an Implementation of the RdfPathLanguage (e.g. the Clerezza based Implementation, or maybe a Jena bases implementation) please let me know. Help would be greatly welcome, because I have already a lot of things on my TODO list before the Meeting in November (such as defining a Proposal for the Stanbol Enhancement Structure).
>> * The Semantic Search Inteface: The Contenthub currently defines it's own query API (supports keyword based search as well as "field -> value" like constraints, supports facets). The LMF directly exposes the RESTful API of the semantic Solr index. I strongly prefer the approach of the LMF, because the two points already described above.
>
> We think that we do not have to make a selection here. We can keep a simple wrap-up on the Solr interface (contenthub's own query API) while providing the Solr RESTful API as is. IMO a wrap-up on Solr interface would be beneficial. On the other hand, in this interface we try to make use of an ontology to be used in OntologyResourceSearchEngine. This might help to figure out new keywords based on the subsumption hierarchy inside the ontology. However, I think this may lead to performance issues and may not be useful at all. We can decide on this later.
You forgot to mention one additional advantage for using the Solr RESTful API: If we do that one could create the Semantic Index and than copy it over to some other SolrServer without the need to run Stanbol directly on the production infrastructure.
In general I would suggest to first focus the discussion on the unique features we would like to provide with the Semantic Search component. I already included three features I would like to have in my first Mail (Query preprocessing, Entity Facets, Semantic Facets). As you now mention the OntologyResourceSearchEngine is very relevant in relation to such features.
However adding such features must not necessarily mean to create an own query language. One could also try to add such features directly to Solr by implementing some Solr extensions.
>
>> But I am also the opinion that a semantic search interface should at least provide the following three additional features:
>> 1. Query preprocessing: e.g. substitute "Paris" in the query with "http://dbpedia.org/resource/Paris";
>> 2. Entity Facets: if a keyword matches a Entity (e.g. "Paris" -> "dbpedia:Paris", "dbpedia:Paris_Texas", "dbpedia:Paris_Hilton") than provide a Facet to the user over such possible matches;
>> 3. Semantic Facets: if a user uses an instance of an ontology type (e.g. a Place, Person, Organization) in a query, that provide facets over semantic relations for such types (e.g. fiends for persons, products/services for Organizations, nearby Points-Of-Interests for Places, Participants for Events, …). To implement features like that we need components that provide query preprocessing capabilities based on data available in the Entityhub, Ontonet … . To me it seams that the contenthub/search/engines/ontologyresource component provides already some functionality related to this so this might be a good starting point.
>
best
Rupert
> --
> Ali Anil SINACI
>
> Software Research, Development and Consultancy Ltd.
> Phone: +90 (312) 2101763
> Fax: +90 (312) 2101837
> E-mail: anil@srdc.com.tr
>
Re: Updates to LMF/Stanbol integration
Posted by Ali Anil SINACI <an...@srdc.com.tr>.
Dear Rupert,
You can find my comments inline.
Regards,
Anil.
On 10/26/2011 02:10 PM, Rupert Westenthaler wrote:
> Hi Stanbol Community
>
> Sebastian, Jakob Frank and my self had a meeting about the next steps in the integration of the LMF and Apache Stanbol. This mail provides an overview about the discussed points and my opinion how the integration with Stanbol could look like.
>
> We decided to start of the integration with the following two assets: (1) RdfPathLanguage [1] and (2) LMF semantic search component [2]. (*)
>
> In the following I will give an short overview about the necessary steps for integrating this two assets with Stanbol:
>
> (1) make the RdfPathLanguage [1] generic so we can use it within Stanbol. The decision was to define an own Java API for this specification and manage it as an own library (outside of Stanbol). This is mainly because we think this specification is of general interest as as query language/indexing language for the Linked Data and therefore it makes sense to keep the Java API definition in depended of any specific project.
>
> * Sebastian will take the lead in defining the API. Jakob Frank has also done a lot of work on this. So he will also actively contribute to that.
> * The current Implementation - based on the Kiwi Triple store will be adapted to the updated API.
> * I will work on implementations for Clerezza and the Entityhub.
> * It was also discussed to provide an implementation based on SPRAQL. This could be used to retrieve data referenced in paths from Sparql Endpoints.
> * In addition I think it would make also sense to provide an implementation based on the CMA adapter to allowing retrieving data from CMS. Suat it would be great if you could check if this would make sense.
> * As a first usage of the RdfPathLanguage within Stanbol I will replace the current FieldMapping infrastructure used by the Entityhub with the RdfPathLanguage. This will also the test case for the Clerezza and Entityhub based implementations.
>
>
> (2) Integrate the LMF semantic search component [2] with Stanbol: The LMF semantic search is based on semantic index configurations that are based on the RdfPathLanguage [1]. Based on this configuration a Solr schema (schema.xml) is generated that is than used to store the data of the semantic index. The actual search interface is the normal Solr RESTful API - just the data stored within the index are smarter. This has two very big advantage: For users - that a lot of programmers do already know how to use Solr and for us - that we do not need to reinvent to rich query interface of Solr (think about features such as facets, /mlt, ranking functions …). (**)
>
> The usage of the LMF semantic search component within Stanbol required the following three features (I already started to work on):
>
> * Enable the RESTful API for SolrCores managed by Stanbol (STANBOL-353): This is needed to allow the usage of the Solr RESTful API to query the semantic index. (nearly finished)
> * Improve the Component that manages internal SolrIncexes to better support update and removal of SolrIndexes. This is needed to allow the LMF to update schema.xml files, deleting and reindexing indexes after incompatible changes. However this will than also allow to "replace" a small default index for DBpedia with a bigger version or a version with additional fields, languages … (plan to start work on that next week)
> * Implementation of the RdfPathLanguage based on Clerezza (to index EnhancementResults) and the Entityhub (to include additional Information for suggested Entities). Optional: Implementation based on the CMS Adapter to also include additional data from a connected CMS. This is required to allow updates to the Semantic Index when running within Stanbol.
>
> The usage of the LMF semantic search component within Stanbol will also require to integrate is within the "/contentub". With the following points I try to highlight the main topics that need further investigations/discussions:
>
> * The LMF semantic search component overlaps greatly with the recently by Anil contributed "contenthub/search/engines/solr" component. Related to this it would be great if Anil could have a look at [2] and check for similarities/differencies and possible integration paths.
>
I had a look on the semantic search component of LMF. As you pointed it
out, LMF semantic search provides a convenient way to index any part of
documents with the help of RDFPath Language. I think that we can make
use of this feature in contenthub. As I described in my previous e-mail,
currently, contenthub indexes a number of semantic fields based on
DBPedia relations. These are hardcoded relations. RDFPath language can
be used to indicate specific semantic fields to be indexed along with
the content itself. Let me describe the thing in our mind in a scenario:
A user provides a domain ontology (e.g. music domain), submits to
Entityhub to be used in the enhancement process. Suppose the domain
ontology includes vast of information about artists, their albums etc...
I assume that this ontology does not include conceptual definitions (it
only includes Abox definitions). User writes an RDF Path Program (in LMF
terminology) to indicate the fields to be indexed when a content item
has an enhancement related with any path in that program. Suppose user
submits a content item along with the RDF Path Program(s) to be used to
determine the fields to be indexed. Enhancement engines find an entity
(or lots of entities). Now, we execute the selected RDF Path Program(s)
and embed the results into the Solr representation of the content item.
If you have any other suggestions, please let me know so that we can
discuss in detail (in SRDC) before the meeting.
> * The Semantic Search Inteface: The Contenthub currently defines it's own query API (supports keyword based search as well as "field -> value" like constraints, supports facets). The LMF directly exposes the RESTful API of the semantic Solr index. I strongly prefer the approach of the LMF, because the two points already described above.
We think that we do not have to make a selection here. We can keep a
simple wrap-up on the Solr interface (contenthub's own query API) while
providing the Solr RESTful API as is. IMO a wrap-up on Solr interface
would be beneficial. On the other hand, in this interface we try to make
use of an ontology to be used in OntologyResourceSearchEngine. This
might help to figure out new keywords based on the subsumption hierarchy
inside the ontology. However, I think this may lead to performance
issues and may not be useful at all. We can decide on this later.
> But I am also the opinion that a semantic search interface should at least provide the following three additional features:
> 1. Query preprocessing: e.g. substitute "Paris" in the query with "http://dbpedia.org/resource/Paris";
> 2. Entity Facets: if a keyword matches a Entity (e.g. "Paris" -> "dbpedia:Paris", "dbpedia:Paris_Texas", "dbpedia:Paris_Hilton") than provide a Facet to the user over such possible matches;
> 3. Semantic Facets: if a user uses an instance of an ontology type (e.g. a Place, Person, Organization) in a query, that provide facets over semantic relations for such types (e.g. fiends for persons, products/services for Organizations, nearby Points-Of-Interests for Places, Participants for Events, …). To implement features like that we need components that provide query preprocessing capabilities based on data available in the Entityhub, Ontonet … . To me it seams that the contenthub/search/engines/ontologyresource component provides already some functionality related to this so this might be a good starting point.
> * How is the relation to the Factstore? As far as I understand the Factstore provides the possibility to first define how facts do look like and than allows to efficiently CRUD over such facts. Related to this one can ask two questions:
> 1. Could one use the RdfPathLanguage to define Fact schemas and/or to create Fact instances?
> 2. could it be possible to integrate the LMF semantic search component as a FactStore implementation into Stanbol?
>
> Next Steps:
>
> Work in the coming weeks need to focus on making the IKS developer meeting end of November in Salzburg [3] as productive as possible. Let me also note that all interested people of the Stanbol Community are also very welcome to join.
>
> To prepare this meeting it would be important if we could already start discussion about the main topics - especially the /contenthub before this meeting on this mailing list. Implementation wise we (the LMF developer and me) will try realize a first integration until the start of the meeting. This will not try to integrate the LMF semantic search component with the content hub but rather concentrate on the RdfPathLanguage and the necessary improvements to the management of Solr indexes within stanbol. The should allow us to run LMF demos like [4] within the Stanbol environment. But without the possibility to update the semantic index or to index additional documents - because this would need than a real integration with the Contenthub.
>
> This two things should than allow us to use the face to face meeting in Salzburg to really work on the integration and hopefully to show a demo that allows to
>
> 1. send a Document to the Contenthub (optionally by using the CMS adapter)
> 2. enhancing it with the enhancer
> 3. storing the enhanced document within the semantic index
> 4. use the faced search interface to navigate through documents
>
> at the end of the meeting.
>
> best
> Rupert Westenthaler
>
> [1] http://code.google.com/p/kiwi/wiki/RdfPathLanguage
> [2] http://code.google.com/p/kiwi/source/browse/#hg%2Flmf-search%2Fsrc%2Fmain%2Fjava%2Fat%2Fnewmedialab%2Flmf%2Fsearch
> [3] http://wiki.iks-project.eu/index.php/IntegrationHackathonSalzburg
> [4] http://labs.newmedialab.at/LMF/sn/search/suche.html
>
> (*) Note that within this we excluded the topic of how to integrate the LMF rule engine. Not because this is not important, because I am do not have the according knowledge about the rule implementation of Stanbol to discuss this.
> (**) As a side note: The use of Solr by the Semantic Search component is very different of the one of the Entityhub SolrYard. First creates a dedicated schema based on a configuration - it can only index specific data, but produces a nice schema that is easy to understand and to use by users - very important for the semantic search use case. The later can store any kind of RDF data with a singe schema - very important when managing very diverse entities but the generic schema needs to use complex prefixes, and suffixes in combination with dynamic fields that would it make very hard for users to write Solr queries.
--
Ali Anil SINACI
Software Research, Development and Consultancy Ltd.
Phone: +90 (312) 2101763
Fax: +90 (312) 2101837
E-mail: anil@srdc.com.tr