You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@stanbol.apache.org by bu...@apache.org on 2012/02/07 10:47:29 UTC
svn commit: r804087 -
/websites/staging/stanbol/trunk/content/stanbol/docs/trunk/contenthub/contenthub5min.html
Author: buildbot
Date: Tue Feb 7 09:47:29 2012
New Revision: 804087
Log:
Staging update by buildbot for stanbol
Modified:
websites/staging/stanbol/trunk/content/stanbol/docs/trunk/contenthub/contenthub5min.html
Modified: websites/staging/stanbol/trunk/content/stanbol/docs/trunk/contenthub/contenthub5min.html
==============================================================================
--- websites/staging/stanbol/trunk/content/stanbol/docs/trunk/contenthub/contenthub5min.html (original)
+++ websites/staging/stanbol/trunk/content/stanbol/docs/trunk/contenthub/contenthub5min.html Tue Feb 7 09:47:29 2012
@@ -57,7 +57,124 @@
<div id="content">
<h1 class="title"></h1>
-
+ <p>Contenthub (5 minutes tutorial)
+The Apache Stanbol Contenthub is an Apache Solr based document repository which enables storage of text-based documents and customizable semantic search facilities. Contenthub exposes an efficient Java API together with the corresponding RESTful services. </p>
+<p>Contenthub is basically a document repository. A document within Contenthub is referred as a "Content Item". A content item consists of metadata of the document in addition to the text-based content of the document. Contenthub has two main subcomponents, namely Store and Search. As their names indicate, Store is specifically responsible for persistent storage of content items. And Search provides strong semantic search facilities on top of the content items.</p>
+<h2 id="contenthub-store">Contenthub Store</h2>
+<p>It is the part of Contenthub which actually stores the documents and their metadata persistently. In current implementation only text/plain documents are supported.</p>
+<p>The storage part of the Contenthub provide basic methods such as create, put, get and delete. When a document is submitted, it delegates the textual content to Stanbol Enhancer to retrieve its enhancements. (Enhancements of a content item are called its metadata within the terminology) While submitting the document, it is also possible to specify external metadata (in addition to the enhancements retrieved from Enhancer) as field:value pairs along with the document.</p>
+<p>The document itself and all metadata are indexed through an embedded Apache Solr core/index which is created specifically for Contenthub. Since documents are given unique IDs while indexing, using its unique ID, a document can be retrieved or deleted from Contenthub. Contenthub provides an HTML interface for its functionalities under the following endpoint, which is available after running the full launcher of Apache Stanbol:</p>
+<div class="codehilite"><pre><span class="n">http:</span><span class="sr">//</span><span class="n">localhost:8080</span><span class="o">/</span><span class="n">contenthub</span>
+</pre></div>
+
+
+<p>Apache Solr can manage several cores (indexes) within the same running instance, and Contenthub makes use of this facility to manage different those cores. This management performed by LDPath programs[1].</p>
+<p>LDPath is a simple path-based query language similar to XPath or SPARQL Property Paths that is particularly well-suited for querying and retrieving resources from the Linked Data Cloud by following RDF links between resources and servers. For example, the following path query would select the names of objects (people) who is known by the context resource (the resource on which this path is being executed):<br />
+</p>
+<div class="codehilite"><pre><span class="err">foaf:knows</span> <span class="err">/</span> <span class="err">foaf:name</span>
+</pre></div>
+
+
+<p>An LDPath program is a collection of path queries. For example, following LDPath program can be executed on the resources which can be retrieved from Stanbol Enhancer as a result of the enhancement process. An LDPath program can be executed on any semantic collection of resources to extract specific information.</p>
+<div class="codehilite"><pre><span class="nv">@prefix</span> <span class="n">rdf</span> <span class="p">:</span> <span class="sr"><http://www.w3.org/1999/02/22-rdf-syntax-ns#></span><span class="p">;</span>
+<span class="nv">@prefix</span> <span class="n">rdfs</span> <span class="p">:</span> <span class="sr"><http://www.w3.org/2000/01/rdf-schema#></span><span class="p">;</span>
+<span class="nv">@prefix</span> <span class="n">db</span><span class="o">-</span><span class="n">ont</span> <span class="p">:</span> <span class="sr"><http://dbpedia.org/ontology/></span><span class="p">;</span>
+<span class="n">title</span> <span class="o">=</span> <span class="n">rdfs:label</span> <span class="o">::</span> <span class="n">xsd:string</span><span class="p">;</span>
+<span class="n">dbpediatype</span> <span class="o">=</span> <span class="n">rdf:type</span> <span class="o">::</span> <span class="n">xsd:anyURI</span><span class="p">;</span>
+<span class="n">population</span> <span class="o">=</span> <span class="n">db</span><span class="o">-</span><span class="n">ont:populationTotal</span> <span class="o">::</span> <span class="n">xsd:int</span><span class="p">;</span>
+</pre></div>
+
+
+<p>Given an LDPath program, Contenthub can create a corresponding Solr core to index the content items through that core. When you submit a document to Contenthub Store by providing an LDPath program, this means the content item (the document content and its metadata/enhancements) will be indexed according to the fields determined by the LDPath program. For instance, the example LDPath program above will lead to a Solr core including the following fields (in addition to default configuration and several default fields)</p>
+<div class="codehilite"><pre><span class="o"><</span><span class="n">field</span> <span class="n">name</span><span class="o">=</span><span class="s">"title"</span> <span class="n">type</span><span class="o">=</span><span class="s">"string"</span> <span class="n">stored</span><span class="o">=</span><span class="s">"true"</span> <span class="n">indexed</span><span class="o">=</span><span class="s">"true"</span> <span class="n">multiValued</span><span class="o">=</span><span class="s">"true"</span><span class="o">/></span>
+<span class="o"><</span><span class="n">field</span> <span class="n">name</span><span class="o">=</span><span class="s">"dbpediatype"</span> <span class="n">type</span><span class="o">=</span><span class="s">"uri"</span> <span class="n">stored</span><span class="o">=</span><span class="s">"true"</span> <span class="n">indexed</span><span class="o">=</span><span class="s">"true"</span> <span class="n">multiValued</span><span class="o">=</span><span class="s">"true"</span><span class="o">/></span>
+<span class="o"><</span><span class="n">field</span> <span class="n">name</span><span class="o">=</span><span class="s">"population"</span> <span class="n">type</span><span class="o">=</span><span class="s">"int"</span> <span class="n">stored</span><span class="o">=</span><span class="s">"true"</span> <span class="n">indexed</span><span class="o">=</span><span class="s">"true"</span> <span class="n">multiValued</span><span class="o">=</span><span class="s">"true"</span><span class="o">/></span>
+</pre></div>
+
+
+<p>To submit an LDPath program, you can use the following command through the REST API of Contenthub</p>
+<div class="codehilite"><pre><span class="n">curl</span> <span class="o">-</span><span class="n">i</span> <span class="o">-</span><span class="n">X</span> <span class="n">POST</span> <span class="o">-</span><span class="n">d</span> <span class="s">"name=myindex&program=@prefix rdf : <http://www.w3.org/1999/02/22-rdf-syntax-ns#>; @prefix rdfs : <http://www.w3.org/2000/01/rdf-schema#>; @prefix db-ont : <http://dbpedia.org/ontology/>; title = rdfs:label :: xsd:string; dbpediatype = rdf:type :: xsd:anyURI; population = db-ont:populationTotal :: xsd:int;"</span> <span class="n">http:</span><span class="sr">//</span><span class="n">localhost:8080</span><span class="sr">/contenthub/</span><span class="n">ldpath</span><span class="o">/</span><span class="n">program</span>
+</pre></div>
+
+
+<p>You can retrieve the list of managed LDPath programs in JSON format with the following command. This is also the list of available Solr cores (except the default Solr core)</p>
+<div class="codehilite"><pre><span class="n">curl</span> <span class="o">-</span><span class="n">i</span> <span class="o">-</span><span class="n">X</span> <span class="n">GET</span> <span class="n">http:</span><span class="sr">//</span><span class="n">localhost:8080</span><span class="sr">/contenthub/</span><span class="n">ldpath</span>
+</pre></div>
+
+
+<p>LDPath related management is performed through SemanticIndexManager of Contenthub. To take advantage of semantic indexes while storing content items, you need to specify the name of the index in the path of the url while submitting the document. Default index for contenthub is named as "contenthub". Hence, following command submits document to the default index:</p>
+<div class="codehilite"><pre><span class="n">curl</span> <span class="o">-</span><span class="n">i</span> <span class="o">-</span><span class="n">X</span> <span class="n">POST</span> <span class="o">-</span><span class="n">H</span> <span class="s">"Content-Type:application/x-www-form-urlencoded"</span> <span class="o">-</span><span class="n">d</span> <span class="s">"title=about me&content=I live in Istanbul.&"</span> <span class="n">http:</span><span class="sr">//</span><span class="n">localhost:8080</span><span class="sr">/contenthub/co</span><span class="n">ntenthub</span><span class="o">/</span><span class="n">store</span>
+</pre></div>
+
+
+<p>Following command will store the content item into Solr core names with "myindex". Therefore, the indexing will be performed through the field properties indicated with the LDPath program named with "myindex".</p>
+<div class="codehilite"><pre><span class="n">curl</span> <span class="o">-</span><span class="n">i</span> <span class="o">-</span><span class="n">X</span> <span class="n">POST</span> <span class="o">-</span><span class="n">H</span> <span class="s">"Content-Type:application/x-www-form-urlencoded"</span> <span class="o">-</span><span class="n">d</span> <span class="s">"title=about me&content=I live in Istanbul.&"</span> <span class="n">http:</span><span class="sr">//</span><span class="n">localhost:8080</span><span class="sr">/contenthub/m</span><span class="n">yindex</span><span class="o">/</span><span class="n">store</span>
+</pre></div>
+
+
+<h2 id="contenthub-search">Contenthub Search</h2>
+<p>Contenthub provides three search interfaces so that capabilities of Stanbol can be adopted by the users through different levels of complexities. These interfaces are;</p>
+<ul>
+<li><strong>SolrSearch</strong>: provides native Solr interface to the outside world.
+ Retrieved the resulting content items (documents) from the Solr
+ backend. SolrJ users can easily make use of this interface. Search
+ is performed on the corresponding Solr index and results are
+ returned in "org.apache.solr.client.solrj.response.QueryResponse"
+ format.</li>
+<li><strong>RelatedKeywordSearch</strong>: provides supporting functionalities for search
+ facilities. Given a keyword, services of this interface finds other
+ related keywords from several sources. Wordnet, domain ontologies
+ and referenced sites are the data sources for these services to
+ retrieve the related keywords.</li>
+<li><strong>FeaturedSearch</strong>: Combines the services of SolrSearch and
+ RelatedKeywordSearch for the users who want the results of a query
+ term all in one interface. Featured search not only returns
+ resulting documents, but also related keywords retrieved from
+ various resources (if the resources are available within the running
+ Stanbol instance) Given a query term, returns the resultant
+ documents from the queried Solr core/index and related keywords from
+ different sources.</li>
+</ul>
+<p>Following request retrieves all documents from the default index (whose name is "contenthub") of Solr:</p>
+<div class="codehilite"><pre><span class="n">http:</span><span class="sr">//</span><span class="n">localhost:8080</span><span class="sr">/solr/</span><span class="n">default</span><span class="sr">/contenthub/s</span><span class="n">elect</span><span class="p">?</span><span class="n">q</span><span class="o">=*</span><span class="p">:</span><span class="o">*</span>
+</pre></div>
+
+
+<p>Following request retrieves all documents from the Solr index named as "myindex":</p>
+<div class="codehilite"><pre><span class="n">http:</span><span class="sr">//</span><span class="n">localhost:8080</span><span class="sr">/solr/</span><span class="n">default</span><span class="sr">/myindex/s</span><span class="n">elect</span><span class="p">?</span><span class="n">q</span><span class="o">=*</span><span class="p">:</span><span class="o">*</span>
+</pre></div>
+
+
+<p>RelatedKeywordSearch is performed by three independent search engines within the Stanbol system, namely: </p>
+<ul>
+<li><strong>OntologyResourceSearch</strong>: If an ontology is already registered to
+ Stanbol (e.g. a domain ontology), it can be used to look for similar
+ keywords, given a keyword. A SPARQL query based on a LARQ index is
+ executed on the specified ontology to find individual and class
+ resources related with the keyword.</li>
+<li><strong>ReferencedSiteSearch</strong>: Referenced sites are used to retrieve the
+ enhancements of a content item. Stanbol Enhancer handles all
+ enhancement operations through the referenced sites. This interface
+ makes use of the referenced sites to look for similar keywords,
+ given a keyword.</li>
+<li><strong>WordnetSearch</strong>: If a Wordnet database is registered to the system
+ (through the OSGi console), this service is ready for use. Looks for
+ several relations among keywords (such as synonyms, hyponyms etc...)
+ and retrieves a list of related keywords from the Wordnet database.</li>
+</ul>
+<p>Following command will retrieve related keywords about "turkey" from referenced sites and wordnet (ReferencedSiteSearch and WordnetSearch). Since no ontology is specified, OntologyResourceSearch will not execute.</p>
+<div class="codehilite"><pre><span class="n">curl</span> <span class="o">-</span><span class="n">i</span> <span class="o">-</span><span class="n">X</span> <span class="n">GET</span> <span class="o">-</span><span class="n">H</span> <span class="s">"Accept: application/json"</span> <span class="n">http:</span><span class="sr">//</span><span class="n">localhost:8080</span><span class="sr">/contenthub/co</span><span class="n">ntenthub</span><span class="sr">/search/</span><span class="n">related</span><span class="p">?</span><span class="n">keyword</span><span class="o">=</span><span class="n">turkey</span>
+</pre></div>
+
+
+<p>If URI of an ontology is also specified with the keyword as follows, result of the service will include related keywords found through the specified ontology in addition to referenced site and wordnet data. Following command will add the related keywords of "turkey" which are retrieved from the ontology identified with "uri-dummy" to the search result of related keyword service.</p>
+<div class="codehilite"><pre><span class="n">curl</span> <span class="o">-</span><span class="n">i</span> <span class="o">-</span><span class="n">X</span> <span class="n">GET</span> <span class="o">-</span><span class="n">H</span> <span class="s">"Accept: application/json"</span> <span class="n">http:</span><span class="sr">//</span><span class="n">localhost:8080</span><span class="sr">/contenthub/co</span><span class="n">ntenthub</span><span class="sr">/search/</span><span class="n">related</span><span class="p">?</span><span class="n">keyword</span><span class="o">=</span><span class="n">turkey</span><span class="o">&</span><span class="n">ontologyURI</span><span class="o">=</span><span class="n">uri</span><span class="o">-</span><span class="n">dummy</span>
+</pre></div>
+
+
+<p>Lastly, Contenthub provides a featured search interface which combines the services of SolrSearch and RelatedKeywordSearch. Results of the services of FeaturedSearch interface includes resultant documents and related keywords of the given query term. Following query will retrieve the documents whose indexed fileds includes the term "turkey" and related keywords from several sources about "turkey".</p>
+<div class="codehilite"><pre><span class="n">curl</span> <span class="o">-</span><span class="n">i</span> <span class="o">-</span><span class="n">X</span> <span class="n">GET</span> <span class="o">-</span><span class="n">H</span> <span class="s">"Accept: application/json"</span> <span class="o">-</span><span class="n">H</span> <span class="s">"Content-Type:text/plain"</span> <span class="n">http:</span><span class="sr">//</span><span class="n">localhost:8080</span><span class="sr">/contenthub/co</span><span class="n">ntenthub</span><span class="sr">/search/</span><span class="n">featured</span><span class="p">?</span><span class="n">queryTerm</span><span class="o">=</span><span class="n">turkey</span>
+</pre></div>
</div>
<div id="footer">