You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@stanbol.apache.org by rw...@apache.org on 2014/05/22 15:24:29 UTC
svn commit: r1596858 - in /stanbol/site/trunk: ./ content/docs/trunk/
content/docs/trunk/components/enhancer/
content/docs/trunk/components/enhancer/chains/
content/docs/trunk/components/enhancer/engines/
content/docs/trunk/components/entityhub/ conten...
Author: rwesten
Date: Thu May 22 13:24:28 2014
New Revision: 1596858
URL: http://svn.apache.org/r1596858
Log:
Added Documentation for STANBOL-488 - Enhancement Properties, STANBOL-336 - DereferenceEngine, STANBOL-1223 - Entityhub Dereference Engine, STANBOL-1165 - Entityhub Apache Marmotta integration
Added:
stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/enhancer-graphchain-enhprop-config.png (with props)
stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/enhancer-listchain-enhprop-config.png (with props)
stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/enhancer-weightedchain-enhprop-config.png (with props)
stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/dereference.mdtext
stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entityhub-dereference-engine-config.png (with props)
stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entityhub-dereference-engine-shared-threadpool-config.png (with props)
stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entityhubdereference.mdtext
stanbol/site/trunk/content/docs/trunk/components/enhancer/enhancementproperties.mdtext
stanbol/site/trunk/content/docs/trunk/components/entityhub/entityhub-manatedsite-sesameyard-config.png (with props)
stanbol/site/trunk/content/docs/trunk/components/entityhub/marmotta-kiwi-repository-service.png (with props)
stanbol/site/trunk/content/docs/trunk/utils/marmotta-kiwi-repository-service-config.png (with props)
stanbol/site/trunk/content/docs/trunk/utils/marmotta-kiwi-repository-service.mdtext
stanbol/site/trunk/content/docs/trunk/utils/marmotta-kiwi-repository-service.png (with props)
Modified:
stanbol/site/trunk/WEBSITE-HOWTO.txt
stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/graphchain.mdtext
stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/listchain.mdtext
stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/list.mdtext
stanbol/site/trunk/content/docs/trunk/components/enhancer/enhancerrest.mdtext
stanbol/site/trunk/content/docs/trunk/components/enhancer/index.mdtext
stanbol/site/trunk/content/docs/trunk/components/entityhub/managedsite.mdtext
stanbol/site/trunk/content/docs/trunk/index.mdtext
stanbol/site/trunk/content/docs/trunk/utils/index.mdtext
Modified: stanbol/site/trunk/WEBSITE-HOWTO.txt
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/WEBSITE-HOWTO.txt?rev=1596858&r1=1596857&r2=1596858&view=diff
==============================================================================
--- stanbol/site/trunk/WEBSITE-HOWTO.txt (original)
+++ stanbol/site/trunk/WEBSITE-HOWTO.txt Thu May 22 13:24:28 2014
@@ -3,7 +3,7 @@ How to update the Stanbol website
INTRO
-----
-The http://incubator.apache.org/stanbol/ is managed by the ASF CMS
+The http://stanbol.apache.org/ is managed by the ASF CMS
(http://cms.apache.org).
All Stanbol committers have read-write access to the website content,
@@ -17,7 +17,7 @@ See also http://wiki.apache.org/general/
WORKING IN SVN
--------------
Committing content under
- https://svn.apache.org/repos/asf/incubator/stanbol/site/trunk
+ https://svn.apache.org/repos/asf/stanbol/site/trunk
causes a staging build of the website to be executed.
The staged content becomes available under
Added: stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/enhancer-graphchain-enhprop-config.png
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/enhancer-graphchain-enhprop-config.png?rev=1596858&view=auto
==============================================================================
Binary file - no diff available.
Propchange: stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/enhancer-graphchain-enhprop-config.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/enhancer-listchain-enhprop-config.png
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/enhancer-listchain-enhprop-config.png?rev=1596858&view=auto
==============================================================================
Binary file - no diff available.
Propchange: stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/enhancer-listchain-enhprop-config.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/enhancer-weightedchain-enhprop-config.png
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/enhancer-weightedchain-enhprop-config.png?rev=1596858&view=auto
==============================================================================
Binary file - no diff available.
Propchange: stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/enhancer-weightedchain-enhprop-config.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Modified: stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/graphchain.mdtext
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/graphchain.mdtext?rev=1596858&r1=1596857&r2=1596858&view=diff
==============================================================================
--- stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/graphchain.mdtext (original)
+++ stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/graphchain.mdtext Thu May 22 13:24:28 2014
@@ -61,6 +61,106 @@ A better visual expression provides this
+## Enhancement Properties Support
+
+__since `0.12.1`__
+
+Starting from `0.12.1` the Graph Chain allows to configure [EnhancementProperties](../enhancementproperties).
+
+### Chain List based Configuration
+
+In case the _Chain List_ type configuration is used properties are configured as follows:
+
+* __chain and engine__ scoped properties are defined as parameters to the engines with the syntax `{engine-name}; {property-name-1}={value-1},{value-2}; {property-name-2}={value-1};`
+
+* __chain__ scoped properties can be configured by using the osgi property key `stanbol.enhancer.chain.chainproperties` by the syntax `{property-name-1}={value-1},{value-2}`. NOTE that `;` is NOT supported as separator for parsing multiple properties as OSGI configurations already define a way for parsing multiple values
+
+All EnhancementProperties configured with a [Chain](chains) are written as RDF to the [ExecutionPlan](chains/executionplan). _Chain_ scoped properties are directly added to the `ep:ExecutionPlan` instance while _chain and engine_ scoped properties are added to the `ep:ExecutionNode` of the according engine.
+
+The following figure and listing provide an example
+
+
+
+The figure shows the maximum number of suggestions is set as a _chain_ scoped property to `5`. In addition two _chain and engine_ scoped properties are set. First for the `dbpedia-fst` engine the minimum confidence is set to `0.85` and second for the `dbpedia-dereference` engine the dereferenced languages are set to English, German and Spanish.
+
+In case of the GraphChain it is typical that _chain and engine_ scoped Enhancement Properties get mixed with parameters of the chain configuration itself. As Enhancement Properties are required to start with `enhancer.` they can be easily separated with chain specific parameters such as `dependsOn`.
+
+The following listing shows the exact same configuration in the `.cfg` format.
+
+ stanbol.enhancer.chain.name="graph-chain"
+ stanbol.enhancer.chain.chainproperties=["enhancer.max-suggestions\=5"]
+ stanbol.enhancer.chain.graph.chainlist=["tika;optional","langdetect;\ dependsOn\=tika",
+ "opennlp-sentence;\ dependsOn\=langdetect","opennlp-token;\ dependsOn\=opennlp-sentence",
+ "opennlp-pos;\ dependsOn\=opennlp-pos","opennlp-chunker;\ optional;\ dependsOn\=opennlp-chunker",
+ "opennlp-ner;\ dependsOn\=opennlp-pos",
+ "dbpedia-fst;\ dependsOn\=opennlp-chunker,opennlp-pos;enhancer.min-confidence\=0.85",
+ "dbpedia-dereference;\ dependsOn\=dbpedia-fst;\ enhancer.engines.dereference.languages\=en,de,es"]
+
+### Graph Resource Configuration
+
+In case the [ExecutionPlan](executionplan) is configured by an RDF file the [EnhancementProperties](../enhancementproperties) need to be directly encoded as RDF.
+
+_Chain_ scoped properties need to be attached to the `ep:ExecutionPlan` instance while _chain and engine_ scoped properties are added to the `ep:ExecutionNode` of the according engine.
+
+Single properties are represented by triples where the execution plan or execution mode instance is the subject. The URI or the enhancement property is the predicate and the value is the object. Multiple valus are represented by multiple triples with the same subject and predicate.
+
+The following listing shows the same example as used in the above section as RDF turtle.
+
+ :::text
+ @prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
+ @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+ @prefix ep: <http://stanbol.apache.org/ontology/enhancer/executionplan#> .
+ @prefix ehp: <http://stanbol.apache.org/ontology/enhancementproperties#> .
+
+ urn:execPlan a ep:ExecutionPlan ;
+ ep:hasExecutionNode urn:node1, urn:node2, urn:node3, urn:node4, urn:node5
+ urn:node6, urn:node7, urn:node8;
+ ep:chain "demoChain" ;
+ ehp:enhancer.max-suggestions "5"^^xsd:int .
+
+ urn:node1 a stanbol:ExecutionNode ;
+ ep:inExecutionPlan urn:execPlan ;
+ ep:engine "langdetect" .
+
+ urn:node2 a ep:ExecutionNode ;
+ ep:inExecutionPlan urn:execPlan ;
+ ep:dependsOn urn:node1 ;
+ ep:engine "opennlp-sentence" .
+
+ urn:node3 a ep:ExecutionNode ;
+ ep:inExecutionPlan urn:execPlan ;
+ ep:dependsOn urn:node2 ;
+ ep:engine "opennlp-token" .
+
+ urn:node4 a ep:ExecutionNode ;
+ ep:inExecutionPlan urn:execPlan ;
+ ep:dependsOn urn:node3 ;
+ ep:engine "opennlp-pos" .
+
+ urn:node5 a ep:ExecutionNode ;
+ ep:inExecutionPlan urn:execPlan ;
+ ep:dependsOn urn:node4 ;
+ ep:engine "opennlp-chunker" .
+
+ urn:node6 a ep:ExecutionNode ;
+ ep:inExecutionPlan urn:execPlan ;
+ ep:dependsOn urn:node4 ;
+ ep:engine "opennlp-ner" .
+
+ urn:node7 a ep:ExecutionNode ;
+ ep:inExecutionPlan urn:execPlan ;
+ ep:dependsOn urn:node5 ;
+ ep:engine "dbpedia-fst" ;
+ ehp:enhancer.min-confidence "0.85"^^xsd:float .
+
+ urn:node8 a ep:ExecutionNode ;
+ ep:inExecutionPlan urn:execPlan ;
+ ep:dependsOn urn:node7 ;
+ ep:engine "dbpedia-dereference" ;
+ ehp:enhancer.engines.dereference.languages "en", "de", "es" .
+
+
+
## Execution
In contrast to other chain implementations the ExecutionPlan must not be calculated but is directly parsed by the user. This provides the most possible freedom in defining how the execution should take place.
Modified: stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/listchain.mdtext
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/listchain.mdtext?rev=1596858&r1=1596857&r2=1596858&view=diff
==============================================================================
--- stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/listchain.mdtext (original)
+++ stanbol/site/trunk/content/docs/trunk/components/enhancer/chains/listchain.mdtext Thu May 22 13:24:28 2014
@@ -29,6 +29,36 @@ To create the same configuration as in t
stanbol.enhancer.chain.name="list"
stanbol.enhancer.chain.list.enginelist=["metaxa;optional","langid","ner","dbpediaLinking"]
+## Enhancement Properties support
+
+__since `0.12.1`__
+
+Starting from `0.12.1` the List Chain allows to configure [EnhancementProperties](../enhancementproperties)
+
+* __chain and engine__ scoped properties are defined as parameters to the engines with the syntax `{engine-name}; {property-name-1}={value-1},{value-2}; {property-name-2}={value-1};`
+
+* __chain__ scoped properties can be configured by using the osgi property key `stanbol.enhancer.chain.chainproperties` by the syntax `{property-name-1}={value-1},{value-2}`. NOTE that `;` is NOT supported as separator for parsing multiple properties as OSGI configurations already define a way for parsing multiple values
+
+All EnhancementProperties configured with a [Chain](chains) are written as RDF to the
+[ExecutionPlan](chains/executionplan). _Chain_ scoped properties are directly added to the
+`ep:ExecutionPlan` instance while _chain and engine_ scoped properties are added to the
+`ep:ExecutionNode` of the according engine.
+
+The following figure and listing provide an example
+
+
+
+The figure shows the definition of two _chain and engine_ scoped and one _chain_ scoped enhancement properties. First the maximum number of suggestions are set on a chain scope to `5`. This is overridden by a specific configuration of the `dbpedia-fst` engine that thats this value to `10` for this engine. Finally the dereferenced languages are set to English, German and French for the `dbpedia-dereference` engine.
+
+The following listing shows the exact same configuration in the `.cfg` format.
+
+ stanbol.enhancer.chain.name="list"
+ stanbol.enhancer.chain.list.enginelist=["tika;optional","langdetect","opennlp-sentence","opennlp-token","opennlp-pos","opennlp-chunker",
+ "dbpedia-fst;\ enhancer.max-suggestions\=10",
+ "dbpedia-dereference;\ enhancer.engine.dereference.languages\=en,de,fr"]
+ stanbol.enhancer.chain.chainproperties=["enhancer.max-suggestions\=5"]
+
+
## Calculation of the ExecutionPlan
The ExecutionPlan is created based on the exact order of the [EnhancementEngines](../engines) provided by the "stanbol.enhancer.chain.list.enginelist" property. The configuration MUST contain at least a single engine. In addition no engine MUST be mentioned twice.
\ No newline at end of file
Added: stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/dereference.mdtext
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/dereference.mdtext?rev=1596858&view=auto
==============================================================================
--- stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/dereference.mdtext (added)
+++ stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/dereference.mdtext Thu May 22 13:24:28 2014
@@ -0,0 +1,165 @@
+title: Entity Dereference Engine
+
+The responsibility of the Dereference Engine is to retrieve information about Entities
+referenced by the Enhancement Results and add them to the metadata of the [Content Item](../contentitem).
+
+## Consumed information
+
+The Entity Dereference Engine consumes the RDF enhancements generated by other Enhancement Engines. Especially the `fise:entity-reference` properties used by `fise:EntityAnnotation` and `fise:TopicAnnotation` are processed by this engine as they do link to the Entities that need to be dereferenced.
+
+* __Language__ (optional): The language detected for the text may be used to determine the set of languages of literals to be dereferenced.
+
+## Design
+
+The Entity Dereference Engine can not directly be used to dereference Entities. It provides the infrastructure to implement actual dereference
+Engines for different technologies and services such as the [Entityhub Dereference Engine](entityhubdereference) for dereferencing Entities via the [Stanbol Entityhub](/docs/trunk/components/entityhub/)).
+
+The engine is defined by the `org.apache.stanbol:org.apache.stanbol.enhancer.dereference.core` module and defines the following two main components:
+
+1. [EnhancementEngine](index) implementation that
+ * processes the Enhancement results and schedules Entities to be dereferenced.
+ * supports the use of a thread pool to dereference multiple entities concurrently.
+ * supports [EnhancementProperties](../enhancementproperties) for _chain_ and _request_ scoped configuration of the dereferenced information.
+2. Definition of the `EntityDerefernece` interface used to dereference scheduled entities. This interface needs to be implemented by Dereference Engines for different technologies/services (e.g. the Entityhub)
+
+## Configuration
+
+The following Configuration parameter are defined by the core Entity Dereference Engine. Actual Dereference Engine implementations might not support all of them.
+
+* __Name__ _(stanbol.enhancer.engine.name)_: The name of the Enhancement engine
+* __URI Prefix__ _(enhancer.engines.dereference.uriPrefix)_: Allows to configure a simple prefix of Entities that can be dereferenced by this engine. If present only Entities referenced in the Enhancement Metadata matching at least one of the configured prefixes (or one of the configured URI Patterns) will be scheduled for dereferencing.
+* __URI Pattern__ _(enhancer.engines.dereference.uriPatter)_: Allows to configure a regex pattern for matching Entity URIs. If present only Entities matching at lease one of the configured regex patterns ((or one of the configured URI Prefixes) will be scheduled for dereferencing.
+* __Fallback Mode__ _(enhancer.engines.dereference.fallback)_: If enabled the EnhancementEngine ordering will get adapted in a way that in case a [Weighted Chain](../chains/weightedchain) is use this Dereference Engine will get called after other with this option deactivated. In addition only Entities with no dereferenced data will get scheduled for dereferencing. This option is only useful in cases where multiple dereference engines are used in the same enhancement chain. It allows to ensure the following workflows
+ 1. First running Dereference Engines for fast/local data sources. Especially those where one can configure an _URI Prefix_ and/or an _URI Pattern_. Second running Dereference Engines for datastes that require remote service calls or for those no _URI Prefix_ nor _URI Pattern_ can be configured. This can be achieved by deactivating _Fallback Mode_ for the first group and activating it for the second.
+ 2. Dereference Entities from slow/remote data sources: E.g. when using a partial local cache of a knowledge base one can configure a Dereference Engine with deactivated _Fallback Mode_ for the local cache and an other for the remote datasource with activated _Fallback Mode_. This setting ensures that only Entities not available in the local cache are requested from the remote service.
+* __Dereference only Content Language Literals__ _(enhancer.engine.dereference.filterContentlanguages)_: If enabled only Literals with the same language as the language detected for the Content will get dereferenced. Literals with no language tag will always get dereferenced.
+* __Dereference Languages__ _(enhancer.engines.dereference.languages)_: A set of languages that are dereferenced. Even if _'Dereference only Content Language Literals'_ is active explicitly configured languages will still get dereferenced. If not present and _'Dereference only Content Language Literals'_ is deactivated literals of any language will get dereferenced.
+* __Dereferenced Fields__ _(enhancer.engines.dereference.fields)_: The dereferenced fields - in RDF terminology 'properties' - to be dereferenced. Typically QNames (e.g. `rdf:label`) can be used for the configuration. However support for QNames is optional. Some Implementations might also support wildcards and exclusions.
+* __Dereference LD Path__ _(enhancer.engines.dereference.ldpath)_: The [LD Path Language](http://marmotta.apache.org/ldpath/language.html) allows to define powerful selectors for dereferenced Entities.
+* __Service Ranking__ _(service.ranking)_: The OSGI service ranking. Will only have an effect if their are two engines with the same name. In such cases the one with the higher service ranking will get called.
+
+_NOTE_ that the configurations for _Dereference Languages_, _Dereferenced Fields_ and _Dereference LD Path_ are just managed by the Core Entity Dereference Engine implementation. Actual support for such properties will depend on the actual `EntityDereferencer` implementation.
+
+## Building a Custom Entity Dereference Engine
+
+This provides information about the necessary steps for building a custom Entity Dereference Engine.
+
+### Entity Dereferencer implementation
+
+The `EntityDereferencer` interface is used to dereference Entities. It also allows the `EntityDereferenceEngine` to check if _OfflineMode_ is supported and to retrieve the `ExecutorService` service.
+
+The following listing shows the signature of the `EntityDereferencer` interface
+
+ :::java
+ EntityDereferencer
+ + supportsOfflineMode() : boolean
+ + getExecutor() : ExecutorService
+ + boolean dereference(UriRef entity, MGraph graph, Lock writeLock,
+ DereferenceContext dereferenceContext) throws DereferenceException;
+
+`supportsOfflineMode` need to return `true` if the implementation does not need to access a remote service for dereferencing entities and `false` if it requires remote services. Stanbol can be started with an option that enforces _Offline Mode_. In this case EntityDereferencers that do not support this mode will be deactivated.
+
+The `ExecutorService` is used by the `EntityDereferenceEngine` to concurrently dereference entities. This means that the `dereference(..)` method of the `EntityDereferencer` implementations will be called in the contexts of threads provided by the returned `ExecutorService`. Returning `null` will deactivate this feature. However note that still multiple threads will be used to call the `dereference(..)` method. Meaning that the implementation MUST BE thread save.
+
+The `dereference(..)` method is used to dereference the Entity with the parsed `UriRef`. Dereferenced information are expected to be written in the parsed `MGraph`. While writing a write lock MUST BE acquired. The `DereferenceContext` provides the configuration (see the following section for more information). If the parsed entity was successfully dereferenced this method is expected to return `true`. Otherwise `false`.
+
+
+### Configuration API
+
+Configuration Parameters supported by the Core Entity Dereference Engine implementation are defined in the `DereferenceConstants` class.
+
+The `DereferenceEngineConfig` class provides an easy - API based - access to those configuration parameters. It is instantiated by using the `Dictionary` parsed by the OSGI as part of the `ComponentContext`.
+
+Finally the `DereferenceContext` is used to parse both the `DereferenceEngineConfig` and the request specific configuration (including _Offline Mode_ state and [Enhancement Properties](../enhancementproperties)) to calls to the `EntityDereferencer`.
+
+Specific EntityDereferencer implementation can use their own custom `DereferenceContext` implementations. Typically they want to do that as this allows to avoid parsing request specific configurations multiple times. For this it is important to understand that one call to the `EntityDereferenceEngine` will result in multiple calls to the `EntityDereferencer` - one call for each scheduled Entity. The same `DereferenceContext` instance will be used for each of those calls. `DereferenceContext` instances are created by using the `DereferenceContextFactory` parsed when constructing the `DereferenceEngine`. So a custom `DereferenceContext` implementation allows to ensure that request specific configurations are only parsed once per request and not for each dereferenced entity.
+
+The following listing shows how to instantiate a `EntityDereferenceEngine` with a custom `DereferenceContext`.
+
+ :::java
+ entityDereferenceEngine = new EntityDereferenceEngine(entityDereferencer, engineConfig,
+ new DereferenceContextFactory() { //we want to use our own DereferenceContext impl
+
+ @Override
+ public DereferenceContext createContext(EntityDereferenceEngine engine,
+ Map<String,Object> enhancementProperties) throws DereferenceConfigurationException {
+ //Instantiate custom DereferenceContext
+ DereferenceContext dereferenceContext = null; //TODO
+ return dereferenceContext;
+ }
+ });
+
+If you apply this code all calls to `EntityDereferencer#dereference(..)` will parse an instance of the custom `DereferenceContext` implementation.
+
+The custom [DereferenceContext](http://svn.apache.org/repos/asf/stanbol/trunk/enhancement-engines/dereference/entityhub/src/main/java/org/apache/stanbol/enhancer/engines/dereference/entityhub/EntityhubDereferenceContext.java) implementation of the Entityhub Dereference Engine is a good example to start from.
+
+### OSGI Component
+
+Finally each Dereference Engine implementation needs to provide an OSGI component. This component is required for parsing the configuration and for implementing the life cycle.
+
+The following listing provide the pseudo code for such a component
+
+ :::java
+ @Component(
+ configurationFactory = true, //allow multiple instances
+ policy = ConfigurationPolicy.REQUIRE, //a configuration is required
+ metatype = true, immediate = true)
+ @Properties(value={
+ @Property(name=PROPERTY_NAME), //the name of the engine
+ //Properties supported by the Core Entity Dereference Engine
+ @Property(name=EntityhubDereferenceEngine.SITE_ID),
+ @Property(name=DereferenceConstants.FALLBACK_MODE,
+ boolValue=DereferenceConstants.DEFAULT_FALLBACK_MODE),
+ @Property(name=DereferenceConstants.URI_PREFIX, cardinality=Integer.MAX_VALUE),
+ @Property(name=DereferenceConstants.URI_PATTERN, cardinality=Integer.MAX_VALUE),
+ @Property(name=DereferenceConstants.FILTER_CONTENT_LANGUAGES,
+ boolValue=DereferenceConstants.DEFAULT_FILTER_CONTENT_LANGUAGES),
+ @Property(name=DEREFERENCE_ENTITIES_FIELDS,cardinality=Integer.MAX_VALUE,
+ value={"rdfs:comment","geo:lat","geo:long","foaf:depiction","dbp-ont:thumbnail"}),
+ @Property(name=DEREFERENCE_ENTITIES_LDPATH, cardinality=Integer.MAX_VALUE),
+ /* add also implementation specific properties */
+ @Property(name=SERVICE_RANKING,intValue=0)
+ })
+ public class YourDereferneceEngineComponent {
+
+ /** support QName configurations */
+ @Reference(cardinality=ReferenceCardinality.OPTIONAL_UNARY)
+ protected NamespacePrefixService prefixService;
+
+ /** The engine instance registered as OSGI service */
+ protected EntityDereferenceEngine entityDereferenceEngine;
+ /** The OSGI service registration */
+ protected ServiceRegistration engineRegistration;
+
+ @Activate
+ protected void activate(ComponentContext ctx) throws ConfigurationException {
+ Dictionary<String,Object> properties = ctx.getProperties();
+ DereferenceEngineConfig engineConfig = new DereferenceEngineConfig(properties, prefixService);
+
+ /* TODO: parse custom configuration properties */
+
+ /* Initialise the custom EntityDereferencer implemenation */
+ EntiyDereferencer dereferencer; //TODO
+
+ //create the Entity Dereference Engine instance
+ entityDereferenceEngine = new EntityDereferenceEngine(entityDereferencer, engineConfig);
+
+ //register the engine as OSGI service
+ engineRegistration = ctx.getBundleContext().registerService(
+ new String[]{EnhancementEngine.class.getName(),
+ ServiceProperties.class.getName()},
+ entityDereferenceEngine, engineConfig.getDict());
+ }
+
+ @Deactivate
+ protected void deactivate(ComponentContext context) {
+ //Unregister the OSGI service
+ if(engineRegistration != null){
+ engineRegistration.unregister();
+ engineRegistration = null;
+ }
+ entityDereferenceEngine = null;
+
+ //TODO: close the dereferencer implementation (if required)
+ }
+ }
\ No newline at end of file
Added: stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entityhub-dereference-engine-config.png
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entityhub-dereference-engine-config.png?rev=1596858&view=auto
==============================================================================
Binary file - no diff available.
Propchange: stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entityhub-dereference-engine-config.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entityhub-dereference-engine-shared-threadpool-config.png
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entityhub-dereference-engine-shared-threadpool-config.png?rev=1596858&view=auto
==============================================================================
Binary file - no diff available.
Propchange: stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entityhub-dereference-engine-shared-threadpool-config.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entityhubdereference.mdtext
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entityhubdereference.mdtext?rev=1596858&view=auto
==============================================================================
--- stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entityhubdereference.mdtext (added)
+++ stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entityhubdereference.mdtext Thu May 22 13:24:28 2014
@@ -0,0 +1,46 @@
+title: Entityhub Dereference Engine
+
+This is a [Entity Dereference Engine](dereference) for the [Stanbol Entityhub](/docs/trunk/components/entityhub). It supports dereferencing Entities from
+
+* Entityhub: locally managed Entities
+* Managed and Referenced Sites
+* SiteManager: Union view over all Managed and Referenced Sites
+
+
+## Configuration
+
+The following figure shows the configuration dialog of the Entityhub Dereference Engine:
+
+
+
+The following Configuration parameter are defined by the core Entity Dereference Engine. Actual Dereference Engine implementations might not support all of them.
+
+* __Name__ _(stanbol.enhancer.engine.name)_: The name of the Enhancement engine
+* __Site__ _(enhancer.engines.dereference.entityhub.siteId)_: The name of the Entityhub Site to be used for dereferencing. `*` will dereference against the SiteManager (union over all Referenced and Managed sites) and `entityhub` will use the entityhub itself for dereferencing.
+* __Fallback Mode__ _(enhancer.engines.dereference.fallback)_: If enabled the EnhancementEngine ordering will get adapted in a way that in case a [Weighted Chain](../chains/weightedchain) is use this Dereference Engine will get called after other with this option deactivated. In addition only Entities with no dereferenced data will get scheduled for dereferencing. This option is only useful in cases where multiple dereference engines are used in the same enhancement chain. It allows to ensure the following workflows
+ 1. First running Dereference Engines for fast/local data sources. Especially those where one can configure an _URI Prefix_ and/or an _URI Pattern_. Second running Dereference Engines for datastes that require remote service calls or for those no _URI Prefix_ nor _URI Pattern_ can be configured. This can be achieved by deactivating _Fallback Mode_ for the first group and activating it for the second.
+ 2. Dereference Entities from slow/remote data sources: E.g. when using a partial local cache of a knowledge base one can configure a Dereference Engine with deactivated _Fallback Mode_ for the local cache and an other for the remote datasource with activated _Fallback Mode_. This setting ensures that only Entities not available in the local cache are requested from the remote service.
+* __URI Prefix__ _(enhancer.engines.dereference.uriPrefix)_: Allows to configure a simple prefix of Entities that can be dereferenced by this engine. If present only Entities referenced in the Enhancement Metadata matching at least one of the configured prefixes (or one of the configured URI Patterns) will be scheduled for dereferencing.
+* __URI Pattern__ _(enhancer.engines.dereference.uriPatter)_: Allows to configure a regex pattern for matching Entity URIs. If present only Entities matching at lease one of the configured regex patterns ((or one of the configured URI Prefixes) will be scheduled for dereferencing.
+* __Dereference only Content Language Literals__ _(enhancer.engine.dereference.filterContentlanguages)_: If enabled only Literals with the same language as the language detected for the Content will get dereferenced. Literals with no language tag will always get dereferenced.
+* __Dereferenced Fields__ _(enhancer.engines.dereference.fields)_: The dereferenced fields - in RDF terminology 'properties' - to be dereferenced. QNames (e.g. `rdf:label`) can be used for the configuration. This Engine supports the use of FieldMappings for the configuration (see the according sub-section for details).
+* __Dereference LD Path__ _(enhancer.engines.dereference.ldpath)_: The [LD Path Language](http://marmotta.apache.org/ldpath/language.html) allows to define powerful selectors for dereferenced Entities.
+* __Use Shared Thread Pool__ _(enhancer.engines.dereference.entityhub.threads.shared)_: If enabled multiple configured Entityhub Dereference Engines will use a shared Thread Pool. The shared Thread pool is provided by an own Component that can be configured independently (see next sub-section). In most cases it is better to enable this feature and to add additional threads to the shared pool if necessary.
+* __Dereference Threads__ _(enhancer.engines.dereference.entityhub.threads.size)_: If no shared Thread pool is used this allows to configure the size of the thread pool just used by this engine. For values < 1 no Thread Pool will be created and the calling thread will get used to dereference entities.
+
+Additional Supported Properties that are not included in the configuration form:
+
+* __Dereference Languages__ _(enhancer.engines.dereference.languages)_: A set of languages that are dereferenced. Even if _'Dereference only Content Language Literals'_ is active explicitly configured languages will still get dereferenced. If not present and _'Dereference only Content Language Literals'_ is deactivated literals of any language will get dereferenced.
+* __Service Ranking__ _(service.ranking)_: The OSGI service ranking. Will only have an effect if their are two engines with the same name. In such cases the one with the higher service ranking will get called.
+
+
+### Shared Thread Pool Configuration
+
+The Shared Thread Pool is a singelton Component used by all Entityhub Dereference Engines with the _'Use Shared Thread Pool'_ option enabled. It has only a single configuration option _(enhancer.engines.dereference.entityhub.sharedthreadpool.size)_ that allows to set the size of the thread pool.
+
+
+
+### Field Mapping Support
+
+The _enhancer.engines.dereference.fields_ configuration does support the Entityhub Field Mapping language.
+
Modified: stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/list.mdtext
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/list.mdtext?rev=1596858&r1=1596857&r2=1596858&view=diff
==============================================================================
--- stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/list.mdtext (original)
+++ stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/list.mdtext Thu May 22 13:24:28 2014
@@ -162,12 +162,6 @@ This category covers enhancement engines
* accesses a remote service, requires a user account
-* _[KeywordLinkingEngine](keywordlinkingengine):_ __depreacted__ use [EntityhubLinkingEngine](entityhublinking) instead!
- * NLP processing using OpenNLP
- * supports multiple languages
- * detects occurrences of untyped entities as concepts, takes local taxonomies as linking target
-
-
### Sentiment Analyses
This includes Engines that perform word/chunk level sentiment classifications on the [AnalyzedText](../nlp/analyzedtext) content part as well as Engines that summarize those lower level annotations to Sentiments for sentences, sections or the whole text. Sentiment summarizations are represented as 'fise:SentimentAnnotation's (TODO: not yet fully specified (see [STANBOL-760](https://issues.apache.org/jira/browse/STANBOL-760)).
@@ -195,11 +189,23 @@ Enhancement Engines in this category can
* adjusts the fise:confidence of existing fise:EntityAnnotations
+
## Postprocessing / Other
-* _NLP 2 RDF Engine:_ __under development__ (see [STANBOL-741](https://issues.apache.org/jira/browse/STANBOL-741))
- * converts NLP processing results stored in the [AnalyzedText](../nlp/analyzedtext) content part to RDF and adds them to the metadata of the [ContentItem](../contentitem)
- * generated RDF uses the NIF (NLP Interchange Format)
+Post-Processing engines are executed after the Semantic Analysis is done. Typical examples of post-processing tasks are to dereference information about linked entities, re-write enhancements, filter annotations (e.g. based on the confidence ...).
+
+### Dereference Entities
+
+This kind of Enhancement Engines are responsible for retrieving additional information about linked Entities. They first query the enhancement results for referenced Entities, second check if an entity can dereferenced and in an third step dereference the entity and add those information to the enhancement results.
+
+Apache Stanbol provide a core implementation of an [Entity Dereference Engine](dereference) that can be extended for different information sources.
+
+* __[Entityhub Dereference Engine](entityhubdereference)__ allows to dereference Entities available through the [Stanbol Entityhub](/docs/trunk/components/entityhub)
+ * Allows to configure the dereferenced languages and fields
+ * Supports [LD Path](http://marmotta.apache.org/ldpath/language.html)
+ * Uses a thread pool to dereference Entities
+
+### Refactor Engines
* __[TextAnnotation new Model Converter Engine](textannotationnewmodel)__
* This engine converts fise:TextAnnotation to include fise:selection-prefix and fise:selection-suffix properties.
@@ -208,5 +214,22 @@ Enhancement Engines in this category can
* transforms enhancements according to a target ontology, requires KRES launcher.
+### Others
+
+* _NLP 2 RDF Engine:_ __under development__ (see [STANBOL-741](https://issues.apache.org/jira/browse/STANBOL-741))
+ * converts NLP processing results stored in the [AnalyzedText](../nlp/analyzedtext) content part to RDF and adds them to the metadata of the [ContentItem](../contentitem)
+ * generated RDF uses the NIF (NLP Interchange Format)
+
+
+## Deprecated
+
+Enhancement Engines listed below are no longer supported or where replaced by others
+
+* _[KeywordLinkingEngine](keywordlinkingengine):_ __depreacted__ use [EntityhubLinkingEngine](entityhublinking) instead!
+ * NLP processing using OpenNLP
+ * supports multiple languages
+ * detects occurrences of untyped entities as concepts, takes local taxonomies as linking target
+
* _CachingDereferencerEngine_ __deprecated__ (see dereferencing support of individual engines as well as [STANBOL-336](https://issues.apache.org/jira/browse/STANBOL-336))
* retrieves additional content for presenting the enhancement results.
+
Added: stanbol/site/trunk/content/docs/trunk/components/enhancer/enhancementproperties.mdtext
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/enhancer/enhancementproperties.mdtext?rev=1596858&view=auto
==============================================================================
--- stanbol/site/trunk/content/docs/trunk/components/enhancer/enhancementproperties.mdtext (added)
+++ stanbol/site/trunk/content/docs/trunk/components/enhancer/enhancementproperties.mdtext Thu May 22 13:24:28 2014
@@ -0,0 +1,162 @@
+Title: Enhancement Properties
+
+__since version `0.12.1`__
+
+Enhancement Properties allow to parametrize the enhancement process of a [ContentItem](contentitem). In contrast to the configuration of [Enhancement Engines](engines) - that is bound to the component life cycle - enhancement properties can be defined for [Enhancement Chain](chains) and single enhancement requests.
+
+
+Naming and definition
+---------------------
+
+EnhancementProperties are defined as string keys similar to Java properties. To represent them in RDF the key is transformed to an URI by using the string key as local name and `http://stanbol.apache.org/ontology/enhancementproperties#` as namespace. The default namespace prefix for enhancement properties is `ehp`.
+
+The String key of Enhancement Properties MUST start with `enhancer.` and SHOULD use the `enhancer.{level-1}.{level-2}.{property-name}` syntax. Properties are case sensitive and SHOULD only use lower case characters. The '-' char shall be used to make properties with multiple names easier to read.
+
+Globally defined properties use '`enhancer.{property-name}`'. Enhancement Engine specific properties a possible shorted/simplified name of the engine should be used as `{level-1}`. Engine specific properties might also use `engines` as `{level-1}` and the name of the engine as `{level-2}`.
+
+Examples: `enhancer.max-suggestions` or `enhancer.min-confidence` are typical examples for globally defined Enhancement Properties. Properties defined by specific Enhancement Engines will look like `enhancer.entity-co-mention.adjust-existing-confidence` or `enhancer.engines.dereference.fields` (as defined by [STANBOL-1287](https://issues.apache.org/jira/browse/STANBOL-1287).
+
+Enhancement Properties can also be defined as RDF datatype properties. This allows to specify the expected XSD data type of
+expected values.
+
+ @prefix ehp <http://stanbol.apache.org/ontology/enhancementproperties#>
+
+ ehp:enhancer.max-suggestions rdf:type rdfs:DatatypeProperty,
+ xsd:datatype xsd:Integer;
+
+ ehp:enhancer.min-confidence rdf:type rdfs:DatatypeProperty,
+ xsd:datatype xsd:Double;
+
+ ehp:enhancer.entity-co-mention.adjust-existing-confidence rdf:type rdfs:DatatypeProperty,
+ xsd:datatype xsd:Double;
+
+ ehp:enhancer.engines.dereference.fields rdf:type rdfs:DatatypeProperty,
+ xsd:datatype xsd:String;
+
+_NOTE_ that the [Java Interface](#java-interface) will parse enhancement properties as `Map<String,Object>`. Regardless of the defined data type Enhancement Engines that support a property MUST support to parse values from string values (the lexical form of the RDF literal). Multiple values may be parsed as Java Collection or an Array.
+
+h2. Scopes
+
+Enhancement Properties can be defined with the following scopes
+
+1. __request and engine__: Properties with this scope are applied for a single request and a specific [Enhancement Engine](engines) part of the executed [Enhancement Chain](chains). They do have the highest priority and will therefore override any property with the same name defined with an different scope.
+2. __request__: Properties valid for a single request that are parsed to every Enhancement Engine part of the executed Enhancement Chain.
+3. __chain and engine__: Properties defined for a specific [Enhancement Engine](engines) of an [Enhancement Chain](chains). This properties will get applied to all calls of the Enhancement Engine part of the execution of the Chain the property is defined for.
+4. __chain__: Properties parsed to every Enhancement Engine called as part of the execution of the defined [Enhancement Chain](chains). Enhancement Properties of this scope do have the lowest priority and will be overridden by any property with the same key and one of the above scopes.
+
+Properties with a higher priority will override properties with an lower priority. Meaning if a property `enhancer.min-confidence=0.5` is defined on a _chain_ scope it can be overridden by `enhancer.min-confidence=0.75` on a __chain and engine__ scope. A single request might still override the value on a __request__ or __request and engine__ scope.
+
+__Chain__ and/or __chain and engine__ scoped properties are configured with [Enhancement Chain](chains) definition. __Request__ and/or __request and engine__ scoped properties can be specified as query parameter of the POST request or via the Java API by accessing the _Request Properties_ content part. See the following sections for detailed information.
+
+Using Enhancement Properties
+-----------------------------------
+
+Enhancement Properties are consumed by [Enhancement Engine](engine)s. This section describes how implementors of engines can retrieve Enhancement Properties from the request - calls to the `computeEnhancements(..)` method.
+
+In version `0.12.1` and `1.*` EnhancementProperties are contained in the [ContentItem](contentitem) parsed to the EnhancementEngine. The `EnhancementEngineHeloer` utility has methods to access them. The following listing shows the code necessary to get the Enhancement Properties from the parsed ContentItem.
+
+ :::java
+ @Override
+ public final void computeEnhancements(ContentItem ci) throws EngineException {
+ Map<String,Object> enhancemntProps = EnhancementEngineHelper.getEnhancementProperties(this, ci);
+ [..]
+ }
+
+With `2.0.0` the EnhancementEngine API will be changed so that the EnhancementProperties are parsed as an additional parameter.
+
+ :::java
+ @Override
+ public final void computeEnhancements(ContentItem ci,
+ Map<String,Object> enhancemntProps) throws EngineException {
+ [..]
+ }
+
+The `Map<String,Object>` containing the EnhancementProperties is a read/write-able copy of the EnhancementProperties present in the ContentItem. That mean that EnhancementEngine implementations are free to change the contents of that map as those changes will not affect the state of the ContentItem.
+
+The keys of in the map are the string keys of the parsed Enhancement Properties (e.g. `enhancer.max-suggestion` or `enhancer.engines.dereference.fields`). Values can be any Object. Arrays and Collections may be used for multi value properties. The `EnhancementEngineHelper` utility provides methods to convert values.
+
+ :::java
+ //define supported enhancement properties as constants
+ public static final String MAX_SUGGESTIONS = "enhancer.max-suggestions";
+ public static final String DEREFERENCED_FIELDS = "enhancer.engines.dereference.fields";
+
+ [..]
+
+ @Override
+ public final void computeEnhancements(ContentItem ci) throws EngineException {
+ Map<String,Object> enhProp = EnhancementEngineHelper.getEnhancementProperties(this, ci);
+ Integer maxSuggestions = EnhancementEngineHelper.getFirstConfigValue(
+ enhProp.get(MAX_SUGGESTIONS), Integer.class);
+
+ Collection<String> fields = EnhancementEngineHelper.getConfigValues(
+ enhProp.get(DEREFERENCED_FIELDS), String.class);
+ }
+
+_TIP_ the same methods can also be used to process the configuration parsed by OSGI.
+
+Definition Chain scoped Enhancement Properties
+----------------------------------------------
+
+Chain scoped EnhancementProperties are represented by RDF in the ExecutionPlan. As in `0.12.1` and `1.*` the ExecutionPlan is generated by the Enhancement Chain implementations needs to support the configuration.
+
+Starting from `0.12.1` the [ListChain](chains/listchain), [WeightedChain](chains/weightedchain) and [GraphChain](chains/graphchain) allow the configuration of EnhancementProperties:
+
+* __chain and engine__ scoped properties are defined as parameters to the engines with the syntax `{engine-name}; {property-name-1}={value-1},{value-2}; {property-name-2}={value-1};`
+
+* __chain__ scoped properties can be configured by using the osgi property key `stanbol.enhancer.chain.chainproperties` by the syntax `{property-name-1}={value-1},{value-2}`. NOTE that `;` is NOT supported as separator for parsing multiple properties as OSGI configurations already define a way for parsing multiple values
+
+All EnhancementProperties configured with a [Chain](chains) are written as RDF to the
+[ExecutionPlan](chains/executionplan). _Chain_ scoped properties are directly added to the
+`ep:ExecutionPlan` instance while _chain and engine_ scoped properties are added to the
+`ep:ExecutionNode` of the according engine.
+
+With version `2.*` of the enhancer it will be possible to directly parse/refer an ExecutionPlan as RDF graph. This will also allow to manage/configure _chain_ scoped enhancement properties in RDF.
+
+Definition of Request scoped Enhancement Properties
+---------------------------------------------------
+
+_Request_ and _request and engine_ scoped EnhancementProperties are commonly called
+__Request Properties_. They can be parsed as Query Parameter with enhancement requests or
+directly set to the RequestProperties contentPart via the Java API.
+
+### Request Properties encoding
+
+Request properties use the following encoding:
+
+* _request_ scoped enhancement properties are directly parsed by their key
+* _request and engine_ scoped enhancement properties are parsed by using `{engine-name}:{property-name}`
+
+As example the request property `enhancer.max-suggestions=5` would set the maximum number
+of suggestions for all engines to five. In contrast the request property `dbpedia-fst:enhancer.max-suggestions=10` would set the maximum number of suggestions for the DBpedia FST linking engine to ten. If both request properties are parsed the DBpedia FST linking engine would be allowed to suggest ten entities while all the other would give five suggestions at max.
+
+### Parsing Request Properties via the Enhancer RESTful Service
+
+Starting with `0.12.1` Enhancement Properties can be parsed as query parameter of Enhancement Requests. For request scoped properties the property name is used as parameter. Request and engine scoped properties need to use `{engine-name}:{property-name}` as parameter.
+
+The following shows the curl request generating the equivalent of the example used in the above section:
+
+ curl -X POST -H "Accept: text/turtle" -H "Content-type: text/plain" \
+ --data "The Eifeltower is located in Paris."
+ http://localhost:8080/enhancer?enhancer.max-suggestions=5&\
+ dbpedia-linking:enhancer.min-confidence=0.33&\
+ conf-filter:enhancer.min-confidence=0.85
+
+### Request Properties Java API
+
+In version `0.12.1` and `1.*` Request Properties (_request_ and _request and engine_ scoped EnhancementProperties) are stored in the ContentPart with the URI `urn:apache.org:stanbol.enhancer:request.properties`. The ContentItemHelper utility provides methods to retrieve and/or init this content part.
+
+The RequestProperties content part uses a simple `Map<Stirng,Object>`. Keys do use the
+Request Properties encoding. Values can be of all types supported by enhancement properties.
+
+The following code segment provides an example on how to set Request Properties via the
+Java API.
+
+ :::java
+ ContentItem ci; //the content item
+ Map<String,Object> reqProp = ContentItemHelper.initEnhancementPropertiesContentPart(ci)
+ //set min confidence to 0.5 for all engines
+ reqProp.put("enhancer.minConfidence","0.5");
+ //set max suggestions to 10 for the linking engine
+ reqProp.put("linking:enhancer.maxSuggestions","10");
+
+_Note_ with the enhancer `2.0` the request properties content part will get removed and replaced by the EnhancementJob API (TBD).
Modified: stanbol/site/trunk/content/docs/trunk/components/enhancer/enhancerrest.mdtext
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/enhancer/enhancerrest.mdtext?rev=1596858&r1=1596857&r2=1596858&view=diff
==============================================================================
--- stanbol/site/trunk/content/docs/trunk/components/enhancer/enhancerrest.mdtext (original)
+++ stanbol/site/trunk/content/docs/trunk/components/enhancer/enhancerrest.mdtext Thu May 22 13:24:28 2014
@@ -48,6 +48,28 @@ In addition this request is directed to
and people such as Bob Marley." \
"http://localhost:8080/enhancer/chain/dbpedia-keyword?uri=urn:fise-example-content-item&executionmetadata=true"
+### Request Properties Support
+
+__since `0.12.1`__
+
+Request Properties allow to parse request specific [Enhancement Properties](enhancementproperties) as additional query parameters of enhancement requests.
+
+The following shows an curl request that parses two enhancement properties:
+
+ curl -X POST -H "Accept: text/turtle" -H "Content-type: text/plain" \
+ --data "The Eifeltower is located in Paris."
+ http://localhost:8080/enhancer?enhancer.max-suggestions=5&\
+ dbpedia-fst:enhancer.min-confidence=0.85&\
+ dbpedia-dereference:enhancer.engines.dereference.languages=de&\
+ dbpedia-dereference:enhancer.engines.dereference.languages=es
+
+The above request parses two _request and engine_ scoped Enhancement Properties. First the minimum confidence value for suggested entities is set for the `dbpedia-fst` engine to `0.85` and second the `dbpedia-dereference` engine is configured to dereference German and English labels in addition to labels in the language detected for the parsed text. Finally the maximum number of suggestions is aset as a _request_ scoped property to `5`. This means that this property will get parsed to all engines executed in the context of the request.
+
+Request properties use the following encoding:
+
+* _request_ scoped enhancement properties are directly parsed by their key. Keys are required to start with `enhancer.` (e.g. `enhancer.max-suggestions` or `enhancer.min-confidence` as used in the above example)
+* _request and engine_ scoped enhancement properties are parsed by using `{engine-name}:{property-key}` (e.g. `dbpedia-fst:enhancer.min-confidence` or `dbpedia-dereference:enhancer.engines.dereference.languages` as used in the above example.
+
## Enhancer Configuration
The Stanbol Enhancer supports several RESTful services to inspect the configuration. This services allow to retrieve currently active [Enhancement Engines](engines) and [Enhancement Chains](chains).
Modified: stanbol/site/trunk/content/docs/trunk/components/enhancer/index.mdtext
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/enhancer/index.mdtext?rev=1596858&r1=1596857&r2=1596858&view=diff
==============================================================================
--- stanbol/site/trunk/content/docs/trunk/components/enhancer/index.mdtext (original)
+++ stanbol/site/trunk/content/docs/trunk/components/enhancer/index.mdtext Thu May 22 13:24:28 2014
@@ -7,6 +7,7 @@ The Apache Stanbol Enhancer provides bot
* [Java API](#Java_API)
* [Main Interfaces and Utility Classes](#Main_Interfaces_and_Utility_Classes)
* [Enhancement Structure](#Enhancement_Structure)
+ * [Enhancement Properties](#Enhancement_Properties)
* [List of Available Enhancement Engines](#List_of_Engines)
Reader should note that this is the technical documentation of the Stanbol Enhancer intended for Developer. For more practical - usage case oriented - introduction to the Stanbol Enhancer as well as other components please have look at the available [Usage Scenarios](../../scenarios.html).
@@ -111,6 +112,14 @@ The enhancement structure defines three
In addition all annotations created by the Stanbol Enhancer do also provide additional meta information defined by the [Enhancement](enhancementstructure.html#fiseenhancement) class.
+<a name="Enhancement_Properties"></a>
+## Enhancement Properties
+
+__since `0.12.1`__
+
+[Enhancement Properties](enhancementproperties) allow to parametrize the enhancement process of a [ContentItem](contentitem). In contrast to the configuration of [Enhancement Engines](engines) - that is bound to the component life cycle - enhancement properties can be defined for [Enhancement Chain](chains) or parsed with single enhancement requests as Query Parameters.
+
+
<a name="List_of_Engines"></a>
## List of Available Enhancement Engines
Added: stanbol/site/trunk/content/docs/trunk/components/entityhub/entityhub-manatedsite-sesameyard-config.png
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/entityhub/entityhub-manatedsite-sesameyard-config.png?rev=1596858&view=auto
==============================================================================
Binary file - no diff available.
Propchange: stanbol/site/trunk/content/docs/trunk/components/entityhub/entityhub-manatedsite-sesameyard-config.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Modified: stanbol/site/trunk/content/docs/trunk/components/entityhub/managedsite.mdtext
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/entityhub/managedsite.mdtext?rev=1596858&r1=1596857&r2=1596858&view=diff
==============================================================================
--- stanbol/site/trunk/content/docs/trunk/components/entityhub/managedsite.mdtext (original)
+++ stanbol/site/trunk/content/docs/trunk/components/entityhub/managedsite.mdtext Thu May 22 13:24:28 2014
@@ -40,7 +40,7 @@ Currently their is a single implementati
For using a YardSite users need to configure two Services:
-1. Yard: The Entityhub currently includes two different Yard implementations. The SolrYard and the ClerezzaYard. The SolrYard is optimal for the use with the Stanbol Enhancer as it allows very fast label based retrieval of Entities. So if you plan to use the ManagedSite primarily with the Stanbol Enhancer this is definitely the Yard implementation to choose. The ClerezzaYard stores the managed Entities within a TripleStore. While the ClerezzaYard is not as efficient for the use with the StanbolEnhancer its data can be queried by using the SPARQL endpoint of Apache Stanbol.
+1. Yard: The Entityhub currently includes three different Yard implementations. The SolrYard, ClerezzaYard and SesameYard. The SolrYard is optimal for the use with the Stanbol Enhancer as it allows very fast label based retrieval of Entities. So if you plan to use the ManagedSite primarily with the Stanbol Enhancer this is definitely the Yard implementation to choose. The ClerezzaYard and the SesameYard store the managed Entities within a TripleStore. Both are not very efficient for label based lookups as required by the Entity Linking engines of the Stanbol Enhancer. But they are well suited for more data focused use cases as well as for the use with the Entity Dereference Engines.
2. YardSite: This configures the ManagedSite. This configuration links to the configured Yard via its id.
#### Configuration of a SolrYard:
@@ -87,6 +87,25 @@ where 'sparqlQuery.txt' refers to a file
}
}
+#### Configuration of a Sesame Yard Site
+
+With [STANBOL-1169](https://issues.apache.org/jira/browse/STANBOL-1169) (since version `0.12.1`) a Sesame Repository registered as OSGI service can be used as Entityhub Yard.
+
+The following figure shows a Apache Marmotta Kiwi Repository registered as OSGI service.
+
+
+
+The highlighted `org.openrdf.repository.Repository.id` key is used to link a specific Sesame Repository to a Sesame Yard Site. All the other keys are implementation specific and not used by the Entityhub Sesame Yard Site.
+
+When configuring a SesameYard one need to set the Repository (`org.openrdf.repository.Repository.id` key) to the value of the Sesame Repository one would like to use as backend. This is especially important if multiple Sesame Repositories are registered as OSGI services.
+
+The following figure shows the configuration dialog for a Sesame Yard. Again the id of the Sesame Repository is highlighted.
+
+
+
+The Context URIs (`org.apache.stanbol.entityhub.yard.sesame.contextUri` key) can be used to configure specific Named Graphs used to read/write RDF triples to/from. An empty value is interpreted as the `null` context. For using the union graph one needs to deactivate the
+Enable Contexts (`org.apache.stanbol.entityhub.yard.sesame.enableContext` key) option. In this case all configured Context URIs will get ignored.
+
#### Configuration of the YardSite
Finally you need to configure the YardSite that uses the previously configured Yard instance (either SolrYard or ClerezzaYard). Again this will show how to configure the YardSite by using the Configuration tab of the Apache Felix Webconsole [http://{stanbol-instance}/system/console/configMgr](http://localhost:8080/system/console/configMgr).
Added: stanbol/site/trunk/content/docs/trunk/components/entityhub/marmotta-kiwi-repository-service.png
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/entityhub/marmotta-kiwi-repository-service.png?rev=1596858&view=auto
==============================================================================
Binary file - no diff available.
Propchange: stanbol/site/trunk/content/docs/trunk/components/entityhub/marmotta-kiwi-repository-service.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Modified: stanbol/site/trunk/content/docs/trunk/index.mdtext
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/index.mdtext?rev=1596858&r1=1596857&r2=1596858&view=diff
==============================================================================
--- stanbol/site/trunk/content/docs/trunk/index.mdtext (original)
+++ stanbol/site/trunk/content/docs/trunk/index.mdtext Thu May 22 13:24:28 2014
@@ -22,6 +22,7 @@ This documentation of Apache Stanbol tar
* [Tools and Utilities](utils/)
* [Enhancer Stress Test Tool](utils/enhancerstresstest) - allows to send concurrent requests to the Stanbol Enhancer for Performace/Stress testing of the Stanbol Enhancer, Chains and Enhancement Engines.
* [Commons Solr](utils/commons-solr) - as [Apache Solr](http://lucene.apache.org/solr/) is used by several Apache Stanbol components Apache Stanbol provides utilities that ease the use of Solr within OSGi; support the initialization of SolrCores with predefined configurations and data; allows to publish the Solr RESTful API on the Stanbol Web UI.
+ * [Marmotta KiWi Repository Service](utils/marmotta-kiwi-repository-service) - allows to use the [Apache Marmotta](http://marmotta.apache.org) [KiWi Triplestore](http://marmotta.apache.org/kiwi) within an OSGI environment.
* [Data File Provider](utils/datafileprovider) - infrastructure used by Apache Stanbol to load data file such as statistical models of NLP frameworks. archives of Knowledge Bases ...
* Demos
* Online demos of the basic/stable features of Apache Stanbol are available [here](http://stanbol.demo.nuxeo.com/) and [here](http://dev.iks-project.eu:8080/). An experimental/full version of Apache Stanbol is available [here](http://dev.iks-project.eu:8081/).
Modified: stanbol/site/trunk/content/docs/trunk/utils/index.mdtext
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/utils/index.mdtext?rev=1596858&r1=1596857&r2=1596858&view=diff
==============================================================================
--- stanbol/site/trunk/content/docs/trunk/utils/index.mdtext (original)
+++ stanbol/site/trunk/content/docs/trunk/utils/index.mdtext Thu May 22 13:24:28 2014
@@ -1,10 +1,14 @@
Title: Apache Stanbol Tools and Utilities
* [Enhancer Stress Test Tool](enhancerstresstest) - allows to send
-[concurrent requests to the Stanbol Enhancer for Performace/Stress testing of
-[the Stanbol Enhancer, Chains and Enhancement Engines.
-* [Commons Solr](commons-solr) - as Apache
-[Solr](http://lucene.apache.org/solr/) is used by several Apache Stanbol
-[components Apache Stanbol provides utilities that ease the use of Solr within
-[OSGi; support the initialization of SolrCores with predefined configurations
-[and data; allows to publish the Solr RESTful API on the Stanbol Web UI.
+concurrent requests to the Stanbol Enhancer for Performace/Stress testing of
+the Stanbol Enhancer, Chains and Enhancement Engines.
+* [Commons Solr](commons-solr) - as Apache [Solr](http://lucene.apache.org/solr/) is used
+by several Apache Stanbol components this module provides utilities that ease the use of
+Solr within OSGi. support the initialization of SolrCores with predefined configurations
+and data; allows to publish the Solr RESTful API on the Stanbol Web UI.
+* [Marmotta KiWi Repository Service](marmotta-kiwi-repository-service) - allows to use the
+[Apache Marmotta](http://marmotta.apache.org) [KiWi Triplestore](http://marmotta.apache.org/kiwi)
+within an OSGI environment.
+* [Data File Provider](datafileprovider) - infrastructure used by Apache Stanbol to load
+data file such as statistical models of NLP frameworks. archives of Knowledge Bases ...
Added: stanbol/site/trunk/content/docs/trunk/utils/marmotta-kiwi-repository-service-config.png
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/utils/marmotta-kiwi-repository-service-config.png?rev=1596858&view=auto
==============================================================================
Binary file - no diff available.
Propchange: stanbol/site/trunk/content/docs/trunk/utils/marmotta-kiwi-repository-service-config.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: stanbol/site/trunk/content/docs/trunk/utils/marmotta-kiwi-repository-service.mdtext
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/utils/marmotta-kiwi-repository-service.mdtext?rev=1596858&view=auto
==============================================================================
--- stanbol/site/trunk/content/docs/trunk/utils/marmotta-kiwi-repository-service.mdtext (added)
+++ stanbol/site/trunk/content/docs/trunk/utils/marmotta-kiwi-repository-service.mdtext Thu May 22 13:24:28 2014
@@ -0,0 +1,58 @@
+Marmotta Kiwi Repository Service
+================================
+
+The `org.apache.stanbol.commons.marmotta.kiwi` module allows to use the
+[Marmota KiWi TripleStore](http://marmotta.apache.org/kiwi/) within an OSGI environment.
+
+Configuration
+-------------
+
+The configuration of a [KiWi TripleStore](http://marmotta.apache.org/kiwi/) is done by an
+OSGI Component that configures the Kiwi Configuration based on the provided OSGI
+configuration. The following figure shows the configuration dialog.
+
+
+
+* `org.openrdf.repository.Repository.id`: The id of the Repository. Intended to be used by
+other components to track a specific repository instance.
+* `marmotta.kiwi.dialect`: The KiWi Database dialect. Currently Marmotta supports the
+H2Dialect, PostgreSQLDialect and MySQLDialect. Note that the selected dialect will select
+different database driver. If those are not available the activation will throw an
+exception. PostgreSQL driver are embedded. H2 drivers are included in the default
+Bundlelist used by Stanbol.
+* `marmotta.kiwi.dburl`: This property can be used to directly configure the DB URL. If
+present this is preferred over the configuration of the `marmotta.kiwi.host`,
+`marmotta.kiwi.port`, `marmotta.kiwi.database` and `marmotta.kiwi.options` parameters.
+* `marmotta.kiwi.user` and `marmotta.kiwi.password` for the database
+* `marmotta.kiwi.cluster`: defines the name of the cluster. Different KiWi Repositories
+might use clusters with different names. If not present or empty clustering will be
+deactivated.
+* `marmotta.kiwi.cluster.address`: The multicast IP Address used for the cluster. Ignored
+if clustering is deactivated.
+* `marmotta.kiwi.cluster.port`: The port used for the cluster. Ignored if clustering is
+deactivated. A value <= 0 will use the default port.
+* `marmotta.kiwi.cluster.cachemode`: The Mode of the Cache. Supported values include
+`LOCAL`, `REPLICATED`, `DISTRIBUTED`
+* `marmotta.kiwi.cluster.cachingbackend`: The Caching Backend used by the KiWi Triplestore.
+Supported are `GUAVA` (local only), `EHCACHE` (local only), `HAZELCAST` (distributed only),
+`INFINISPAN_CLUSTERED`(distributed or replicated) and `INFINISPAN_HOTROD` (client/server).
+Please note the Marmotta KiWi documentation for details. Caching backend implementations
+do have additional dependencies. `GUAVA` and `HAZELCAST` are included in Stanbol. For the
+other please have a look for commented dependencies in the
+[Marmotta Kiwi Bundlelist](http://svn.apache.org/repos/asf/stanbol/branches/release-0.12/launchers/bundlelists/marmotta/kiwi/src/main/bundles/list.xml)
+
+Repository Service
+------------------
+
+Providing a OSGI configuration for the `KiWiRepositoryService` component will cause a
+KiWi Repository to be configured during activation. This Sesame Repository will get
+registered as OSGI service with the parameters as shown in the following figure.
+
+
+
+The marked `org.openrdf.repository.Repository.id` property is of special interest as it
+can be used to track for a Sesame Repository with a specific name. As an Example the
+Repository with the name `dummy` can be tracked with the Filter
+`(&(objectClass=org.openrdf.repository.Repository)(org.openrdf.repository.Repository.id=dummy))`
+
+
Added: stanbol/site/trunk/content/docs/trunk/utils/marmotta-kiwi-repository-service.png
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/utils/marmotta-kiwi-repository-service.png?rev=1596858&view=auto
==============================================================================
Binary file - no diff available.
Propchange: stanbol/site/trunk/content/docs/trunk/utils/marmotta-kiwi-repository-service.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream