You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@stanbol.apache.org by rw...@apache.org on 2012/11/23 14:21:33 UTC
svn commit: r1412877 - in
/stanbol/site/trunk/content/docs/trunk/components/enhancer:
engines/entitylinking.mdtext nlp/analyzedtext.mdtext
nlp/nlpannotations.mdtext
Author: rwesten
Date: Fri Nov 23 13:21:32 2012
New Revision: 1412877
URL: http://svn.apache.org/viewvc?rev=1412877&view=rev
Log:
STANBOL-733 - minor formatting related changes
Modified:
stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entitylinking.mdtext
stanbol/site/trunk/content/docs/trunk/components/enhancer/nlp/analyzedtext.mdtext
stanbol/site/trunk/content/docs/trunk/components/enhancer/nlp/nlpannotations.mdtext
Modified: stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entitylinking.mdtext
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entitylinking.mdtext?rev=1412877&r1=1412876&r2=1412877&view=diff
==============================================================================
--- stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entitylinking.mdtext (original)
+++ stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entitylinking.mdtext Fri Nov 23 13:21:32 2012
@@ -102,7 +102,7 @@ When activating "Proper Noun Linking" us
1. the POS tagging for given languages do support _Pos#ProperNoun_. If this is not the case for some languages than language specific configurations need to be used to manually adjust configurations for such languages. The next section provides examples for that.
2. the Entities in the Vocabulary linked against need typically be mentioned as Proper Nouns in the Text. Users that need to link Vocabularies with Entities that use common nouns as their labels (e.g. House, Mountain, Summer, ...) can typically not use "Proper Noun Linking" with the following exceptions:
- * Entities with labels comprised of multiple common nouns (e.g. White House) can be detected in cases where _Chunk_s are supported and the _Link Multiple Matchable Tokens in Phrases_ option is enabled (see the next sub-section for details).
+ * Entities with labels comprised of multiple common nouns (e.g. White House) can be detected in cases where _Chunks_ are supported and the _Link Multiple Matchable Tokens in Phrases_ option is enabled (see the next sub-section for details).
* In case Entities mentioned in the text are written as upper case tokens that the _Upper Case Token Mode_ can be set to "LINK" (see the next sub-section for details)
If suitable it is strongly recommended to activate "Proper Noun Linking" as it highly increases the performance because in typical text only around 1/10 of the Nouns are marked as Proper Nouns and therefore the amount of vocabulary lookups also decreases by this amount.
@@ -196,7 +196,7 @@ The following properties define how Link
* __Default Matching Language__ _(enhancer.engines.linking.defaultMatchingLanguage)_: Linking is always done in the language of the processed text and in the _Default Matching Language_. By default the default language are labels without an language tag, but this parameter allows to override this to a specific language. This is e.g. useful for [DBpedia](http://dbpedia.org) where all labels are marked with the language of the source Wikipedia data. So it makes sense to configure the default matching language to this value.
* __Max Search Token Distance__ _(enhancer.engines.linking.maxSearchTokenDistance)_: The maximum number of Tokens searched around a linked token to search for additional matchable tokens to be included for searches for Entities. The default value is '3'. As an Example in the text section "at the University of Munich a new procedure to" only "Munich" would be marked as linkable token if _Proper Noun Linking_ is activated. However for searching Entities it makes sense to also use the matchable term 'University', because otherwise a search would potentially return an huge number of candidates of Entities mentioning 'Munich' in their labels. This parameter allows to configure the maximum distance of tokens so that the EntityLinkingEngine may include them as additional optional constraints for queries via the EntitySearcher interface. _NOTE_ that this parameter will not allow to include tokens outside of a _processable chunk_ if the _linked token_ is within an such.
* __Max Search Tokens__ _(enhancer.engines.linking.maxSearchTokens)_: The maximum number of Tokens used for searches via the _EntitySearcher_ interface. The default value is '2'. In case more _matchable tokens_ are within the configured _Max Search Token Distance_ than those closer & trailing the _linkable token_ are preferred. E.g. the text "president Barack Obama" where 'Barack' is the currently active _linkable token_ will result in a query with the tokens 'Barack' OR 'Obama' if _Max Search Tokens_=2 and _Max Search Token Distance_>=1 because both 'president' and 'Obama' do have a distance of 1 but trailing Tokens are preferred.
-* __Lemma based Matching__ _(enhancer.engines.linking.lemmaMatching)_: If this feature in enabled than the _MorphoFeatures#getLemma()_ values are used instead of the _Token#getSpan()_s if present.
+* __Lemma based Matching__ _(enhancer.engines.linking.lemmaMatching)_: If this feature in enabled than the _MorphoFeatures#getLemma()_ values are used instead of the _Token#getSpan()s_ if present.
* __Min Search Token Length__ _(enhancer.engines.linking.minSearchTokenLength)_: This is used as fallback if the _Tokens_ in the _[AnalyzedText](../nlp/analyzedtext)_ do not contain Part of Speech annotations or if the confidence of those annotations is to low. The default value is '3' meaning that in such cases all tokens with more than '3' characters are linked with the vocabulary. _NOTE_ that this configuration might move to the _Text Processing Configuration_ in future versions.
The parameters below are used to configure the matching process.
@@ -263,7 +263,7 @@ This method is called with the 'id' of a
* __Entity Search__ _lookup(String field, Set<String> includeFields, List<String> search, String[] languages,Integer limit)::Collection<Representation>_
-This method is used for searching entities in the controlled vocabulary. The configured _Label Field_ is parsed in the 'field' parameter. The 'includedFileds' contain all fields required for the linking process. _Representation_s returned as result need to include values for those fields. The 'search' parameter includes the tokens used for the search. Values should be considered optional however Results are considered to rank Entities that match more search tokens first. The array of 'languages' is used to parse the languages that need to be considered for the search. If 'languages' contains NULL or '' it means that also labels without an language tag need to be included in the search (NOTE that this DOES NOT mean to include labels of any language!). Finally the 'limit' parameter is used to specify the maximum number of results. If NULL than the implementation can choose an meaningful default.
+This method is used for searching entities in the controlled vocabulary. The configured _Label Field_ is parsed in the 'field' parameter. The 'includedFileds' contain all fields required for the linking process. _Representations_ returned as result need to include values for those fields. The 'search' parameter includes the tokens used for the search. Values should be considered optional however Results are considered to rank Entities that match more search tokens first. The array of 'languages' is used to parse the languages that need to be considered for the search. If 'languages' contains NULL or '' it means that also labels without an language tag need to be included in the search (NOTE that this DOES NOT mean to include labels of any language!). Finally the 'limit' parameter is used to specify the maximum number of results. If NULL than the implementation can choose an meaningful default.
* __Offline Mode__ _supportsOfflineMode()::boolean_ : indicates if the EntitySearcher implementation needs to connect an remote service. This is needed to deactivate the EntityLinkingEngine in cases where Apache Stanbol is started in OfflineMode
* __Serach Result Limit__ _getLimit()::Integer_ : The maximum number of search results supported by the EntitySearcher implementation. Can return NULL if not applicable or unknown.
@@ -281,9 +281,9 @@ The _LabelTokenizer_ interface defines o
#### MainLabelTokenizer
-As it might very likely be the case that users will want to use multiple LabelTokenizer for different languages the EntityLinkingEngine comes with an MainLabelTokenizer implementation. It registers itself as LabelTokenizer with highest possible OSGI 'service.ranking' and tracks all other registered _LabelTokenizer_s.
+As it might very likely be the case that users will want to use multiple LabelTokenizer for different languages the EntityLinkingEngine comes with an MainLabelTokenizer implementation. It registers itself as LabelTokenizer with highest possible OSGI 'service.ranking' and tracks all other registered _LabelTokenizers_.
-So if custom _LabelTokenizer_s register themselves as OSGI service than the MainLabelTokenizer can forward requests to them. It will do so in the order of the '<code>service.ranking</code>'s. in addition _LabelTokenizer_ can use the '<code>enhancer.engines.keywordextraction.labeltokenizer.languages</code>' property to formally specify the languages they are supporting. This property does use the language configuration syntax (e.g. "en,de" would include English and German; "!it,!fr,*" would specify all languages expect Italian and French). If no configuration is provided than "*" (all languages) is assumed - what is fine as default as long as _LabelTokenizer_ correctly return NULL for languages they do not support.
+So if custom _LabelTokenizers_ register themselves as OSGI service than the MainLabelTokenizer can forward requests to them. It will do so in the order of the '<code>service.ranking</code>'s. in addition _LabelTokenizer_ can use the '<code>enhancer.engines.keywordextraction.labeltokenizer.languages</code>' property to formally specify the languages they are supporting. This property does use the language configuration syntax (e.g. "en,de" would include English and German; "!it,!fr,*" would specify all languages expect Italian and French). If no configuration is provided than "*" (all languages) is assumed - what is fine as default as long as _LabelTokenizer_ correctly return NULL for languages they do not support.
The MainLabelTokenizer forwards tokenize requests to all available LabelTokenizer implementations that support a specific language sorted by their '<code>service.ranking</code>' until the first one does NOT return NULL. If no LabelTokenizer was found or all returned NULL it will also return NULL.
@@ -306,7 +306,7 @@ This will inject the MainLabelTokenizer
textProcessingConfig, linkerConfig, //config
labelTokenizer); //the MainLabelTokenizer
-Configuring the NamedEntityLinkingEngine like this ensures that all registered _LabelTokenizer_s are considered for tokenizing.
+Configuring the NamedEntityLinkingEngine like this ensures that all registered _LabelTokenizers_ are considered for tokenizing.s_
#### OpenNLP LabelTokenizer
Modified: stanbol/site/trunk/content/docs/trunk/components/enhancer/nlp/analyzedtext.mdtext
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/enhancer/nlp/analyzedtext.mdtext?rev=1412877&r1=1412876&r2=1412877&view=diff
==============================================================================
--- stanbol/site/trunk/content/docs/trunk/components/enhancer/nlp/analyzedtext.mdtext (original)
+++ stanbol/site/trunk/content/docs/trunk/components/enhancer/nlp/analyzedtext.mdtext Fri Nov 23 13:21:32 2012
@@ -67,7 +67,7 @@ This order is used by all Iterators retu
### Concurrent Modifications and Iterators
-Iterators returned by the AnalyzedText API MUST throw _ConcurrentModificationException_s but rather reflect changes to the underlaying model. While this is not constant with the default behavior of Iterators in Java this is central for the effective usage of the AnalyzedText API - e.g. when Iterating over Sentences while adding Tokens.
+Iterators returned by the AnalyzedText API MUST throw _ConcurrentModificationExceptions_ but rather reflect changes to the underlaying model. While this is not constant with the default behavior of Iterators in Java this is central for the effective usage of the AnalyzedText API - e.g. when Iterating over Sentences while adding Tokens.
### Code Samples:
@@ -131,7 +131,7 @@ The following example shows the intended
...
}
-2. Defined _Annotation_ are used to add information to an _Annotated_ instance (like a Span). For adding annotations the use of _Annotation_s is required to ensure type safety. The following code snippet shows how to add an PosTag with the probability 0.95.
+2. Defined _Annotation_ are used to add information to an _Annotated_ instance (like a Span). For adding annotations the use of _Annotations_ is required to ensure type safety. The following code snippet shows how to add an PosTag with the probability 0.95.
:::java
PosTag tag = new PosTag("N"); //a simple POS tag
Modified: stanbol/site/trunk/content/docs/trunk/components/enhancer/nlp/nlpannotations.mdtext
URL: http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/enhancer/nlp/nlpannotations.mdtext?rev=1412877&r1=1412876&r2=1412877&view=diff
==============================================================================
--- stanbol/site/trunk/content/docs/trunk/components/enhancer/nlp/nlpannotations.mdtext (original)
+++ stanbol/site/trunk/content/docs/trunk/components/enhancer/nlp/nlpannotations.mdtext Fri Nov 23 13:21:32 2012
@@ -39,7 +39,7 @@ The first parameter is the String POS ta
_TagSet_ is the other important class as it allows to manage the set of PosTag instances. _TagSet_ has two main functions: First it allows an integrator of an POS tagger with Stanbol to define the mappings from the string POS tags used by the Pos Tagger to the LexicalCategory and Pos enumeration members as preferable used by the Stanbol NLP chain. Second it ensures that there is only a single instance of PosTag used to annotate all Tokens with the same type.
-_TagSet_s are typically specified as static members of utility classes. The following code snippet shows an example
+TagSets are typically specified as static members of utility classes. The following code snippet shows an example
:::java
//Tagset is generically typed. We need a TagSet for PosTag's
@@ -56,10 +56,10 @@ _TagSet_s are typically specified as sta
STTS.addTag(new PosTag("ADJA", Pos.AttributiveAdjective));
STTS.addTag(new PosTag("ADJD", Pos.PredicativeAdjective));
STTS.addTag(new PosTag("ADV", LexicalCategory.Adverb));
-//[...]
+ //[...]
}
-The string tag (first parameter) of the _PosTag_ is used as unique key by the _TagSet_. Adding an 2nd _PasTag_ with the same tag will override the first one. _PosTag_s that are added to a _TagSet_ have the _Tag#getAnnotationModel()_ property set to that model.
+The string tag (first parameter) of the _PosTag_ is used as unique key by the _TagSet_. Adding an 2nd _PasTag_ with the same tag will override the first one. _PosTags_ that are added to a _TagSet_ have the _Tag#getAnnotationModel()_ property set to that model.
The final example shows a code snippet shows the core part of an POS tagging engine using the both the [AnalyzedText](analyzedtext) and the _PosTag_ and _TagSet_ APIs.