You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stanbol.apache.org by Pablo Mendes <pa...@gmail.com> on 2012/06/01 17:37:23 UTC
DBpedia Spotlight and Stanbol
*Hi all,*
*Iavor [1] and myself [2] have been working on integrating DBpedia
Spotlight functionality into Stanbol as part of the EAP [3]. **This message
is to give you a status update.*
*In DBpedia Spotlight content enhancement goes through Spotting and
Disambiguation. First, spotting recognizes phrases that should be annotated
in the input text. If all you care about are named entities, you can use a
named entity recognizer (NER) in this step. DBpedia Spotlight includes NER,
Keyphrase Extraction, and other spotting techniques that can find phrases
referring to all 3.5M entities in DBpedia. Second, disambiguation (a.k.a.
Entity Linking) attempts to map each recognized phrase to its correct
identifier in DBpedia. The entire content enhancement process is
implemented in DBpedia Spotlight via the /annotate REST endpoint.
We have developed a new enhancement engine for Stanbol using /annotate
endpoint in DBpedia Spotlight. The current version is currently being
tested and improved, and can be accessed at:
http://spotlight.dbpedia.org/stanbol/<http://spotlight.dbpedia.org/stanbol/system/console>
enhancer <http://88.198.155.99:9090/enhancer>
Configuration:
http://spotlight.dbpedia.org/stanbol/system/console
The account data for the felix configuration is the default one. The Engine
can be configured there as well. For instance, the option onlyNER (boolean
value, inspired by the OpenCalais plug-in) runs only the phrase
recognition (spotting) stage and creates TextAnnotations with the results,
which can be used in a chain context by other engines (e.g. DBpedia Linking)
Our next step is to create an enhancement chain with two enhancement
engines: DBpedia Spotlight Spotting and DBpedia Spotlight Disambiguation.
We have performed preliminary evaluations of the new enhancement engine
using the Stanbol Benchmark Component (SBC). The SBC allows evaluating
content enhancement engines based on examples of desired and undesired
behavior defined through Benchmark Definition Language (BDL) statements. We
have transformed the dataset from Kulkarni et al. 2009 [4] into BDL. The
BDL data set is available from:
http://spotlight.dbpedia.org/download/stanbol/
The SBC is a nice way to perform manual inspection of the behavior of the
enhancement chain for different examples in the evaluation dataset.
However, for evaluations with several hundreds of examples, it would be
interesting to have scores that summarize the performance for the entire
dataset. We are in the process of conducting large scale experiments with
existing datasets, aiming at producing precision and recall figures for
different enhancement chains.*
Any comments are very welcome.
Cheers,
Pablo
[1] http://www.iks-project.eu/community/people/pablo-n-mendes
[2] http://www.iks-project.eu/community/people/iavor-jelev
[3] http://wiki.iks-project.eu/index.php/GzEvD_Proposal
[4] http://www.cse.iitb.ac.in/soumen/doc/QCQ/
DBpedia Spotlight and Stanbol
Posted by Iavor Jelev <ia...@babelmonkeys.com>.
Hi all,
on 01.06. Pablo and I gave you an update of the DBpedia Spotlight
integration in Stanbol (I replied to it with this mail, so you can
review it again if you wis). We had implemented an EnhancementEngine,
which did the entire annotation process (spotting, candidates selection
and disambiguation), to demo the functionality DBpedia Spotlight would
bring to Stanbol. In the last two weeks, apart from running evaluations,
we created 3 further Enhancement Engines, which allow the combination of
DBpedia Spotlight functionality with other stanbol engines in a chain
context, as follows:
URL abbreviations:
engineURL: http://spotlight.dbpedia.org/stanbol/enhancer/engine/
chainURL: http://spotlight.dbpedia.org/stanbol/enhancer/chain/
engineUrl/dbpspotlightannotate
---------------------
This was the EnhancementEngine we were reffering to above. The only
thing that changed here is, that the parameter "onlyNER", which could be
used to do only spotting, was removed, as we extracted spotting in a
separate engine. This engine performs the entire annotation process,
from natural language text to DBpedia URIs.
engineUrl/dbpspotlightspot
---------------------
This engine does only NER, and stores the results as TextAnnotations,
which can be input to a stanbol engine which does linking. It also
performs other kinds of phrase recognition besides NER [1].
engineUrl/dbpspotlightdisambiguate
---------------------
This engine reads TextAnnotations, stored by a stanbol NER engine, and
does candidate selection, disambiguation and linking.
engineUrl/dbpspotlightcandidates
---------------------
This engine equivalent to annotate, except all possible disambiguations
for a TextAnnotation are returned, and not only the best disambiguation
as in /dbpspotlightannotate
Apart from the EnhancementEngines, the following EnhancementChains were
defined:
chainURL/dbpspotlight
---------------------
This chain replicates the functionality of /dbpspotlightannotate, by
chaining /dbpspotlightspot and /dbpspotlightdisambiguate. Note: langid
is run first, and only english texts are processed
chainURL/dbpspotlightonlyspot
---------------------
Demonstrates the use of /dbpspotlightspot with a diferent linker, in
this case /dbpediaLinking
chainURL/dbpspotlightonlydisambiguate
---------------------
Demonstrates the use of /dbpspotlightdisambiguate with a diferent NER
engine, in this case /ner
We would greatly appreciate feedback and improvement suggestions.
[1] https://github.com/dbpedia-spotlight/dbpedia-spotlight/wiki/Spotting
kind regards,
Iavor
Am 01.06.2012 17:37, schrieb Pablo Mendes:
> *Hi all,*
> *Iavor [1] and myself [2] have been working on integrating DBpedia
> Spotlight functionality into Stanbol as part of the EAP [3]. **This message
> is to give you a status update.*
>
> *In DBpedia Spotlight content enhancement goes through Spotting and
> Disambiguation. First, spotting recognizes phrases that should be annotated
> in the input text. If all you care about are named entities, you can use a
> named entity recognizer (NER) in this step. DBpedia Spotlight includes NER,
> Keyphrase Extraction, and other spotting techniques that can find phrases
> referring to all 3.5M entities in DBpedia. Second, disambiguation (a.k.a.
> Entity Linking) attempts to map each recognized phrase to its correct
> identifier in DBpedia. The entire content enhancement process is
> implemented in DBpedia Spotlight via the /annotate REST endpoint.
>
> We have developed a new enhancement engine for Stanbol using /annotate
> endpoint in DBpedia Spotlight. The current version is currently being
> tested and improved, and can be accessed at:
> http://spotlight.dbpedia.org/stanbol/<http://spotlight.dbpedia.org/stanbol/system/console>
> enhancer <http://88.198.155.99:9090/enhancer>
> Configuration:
> http://spotlight.dbpedia.org/stanbol/system/console
> The account data for the felix configuration is the default one. The Engine
> can be configured there as well. For instance, the option onlyNER (boolean
> value, inspired by the OpenCalais plug-in) runs only the phrase
> recognition (spotting) stage and creates TextAnnotations with the results,
> which can be used in a chain context by other engines (e.g. DBpedia Linking)
>
> Our next step is to create an enhancement chain with two enhancement
> engines: DBpedia Spotlight Spotting and DBpedia Spotlight Disambiguation.
>
> We have performed preliminary evaluations of the new enhancement engine
> using the Stanbol Benchmark Component (SBC). The SBC allows evaluating
> content enhancement engines based on examples of desired and undesired
> behavior defined through Benchmark Definition Language (BDL) statements. We
> have transformed the dataset from Kulkarni et al. 2009 [4] into BDL. The
> BDL data set is available from:
> http://spotlight.dbpedia.org/download/stanbol/
>
> The SBC is a nice way to perform manual inspection of the behavior of the
> enhancement chain for different examples in the evaluation dataset.
> However, for evaluations with several hundreds of examples, it would be
> interesting to have scores that summarize the performance for the entire
> dataset. We are in the process of conducting large scale experiments with
> existing datasets, aiming at producing precision and recall figures for
> different enhancement chains.*
>
> Any comments are very welcome.
>
> Cheers,
> Pablo
>
> [1] http://www.iks-project.eu/community/people/pablo-n-mendes
> [2] http://www.iks-project.eu/community/people/iavor-jelev
> [3] http://wiki.iks-project.eu/index.php/GzEvD_Proposal
> [4] http://www.cse.iitb.ac.in/soumen/doc/QCQ/
>
Re: DBpedia Spotlight and Stanbol
Posted by Pablo Mendes <pa...@gmail.com>.
Hi Rupert,
Thanks for the info. I will talk to Iavor about your suggestions to the
engines.
> BTW: do you plan to contribute your Enhancement Engines to Stanbol, or
> would you rather package/promote it via Spotlight?
We'd like to contribute to Stanbol, so that it is easier for users to just
use them directly from Stanbol. But we will also document them on our
website and promote via our channels.
Cheers,
Pablo
On Mon, Jun 4, 2012 at 1:10 PM, Rupert Westenthaler <
rupert.westenthaler@gmail.com> wrote:
> On Mon, Jun 4, 2012 at 12:51 PM, Pablo Mendes <pa...@gmail.com>
> wrote:
> > Hi Rupert,
> >
> > 1. Do you think it might make sense to allow multiple EngineInstances
> >> using different Spotting algorithms?
> >>
> >
> > You mean that instead of a "/spot" you would have a /lingpipespot, /ner
> and
> > /keyphrases? Can be done, but do you have a use case for that? Could this
> > also be done as some sort of URL rewrite? From
> > /spot?spotter=LingPipeSpotter to /lingpipespotter
>
> No much simpler. Just to use the
>
> @Component(configurationFactory = true, [..])
>
> to allow users to configure multiple instances the DBpedia Spotlight
> Spotting engine. Than he can configure different spotting algorithms
> for the different instances.
>
> >
> >> Our next step is to create an enhancement chain with two enhancement
> >> > engines: DBpedia Spotlight Spotting and DBpedia Spotlight
> Disambiguation.
> >>
> >> So basically to split this engine in to separate one, right?
> >
> >
> > Correct. Split this engine into two, and have a chain connecting the
> > engines.
>
> Note also that you can configure the Engines and the chain by using the
>
> <Install-Path>{path}</Install-Path>
>
> bundle extension.
>
> The OSGI service configuration need than to be located at
>
> {module}/src/main/resources/{path}
>
> If you add this to the module of the engines, than users would
> automatically get the default configuration as soon as they add the
> engines to Stanbol.
> As example you can use the default configuration module of the Enhancer [1]
>
> [1]
> http://svn.apache.org/repos/asf/incubator/stanbol/trunk/enhancer/defaults/pom.xml
>
> >
> > Had not yet time to look at the examples in detail, but if the license
> >> if [4] allows and you agree we could think about to make them
> >> available as part of the Stanbol Enhancer.
> >
> >
> > I could not find licensing information. I can ask the authors directly.
> >
> >
> >> This is completely true. Can you start an Jira Issue about that. I
> >> will definitely help with implementing this.
> >
> >
> > Sure. Here: https://issues.apache.org/jira/browse/STANBOL-652
> > I have already implemented quite a bit of evaluation code. If you want,
> you
> > can use it as a starting point. My approach is to go over the dataset and
> > write out a log of each annotation attempt. Based on this log, I run a
> > series of R scripts to interpret the evaluation results.
> >
>
> +1
>
> BTW: do you plan to contribute your Enhancement Engines to Stanbol, or
> would you rather package/promote it via Spotlight?
>
> best
> Rupert
>
> > Cheers,
> > Pablo
> >
> >
> > On Mon, Jun 4, 2012 at 11:45 AM, Rupert Westenthaler <
> > rupert.westenthaler@gmail.com> wrote:
> >
> >> Hi Pablo
> >>
> >> I made som tests and the spotting looks great. Also tried some some of
> >> the different Spotting algorithms (NER, LingPipeSpotter (very slow)
> >> and Kea).
> >>
> >> Here are some Questions/Suggestions related to the engine.
> >>
> >> 1. Do you think it might make sense to allow multiple EngineInstances
> >> using different Spotting algorithms?
> >>
> >> 2. I noticed that created TextAnnotations do not have "dc-terms:type"
> >> information. This property is used to represent the "nature" (e.g.
> >> Person, Organisation, Place in case of Named Entities) by the the
> >> Stanbol Enhancement Structure. So if such information are available it
> >> would be great to set it.
> >>
> >> 3. I would suggest to add support for the type suggestion filter
> >> feature as shown in the 2nd example of the user manuel [1]
> >>
> >> [1] http://wiki.dbpedia.org/spotlight/usersmanual#h139-10
> >>
> >> On Fri, Jun 1, 2012 at 5:37 PM, Pablo Mendes <pa...@gmail.com>
> >> wrote:
> >> > Our next step is to create an enhancement chain with two enhancement
> >> > engines: DBpedia Spotlight Spotting and DBpedia Spotlight
> Disambiguation.
> >>
> >> So basically to split this engine in to separate one, right?
> >>
> >> > We have performed preliminary evaluations of the new enhancement
> engine
> >> > using the Stanbol Benchmark Component (SBC). The SBC allows evaluating
> >> > content enhancement engines based on examples of desired and undesired
> >> > behavior defined through Benchmark Definition Language (BDL)
> statements.
> >> We
> >> > have transformed the dataset from Kulkarni et al. 2009 [4] into BDL.
> The
> >> > BDL data set is available from:
> >> > http://spotlight.dbpedia.org/download/stanbol/
> >> >
> >>
> >> Had not yet time to look at the examples in detail, but if the license
> >> if [4] allows and you agree we could think about to make them
> >> available as part of the Stanbol Enhancer.
> >>
> >> > The SBC is a nice way to perform manual inspection of the behavior of
> the
> >> > enhancement chain for different examples in the evaluation dataset.
> >> > However, for evaluations with several hundreds of examples, it would
> be
> >> > interesting to have scores that summarize the performance for the
> entire
> >> > dataset. We are in the process of conducting large scale experiments
> with
> >> > existing datasets, aiming at producing precision and recall figures
> for
> >> > different enhancement chains.*
> >> >
> >>
> >> This is completely true. Can you start an Jira Issue about that. I
> >> will definitely help with implementing this.
> >>
> >> best
> >> Rupert
> >>
> >>
> >>
> >> --
> >> | Rupert Westenthaler rupert.westenthaler@gmail.com
> >> | Bodenlehenstraße 11 ++43-699-11108907
> >> | A-5500 Bischofshofen
> >>
>
>
>
> --
> | Rupert Westenthaler rupert.westenthaler@gmail.com
> | Bodenlehenstraße 11 ++43-699-11108907
> | A-5500 Bischofshofen
>
Re: DBpedia Spotlight and Stanbol
Posted by Iavor Jelev <ia...@babelmonkeys.com>.
Hi Rupert,
thanks for your suggestions! I will look into incorporating them in our
implementation.
Am 04.06.2012 13:10, schrieb Rupert Westenthaler:
> No much simpler. Just to use the
>
> @Component(configurationFactory = true, [..])
>
> to allow users to configure multiple instances the DBpedia Spotlight
> Spotting engine. Than he can configure different spotting algorithms
> for the different instances.
>
> Note also that you can configure the Engines and the chain by using the
>
> <Install-Path>{path}</Install-Path>
>
> bundle extension.
>
> The OSGI service configuration need than to be located at
>
> {module}/src/main/resources/{path}
best,
Iavor
>>
>> On Mon, Jun 4, 2012 at 11:45 AM, Rupert Westenthaler <
>> rupert.westenthaler@gmail.com> wrote:
>>
>>> Hi Pablo
>>>
>>> I made som tests and the spotting looks great. Also tried some some of
>>> the different Spotting algorithms (NER, LingPipeSpotter (very slow)
>>> and Kea).
>>>
>>> Here are some Questions/Suggestions related to the engine.
>>>
>>> 1. Do you think it might make sense to allow multiple EngineInstances
>>> using different Spotting algorithms?
>>>
>>> 2. I noticed that created TextAnnotations do not have "dc-terms:type"
>>> information. This property is used to represent the "nature" (e.g.
>>> Person, Organisation, Place in case of Named Entities) by the the
>>> Stanbol Enhancement Structure. So if such information are available it
>>> would be great to set it.
>>>
>>> 3. I would suggest to add support for the type suggestion filter
>>> feature as shown in the 2nd example of the user manuel [1]
>>>
>>> [1] http://wiki.dbpedia.org/spotlight/usersmanual#h139-10
>>>
>>> On Fri, Jun 1, 2012 at 5:37 PM, Pablo Mendes <pa...@gmail.com>
>>> wrote:
>>>> Our next step is to create an enhancement chain with two enhancement
>>>> engines: DBpedia Spotlight Spotting and DBpedia Spotlight Disambiguation.
>>>
>>> So basically to split this engine in to separate one, right?
>>>
>>>> We have performed preliminary evaluations of the new enhancement engine
>>>> using the Stanbol Benchmark Component (SBC). The SBC allows evaluating
>>>> content enhancement engines based on examples of desired and undesired
>>>> behavior defined through Benchmark Definition Language (BDL) statements.
>>> We
>>>> have transformed the dataset from Kulkarni et al. 2009 [4] into BDL. The
>>>> BDL data set is available from:
>>>> http://spotlight.dbpedia.org/download/stanbol/
>>>>
>>>
>>> Had not yet time to look at the examples in detail, but if the license
>>> if [4] allows and you agree we could think about to make them
>>> available as part of the Stanbol Enhancer.
>>>
>>>> The SBC is a nice way to perform manual inspection of the behavior of the
>>>> enhancement chain for different examples in the evaluation dataset.
>>>> However, for evaluations with several hundreds of examples, it would be
>>>> interesting to have scores that summarize the performance for the entire
>>>> dataset. We are in the process of conducting large scale experiments with
>>>> existing datasets, aiming at producing precision and recall figures for
>>>> different enhancement chains.*
>>>>
>>>
>>> This is completely true. Can you start an Jira Issue about that. I
>>> will definitely help with implementing this.
>>>
>>> best
>>> Rupert
>>>
>>>
>>>
>>> --
>>> | Rupert Westenthaler rupert.westenthaler@gmail.com
>>> | Bodenlehenstraße 11 ++43-699-11108907
>>> | A-5500 Bischofshofen
>>>
>
>
>
Re: DBpedia Spotlight and Stanbol
Posted by Rupert Westenthaler <ru...@gmail.com>.
On Mon, Jun 4, 2012 at 12:51 PM, Pablo Mendes <pa...@gmail.com> wrote:
> Hi Rupert,
>
> 1. Do you think it might make sense to allow multiple EngineInstances
>> using different Spotting algorithms?
>>
>
> You mean that instead of a "/spot" you would have a /lingpipespot, /ner and
> /keyphrases? Can be done, but do you have a use case for that? Could this
> also be done as some sort of URL rewrite? From
> /spot?spotter=LingPipeSpotter to /lingpipespotter
No much simpler. Just to use the
@Component(configurationFactory = true, [..])
to allow users to configure multiple instances the DBpedia Spotlight
Spotting engine. Than he can configure different spotting algorithms
for the different instances.
>
>> Our next step is to create an enhancement chain with two enhancement
>> > engines: DBpedia Spotlight Spotting and DBpedia Spotlight Disambiguation.
>>
>> So basically to split this engine in to separate one, right?
>
>
> Correct. Split this engine into two, and have a chain connecting the
> engines.
Note also that you can configure the Engines and the chain by using the
<Install-Path>{path}</Install-Path>
bundle extension.
The OSGI service configuration need than to be located at
{module}/src/main/resources/{path}
If you add this to the module of the engines, than users would
automatically get the default configuration as soon as they add the
engines to Stanbol.
As example you can use the default configuration module of the Enhancer [1]
[1] http://svn.apache.org/repos/asf/incubator/stanbol/trunk/enhancer/defaults/pom.xml
>
> Had not yet time to look at the examples in detail, but if the license
>> if [4] allows and you agree we could think about to make them
>> available as part of the Stanbol Enhancer.
>
>
> I could not find licensing information. I can ask the authors directly.
>
>
>> This is completely true. Can you start an Jira Issue about that. I
>> will definitely help with implementing this.
>
>
> Sure. Here: https://issues.apache.org/jira/browse/STANBOL-652
> I have already implemented quite a bit of evaluation code. If you want, you
> can use it as a starting point. My approach is to go over the dataset and
> write out a log of each annotation attempt. Based on this log, I run a
> series of R scripts to interpret the evaluation results.
>
+1
BTW: do you plan to contribute your Enhancement Engines to Stanbol, or
would you rather package/promote it via Spotlight?
best
Rupert
> Cheers,
> Pablo
>
>
> On Mon, Jun 4, 2012 at 11:45 AM, Rupert Westenthaler <
> rupert.westenthaler@gmail.com> wrote:
>
>> Hi Pablo
>>
>> I made som tests and the spotting looks great. Also tried some some of
>> the different Spotting algorithms (NER, LingPipeSpotter (very slow)
>> and Kea).
>>
>> Here are some Questions/Suggestions related to the engine.
>>
>> 1. Do you think it might make sense to allow multiple EngineInstances
>> using different Spotting algorithms?
>>
>> 2. I noticed that created TextAnnotations do not have "dc-terms:type"
>> information. This property is used to represent the "nature" (e.g.
>> Person, Organisation, Place in case of Named Entities) by the the
>> Stanbol Enhancement Structure. So if such information are available it
>> would be great to set it.
>>
>> 3. I would suggest to add support for the type suggestion filter
>> feature as shown in the 2nd example of the user manuel [1]
>>
>> [1] http://wiki.dbpedia.org/spotlight/usersmanual#h139-10
>>
>> On Fri, Jun 1, 2012 at 5:37 PM, Pablo Mendes <pa...@gmail.com>
>> wrote:
>> > Our next step is to create an enhancement chain with two enhancement
>> > engines: DBpedia Spotlight Spotting and DBpedia Spotlight Disambiguation.
>>
>> So basically to split this engine in to separate one, right?
>>
>> > We have performed preliminary evaluations of the new enhancement engine
>> > using the Stanbol Benchmark Component (SBC). The SBC allows evaluating
>> > content enhancement engines based on examples of desired and undesired
>> > behavior defined through Benchmark Definition Language (BDL) statements.
>> We
>> > have transformed the dataset from Kulkarni et al. 2009 [4] into BDL. The
>> > BDL data set is available from:
>> > http://spotlight.dbpedia.org/download/stanbol/
>> >
>>
>> Had not yet time to look at the examples in detail, but if the license
>> if [4] allows and you agree we could think about to make them
>> available as part of the Stanbol Enhancer.
>>
>> > The SBC is a nice way to perform manual inspection of the behavior of the
>> > enhancement chain for different examples in the evaluation dataset.
>> > However, for evaluations with several hundreds of examples, it would be
>> > interesting to have scores that summarize the performance for the entire
>> > dataset. We are in the process of conducting large scale experiments with
>> > existing datasets, aiming at producing precision and recall figures for
>> > different enhancement chains.*
>> >
>>
>> This is completely true. Can you start an Jira Issue about that. I
>> will definitely help with implementing this.
>>
>> best
>> Rupert
>>
>>
>>
>> --
>> | Rupert Westenthaler rupert.westenthaler@gmail.com
>> | Bodenlehenstraße 11 ++43-699-11108907
>> | A-5500 Bischofshofen
>>
--
| Rupert Westenthaler rupert.westenthaler@gmail.com
| Bodenlehenstraße 11 ++43-699-11108907
| A-5500 Bischofshofen
Re: DBpedia Spotlight and Stanbol
Posted by Pablo Mendes <pa...@gmail.com>.
Hi Rupert,
1. Do you think it might make sense to allow multiple EngineInstances
> using different Spotting algorithms?
>
You mean that instead of a "/spot" you would have a /lingpipespot, /ner and
/keyphrases? Can be done, but do you have a use case for that? Could this
also be done as some sort of URL rewrite? From
/spot?spotter=LingPipeSpotter to /lingpipespotter
> Our next step is to create an enhancement chain with two enhancement
> > engines: DBpedia Spotlight Spotting and DBpedia Spotlight Disambiguation.
>
> So basically to split this engine in to separate one, right?
Correct. Split this engine into two, and have a chain connecting the
engines.
Had not yet time to look at the examples in detail, but if the license
> if [4] allows and you agree we could think about to make them
> available as part of the Stanbol Enhancer.
I could not find licensing information. I can ask the authors directly.
> This is completely true. Can you start an Jira Issue about that. I
> will definitely help with implementing this.
Sure. Here: https://issues.apache.org/jira/browse/STANBOL-652
I have already implemented quite a bit of evaluation code. If you want, you
can use it as a starting point. My approach is to go over the dataset and
write out a log of each annotation attempt. Based on this log, I run a
series of R scripts to interpret the evaluation results.
Cheers,
Pablo
On Mon, Jun 4, 2012 at 11:45 AM, Rupert Westenthaler <
rupert.westenthaler@gmail.com> wrote:
> Hi Pablo
>
> I made som tests and the spotting looks great. Also tried some some of
> the different Spotting algorithms (NER, LingPipeSpotter (very slow)
> and Kea).
>
> Here are some Questions/Suggestions related to the engine.
>
> 1. Do you think it might make sense to allow multiple EngineInstances
> using different Spotting algorithms?
>
> 2. I noticed that created TextAnnotations do not have "dc-terms:type"
> information. This property is used to represent the "nature" (e.g.
> Person, Organisation, Place in case of Named Entities) by the the
> Stanbol Enhancement Structure. So if such information are available it
> would be great to set it.
>
> 3. I would suggest to add support for the type suggestion filter
> feature as shown in the 2nd example of the user manuel [1]
>
> [1] http://wiki.dbpedia.org/spotlight/usersmanual#h139-10
>
> On Fri, Jun 1, 2012 at 5:37 PM, Pablo Mendes <pa...@gmail.com>
> wrote:
> > Our next step is to create an enhancement chain with two enhancement
> > engines: DBpedia Spotlight Spotting and DBpedia Spotlight Disambiguation.
>
> So basically to split this engine in to separate one, right?
>
> > We have performed preliminary evaluations of the new enhancement engine
> > using the Stanbol Benchmark Component (SBC). The SBC allows evaluating
> > content enhancement engines based on examples of desired and undesired
> > behavior defined through Benchmark Definition Language (BDL) statements.
> We
> > have transformed the dataset from Kulkarni et al. 2009 [4] into BDL. The
> > BDL data set is available from:
> > http://spotlight.dbpedia.org/download/stanbol/
> >
>
> Had not yet time to look at the examples in detail, but if the license
> if [4] allows and you agree we could think about to make them
> available as part of the Stanbol Enhancer.
>
> > The SBC is a nice way to perform manual inspection of the behavior of the
> > enhancement chain for different examples in the evaluation dataset.
> > However, for evaluations with several hundreds of examples, it would be
> > interesting to have scores that summarize the performance for the entire
> > dataset. We are in the process of conducting large scale experiments with
> > existing datasets, aiming at producing precision and recall figures for
> > different enhancement chains.*
> >
>
> This is completely true. Can you start an Jira Issue about that. I
> will definitely help with implementing this.
>
> best
> Rupert
>
>
>
> --
> | Rupert Westenthaler rupert.westenthaler@gmail.com
> | Bodenlehenstraße 11 ++43-699-11108907
> | A-5500 Bischofshofen
>
Re: DBpedia Spotlight and Stanbol
Posted by Iavor Jelev <ia...@babelmonkeys.com>.
Hi Rupert,
thanks for your feedback! I would like to answer 2 and 3, in order to
save Pablo some time.
2) this actually is a bug in my implementation, which will be fixed.
Thanks for pointing me to it.
3) the parameter "Types Restriction" in the configuration of the engine
is the one responsible for that. I'm pretty sure the bug mentioned in 2
prevents this functionality at this point. Will be fixed as well.
best regards,
Iavor
Am 04.06.2012 11:45, schrieb Rupert Westenthaler:
> Hi Pablo
>
> I made som tests and the spotting looks great. Also tried some some of
> the different Spotting algorithms (NER, LingPipeSpotter (very slow)
> and Kea).
>
> Here are some Questions/Suggestions related to the engine.
>
> 1. Do you think it might make sense to allow multiple EngineInstances
> using different Spotting algorithms?
>
> 2. I noticed that created TextAnnotations do not have "dc-terms:type"
> information. This property is used to represent the "nature" (e.g.
> Person, Organisation, Place in case of Named Entities) by the the
> Stanbol Enhancement Structure. So if such information are available it
> would be great to set it.
>
> 3. I would suggest to add support for the type suggestion filter
> feature as shown in the 2nd example of the user manuel [1]
>
> [1] http://wiki.dbpedia.org/spotlight/usersmanual#h139-10
>
> On Fri, Jun 1, 2012 at 5:37 PM, Pablo Mendes <pa...@gmail.com> wrote:
>> Our next step is to create an enhancement chain with two enhancement
>> engines: DBpedia Spotlight Spotting and DBpedia Spotlight Disambiguation.
>
> So basically to split this engine in to separate one, right?
>
>> We have performed preliminary evaluations of the new enhancement engine
>> using the Stanbol Benchmark Component (SBC). The SBC allows evaluating
>> content enhancement engines based on examples of desired and undesired
>> behavior defined through Benchmark Definition Language (BDL) statements. We
>> have transformed the dataset from Kulkarni et al. 2009 [4] into BDL. The
>> BDL data set is available from:
>> http://spotlight.dbpedia.org/download/stanbol/
>>
>
> Had not yet time to look at the examples in detail, but if the license
> if [4] allows and you agree we could think about to make them
> available as part of the Stanbol Enhancer.
>
>> The SBC is a nice way to perform manual inspection of the behavior of the
>> enhancement chain for different examples in the evaluation dataset.
>> However, for evaluations with several hundreds of examples, it would be
>> interesting to have scores that summarize the performance for the entire
>> dataset. We are in the process of conducting large scale experiments with
>> existing datasets, aiming at producing precision and recall figures for
>> different enhancement chains.*
>>
>
> This is completely true. Can you start an Jira Issue about that. I
> will definitely help with implementing this.
>
> best
> Rupert
>
>
>
Re: DBpedia Spotlight and Stanbol
Posted by Rupert Westenthaler <ru...@gmail.com>.
Hi Pablo
I made som tests and the spotting looks great. Also tried some some of
the different Spotting algorithms (NER, LingPipeSpotter (very slow)
and Kea).
Here are some Questions/Suggestions related to the engine.
1. Do you think it might make sense to allow multiple EngineInstances
using different Spotting algorithms?
2. I noticed that created TextAnnotations do not have "dc-terms:type"
information. This property is used to represent the "nature" (e.g.
Person, Organisation, Place in case of Named Entities) by the the
Stanbol Enhancement Structure. So if such information are available it
would be great to set it.
3. I would suggest to add support for the type suggestion filter
feature as shown in the 2nd example of the user manuel [1]
[1] http://wiki.dbpedia.org/spotlight/usersmanual#h139-10
On Fri, Jun 1, 2012 at 5:37 PM, Pablo Mendes <pa...@gmail.com> wrote:
> Our next step is to create an enhancement chain with two enhancement
> engines: DBpedia Spotlight Spotting and DBpedia Spotlight Disambiguation.
So basically to split this engine in to separate one, right?
> We have performed preliminary evaluations of the new enhancement engine
> using the Stanbol Benchmark Component (SBC). The SBC allows evaluating
> content enhancement engines based on examples of desired and undesired
> behavior defined through Benchmark Definition Language (BDL) statements. We
> have transformed the dataset from Kulkarni et al. 2009 [4] into BDL. The
> BDL data set is available from:
> http://spotlight.dbpedia.org/download/stanbol/
>
Had not yet time to look at the examples in detail, but if the license
if [4] allows and you agree we could think about to make them
available as part of the Stanbol Enhancer.
> The SBC is a nice way to perform manual inspection of the behavior of the
> enhancement chain for different examples in the evaluation dataset.
> However, for evaluations with several hundreds of examples, it would be
> interesting to have scores that summarize the performance for the entire
> dataset. We are in the process of conducting large scale experiments with
> existing datasets, aiming at producing precision and recall figures for
> different enhancement chains.*
>
This is completely true. Can you start an Jira Issue about that. I
will definitely help with implementing this.
best
Rupert
--
| Rupert Westenthaler rupert.westenthaler@gmail.com
| Bodenlehenstraße 11 ++43-699-11108907
| A-5500 Bischofshofen