You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stanbol.apache.org by Antonio Perez <ap...@zaizi.com> on 2013/04/25 16:04:21 UTC
[GSOC Idea] Disambiguation algorithm for Apache Stanbol
Hi everybody
I'm Antonio David Pérez, a new Zaizi team member and a student for a MSc at
the University of Seville. Lastly, I've been involved in the development of
a semantic CMS solution in a Spanish Company called Ximdex working with
several technologies like Apache Nutch, Apache Solr and also Apache Stanbol.
Currently, I've been assigned to a project that involves different
technologies like Apache Stanbol and Apache ManifoldCF. So, related to
Stanbol, I'm interested in the disambiguation problem, so I would like to
prepare a proposal for GSoC about this topic.
I have been following last mails about disambiguation and WebID protocol. I
would be more interesting in develop disambiguation systems within Stanbol
using the major semantic knowledge bases. Actually, my initial idea is to
use Freebase with the aim to make it extensible to any other database like
Wikipedia and DBpedia. Following STANBOL-1037 [1], the main goal is to
implement a couple of global-approach disambiguation algorithms to be used
in Stanbol.
For this, I would like to discuss some topics about the proposal:
- Knowledge Base: I have decided to stick first to Freebase, because it has
a REST API allowing 100k calls per day for read and 10k for write. Besides
the REST API, an alternative could be to integrate the whole freebase graph
in Stanbol and use their Java API to manage it. Ideally, the management
framework should be valid for others knowledge bases as Wikipedia or
DBpedia.
- Resources: As have been pointed before in the mailing lists, google has
released a couple of resources to be used in disambiguation applications.
One if a dictionary of concepts from Wikipedia, using anchor text labels in
Wikipedia internal links to create an index of entities possible names [2].
The second one is a dataset of texts that links to concepts in the
Wikipedia [3] that can be used as disambiguation contexts according to
STANBOL-1037. I need to research if similar information can be retrieved
directly from freebase or , in other words, to check if this information is
already incorporated in Freebase.
Moreover, the proposal design will try to be as generic as possible in
order to be adaptable to any other Knowledge Base.
Waiting for your comments and valuable suggestions.
Thanks
Regards
[1] https://issues.apache.org/jira/browse/STANBOL-1037
[2]
http://googleresearch.blogspot.com.es/2012/05/from-words-to-concepts-and-back.html
[3] https://code.google.com/p/wiki-links/
--
------------------------------
This message should be regarded as confidential. If you have received this
email in error please notify the sender and destroy it immediately.
Statements of intent shall only become binding when confirmed in hard copy
by an authorised signatory.
Zaizi Ltd is registered in England and Wales with the registration number
6440931. The Registered Office is 222 Westbourne Studios, 242 Acklam Road,
London W10 5JJ, UK.
Re: [GSOC Idea] Disambiguation algorithm for Apache Stanbol
Posted by Rupert Westenthaler <ru...@gmail.com>.
On Tue, Apr 30, 2013 at 6:03 PM, Rafa Haro <rh...@zaizi.com> wrote:
> Hi Rupert, Antonio, all
>
> El 27/04/13 16:35, Rupert Westenthaler escribió:
>
>>> For this, I would like to discuss some topics about the proposal:
>>> >
>>> >- Knowledge Base: I have decided to stick first to Freebase, because it
>>> > has
>>> >a REST API allowing 100k calls per day for read and 10k for write.
>>> > Besides
>>> >the REST API, an alternative could be to integrate the whole freebase
>>> > graph
>>> >in Stanbol and use their Java API to manage it. Ideally, the management
>>> >framework should be valid for others knowledge bases as Wikipedia or
>>> >DBpedia.
>>> >
>>
>> I recently created my first Freebase index for Stanbol (see
>> STANBOL-1014 for the Indexing tool). First test on an Index with all
>> Freebase Topics and all languages have shown very nice result! IMO
>> Freebase is currently for sure the better choice over DBpedia. However
>> one needs to see/wait how Freebase compares to the Wikidata project
>> [4] that only recently entered phase 2.
>>
>> Designing disambiguation in a way that it can be applied to other
>> datasets would be for sure a great bonus. But given the good results
>> one can get with Freebase I would even be very interested if the
>> results would only work on Freebase ^^
>
> Following Rupert's idea, I agree that maybe the best is to develop a
> Knowledge Base manager within Stanbol for disambiguation purposes. IMO, it
> would be a mistake to try to come with an universal solution. I suppose that
> one wants to generate its knowledge base differently according to custom
> data domains. For instance, a graph representation is more suitable in "real
> world" knowledge bases, while most domains are well covered with a taxonomy
> structure.
>
> It would be important to develop tools to allow Stanbol to interact with
> these knowledge bases from-to EntityHub sites. Of course, a good way to
> learn how to do that could be developing first a nice solution only for
> Freebase.
>
There is already the "2-layered storage infrastructure" [1] for the
Contenthub. Developed mainly by Suat and Anil in an own branch. In
this trunk there is also a new commons.semanticindexing [2] package.
This architecture would allow for using a "knowledge base" as
"Indexing Source" and use an Entityhub Site as "Indexing Destination".
So it goes exactly in the proposed direction.
I have planed for long to adapt this for the Entityhub, but
development in this branch has not shown much progress in the recent
time and I do also have higher priority task ATM.
To be clear: I would not recommend to use this for a GSoC project but
rather use available APIs of the Entityhub as this is clearly off
topic to disambiguation. However the results of the propoesed GSoC
project could be a nice application/validation case for [1].
[1] https://issues.apache.org/jira/browse/STANBOL-471
[2] https://issues.apache.org/jira/browse/STANBOL-701
best
Rupert
--
| Rupert Westenthaler rupert.westenthaler@gmail.com
| Bodenlehenstraße 11 ++43-699-11108907
| A-5500 Bischofshofen
Re: [GSOC Idea] Disambiguation algorithm for Apache Stanbol
Posted by Rafa Haro <rh...@zaizi.com>.
Hi Rupert, Antonio, all
El 27/04/13 16:35, Rupert Westenthaler escribió:
>> For this, I would like to discuss some topics about the proposal:
>> >
>> >- Knowledge Base: I have decided to stick first to Freebase, because it has
>> >a REST API allowing 100k calls per day for read and 10k for write. Besides
>> >the REST API, an alternative could be to integrate the whole freebase graph
>> >in Stanbol and use their Java API to manage it. Ideally, the management
>> >framework should be valid for others knowledge bases as Wikipedia or
>> >DBpedia.
>> >
> I recently created my first Freebase index for Stanbol (see
> STANBOL-1014 for the Indexing tool). First test on an Index with all
> Freebase Topics and all languages have shown very nice result! IMO
> Freebase is currently for sure the better choice over DBpedia. However
> one needs to see/wait how Freebase compares to the Wikidata project
> [4] that only recently entered phase 2.
>
> Designing disambiguation in a way that it can be applied to other
> datasets would be for sure a great bonus. But given the good results
> one can get with Freebase I would even be very interested if the
> results would only work on Freebase ^^
Following Rupert's idea, I agree that maybe the best is to develop a
Knowledge Base manager within Stanbol for disambiguation purposes. IMO,
it would be a mistake to try to come with an universal solution. I
suppose that one wants to generate its knowledge base differently
according to custom data domains. For instance, a graph representation
is more suitable in "real world" knowledge bases, while most domains are
well covered with a taxonomy structure.
It would be important to develop tools to allow Stanbol to interact with
these knowledge bases from-to EntityHub sites. Of course, a good way to
learn how to do that could be developing first a nice solution only for
Freebase.
>
>> >- Resources: As have been pointed before in the mailing lists, google has
>> >released a couple of resources to be used in disambiguation applications.
>> >One if a dictionary of concepts from Wikipedia, using anchor text labels in
>> >Wikipedia internal links to create an index of entities possible names [2].
>> >The second one is a dataset of texts that links to concepts in the
>> >Wikipedia [3] that can be used as disambiguation contexts according to
>> >STANBOL-1037. I need to research if similar information can be retrieved
>> >directly from freebase or , in other words, to check if this information is
>> >already incorporated in Freebase.
>> >
> I think you can even use [2] and [3] for disambiguation on top of
> Freebase as there is anyway a mapping between Freebase and DBpedia
> concepts. However you will likely need a higher quality mapping as it
> is currently available. Because of that I would suggest you to start
> of with implementing STANBOL-1046 [5]. For possible names (or surface
> forms as they are also often called) one can use the Alias in
> Freebase. However AFAIK there are no information available in Freebase
> similar to [3]. Related to this I fond however an interesting pager
> [6]. The semi-supervised approach suggested in chapter III could
> nicely work. Especially if one considers that users could manually
> disambiguate Entities. In combination with other mentions extracted by
> the Stanbol Enhancer this could be used to acquire the required data.
I also suggest to compare both resources and try to improve it. For
example, AFAIK, DBpedia Spotlight also uses Wikipedia's Disambiguation
and Redirect pages to collect more surface forms. My impression is that
we can improve Google Concept Dictionary by bringing together entities'
names data in Freebase and Wikipedia. Google's dictionary seems to
contain only labels used in anchor texts from internal links
>
>> >Moreover, the proposal design will try to be as generic as possible in
>> >order to be adaptable to any other Knowledge Base.
>> >
> Disambiguation is not something easy and making something "generic"
> makes it even harder. So IMO having one/several more specific options
> would not hurt a GSoC proposal. It would also make it easier to
> evaluate the proposal.
Go for it!!
Cheers
Rafa
--
------------------------------
This message should be regarded as confidential. If you have received this
email in error please notify the sender and destroy it immediately.
Statements of intent shall only become binding when confirmed in hard copy
by an authorised signatory.
Zaizi Ltd is registered in England and Wales with the registration number
6440931. The Registered Office is 222 Westbourne Studios, 242 Acklam Road,
London W10 5JJ, UK.
Re: [GSOC Idea] Disambiguation algorithm for Apache Stanbol
Posted by Rupert Westenthaler <ru...@gmail.com>.
Hi harish
On Sat, Apr 27, 2013 at 8:00 PM, harish suvarna <hs...@gmail.com> wrote:
> Rupert,
> Is the freebase index available for use? From where I can get it?
> I can compare the entities I get from dbpedia and freebase on some of my
> test files.
The index is installed on
http://dev.iks-project.eu:8083/entityhub/site/freebase/
you can also use it with the enhancer by using
http://dev.iks-project.eu:8083/enhancer/chain/freebase-proper-noun
I would be happy to provide the index for download, but for now it is
not available because of its size. The 'freebase.solrindex.zip' file
has ~17GByte and if multiple users would try to download it this could
very well 'crash' the Internet connection of our company. If someone
has the infrastructure around to serve indexes of that size I am happy
to upload it their.
>
> Is [2]
> http://googleresearch.blogspot.com.es/2012/05/from-words-to-concepts-and-back.htmlused
> by any engine now in stanbol?
Not until now
> Are there any programming apis avaialble to access the concept dictionary?
>
Not as part of Stanbol.
> -harish
>
>
> On Sat, Apr 27, 2013 at 7:35 AM, Rupert Westenthaler <
> rupert.westenthaler@gmail.com> wrote:
>
>> Hi Antonio
>>
>> First of all thx for your interest!
>>
>> On Thu, Apr 25, 2013 at 4:04 PM, Antonio Perez <ap...@zaizi.com> wrote:
>> > Hi everybody
>> >
>> > I'm Antonio David Pérez, a new Zaizi team member and a student for a MSc
>> at
>> > the University of Seville. Lastly, I've been involved in the development
>> of
>> > a semantic CMS solution in a Spanish Company called Ximdex working with
>> > several technologies like Apache Nutch, Apache Solr and also Apache
>> Stanbol.
>> >
>> > Currently, I've been assigned to a project that involves different
>> > technologies like Apache Stanbol and Apache ManifoldCF. So, related to
>> > Stanbol, I'm interested in the disambiguation problem, so I would like to
>> > prepare a proposal for GSoC about this topic.
>> >
>>
>> If you do have already some experiences with Apache Stanbol, this
>> would be fore sure a big help for a GSoC project.
>>
>> > I have been following last mails about disambiguation and WebID
>> protocol. I
>> > would be more interesting in develop disambiguation systems within
>> Stanbol
>> > using the major semantic knowledge bases. Actually, my initial idea is to
>> > use Freebase with the aim to make it extensible to any other database
>> like
>> > Wikipedia and DBpedia. Following STANBOL-1037 [1], the main goal is to
>> > implement a couple of global-approach disambiguation algorithms to be
>> used
>> > in Stanbol.
>> >
>>
>> Disambiguation on "World Domain" datasets is a very important feature
>> for a lot of usage scenarios. So definitely very interesting and
>> relevant for Apache Stanbol.
>>
>> > For this, I would like to discuss some topics about the proposal:
>> >
>> > - Knowledge Base: I have decided to stick first to Freebase, because it
>> has
>> > a REST API allowing 100k calls per day for read and 10k for write.
>> Besides
>> > the REST API, an alternative could be to integrate the whole freebase
>> graph
>> > in Stanbol and use their Java API to manage it. Ideally, the management
>> > framework should be valid for others knowledge bases as Wikipedia or
>> > DBpedia.
>> >
>>
>> I recently created my first Freebase index for Stanbol (see
>> STANBOL-1014 for the Indexing tool). First test on an Index with all
>> Freebase Topics and all languages have shown very nice result! IMO
>> Freebase is currently for sure the better choice over DBpedia. However
>> one needs to see/wait how Freebase compares to the Wikidata project
>> [4] that only recently entered phase 2.
>>
>> Designing disambiguation in a way that it can be applied to other
>> datasets would be for sure a great bonus. But given the good results
>> one can get with Freebase I would even be very interested if the
>> results would only work on Freebase ^^
>>
>> > - Resources: As have been pointed before in the mailing lists, google has
>> > released a couple of resources to be used in disambiguation applications.
>> > One if a dictionary of concepts from Wikipedia, using anchor text labels
>> in
>> > Wikipedia internal links to create an index of entities possible names
>> [2].
>> > The second one is a dataset of texts that links to concepts in the
>> > Wikipedia [3] that can be used as disambiguation contexts according to
>> > STANBOL-1037. I need to research if similar information can be retrieved
>> > directly from freebase or , in other words, to check if this information
>> is
>> > already incorporated in Freebase.
>> >
>>
>> I think you can even use [2] and [3] for disambiguation on top of
>> Freebase as there is anyway a mapping between Freebase and DBpedia
>> concepts. However you will likely need a higher quality mapping as it
>> is currently available. Because of that I would suggest you to start
>> of with implementing STANBOL-1046 [5]. For possible names (or surface
>> forms as they are also often called) one can use the Alias in
>> Freebase. However AFAIK there are no information available in Freebase
>> similar to [3]. Related to this I fond however an interesting pager
>> [6]. The semi-supervised approach suggested in chapter III could
>> nicely work. Especially if one considers that users could manually
>> disambiguate Entities. In combination with other mentions extracted by
>> the Stanbol Enhancer this could be used to acquire the required data.
>>
>> > Moreover, the proposal design will try to be as generic as possible in
>> > order to be adaptable to any other Knowledge Base.
>> >
>>
>> Disambiguation is not something easy and making something "generic"
>> makes it even harder. So IMO having one/several more specific options
>> would not hurt a GSoC proposal. It would also make it easier to
>> evaluate the proposal.
>>
>> > Waiting for your comments and valuable suggestions.
>> >
>>
>> Hope my comments provided at least some valuable information.
>>
>> best
>> Rupert
>>
>> References:
>>
>> > [1] https://issues.apache.org/jira/browse/STANBOL-1037
>> > [2]
>> >
>> http://googleresearch.blogspot.com.es/2012/05/from-words-to-concepts-and-back.html
>> > [3] https://code.google.com/p/wiki-links/
>> [4] https://www.wikidata.org/wiki/Wikidata:Main_Page
>> [5] https://issues.apache.org/jira/browse/STANBOL-1046
>> [6]
>> http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en//pubs/archive/38389.pdf
>>
>> >
>> > --
>> >
>> > ------------------------------
>> > This message should be regarded as confidential. If you have received
>> this
>> > email in error please notify the sender and destroy it immediately.
>> > Statements of intent shall only become binding when confirmed in hard
>> copy
>> > by an authorised signatory.
>> >
>> > Zaizi Ltd is registered in England and Wales with the registration number
>> > 6440931. The Registered Office is 222 Westbourne Studios, 242 Acklam
>> Road,
>> > London W10 5JJ, UK.
>>
>>
>>
>> --
>> | Rupert Westenthaler rupert.westenthaler@gmail.com
>> | Bodenlehenstraße 11 ++43-699-11108907
>> | A-5500 Bischofshofen
>>
>
>
>
> --
> Thanks
> Harish
--
| Rupert Westenthaler rupert.westenthaler@gmail.com
| Bodenlehenstraße 11 ++43-699-11108907
| A-5500 Bischofshofen
Re: [GSOC Idea] Disambiguation algorithm for Apache Stanbol
Posted by harish suvarna <hs...@gmail.com>.
Rupert,
Is the freebase index available for use? From where I can get it?
I can compare the entities I get from dbpedia and freebase on some of my
test files.
Is [2]
http://googleresearch.blogspot.com.es/2012/05/from-words-to-concepts-and-back.htmlused
by any engine now in stanbol?
Are there any programming apis avaialble to access the concept dictionary?
-harish
On Sat, Apr 27, 2013 at 7:35 AM, Rupert Westenthaler <
rupert.westenthaler@gmail.com> wrote:
> Hi Antonio
>
> First of all thx for your interest!
>
> On Thu, Apr 25, 2013 at 4:04 PM, Antonio Perez <ap...@zaizi.com> wrote:
> > Hi everybody
> >
> > I'm Antonio David Pérez, a new Zaizi team member and a student for a MSc
> at
> > the University of Seville. Lastly, I've been involved in the development
> of
> > a semantic CMS solution in a Spanish Company called Ximdex working with
> > several technologies like Apache Nutch, Apache Solr and also Apache
> Stanbol.
> >
> > Currently, I've been assigned to a project that involves different
> > technologies like Apache Stanbol and Apache ManifoldCF. So, related to
> > Stanbol, I'm interested in the disambiguation problem, so I would like to
> > prepare a proposal for GSoC about this topic.
> >
>
> If you do have already some experiences with Apache Stanbol, this
> would be fore sure a big help for a GSoC project.
>
> > I have been following last mails about disambiguation and WebID
> protocol. I
> > would be more interesting in develop disambiguation systems within
> Stanbol
> > using the major semantic knowledge bases. Actually, my initial idea is to
> > use Freebase with the aim to make it extensible to any other database
> like
> > Wikipedia and DBpedia. Following STANBOL-1037 [1], the main goal is to
> > implement a couple of global-approach disambiguation algorithms to be
> used
> > in Stanbol.
> >
>
> Disambiguation on "World Domain" datasets is a very important feature
> for a lot of usage scenarios. So definitely very interesting and
> relevant for Apache Stanbol.
>
> > For this, I would like to discuss some topics about the proposal:
> >
> > - Knowledge Base: I have decided to stick first to Freebase, because it
> has
> > a REST API allowing 100k calls per day for read and 10k for write.
> Besides
> > the REST API, an alternative could be to integrate the whole freebase
> graph
> > in Stanbol and use their Java API to manage it. Ideally, the management
> > framework should be valid for others knowledge bases as Wikipedia or
> > DBpedia.
> >
>
> I recently created my first Freebase index for Stanbol (see
> STANBOL-1014 for the Indexing tool). First test on an Index with all
> Freebase Topics and all languages have shown very nice result! IMO
> Freebase is currently for sure the better choice over DBpedia. However
> one needs to see/wait how Freebase compares to the Wikidata project
> [4] that only recently entered phase 2.
>
> Designing disambiguation in a way that it can be applied to other
> datasets would be for sure a great bonus. But given the good results
> one can get with Freebase I would even be very interested if the
> results would only work on Freebase ^^
>
> > - Resources: As have been pointed before in the mailing lists, google has
> > released a couple of resources to be used in disambiguation applications.
> > One if a dictionary of concepts from Wikipedia, using anchor text labels
> in
> > Wikipedia internal links to create an index of entities possible names
> [2].
> > The second one is a dataset of texts that links to concepts in the
> > Wikipedia [3] that can be used as disambiguation contexts according to
> > STANBOL-1037. I need to research if similar information can be retrieved
> > directly from freebase or , in other words, to check if this information
> is
> > already incorporated in Freebase.
> >
>
> I think you can even use [2] and [3] for disambiguation on top of
> Freebase as there is anyway a mapping between Freebase and DBpedia
> concepts. However you will likely need a higher quality mapping as it
> is currently available. Because of that I would suggest you to start
> of with implementing STANBOL-1046 [5]. For possible names (or surface
> forms as they are also often called) one can use the Alias in
> Freebase. However AFAIK there are no information available in Freebase
> similar to [3]. Related to this I fond however an interesting pager
> [6]. The semi-supervised approach suggested in chapter III could
> nicely work. Especially if one considers that users could manually
> disambiguate Entities. In combination with other mentions extracted by
> the Stanbol Enhancer this could be used to acquire the required data.
>
> > Moreover, the proposal design will try to be as generic as possible in
> > order to be adaptable to any other Knowledge Base.
> >
>
> Disambiguation is not something easy and making something "generic"
> makes it even harder. So IMO having one/several more specific options
> would not hurt a GSoC proposal. It would also make it easier to
> evaluate the proposal.
>
> > Waiting for your comments and valuable suggestions.
> >
>
> Hope my comments provided at least some valuable information.
>
> best
> Rupert
>
> References:
>
> > [1] https://issues.apache.org/jira/browse/STANBOL-1037
> > [2]
> >
> http://googleresearch.blogspot.com.es/2012/05/from-words-to-concepts-and-back.html
> > [3] https://code.google.com/p/wiki-links/
> [4] https://www.wikidata.org/wiki/Wikidata:Main_Page
> [5] https://issues.apache.org/jira/browse/STANBOL-1046
> [6]
> http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en//pubs/archive/38389.pdf
>
> >
> > --
> >
> > ------------------------------
> > This message should be regarded as confidential. If you have received
> this
> > email in error please notify the sender and destroy it immediately.
> > Statements of intent shall only become binding when confirmed in hard
> copy
> > by an authorised signatory.
> >
> > Zaizi Ltd is registered in England and Wales with the registration number
> > 6440931. The Registered Office is 222 Westbourne Studios, 242 Acklam
> Road,
> > London W10 5JJ, UK.
>
>
>
> --
> | Rupert Westenthaler rupert.westenthaler@gmail.com
> | Bodenlehenstraße 11 ++43-699-11108907
> | A-5500 Bischofshofen
>
--
Thanks
Harish
Re: [GSOC Idea] Disambiguation algorithm for Apache Stanbol
Posted by Rupert Westenthaler <ru...@gmail.com>.
Hi Antonio
First of all thx for your interest!
On Thu, Apr 25, 2013 at 4:04 PM, Antonio Perez <ap...@zaizi.com> wrote:
> Hi everybody
>
> I'm Antonio David Pérez, a new Zaizi team member and a student for a MSc at
> the University of Seville. Lastly, I've been involved in the development of
> a semantic CMS solution in a Spanish Company called Ximdex working with
> several technologies like Apache Nutch, Apache Solr and also Apache Stanbol.
>
> Currently, I've been assigned to a project that involves different
> technologies like Apache Stanbol and Apache ManifoldCF. So, related to
> Stanbol, I'm interested in the disambiguation problem, so I would like to
> prepare a proposal for GSoC about this topic.
>
If you do have already some experiences with Apache Stanbol, this
would be fore sure a big help for a GSoC project.
> I have been following last mails about disambiguation and WebID protocol. I
> would be more interesting in develop disambiguation systems within Stanbol
> using the major semantic knowledge bases. Actually, my initial idea is to
> use Freebase with the aim to make it extensible to any other database like
> Wikipedia and DBpedia. Following STANBOL-1037 [1], the main goal is to
> implement a couple of global-approach disambiguation algorithms to be used
> in Stanbol.
>
Disambiguation on "World Domain" datasets is a very important feature
for a lot of usage scenarios. So definitely very interesting and
relevant for Apache Stanbol.
> For this, I would like to discuss some topics about the proposal:
>
> - Knowledge Base: I have decided to stick first to Freebase, because it has
> a REST API allowing 100k calls per day for read and 10k for write. Besides
> the REST API, an alternative could be to integrate the whole freebase graph
> in Stanbol and use their Java API to manage it. Ideally, the management
> framework should be valid for others knowledge bases as Wikipedia or
> DBpedia.
>
I recently created my first Freebase index for Stanbol (see
STANBOL-1014 for the Indexing tool). First test on an Index with all
Freebase Topics and all languages have shown very nice result! IMO
Freebase is currently for sure the better choice over DBpedia. However
one needs to see/wait how Freebase compares to the Wikidata project
[4] that only recently entered phase 2.
Designing disambiguation in a way that it can be applied to other
datasets would be for sure a great bonus. But given the good results
one can get with Freebase I would even be very interested if the
results would only work on Freebase ^^
> - Resources: As have been pointed before in the mailing lists, google has
> released a couple of resources to be used in disambiguation applications.
> One if a dictionary of concepts from Wikipedia, using anchor text labels in
> Wikipedia internal links to create an index of entities possible names [2].
> The second one is a dataset of texts that links to concepts in the
> Wikipedia [3] that can be used as disambiguation contexts according to
> STANBOL-1037. I need to research if similar information can be retrieved
> directly from freebase or , in other words, to check if this information is
> already incorporated in Freebase.
>
I think you can even use [2] and [3] for disambiguation on top of
Freebase as there is anyway a mapping between Freebase and DBpedia
concepts. However you will likely need a higher quality mapping as it
is currently available. Because of that I would suggest you to start
of with implementing STANBOL-1046 [5]. For possible names (or surface
forms as they are also often called) one can use the Alias in
Freebase. However AFAIK there are no information available in Freebase
similar to [3]. Related to this I fond however an interesting pager
[6]. The semi-supervised approach suggested in chapter III could
nicely work. Especially if one considers that users could manually
disambiguate Entities. In combination with other mentions extracted by
the Stanbol Enhancer this could be used to acquire the required data.
> Moreover, the proposal design will try to be as generic as possible in
> order to be adaptable to any other Knowledge Base.
>
Disambiguation is not something easy and making something "generic"
makes it even harder. So IMO having one/several more specific options
would not hurt a GSoC proposal. It would also make it easier to
evaluate the proposal.
> Waiting for your comments and valuable suggestions.
>
Hope my comments provided at least some valuable information.
best
Rupert
References:
> [1] https://issues.apache.org/jira/browse/STANBOL-1037
> [2]
> http://googleresearch.blogspot.com.es/2012/05/from-words-to-concepts-and-back.html
> [3] https://code.google.com/p/wiki-links/
[4] https://www.wikidata.org/wiki/Wikidata:Main_Page
[5] https://issues.apache.org/jira/browse/STANBOL-1046
[6] http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en//pubs/archive/38389.pdf
>
> --
>
> ------------------------------
> This message should be regarded as confidential. If you have received this
> email in error please notify the sender and destroy it immediately.
> Statements of intent shall only become binding when confirmed in hard copy
> by an authorised signatory.
>
> Zaizi Ltd is registered in England and Wales with the registration number
> 6440931. The Registered Office is 222 Westbourne Studios, 242 Acklam Road,
> London W10 5JJ, UK.
--
| Rupert Westenthaler rupert.westenthaler@gmail.com
| Bodenlehenstraße 11 ++43-699-11108907
| A-5500 Bischofshofen