You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@stanbol.apache.org by "Bertrand Delacretaz (JIRA)" <ji...@apache.org> on 2011/04/01 13:54:06 UTC

[jira] [Created] (STANBOL-138) Enhancer benchmark tool

Enhancer benchmark tool
-----------------------

                 Key: STANBOL-138
                 URL: https://issues.apache.org/jira/browse/STANBOL-138
             Project: Stanbol
          Issue Type: New Feature
          Components: Enhancer, Testing
            Reporter: Bertrand Delacretaz
            Assignee: Bertrand Delacretaz
            Priority: Minor


I'm working on a benchmark tool for the stanbol enhancer, where content is fed to the enhancer and the output is checked according to a set of expectations.

Here's an example benchmark - the syntax might evolve as we implement this, but it explains the general idea. 

To run the benchmark we'll POST a plain text document containing one or several such benchmarks, and get an HTML document back with the results.

= INPUT =
Bob Marley was born in Kingston, Jamaica

= EXPECT =
# Comments such as this one are ignored
# EXPECT defines groups of predicate/object matchers that we expect to find in the output
# Each group applies to one given enhancement: for the expectation to succeed, at least
# one enhancement must match all lines in the group
http://fise.iks-project.eu/ontology/entity-reference URI http://dbpedia.org/resource/Kingston%2C_Jamaica
http://purl.org/dc/terms/creator STRING org.apache..*ReferencedSiteEntityTaggingEnhancementEngine

# Groups are separated by empty lines, here's a new one
http://fise.iks-project.eu/ontology/entity-type URI http://dbpedia.org/ontology/MusicalArtist
http://fise.iks-project.eu/ontology/entity-reference URI http://dbpedia.org/resource/Bob_Marley
http://purl.org/dc/terms/creator REGEXP org.apache.*EntityMentionEnhancementEngine

= COMPLAIN =
# COMPLAIN statements are similar to EXPECT but we expect them *not* to be fulfilled by
# the enhancer output. 


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (STANBOL-138) Enhancer benchmark tool

Posted by "Bertrand Delacretaz (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/STANBOL-138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13078716#comment-13078716 ] 

Bertrand Delacretaz commented on STANBOL-138:
---------------------------------------------

See also http://www.slideshare.net/bdelacretaz/bertrand-stanbolbenchmarksapril2011 for a set of slides about this

> Enhancer benchmark tool
> -----------------------
>
>                 Key: STANBOL-138
>                 URL: https://issues.apache.org/jira/browse/STANBOL-138
>             Project: Stanbol
>          Issue Type: New Feature
>          Components: Enhancer, Testing
>            Reporter: Bertrand Delacretaz
>            Assignee: Bertrand Delacretaz
>            Priority: Minor
>
> I'm working on a benchmark tool for the stanbol enhancer, where content is fed to the enhancer and the output is checked according to a set of expectations.
> Here's an example benchmark - the syntax might evolve as we implement this, but it explains the general idea. 
> To run the benchmark we'll POST a plain text document containing one or several such benchmarks, and get an HTML document back with the results.
> = INPUT =
> Bob Marley was born in Kingston, Jamaica
> = EXPECT =
> # Comments such as this one are ignored
> # EXPECT defines groups of predicate/object matchers that we expect to find in the output
> # Each group applies to one given enhancement: for the expectation to succeed, at least
> # one enhancement must match all lines in the group
> http://fise.iks-project.eu/ontology/entity-reference URI http://dbpedia.org/resource/Kingston%2C_Jamaica
> http://purl.org/dc/terms/creator STRING org.apache..*ReferencedSiteEntityTaggingEnhancementEngine
> # Groups are separated by empty lines, here's a new one
> http://fise.iks-project.eu/ontology/entity-type URI http://dbpedia.org/ontology/MusicalArtist
> http://fise.iks-project.eu/ontology/entity-reference URI http://dbpedia.org/resource/Bob_Marley
> http://purl.org/dc/terms/creator REGEXP org.apache.*EntityMentionEnhancementEngine
> = COMPLAIN =
> # COMPLAIN statements are similar to EXPECT but we expect them *not* to be fulfilled by
> # the enhancer output. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (STANBOL-138) Enhancer benchmark tool

Posted by "Bertrand Delacretaz (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/STANBOL-138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015450#comment-13015450 ] 

Bertrand Delacretaz commented on STANBOL-138:
---------------------------------------------

Committed a better presentation in revision 1088621 and 1088624, and integrated in enhancer/launcher/full in revision 1088635

> Enhancer benchmark tool
> -----------------------
>
>                 Key: STANBOL-138
>                 URL: https://issues.apache.org/jira/browse/STANBOL-138
>             Project: Stanbol
>          Issue Type: New Feature
>          Components: Enhancer, Testing
>            Reporter: Bertrand Delacretaz
>            Assignee: Bertrand Delacretaz
>            Priority: Minor
>
> I'm working on a benchmark tool for the stanbol enhancer, where content is fed to the enhancer and the output is checked according to a set of expectations.
> Here's an example benchmark - the syntax might evolve as we implement this, but it explains the general idea. 
> To run the benchmark we'll POST a plain text document containing one or several such benchmarks, and get an HTML document back with the results.
> = INPUT =
> Bob Marley was born in Kingston, Jamaica
> = EXPECT =
> # Comments such as this one are ignored
> # EXPECT defines groups of predicate/object matchers that we expect to find in the output
> # Each group applies to one given enhancement: for the expectation to succeed, at least
> # one enhancement must match all lines in the group
> http://fise.iks-project.eu/ontology/entity-reference URI http://dbpedia.org/resource/Kingston%2C_Jamaica
> http://purl.org/dc/terms/creator STRING org.apache..*ReferencedSiteEntityTaggingEnhancementEngine
> # Groups are separated by empty lines, here's a new one
> http://fise.iks-project.eu/ontology/entity-type URI http://dbpedia.org/ontology/MusicalArtist
> http://fise.iks-project.eu/ontology/entity-reference URI http://dbpedia.org/resource/Bob_Marley
> http://purl.org/dc/terms/creator REGEXP org.apache.*EntityMentionEnhancementEngine
> = COMPLAIN =
> # COMPLAIN statements are similar to EXPECT but we expect them *not* to be fulfilled by
> # the enhancer output. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (STANBOL-138) Enhancer benchmark tool

Posted by "Bertrand Delacretaz (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/STANBOL-138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bertrand Delacretaz resolved STANBOL-138.
-----------------------------------------

    Resolution: Fixed

A screencast created with revision 1089862 is available at http://www.youtube.com/watch?v=QQ4G_mFLzVw

> Enhancer benchmark tool
> -----------------------
>
>                 Key: STANBOL-138
>                 URL: https://issues.apache.org/jira/browse/STANBOL-138
>             Project: Stanbol
>          Issue Type: New Feature
>          Components: Enhancer, Testing
>            Reporter: Bertrand Delacretaz
>            Assignee: Bertrand Delacretaz
>            Priority: Minor
>
> I'm working on a benchmark tool for the stanbol enhancer, where content is fed to the enhancer and the output is checked according to a set of expectations.
> Here's an example benchmark - the syntax might evolve as we implement this, but it explains the general idea. 
> To run the benchmark we'll POST a plain text document containing one or several such benchmarks, and get an HTML document back with the results.
> = INPUT =
> Bob Marley was born in Kingston, Jamaica
> = EXPECT =
> # Comments such as this one are ignored
> # EXPECT defines groups of predicate/object matchers that we expect to find in the output
> # Each group applies to one given enhancement: for the expectation to succeed, at least
> # one enhancement must match all lines in the group
> http://fise.iks-project.eu/ontology/entity-reference URI http://dbpedia.org/resource/Kingston%2C_Jamaica
> http://purl.org/dc/terms/creator STRING org.apache..*ReferencedSiteEntityTaggingEnhancementEngine
> # Groups are separated by empty lines, here's a new one
> http://fise.iks-project.eu/ontology/entity-type URI http://dbpedia.org/ontology/MusicalArtist
> http://fise.iks-project.eu/ontology/entity-reference URI http://dbpedia.org/resource/Bob_Marley
> http://purl.org/dc/terms/creator REGEXP org.apache.*EntityMentionEnhancementEngine
> = COMPLAIN =
> # COMPLAIN statements are similar to EXPECT but we expect them *not* to be fulfilled by
> # the enhancer output. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (STANBOL-138) Enhancer benchmark tool

Posted by "Bertrand Delacretaz (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/STANBOL-138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13014745#comment-13014745 ] 

Bertrand Delacretaz commented on STANBOL-138:
---------------------------------------------

First version implemented in revision 1087804 - work in progress, not added to the list of standard bundles yet.

To test, install the enhancer/benchmark bundle and open http://localhost:8080/benchmark - displays a form that runs a simple example benchmark. Presentation is stil rough

> Enhancer benchmark tool
> -----------------------
>
>                 Key: STANBOL-138
>                 URL: https://issues.apache.org/jira/browse/STANBOL-138
>             Project: Stanbol
>          Issue Type: New Feature
>          Components: Enhancer, Testing
>            Reporter: Bertrand Delacretaz
>            Assignee: Bertrand Delacretaz
>            Priority: Minor
>
> I'm working on a benchmark tool for the stanbol enhancer, where content is fed to the enhancer and the output is checked according to a set of expectations.
> Here's an example benchmark - the syntax might evolve as we implement this, but it explains the general idea. 
> To run the benchmark we'll POST a plain text document containing one or several such benchmarks, and get an HTML document back with the results.
> = INPUT =
> Bob Marley was born in Kingston, Jamaica
> = EXPECT =
> # Comments such as this one are ignored
> # EXPECT defines groups of predicate/object matchers that we expect to find in the output
> # Each group applies to one given enhancement: for the expectation to succeed, at least
> # one enhancement must match all lines in the group
> http://fise.iks-project.eu/ontology/entity-reference URI http://dbpedia.org/resource/Kingston%2C_Jamaica
> http://purl.org/dc/terms/creator STRING org.apache..*ReferencedSiteEntityTaggingEnhancementEngine
> # Groups are separated by empty lines, here's a new one
> http://fise.iks-project.eu/ontology/entity-type URI http://dbpedia.org/ontology/MusicalArtist
> http://fise.iks-project.eu/ontology/entity-reference URI http://dbpedia.org/resource/Bob_Marley
> http://purl.org/dc/terms/creator REGEXP org.apache.*EntityMentionEnhancementEngine
> = COMPLAIN =
> # COMPLAIN statements are similar to EXPECT but we expect them *not* to be fulfilled by
> # the enhancer output. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira