You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@stanbol.apache.org by "Rupert Westenthaler (JIRA)" <ji...@apache.org> on 2013/04/15 09:24:16 UTC
[jira] [Resolved] (STANBOL-1016) Add RDF Triple Filter support to
the Jena TDB Indexing Source
[ https://issues.apache.org/jira/browse/STANBOL-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rupert Westenthaler resolved STANBOL-1016.
------------------------------------------
Resolution: Fixed
Implemented with http://svn.apache.org/r1467865
> Add RDF Triple Filter support to the Jena TDB Indexing Source
> -------------------------------------------------------------
>
> Key: STANBOL-1016
> URL: https://issues.apache.org/jira/browse/STANBOL-1016
> Project: Stanbol
> Issue Type: Sub-task
> Components: Entityhub
> Reporter: Rupert Westenthaler
> Assignee: Rupert Westenthaler
>
> The freebase.com dump has ~1.200.000.000 triples. Loading those triples to Jena TDB takes ages if the RAM (available to the memory mapped files) is not huge enough to hold the data. If the number of imported triples exceeds the available RAM the import speed deceases to ~7k triples/sec on an SSD. For reaching those 7k triple/sec the logs show 1,5k reads and 1k writes per second so import speeds on normal hard discs should be much slower.
> As most of the Triples contained in the freebase dump are not relevant for indexing this issue will introduce a new feature to the Jena TDB Indexing Source that allows - on a very low level - to filter out triples.
> This Filter will be based on Triples provided by the Riot parser and define a single method
> accept(Node subject, Node predicate, Node object) : boolean
> In addition the interface will extend IndexingComponent, what will allow to configure it via the configuration file of the
> org.apache.stanbol.entityhub.indexing.source.jenatdb.RdfIndexingSource
> The parameter used to configure the filter will be called "import-filter" and the value MUST BE the Class name of the used implementation.
> The configuration of the jenatdb.RdfIndexingSource will be parsed to the Import Filters #setConfiguration(..) method. This means that users will need to add configuration properties of for the Import Filter to the configuration of the RdfIndexingSource.
> To keep things simple the RdfImportFilter interface will be specific to the Jena TDB Indexing Source.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira