You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by afs <gi...@git.apache.org> on 2016/03/05 15:32:36 UTC
[GitHub] jena pull request: JENA-1147 : Introduce FactoryRDF
GitHub user afs opened a pull request:
https://github.com/apache/jena/pull/127
JENA-1147 : Introduce FactoryRDF
FactoryRDF is an interface to abstract the creation of nodes, triples and quads from constituent elements, e.g. to create a URI Node, the operation takes a string for the URI. This allows different policies to be slotted in. This PR covers the use of FactoryRDF by ParserProfile. A PR to follow will use a caching implementation of FactoryRDF as the default mechanism when parsing.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/afs/jena master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/jena/pull/127.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #127
----
commit a19aa45a1106ae9ece835a9f17a39663f74f4b78
Author: Andy Seaborne <an...@apache.org>
Date: 2016-03-05T14:28:18Z
Introduce FactoryRDF and use in ParserProfile.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request: JENA-1147 : Introduce FactoryRDF
Posted by afs <gi...@git.apache.org>.
Github user afs commented on the pull request:
https://github.com/apache/jena/pull/127#issuecomment-193008707
This has got messy. I'll resubmit a cleaned up version.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request: JENA-1147 : Introduce FactoryRDF
Posted by afs <gi...@git.apache.org>.
Github user afs closed the pull request at:
https://github.com/apache/jena/pull/127
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
Background to PR#127 JENA-1147 : Introduce FactoryRDF
Posted by Andy Seaborne <an...@apache.org>.
On 05/03/16 14:32, afs wrote:
> GitHub user afs opened a pull request:
>
> https://github.com/apache/jena/pull/127
>
> JENA-1147 : Introduce FactoryRDF
This PR is the refactoring and better separates the concerns of the
ParserProfile which couples parsers to Node/triple etc.
In initial testing with caching, I'm getting space saving of 30-50% when
parsing into memory datasets.
For example:
Data: chembl_20.0_unichem.ttl.gz
which is 12,643,734 triples
General/Standard Space=6632.41 MB
General/Caching Space=3513.50 MB Cache=77.9%
TIM/Standard Space=9382.10 MB
TIM/Caching Space=6261.84 MB Cache=77.9%
i.e. saves 3G of Node space.
General:
the usual DatasetGraph implementation based on holding Graph objects.
TIM:
the transactional in-memory DatasetGraph
/Standard:
With this PR, no caching
/Caching:
With this PR and using a 10,000 slot LRU cache on createURI
10,000 is an untuned random guess at the moment.
%-age - the cache hit rate.
(see FactoryRDFCaching in the PR as an illustration - this is not on the
parser path in the PR as I want to test the setting a bit first).
Parsing speed is not adversely affected when parsing and storing
in-memory - in fact, it's slightly faster with caching (lower GC costs?).
I'm still investigating the impact on parsing-only of caching. Parsing
to a sink, after triple creation, is currently measurably slower with
caching and I want to investigate further.
Using:
Chembl/chembl_20.0_assay.ttl.gz
49,952,240 triples : Parse/Cache 131.40s (Avg: 380,154)
49,952,240 triples : Parse/Standard 108.83s (Avg: 459,014)
Old style RIOT before this PR
111.67 sec 49,952,240 triples 447,320.14 TPS
(I'm seeing +/- ~5s variability in these timings)
Andy