You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Rinne Mikko <mi...@aalto.fi> on 2011/10/28 10:16:04 UTC
Parallel SPARQL queries with ARQ?
Hi!
New to Jena and the list, please bear with me if this has been explained over and over again. I had so far no luck with the documentation, mailing list archives or googling, so here we go:
Can ARQ be used to execute multiple parallel SPARQL queries?
I would like to configure e.g. 100 or 1000 queries and then run them against a single file of triples. I wrote a piece of code to run the queries in sequence and got suprisingly good performance with brute force, but I would expect going through the dataset only once to perform much better.
If ARQ doesn't support this, is it the Jena forward-chaining RETE engine<http://jena.sourceforge.net/inference/> I should be looking at, and translate the SPARQL queries manually?
Ultimately I would like to track the processing of each new triple from the dataset, in case it matches a query. Any proposals on good documentation? Which level should I try to interface on?
Thanks!
Mikko
Re: Parallel SPARQL queries with ARQ?
Posted by Rinne Mikko <mi...@aalto.fi>.
Hi Dave!
This was exactly the answer I was looking for - thank you very much! I'll dive into JenaRules and see, how accurately the current SPARQL-queries would translate.
I only saw your mail after sending mine; sorry for the extra traffic on the list.
Mikko
On 28. Oct 2011, at 1:14 PM, Dave Reynolds wrote:
> Hi,
>
> On Fri, 2011-10-28 at 08:16 +0000, Rinne Mikko wrote:
>> Hi!
>>
>> New to Jena and the list, please bear with me if this has been explained over and over again. I had so far no luck with the documentation, mailing list archives or googling, so here we go:
>>
>> Can ARQ be used to execute multiple parallel SPARQL queries?
>>
>> I would like to configure e.g. 100 or 1000 queries and then run them against a single file of triples. I wrote a piece of code to run the queries in sequence and got suprisingly good performance with brute force, but I would expect going through the dataset only once to perform much better.
>>
>> If ARQ doesn't support this, is it the Jena forward-chaining RETE engine<http://jena.sourceforge.net/inference/> I should be looking at, and translate the SPARQL queries manually?
>
> Like Paolo I'm not quite sure what you are trying to do but based on
> that question let me take a guess ...
>
> It sounds like you have your data, maybe in a memory model, and want to
> run a *lot* of queries over that single data set. You suspect that
> instead of each query starting over again maybe you could stream the
> data once through some sort of query sieve to do all the queries at once
> in one pass. Is that about right?
>
> If so then there is no specific parallel-SPARQL-query support in Jena
> but as you say it might be possible to use the RETE engine depending on
> the specifics of what you are doing.
>
> As an aside, note it is possible issue SPARQL queries in parallel (most
> of the Jena stores are Multiple Reader Single Writer), so on a
> multi-core machine you might get extra speed from the brute force query
> approach by spreading the queries across a small number of threads.
>
> The RETE engine works by keeping tables of partially matched triple
> patterns so that each new triple is matched against the rules
> incrementally. Which does seem related to what you want.
>
> The problem is that JenaRules is not SPARQL - there's no equivalents of
> SPARQL constructs like UNION, ORDER BY, DISTINCT etc and the set of
> built in predicates for filtering is different. Furthermore all you can
> do as a result of a rule matching is assert a set of triples as a result
> (or call some java code like Print) - you don't have access to a stream
> of binding results in the way you do with SPARQL.
>
> However, if your queries are primarily just basic graph patterns and if
> the results from your queries can be expressed as new triples then you
> could indeed use the RETE engine.
>
> Whether it will gain you any benefit depends on the specifics. If there
> is a lot of shared patterns between your rules then it might. If not
> the overheads of the rule machinery may outweigh the gain from reuse of
> partial matches.
>
> I would suggest you try a small experiment first to measure the
> cost/gains before committing to it.
>
>> Ultimately I would like to track the processing of each new triple from the dataset, in case it matches a query.
>
> That is the way the RETE engine works. So long as you are only adding
> and not removing triples (and so long as you don't have any nasty
> non-monotonic operators in your rules) then each triple added to the
> model is filtered through the RETE network to see if it triggers more
> rules.
>
>> Any proposals on good documentation?
>
> The primary documentation for the rules engine is:
>
> http://incubator.apache.org/jena/documentation/inference/index.html#rules
>
> Dave
>
>
Re: Parallel SPARQL queries with ARQ?
Posted by Dave Reynolds <da...@gmail.com>.
Hi,
On Fri, 2011-10-28 at 08:16 +0000, Rinne Mikko wrote:
> Hi!
>
> New to Jena and the list, please bear with me if this has been explained over and over again. I had so far no luck with the documentation, mailing list archives or googling, so here we go:
>
> Can ARQ be used to execute multiple parallel SPARQL queries?
>
> I would like to configure e.g. 100 or 1000 queries and then run them against a single file of triples. I wrote a piece of code to run the queries in sequence and got suprisingly good performance with brute force, but I would expect going through the dataset only once to perform much better.
>
> If ARQ doesn't support this, is it the Jena forward-chaining RETE engine<http://jena.sourceforge.net/inference/> I should be looking at, and translate the SPARQL queries manually?
Like Paolo I'm not quite sure what you are trying to do but based on
that question let me take a guess ...
It sounds like you have your data, maybe in a memory model, and want to
run a *lot* of queries over that single data set. You suspect that
instead of each query starting over again maybe you could stream the
data once through some sort of query sieve to do all the queries at once
in one pass. Is that about right?
If so then there is no specific parallel-SPARQL-query support in Jena
but as you say it might be possible to use the RETE engine depending on
the specifics of what you are doing.
As an aside, note it is possible issue SPARQL queries in parallel (most
of the Jena stores are Multiple Reader Single Writer), so on a
multi-core machine you might get extra speed from the brute force query
approach by spreading the queries across a small number of threads.
The RETE engine works by keeping tables of partially matched triple
patterns so that each new triple is matched against the rules
incrementally. Which does seem related to what you want.
The problem is that JenaRules is not SPARQL - there's no equivalents of
SPARQL constructs like UNION, ORDER BY, DISTINCT etc and the set of
built in predicates for filtering is different. Furthermore all you can
do as a result of a rule matching is assert a set of triples as a result
(or call some java code like Print) - you don't have access to a stream
of binding results in the way you do with SPARQL.
However, if your queries are primarily just basic graph patterns and if
the results from your queries can be expressed as new triples then you
could indeed use the RETE engine.
Whether it will gain you any benefit depends on the specifics. If there
is a lot of shared patterns between your rules then it might. If not
the overheads of the rule machinery may outweigh the gain from reuse of
partial matches.
I would suggest you try a small experiment first to measure the
cost/gains before committing to it.
> Ultimately I would like to track the processing of each new triple from the dataset, in case it matches a query.
That is the way the RETE engine works. So long as you are only adding
and not removing triples (and so long as you don't have any nasty
non-monotonic operators in your rules) then each triple added to the
model is filtered through the RETE network to see if it triggers more
rules.
> Any proposals on good documentation?
The primary documentation for the rules engine is:
http://incubator.apache.org/jena/documentation/inference/index.html#rules
Dave
Re: Parallel SPARQL queries with ARQ?
Posted by Andy Seaborne <an...@apache.org>.
On 28/10/11 12:13, Damian Steer wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 28/10/11 11:59, Paolo Castagna wrote:
>> Hi Ian
>>
>> Ian Dickinson wrote:
>>> On 28/10/11 09:45, Paolo Castagna wrote (on jena-users):
>>>> (should we put<meta http-equiv="refresh" content="0;
>>>> url=http://incubator.apache.org/jena/..."> on the HTML pages on
>>>> SF?)
>>> I think we're getting close to the point where we could put a
>>> .htaccess file on both jena.sf.net and openjena.org to redirect
>>> to the Apache site. Not quite there yet, but close. That's
>>> assuming we can put a .htaccess on SourceForge, but I don't see
>>> why not.
>>
>> I am not sure if we put a .htaccess file on SF. If so, yes. Not a
>> bad idea when all the content is moved.
>>
>> The<meta http-equiv="refresh" ...> allows us to selectively
>> redirect from the existing pages we have already ported over to the
>> new website. Leaving the content we have not ported yet,
>> unchanged.
>>
>> Paolo
>
> I believe google and yahoo treat refreshes with no delay as equivalent
> to 301s, which is presumably what we want. Searches will switch to the
> new layout, etc.
>
> So even if we can't use .htaccess it isn't a serious impediment.
We can put in .htaccess files on SF - we already do.
Andy
>
> Damian
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk6qjlMACgkQAyLCB+mTtymcvgCgkqULycWMURA3k0ryz4pzlPiP
> XaEAoNCgZJpjHyUMDg++4Gh3+D5kRtE/
> =2tHV
> -----END PGP SIGNATURE-----
Re: Parallel SPARQL queries with ARQ?
Posted by Damian Steer <d....@bristol.ac.uk>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 28/10/11 11:59, Paolo Castagna wrote:
> Hi Ian
>
> Ian Dickinson wrote:
>> On 28/10/11 09:45, Paolo Castagna wrote (on jena-users):
>>> (should we put <meta http-equiv="refresh" content="0;
>>> url=http://incubator.apache.org/jena/..."> on the HTML pages on
>>> SF?)
>> I think we're getting close to the point where we could put a
>> .htaccess file on both jena.sf.net and openjena.org to redirect
>> to the Apache site. Not quite there yet, but close. That's
>> assuming we can put a .htaccess on SourceForge, but I don't see
>> why not.
>
> I am not sure if we put a .htaccess file on SF. If so, yes. Not a
> bad idea when all the content is moved.
>
> The <meta http-equiv="refresh" ...> allows us to selectively
> redirect from the existing pages we have already ported over to the
> new website. Leaving the content we have not ported yet,
> unchanged.
>
> Paolo
I believe google and yahoo treat refreshes with no delay as equivalent
to 301s, which is presumably what we want. Searches will switch to the
new layout, etc.
So even if we can't use .htaccess it isn't a serious impediment.
Damian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6qjlMACgkQAyLCB+mTtymcvgCgkqULycWMURA3k0ryz4pzlPiP
XaEAoNCgZJpjHyUMDg++4Gh3+D5kRtE/
=2tHV
-----END PGP SIGNATURE-----
Re: Parallel SPARQL queries with ARQ?
Posted by Paolo Castagna <ca...@googlemail.com>.
Hi Ian
Ian Dickinson wrote:
> On 28/10/11 09:45, Paolo Castagna wrote (on jena-users):
>> (should we put <meta http-equiv="refresh" content="0;
>> url=http://incubator.apache.org/jena/..."> on the HTML pages on SF?)
> I think we're getting close to the point where we could put a .htaccess
> file on both jena.sf.net and openjena.org to redirect to the Apache
> site. Not quite there yet, but close. That's assuming we can put a
> .htaccess on SourceForge, but I don't see why not.
I am not sure if we put a .htaccess file on SF. If so, yes.
Not a bad idea when all the content is moved.
The <meta http-equiv="refresh" ...> allows us to selectively
redirect from the existing pages we have already ported over
to the new website. Leaving the content we have not ported
yet, unchanged.
Paolo
>
> Ian
>
>
Re: Parallel SPARQL queries with ARQ?
Posted by Ian Dickinson <ia...@epimorphics.com>.
On 28/10/11 09:45, Paolo Castagna wrote (on jena-users):
> (should we put <meta http-equiv="refresh" content="0;
> url=http://incubator.apache.org/jena/..."> on the HTML pages on SF?)
I think we're getting close to the point where we could put a .htaccess
file on both jena.sf.net and openjena.org to redirect to the Apache
site. Not quite there yet, but close. That's assuming we can put a
.htaccess on SourceForge, but I don't see why not.
Ian
--
____________________________________________________________
Ian Dickinson Epimorphics Ltd, Bristol, UK
mailto:ian@epimorphics.com http://www.epimorphics.com
cell: +44-7786-850536 landline: +44-1275-399069
------------------------------------------------------------
Epimorphics Ltd. is a limited company registered in England
(no. 7016688). Registered address: Court Lodge, 105 High St,
Portishead, Bristol BS20 6PT, UK
Re: Parallel SPARQL queries with ARQ?
Posted by Paolo Castagna <ca...@googlemail.com>.
Rinne Mikko wrote:
> is it the Jena forward-chaining RETE engine<http://jena.sourceforge.net/inference/>
By the way, you are on the right mailing list.
But on the "wrong" website.
Your link has been moved here now:
http://incubator.apache.org/jena/documentation/inference/
(should we put <meta http-equiv="refresh" content="0;
url=http://incubator.apache.org/jena/..."> on the HTML pages on SF?)
Paolo
Re: Parallel SPARQL queries with ARQ?
Posted by Paolo Castagna <ca...@googlemail.com>.
Hi Mikko
Rinne Mikko wrote:
> Hi Paolo!
>
> Many thanks for the quick reply! Sorry if I was not clear in the first mail - it is always a compromise between compact and comprehensive. I'll try to elaborate a bit more.
>
> First, on the use case: I'm ultimately aiming at an event processor (a.k.a subscription matcher), where subscriptions are defined as SPARQL queries and events are input as N3-serialized RDF streams.
Why N3 (and not Turtle)?
Or, even better (i.e. faster) N-Triples (maybe compressed)?
> I'm trying to evaluate the suitability of Jena/ARQ for this kind of task.
Processing one SPARQL-query at a time on an N3-file works well, but next I
should understand how to move beyond that and whether this ultimate goal is
reachable (and how much new code would it take).
You might find this interesting:
http://wiki.larkc.eu/c-sparql
http://streamreasoning.org/download
It's on my long list of the things I'd like to learn more about
it and look at the sources (if available).
> This is what I'm doing now:
>
> 1) Read the triples into a model (only once :-), although I should prepare for the case where the model is a stream instead of a small file)
>
> 2) Loop through the set of SPARQL-filters (a.k.a subscriptions), pre-creating QueryExecutionFactories into an array.
>
> 3) Run the execSelects in another loop (which I'm timing to observe how long it takes to execute the queries in sequence)
Make sure you actually iterate through all the results.
ARQ streams the results when possible, so execSelects might return
instantly, but you actually have not computed all the results until
you retrieve them. I am sure you are doing this already, but it's
better to double check.
>
> ----------
>
> The next step I'd like to do, is to "load" all the queries first, and execute my model against the whole set of queries at once.
Ah... now I understand! :-)
Interesting use case/idea. It's a sort of continuous query running system
you want, isn't it? You stream events in and you want to stream new data
out to 1000s of destinations.
> Ultimately, I'd like to load all the queries, stream in N3 triple by triple and observe the performance (notification speed with a large number of subscriptions) in that scenario.
>
>> Mikko, maybe you can "payback" with a small example on how you could
>> use SPARQL CONSTRUCT queries to do small "inference" over RDF data.
>
> I'm afraid that the simple SPARQL-queries I'm using for testing at the moment would not help at all. :-( Right now I'm just using some simple filtering to give each imaginary "subscriber" their slice of the test data. Once we proceed to test with something more elaborate, I'll let you know. :-)
Now I better understand your use case.
You mentioned inference and the RETE algorithm and that confused me.
But, now I understand why you were thinking about that... it's not at
all a bad idea since many of your SPARQL queries would probably share
common patterns/parts, so you want to compute those only once.
Ymris is still there on the /Import
http://svn.apache.org/repos/asf/incubator/jena/Import/Jena-SVN/Ymris/trunk/
... but I don't think anyone is working on it actively.
You could have a look at that as well, keeping in mind that it is
"experimental", unreleased and there are no plans (that I am aware of) to
work on it (unfortunately!), so... you probably are on your own with that.
But, it's interesting!
S4.io is also in the Apache incubator now:
http://s4.io/
Paolo
>
> Thank you for the links - didn't check all yet, but I will!
>
> Best Wishes,
>
> Mikko
>
>
> On 28. Oct 2011, at 11:41 AM, Paolo Castagna wrote:
>
>> Hi Mikko,
>> I am not giving you answers to all of your questions, just a few.
>>
>> Rinne Mikko wrote:
>>> Hi!
>>> New to Jena and the list, please bear with me if this has been explained over and over again. I had so far no luck with the documentation, mailing list archives or googling, so here we go:
>>> Can ARQ be used to execute multiple parallel SPARQL queries?
>> I am not sure I follow what you are doing exactly.
>> But, are you reading a file with your data each time you execute a query?
>>
>> A better approach would be to use something like TDB to load (and index)
>> your data once and then run your queries against the data stored in TDB
>> (using ARQ as you are doing now). See: http://openjena.org/wiki/TDB
>>
>> Another option for you might be to use a SPARQL endpoint (and issue queries
>> in parallel via HTTP clients), see: http://openjena.org/wiki/Fuseki
>>
>>> I would like to configure e.g. 100 or 1000 queries and then run them against a single file of triples. I wrote a piece of code to run the queries in sequence and got suprisingly good performance with brute force, but I would expect going through the dataset only once to perform much better.
>> This is what made me think that you are parsing and reading in your data
>> from a file and doing this for each of your queries.
>>
>> If so, bad.
>>
>> Read the data once and keep it in memory if it's small.
>> Then run all your queries against that Jena Model.
>>
>>> If ARQ doesn't support this, is it the Jena forward-chaining RETE engine<http://jena.sourceforge.net/inference/> I should be looking at, and translate the SPARQL queries manually?
>> Now you got me confused.
>>
>> Because I do not understand what you are trying to achieve, I don't know
>> your use case.
>>
>> (However, it's seems something interesting... it seems to me you are
>> doing some "inference" via SPARQL and then you would like to keep data
>> up-to-date as you add/remove RDF data to/from your system).
>>
>>> Ultimately I would like to track the processing of each new triple from the dataset, in case it matches a query. Any proposals on good documentation? Which level should I try to interface on?
>> Interesting question... I'd love to have a clear and simple answer to
>> your question and a good example to point you at. But, I don't.
>>
>> A related (and recent) thread from jena-users:
>> http://markmail.org/thread/l4ymug3ujoqifnty
>>
>> You might find this example interesting and useful:
>> https://github.com/castagna/Apache-Jena-Examples/blob/master/src/main/java/org/apache/jena/examples/ExampleTDB_04.java
>> This is from GitHub, therefore it's not "official".
>> Some people learn best from examples.
>>
>> When I need to learn how to use a new software and its APIs I prefer
>> Java code and many examples (with a few comments in it).
>>
>> (If others want to contribute more small examples, go ahead: fork it
>> and send pull requests! ;-))
>>
>> Mikko, maybe you can "payback" with a small example on how you could
>> use SPARQL CONSTRUCT queries to do small "inference" over RDF data.
>>
>> Paolo
>>
>>> Thanks!
>>> Mikko
>
Re: Parallel SPARQL queries with ARQ?
Posted by Rinne Mikko <mi...@aalto.fi>.
Hi Paolo!
Many thanks for the quick reply! Sorry if I was not clear in the first mail - it is always a compromise between compact and comprehensive. I'll try to elaborate a bit more.
First, on the use case: I'm ultimately aiming at an event processor (a.k.a subscription matcher), where subscriptions are defined as SPARQL queries and events are input as N3-serialized RDF streams. I'm trying to evaluate the suitability of Jena/ARQ for this kind of task. Processing one SPARQL-query at a time on an N3-file works well, but next I should understand how to move beyond that and whether this ultimate goal is reachable (and how much new code would it take).
This is what I'm doing now:
1) Read the triples into a model (only once :-), although I should prepare for the case where the model is a stream instead of a small file)
2) Loop through the set of SPARQL-filters (a.k.a subscriptions), pre-creating QueryExecutionFactories into an array.
3) Run the execSelects in another loop (which I'm timing to observe how long it takes to execute the queries in sequence)
----------
The next step I'd like to do, is to "load" all the queries first, and execute my model against the whole set of queries at once.
Ultimately, I'd like to load all the queries, stream in N3 triple by triple and observe the performance (notification speed with a large number of subscriptions) in that scenario.
> Mikko, maybe you can "payback" with a small example on how you could
> use SPARQL CONSTRUCT queries to do small "inference" over RDF data.
I'm afraid that the simple SPARQL-queries I'm using for testing at the moment would not help at all. :-( Right now I'm just using some simple filtering to give each imaginary "subscriber" their slice of the test data. Once we proceed to test with something more elaborate, I'll let you know. :-)
Thank you for the links - didn't check all yet, but I will!
Best Wishes,
Mikko
On 28. Oct 2011, at 11:41 AM, Paolo Castagna wrote:
> Hi Mikko,
> I am not giving you answers to all of your questions, just a few.
>
> Rinne Mikko wrote:
>> Hi!
>> New to Jena and the list, please bear with me if this has been explained over and over again. I had so far no luck with the documentation, mailing list archives or googling, so here we go:
>> Can ARQ be used to execute multiple parallel SPARQL queries?
>
> I am not sure I follow what you are doing exactly.
> But, are you reading a file with your data each time you execute a query?
>
> A better approach would be to use something like TDB to load (and index)
> your data once and then run your queries against the data stored in TDB
> (using ARQ as you are doing now). See: http://openjena.org/wiki/TDB
>
> Another option for you might be to use a SPARQL endpoint (and issue queries
> in parallel via HTTP clients), see: http://openjena.org/wiki/Fuseki
>
>> I would like to configure e.g. 100 or 1000 queries and then run them against a single file of triples. I wrote a piece of code to run the queries in sequence and got suprisingly good performance with brute force, but I would expect going through the dataset only once to perform much better.
>
> This is what made me think that you are parsing and reading in your data
> from a file and doing this for each of your queries.
>
> If so, bad.
>
> Read the data once and keep it in memory if it's small.
> Then run all your queries against that Jena Model.
>
>> If ARQ doesn't support this, is it the Jena forward-chaining RETE engine<http://jena.sourceforge.net/inference/> I should be looking at, and translate the SPARQL queries manually?
>
> Now you got me confused.
>
> Because I do not understand what you are trying to achieve, I don't know
> your use case.
>
> (However, it's seems something interesting... it seems to me you are
> doing some "inference" via SPARQL and then you would like to keep data
> up-to-date as you add/remove RDF data to/from your system).
>
>> Ultimately I would like to track the processing of each new triple from the dataset, in case it matches a query. Any proposals on good documentation? Which level should I try to interface on?
>
> Interesting question... I'd love to have a clear and simple answer to
> your question and a good example to point you at. But, I don't.
>
> A related (and recent) thread from jena-users:
> http://markmail.org/thread/l4ymug3ujoqifnty
>
> You might find this example interesting and useful:
> https://github.com/castagna/Apache-Jena-Examples/blob/master/src/main/java/org/apache/jena/examples/ExampleTDB_04.java
> This is from GitHub, therefore it's not "official".
> Some people learn best from examples.
>
> When I need to learn how to use a new software and its APIs I prefer
> Java code and many examples (with a few comments in it).
>
> (If others want to contribute more small examples, go ahead: fork it
> and send pull requests! ;-))
>
> Mikko, maybe you can "payback" with a small example on how you could
> use SPARQL CONSTRUCT queries to do small "inference" over RDF data.
>
> Paolo
>
>> Thanks!
>> Mikko
>
Re: Parallel SPARQL queries with ARQ?
Posted by Paolo Castagna <ca...@googlemail.com>.
Hi Mikko,
I am not giving you answers to all of your questions, just a few.
Rinne Mikko wrote:
> Hi!
>
> New to Jena and the list, please bear with me if this has been explained over and over again. I had so far no luck with the documentation, mailing list archives or googling, so here we go:
>
> Can ARQ be used to execute multiple parallel SPARQL queries?
I am not sure I follow what you are doing exactly.
But, are you reading a file with your data each time you execute a query?
A better approach would be to use something like TDB to load (and index)
your data once and then run your queries against the data stored in TDB
(using ARQ as you are doing now). See: http://openjena.org/wiki/TDB
Another option for you might be to use a SPARQL endpoint (and issue queries
in parallel via HTTP clients), see: http://openjena.org/wiki/Fuseki
> I would like to configure e.g. 100 or 1000 queries and then run them against a single file of triples. I wrote a piece of code to run the queries in sequence and got suprisingly good performance with brute force, but I would expect going through the dataset only once to perform much better.
This is what made me think that you are parsing and reading in your data
from a file and doing this for each of your queries.
If so, bad.
Read the data once and keep it in memory if it's small.
Then run all your queries against that Jena Model.
> If ARQ doesn't support this, is it the Jena forward-chaining RETE engine<http://jena.sourceforge.net/inference/> I should be looking at, and translate the SPARQL queries manually?
Now you got me confused.
Because I do not understand what you are trying to achieve, I don't know
your use case.
(However, it's seems something interesting... it seems to me you are
doing some "inference" via SPARQL and then you would like to keep data
up-to-date as you add/remove RDF data to/from your system).
> Ultimately I would like to track the processing of each new triple from the dataset, in case it matches a query. Any proposals on good documentation? Which level should I try to interface on?
Interesting question... I'd love to have a clear and simple answer to
your question and a good example to point you at. But, I don't.
A related (and recent) thread from jena-users:
http://markmail.org/thread/l4ymug3ujoqifnty
You might find this example interesting and useful:
https://github.com/castagna/Apache-Jena-Examples/blob/master/src/main/java/org/apache/jena/examples/ExampleTDB_04.java
This is from GitHub, therefore it's not "official".
Some people learn best from examples.
When I need to learn how to use a new software and its APIs I prefer
Java code and many examples (with a few comments in it).
(If others want to contribute more small examples, go ahead: fork it
and send pull requests! ;-))
Mikko, maybe you can "payback" with a small example on how you could
use SPARQL CONSTRUCT queries to do small "inference" over RDF data.
Paolo
>
> Thanks!
>
> Mikko
>