You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Christian Schwaderer <c_...@hotmail.com> on 2015/04/07 16:31:19 UTC

Randomness and performance

Dear all,

ok, first things first: It's great that there is a free RDF/SPARQL framework like Apache enabling beginners like me to build up there own RDF application!

However, I'm not quite content with the documentation. I finally found out, that TDB is (probably) the component I was looking for. Despite there being a "Getting started" section I had to search extern sources to get my first steps done. Only this epic Stack Overflow answer http://stackoverflow.com/questions/16608265/querying-open-data-communities-data-with-sparql/16610663#16610663 finally informed me about two important things: That I first have to load my data into TDB via the command tdbloader and then query it with tdbquery. It sounds trivial, but if you do not know that, you can hardly figure out that by yourself. So, a section "First steps with TDB" would be very usefull.
Also, I'd suggest expanding the documention at least on these two commands, using the information available with "--help" and including some examples. For instance, on tdbloader2 the documentation says: "It only accepts the --loc argument and a list of files to load". Honestely, I haven't understood yet, what the "loc" argument is all about (just used it without knowing what I'm doing). Even "list of files" is not 100% clear without further explanation or examples, because at least I don't know which format the list of files should be in: white space separated, comma separated, colon separated?
Please don't get me wrong: These are just suggestions not demands. I know that writing documentions is sometimes anoying. But good documentations really help beginners.

Furthermore, I have two questions:

1. Is there any implemation of (pseudo)randomness in Jena? The SPARQL functions (sort by) rand() and sample() return in most cases the very same triple (out of hundreds). A more random outcome would be very handy in my case.

2. After having loaded my data to TDB (about 170 00 triples and 7MB) with tdbloader I now query this data in my web application using PHP and the shell_exec function giving the query as a string. Every query, however simple, takes more than one second to be executed, which is a little bit too much. Is there any way to improve the performance? Would using Fuseki help?

Thanks in advance and best regards
Christian
 		 	   		  

Re: Randomness and performance

Posted by Andy Seaborne <an...@apache.org>.
On 07/04/15 15:31, Christian Schwaderer wrote:
> Dear all,
>
> ok, first things first: It's great that there is a free RDF/SPARQL framework like Apache enabling beginners like me to build up there own RDF application!
>
> However, I'm not quite content with the documentation. I finally found out, that TDB is (probably) the component I was looking for. Despite there being a "Getting started" section I had to search extern sources to get my first steps done. Only this epic Stack Overflow answer http://stackoverflow.com/questions/16608265/querying-open-data-communities-data-with-sparql/16610663#16610663 finally informed me about two important things: That I first have to load my data into TDB via the command tdbloader and then query it with tdbquery. It sounds trivial, but if you do not know that, you can hardly figure out that by yourself. So, a section "First steps with TDB" would be very usefull.
> Also, I'd suggest expanding the documention at least on these two commands, using the information available with "--help" and including some examples. For instance, on tdbloader2 the documentation says: "It only accepts the --loc argument and a list of files to load". Honestely, I haven't understood yet, what the "loc" argument is all about (just used it without knowing what I'm doing). Even "list of files" is not 100% clear without further explanation or examples, because at least I don't know which format the list of files should be in: white space separated, comma separated, colon separated?
> Please don't get me wrong: These are just suggestions not demands. I know that writing documentions is sometimes anoying. But good documentations really help beginners.
>
> Furthermore, I have two questions:
>
> 1. Is there any implemation of (pseudo)randomness in Jena? The SPARQL
> functions  (sort by) rand() and sample() return in most cases the very same triple
> (out of hundreds). A more random outcome would be very handy in my case.

Yes - SAMPLE is deterministic with the group.

RAND() is a new random number (as generated by java secure random number 
generator) but sorting by it is not possible.  (The sort isn't stable).

Assign a RAND to each row and use MAX to pick it out.

> 2. After having loaded my data to TDB (about 170 00 triples and 7MB)
> with tdbloader I now query this data in my web application using PHP
> and the shell_exec function giving the query as a string. Every
> query, however simple, takes more than one second to be executed,
> which is a little bit too much. Is there any way to improve the
> performance? Would using Fuseki help?

Yes, Fuseki will help.

Calling shell_exec is starting the database from cold every time.  No 
caching - everything needed by the query is read from disk.  Starting 
the JVM is slow as well.

	Andy

>
> Thanks in advance and best regards
> Christian
>   		 	   		
>