You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Ankur Padia <pa...@gmail.com> on 2015/08/24 15:28:08 UTC
Clearing cache does not effect query execution time
Hello everyone,
I want to execute a bunch of queries on Fuseki server using default
settings and want to clear cache. To accomplish the goal mentioned, I used
following linux command to clean the cache,
*sync; echo 3 | sudo tee /proc/sys/vm/drop_caches*
However, instead of getting execution time in seconds which means results
was fetched from harddisk, I am getting results in millisecond .
Also, the very first query takes time in seconds however the later queries
following it takes few millisecond both having same number of triple
pattern.
Can anyone please guide me through the phenomena happening inside Fuseki ?
Fuseki was hosted on linux, Fedora 17.
Ankur Padia.
Re: Clearing cache does not effect query execution time
Posted by Rob Vesse <rv...@dotnetrdf.org>.
You already received two replies from Jena developers to this question
which you originally sent to the users@jena.apache.org mailing list so
sending it again to the dev list is unlikely to elicit any further
responses
If those replies didn't answer your question then please explain why they
didn't answer the question?
Rob
On 03/09/2015 16:00, "Ankur Padia" <pa...@gmail.com> wrote:
>---------- Forwarded message ----------
>From: Ankur Padia <pa...@gmail.com>
>Date: Mon, Aug 24, 2015 at 9:28 AM
>Subject: Clearing cache does not effect query execution time
>To: users@jena.apache.org
>
>
>Hello everyone,
>
> I want to execute a bunch of queries on Fuseki server using default
>settings and want to clear cache. To accomplish the goal mentioned, I used
>following linux command to clean the cache,
>
>*sync; echo 3 | sudo tee /proc/sys/vm/drop_caches*
>
>However, instead of getting execution time in seconds which means results
>was fetched from harddisk, I am getting results in millisecond .
>
>Also, the very first query takes time in seconds however the later queries
>following it takes few millisecond both having same number of triple
>pattern.
>
>Can anyone please guide me through the phenomena happening inside Fuseki ?
>Fuseki was hosted on linux, Fedora 17.
>
>
>Ankur Padia.
Fwd: Clearing cache does not effect query execution time
Posted by Ankur Padia <pa...@gmail.com>.
---------- Forwarded message ----------
From: Ankur Padia <pa...@gmail.com>
Date: Mon, Aug 24, 2015 at 9:28 AM
Subject: Clearing cache does not effect query execution time
To: users@jena.apache.org
Hello everyone,
I want to execute a bunch of queries on Fuseki server using default
settings and want to clear cache. To accomplish the goal mentioned, I used
following linux command to clean the cache,
*sync; echo 3 | sudo tee /proc/sys/vm/drop_caches*
However, instead of getting execution time in seconds which means results
was fetched from harddisk, I am getting results in millisecond .
Also, the very first query takes time in seconds however the later queries
following it takes few millisecond both having same number of triple
pattern.
Can anyone please guide me through the phenomena happening inside Fuseki ?
Fuseki was hosted on linux, Fedora 17.
Ankur Padia.
Re: Clearing cache does not effect query execution time
Posted by Andy Seaborne <an...@apache.org>.
There is another important cache - the node table cache is not a filing
system cache at all. It's in-process, in Java, cache. It caches what
is otherwise random I/O and potentially expensive.
Depends on what your queries are as to which caching effects dominate
execution.
SELECT * { ?s ?p ?o } is almost all node table - the index is a scan and
very read-friendly even when cold. This is especially true if the
database has had little in the way of updates since first built - the
loaders generate some indexes, like the one that query will use, in
large, disk aligned units.
The only way to get reliable (and realistic - servers run for a long
time) results is to pre-warm the system as Rob says.
Andy
On 24/08/15 15:09, Rob Vesse wrote:
> Comments inline:
>
> On 24/08/2015 14:28, "Ankur Padia" <pa...@gmail.com> wrote:
>
>> Hello everyone,
>>
>> I want to execute a bunch of queries on Fuseki server using default
>> settings and want to clear cache. To accomplish the goal mentioned, I used
>> following linux command to clean the cache,
>>
>> *sync; echo 3 | sudo tee /proc/sys/vm/drop_caches*
>>
>> However, instead of getting execution time in seconds which means results
>> was fetched from harddisk, I am getting results in millisecond .
>
> TDB the storage layer used underneath Fuseki by default uses memory mapped
> files for its own caching which is separate and distinct from any OS level
> caching
>
> Dropping the OS caches thus has no effect because TDB does not rely on OS
> caching
>
>
>>
>> Also, the very first query takes time in seconds however the later queries
>> following it takes few millisecond both having same number of triple
>> pattern.
>
> There is also the issue of JVM class loading, the first few queries will
> likely cause many classes to be loaded by the JVM which is a one time
> start up cost paid by any Java application.
>
> If you are trying to do benchmarking then you should always run warm-ups
> to bring the system to a hot state since running from a cold state is
> always likely to hit discrepancies like this and won't give you
> reproducible figures
>
> Rob
>
>>
>> Can anyone please guide me through the phenomena happening inside Fuseki ?
>> Fuseki was hosted on linux, Fedora 17.
>>
>>
>> Ankur Padia.
>
>
>
>
Re: Clearing cache does not effect query execution time
Posted by Rob Vesse <rv...@dotnetrdf.org>.
Comments inline:
On 24/08/2015 14:28, "Ankur Padia" <pa...@gmail.com> wrote:
>Hello everyone,
>
> I want to execute a bunch of queries on Fuseki server using default
>settings and want to clear cache. To accomplish the goal mentioned, I used
>following linux command to clean the cache,
>
>*sync; echo 3 | sudo tee /proc/sys/vm/drop_caches*
>
>However, instead of getting execution time in seconds which means results
>was fetched from harddisk, I am getting results in millisecond .
TDB the storage layer used underneath Fuseki by default uses memory mapped
files for its own caching which is separate and distinct from any OS level
caching
Dropping the OS caches thus has no effect because TDB does not rely on OS
caching
>
>Also, the very first query takes time in seconds however the later queries
>following it takes few millisecond both having same number of triple
>pattern.
There is also the issue of JVM class loading, the first few queries will
likely cause many classes to be loaded by the JVM which is a one time
start up cost paid by any Java application.
If you are trying to do benchmarking then you should always run warm-ups
to bring the system to a hot state since running from a cold state is
always likely to hit discrepancies like this and won't give you
reproducible figures
Rob
>
>Can anyone please guide me through the phenomena happening inside Fuseki ?
>Fuseki was hosted on linux, Fedora 17.
>
>
>Ankur Padia.