You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Hashim Khan <en...@gmail.com> on 2020/05/06 21:59:15 UTC
Fwd: benchmark results
---------- Forwarded message ---------
From: Hashim Khan <en...@gmail.com>
Date: Wed 6 May 2020 18:12
Subject: benchmark results
To: <us...@jena.apache.org>, <us...@jena.apache.org>
Hi,
I have some benchmark results for academic purpose. Since I have to publish
these results for academic purpose, therefore, ethically I should approve
it from you. Please have a look on it and comment. The experiments are
performed for testing Jena Fuseki for throughput in the form of
Queries-per-Second (QpS) by concurrent querying users. Results are as
follows:
-----------------------------------------------------------------------------------------------------------------------
Queries-per-Second (QpS) value (on right) against number of querying user/s
on left. Dataset used in WatDiv and its corresponding generated queries.
*1 user* *16.3095428042943*
*2* *28.7767042229086*
*4* *37.7873613755524*
*8* *41.4869801140127*
*16* *40.3444765286102*
*32* *18.2336058348627*
*64* *4.99362748348325*
*128* *3.14705862217391*
-----------------------------------------------------------------------------------------------------------------------
Queries-per-Second (QpS) value (on right) against number of querying user/s
on left. Dataset used in DBpedia3.5.1, with FEASIBLE benchmark generated
queries.
*1 user* *34.2716918435139*
*2* *41.4967058060893*
*4* *46.133575439712*
*8* *90.7590012425525*
*16* *23.324348385717*
*32* *0.184894512966109*
*64* *0.000185155317588*
*128* *0.004903533091092*
-------------------------------------------------------------------------------------------------------------------------
Further details are;
WatDiv generated dataset having 108M triples source:
https://dsg.uwaterloo.ca/watdiv/
and the corresponding queries generated by Query Generator tool of the same
source.
DBpedia3.5.1 dataset having about 280M triples source: *dbpedia.org
<http://dbpedia.org>*
with FEASIBEL Benchmark queries source:
https://link.springer.com/chapter/10.1007/978-3-319-25007-6_4
Benchmark execution framwork IGUANA source:
https://github.com/dice-group/IGUANA
is used.
All experiments were performed on a machine with 32 X Intel(R) Xeon(R) CPU
@ 2.10GHz, 256GB RAM, 4TB HDD and running Ubuntu 4.18.0-25-generic.
Jena-TDB (Fuseki) Version 2.0 with a Java heap size set to 16GB,
-- I will be waiting for your reply.
Best Regards,
--
*Hashim Khan*
Re: Fwd: benchmark results
Posted by Andy Seaborne <an...@apache.org>.
On 09/05/2020 16:53, Hashim Khan wrote:
> HI,
>
> Thanks for the guidance. Its Apache Jena Fuseki 2.0 with TDB - Can you
> please tell me about the results I attached in the previous email?
100 or 200M triples should cause a high degree of caching so ocne warmed
up, many queries are on going near the disk but it's not a benchmark I'm
familiar with.
Other than the obvious point that some contention arises at about 8
threads, not much I can say. The servlet container (Tomcat, Jetty) may
have a limit on the number of real threads.
If this is running in Tomcat, maybe (speculation) there is an active
real thread limit there.
Fuseki 2.0 was a lonng time ago but the source is available to you.
Andy
>
>
>
> On Thu, May 7, 2020 at 12:47 PM Andy Seaborne <an...@apache.org> wrote:
>
>> Hi Hashim,
>>
>> > Jena-TDB (Fuseki) Version 2.0 with a Java heap size set to 16GB,
>>
>> When you reference Jena or Fuseki the first time, it should be the full
>> form such as "Apache Jena Fuseki"; after that you can use the shorter
>> name "Fuseki" (this is the general conventional way to handle
>> trademarks, full name first and the full name is "Apache Jena").
>>
>> For version of Fuseki (there is the now removed Fuseki1 and Fuseki2,
>> which is what people usually mean bu Fuseki), if you really do mean
>> Fuseki2 2.0 -- which was release 2015-03-08 -- it's some way behind.
>>
>> The current on is Fuseki2 version 3.14.0. We joined the versioning of
>> Fuseki to the rest of Jena.
>>
>> Then you should say which TDB - we have TDB1 (often written just TDB)
>> and TDB2.
>>
>> Thanks for checking and I hope these details will help you identify
>> which code you are benchmarking.
>>
>> Andy
>>
>> On 06/05/2020 22:59, Hashim Khan wrote:
>>> ---------- Forwarded message ---------
>>> From: Hashim Khan <en...@gmail.com>
>>> Date: Wed 6 May 2020 18:12
>>> Subject: benchmark results
>>> To: <us...@jena.apache.org>, <us...@jena.apache.org>
>>>
>>>
>>> Hi,
>>> I have some benchmark results for academic purpose. Since I have to
>> publish
>>> these results for academic purpose, therefore, ethically I should approve
>>> it from you. Please have a look on it and comment. The experiments are
>>> performed for testing Jena Fuseki for throughput in the form of
>>> Queries-per-Second (QpS) by concurrent querying users. Results are as
>>> follows:
>>>
>> -----------------------------------------------------------------------------------------------------------------------
>>> Queries-per-Second (QpS) value (on right) against number of querying
>> user/s
>>> on left. Dataset used in WatDiv and its corresponding generated queries.
>>>
>>>
>>> *1 user* *16.3095428042943*
>>> *2* *28.7767042229086*
>>> *4* *37.7873613755524*
>>> *8* *41.4869801140127*
>>> *16* *40.3444765286102*
>>> *32* *18.2336058348627*
>>> *64* *4.99362748348325*
>>> *128* *3.14705862217391*
>>>
>> -----------------------------------------------------------------------------------------------------------------------
>>> Queries-per-Second (QpS) value (on right) against number of querying
>> user/s
>>> on left. Dataset used in DBpedia3.5.1, with FEASIBLE benchmark generated
>>> queries.
>>>
>>> *1 user* *34.2716918435139*
>>> *2* *41.4967058060893*
>>> *4* *46.133575439712*
>>> *8* *90.7590012425525*
>>> *16* *23.324348385717*
>>> *32* *0.184894512966109*
>>> *64* *0.000185155317588*
>>> *128* *0.004903533091092*
>>>
>> -------------------------------------------------------------------------------------------------------------------------
>>> Further details are;
>>>
>>> WatDiv generated dataset having 108M triples source:
>>> https://dsg.uwaterloo.ca/watdiv/
>>> and the corresponding queries generated by Query Generator tool of the
>> same
>>> source.
>>>
>>> DBpedia3.5.1 dataset having about 280M triples source: *dbpedia.org
>>> <http://dbpedia.org>*
>>> with FEASIBEL Benchmark queries source:
>>> https://link.springer.com/chapter/10.1007/978-3-319-25007-6_4
>>>
>>> Benchmark execution framwork IGUANA source:
>>> https://github.com/dice-group/IGUANA
>>> is used.
>>>
>>> All experiments were performed on a machine with 32 X Intel(R) Xeon(R)
>> CPU
>>> @ 2.10GHz, 256GB RAM, 4TB HDD and running Ubuntu 4.18.0-25-generic.
>>>
>>> Jena-TDB (Fuseki) Version 2.0 with a Java heap size set to 16GB,
>>>
>>> -- I will be waiting for your reply.
>>>
>>>
>>>
>>> Best Regards,
>>>
>>
>
>
Re: Fwd: benchmark results
Posted by Hashim Khan <en...@gmail.com>.
HI,
Thanks for the guidance. Its Apache Jena Fuseki 2.0 with TDB - Can you
please tell me about the results I attached in the previous email?
On Thu, May 7, 2020 at 12:47 PM Andy Seaborne <an...@apache.org> wrote:
> Hi Hashim,
>
> > Jena-TDB (Fuseki) Version 2.0 with a Java heap size set to 16GB,
>
> When you reference Jena or Fuseki the first time, it should be the full
> form such as "Apache Jena Fuseki"; after that you can use the shorter
> name "Fuseki" (this is the general conventional way to handle
> trademarks, full name first and the full name is "Apache Jena").
>
> For version of Fuseki (there is the now removed Fuseki1 and Fuseki2,
> which is what people usually mean bu Fuseki), if you really do mean
> Fuseki2 2.0 -- which was release 2015-03-08 -- it's some way behind.
>
> The current on is Fuseki2 version 3.14.0. We joined the versioning of
> Fuseki to the rest of Jena.
>
> Then you should say which TDB - we have TDB1 (often written just TDB)
> and TDB2.
>
> Thanks for checking and I hope these details will help you identify
> which code you are benchmarking.
>
> Andy
>
> On 06/05/2020 22:59, Hashim Khan wrote:
> > ---------- Forwarded message ---------
> > From: Hashim Khan <en...@gmail.com>
> > Date: Wed 6 May 2020 18:12
> > Subject: benchmark results
> > To: <us...@jena.apache.org>, <us...@jena.apache.org>
> >
> >
> > Hi,
> > I have some benchmark results for academic purpose. Since I have to
> publish
> > these results for academic purpose, therefore, ethically I should approve
> > it from you. Please have a look on it and comment. The experiments are
> > performed for testing Jena Fuseki for throughput in the form of
> > Queries-per-Second (QpS) by concurrent querying users. Results are as
> > follows:
> >
> -----------------------------------------------------------------------------------------------------------------------
> > Queries-per-Second (QpS) value (on right) against number of querying
> user/s
> > on left. Dataset used in WatDiv and its corresponding generated queries.
> >
> >
> > *1 user* *16.3095428042943*
> > *2* *28.7767042229086*
> > *4* *37.7873613755524*
> > *8* *41.4869801140127*
> > *16* *40.3444765286102*
> > *32* *18.2336058348627*
> > *64* *4.99362748348325*
> > *128* *3.14705862217391*
> >
> -----------------------------------------------------------------------------------------------------------------------
> > Queries-per-Second (QpS) value (on right) against number of querying
> user/s
> > on left. Dataset used in DBpedia3.5.1, with FEASIBLE benchmark generated
> > queries.
> >
> > *1 user* *34.2716918435139*
> > *2* *41.4967058060893*
> > *4* *46.133575439712*
> > *8* *90.7590012425525*
> > *16* *23.324348385717*
> > *32* *0.184894512966109*
> > *64* *0.000185155317588*
> > *128* *0.004903533091092*
> >
> -------------------------------------------------------------------------------------------------------------------------
> > Further details are;
> >
> > WatDiv generated dataset having 108M triples source:
> > https://dsg.uwaterloo.ca/watdiv/
> > and the corresponding queries generated by Query Generator tool of the
> same
> > source.
> >
> > DBpedia3.5.1 dataset having about 280M triples source: *dbpedia.org
> > <http://dbpedia.org>*
> > with FEASIBEL Benchmark queries source:
> > https://link.springer.com/chapter/10.1007/978-3-319-25007-6_4
> >
> > Benchmark execution framwork IGUANA source:
> > https://github.com/dice-group/IGUANA
> > is used.
> >
> > All experiments were performed on a machine with 32 X Intel(R) Xeon(R)
> CPU
> > @ 2.10GHz, 256GB RAM, 4TB HDD and running Ubuntu 4.18.0-25-generic.
> >
> > Jena-TDB (Fuseki) Version 2.0 with a Java heap size set to 16GB,
> >
> > -- I will be waiting for your reply.
> >
> >
> >
> > Best Regards,
> >
>
--
*Hashim Khan*
Re: Fwd: benchmark results
Posted by Andy Seaborne <an...@apache.org>.
Hi Hashim,
> Jena-TDB (Fuseki) Version 2.0 with a Java heap size set to 16GB,
When you reference Jena or Fuseki the first time, it should be the full
form such as "Apache Jena Fuseki"; after that you can use the shorter
name "Fuseki" (this is the general conventional way to handle
trademarks, full name first and the full name is "Apache Jena").
For version of Fuseki (there is the now removed Fuseki1 and Fuseki2,
which is what people usually mean bu Fuseki), if you really do mean
Fuseki2 2.0 -- which was release 2015-03-08 -- it's some way behind.
The current on is Fuseki2 version 3.14.0. We joined the versioning of
Fuseki to the rest of Jena.
Then you should say which TDB - we have TDB1 (often written just TDB)
and TDB2.
Thanks for checking and I hope these details will help you identify
which code you are benchmarking.
Andy
On 06/05/2020 22:59, Hashim Khan wrote:
> ---------- Forwarded message ---------
> From: Hashim Khan <en...@gmail.com>
> Date: Wed 6 May 2020 18:12
> Subject: benchmark results
> To: <us...@jena.apache.org>, <us...@jena.apache.org>
>
>
> Hi,
> I have some benchmark results for academic purpose. Since I have to publish
> these results for academic purpose, therefore, ethically I should approve
> it from you. Please have a look on it and comment. The experiments are
> performed for testing Jena Fuseki for throughput in the form of
> Queries-per-Second (QpS) by concurrent querying users. Results are as
> follows:
> -----------------------------------------------------------------------------------------------------------------------
> Queries-per-Second (QpS) value (on right) against number of querying user/s
> on left. Dataset used in WatDiv and its corresponding generated queries.
>
>
> *1 user* *16.3095428042943*
> *2* *28.7767042229086*
> *4* *37.7873613755524*
> *8* *41.4869801140127*
> *16* *40.3444765286102*
> *32* *18.2336058348627*
> *64* *4.99362748348325*
> *128* *3.14705862217391*
> -----------------------------------------------------------------------------------------------------------------------
> Queries-per-Second (QpS) value (on right) against number of querying user/s
> on left. Dataset used in DBpedia3.5.1, with FEASIBLE benchmark generated
> queries.
>
> *1 user* *34.2716918435139*
> *2* *41.4967058060893*
> *4* *46.133575439712*
> *8* *90.7590012425525*
> *16* *23.324348385717*
> *32* *0.184894512966109*
> *64* *0.000185155317588*
> *128* *0.004903533091092*
> -------------------------------------------------------------------------------------------------------------------------
> Further details are;
>
> WatDiv generated dataset having 108M triples source:
> https://dsg.uwaterloo.ca/watdiv/
> and the corresponding queries generated by Query Generator tool of the same
> source.
>
> DBpedia3.5.1 dataset having about 280M triples source: *dbpedia.org
> <http://dbpedia.org>*
> with FEASIBEL Benchmark queries source:
> https://link.springer.com/chapter/10.1007/978-3-319-25007-6_4
>
> Benchmark execution framwork IGUANA source:
> https://github.com/dice-group/IGUANA
> is used.
>
> All experiments were performed on a machine with 32 X Intel(R) Xeon(R) CPU
> @ 2.10GHz, 256GB RAM, 4TB HDD and running Ubuntu 4.18.0-25-generic.
>
> Jena-TDB (Fuseki) Version 2.0 with a Java heap size set to 16GB,
>
> -- I will be waiting for your reply.
>
>
>
> Best Regards,
>
Re: benchmark results
Posted by Luis Enrique Ramos García <lu...@googlemail.com.INVALID>.
Hi Hashim,
Thanks for the information, could ou please tell me the size of the data
set in bytes,
Luis
El mié., 6 may. 2020 a las 23:59, Hashim Khan (<en...@gmail.com>)
escribió:
> ---------- Forwarded message ---------
> From: Hashim Khan <en...@gmail.com>
> Date: Wed 6 May 2020 18:12
> Subject: benchmark results
> To: <us...@jena.apache.org>, <us...@jena.apache.org>
>
>
> Hi,
> I have some benchmark results for academic purpose. Since I have to publish
> these results for academic purpose, therefore, ethically I should approve
> it from you. Please have a look on it and comment. The experiments are
> performed for testing Jena Fuseki for throughput in the form of
> Queries-per-Second (QpS) by concurrent querying users. Results are as
> follows:
>
> -----------------------------------------------------------------------------------------------------------------------
> Queries-per-Second (QpS) value (on right) against number of querying user/s
> on left. Dataset used in WatDiv and its corresponding generated queries.
>
>
> *1 user* *16.3095428042943*
> *2* *28.7767042229086*
> *4* *37.7873613755524*
> *8* *41.4869801140127*
> *16* *40.3444765286102*
> *32* *18.2336058348627*
> *64* *4.99362748348325*
> *128* *3.14705862217391*
>
> -----------------------------------------------------------------------------------------------------------------------
> Queries-per-Second (QpS) value (on right) against number of querying user/s
> on left. Dataset used in DBpedia3.5.1, with FEASIBLE benchmark generated
> queries.
>
> *1 user* *34.2716918435139*
> *2* *41.4967058060893*
> *4* *46.133575439712*
> *8* *90.7590012425525*
> *16* *23.324348385717*
> *32* *0.184894512966109*
> *64* *0.000185155317588*
> *128* *0.004903533091092*
>
> -------------------------------------------------------------------------------------------------------------------------
> Further details are;
>
> WatDiv generated dataset having 108M triples source:
> https://dsg.uwaterloo.ca/watdiv/
> and the corresponding queries generated by Query Generator tool of the same
> source.
>
> DBpedia3.5.1 dataset having about 280M triples source: *dbpedia.org
> <http://dbpedia.org>*
> with FEASIBEL Benchmark queries source:
> https://link.springer.com/chapter/10.1007/978-3-319-25007-6_4
>
> Benchmark execution framwork IGUANA source:
> https://github.com/dice-group/IGUANA
> is used.
>
> All experiments were performed on a machine with 32 X Intel(R) Xeon(R) CPU
> @ 2.10GHz, 256GB RAM, 4TB HDD and running Ubuntu 4.18.0-25-generic.
>
> Jena-TDB (Fuseki) Version 2.0 with a Java heap size set to 16GB,
>
> -- I will be waiting for your reply.
>
>
>
> Best Regards,
> --
> *Hashim Khan*
>