You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@deltaspike.apache.org by Schäfer,
Johannes <js...@psi.de> on 2017/06/01 09:35:53 UTC
Performance of DeltaSpike Data
Hi,
My company is thinking about using DeltaSpike Data. But before we integrate this into our development I was asked to prepare some benchmarks, comparing the usage of DeltaSpike Data with the usage of a plain EntityManager.
I prepared some benchmarks and I was surprised that there is a big difference in the write performance. I already got some hints in the delta spike irc channel, but the performance is still bad.
Based on a template from os890 I implemented my tests and prepared a github project.
https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ds_project_template
Basically I'm talking about this test:
https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ds_project_template/blob/master/src/test/java/de/psi/metals/futurelab/repo/benchmark/SaveTest.java
It just saves an entity into a DB in a loop. Depending of the number of iterations the difference is quite big.
SaveTest
_____________________________________________________________________________________________________________________________________________________
| | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 | iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter 10240 |
|====================================================================================================================================================|
| DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639| 0.043319612| 0.281807839| 1.308948835| 1.370535533| 8.186996818| 20.920141274| 93.631768475|
| EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958| 0.028243393| 0.035484616| 0.038600595| 0.088904458| 0.339158674| 0.745850523 | 0.853543234 |
Also the difference between a run with 5120 and 10240 iteration is not just the double amount of time, it is more than 4 times more.
Do you have some hints to me what I'm doing wrong there?
Regards
Johannes
Re: Performance of DeltaSpike Data
Posted by Mark Struberg <st...@yahoo.de.INVALID>.
Would be cool if we can get this into 1.8.1?
txs and LieGrue,
strub
> Am 15.06.2017 um 16:02 schrieb Thomas Andraschko <an...@gmail.com>:
>
> Jfyi:
> I think the performance of the data module is really good but we can
> improve the proxy module.
> If the performance is better with owb in your case, the performance will
> likely be much better with weld after i optimzed the proxy module as we can
> remove the dynamic bean lookups in the proxy and cache the bean references.
> Will likely do it in 1-2 weeks.
>
> Am Donnerstag, 15. Juni 2017 schrieb Thomas Andraschko :
>
>> Hi,
>>
>> It would be great if you could try your sample with OWB instead of weld.
>> I always to such tests with OWB.
>>
>> Regards,
>> Thomas
>>
>> Am Donnerstag, 15. Juni 2017 schrieb Schäfer, Johannes :
>>
>>> Hi,
>>>
>>> Thanks for your investigation. I tried it today again with the last
>>> version from Github. For my test setup I still have the same numbers. So in
>>> average DeltaSpike needs 3 times longer that a pure EM implementation for
>>> finding entities by a query.
>>> For now I stopped my evaluation of DeltaSpike Data. My personal feeling
>>> is quite mixed. I really like to concept of DeltaSpike Data, but the
>>> performance loss is sometimes quite too big.
>>> I hope that there will be in future some improvements.
>>>
>>> Regards
>>> Johannes
>>>
>>> -----Original Message-----
>>> From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
>>> Sent: Friday, June 9, 2017 4:40 PM
>>> To: users@deltaspike.apache.org
>>> Subject: Re: Performance of DeltaSpike Data
>>>
>>> FYI:
>>>
>>> i created a similar example based on the unittests in the ds data module.
>>> Here is the result of 100.000 selects, running on tomee and hsqldb:
>>>
>>> DS: 2600ms
>>> JPA: 1800ms
>>>
>>>
>>> 200-300ms are taken by our interceptor lookup, which can be improved a
>>> bit by caching the information about interceptors in the proxy instance
>>> instead of doing a bean+cache lookup each time.
>>> another 200-300ms are taken by DS Data directly - not sure if we can
>>> improve it further.
>>>
>>> Not sure about the other 200-400ms.
>>>
>>> 2017-06-08 16:33 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
>>>
>>>> No chances with @ApplicationScoped.
>>>> To run my test just run "mvn clean install -PEmbeddedTestWeld".
>>>> It is using a in memory H2 DB. Maybe pipe the output into a log to
>>>> find the test results in the output.
>>>>
>>>> Grüße
>>>> Johannes
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
>>>> Sent: Thursday, June 8, 2017 3:12 PM
>>>> To: users@deltaspike.apache.org
>>>> Subject: Re: Performance of DeltaSpike Data
>>>>
>>>> Could you also try to make your repository @ApplicationScoped?
>>>>
>>>> How can i run your tests? I don't like to install a AS or database ;)
>>>>
>>>> 2017-06-08 13:28 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
>>>>
>>>>> Same with a locally compile version.
>>>>> Mysql:
>>>>> ____________________________________________________________
>>>>> ____________________________________________________________
>>>>> _____________________________
>>>>> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
>>>>> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
>>>>> 10240 |
>>>>> |===========================================================
>>>>> ============================================================
>>>>> =============================|
>>>>> | DS| 0.028479446| 0.088102335| 0.138260967| 0.255074252|
>>>>> | DS| 0.385907351|
>>>>> 0.734428279| 1.836123535| 4.125717222| 6.175937816| 13.217757392|
>>>>> 25.372525787|
>>>>> | EM| 0.010955534| 0.020851247| 0.041094277| 0.076565573|
>>>>> | EM| 0.195617863|
>>>>> 0.386509868| 0.812829151| 1.4044238 | 3.007676477| 6.232350452 |
>>>>> 11.726264467|
>>>>>
>>>>> Grüße
>>>>> Johannes
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
>>>>> Sent: Thursday, June 8, 2017 1:12 PM
>>>>> To: users@deltaspike.apache.org
>>>>> Subject: Re: Performance of DeltaSpike Data
>>>>>
>>>>> please build DS from source, i don't think that SNAPSHOT is up to
>>> date.
>>>>>
>>>>> 2017-06-08 13:05 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Thanks for your great support. Now I had the time to run the tests.
>>>>>> Unfortunately no improvement. :-(
>>>>>> I used Mysql and H2 and both still have a significant difference.
>>>>>> mysql:
>>>>>> ____________________________________________________________
>>>>>> ____________________________________________________________
>>>>>> _____________________________
>>>>>> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
>>> |
>>>>>> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
>>> iter
>>>>>> 10240 |
>>>>>> |===========================================================
>>>>>> ============================================================
>>>>>> =============================|
>>>>>> | DS| 0.042993818| 0.070756327| 0.139015158| 0.249963317|
>>>>>> | DS| 0.489673972|
>>>>>> 1.000932095| 1.418196146| 3.396942334| 6.268094687| 12.142304859|
>>>>>> 24.631240985|
>>>>>> | EM| 0.016741971| 0.034018415| 0.042539175| 0.097203944|
>>>>>> | EM| 0.15662194
>>>>>> | EM| |
>>>>>> 0.32694476 | 0.665341891| 1.582051703| 2.602520533| 5.710082816 |
>>>>>> 10.856276852|
>>>>>>
>>>>>> h2:
>>>>>> ____________________________________________________________
>>>>>> ____________________________________________________________
>>>>>> __________________________
>>>>>> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
>>> |
>>>>>> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
>>>>> 10240 |
>>>>>> |===========================================================
>>>>>> ============================================================
>>>>>> ==========================|
>>>>>> | DS| 0.007259847| 0.009839833| 0.024182004| 0.040194493|
>>>>>> | DS| 0.037355253|
>>>>>> 0.038992501| 0.15695646| 0.157184542| 0.277937182| 0.575950893|
>>>>>> 0.848326297|
>>>>>> | EM| 7.12756E-4 | 7.15797E-4 | 0.0035079 | 0.001897262|
>>>>>> | EM| 0.003144109|
>>>>>> 0.007000594| 0.01269694| 0.024183904| 0.037443446| 0.108577248|
>>>>>> 0.217664259|
>>>>>>
>>>>>> I used version 1.8.1-SNAPSHOT for testing.
>>>>>> See https://github.com/johannesschaefer/javaee_jsf_
>>>>>> cdi_jpa_data_ds_project_template
>>>>>>
>>>>>> Grüße
>>>>>> Johannes
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
>>>>>> Sent: Thursday, June 8, 2017 10:36 AM
>>>>>> To: users@deltaspike.apache.org
>>>>>> Subject: Re: Performance of DeltaSpike Data
>>>>>>
>>>>>> I did a major improvement and in my tests, both plain JPA and DS
>>>>>> Data are now very similar.
>>>>>> Would be great if you could provide the new numbers.
>>>>>>
>>>>>> 2017-06-07 14:33 GMT+02:00 Thomas Andraschko
>>>>>> <andraschko.thomas@gmail.com
>>>>>>> :
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> could you please try to run your test again against the github
>>>> master?
>>>>>>> I already did a small improvement and refactored a little bit on
>>>>>>> the weekend.
>>>>>>>
>>>>>>> 2017-06-06 8:54 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> So after the a long weekend, I'm back with my results.
>>>>>>>> For the write, findByPK and findAll tests I get now good numbers.
>>>>>>>> See:
>>>>>>>> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
>>>>>>>> ds_project_template/blob/master/src/test/java/de/psi/
>>>>>>>> metals/futurelab/repo/benchmark/SaveTest.java
>>>>>>>> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
>>>>>>>> ds_project_template/blob/master/src/test/java/de/psi/
>>>>>>>> metals/futurelab/repo/benchmark/ReadTest.java
>>>>>>>> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
>>>>>>>> ds_project_template/blob/master/src/test/java/de/psi/
>>>>>>>> metals/futurelab/repo/benchmark/ReadAllTest.java
>>>>>>>>
>>>>>>>> The difference between delta spike and plain EM are just a few
>>>>>>>> percent, in both directions ;-) .
>>>>>>>>
>>>>>>>> But I wrote a new test case were I try to find entities by an
>>> query.
>>>>>>>> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
>>>>>>>> ds_project_template/blob/master/src/test/java/de/psi/
>>>>>>>> metals/futurelab/repo/benchmark/ReadQueryTest.java
>>>>>>>> So I compare
>>>>>>>> TypedQuery< Material > query = eml.createQuery(
>>>>>>>> "SELECT m FROM Material m WHERE grade = :grade
>>>>>>>> AND width = :width AND thickness = :thickness",
>>>>>>>> Material.class );
>>>>>>>> query.setParameter( "grade", "AAA" );
>>>>>>>> query.setParameter( "width", 5 );
>>>>>>>> query.setParameter( "thickness", 5. ); List<
>>>>>>>> Material
>>>>>>>>> mats = query.getResultList();
>>>>>>>>
>>>>>>>> to
>>>>>>>> List< Material > mats =
>>>>>>>> matRepo.findByGradeAndWidthAndThickness(
>>>>>>>> "AAA", 5, 5. );
>>>>>>>>
>>>>>>>> Here again the difference is quite high.
>>>>>>>> | | iter 10 | iter 20 | iter 40 | iter 80 | iter
>>> 160
>>>> |
>>>>>>>> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
>>>> iter
>>>>>>>> 10240 |
>>>>>>>> |===========================================================
>>>>>>>> ============================================================
>>>>>>>> =============================|
>>>>>>>> | DS| 0.03988012 | 0.151870613| 0.144881044| 0.270389952|
>>>>>>>> | DS| 0.526700787|
>>>>>>>> 1.023574545| 1.806960223| 3.426772405| 6.969935385|
>>>>>>>> 13.963582543| 26.785764953|
>>>>>>>> | EM| 0.010984804| 0.021940339| 0.059921297| 0.087386918|
>>>>>>>> | EM| 0.171045079|
>>>>>>>> 0.375059016| 0.747171594| 1.560946145| 2.968347174| 6.446844753
>>>>>>>> | 12.361550486|
>>>>>>>>
>>>>>>>> So as you can see the DeltaSpike implementation needs at least
>>>>>>>> the double amount of time.
>>>>>>>>
>>>>>>>> Any hints for improving the performance?
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Johannes
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Schäfer, Johannes [mailto:jschaefer@psi.de]
>>>>>>>> Sent: Thursday, June 1, 2017 2:27 PM
>>>>>>>> To: users@deltaspike.apache.org
>>>>>>>> Subject: RE: Performance of DeltaSpike Data
>>>>>>>>
>>>>>>>> Right. Copy and paste error.
>>>>>>>> I added also a flush to the EM test.
>>>>>>>> Now I have similar numbers.
>>>>>>>> ____________________________________________________________
>>>>>>>> ____________________________________________________________
>>>>>>>> ______________________________
>>>>>>>> | | iter 10 | iter 20 | iter 40 | iter 80 | iter
>>> 160
>>>> |
>>>>>>>> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
>>>> iter
>>>>>>>> 10240 |
>>>>>>>> |==============================================================
>>>>>>>> |==
>>>>>>>> |==
>>>>>>>> |==
>>>>>>>> |===
>>>>>>>> |==============================================================
>>>>>>>> |==
>>>>>>>> |==
>>>>>>>> |==
>>>>>>>> |===
>>>>>>>> |=======|
>>>>>>>> | DS| 0.001588214| 0.004130191| 0.007351854| 0.014062036|
>>>>>>>> | DS| 0.048373222| 0.593463008| 0.741351593| 1.697058004|
>>>>>>>> | DS| 6.049719288| 34.101109279| 101.589077365|
>>>>>>>> | EM| 0.001385601| 0.002662861| 0.004092937| 0.108730649|
>>>>>>>> | EM| 0.046299193| 0.106900289| 0.461147505| 1.688040769|
>>>>>>>> | EM| 5.960683928| 25.604583163| 106.688041149|
>>>>>>>>
>>>>>>>> It's a little bit strange for me, why I have to compare
>>>>>>>> EntityPersistenceRepository.save with a em.persist + em.flush.
>>>>>>>> I would expect that an simple EntityPersistenceRepository.save
>>>>>>>> don't have a flush (there is a separate
>>> EntityPersistenceRepository.
>>>>>> saveAndFlush).
>>>>>>>>
>>>>>>>> When I do a run with EntityPersistenceRepository.saveAndFlush I
>>>>>>>> get the following numbers.
>>>>>>>> ____________________________________________________________
>>>>>>>> ____________________________________________________________
>>>>>>>> ______________________________
>>>>>>>> | | iter 10 | iter 20 | iter 40 | iter 80 | iter
>>> 160
>>>> |
>>>>>>>> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
>>>> iter
>>>>>>>> 10240 |
>>>>>>>> |==============================================================
>>>>>>>> |==
>>>>>>>> |==
>>>>>>>> |==
>>>>>>>> |===
>>>>>>>> |==============================================================
>>>>>>>> |==
>>>>>>>> |==
>>>>>>>> |==
>>>>>>>> |===
>>>>>>>> |=======|
>>>>>>>> | DS| 0.001703015| 0.003457728| 0.008079817| 0.019099994|
>>>>>>>> | DS| 0.053865065| 0.940319597| 0.643245399| 2.292716685|
>>>>>>>> | DS| 9.902395587| 40.84301017 | 172.179435413|
>>>>>>>> | EM| 0.001677545| 0.004168205| 0.005779986| 0.014491211|
>>>>>>>> | EM| 0.031066334| 0.110747277| 0.4051742 | 1.925682412|
>>>>>>>> | EM| 5.842606084| 23.540393571| 132.817886521|
>>>>>>>>
>>>>>>>> So I have the feeling that there is still something wrong.
>>>>>>>>
>>>>>>>> Thanks to Gerhard for his additional hints.
>>>>>>>> I committed all changes to the github repo.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Johannes
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Gerhard Petracek [mailto:gpetracek@apache.org]
>>>>>>>> Sent: Thursday, June 1, 2017 1:21 PM
>>>>>>>> To: users@deltaspike.apache.org
>>>>>>>> Subject: Re: Performance of DeltaSpike Data
>>>>>>>>
>>>>>>>> @johannes:
>>>>>>>> as mentioned yesterday you have to move EntityTransaction#begin
>>>>>>>> and EntityTransaction#commit into the for-loop.
>>>>>>>>
>>>>>>>> regards,
>>>>>>>> gerhard
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> 2017-06-01 12:58 GMT+02:00 Thomas Andraschko
>>>>>>>> <andraschko.thomas@gmail.com
>>>>>>>>> :
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> ~1 year ago i did many optimizations in the data module and
>>>>>>>>> AFAIR DS Data was only a little bit slower.
>>>>>>>>> After i compared my testcase with a benchmark from a user,
>>>>>>>>> the big different came from the transaction handling which
>>>>>>>>> was different in both testcases.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Thomas
>>>>>>>>>
>>>>>>>>> 2017-06-01 12:33 GMT+02:00 Gerhard Petracek
>>>>>>>>> <gpetracek@apache.org
>>>>> :
>>>>>>>>>
>>>>>>>>>> hi johannes,
>>>>>>>>>>
>>>>>>>>>> after refactoring your initial code to ds-test-control i
>>>>>>>>>> saw
>>>> e.g.
>>>>>>>>>> ~6s vs 7,5s for 2560 iterations.
>>>>>>>>>> i'll compare my local version with your new version
>>>>>>>>>> (mentioned in your mail).
>>>>>>>>>>
>>>>>>>>>> regards,
>>>>>>>>>> gerhard
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes
>>>>>>>>>> <jschaefer@psi.de
>>>>> :
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> My company is thinking about using DeltaSpike Data. But
>>>>>>>>>>> before we integrate this into our development I was asked
>>>>>>>>>>> to prepare some
>>>>>>>>>> benchmarks,
>>>>>>>>>>> comparing the usage of DeltaSpike Data with the usage of
>>>>>>>>>>> a plain EntityManager.
>>>>>>>>>>> I prepared some benchmarks and I was surprised that there
>>>>>>>>>>> is a big difference in the write performance. I already
>>>>>>>>>>> got some hints in the
>>>>>>>>>> delta
>>>>>>>>>>> spike irc channel, but the performance is still bad.
>>>>>>>>>>> Based on a template from os890 I implemented my tests and
>>>>>>>>>>> prepared a github project.
>>>>>>>>>>> https://github.com/johannesschaefer/javaee_jsf_
>>>>>>>>> cdi_jpa_data_ds_project_
>>>>>>>>>>> template
>>>>>>>>>>> Basically I'm talking about this test:
>>>>>>>>>>> https://github.com/johannesschaefer/javaee_jsf_
>>>>>>>>> cdi_jpa_data_ds_project_
>>>>>>>>>>> template/blob/master/src/test/java/de/psi/metals/futurela
>>>>>>>>>>> b/ repo/benchmark/SaveTest.java
>>>>>>>>>>>
>>>>>>>>>>> It just saves an entity into a DB in a loop. Depending of
>>>>>>>>>>> the number of iterations the difference is quite big.
>>>>>>>>>>>
>>>>>>>>>>> SaveTest
>>>>>>>>>>> _________________________________________________________
>>>>>>>>>>> __
>>>>>>>>>>> _
>>>>>>>>>>> _________________________________________________________
>>>>>>>>>>> __ _ _____________________________
>>>>>>>>>>> | | iter 10 | iter 20 | iter 40 | iter 80 |
>>> iter
>>>>> 160
>>>>>>>> |
>>>>>>>>>>> iter 320 | iter 640 | iter 1280 | iter 2560 | iter
>>> 5120
>>>>> |
>>>>>>>> iter
>>>>>>>>>>> 10240 |
>>>>>>>>>>> |========================================================
>>>>>>>>>>> |==
>>>>>>>>>>> |=
>>>>>>>>>>> =========================================================
>>>>>>>>>>> == = =============================|
>>>>>>>>>>> | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639|
>>>>>>>>>>> | DS| 0.043319612|
>>>>>>>>>>> 0.281807839| 1.308948835| 1.370535533| 8.186996818|
>>>>>>>>>>> 20.920141274| 93.631768475|
>>>>>>>>>>> | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958|
>>>>>>>>>>> | EM| 0.028243393|
>>>>>>>>>>> 0.035484616| 0.038600595| 0.088904458| 0.339158674|
>>>>>>>>>>> 0.745850523
>>>>>>>>>>> |
>>>>>>>>>>> 0.853543234 |
>>>>>>>>>>>
>>>>>>>>>>> Also the difference between a run with 5120 and 10240
>>>>>>>>>>> iteration is not just the double amount of time, it is
>>>>>>>>>>> more than 4 times
>>>>>> more.
>>>>>>>>>>>
>>>>>>>>>>> Do you have some hints to me what I'm doing wrong there?
>>>>>>>>>>>
>>>>>>>>>>> Regards
>>>>>>>>>>> Johannes
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
Re: Performance of DeltaSpike Data
Posted by Thomas Andraschko <an...@gmail.com>.
Jfyi:
I think the performance of the data module is really good but we can
improve the proxy module.
If the performance is better with owb in your case, the performance will
likely be much better with weld after i optimzed the proxy module as we can
remove the dynamic bean lookups in the proxy and cache the bean references.
Will likely do it in 1-2 weeks.
Am Donnerstag, 15. Juni 2017 schrieb Thomas Andraschko :
> Hi,
>
> It would be great if you could try your sample with OWB instead of weld.
> I always to such tests with OWB.
>
> Regards,
> Thomas
>
> Am Donnerstag, 15. Juni 2017 schrieb Schäfer, Johannes :
>
>> Hi,
>>
>> Thanks for your investigation. I tried it today again with the last
>> version from Github. For my test setup I still have the same numbers. So in
>> average DeltaSpike needs 3 times longer that a pure EM implementation for
>> finding entities by a query.
>> For now I stopped my evaluation of DeltaSpike Data. My personal feeling
>> is quite mixed. I really like to concept of DeltaSpike Data, but the
>> performance loss is sometimes quite too big.
>> I hope that there will be in future some improvements.
>>
>> Regards
>> Johannes
>>
>> -----Original Message-----
>> From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
>> Sent: Friday, June 9, 2017 4:40 PM
>> To: users@deltaspike.apache.org
>> Subject: Re: Performance of DeltaSpike Data
>>
>> FYI:
>>
>> i created a similar example based on the unittests in the ds data module.
>> Here is the result of 100.000 selects, running on tomee and hsqldb:
>>
>> DS: 2600ms
>> JPA: 1800ms
>>
>>
>> 200-300ms are taken by our interceptor lookup, which can be improved a
>> bit by caching the information about interceptors in the proxy instance
>> instead of doing a bean+cache lookup each time.
>> another 200-300ms are taken by DS Data directly - not sure if we can
>> improve it further.
>>
>> Not sure about the other 200-400ms.
>>
>> 2017-06-08 16:33 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
>>
>> > No chances with @ApplicationScoped.
>> > To run my test just run "mvn clean install -PEmbeddedTestWeld".
>> > It is using a in memory H2 DB. Maybe pipe the output into a log to
>> > find the test results in the output.
>> >
>> > Grüße
>> > Johannes
>> >
>> >
>> > -----Original Message-----
>> > From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
>> > Sent: Thursday, June 8, 2017 3:12 PM
>> > To: users@deltaspike.apache.org
>> > Subject: Re: Performance of DeltaSpike Data
>> >
>> > Could you also try to make your repository @ApplicationScoped?
>> >
>> > How can i run your tests? I don't like to install a AS or database ;)
>> >
>> > 2017-06-08 13:28 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
>> >
>> > > Same with a locally compile version.
>> > > Mysql:
>> > > ____________________________________________________________
>> > > ____________________________________________________________
>> > > _____________________________
>> > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
>> > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
>> > > 10240 |
>> > > |===========================================================
>> > > ============================================================
>> > > =============================|
>> > > | DS| 0.028479446| 0.088102335| 0.138260967| 0.255074252|
>> > > | DS| 0.385907351|
>> > > 0.734428279| 1.836123535| 4.125717222| 6.175937816| 13.217757392|
>> > > 25.372525787|
>> > > | EM| 0.010955534| 0.020851247| 0.041094277| 0.076565573|
>> > > | EM| 0.195617863|
>> > > 0.386509868| 0.812829151| 1.4044238 | 3.007676477| 6.232350452 |
>> > > 11.726264467|
>> > >
>> > > Grüße
>> > > Johannes
>> > >
>> > >
>> > > -----Original Message-----
>> > > From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
>> > > Sent: Thursday, June 8, 2017 1:12 PM
>> > > To: users@deltaspike.apache.org
>> > > Subject: Re: Performance of DeltaSpike Data
>> > >
>> > > please build DS from source, i don't think that SNAPSHOT is up to
>> date.
>> > >
>> > > 2017-06-08 13:05 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
>> > >
>> > > > Hi,
>> > > >
>> > > > Thanks for your great support. Now I had the time to run the tests.
>> > > > Unfortunately no improvement. :-(
>> > > > I used Mysql and H2 and both still have a significant difference.
>> > > > mysql:
>> > > > ____________________________________________________________
>> > > > ____________________________________________________________
>> > > > _____________________________
>> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
>> |
>> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
>> iter
>> > > > 10240 |
>> > > > |===========================================================
>> > > > ============================================================
>> > > > =============================|
>> > > > | DS| 0.042993818| 0.070756327| 0.139015158| 0.249963317|
>> > > > | DS| 0.489673972|
>> > > > 1.000932095| 1.418196146| 3.396942334| 6.268094687| 12.142304859|
>> > > > 24.631240985|
>> > > > | EM| 0.016741971| 0.034018415| 0.042539175| 0.097203944|
>> > > > | EM| 0.15662194
>> > > > | EM| |
>> > > > 0.32694476 | 0.665341891| 1.582051703| 2.602520533| 5.710082816 |
>> > > > 10.856276852|
>> > > >
>> > > > h2:
>> > > > ____________________________________________________________
>> > > > ____________________________________________________________
>> > > > __________________________
>> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
>> |
>> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
>> > > 10240 |
>> > > > |===========================================================
>> > > > ============================================================
>> > > > ==========================|
>> > > > | DS| 0.007259847| 0.009839833| 0.024182004| 0.040194493|
>> > > > | DS| 0.037355253|
>> > > > 0.038992501| 0.15695646| 0.157184542| 0.277937182| 0.575950893|
>> > > > 0.848326297|
>> > > > | EM| 7.12756E-4 | 7.15797E-4 | 0.0035079 | 0.001897262|
>> > > > | EM| 0.003144109|
>> > > > 0.007000594| 0.01269694| 0.024183904| 0.037443446| 0.108577248|
>> > > > 0.217664259|
>> > > >
>> > > > I used version 1.8.1-SNAPSHOT for testing.
>> > > > See https://github.com/johannesschaefer/javaee_jsf_
>> > > > cdi_jpa_data_ds_project_template
>> > > >
>> > > > Grüße
>> > > > Johannes
>> > > >
>> > > > -----Original Message-----
>> > > > From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
>> > > > Sent: Thursday, June 8, 2017 10:36 AM
>> > > > To: users@deltaspike.apache.org
>> > > > Subject: Re: Performance of DeltaSpike Data
>> > > >
>> > > > I did a major improvement and in my tests, both plain JPA and DS
>> > > > Data are now very similar.
>> > > > Would be great if you could provide the new numbers.
>> > > >
>> > > > 2017-06-07 14:33 GMT+02:00 Thomas Andraschko
>> > > > <andraschko.thomas@gmail.com
>> > > > >:
>> > > >
>> > > > > Hi,
>> > > > >
>> > > > > could you please try to run your test again against the github
>> > master?
>> > > > > I already did a small improvement and refactored a little bit on
>> > > > > the weekend.
>> > > > >
>> > > > > 2017-06-06 8:54 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
>> > > > >
>> > > > >> Hi,
>> > > > >>
>> > > > >> So after the a long weekend, I'm back with my results.
>> > > > >> For the write, findByPK and findAll tests I get now good numbers.
>> > > > >> See:
>> > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
>> > > > >> ds_project_template/blob/master/src/test/java/de/psi/
>> > > > >> metals/futurelab/repo/benchmark/SaveTest.java
>> > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
>> > > > >> ds_project_template/blob/master/src/test/java/de/psi/
>> > > > >> metals/futurelab/repo/benchmark/ReadTest.java
>> > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
>> > > > >> ds_project_template/blob/master/src/test/java/de/psi/
>> > > > >> metals/futurelab/repo/benchmark/ReadAllTest.java
>> > > > >>
>> > > > >> The difference between delta spike and plain EM are just a few
>> > > > >> percent, in both directions ;-) .
>> > > > >>
>> > > > >> But I wrote a new test case were I try to find entities by an
>> query.
>> > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
>> > > > >> ds_project_template/blob/master/src/test/java/de/psi/
>> > > > >> metals/futurelab/repo/benchmark/ReadQueryTest.java
>> > > > >> So I compare
>> > > > >> TypedQuery< Material > query = eml.createQuery(
>> > > > >> "SELECT m FROM Material m WHERE grade = :grade
>> > > > >> AND width = :width AND thickness = :thickness",
>> > > > >> Material.class );
>> > > > >> query.setParameter( "grade", "AAA" );
>> > > > >> query.setParameter( "width", 5 );
>> > > > >> query.setParameter( "thickness", 5. ); List<
>> > > > >> Material
>> > > > >> > mats = query.getResultList();
>> > > > >>
>> > > > >> to
>> > > > >> List< Material > mats =
>> > > > >> matRepo.findByGradeAndWidthAndThickness(
>> > > > >> "AAA", 5, 5. );
>> > > > >>
>> > > > >> Here again the difference is quite high.
>> > > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter
>> 160
>> > |
>> > > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
>> > iter
>> > > > >> 10240 |
>> > > > >> |===========================================================
>> > > > >> ============================================================
>> > > > >> =============================|
>> > > > >> | DS| 0.03988012 | 0.151870613| 0.144881044| 0.270389952|
>> > > > >> | DS| 0.526700787|
>> > > > >> 1.023574545| 1.806960223| 3.426772405| 6.969935385|
>> > > > >> 13.963582543| 26.785764953|
>> > > > >> | EM| 0.010984804| 0.021940339| 0.059921297| 0.087386918|
>> > > > >> | EM| 0.171045079|
>> > > > >> 0.375059016| 0.747171594| 1.560946145| 2.968347174| 6.446844753
>> > > > >> | 12.361550486|
>> > > > >>
>> > > > >> So as you can see the DeltaSpike implementation needs at least
>> > > > >> the double amount of time.
>> > > > >>
>> > > > >> Any hints for improving the performance?
>> > > > >>
>> > > > >> Regards,
>> > > > >> Johannes
>> > > > >>
>> > > > >> -----Original Message-----
>> > > > >> From: Schäfer, Johannes [mailto:jschaefer@psi.de]
>> > > > >> Sent: Thursday, June 1, 2017 2:27 PM
>> > > > >> To: users@deltaspike.apache.org
>> > > > >> Subject: RE: Performance of DeltaSpike Data
>> > > > >>
>> > > > >> Right. Copy and paste error.
>> > > > >> I added also a flush to the EM test.
>> > > > >> Now I have similar numbers.
>> > > > >> ____________________________________________________________
>> > > > >> ____________________________________________________________
>> > > > >> ______________________________
>> > > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter
>> 160
>> > |
>> > > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
>> > iter
>> > > > >> 10240 |
>> > > > >> |==============================================================
>> > > > >> |==
>> > > > >> |==
>> > > > >> |==
>> > > > >> |===
>> > > > >> |==============================================================
>> > > > >> |==
>> > > > >> |==
>> > > > >> |==
>> > > > >> |===
>> > > > >> |=======|
>> > > > >> | DS| 0.001588214| 0.004130191| 0.007351854| 0.014062036|
>> > > > >> | DS| 0.048373222| 0.593463008| 0.741351593| 1.697058004|
>> > > > >> | DS| 6.049719288| 34.101109279| 101.589077365|
>> > > > >> | EM| 0.001385601| 0.002662861| 0.004092937| 0.108730649|
>> > > > >> | EM| 0.046299193| 0.106900289| 0.461147505| 1.688040769|
>> > > > >> | EM| 5.960683928| 25.604583163| 106.688041149|
>> > > > >>
>> > > > >> It's a little bit strange for me, why I have to compare
>> > > > >> EntityPersistenceRepository.save with a em.persist + em.flush.
>> > > > >> I would expect that an simple EntityPersistenceRepository.save
>> > > > >> don't have a flush (there is a separate
>> EntityPersistenceRepository.
>> > > > saveAndFlush).
>> > > > >>
>> > > > >> When I do a run with EntityPersistenceRepository.saveAndFlush I
>> > > > >> get the following numbers.
>> > > > >> ____________________________________________________________
>> > > > >> ____________________________________________________________
>> > > > >> ______________________________
>> > > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter
>> 160
>> > |
>> > > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
>> > iter
>> > > > >> 10240 |
>> > > > >> |==============================================================
>> > > > >> |==
>> > > > >> |==
>> > > > >> |==
>> > > > >> |===
>> > > > >> |==============================================================
>> > > > >> |==
>> > > > >> |==
>> > > > >> |==
>> > > > >> |===
>> > > > >> |=======|
>> > > > >> | DS| 0.001703015| 0.003457728| 0.008079817| 0.019099994|
>> > > > >> | DS| 0.053865065| 0.940319597| 0.643245399| 2.292716685|
>> > > > >> | DS| 9.902395587| 40.84301017 | 172.179435413|
>> > > > >> | EM| 0.001677545| 0.004168205| 0.005779986| 0.014491211|
>> > > > >> | EM| 0.031066334| 0.110747277| 0.4051742 | 1.925682412|
>> > > > >> | EM| 5.842606084| 23.540393571| 132.817886521|
>> > > > >>
>> > > > >> So I have the feeling that there is still something wrong.
>> > > > >>
>> > > > >> Thanks to Gerhard for his additional hints.
>> > > > >> I committed all changes to the github repo.
>> > > > >>
>> > > > >> Regards,
>> > > > >> Johannes
>> > > > >>
>> > > > >> -----Original Message-----
>> > > > >> From: Gerhard Petracek [mailto:gpetracek@apache.org]
>> > > > >> Sent: Thursday, June 1, 2017 1:21 PM
>> > > > >> To: users@deltaspike.apache.org
>> > > > >> Subject: Re: Performance of DeltaSpike Data
>> > > > >>
>> > > > >> @johannes:
>> > > > >> as mentioned yesterday you have to move EntityTransaction#begin
>> > > > >> and EntityTransaction#commit into the for-loop.
>> > > > >>
>> > > > >> regards,
>> > > > >> gerhard
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >> 2017-06-01 12:58 GMT+02:00 Thomas Andraschko
>> > > > >> <andraschko.thomas@gmail.com
>> > > > >> >:
>> > > > >>
>> > > > >> > Hi,
>> > > > >> >
>> > > > >> > ~1 year ago i did many optimizations in the data module and
>> > > > >> > AFAIR DS Data was only a little bit slower.
>> > > > >> > After i compared my testcase with a benchmark from a user,
>> > > > >> > the big different came from the transaction handling which
>> > > > >> > was different in both testcases.
>> > > > >> >
>> > > > >> > Regards,
>> > > > >> > Thomas
>> > > > >> >
>> > > > >> > 2017-06-01 12:33 GMT+02:00 Gerhard Petracek
>> > > > >> > <gpetracek@apache.org
>> > >:
>> > > > >> >
>> > > > >> > > hi johannes,
>> > > > >> > >
>> > > > >> > > after refactoring your initial code to ds-test-control i
>> > > > >> > > saw
>> > e.g.
>> > > > >> > > ~6s vs 7,5s for 2560 iterations.
>> > > > >> > > i'll compare my local version with your new version
>> > > > >> > > (mentioned in your mail).
>> > > > >> > >
>> > > > >> > > regards,
>> > > > >> > > gerhard
>> > > > >> > >
>> > > > >> > >
>> > > > >> > >
>> > > > >> > > 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes
>> > > > >> > > <jschaefer@psi.de
>> > >:
>> > > > >> > >
>> > > > >> > > > Hi,
>> > > > >> > > >
>> > > > >> > > > My company is thinking about using DeltaSpike Data. But
>> > > > >> > > > before we integrate this into our development I was asked
>> > > > >> > > > to prepare some
>> > > > >> > > benchmarks,
>> > > > >> > > > comparing the usage of DeltaSpike Data with the usage of
>> > > > >> > > > a plain EntityManager.
>> > > > >> > > > I prepared some benchmarks and I was surprised that there
>> > > > >> > > > is a big difference in the write performance. I already
>> > > > >> > > > got some hints in the
>> > > > >> > > delta
>> > > > >> > > > spike irc channel, but the performance is still bad.
>> > > > >> > > > Based on a template from os890 I implemented my tests and
>> > > > >> > > > prepared a github project.
>> > > > >> > > > https://github.com/johannesschaefer/javaee_jsf_
>> > > > >> > cdi_jpa_data_ds_project_
>> > > > >> > > > template
>> > > > >> > > > Basically I'm talking about this test:
>> > > > >> > > > https://github.com/johannesschaefer/javaee_jsf_
>> > > > >> > cdi_jpa_data_ds_project_
>> > > > >> > > > template/blob/master/src/test/java/de/psi/metals/futurela
>> > > > >> > > > b/ repo/benchmark/SaveTest.java
>> > > > >> > > >
>> > > > >> > > > It just saves an entity into a DB in a loop. Depending of
>> > > > >> > > > the number of iterations the difference is quite big.
>> > > > >> > > >
>> > > > >> > > > SaveTest
>> > > > >> > > > _________________________________________________________
>> > > > >> > > > __
>> > > > >> > > > _
>> > > > >> > > > _________________________________________________________
>> > > > >> > > > __ _ _____________________________
>> > > > >> > > > | | iter 10 | iter 20 | iter 40 | iter 80 |
>> iter
>> > > 160
>> > > > >> |
>> > > > >> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter
>> 5120
>> > > |
>> > > > >> iter
>> > > > >> > > > 10240 |
>> > > > >> > > > |========================================================
>> > > > >> > > > |==
>> > > > >> > > > |=
>> > > > >> > > > =========================================================
>> > > > >> > > > == = =============================|
>> > > > >> > > > | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639|
>> > > > >> > > > | DS| 0.043319612|
>> > > > >> > > > 0.281807839| 1.308948835| 1.370535533| 8.186996818|
>> > > > >> > > > 20.920141274| 93.631768475|
>> > > > >> > > > | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958|
>> > > > >> > > > | EM| 0.028243393|
>> > > > >> > > > 0.035484616| 0.038600595| 0.088904458| 0.339158674|
>> > > > >> > > > 0.745850523
>> > > > >> > > > |
>> > > > >> > > > 0.853543234 |
>> > > > >> > > >
>> > > > >> > > > Also the difference between a run with 5120 and 10240
>> > > > >> > > > iteration is not just the double amount of time, it is
>> > > > >> > > > more than 4 times
>> > > > more.
>> > > > >> > > >
>> > > > >> > > > Do you have some hints to me what I'm doing wrong there?
>> > > > >> > > >
>> > > > >> > > > Regards
>> > > > >> > > > Johannes
>> > > > >> > > >
>> > > > >> > > >
>> > > > >> > > >
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
RE: Performance of DeltaSpike Data
Posted by Schäfer,
Johannes <js...@psi.de>.
Hi,
Just tested with OWB.
OWB is faster, but there is still an overhead of 2.5 (instead of 3).
Here are my results:
| | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 | iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter 10240 |
|====================================================================================================================================================|
| DS| 0.029778038| 0.068785804| 0.17412749 | 0.386697864| 0.349159707| 0.7658348 | 1.664042074| 2.957301103| 5.538809715| 12.854707047| 22.860419748|
| EM| 0.011113307| 0.018919042| 0.071415593| 0.080011252| 0.238572129| 0.309746715| 0.6318897 | 1.533790385| 2.543436744| 5.104184549 | 10.577758559|
Grüße
Johannes
-----Original Message-----
From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
Sent: Thursday, June 15, 2017 3:35 PM
To: users@deltaspike.apache.org
Subject: Re: Performance of DeltaSpike Data
Hi,
It would be great if you could try your sample with OWB instead of weld.
I always to such tests with OWB.
Regards,
Thomas
Am Donnerstag, 15. Juni 2017 schrieb Schäfer, Johannes :
> Hi,
>
> Thanks for your investigation. I tried it today again with the last
> version from Github. For my test setup I still have the same numbers.
> So in average DeltaSpike needs 3 times longer that a pure EM
> implementation for finding entities by a query.
> For now I stopped my evaluation of DeltaSpike Data. My personal
> feeling is quite mixed. I really like to concept of DeltaSpike Data,
> but the performance loss is sometimes quite too big.
> I hope that there will be in future some improvements.
>
> Regards
> Johannes
>
> -----Original Message-----
> From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com
> <javascript:;> ]
> Sent: Friday, June 9, 2017 4:40 PM
> To: users@deltaspike.apache.org <javascript:;>
> Subject: Re: Performance of DeltaSpike Data
>
> FYI:
>
> i created a similar example based on the unittests in the ds data module.
> Here is the result of 100.000 selects, running on tomee and hsqldb:
>
> DS: 2600ms
> JPA: 1800ms
>
>
> 200-300ms are taken by our interceptor lookup, which can be improved a
> bit by caching the information about interceptors in the proxy
> instance instead of doing a bean+cache lookup each time.
> another 200-300ms are taken by DS Data directly - not sure if we can
> improve it further.
>
> Not sure about the other 200-400ms.
>
> 2017-06-08 16:33 GMT+02:00 Schäfer, Johannes <jschaefer@psi.de
> <javascript:;>>:
>
> > No chances with @ApplicationScoped.
> > To run my test just run "mvn clean install -PEmbeddedTestWeld".
> > It is using a in memory H2 DB. Maybe pipe the output into a log to
> > find the test results in the output.
> >
> > Grüße
> > Johannes
> >
> >
> > -----Original Message-----
> > From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com
> <javascript:;>]
> > Sent: Thursday, June 8, 2017 3:12 PM
> > To: users@deltaspike.apache.org <javascript:;>
> > Subject: Re: Performance of DeltaSpike Data
> >
> > Could you also try to make your repository @ApplicationScoped?
> >
> > How can i run your tests? I don't like to install a AS or database
> > ;)
> >
> > 2017-06-08 13:28 GMT+02:00 Schäfer, Johannes <jschaefer@psi.de
> <javascript:;>>:
> >
> > > Same with a locally compile version.
> > > Mysql:
> > > ____________________________________________________________
> > > ____________________________________________________________
> > > _____________________________
> > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > > 10240 |
> > > |===========================================================
> > > ============================================================
> > > =============================|
> > > | DS| 0.028479446| 0.088102335| 0.138260967| 0.255074252|
> > > | DS| 0.385907351|
> > > 0.734428279| 1.836123535| 4.125717222| 6.175937816| 13.217757392|
> > > 25.372525787|
> > > | EM| 0.010955534| 0.020851247| 0.041094277| 0.076565573|
> > > | EM| 0.195617863|
> > > 0.386509868| 0.812829151| 1.4044238 | 3.007676477| 6.232350452 |
> > > 11.726264467|
> > >
> > > Grüße
> > > Johannes
> > >
> > >
> > > -----Original Message-----
> > > From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com
> <javascript:;>]
> > > Sent: Thursday, June 8, 2017 1:12 PM
> > > To: users@deltaspike.apache.org <javascript:;>
> > > Subject: Re: Performance of DeltaSpike Data
> > >
> > > please build DS from source, i don't think that SNAPSHOT is up to date.
> > >
> > > 2017-06-08 13:05 GMT+02:00 Schäfer, Johannes <jschaefer@psi.de
> <javascript:;>>:
> > >
> > > > Hi,
> > > >
> > > > Thanks for your great support. Now I had the time to run the tests.
> > > > Unfortunately no improvement. :-( I used Mysql and H2 and both
> > > > still have a significant difference.
> > > > mysql:
> > > > ____________________________________________________________
> > > > ____________________________________________________________
> > > > _____________________________
> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
> |
> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
> iter
> > > > 10240 |
> > > > |===========================================================
> > > > ============================================================
> > > > =============================|
> > > > | DS| 0.042993818| 0.070756327| 0.139015158| 0.249963317|
> > > > | DS| 0.489673972|
> > > > 1.000932095| 1.418196146| 3.396942334| 6.268094687|
> > > > 12.142304859| 24.631240985|
> > > > | EM| 0.016741971| 0.034018415| 0.042539175| 0.097203944|
> > > > | EM| 0.15662194
> > > > | EM| |
> > > > 0.32694476 | 0.665341891| 1.582051703| 2.602520533| 5.710082816
> > > > | 10.856276852|
> > > >
> > > > h2:
> > > > ____________________________________________________________
> > > > ____________________________________________________________
> > > > __________________________
> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
> |
> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > > 10240 |
> > > > |===========================================================
> > > > ============================================================
> > > > ==========================|
> > > > | DS| 0.007259847| 0.009839833| 0.024182004| 0.040194493|
> > > > | DS| 0.037355253|
> > > > 0.038992501| 0.15695646| 0.157184542| 0.277937182| 0.575950893|
> > > > 0.848326297|
> > > > | EM| 7.12756E-4 | 7.15797E-4 | 0.0035079 | 0.001897262|
> > > > | EM| 0.003144109|
> > > > 0.007000594| 0.01269694| 0.024183904| 0.037443446| 0.108577248|
> > > > 0.217664259|
> > > >
> > > > I used version 1.8.1-SNAPSHOT for testing.
> > > > See https://github.com/johannesschaefer/javaee_jsf_
> > > > cdi_jpa_data_ds_project_template
> > > >
> > > > Grüße
> > > > Johannes
> > > >
> > > > -----Original Message-----
> > > > From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com
> <javascript:;>]
> > > > Sent: Thursday, June 8, 2017 10:36 AM
> > > > To: users@deltaspike.apache.org <javascript:;>
> > > > Subject: Re: Performance of DeltaSpike Data
> > > >
> > > > I did a major improvement and in my tests, both plain JPA and DS
> > > > Data are now very similar.
> > > > Would be great if you could provide the new numbers.
> > > >
> > > > 2017-06-07 14:33 GMT+02:00 Thomas Andraschko
> > > > <andraschko.thomas@gmail.com <javascript:;>
> > > > >:
> > > >
> > > > > Hi,
> > > > >
> > > > > could you please try to run your test again against the github
> > master?
> > > > > I already did a small improvement and refactored a little bit
> > > > > on the weekend.
> > > > >
> > > > > 2017-06-06 8:54 GMT+02:00 Schäfer, Johannes <jschaefer@psi.de
> <javascript:;>>:
> > > > >
> > > > >> Hi,
> > > > >>
> > > > >> So after the a long weekend, I'm back with my results.
> > > > >> For the write, findByPK and findAll tests I get now good numbers.
> > > > >> See:
> > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > > >> metals/futurelab/repo/benchmark/SaveTest.java
> > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > > >> metals/futurelab/repo/benchmark/ReadTest.java
> > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > > >> metals/futurelab/repo/benchmark/ReadAllTest.java
> > > > >>
> > > > >> The difference between delta spike and plain EM are just a
> > > > >> few percent, in both directions ;-) .
> > > > >>
> > > > >> But I wrote a new test case were I try to find entities by an
> query.
> > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > > >> metals/futurelab/repo/benchmark/ReadQueryTest.java
> > > > >> So I compare
> > > > >> TypedQuery< Material > query = eml.createQuery(
> > > > >> "SELECT m FROM Material m WHERE grade =
> > > > >> :grade AND width = :width AND thickness = :thickness",
> > > > >> Material.class );
> > > > >> query.setParameter( "grade", "AAA" );
> > > > >> query.setParameter( "width", 5 );
> > > > >> query.setParameter( "thickness", 5. ); List<
> > > > >> Material
> > > > >> > mats = query.getResultList();
> > > > >>
> > > > >> to
> > > > >> List< Material > mats =
> > > > >> matRepo.findByGradeAndWidthAndThickness(
> > > > >> "AAA", 5, 5. );
> > > > >>
> > > > >> Here again the difference is quite high.
> > > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
> > |
> > > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
> > iter
> > > > >> 10240 |
> > > > >> |===========================================================
> > > > >> ============================================================
> > > > >> =============================|
> > > > >> | DS| 0.03988012 | 0.151870613| 0.144881044| 0.270389952|
> > > > >> | DS| 0.526700787|
> > > > >> 1.023574545| 1.806960223| 3.426772405| 6.969935385|
> > > > >> 13.963582543| 26.785764953|
> > > > >> | EM| 0.010984804| 0.021940339| 0.059921297| 0.087386918|
> > > > >> | EM| 0.171045079|
> > > > >> 0.375059016| 0.747171594| 1.560946145| 2.968347174|
> > > > >> 6.446844753
> > > > >> | 12.361550486|
> > > > >>
> > > > >> So as you can see the DeltaSpike implementation needs at
> > > > >> least the double amount of time.
> > > > >>
> > > > >> Any hints for improving the performance?
> > > > >>
> > > > >> Regards,
> > > > >> Johannes
> > > > >>
> > > > >> -----Original Message-----
> > > > >> From: Schäfer, Johannes [mailto:jschaefer@psi.de
> > > > >> <javascript:;>]
> > > > >> Sent: Thursday, June 1, 2017 2:27 PM
> > > > >> To: users@deltaspike.apache.org <javascript:;>
> > > > >> Subject: RE: Performance of DeltaSpike Data
> > > > >>
> > > > >> Right. Copy and paste error.
> > > > >> I added also a flush to the EM test.
> > > > >> Now I have similar numbers.
> > > > >> ____________________________________________________________
> > > > >> ____________________________________________________________
> > > > >> ______________________________
> > > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
> > |
> > > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
> > iter
> > > > >> 10240 |
> > > > >> |============================================================
> > > > >> |==
> > > > >> |==
> > > > >> |==
> > > > >> |==
> > > > >> |===
> > > > >> |============================================================
> > > > >> |==
> > > > >> |==
> > > > >> |==
> > > > >> |==
> > > > >> |===
> > > > >> |=======|
> > > > >> | DS| 0.001588214| 0.004130191| 0.007351854| 0.014062036|
> > > > >> | DS| 0.048373222| 0.593463008| 0.741351593| 1.697058004|
> > > > >> | DS| 6.049719288| 34.101109279| 101.589077365|
> > > > >> | EM| 0.001385601| 0.002662861| 0.004092937| 0.108730649|
> > > > >> | EM| 0.046299193| 0.106900289| 0.461147505| 1.688040769|
> > > > >> | EM| 5.960683928| 25.604583163| 106.688041149|
> > > > >>
> > > > >> It's a little bit strange for me, why I have to compare
> > > > >> EntityPersistenceRepository.save with a em.persist + em.flush.
> > > > >> I would expect that an simple
> > > > >> EntityPersistenceRepository.save don't have a flush (there is
> > > > >> a separate
> EntityPersistenceRepository.
> > > > saveAndFlush).
> > > > >>
> > > > >> When I do a run with EntityPersistenceRepository.saveAndFlush
> > > > >> I get the following numbers.
> > > > >> ____________________________________________________________
> > > > >> ____________________________________________________________
> > > > >> ______________________________
> > > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
> > |
> > > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
> > iter
> > > > >> 10240 |
> > > > >> |============================================================
> > > > >> |==
> > > > >> |==
> > > > >> |==
> > > > >> |==
> > > > >> |===
> > > > >> |============================================================
> > > > >> |==
> > > > >> |==
> > > > >> |==
> > > > >> |==
> > > > >> |===
> > > > >> |=======|
> > > > >> | DS| 0.001703015| 0.003457728| 0.008079817| 0.019099994|
> > > > >> | DS| 0.053865065| 0.940319597| 0.643245399| 2.292716685|
> > > > >> | DS| 9.902395587| 40.84301017 | 172.179435413|
> > > > >> | EM| 0.001677545| 0.004168205| 0.005779986| 0.014491211|
> > > > >> | EM| 0.031066334| 0.110747277| 0.4051742 | 1.925682412|
> > > > >> | EM| 5.842606084| 23.540393571| 132.817886521|
> > > > >>
> > > > >> So I have the feeling that there is still something wrong.
> > > > >>
> > > > >> Thanks to Gerhard for his additional hints.
> > > > >> I committed all changes to the github repo.
> > > > >>
> > > > >> Regards,
> > > > >> Johannes
> > > > >>
> > > > >> -----Original Message-----
> > > > >> From: Gerhard Petracek [mailto:gpetracek@apache.org
> <javascript:;>]
> > > > >> Sent: Thursday, June 1, 2017 1:21 PM
> > > > >> To: users@deltaspike.apache.org <javascript:;>
> > > > >> Subject: Re: Performance of DeltaSpike Data
> > > > >>
> > > > >> @johannes:
> > > > >> as mentioned yesterday you have to move
> > > > >> EntityTransaction#begin and EntityTransaction#commit into the for-loop.
> > > > >>
> > > > >> regards,
> > > > >> gerhard
> > > > >>
> > > > >>
> > > > >>
> > > > >> 2017-06-01 12:58 GMT+02:00 Thomas Andraschko
> > > > >> <andraschko.thomas@gmail.com <javascript:;>
> > > > >> >:
> > > > >>
> > > > >> > Hi,
> > > > >> >
> > > > >> > ~1 year ago i did many optimizations in the data module and
> > > > >> > AFAIR DS Data was only a little bit slower.
> > > > >> > After i compared my testcase with a benchmark from a user,
> > > > >> > the big different came from the transaction handling which
> > > > >> > was different in both testcases.
> > > > >> >
> > > > >> > Regards,
> > > > >> > Thomas
> > > > >> >
> > > > >> > 2017-06-01 12:33 GMT+02:00 Gerhard Petracek
> > > > >> > <gpetracek@apache.org <javascript:;>
> > >:
> > > > >> >
> > > > >> > > hi johannes,
> > > > >> > >
> > > > >> > > after refactoring your initial code to ds-test-control i
> > > > >> > > saw
> > e.g.
> > > > >> > > ~6s vs 7,5s for 2560 iterations.
> > > > >> > > i'll compare my local version with your new version
> > > > >> > > (mentioned in your mail).
> > > > >> > >
> > > > >> > > regards,
> > > > >> > > gerhard
> > > > >> > >
> > > > >> > >
> > > > >> > >
> > > > >> > > 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes
> > > > >> > > <jschaefer@psi.de <javascript:;>
> > >:
> > > > >> > >
> > > > >> > > > Hi,
> > > > >> > > >
> > > > >> > > > My company is thinking about using DeltaSpike Data. But
> > > > >> > > > before we integrate this into our development I was
> > > > >> > > > asked to prepare some
> > > > >> > > benchmarks,
> > > > >> > > > comparing the usage of DeltaSpike Data with the usage
> > > > >> > > > of a plain EntityManager.
> > > > >> > > > I prepared some benchmarks and I was surprised that
> > > > >> > > > there is a big difference in the write performance. I
> > > > >> > > > already got some hints in the
> > > > >> > > delta
> > > > >> > > > spike irc channel, but the performance is still bad.
> > > > >> > > > Based on a template from os890 I implemented my tests
> > > > >> > > > and prepared a github project.
> > > > >> > > > https://github.com/johannesschaefer/javaee_jsf_
> > > > >> > cdi_jpa_data_ds_project_
> > > > >> > > > template
> > > > >> > > > Basically I'm talking about this test:
> > > > >> > > > https://github.com/johannesschaefer/javaee_jsf_
> > > > >> > cdi_jpa_data_ds_project_
> > > > >> > > > template/blob/master/src/test/java/de/psi/metals/future
> > > > >> > > > la b/ repo/benchmark/SaveTest.java
> > > > >> > > >
> > > > >> > > > It just saves an entity into a DB in a loop. Depending
> > > > >> > > > of the number of iterations the difference is quite big.
> > > > >> > > >
> > > > >> > > > SaveTest
> > > > >> > > > _______________________________________________________
> > > > >> > > > __
> > > > >> > > > __
> > > > >> > > > _
> > > > >> > > > _______________________________________________________
> > > > >> > > > __ __ _ _____________________________
> > > > >> > > > | | iter 10 | iter 20 | iter 40 | iter 80 |
> iter
> > > 160
> > > > >> |
> > > > >> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter
> 5120
> > > |
> > > > >> iter
> > > > >> > > > 10240 |
> > > > >> > > > |======================================================
> > > > >> > > > |==
> > > > >> > > > |==
> > > > >> > > > |=
> > > > >> > > > =======================================================
> > > > >> > > > == == = =============================|
> > > > >> > > > | DS| 0.004911746| 0.03597043 | 0.015765787|
> > > > >> > > > | DS| 0.016966639| 0.043319612|
> > > > >> > > > 0.281807839| 1.308948835| 1.370535533| 8.186996818|
> > > > >> > > > 20.920141274| 93.631768475|
> > > > >> > > > | EM| 0.004557839| 0.003256631| 0.005775416|
> > > > >> > > > | EM| 0.004834958| 0.028243393|
> > > > >> > > > 0.035484616| 0.038600595| 0.088904458| 0.339158674|
> > > > >> > > > 0.745850523
> > > > >> > > > |
> > > > >> > > > 0.853543234 |
> > > > >> > > >
> > > > >> > > > Also the difference between a run with 5120 and 10240
> > > > >> > > > iteration is not just the double amount of time, it is
> > > > >> > > > more than 4 times
> > > > more.
> > > > >> > > >
> > > > >> > > > Do you have some hints to me what I'm doing wrong there?
> > > > >> > > >
> > > > >> > > > Regards
> > > > >> > > > Johannes
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: Performance of DeltaSpike Data
Posted by Thomas Andraschko <an...@gmail.com>.
Hi,
It would be great if you could try your sample with OWB instead of weld.
I always to such tests with OWB.
Regards,
Thomas
Am Donnerstag, 15. Juni 2017 schrieb Schäfer, Johannes :
> Hi,
>
> Thanks for your investigation. I tried it today again with the last
> version from Github. For my test setup I still have the same numbers. So in
> average DeltaSpike needs 3 times longer that a pure EM implementation for
> finding entities by a query.
> For now I stopped my evaluation of DeltaSpike Data. My personal feeling is
> quite mixed. I really like to concept of DeltaSpike Data, but the
> performance loss is sometimes quite too big.
> I hope that there will be in future some improvements.
>
> Regards
> Johannes
>
> -----Original Message-----
> From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com <javascript:;>
> ]
> Sent: Friday, June 9, 2017 4:40 PM
> To: users@deltaspike.apache.org <javascript:;>
> Subject: Re: Performance of DeltaSpike Data
>
> FYI:
>
> i created a similar example based on the unittests in the ds data module.
> Here is the result of 100.000 selects, running on tomee and hsqldb:
>
> DS: 2600ms
> JPA: 1800ms
>
>
> 200-300ms are taken by our interceptor lookup, which can be improved a bit
> by caching the information about interceptors in the proxy instance instead
> of doing a bean+cache lookup each time.
> another 200-300ms are taken by DS Data directly - not sure if we can
> improve it further.
>
> Not sure about the other 200-400ms.
>
> 2017-06-08 16:33 GMT+02:00 Schäfer, Johannes <jschaefer@psi.de
> <javascript:;>>:
>
> > No chances with @ApplicationScoped.
> > To run my test just run "mvn clean install -PEmbeddedTestWeld".
> > It is using a in memory H2 DB. Maybe pipe the output into a log to
> > find the test results in the output.
> >
> > Grüße
> > Johannes
> >
> >
> > -----Original Message-----
> > From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com
> <javascript:;>]
> > Sent: Thursday, June 8, 2017 3:12 PM
> > To: users@deltaspike.apache.org <javascript:;>
> > Subject: Re: Performance of DeltaSpike Data
> >
> > Could you also try to make your repository @ApplicationScoped?
> >
> > How can i run your tests? I don't like to install a AS or database ;)
> >
> > 2017-06-08 13:28 GMT+02:00 Schäfer, Johannes <jschaefer@psi.de
> <javascript:;>>:
> >
> > > Same with a locally compile version.
> > > Mysql:
> > > ____________________________________________________________
> > > ____________________________________________________________
> > > _____________________________
> > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > > 10240 |
> > > |===========================================================
> > > ============================================================
> > > =============================|
> > > | DS| 0.028479446| 0.088102335| 0.138260967| 0.255074252|
> > > | DS| 0.385907351|
> > > 0.734428279| 1.836123535| 4.125717222| 6.175937816| 13.217757392|
> > > 25.372525787|
> > > | EM| 0.010955534| 0.020851247| 0.041094277| 0.076565573|
> > > | EM| 0.195617863|
> > > 0.386509868| 0.812829151| 1.4044238 | 3.007676477| 6.232350452 |
> > > 11.726264467|
> > >
> > > Grüße
> > > Johannes
> > >
> > >
> > > -----Original Message-----
> > > From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com
> <javascript:;>]
> > > Sent: Thursday, June 8, 2017 1:12 PM
> > > To: users@deltaspike.apache.org <javascript:;>
> > > Subject: Re: Performance of DeltaSpike Data
> > >
> > > please build DS from source, i don't think that SNAPSHOT is up to date.
> > >
> > > 2017-06-08 13:05 GMT+02:00 Schäfer, Johannes <jschaefer@psi.de
> <javascript:;>>:
> > >
> > > > Hi,
> > > >
> > > > Thanks for your great support. Now I had the time to run the tests.
> > > > Unfortunately no improvement. :-(
> > > > I used Mysql and H2 and both still have a significant difference.
> > > > mysql:
> > > > ____________________________________________________________
> > > > ____________________________________________________________
> > > > _____________________________
> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
> |
> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
> iter
> > > > 10240 |
> > > > |===========================================================
> > > > ============================================================
> > > > =============================|
> > > > | DS| 0.042993818| 0.070756327| 0.139015158| 0.249963317|
> > > > | DS| 0.489673972|
> > > > 1.000932095| 1.418196146| 3.396942334| 6.268094687| 12.142304859|
> > > > 24.631240985|
> > > > | EM| 0.016741971| 0.034018415| 0.042539175| 0.097203944|
> > > > | EM| 0.15662194
> > > > | EM| |
> > > > 0.32694476 | 0.665341891| 1.582051703| 2.602520533| 5.710082816 |
> > > > 10.856276852|
> > > >
> > > > h2:
> > > > ____________________________________________________________
> > > > ____________________________________________________________
> > > > __________________________
> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
> |
> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > > 10240 |
> > > > |===========================================================
> > > > ============================================================
> > > > ==========================|
> > > > | DS| 0.007259847| 0.009839833| 0.024182004| 0.040194493|
> > > > | DS| 0.037355253|
> > > > 0.038992501| 0.15695646| 0.157184542| 0.277937182| 0.575950893|
> > > > 0.848326297|
> > > > | EM| 7.12756E-4 | 7.15797E-4 | 0.0035079 | 0.001897262|
> > > > | EM| 0.003144109|
> > > > 0.007000594| 0.01269694| 0.024183904| 0.037443446| 0.108577248|
> > > > 0.217664259|
> > > >
> > > > I used version 1.8.1-SNAPSHOT for testing.
> > > > See https://github.com/johannesschaefer/javaee_jsf_
> > > > cdi_jpa_data_ds_project_template
> > > >
> > > > Grüße
> > > > Johannes
> > > >
> > > > -----Original Message-----
> > > > From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com
> <javascript:;>]
> > > > Sent: Thursday, June 8, 2017 10:36 AM
> > > > To: users@deltaspike.apache.org <javascript:;>
> > > > Subject: Re: Performance of DeltaSpike Data
> > > >
> > > > I did a major improvement and in my tests, both plain JPA and DS
> > > > Data are now very similar.
> > > > Would be great if you could provide the new numbers.
> > > >
> > > > 2017-06-07 14:33 GMT+02:00 Thomas Andraschko
> > > > <andraschko.thomas@gmail.com <javascript:;>
> > > > >:
> > > >
> > > > > Hi,
> > > > >
> > > > > could you please try to run your test again against the github
> > master?
> > > > > I already did a small improvement and refactored a little bit on
> > > > > the weekend.
> > > > >
> > > > > 2017-06-06 8:54 GMT+02:00 Schäfer, Johannes <jschaefer@psi.de
> <javascript:;>>:
> > > > >
> > > > >> Hi,
> > > > >>
> > > > >> So after the a long weekend, I'm back with my results.
> > > > >> For the write, findByPK and findAll tests I get now good numbers.
> > > > >> See:
> > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > > >> metals/futurelab/repo/benchmark/SaveTest.java
> > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > > >> metals/futurelab/repo/benchmark/ReadTest.java
> > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > > >> metals/futurelab/repo/benchmark/ReadAllTest.java
> > > > >>
> > > > >> The difference between delta spike and plain EM are just a few
> > > > >> percent, in both directions ;-) .
> > > > >>
> > > > >> But I wrote a new test case were I try to find entities by an
> query.
> > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > > >> metals/futurelab/repo/benchmark/ReadQueryTest.java
> > > > >> So I compare
> > > > >> TypedQuery< Material > query = eml.createQuery(
> > > > >> "SELECT m FROM Material m WHERE grade = :grade
> > > > >> AND width = :width AND thickness = :thickness",
> > > > >> Material.class );
> > > > >> query.setParameter( "grade", "AAA" );
> > > > >> query.setParameter( "width", 5 );
> > > > >> query.setParameter( "thickness", 5. ); List<
> > > > >> Material
> > > > >> > mats = query.getResultList();
> > > > >>
> > > > >> to
> > > > >> List< Material > mats =
> > > > >> matRepo.findByGradeAndWidthAndThickness(
> > > > >> "AAA", 5, 5. );
> > > > >>
> > > > >> Here again the difference is quite high.
> > > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
> > |
> > > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
> > iter
> > > > >> 10240 |
> > > > >> |===========================================================
> > > > >> ============================================================
> > > > >> =============================|
> > > > >> | DS| 0.03988012 | 0.151870613| 0.144881044| 0.270389952|
> > > > >> | DS| 0.526700787|
> > > > >> 1.023574545| 1.806960223| 3.426772405| 6.969935385|
> > > > >> 13.963582543| 26.785764953|
> > > > >> | EM| 0.010984804| 0.021940339| 0.059921297| 0.087386918|
> > > > >> | EM| 0.171045079|
> > > > >> 0.375059016| 0.747171594| 1.560946145| 2.968347174| 6.446844753
> > > > >> | 12.361550486|
> > > > >>
> > > > >> So as you can see the DeltaSpike implementation needs at least
> > > > >> the double amount of time.
> > > > >>
> > > > >> Any hints for improving the performance?
> > > > >>
> > > > >> Regards,
> > > > >> Johannes
> > > > >>
> > > > >> -----Original Message-----
> > > > >> From: Schäfer, Johannes [mailto:jschaefer@psi.de <javascript:;>]
> > > > >> Sent: Thursday, June 1, 2017 2:27 PM
> > > > >> To: users@deltaspike.apache.org <javascript:;>
> > > > >> Subject: RE: Performance of DeltaSpike Data
> > > > >>
> > > > >> Right. Copy and paste error.
> > > > >> I added also a flush to the EM test.
> > > > >> Now I have similar numbers.
> > > > >> ____________________________________________________________
> > > > >> ____________________________________________________________
> > > > >> ______________________________
> > > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
> > |
> > > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
> > iter
> > > > >> 10240 |
> > > > >> |==============================================================
> > > > >> |==
> > > > >> |==
> > > > >> |==
> > > > >> |===
> > > > >> |==============================================================
> > > > >> |==
> > > > >> |==
> > > > >> |==
> > > > >> |===
> > > > >> |=======|
> > > > >> | DS| 0.001588214| 0.004130191| 0.007351854| 0.014062036|
> > > > >> | DS| 0.048373222| 0.593463008| 0.741351593| 1.697058004|
> > > > >> | DS| 6.049719288| 34.101109279| 101.589077365|
> > > > >> | EM| 0.001385601| 0.002662861| 0.004092937| 0.108730649|
> > > > >> | EM| 0.046299193| 0.106900289| 0.461147505| 1.688040769|
> > > > >> | EM| 5.960683928| 25.604583163| 106.688041149|
> > > > >>
> > > > >> It's a little bit strange for me, why I have to compare
> > > > >> EntityPersistenceRepository.save with a em.persist + em.flush.
> > > > >> I would expect that an simple EntityPersistenceRepository.save
> > > > >> don't have a flush (there is a separate
> EntityPersistenceRepository.
> > > > saveAndFlush).
> > > > >>
> > > > >> When I do a run with EntityPersistenceRepository.saveAndFlush I
> > > > >> get the following numbers.
> > > > >> ____________________________________________________________
> > > > >> ____________________________________________________________
> > > > >> ______________________________
> > > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
> > |
> > > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
> > iter
> > > > >> 10240 |
> > > > >> |==============================================================
> > > > >> |==
> > > > >> |==
> > > > >> |==
> > > > >> |===
> > > > >> |==============================================================
> > > > >> |==
> > > > >> |==
> > > > >> |==
> > > > >> |===
> > > > >> |=======|
> > > > >> | DS| 0.001703015| 0.003457728| 0.008079817| 0.019099994|
> > > > >> | DS| 0.053865065| 0.940319597| 0.643245399| 2.292716685|
> > > > >> | DS| 9.902395587| 40.84301017 | 172.179435413|
> > > > >> | EM| 0.001677545| 0.004168205| 0.005779986| 0.014491211|
> > > > >> | EM| 0.031066334| 0.110747277| 0.4051742 | 1.925682412|
> > > > >> | EM| 5.842606084| 23.540393571| 132.817886521|
> > > > >>
> > > > >> So I have the feeling that there is still something wrong.
> > > > >>
> > > > >> Thanks to Gerhard for his additional hints.
> > > > >> I committed all changes to the github repo.
> > > > >>
> > > > >> Regards,
> > > > >> Johannes
> > > > >>
> > > > >> -----Original Message-----
> > > > >> From: Gerhard Petracek [mailto:gpetracek@apache.org
> <javascript:;>]
> > > > >> Sent: Thursday, June 1, 2017 1:21 PM
> > > > >> To: users@deltaspike.apache.org <javascript:;>
> > > > >> Subject: Re: Performance of DeltaSpike Data
> > > > >>
> > > > >> @johannes:
> > > > >> as mentioned yesterday you have to move EntityTransaction#begin
> > > > >> and EntityTransaction#commit into the for-loop.
> > > > >>
> > > > >> regards,
> > > > >> gerhard
> > > > >>
> > > > >>
> > > > >>
> > > > >> 2017-06-01 12:58 GMT+02:00 Thomas Andraschko
> > > > >> <andraschko.thomas@gmail.com <javascript:;>
> > > > >> >:
> > > > >>
> > > > >> > Hi,
> > > > >> >
> > > > >> > ~1 year ago i did many optimizations in the data module and
> > > > >> > AFAIR DS Data was only a little bit slower.
> > > > >> > After i compared my testcase with a benchmark from a user,
> > > > >> > the big different came from the transaction handling which
> > > > >> > was different in both testcases.
> > > > >> >
> > > > >> > Regards,
> > > > >> > Thomas
> > > > >> >
> > > > >> > 2017-06-01 12:33 GMT+02:00 Gerhard Petracek
> > > > >> > <gpetracek@apache.org <javascript:;>
> > >:
> > > > >> >
> > > > >> > > hi johannes,
> > > > >> > >
> > > > >> > > after refactoring your initial code to ds-test-control i
> > > > >> > > saw
> > e.g.
> > > > >> > > ~6s vs 7,5s for 2560 iterations.
> > > > >> > > i'll compare my local version with your new version
> > > > >> > > (mentioned in your mail).
> > > > >> > >
> > > > >> > > regards,
> > > > >> > > gerhard
> > > > >> > >
> > > > >> > >
> > > > >> > >
> > > > >> > > 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes
> > > > >> > > <jschaefer@psi.de <javascript:;>
> > >:
> > > > >> > >
> > > > >> > > > Hi,
> > > > >> > > >
> > > > >> > > > My company is thinking about using DeltaSpike Data. But
> > > > >> > > > before we integrate this into our development I was asked
> > > > >> > > > to prepare some
> > > > >> > > benchmarks,
> > > > >> > > > comparing the usage of DeltaSpike Data with the usage of
> > > > >> > > > a plain EntityManager.
> > > > >> > > > I prepared some benchmarks and I was surprised that there
> > > > >> > > > is a big difference in the write performance. I already
> > > > >> > > > got some hints in the
> > > > >> > > delta
> > > > >> > > > spike irc channel, but the performance is still bad.
> > > > >> > > > Based on a template from os890 I implemented my tests and
> > > > >> > > > prepared a github project.
> > > > >> > > > https://github.com/johannesschaefer/javaee_jsf_
> > > > >> > cdi_jpa_data_ds_project_
> > > > >> > > > template
> > > > >> > > > Basically I'm talking about this test:
> > > > >> > > > https://github.com/johannesschaefer/javaee_jsf_
> > > > >> > cdi_jpa_data_ds_project_
> > > > >> > > > template/blob/master/src/test/java/de/psi/metals/futurela
> > > > >> > > > b/ repo/benchmark/SaveTest.java
> > > > >> > > >
> > > > >> > > > It just saves an entity into a DB in a loop. Depending of
> > > > >> > > > the number of iterations the difference is quite big.
> > > > >> > > >
> > > > >> > > > SaveTest
> > > > >> > > > _________________________________________________________
> > > > >> > > > __
> > > > >> > > > _
> > > > >> > > > _________________________________________________________
> > > > >> > > > __ _ _____________________________
> > > > >> > > > | | iter 10 | iter 20 | iter 40 | iter 80 |
> iter
> > > 160
> > > > >> |
> > > > >> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter
> 5120
> > > |
> > > > >> iter
> > > > >> > > > 10240 |
> > > > >> > > > |========================================================
> > > > >> > > > |==
> > > > >> > > > |=
> > > > >> > > > =========================================================
> > > > >> > > > == = =============================|
> > > > >> > > > | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639|
> > > > >> > > > | DS| 0.043319612|
> > > > >> > > > 0.281807839| 1.308948835| 1.370535533| 8.186996818|
> > > > >> > > > 20.920141274| 93.631768475|
> > > > >> > > > | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958|
> > > > >> > > > | EM| 0.028243393|
> > > > >> > > > 0.035484616| 0.038600595| 0.088904458| 0.339158674|
> > > > >> > > > 0.745850523
> > > > >> > > > |
> > > > >> > > > 0.853543234 |
> > > > >> > > >
> > > > >> > > > Also the difference between a run with 5120 and 10240
> > > > >> > > > iteration is not just the double amount of time, it is
> > > > >> > > > more than 4 times
> > > > more.
> > > > >> > > >
> > > > >> > > > Do you have some hints to me what I'm doing wrong there?
> > > > >> > > >
> > > > >> > > > Regards
> > > > >> > > > Johannes
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
RE: Performance of DeltaSpike Data
Posted by Schäfer,
Johannes <js...@psi.de>.
Hi,
Thanks for your investigation. I tried it today again with the last version from Github. For my test setup I still have the same numbers. So in average DeltaSpike needs 3 times longer that a pure EM implementation for finding entities by a query.
For now I stopped my evaluation of DeltaSpike Data. My personal feeling is quite mixed. I really like to concept of DeltaSpike Data, but the performance loss is sometimes quite too big.
I hope that there will be in future some improvements.
Regards
Johannes
-----Original Message-----
From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
Sent: Friday, June 9, 2017 4:40 PM
To: users@deltaspike.apache.org
Subject: Re: Performance of DeltaSpike Data
FYI:
i created a similar example based on the unittests in the ds data module.
Here is the result of 100.000 selects, running on tomee and hsqldb:
DS: 2600ms
JPA: 1800ms
200-300ms are taken by our interceptor lookup, which can be improved a bit by caching the information about interceptors in the proxy instance instead of doing a bean+cache lookup each time.
another 200-300ms are taken by DS Data directly - not sure if we can improve it further.
Not sure about the other 200-400ms.
2017-06-08 16:33 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> No chances with @ApplicationScoped.
> To run my test just run "mvn clean install -PEmbeddedTestWeld".
> It is using a in memory H2 DB. Maybe pipe the output into a log to
> find the test results in the output.
>
> Grüße
> Johannes
>
>
> -----Original Message-----
> From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
> Sent: Thursday, June 8, 2017 3:12 PM
> To: users@deltaspike.apache.org
> Subject: Re: Performance of DeltaSpike Data
>
> Could you also try to make your repository @ApplicationScoped?
>
> How can i run your tests? I don't like to install a AS or database ;)
>
> 2017-06-08 13:28 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
>
> > Same with a locally compile version.
> > Mysql:
> > ____________________________________________________________
> > ____________________________________________________________
> > _____________________________
> > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > 10240 |
> > |===========================================================
> > ============================================================
> > =============================|
> > | DS| 0.028479446| 0.088102335| 0.138260967| 0.255074252|
> > | DS| 0.385907351|
> > 0.734428279| 1.836123535| 4.125717222| 6.175937816| 13.217757392|
> > 25.372525787|
> > | EM| 0.010955534| 0.020851247| 0.041094277| 0.076565573|
> > | EM| 0.195617863|
> > 0.386509868| 0.812829151| 1.4044238 | 3.007676477| 6.232350452 |
> > 11.726264467|
> >
> > Grüße
> > Johannes
> >
> >
> > -----Original Message-----
> > From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
> > Sent: Thursday, June 8, 2017 1:12 PM
> > To: users@deltaspike.apache.org
> > Subject: Re: Performance of DeltaSpike Data
> >
> > please build DS from source, i don't think that SNAPSHOT is up to date.
> >
> > 2017-06-08 13:05 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> >
> > > Hi,
> > >
> > > Thanks for your great support. Now I had the time to run the tests.
> > > Unfortunately no improvement. :-(
> > > I used Mysql and H2 and both still have a significant difference.
> > > mysql:
> > > ____________________________________________________________
> > > ____________________________________________________________
> > > _____________________________
> > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > > 10240 |
> > > |===========================================================
> > > ============================================================
> > > =============================|
> > > | DS| 0.042993818| 0.070756327| 0.139015158| 0.249963317|
> > > | DS| 0.489673972|
> > > 1.000932095| 1.418196146| 3.396942334| 6.268094687| 12.142304859|
> > > 24.631240985|
> > > | EM| 0.016741971| 0.034018415| 0.042539175| 0.097203944|
> > > | EM| 0.15662194
> > > | EM| |
> > > 0.32694476 | 0.665341891| 1.582051703| 2.602520533| 5.710082816 |
> > > 10.856276852|
> > >
> > > h2:
> > > ____________________________________________________________
> > > ____________________________________________________________
> > > __________________________
> > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > 10240 |
> > > |===========================================================
> > > ============================================================
> > > ==========================|
> > > | DS| 0.007259847| 0.009839833| 0.024182004| 0.040194493|
> > > | DS| 0.037355253|
> > > 0.038992501| 0.15695646| 0.157184542| 0.277937182| 0.575950893|
> > > 0.848326297|
> > > | EM| 7.12756E-4 | 7.15797E-4 | 0.0035079 | 0.001897262|
> > > | EM| 0.003144109|
> > > 0.007000594| 0.01269694| 0.024183904| 0.037443446| 0.108577248|
> > > 0.217664259|
> > >
> > > I used version 1.8.1-SNAPSHOT for testing.
> > > See https://github.com/johannesschaefer/javaee_jsf_
> > > cdi_jpa_data_ds_project_template
> > >
> > > Grüße
> > > Johannes
> > >
> > > -----Original Message-----
> > > From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
> > > Sent: Thursday, June 8, 2017 10:36 AM
> > > To: users@deltaspike.apache.org
> > > Subject: Re: Performance of DeltaSpike Data
> > >
> > > I did a major improvement and in my tests, both plain JPA and DS
> > > Data are now very similar.
> > > Would be great if you could provide the new numbers.
> > >
> > > 2017-06-07 14:33 GMT+02:00 Thomas Andraschko
> > > <andraschko.thomas@gmail.com
> > > >:
> > >
> > > > Hi,
> > > >
> > > > could you please try to run your test again against the github
> master?
> > > > I already did a small improvement and refactored a little bit on
> > > > the weekend.
> > > >
> > > > 2017-06-06 8:54 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> > > >
> > > >> Hi,
> > > >>
> > > >> So after the a long weekend, I'm back with my results.
> > > >> For the write, findByPK and findAll tests I get now good numbers.
> > > >> See:
> > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > >> metals/futurelab/repo/benchmark/SaveTest.java
> > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > >> metals/futurelab/repo/benchmark/ReadTest.java
> > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > >> metals/futurelab/repo/benchmark/ReadAllTest.java
> > > >>
> > > >> The difference between delta spike and plain EM are just a few
> > > >> percent, in both directions ;-) .
> > > >>
> > > >> But I wrote a new test case were I try to find entities by an query.
> > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > >> metals/futurelab/repo/benchmark/ReadQueryTest.java
> > > >> So I compare
> > > >> TypedQuery< Material > query = eml.createQuery(
> > > >> "SELECT m FROM Material m WHERE grade = :grade
> > > >> AND width = :width AND thickness = :thickness",
> > > >> Material.class );
> > > >> query.setParameter( "grade", "AAA" );
> > > >> query.setParameter( "width", 5 );
> > > >> query.setParameter( "thickness", 5. ); List<
> > > >> Material
> > > >> > mats = query.getResultList();
> > > >>
> > > >> to
> > > >> List< Material > mats =
> > > >> matRepo.findByGradeAndWidthAndThickness(
> > > >> "AAA", 5, 5. );
> > > >>
> > > >> Here again the difference is quite high.
> > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
> |
> > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
> iter
> > > >> 10240 |
> > > >> |===========================================================
> > > >> ============================================================
> > > >> =============================|
> > > >> | DS| 0.03988012 | 0.151870613| 0.144881044| 0.270389952|
> > > >> | DS| 0.526700787|
> > > >> 1.023574545| 1.806960223| 3.426772405| 6.969935385|
> > > >> 13.963582543| 26.785764953|
> > > >> | EM| 0.010984804| 0.021940339| 0.059921297| 0.087386918|
> > > >> | EM| 0.171045079|
> > > >> 0.375059016| 0.747171594| 1.560946145| 2.968347174| 6.446844753
> > > >> | 12.361550486|
> > > >>
> > > >> So as you can see the DeltaSpike implementation needs at least
> > > >> the double amount of time.
> > > >>
> > > >> Any hints for improving the performance?
> > > >>
> > > >> Regards,
> > > >> Johannes
> > > >>
> > > >> -----Original Message-----
> > > >> From: Schäfer, Johannes [mailto:jschaefer@psi.de]
> > > >> Sent: Thursday, June 1, 2017 2:27 PM
> > > >> To: users@deltaspike.apache.org
> > > >> Subject: RE: Performance of DeltaSpike Data
> > > >>
> > > >> Right. Copy and paste error.
> > > >> I added also a flush to the EM test.
> > > >> Now I have similar numbers.
> > > >> ____________________________________________________________
> > > >> ____________________________________________________________
> > > >> ______________________________
> > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
> |
> > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
> iter
> > > >> 10240 |
> > > >> |==============================================================
> > > >> |==
> > > >> |==
> > > >> |==
> > > >> |===
> > > >> |==============================================================
> > > >> |==
> > > >> |==
> > > >> |==
> > > >> |===
> > > >> |=======|
> > > >> | DS| 0.001588214| 0.004130191| 0.007351854| 0.014062036|
> > > >> | DS| 0.048373222| 0.593463008| 0.741351593| 1.697058004|
> > > >> | DS| 6.049719288| 34.101109279| 101.589077365|
> > > >> | EM| 0.001385601| 0.002662861| 0.004092937| 0.108730649|
> > > >> | EM| 0.046299193| 0.106900289| 0.461147505| 1.688040769|
> > > >> | EM| 5.960683928| 25.604583163| 106.688041149|
> > > >>
> > > >> It's a little bit strange for me, why I have to compare
> > > >> EntityPersistenceRepository.save with a em.persist + em.flush.
> > > >> I would expect that an simple EntityPersistenceRepository.save
> > > >> don't have a flush (there is a separate EntityPersistenceRepository.
> > > saveAndFlush).
> > > >>
> > > >> When I do a run with EntityPersistenceRepository.saveAndFlush I
> > > >> get the following numbers.
> > > >> ____________________________________________________________
> > > >> ____________________________________________________________
> > > >> ______________________________
> > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
> |
> > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
> iter
> > > >> 10240 |
> > > >> |==============================================================
> > > >> |==
> > > >> |==
> > > >> |==
> > > >> |===
> > > >> |==============================================================
> > > >> |==
> > > >> |==
> > > >> |==
> > > >> |===
> > > >> |=======|
> > > >> | DS| 0.001703015| 0.003457728| 0.008079817| 0.019099994|
> > > >> | DS| 0.053865065| 0.940319597| 0.643245399| 2.292716685|
> > > >> | DS| 9.902395587| 40.84301017 | 172.179435413|
> > > >> | EM| 0.001677545| 0.004168205| 0.005779986| 0.014491211|
> > > >> | EM| 0.031066334| 0.110747277| 0.4051742 | 1.925682412|
> > > >> | EM| 5.842606084| 23.540393571| 132.817886521|
> > > >>
> > > >> So I have the feeling that there is still something wrong.
> > > >>
> > > >> Thanks to Gerhard for his additional hints.
> > > >> I committed all changes to the github repo.
> > > >>
> > > >> Regards,
> > > >> Johannes
> > > >>
> > > >> -----Original Message-----
> > > >> From: Gerhard Petracek [mailto:gpetracek@apache.org]
> > > >> Sent: Thursday, June 1, 2017 1:21 PM
> > > >> To: users@deltaspike.apache.org
> > > >> Subject: Re: Performance of DeltaSpike Data
> > > >>
> > > >> @johannes:
> > > >> as mentioned yesterday you have to move EntityTransaction#begin
> > > >> and EntityTransaction#commit into the for-loop.
> > > >>
> > > >> regards,
> > > >> gerhard
> > > >>
> > > >>
> > > >>
> > > >> 2017-06-01 12:58 GMT+02:00 Thomas Andraschko
> > > >> <andraschko.thomas@gmail.com
> > > >> >:
> > > >>
> > > >> > Hi,
> > > >> >
> > > >> > ~1 year ago i did many optimizations in the data module and
> > > >> > AFAIR DS Data was only a little bit slower.
> > > >> > After i compared my testcase with a benchmark from a user,
> > > >> > the big different came from the transaction handling which
> > > >> > was different in both testcases.
> > > >> >
> > > >> > Regards,
> > > >> > Thomas
> > > >> >
> > > >> > 2017-06-01 12:33 GMT+02:00 Gerhard Petracek
> > > >> > <gpetracek@apache.org
> >:
> > > >> >
> > > >> > > hi johannes,
> > > >> > >
> > > >> > > after refactoring your initial code to ds-test-control i
> > > >> > > saw
> e.g.
> > > >> > > ~6s vs 7,5s for 2560 iterations.
> > > >> > > i'll compare my local version with your new version
> > > >> > > (mentioned in your mail).
> > > >> > >
> > > >> > > regards,
> > > >> > > gerhard
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes
> > > >> > > <jschaefer@psi.de
> >:
> > > >> > >
> > > >> > > > Hi,
> > > >> > > >
> > > >> > > > My company is thinking about using DeltaSpike Data. But
> > > >> > > > before we integrate this into our development I was asked
> > > >> > > > to prepare some
> > > >> > > benchmarks,
> > > >> > > > comparing the usage of DeltaSpike Data with the usage of
> > > >> > > > a plain EntityManager.
> > > >> > > > I prepared some benchmarks and I was surprised that there
> > > >> > > > is a big difference in the write performance. I already
> > > >> > > > got some hints in the
> > > >> > > delta
> > > >> > > > spike irc channel, but the performance is still bad.
> > > >> > > > Based on a template from os890 I implemented my tests and
> > > >> > > > prepared a github project.
> > > >> > > > https://github.com/johannesschaefer/javaee_jsf_
> > > >> > cdi_jpa_data_ds_project_
> > > >> > > > template
> > > >> > > > Basically I'm talking about this test:
> > > >> > > > https://github.com/johannesschaefer/javaee_jsf_
> > > >> > cdi_jpa_data_ds_project_
> > > >> > > > template/blob/master/src/test/java/de/psi/metals/futurela
> > > >> > > > b/ repo/benchmark/SaveTest.java
> > > >> > > >
> > > >> > > > It just saves an entity into a DB in a loop. Depending of
> > > >> > > > the number of iterations the difference is quite big.
> > > >> > > >
> > > >> > > > SaveTest
> > > >> > > > _________________________________________________________
> > > >> > > > __
> > > >> > > > _
> > > >> > > > _________________________________________________________
> > > >> > > > __ _ _____________________________
> > > >> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter
> > 160
> > > >> |
> > > >> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120
> > |
> > > >> iter
> > > >> > > > 10240 |
> > > >> > > > |========================================================
> > > >> > > > |==
> > > >> > > > |=
> > > >> > > > =========================================================
> > > >> > > > == = =============================|
> > > >> > > > | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639|
> > > >> > > > | DS| 0.043319612|
> > > >> > > > 0.281807839| 1.308948835| 1.370535533| 8.186996818|
> > > >> > > > 20.920141274| 93.631768475|
> > > >> > > > | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958|
> > > >> > > > | EM| 0.028243393|
> > > >> > > > 0.035484616| 0.038600595| 0.088904458| 0.339158674|
> > > >> > > > 0.745850523
> > > >> > > > |
> > > >> > > > 0.853543234 |
> > > >> > > >
> > > >> > > > Also the difference between a run with 5120 and 10240
> > > >> > > > iteration is not just the double amount of time, it is
> > > >> > > > more than 4 times
> > > more.
> > > >> > > >
> > > >> > > > Do you have some hints to me what I'm doing wrong there?
> > > >> > > >
> > > >> > > > Regards
> > > >> > > > Johannes
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>
Re: Performance of DeltaSpike Data
Posted by Thomas Andraschko <an...@gmail.com>.
FYI:
i created a similar example based on the unittests in the ds data module.
Here is the result of 100.000 selects, running on tomee and hsqldb:
DS: 2600ms
JPA: 1800ms
200-300ms are taken by our interceptor lookup, which can be improved a bit
by caching the information about interceptors in the proxy instance instead
of doing a bean+cache lookup each time.
another 200-300ms are taken by DS Data directly - not sure if we can
improve it further.
Not sure about the other 200-400ms.
2017-06-08 16:33 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> No chances with @ApplicationScoped.
> To run my test just run "mvn clean install -PEmbeddedTestWeld".
> It is using a in memory H2 DB. Maybe pipe the output into a log to find
> the test results in the output.
>
> Grüße
> Johannes
>
>
> -----Original Message-----
> From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
> Sent: Thursday, June 8, 2017 3:12 PM
> To: users@deltaspike.apache.org
> Subject: Re: Performance of DeltaSpike Data
>
> Could you also try to make your repository @ApplicationScoped?
>
> How can i run your tests? I don't like to install a AS or database ;)
>
> 2017-06-08 13:28 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
>
> > Same with a locally compile version.
> > Mysql:
> > ____________________________________________________________
> > ____________________________________________________________
> > _____________________________
> > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > 10240 |
> > |===========================================================
> > ============================================================
> > =============================|
> > | DS| 0.028479446| 0.088102335| 0.138260967| 0.255074252| 0.385907351|
> > 0.734428279| 1.836123535| 4.125717222| 6.175937816| 13.217757392|
> > 25.372525787|
> > | EM| 0.010955534| 0.020851247| 0.041094277| 0.076565573| 0.195617863|
> > 0.386509868| 0.812829151| 1.4044238 | 3.007676477| 6.232350452 |
> > 11.726264467|
> >
> > Grüße
> > Johannes
> >
> >
> > -----Original Message-----
> > From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
> > Sent: Thursday, June 8, 2017 1:12 PM
> > To: users@deltaspike.apache.org
> > Subject: Re: Performance of DeltaSpike Data
> >
> > please build DS from source, i don't think that SNAPSHOT is up to date.
> >
> > 2017-06-08 13:05 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> >
> > > Hi,
> > >
> > > Thanks for your great support. Now I had the time to run the tests.
> > > Unfortunately no improvement. :-(
> > > I used Mysql and H2 and both still have a significant difference.
> > > mysql:
> > > ____________________________________________________________
> > > ____________________________________________________________
> > > _____________________________
> > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > > 10240 |
> > > |===========================================================
> > > ============================================================
> > > =============================|
> > > | DS| 0.042993818| 0.070756327| 0.139015158| 0.249963317|
> > > | DS| 0.489673972|
> > > 1.000932095| 1.418196146| 3.396942334| 6.268094687| 12.142304859|
> > > 24.631240985|
> > > | EM| 0.016741971| 0.034018415| 0.042539175| 0.097203944| 0.15662194
> > > | EM| |
> > > 0.32694476 | 0.665341891| 1.582051703| 2.602520533| 5.710082816 |
> > > 10.856276852|
> > >
> > > h2:
> > > ____________________________________________________________
> > > ____________________________________________________________
> > > __________________________
> > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > 10240 |
> > > |===========================================================
> > > ============================================================
> > > ==========================|
> > > | DS| 0.007259847| 0.009839833| 0.024182004| 0.040194493|
> > > | DS| 0.037355253|
> > > 0.038992501| 0.15695646| 0.157184542| 0.277937182| 0.575950893|
> > > 0.848326297|
> > > | EM| 7.12756E-4 | 7.15797E-4 | 0.0035079 | 0.001897262|
> > > | EM| 0.003144109|
> > > 0.007000594| 0.01269694| 0.024183904| 0.037443446| 0.108577248|
> > > 0.217664259|
> > >
> > > I used version 1.8.1-SNAPSHOT for testing.
> > > See https://github.com/johannesschaefer/javaee_jsf_
> > > cdi_jpa_data_ds_project_template
> > >
> > > Grüße
> > > Johannes
> > >
> > > -----Original Message-----
> > > From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
> > > Sent: Thursday, June 8, 2017 10:36 AM
> > > To: users@deltaspike.apache.org
> > > Subject: Re: Performance of DeltaSpike Data
> > >
> > > I did a major improvement and in my tests, both plain JPA and DS
> > > Data are now very similar.
> > > Would be great if you could provide the new numbers.
> > >
> > > 2017-06-07 14:33 GMT+02:00 Thomas Andraschko
> > > <andraschko.thomas@gmail.com
> > > >:
> > >
> > > > Hi,
> > > >
> > > > could you please try to run your test again against the github
> master?
> > > > I already did a small improvement and refactored a little bit on
> > > > the weekend.
> > > >
> > > > 2017-06-06 8:54 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> > > >
> > > >> Hi,
> > > >>
> > > >> So after the a long weekend, I'm back with my results.
> > > >> For the write, findByPK and findAll tests I get now good numbers.
> > > >> See:
> > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > >> metals/futurelab/repo/benchmark/SaveTest.java
> > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > >> metals/futurelab/repo/benchmark/ReadTest.java
> > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > >> metals/futurelab/repo/benchmark/ReadAllTest.java
> > > >>
> > > >> The difference between delta spike and plain EM are just a few
> > > >> percent, in both directions ;-) .
> > > >>
> > > >> But I wrote a new test case were I try to find entities by an query.
> > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > > >> ds_project_template/blob/master/src/test/java/de/psi/
> > > >> metals/futurelab/repo/benchmark/ReadQueryTest.java
> > > >> So I compare
> > > >> TypedQuery< Material > query = eml.createQuery(
> > > >> "SELECT m FROM Material m WHERE grade = :grade
> > > >> AND width = :width AND thickness = :thickness",
> > > >> Material.class );
> > > >> query.setParameter( "grade", "AAA" );
> > > >> query.setParameter( "width", 5 );
> > > >> query.setParameter( "thickness", 5. ); List< Material
> > > >> > mats = query.getResultList();
> > > >>
> > > >> to
> > > >> List< Material > mats = matRepo.findByGradeAndWidthAndThickness(
> > > >> "AAA", 5, 5. );
> > > >>
> > > >> Here again the difference is quite high.
> > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
> |
> > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
> iter
> > > >> 10240 |
> > > >> |===========================================================
> > > >> ============================================================
> > > >> =============================|
> > > >> | DS| 0.03988012 | 0.151870613| 0.144881044| 0.270389952|
> > > >> | DS| 0.526700787|
> > > >> 1.023574545| 1.806960223| 3.426772405| 6.969935385| 13.963582543|
> > > >> 26.785764953|
> > > >> | EM| 0.010984804| 0.021940339| 0.059921297| 0.087386918|
> > > >> | EM| 0.171045079|
> > > >> 0.375059016| 0.747171594| 1.560946145| 2.968347174| 6.446844753 |
> > > >> 12.361550486|
> > > >>
> > > >> So as you can see the DeltaSpike implementation needs at least
> > > >> the double amount of time.
> > > >>
> > > >> Any hints for improving the performance?
> > > >>
> > > >> Regards,
> > > >> Johannes
> > > >>
> > > >> -----Original Message-----
> > > >> From: Schäfer, Johannes [mailto:jschaefer@psi.de]
> > > >> Sent: Thursday, June 1, 2017 2:27 PM
> > > >> To: users@deltaspike.apache.org
> > > >> Subject: RE: Performance of DeltaSpike Data
> > > >>
> > > >> Right. Copy and paste error.
> > > >> I added also a flush to the EM test.
> > > >> Now I have similar numbers.
> > > >> ____________________________________________________________
> > > >> ____________________________________________________________
> > > >> ______________________________
> > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
> |
> > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
> iter
> > > >> 10240 |
> > > >> |================================================================
> > > >> |==
> > > >> |==
> > > >> |===
> > > >> |================================================================
> > > >> |==
> > > >> |==
> > > >> |===
> > > >> |=======|
> > > >> | DS| 0.001588214| 0.004130191| 0.007351854| 0.014062036|
> > > >> | DS| 0.048373222| 0.593463008| 0.741351593| 1.697058004|
> > > >> | DS| 6.049719288| 34.101109279| 101.589077365|
> > > >> | EM| 0.001385601| 0.002662861| 0.004092937| 0.108730649|
> > > >> | EM| 0.046299193| 0.106900289| 0.461147505| 1.688040769|
> > > >> | EM| 5.960683928| 25.604583163| 106.688041149|
> > > >>
> > > >> It's a little bit strange for me, why I have to compare
> > > >> EntityPersistenceRepository.save with a em.persist + em.flush. I
> > > >> would expect that an simple EntityPersistenceRepository.save
> > > >> don't have a flush (there is a separate EntityPersistenceRepository.
> > > saveAndFlush).
> > > >>
> > > >> When I do a run with EntityPersistenceRepository.saveAndFlush I
> > > >> get the following numbers.
> > > >> ____________________________________________________________
> > > >> ____________________________________________________________
> > > >> ______________________________
> > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
> |
> > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
> iter
> > > >> 10240 |
> > > >> |================================================================
> > > >> |==
> > > >> |==
> > > >> |===
> > > >> |================================================================
> > > >> |==
> > > >> |==
> > > >> |===
> > > >> |=======|
> > > >> | DS| 0.001703015| 0.003457728| 0.008079817| 0.019099994|
> > > >> | DS| 0.053865065| 0.940319597| 0.643245399| 2.292716685|
> > > >> | DS| 9.902395587| 40.84301017 | 172.179435413|
> > > >> | EM| 0.001677545| 0.004168205| 0.005779986| 0.014491211|
> > > >> | EM| 0.031066334| 0.110747277| 0.4051742 | 1.925682412|
> > > >> | EM| 5.842606084| 23.540393571| 132.817886521|
> > > >>
> > > >> So I have the feeling that there is still something wrong.
> > > >>
> > > >> Thanks to Gerhard for his additional hints.
> > > >> I committed all changes to the github repo.
> > > >>
> > > >> Regards,
> > > >> Johannes
> > > >>
> > > >> -----Original Message-----
> > > >> From: Gerhard Petracek [mailto:gpetracek@apache.org]
> > > >> Sent: Thursday, June 1, 2017 1:21 PM
> > > >> To: users@deltaspike.apache.org
> > > >> Subject: Re: Performance of DeltaSpike Data
> > > >>
> > > >> @johannes:
> > > >> as mentioned yesterday you have to move EntityTransaction#begin
> > > >> and EntityTransaction#commit into the for-loop.
> > > >>
> > > >> regards,
> > > >> gerhard
> > > >>
> > > >>
> > > >>
> > > >> 2017-06-01 12:58 GMT+02:00 Thomas Andraschko
> > > >> <andraschko.thomas@gmail.com
> > > >> >:
> > > >>
> > > >> > Hi,
> > > >> >
> > > >> > ~1 year ago i did many optimizations in the data module and
> > > >> > AFAIR DS Data was only a little bit slower.
> > > >> > After i compared my testcase with a benchmark from a user, the
> > > >> > big different came from the transaction handling which was
> > > >> > different in both testcases.
> > > >> >
> > > >> > Regards,
> > > >> > Thomas
> > > >> >
> > > >> > 2017-06-01 12:33 GMT+02:00 Gerhard Petracek <gpetracek@apache.org
> >:
> > > >> >
> > > >> > > hi johannes,
> > > >> > >
> > > >> > > after refactoring your initial code to ds-test-control i saw
> e.g.
> > > >> > > ~6s vs 7,5s for 2560 iterations.
> > > >> > > i'll compare my local version with your new version
> > > >> > > (mentioned in your mail).
> > > >> > >
> > > >> > > regards,
> > > >> > > gerhard
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes <jschaefer@psi.de
> >:
> > > >> > >
> > > >> > > > Hi,
> > > >> > > >
> > > >> > > > My company is thinking about using DeltaSpike Data. But
> > > >> > > > before we integrate this into our development I was asked
> > > >> > > > to prepare some
> > > >> > > benchmarks,
> > > >> > > > comparing the usage of DeltaSpike Data with the usage of a
> > > >> > > > plain EntityManager.
> > > >> > > > I prepared some benchmarks and I was surprised that there
> > > >> > > > is a big difference in the write performance. I already got
> > > >> > > > some hints in the
> > > >> > > delta
> > > >> > > > spike irc channel, but the performance is still bad.
> > > >> > > > Based on a template from os890 I implemented my tests and
> > > >> > > > prepared a github project.
> > > >> > > > https://github.com/johannesschaefer/javaee_jsf_
> > > >> > cdi_jpa_data_ds_project_
> > > >> > > > template
> > > >> > > > Basically I'm talking about this test:
> > > >> > > > https://github.com/johannesschaefer/javaee_jsf_
> > > >> > cdi_jpa_data_ds_project_
> > > >> > > > template/blob/master/src/test/java/de/psi/metals/futurelab/
> > > >> > > > repo/benchmark/SaveTest.java
> > > >> > > >
> > > >> > > > It just saves an entity into a DB in a loop. Depending of
> > > >> > > > the number of iterations the difference is quite big.
> > > >> > > >
> > > >> > > > SaveTest
> > > >> > > > ___________________________________________________________
> > > >> > > > _
> > > >> > > > ___________________________________________________________
> > > >> > > > _ _____________________________
> > > >> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter
> > 160
> > > >> |
> > > >> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120
> > |
> > > >> iter
> > > >> > > > 10240 |
> > > >> > > > |==========================================================
> > > >> > > > |=
> > > >> > > > ===========================================================
> > > >> > > > = =============================|
> > > >> > > > | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639|
> > > >> > > > | DS| 0.043319612|
> > > >> > > > 0.281807839| 1.308948835| 1.370535533| 8.186996818|
> > > >> > > > 20.920141274| 93.631768475|
> > > >> > > > | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958|
> > > >> > > > | EM| 0.028243393|
> > > >> > > > 0.035484616| 0.038600595| 0.088904458| 0.339158674|
> > > >> > > > 0.745850523
> > > >> > > > |
> > > >> > > > 0.853543234 |
> > > >> > > >
> > > >> > > > Also the difference between a run with 5120 and 10240
> > > >> > > > iteration is not just the double amount of time, it is more
> > > >> > > > than 4 times
> > > more.
> > > >> > > >
> > > >> > > > Do you have some hints to me what I'm doing wrong there?
> > > >> > > >
> > > >> > > > Regards
> > > >> > > > Johannes
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>
RE: Performance of DeltaSpike Data
Posted by Schäfer,
Johannes <js...@psi.de>.
No chances with @ApplicationScoped.
To run my test just run "mvn clean install -PEmbeddedTestWeld".
It is using a in memory H2 DB. Maybe pipe the output into a log to find the test results in the output.
Grüße
Johannes
-----Original Message-----
From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
Sent: Thursday, June 8, 2017 3:12 PM
To: users@deltaspike.apache.org
Subject: Re: Performance of DeltaSpike Data
Could you also try to make your repository @ApplicationScoped?
How can i run your tests? I don't like to install a AS or database ;)
2017-06-08 13:28 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> Same with a locally compile version.
> Mysql:
> ____________________________________________________________
> ____________________________________________________________
> _____________________________
> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> 10240 |
> |===========================================================
> ============================================================
> =============================|
> | DS| 0.028479446| 0.088102335| 0.138260967| 0.255074252| 0.385907351|
> 0.734428279| 1.836123535| 4.125717222| 6.175937816| 13.217757392|
> 25.372525787|
> | EM| 0.010955534| 0.020851247| 0.041094277| 0.076565573| 0.195617863|
> 0.386509868| 0.812829151| 1.4044238 | 3.007676477| 6.232350452 |
> 11.726264467|
>
> Grüße
> Johannes
>
>
> -----Original Message-----
> From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
> Sent: Thursday, June 8, 2017 1:12 PM
> To: users@deltaspike.apache.org
> Subject: Re: Performance of DeltaSpike Data
>
> please build DS from source, i don't think that SNAPSHOT is up to date.
>
> 2017-06-08 13:05 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
>
> > Hi,
> >
> > Thanks for your great support. Now I had the time to run the tests.
> > Unfortunately no improvement. :-(
> > I used Mysql and H2 and both still have a significant difference.
> > mysql:
> > ____________________________________________________________
> > ____________________________________________________________
> > _____________________________
> > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > 10240 |
> > |===========================================================
> > ============================================================
> > =============================|
> > | DS| 0.042993818| 0.070756327| 0.139015158| 0.249963317|
> > | DS| 0.489673972|
> > 1.000932095| 1.418196146| 3.396942334| 6.268094687| 12.142304859|
> > 24.631240985|
> > | EM| 0.016741971| 0.034018415| 0.042539175| 0.097203944| 0.15662194
> > | EM| |
> > 0.32694476 | 0.665341891| 1.582051703| 2.602520533| 5.710082816 |
> > 10.856276852|
> >
> > h2:
> > ____________________________________________________________
> > ____________________________________________________________
> > __________________________
> > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> 10240 |
> > |===========================================================
> > ============================================================
> > ==========================|
> > | DS| 0.007259847| 0.009839833| 0.024182004| 0.040194493|
> > | DS| 0.037355253|
> > 0.038992501| 0.15695646| 0.157184542| 0.277937182| 0.575950893|
> > 0.848326297|
> > | EM| 7.12756E-4 | 7.15797E-4 | 0.0035079 | 0.001897262|
> > | EM| 0.003144109|
> > 0.007000594| 0.01269694| 0.024183904| 0.037443446| 0.108577248|
> > 0.217664259|
> >
> > I used version 1.8.1-SNAPSHOT for testing.
> > See https://github.com/johannesschaefer/javaee_jsf_
> > cdi_jpa_data_ds_project_template
> >
> > Grüße
> > Johannes
> >
> > -----Original Message-----
> > From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
> > Sent: Thursday, June 8, 2017 10:36 AM
> > To: users@deltaspike.apache.org
> > Subject: Re: Performance of DeltaSpike Data
> >
> > I did a major improvement and in my tests, both plain JPA and DS
> > Data are now very similar.
> > Would be great if you could provide the new numbers.
> >
> > 2017-06-07 14:33 GMT+02:00 Thomas Andraschko
> > <andraschko.thomas@gmail.com
> > >:
> >
> > > Hi,
> > >
> > > could you please try to run your test again against the github master?
> > > I already did a small improvement and refactored a little bit on
> > > the weekend.
> > >
> > > 2017-06-06 8:54 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> > >
> > >> Hi,
> > >>
> > >> So after the a long weekend, I'm back with my results.
> > >> For the write, findByPK and findAll tests I get now good numbers.
> > >> See:
> > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > >> ds_project_template/blob/master/src/test/java/de/psi/
> > >> metals/futurelab/repo/benchmark/SaveTest.java
> > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > >> ds_project_template/blob/master/src/test/java/de/psi/
> > >> metals/futurelab/repo/benchmark/ReadTest.java
> > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > >> ds_project_template/blob/master/src/test/java/de/psi/
> > >> metals/futurelab/repo/benchmark/ReadAllTest.java
> > >>
> > >> The difference between delta spike and plain EM are just a few
> > >> percent, in both directions ;-) .
> > >>
> > >> But I wrote a new test case were I try to find entities by an query.
> > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > >> ds_project_template/blob/master/src/test/java/de/psi/
> > >> metals/futurelab/repo/benchmark/ReadQueryTest.java
> > >> So I compare
> > >> TypedQuery< Material > query = eml.createQuery(
> > >> "SELECT m FROM Material m WHERE grade = :grade
> > >> AND width = :width AND thickness = :thickness",
> > >> Material.class );
> > >> query.setParameter( "grade", "AAA" );
> > >> query.setParameter( "width", 5 );
> > >> query.setParameter( "thickness", 5. ); List< Material
> > >> > mats = query.getResultList();
> > >>
> > >> to
> > >> List< Material > mats = matRepo.findByGradeAndWidthAndThickness(
> > >> "AAA", 5, 5. );
> > >>
> > >> Here again the difference is quite high.
> > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > >> 10240 |
> > >> |===========================================================
> > >> ============================================================
> > >> =============================|
> > >> | DS| 0.03988012 | 0.151870613| 0.144881044| 0.270389952|
> > >> | DS| 0.526700787|
> > >> 1.023574545| 1.806960223| 3.426772405| 6.969935385| 13.963582543|
> > >> 26.785764953|
> > >> | EM| 0.010984804| 0.021940339| 0.059921297| 0.087386918|
> > >> | EM| 0.171045079|
> > >> 0.375059016| 0.747171594| 1.560946145| 2.968347174| 6.446844753 |
> > >> 12.361550486|
> > >>
> > >> So as you can see the DeltaSpike implementation needs at least
> > >> the double amount of time.
> > >>
> > >> Any hints for improving the performance?
> > >>
> > >> Regards,
> > >> Johannes
> > >>
> > >> -----Original Message-----
> > >> From: Schäfer, Johannes [mailto:jschaefer@psi.de]
> > >> Sent: Thursday, June 1, 2017 2:27 PM
> > >> To: users@deltaspike.apache.org
> > >> Subject: RE: Performance of DeltaSpike Data
> > >>
> > >> Right. Copy and paste error.
> > >> I added also a flush to the EM test.
> > >> Now I have similar numbers.
> > >> ____________________________________________________________
> > >> ____________________________________________________________
> > >> ______________________________
> > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > >> 10240 |
> > >> |================================================================
> > >> |==
> > >> |==
> > >> |===
> > >> |================================================================
> > >> |==
> > >> |==
> > >> |===
> > >> |=======|
> > >> | DS| 0.001588214| 0.004130191| 0.007351854| 0.014062036|
> > >> | DS| 0.048373222| 0.593463008| 0.741351593| 1.697058004|
> > >> | DS| 6.049719288| 34.101109279| 101.589077365|
> > >> | EM| 0.001385601| 0.002662861| 0.004092937| 0.108730649|
> > >> | EM| 0.046299193| 0.106900289| 0.461147505| 1.688040769|
> > >> | EM| 5.960683928| 25.604583163| 106.688041149|
> > >>
> > >> It's a little bit strange for me, why I have to compare
> > >> EntityPersistenceRepository.save with a em.persist + em.flush. I
> > >> would expect that an simple EntityPersistenceRepository.save
> > >> don't have a flush (there is a separate EntityPersistenceRepository.
> > saveAndFlush).
> > >>
> > >> When I do a run with EntityPersistenceRepository.saveAndFlush I
> > >> get the following numbers.
> > >> ____________________________________________________________
> > >> ____________________________________________________________
> > >> ______________________________
> > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > >> 10240 |
> > >> |================================================================
> > >> |==
> > >> |==
> > >> |===
> > >> |================================================================
> > >> |==
> > >> |==
> > >> |===
> > >> |=======|
> > >> | DS| 0.001703015| 0.003457728| 0.008079817| 0.019099994|
> > >> | DS| 0.053865065| 0.940319597| 0.643245399| 2.292716685|
> > >> | DS| 9.902395587| 40.84301017 | 172.179435413|
> > >> | EM| 0.001677545| 0.004168205| 0.005779986| 0.014491211|
> > >> | EM| 0.031066334| 0.110747277| 0.4051742 | 1.925682412|
> > >> | EM| 5.842606084| 23.540393571| 132.817886521|
> > >>
> > >> So I have the feeling that there is still something wrong.
> > >>
> > >> Thanks to Gerhard for his additional hints.
> > >> I committed all changes to the github repo.
> > >>
> > >> Regards,
> > >> Johannes
> > >>
> > >> -----Original Message-----
> > >> From: Gerhard Petracek [mailto:gpetracek@apache.org]
> > >> Sent: Thursday, June 1, 2017 1:21 PM
> > >> To: users@deltaspike.apache.org
> > >> Subject: Re: Performance of DeltaSpike Data
> > >>
> > >> @johannes:
> > >> as mentioned yesterday you have to move EntityTransaction#begin
> > >> and EntityTransaction#commit into the for-loop.
> > >>
> > >> regards,
> > >> gerhard
> > >>
> > >>
> > >>
> > >> 2017-06-01 12:58 GMT+02:00 Thomas Andraschko
> > >> <andraschko.thomas@gmail.com
> > >> >:
> > >>
> > >> > Hi,
> > >> >
> > >> > ~1 year ago i did many optimizations in the data module and
> > >> > AFAIR DS Data was only a little bit slower.
> > >> > After i compared my testcase with a benchmark from a user, the
> > >> > big different came from the transaction handling which was
> > >> > different in both testcases.
> > >> >
> > >> > Regards,
> > >> > Thomas
> > >> >
> > >> > 2017-06-01 12:33 GMT+02:00 Gerhard Petracek <gp...@apache.org>:
> > >> >
> > >> > > hi johannes,
> > >> > >
> > >> > > after refactoring your initial code to ds-test-control i saw e.g.
> > >> > > ~6s vs 7,5s for 2560 iterations.
> > >> > > i'll compare my local version with your new version
> > >> > > (mentioned in your mail).
> > >> > >
> > >> > > regards,
> > >> > > gerhard
> > >> > >
> > >> > >
> > >> > >
> > >> > > 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> > >> > >
> > >> > > > Hi,
> > >> > > >
> > >> > > > My company is thinking about using DeltaSpike Data. But
> > >> > > > before we integrate this into our development I was asked
> > >> > > > to prepare some
> > >> > > benchmarks,
> > >> > > > comparing the usage of DeltaSpike Data with the usage of a
> > >> > > > plain EntityManager.
> > >> > > > I prepared some benchmarks and I was surprised that there
> > >> > > > is a big difference in the write performance. I already got
> > >> > > > some hints in the
> > >> > > delta
> > >> > > > spike irc channel, but the performance is still bad.
> > >> > > > Based on a template from os890 I implemented my tests and
> > >> > > > prepared a github project.
> > >> > > > https://github.com/johannesschaefer/javaee_jsf_
> > >> > cdi_jpa_data_ds_project_
> > >> > > > template
> > >> > > > Basically I'm talking about this test:
> > >> > > > https://github.com/johannesschaefer/javaee_jsf_
> > >> > cdi_jpa_data_ds_project_
> > >> > > > template/blob/master/src/test/java/de/psi/metals/futurelab/
> > >> > > > repo/benchmark/SaveTest.java
> > >> > > >
> > >> > > > It just saves an entity into a DB in a loop. Depending of
> > >> > > > the number of iterations the difference is quite big.
> > >> > > >
> > >> > > > SaveTest
> > >> > > > ___________________________________________________________
> > >> > > > _
> > >> > > > ___________________________________________________________
> > >> > > > _ _____________________________
> > >> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter
> 160
> > >> |
> > >> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120
> |
> > >> iter
> > >> > > > 10240 |
> > >> > > > |==========================================================
> > >> > > > |=
> > >> > > > ===========================================================
> > >> > > > = =============================|
> > >> > > > | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639|
> > >> > > > | DS| 0.043319612|
> > >> > > > 0.281807839| 1.308948835| 1.370535533| 8.186996818|
> > >> > > > 20.920141274| 93.631768475|
> > >> > > > | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958|
> > >> > > > | EM| 0.028243393|
> > >> > > > 0.035484616| 0.038600595| 0.088904458| 0.339158674|
> > >> > > > 0.745850523
> > >> > > > |
> > >> > > > 0.853543234 |
> > >> > > >
> > >> > > > Also the difference between a run with 5120 and 10240
> > >> > > > iteration is not just the double amount of time, it is more
> > >> > > > than 4 times
> > more.
> > >> > > >
> > >> > > > Do you have some hints to me what I'm doing wrong there?
> > >> > > >
> > >> > > > Regards
> > >> > > > Johannes
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>
Re: Performance of DeltaSpike Data
Posted by Thomas Andraschko <an...@gmail.com>.
Could you also try to make your repository @ApplicationScoped?
How can i run your tests? I don't like to install a AS or database ;)
2017-06-08 13:28 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> Same with a locally compile version.
> Mysql:
> ____________________________________________________________
> ____________________________________________________________
> _____________________________
> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> 10240 |
> |===========================================================
> ============================================================
> =============================|
> | DS| 0.028479446| 0.088102335| 0.138260967| 0.255074252| 0.385907351|
> 0.734428279| 1.836123535| 4.125717222| 6.175937816| 13.217757392|
> 25.372525787|
> | EM| 0.010955534| 0.020851247| 0.041094277| 0.076565573| 0.195617863|
> 0.386509868| 0.812829151| 1.4044238 | 3.007676477| 6.232350452 |
> 11.726264467|
>
> Grüße
> Johannes
>
>
> -----Original Message-----
> From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
> Sent: Thursday, June 8, 2017 1:12 PM
> To: users@deltaspike.apache.org
> Subject: Re: Performance of DeltaSpike Data
>
> please build DS from source, i don't think that SNAPSHOT is up to date.
>
> 2017-06-08 13:05 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
>
> > Hi,
> >
> > Thanks for your great support. Now I had the time to run the tests.
> > Unfortunately no improvement. :-(
> > I used Mysql and H2 and both still have a significant difference.
> > mysql:
> > ____________________________________________________________
> > ____________________________________________________________
> > _____________________________
> > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > 10240 |
> > |===========================================================
> > ============================================================
> > =============================|
> > | DS| 0.042993818| 0.070756327| 0.139015158| 0.249963317| 0.489673972|
> > 1.000932095| 1.418196146| 3.396942334| 6.268094687| 12.142304859|
> > 24.631240985|
> > | EM| 0.016741971| 0.034018415| 0.042539175| 0.097203944| 0.15662194 |
> > 0.32694476 | 0.665341891| 1.582051703| 2.602520533| 5.710082816 |
> > 10.856276852|
> >
> > h2:
> > ____________________________________________________________
> > ____________________________________________________________
> > __________________________
> > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> 10240 |
> > |===========================================================
> > ============================================================
> > ==========================|
> > | DS| 0.007259847| 0.009839833| 0.024182004| 0.040194493| 0.037355253|
> > 0.038992501| 0.15695646| 0.157184542| 0.277937182| 0.575950893|
> > 0.848326297|
> > | EM| 7.12756E-4 | 7.15797E-4 | 0.0035079 | 0.001897262| 0.003144109|
> > 0.007000594| 0.01269694| 0.024183904| 0.037443446| 0.108577248|
> > 0.217664259|
> >
> > I used version 1.8.1-SNAPSHOT for testing.
> > See https://github.com/johannesschaefer/javaee_jsf_
> > cdi_jpa_data_ds_project_template
> >
> > Grüße
> > Johannes
> >
> > -----Original Message-----
> > From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
> > Sent: Thursday, June 8, 2017 10:36 AM
> > To: users@deltaspike.apache.org
> > Subject: Re: Performance of DeltaSpike Data
> >
> > I did a major improvement and in my tests, both plain JPA and DS Data
> > are now very similar.
> > Would be great if you could provide the new numbers.
> >
> > 2017-06-07 14:33 GMT+02:00 Thomas Andraschko
> > <andraschko.thomas@gmail.com
> > >:
> >
> > > Hi,
> > >
> > > could you please try to run your test again against the github master?
> > > I already did a small improvement and refactored a little bit on the
> > > weekend.
> > >
> > > 2017-06-06 8:54 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> > >
> > >> Hi,
> > >>
> > >> So after the a long weekend, I'm back with my results.
> > >> For the write, findByPK and findAll tests I get now good numbers.
> > >> See:
> > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > >> ds_project_template/blob/master/src/test/java/de/psi/
> > >> metals/futurelab/repo/benchmark/SaveTest.java
> > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > >> ds_project_template/blob/master/src/test/java/de/psi/
> > >> metals/futurelab/repo/benchmark/ReadTest.java
> > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > >> ds_project_template/blob/master/src/test/java/de/psi/
> > >> metals/futurelab/repo/benchmark/ReadAllTest.java
> > >>
> > >> The difference between delta spike and plain EM are just a few
> > >> percent, in both directions ;-) .
> > >>
> > >> But I wrote a new test case were I try to find entities by an query.
> > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> > >> ds_project_template/blob/master/src/test/java/de/psi/
> > >> metals/futurelab/repo/benchmark/ReadQueryTest.java
> > >> So I compare
> > >> TypedQuery< Material > query = eml.createQuery(
> > >> "SELECT m FROM Material m WHERE grade = :grade AND
> > >> width = :width AND thickness = :thickness",
> > >> Material.class );
> > >> query.setParameter( "grade", "AAA" );
> > >> query.setParameter( "width", 5 );
> > >> query.setParameter( "thickness", 5. ); List< Material >
> > >> mats = query.getResultList();
> > >>
> > >> to
> > >> List< Material > mats = matRepo.findByGradeAndWidthAndThickness(
> > >> "AAA", 5, 5. );
> > >>
> > >> Here again the difference is quite high.
> > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > >> 10240 |
> > >> |===========================================================
> > >> ============================================================
> > >> =============================|
> > >> | DS| 0.03988012 | 0.151870613| 0.144881044| 0.270389952|
> > >> | DS| 0.526700787|
> > >> 1.023574545| 1.806960223| 3.426772405| 6.969935385| 13.963582543|
> > >> 26.785764953|
> > >> | EM| 0.010984804| 0.021940339| 0.059921297| 0.087386918|
> > >> | EM| 0.171045079|
> > >> 0.375059016| 0.747171594| 1.560946145| 2.968347174| 6.446844753 |
> > >> 12.361550486|
> > >>
> > >> So as you can see the DeltaSpike implementation needs at least the
> > >> double amount of time.
> > >>
> > >> Any hints for improving the performance?
> > >>
> > >> Regards,
> > >> Johannes
> > >>
> > >> -----Original Message-----
> > >> From: Schäfer, Johannes [mailto:jschaefer@psi.de]
> > >> Sent: Thursday, June 1, 2017 2:27 PM
> > >> To: users@deltaspike.apache.org
> > >> Subject: RE: Performance of DeltaSpike Data
> > >>
> > >> Right. Copy and paste error.
> > >> I added also a flush to the EM test.
> > >> Now I have similar numbers.
> > >> ____________________________________________________________
> > >> ____________________________________________________________
> > >> ______________________________
> > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > >> 10240 |
> > >> |==================================================================
> > >> |==
> > >> |===
> > >> |==================================================================
> > >> |==
> > >> |===
> > >> |=======|
> > >> | DS| 0.001588214| 0.004130191| 0.007351854| 0.014062036|
> > >> | DS| 0.048373222| 0.593463008| 0.741351593| 1.697058004|
> > >> | DS| 6.049719288| 34.101109279| 101.589077365|
> > >> | EM| 0.001385601| 0.002662861| 0.004092937| 0.108730649|
> > >> | EM| 0.046299193| 0.106900289| 0.461147505| 1.688040769|
> > >> | EM| 5.960683928| 25.604583163| 106.688041149|
> > >>
> > >> It's a little bit strange for me, why I have to compare
> > >> EntityPersistenceRepository.save with a em.persist + em.flush. I
> > >> would expect that an simple EntityPersistenceRepository.save don't
> > >> have a flush (there is a separate EntityPersistenceRepository.
> > saveAndFlush).
> > >>
> > >> When I do a run with EntityPersistenceRepository.saveAndFlush I get
> > >> the following numbers.
> > >> ____________________________________________________________
> > >> ____________________________________________________________
> > >> ______________________________
> > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > >> 10240 |
> > >> |==================================================================
> > >> |==
> > >> |===
> > >> |==================================================================
> > >> |==
> > >> |===
> > >> |=======|
> > >> | DS| 0.001703015| 0.003457728| 0.008079817| 0.019099994|
> > >> | DS| 0.053865065| 0.940319597| 0.643245399| 2.292716685|
> > >> | DS| 9.902395587| 40.84301017 | 172.179435413|
> > >> | EM| 0.001677545| 0.004168205| 0.005779986| 0.014491211|
> > >> | EM| 0.031066334| 0.110747277| 0.4051742 | 1.925682412|
> > >> | EM| 5.842606084| 23.540393571| 132.817886521|
> > >>
> > >> So I have the feeling that there is still something wrong.
> > >>
> > >> Thanks to Gerhard for his additional hints.
> > >> I committed all changes to the github repo.
> > >>
> > >> Regards,
> > >> Johannes
> > >>
> > >> -----Original Message-----
> > >> From: Gerhard Petracek [mailto:gpetracek@apache.org]
> > >> Sent: Thursday, June 1, 2017 1:21 PM
> > >> To: users@deltaspike.apache.org
> > >> Subject: Re: Performance of DeltaSpike Data
> > >>
> > >> @johannes:
> > >> as mentioned yesterday you have to move EntityTransaction#begin and
> > >> EntityTransaction#commit into the for-loop.
> > >>
> > >> regards,
> > >> gerhard
> > >>
> > >>
> > >>
> > >> 2017-06-01 12:58 GMT+02:00 Thomas Andraschko
> > >> <andraschko.thomas@gmail.com
> > >> >:
> > >>
> > >> > Hi,
> > >> >
> > >> > ~1 year ago i did many optimizations in the data module and AFAIR
> > >> > DS Data was only a little bit slower.
> > >> > After i compared my testcase with a benchmark from a user, the
> > >> > big different came from the transaction handling which was
> > >> > different in both testcases.
> > >> >
> > >> > Regards,
> > >> > Thomas
> > >> >
> > >> > 2017-06-01 12:33 GMT+02:00 Gerhard Petracek <gp...@apache.org>:
> > >> >
> > >> > > hi johannes,
> > >> > >
> > >> > > after refactoring your initial code to ds-test-control i saw e.g.
> > >> > > ~6s vs 7,5s for 2560 iterations.
> > >> > > i'll compare my local version with your new version (mentioned
> > >> > > in your mail).
> > >> > >
> > >> > > regards,
> > >> > > gerhard
> > >> > >
> > >> > >
> > >> > >
> > >> > > 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> > >> > >
> > >> > > > Hi,
> > >> > > >
> > >> > > > My company is thinking about using DeltaSpike Data. But
> > >> > > > before we integrate this into our development I was asked to
> > >> > > > prepare some
> > >> > > benchmarks,
> > >> > > > comparing the usage of DeltaSpike Data with the usage of a
> > >> > > > plain EntityManager.
> > >> > > > I prepared some benchmarks and I was surprised that there is
> > >> > > > a big difference in the write performance. I already got some
> > >> > > > hints in the
> > >> > > delta
> > >> > > > spike irc channel, but the performance is still bad.
> > >> > > > Based on a template from os890 I implemented my tests and
> > >> > > > prepared a github project.
> > >> > > > https://github.com/johannesschaefer/javaee_jsf_
> > >> > cdi_jpa_data_ds_project_
> > >> > > > template
> > >> > > > Basically I'm talking about this test:
> > >> > > > https://github.com/johannesschaefer/javaee_jsf_
> > >> > cdi_jpa_data_ds_project_
> > >> > > > template/blob/master/src/test/java/de/psi/metals/futurelab/
> > >> > > > repo/benchmark/SaveTest.java
> > >> > > >
> > >> > > > It just saves an entity into a DB in a loop. Depending of the
> > >> > > > number of iterations the difference is quite big.
> > >> > > >
> > >> > > > SaveTest
> > >> > > > ____________________________________________________________
> > >> > > > ____________________________________________________________
> > >> > > > _____________________________
> > >> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter
> 160
> > >> |
> > >> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120
> |
> > >> iter
> > >> > > > 10240 |
> > >> > > > |===========================================================
> > >> > > > ============================================================
> > >> > > > =============================|
> > >> > > > | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639|
> > >> > > > | DS| 0.043319612|
> > >> > > > 0.281807839| 1.308948835| 1.370535533| 8.186996818|
> > >> > > > 20.920141274| 93.631768475|
> > >> > > > | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958|
> > >> > > > | EM| 0.028243393|
> > >> > > > 0.035484616| 0.038600595| 0.088904458| 0.339158674|
> > >> > > > 0.745850523
> > >> > > > |
> > >> > > > 0.853543234 |
> > >> > > >
> > >> > > > Also the difference between a run with 5120 and 10240
> > >> > > > iteration is not just the double amount of time, it is more
> > >> > > > than 4 times
> > more.
> > >> > > >
> > >> > > > Do you have some hints to me what I'm doing wrong there?
> > >> > > >
> > >> > > > Regards
> > >> > > > Johannes
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>
RE: Performance of DeltaSpike Data
Posted by Schäfer,
Johannes <js...@psi.de>.
Same with a locally compile version.
Mysql:
_____________________________________________________________________________________________________________________________________________________
| | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 | iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter 10240 |
|====================================================================================================================================================|
| DS| 0.028479446| 0.088102335| 0.138260967| 0.255074252| 0.385907351| 0.734428279| 1.836123535| 4.125717222| 6.175937816| 13.217757392| 25.372525787|
| EM| 0.010955534| 0.020851247| 0.041094277| 0.076565573| 0.195617863| 0.386509868| 0.812829151| 1.4044238 | 3.007676477| 6.232350452 | 11.726264467|
Grüße
Johannes
-----Original Message-----
From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
Sent: Thursday, June 8, 2017 1:12 PM
To: users@deltaspike.apache.org
Subject: Re: Performance of DeltaSpike Data
please build DS from source, i don't think that SNAPSHOT is up to date.
2017-06-08 13:05 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> Hi,
>
> Thanks for your great support. Now I had the time to run the tests.
> Unfortunately no improvement. :-(
> I used Mysql and H2 and both still have a significant difference.
> mysql:
> ____________________________________________________________
> ____________________________________________________________
> _____________________________
> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> 10240 |
> |===========================================================
> ============================================================
> =============================|
> | DS| 0.042993818| 0.070756327| 0.139015158| 0.249963317| 0.489673972|
> 1.000932095| 1.418196146| 3.396942334| 6.268094687| 12.142304859|
> 24.631240985|
> | EM| 0.016741971| 0.034018415| 0.042539175| 0.097203944| 0.15662194 |
> 0.32694476 | 0.665341891| 1.582051703| 2.602520533| 5.710082816 |
> 10.856276852|
>
> h2:
> ____________________________________________________________
> ____________________________________________________________
> __________________________
> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter 10240 |
> |===========================================================
> ============================================================
> ==========================|
> | DS| 0.007259847| 0.009839833| 0.024182004| 0.040194493| 0.037355253|
> 0.038992501| 0.15695646| 0.157184542| 0.277937182| 0.575950893|
> 0.848326297|
> | EM| 7.12756E-4 | 7.15797E-4 | 0.0035079 | 0.001897262| 0.003144109|
> 0.007000594| 0.01269694| 0.024183904| 0.037443446| 0.108577248|
> 0.217664259|
>
> I used version 1.8.1-SNAPSHOT for testing.
> See https://github.com/johannesschaefer/javaee_jsf_
> cdi_jpa_data_ds_project_template
>
> Grüße
> Johannes
>
> -----Original Message-----
> From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
> Sent: Thursday, June 8, 2017 10:36 AM
> To: users@deltaspike.apache.org
> Subject: Re: Performance of DeltaSpike Data
>
> I did a major improvement and in my tests, both plain JPA and DS Data
> are now very similar.
> Would be great if you could provide the new numbers.
>
> 2017-06-07 14:33 GMT+02:00 Thomas Andraschko
> <andraschko.thomas@gmail.com
> >:
>
> > Hi,
> >
> > could you please try to run your test again against the github master?
> > I already did a small improvement and refactored a little bit on the
> > weekend.
> >
> > 2017-06-06 8:54 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> >
> >> Hi,
> >>
> >> So after the a long weekend, I'm back with my results.
> >> For the write, findByPK and findAll tests I get now good numbers.
> >> See:
> >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> >> ds_project_template/blob/master/src/test/java/de/psi/
> >> metals/futurelab/repo/benchmark/SaveTest.java
> >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> >> ds_project_template/blob/master/src/test/java/de/psi/
> >> metals/futurelab/repo/benchmark/ReadTest.java
> >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> >> ds_project_template/blob/master/src/test/java/de/psi/
> >> metals/futurelab/repo/benchmark/ReadAllTest.java
> >>
> >> The difference between delta spike and plain EM are just a few
> >> percent, in both directions ;-) .
> >>
> >> But I wrote a new test case were I try to find entities by an query.
> >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> >> ds_project_template/blob/master/src/test/java/de/psi/
> >> metals/futurelab/repo/benchmark/ReadQueryTest.java
> >> So I compare
> >> TypedQuery< Material > query = eml.createQuery(
> >> "SELECT m FROM Material m WHERE grade = :grade AND
> >> width = :width AND thickness = :thickness",
> >> Material.class );
> >> query.setParameter( "grade", "AAA" );
> >> query.setParameter( "width", 5 );
> >> query.setParameter( "thickness", 5. ); List< Material >
> >> mats = query.getResultList();
> >>
> >> to
> >> List< Material > mats = matRepo.findByGradeAndWidthAndThickness(
> >> "AAA", 5, 5. );
> >>
> >> Here again the difference is quite high.
> >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> >> 10240 |
> >> |===========================================================
> >> ============================================================
> >> =============================|
> >> | DS| 0.03988012 | 0.151870613| 0.144881044| 0.270389952|
> >> | DS| 0.526700787|
> >> 1.023574545| 1.806960223| 3.426772405| 6.969935385| 13.963582543|
> >> 26.785764953|
> >> | EM| 0.010984804| 0.021940339| 0.059921297| 0.087386918|
> >> | EM| 0.171045079|
> >> 0.375059016| 0.747171594| 1.560946145| 2.968347174| 6.446844753 |
> >> 12.361550486|
> >>
> >> So as you can see the DeltaSpike implementation needs at least the
> >> double amount of time.
> >>
> >> Any hints for improving the performance?
> >>
> >> Regards,
> >> Johannes
> >>
> >> -----Original Message-----
> >> From: Schäfer, Johannes [mailto:jschaefer@psi.de]
> >> Sent: Thursday, June 1, 2017 2:27 PM
> >> To: users@deltaspike.apache.org
> >> Subject: RE: Performance of DeltaSpike Data
> >>
> >> Right. Copy and paste error.
> >> I added also a flush to the EM test.
> >> Now I have similar numbers.
> >> ____________________________________________________________
> >> ____________________________________________________________
> >> ______________________________
> >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> >> 10240 |
> >> |==================================================================
> >> |==
> >> |===
> >> |==================================================================
> >> |==
> >> |===
> >> |=======|
> >> | DS| 0.001588214| 0.004130191| 0.007351854| 0.014062036|
> >> | DS| 0.048373222| 0.593463008| 0.741351593| 1.697058004|
> >> | DS| 6.049719288| 34.101109279| 101.589077365|
> >> | EM| 0.001385601| 0.002662861| 0.004092937| 0.108730649|
> >> | EM| 0.046299193| 0.106900289| 0.461147505| 1.688040769|
> >> | EM| 5.960683928| 25.604583163| 106.688041149|
> >>
> >> It's a little bit strange for me, why I have to compare
> >> EntityPersistenceRepository.save with a em.persist + em.flush. I
> >> would expect that an simple EntityPersistenceRepository.save don't
> >> have a flush (there is a separate EntityPersistenceRepository.
> saveAndFlush).
> >>
> >> When I do a run with EntityPersistenceRepository.saveAndFlush I get
> >> the following numbers.
> >> ____________________________________________________________
> >> ____________________________________________________________
> >> ______________________________
> >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> >> 10240 |
> >> |==================================================================
> >> |==
> >> |===
> >> |==================================================================
> >> |==
> >> |===
> >> |=======|
> >> | DS| 0.001703015| 0.003457728| 0.008079817| 0.019099994|
> >> | DS| 0.053865065| 0.940319597| 0.643245399| 2.292716685|
> >> | DS| 9.902395587| 40.84301017 | 172.179435413|
> >> | EM| 0.001677545| 0.004168205| 0.005779986| 0.014491211|
> >> | EM| 0.031066334| 0.110747277| 0.4051742 | 1.925682412|
> >> | EM| 5.842606084| 23.540393571| 132.817886521|
> >>
> >> So I have the feeling that there is still something wrong.
> >>
> >> Thanks to Gerhard for his additional hints.
> >> I committed all changes to the github repo.
> >>
> >> Regards,
> >> Johannes
> >>
> >> -----Original Message-----
> >> From: Gerhard Petracek [mailto:gpetracek@apache.org]
> >> Sent: Thursday, June 1, 2017 1:21 PM
> >> To: users@deltaspike.apache.org
> >> Subject: Re: Performance of DeltaSpike Data
> >>
> >> @johannes:
> >> as mentioned yesterday you have to move EntityTransaction#begin and
> >> EntityTransaction#commit into the for-loop.
> >>
> >> regards,
> >> gerhard
> >>
> >>
> >>
> >> 2017-06-01 12:58 GMT+02:00 Thomas Andraschko
> >> <andraschko.thomas@gmail.com
> >> >:
> >>
> >> > Hi,
> >> >
> >> > ~1 year ago i did many optimizations in the data module and AFAIR
> >> > DS Data was only a little bit slower.
> >> > After i compared my testcase with a benchmark from a user, the
> >> > big different came from the transaction handling which was
> >> > different in both testcases.
> >> >
> >> > Regards,
> >> > Thomas
> >> >
> >> > 2017-06-01 12:33 GMT+02:00 Gerhard Petracek <gp...@apache.org>:
> >> >
> >> > > hi johannes,
> >> > >
> >> > > after refactoring your initial code to ds-test-control i saw e.g.
> >> > > ~6s vs 7,5s for 2560 iterations.
> >> > > i'll compare my local version with your new version (mentioned
> >> > > in your mail).
> >> > >
> >> > > regards,
> >> > > gerhard
> >> > >
> >> > >
> >> > >
> >> > > 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> >> > >
> >> > > > Hi,
> >> > > >
> >> > > > My company is thinking about using DeltaSpike Data. But
> >> > > > before we integrate this into our development I was asked to
> >> > > > prepare some
> >> > > benchmarks,
> >> > > > comparing the usage of DeltaSpike Data with the usage of a
> >> > > > plain EntityManager.
> >> > > > I prepared some benchmarks and I was surprised that there is
> >> > > > a big difference in the write performance. I already got some
> >> > > > hints in the
> >> > > delta
> >> > > > spike irc channel, but the performance is still bad.
> >> > > > Based on a template from os890 I implemented my tests and
> >> > > > prepared a github project.
> >> > > > https://github.com/johannesschaefer/javaee_jsf_
> >> > cdi_jpa_data_ds_project_
> >> > > > template
> >> > > > Basically I'm talking about this test:
> >> > > > https://github.com/johannesschaefer/javaee_jsf_
> >> > cdi_jpa_data_ds_project_
> >> > > > template/blob/master/src/test/java/de/psi/metals/futurelab/
> >> > > > repo/benchmark/SaveTest.java
> >> > > >
> >> > > > It just saves an entity into a DB in a loop. Depending of the
> >> > > > number of iterations the difference is quite big.
> >> > > >
> >> > > > SaveTest
> >> > > > ____________________________________________________________
> >> > > > ____________________________________________________________
> >> > > > _____________________________
> >> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
> >> |
> >> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
> >> iter
> >> > > > 10240 |
> >> > > > |===========================================================
> >> > > > ============================================================
> >> > > > =============================|
> >> > > > | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639|
> >> > > > | DS| 0.043319612|
> >> > > > 0.281807839| 1.308948835| 1.370535533| 8.186996818|
> >> > > > 20.920141274| 93.631768475|
> >> > > > | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958|
> >> > > > | EM| 0.028243393|
> >> > > > 0.035484616| 0.038600595| 0.088904458| 0.339158674|
> >> > > > 0.745850523
> >> > > > |
> >> > > > 0.853543234 |
> >> > > >
> >> > > > Also the difference between a run with 5120 and 10240
> >> > > > iteration is not just the double amount of time, it is more
> >> > > > than 4 times
> more.
> >> > > >
> >> > > > Do you have some hints to me what I'm doing wrong there?
> >> > > >
> >> > > > Regards
> >> > > > Johannes
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
Re: Performance of DeltaSpike Data
Posted by Thomas Andraschko <an...@gmail.com>.
please build DS from source, i don't think that SNAPSHOT is up to date.
2017-06-08 13:05 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> Hi,
>
> Thanks for your great support. Now I had the time to run the tests.
> Unfortunately no improvement. :-(
> I used Mysql and H2 and both still have a significant difference.
> mysql:
> ____________________________________________________________
> ____________________________________________________________
> _____________________________
> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> 10240 |
> |===========================================================
> ============================================================
> =============================|
> | DS| 0.042993818| 0.070756327| 0.139015158| 0.249963317| 0.489673972|
> 1.000932095| 1.418196146| 3.396942334| 6.268094687| 12.142304859|
> 24.631240985|
> | EM| 0.016741971| 0.034018415| 0.042539175| 0.097203944| 0.15662194 |
> 0.32694476 | 0.665341891| 1.582051703| 2.602520533| 5.710082816 |
> 10.856276852|
>
> h2:
> ____________________________________________________________
> ____________________________________________________________
> __________________________
> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter 10240 |
> |===========================================================
> ============================================================
> ==========================|
> | DS| 0.007259847| 0.009839833| 0.024182004| 0.040194493| 0.037355253|
> 0.038992501| 0.15695646| 0.157184542| 0.277937182| 0.575950893| 0.848326297|
> | EM| 7.12756E-4 | 7.15797E-4 | 0.0035079 | 0.001897262| 0.003144109|
> 0.007000594| 0.01269694| 0.024183904| 0.037443446| 0.108577248| 0.217664259|
>
> I used version 1.8.1-SNAPSHOT for testing.
> See https://github.com/johannesschaefer/javaee_jsf_
> cdi_jpa_data_ds_project_template
>
> Grüße
> Johannes
>
> -----Original Message-----
> From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
> Sent: Thursday, June 8, 2017 10:36 AM
> To: users@deltaspike.apache.org
> Subject: Re: Performance of DeltaSpike Data
>
> I did a major improvement and in my tests, both plain JPA and DS Data are
> now very similar.
> Would be great if you could provide the new numbers.
>
> 2017-06-07 14:33 GMT+02:00 Thomas Andraschko <andraschko.thomas@gmail.com
> >:
>
> > Hi,
> >
> > could you please try to run your test again against the github master?
> > I already did a small improvement and refactored a little bit on the
> > weekend.
> >
> > 2017-06-06 8:54 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> >
> >> Hi,
> >>
> >> So after the a long weekend, I'm back with my results.
> >> For the write, findByPK and findAll tests I get now good numbers.
> >> See:
> >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> >> ds_project_template/blob/master/src/test/java/de/psi/
> >> metals/futurelab/repo/benchmark/SaveTest.java
> >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> >> ds_project_template/blob/master/src/test/java/de/psi/
> >> metals/futurelab/repo/benchmark/ReadTest.java
> >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> >> ds_project_template/blob/master/src/test/java/de/psi/
> >> metals/futurelab/repo/benchmark/ReadAllTest.java
> >>
> >> The difference between delta spike and plain EM are just a few
> >> percent, in both directions ;-) .
> >>
> >> But I wrote a new test case were I try to find entities by an query.
> >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
> >> ds_project_template/blob/master/src/test/java/de/psi/
> >> metals/futurelab/repo/benchmark/ReadQueryTest.java
> >> So I compare
> >> TypedQuery< Material > query = eml.createQuery(
> >> "SELECT m FROM Material m WHERE grade = :grade AND
> >> width = :width AND thickness = :thickness",
> >> Material.class );
> >> query.setParameter( "grade", "AAA" );
> >> query.setParameter( "width", 5 );
> >> query.setParameter( "thickness", 5. ); List< Material >
> >> mats = query.getResultList();
> >>
> >> to
> >> List< Material > mats = matRepo.findByGradeAndWidthAndThickness(
> >> "AAA", 5, 5. );
> >>
> >> Here again the difference is quite high.
> >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> >> 10240 |
> >> |===========================================================
> >> ============================================================
> >> =============================|
> >> | DS| 0.03988012 | 0.151870613| 0.144881044| 0.270389952|
> >> | DS| 0.526700787|
> >> 1.023574545| 1.806960223| 3.426772405| 6.969935385| 13.963582543|
> >> 26.785764953|
> >> | EM| 0.010984804| 0.021940339| 0.059921297| 0.087386918|
> >> | EM| 0.171045079|
> >> 0.375059016| 0.747171594| 1.560946145| 2.968347174| 6.446844753 |
> >> 12.361550486|
> >>
> >> So as you can see the DeltaSpike implementation needs at least the
> >> double amount of time.
> >>
> >> Any hints for improving the performance?
> >>
> >> Regards,
> >> Johannes
> >>
> >> -----Original Message-----
> >> From: Schäfer, Johannes [mailto:jschaefer@psi.de]
> >> Sent: Thursday, June 1, 2017 2:27 PM
> >> To: users@deltaspike.apache.org
> >> Subject: RE: Performance of DeltaSpike Data
> >>
> >> Right. Copy and paste error.
> >> I added also a flush to the EM test.
> >> Now I have similar numbers.
> >> ____________________________________________________________
> >> ____________________________________________________________
> >> ______________________________
> >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> >> 10240 |
> >> |====================================================================
> >> |===
> >> |====================================================================
> >> |===
> >> |=======|
> >> | DS| 0.001588214| 0.004130191| 0.007351854| 0.014062036|
> >> | DS| 0.048373222| 0.593463008| 0.741351593| 1.697058004|
> >> | DS| 6.049719288| 34.101109279| 101.589077365|
> >> | EM| 0.001385601| 0.002662861| 0.004092937| 0.108730649|
> >> | EM| 0.046299193| 0.106900289| 0.461147505| 1.688040769|
> >> | EM| 5.960683928| 25.604583163| 106.688041149|
> >>
> >> It's a little bit strange for me, why I have to compare
> >> EntityPersistenceRepository.save with a em.persist + em.flush. I
> >> would expect that an simple EntityPersistenceRepository.save don't
> >> have a flush (there is a separate EntityPersistenceRepository.
> saveAndFlush).
> >>
> >> When I do a run with EntityPersistenceRepository.saveAndFlush I get
> >> the following numbers.
> >> ____________________________________________________________
> >> ____________________________________________________________
> >> ______________________________
> >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> >> 10240 |
> >> |====================================================================
> >> |===
> >> |====================================================================
> >> |===
> >> |=======|
> >> | DS| 0.001703015| 0.003457728| 0.008079817| 0.019099994|
> >> | DS| 0.053865065| 0.940319597| 0.643245399| 2.292716685|
> >> | DS| 9.902395587| 40.84301017 | 172.179435413|
> >> | EM| 0.001677545| 0.004168205| 0.005779986| 0.014491211|
> >> | EM| 0.031066334| 0.110747277| 0.4051742 | 1.925682412|
> >> | EM| 5.842606084| 23.540393571| 132.817886521|
> >>
> >> So I have the feeling that there is still something wrong.
> >>
> >> Thanks to Gerhard for his additional hints.
> >> I committed all changes to the github repo.
> >>
> >> Regards,
> >> Johannes
> >>
> >> -----Original Message-----
> >> From: Gerhard Petracek [mailto:gpetracek@apache.org]
> >> Sent: Thursday, June 1, 2017 1:21 PM
> >> To: users@deltaspike.apache.org
> >> Subject: Re: Performance of DeltaSpike Data
> >>
> >> @johannes:
> >> as mentioned yesterday you have to move EntityTransaction#begin and
> >> EntityTransaction#commit into the for-loop.
> >>
> >> regards,
> >> gerhard
> >>
> >>
> >>
> >> 2017-06-01 12:58 GMT+02:00 Thomas Andraschko
> >> <andraschko.thomas@gmail.com
> >> >:
> >>
> >> > Hi,
> >> >
> >> > ~1 year ago i did many optimizations in the data module and AFAIR
> >> > DS Data was only a little bit slower.
> >> > After i compared my testcase with a benchmark from a user, the big
> >> > different came from the transaction handling which was different in
> >> > both testcases.
> >> >
> >> > Regards,
> >> > Thomas
> >> >
> >> > 2017-06-01 12:33 GMT+02:00 Gerhard Petracek <gp...@apache.org>:
> >> >
> >> > > hi johannes,
> >> > >
> >> > > after refactoring your initial code to ds-test-control i saw e.g.
> >> > > ~6s vs 7,5s for 2560 iterations.
> >> > > i'll compare my local version with your new version (mentioned in
> >> > > your mail).
> >> > >
> >> > > regards,
> >> > > gerhard
> >> > >
> >> > >
> >> > >
> >> > > 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> >> > >
> >> > > > Hi,
> >> > > >
> >> > > > My company is thinking about using DeltaSpike Data. But before
> >> > > > we integrate this into our development I was asked to prepare
> >> > > > some
> >> > > benchmarks,
> >> > > > comparing the usage of DeltaSpike Data with the usage of a
> >> > > > plain EntityManager.
> >> > > > I prepared some benchmarks and I was surprised that there is a
> >> > > > big difference in the write performance. I already got some
> >> > > > hints in the
> >> > > delta
> >> > > > spike irc channel, but the performance is still bad.
> >> > > > Based on a template from os890 I implemented my tests and
> >> > > > prepared a github project.
> >> > > > https://github.com/johannesschaefer/javaee_jsf_
> >> > cdi_jpa_data_ds_project_
> >> > > > template
> >> > > > Basically I'm talking about this test:
> >> > > > https://github.com/johannesschaefer/javaee_jsf_
> >> > cdi_jpa_data_ds_project_
> >> > > > template/blob/master/src/test/java/de/psi/metals/futurelab/
> >> > > > repo/benchmark/SaveTest.java
> >> > > >
> >> > > > It just saves an entity into a DB in a loop. Depending of the
> >> > > > number of iterations the difference is quite big.
> >> > > >
> >> > > > SaveTest
> >> > > > ____________________________________________________________
> >> > > > ____________________________________________________________
> >> > > > _____________________________
> >> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
> >> |
> >> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
> >> iter
> >> > > > 10240 |
> >> > > > |===========================================================
> >> > > > ============================================================
> >> > > > =============================|
> >> > > > | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639|
> >> > > > | DS| 0.043319612|
> >> > > > 0.281807839| 1.308948835| 1.370535533| 8.186996818|
> >> > > > 20.920141274| 93.631768475|
> >> > > > | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958|
> >> > > > | EM| 0.028243393|
> >> > > > 0.035484616| 0.038600595| 0.088904458| 0.339158674| 0.745850523
> >> > > > |
> >> > > > 0.853543234 |
> >> > > >
> >> > > > Also the difference between a run with 5120 and 10240 iteration
> >> > > > is not just the double amount of time, it is more than 4 times
> more.
> >> > > >
> >> > > > Do you have some hints to me what I'm doing wrong there?
> >> > > >
> >> > > > Regards
> >> > > > Johannes
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
RE: Performance of DeltaSpike Data
Posted by Schäfer,
Johannes <js...@psi.de>.
Hi,
Thanks for your great support. Now I had the time to run the tests.
Unfortunately no improvement. :-(
I used Mysql and H2 and both still have a significant difference.
mysql:
_____________________________________________________________________________________________________________________________________________________
| | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 | iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter 10240 |
|====================================================================================================================================================|
| DS| 0.042993818| 0.070756327| 0.139015158| 0.249963317| 0.489673972| 1.000932095| 1.418196146| 3.396942334| 6.268094687| 12.142304859| 24.631240985|
| EM| 0.016741971| 0.034018415| 0.042539175| 0.097203944| 0.15662194 | 0.32694476 | 0.665341891| 1.582051703| 2.602520533| 5.710082816 | 10.856276852|
h2:
__________________________________________________________________________________________________________________________________________________
| | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 | iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter 10240 |
|=================================================================================================================================================|
| DS| 0.007259847| 0.009839833| 0.024182004| 0.040194493| 0.037355253| 0.038992501| 0.15695646| 0.157184542| 0.277937182| 0.575950893| 0.848326297|
| EM| 7.12756E-4 | 7.15797E-4 | 0.0035079 | 0.001897262| 0.003144109| 0.007000594| 0.01269694| 0.024183904| 0.037443446| 0.108577248| 0.217664259|
I used version 1.8.1-SNAPSHOT for testing.
See https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ds_project_template
Grüße
Johannes
-----Original Message-----
From: Thomas Andraschko [mailto:andraschko.thomas@gmail.com]
Sent: Thursday, June 8, 2017 10:36 AM
To: users@deltaspike.apache.org
Subject: Re: Performance of DeltaSpike Data
I did a major improvement and in my tests, both plain JPA and DS Data are now very similar.
Would be great if you could provide the new numbers.
2017-06-07 14:33 GMT+02:00 Thomas Andraschko <an...@gmail.com>:
> Hi,
>
> could you please try to run your test again against the github master?
> I already did a small improvement and refactored a little bit on the
> weekend.
>
> 2017-06-06 8:54 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
>
>> Hi,
>>
>> So after the a long weekend, I'm back with my results.
>> For the write, findByPK and findAll tests I get now good numbers.
>> See:
>> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
>> ds_project_template/blob/master/src/test/java/de/psi/
>> metals/futurelab/repo/benchmark/SaveTest.java
>> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
>> ds_project_template/blob/master/src/test/java/de/psi/
>> metals/futurelab/repo/benchmark/ReadTest.java
>> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
>> ds_project_template/blob/master/src/test/java/de/psi/
>> metals/futurelab/repo/benchmark/ReadAllTest.java
>>
>> The difference between delta spike and plain EM are just a few
>> percent, in both directions ;-) .
>>
>> But I wrote a new test case were I try to find entities by an query.
>> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
>> ds_project_template/blob/master/src/test/java/de/psi/
>> metals/futurelab/repo/benchmark/ReadQueryTest.java
>> So I compare
>> TypedQuery< Material > query = eml.createQuery(
>> "SELECT m FROM Material m WHERE grade = :grade AND
>> width = :width AND thickness = :thickness",
>> Material.class );
>> query.setParameter( "grade", "AAA" );
>> query.setParameter( "width", 5 );
>> query.setParameter( "thickness", 5. ); List< Material >
>> mats = query.getResultList();
>>
>> to
>> List< Material > mats = matRepo.findByGradeAndWidthAndThickness(
>> "AAA", 5, 5. );
>>
>> Here again the difference is quite high.
>> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
>> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
>> 10240 |
>> |===========================================================
>> ============================================================
>> =============================|
>> | DS| 0.03988012 | 0.151870613| 0.144881044| 0.270389952|
>> | DS| 0.526700787|
>> 1.023574545| 1.806960223| 3.426772405| 6.969935385| 13.963582543|
>> 26.785764953|
>> | EM| 0.010984804| 0.021940339| 0.059921297| 0.087386918|
>> | EM| 0.171045079|
>> 0.375059016| 0.747171594| 1.560946145| 2.968347174| 6.446844753 |
>> 12.361550486|
>>
>> So as you can see the DeltaSpike implementation needs at least the
>> double amount of time.
>>
>> Any hints for improving the performance?
>>
>> Regards,
>> Johannes
>>
>> -----Original Message-----
>> From: Schäfer, Johannes [mailto:jschaefer@psi.de]
>> Sent: Thursday, June 1, 2017 2:27 PM
>> To: users@deltaspike.apache.org
>> Subject: RE: Performance of DeltaSpike Data
>>
>> Right. Copy and paste error.
>> I added also a flush to the EM test.
>> Now I have similar numbers.
>> ____________________________________________________________
>> ____________________________________________________________
>> ______________________________
>> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
>> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
>> 10240 |
>> |====================================================================
>> |===
>> |====================================================================
>> |===
>> |=======|
>> | DS| 0.001588214| 0.004130191| 0.007351854| 0.014062036|
>> | DS| 0.048373222| 0.593463008| 0.741351593| 1.697058004|
>> | DS| 6.049719288| 34.101109279| 101.589077365|
>> | EM| 0.001385601| 0.002662861| 0.004092937| 0.108730649|
>> | EM| 0.046299193| 0.106900289| 0.461147505| 1.688040769|
>> | EM| 5.960683928| 25.604583163| 106.688041149|
>>
>> It's a little bit strange for me, why I have to compare
>> EntityPersistenceRepository.save with a em.persist + em.flush. I
>> would expect that an simple EntityPersistenceRepository.save don't
>> have a flush (there is a separate EntityPersistenceRepository.saveAndFlush).
>>
>> When I do a run with EntityPersistenceRepository.saveAndFlush I get
>> the following numbers.
>> ____________________________________________________________
>> ____________________________________________________________
>> ______________________________
>> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
>> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
>> 10240 |
>> |====================================================================
>> |===
>> |====================================================================
>> |===
>> |=======|
>> | DS| 0.001703015| 0.003457728| 0.008079817| 0.019099994|
>> | DS| 0.053865065| 0.940319597| 0.643245399| 2.292716685|
>> | DS| 9.902395587| 40.84301017 | 172.179435413|
>> | EM| 0.001677545| 0.004168205| 0.005779986| 0.014491211|
>> | EM| 0.031066334| 0.110747277| 0.4051742 | 1.925682412|
>> | EM| 5.842606084| 23.540393571| 132.817886521|
>>
>> So I have the feeling that there is still something wrong.
>>
>> Thanks to Gerhard for his additional hints.
>> I committed all changes to the github repo.
>>
>> Regards,
>> Johannes
>>
>> -----Original Message-----
>> From: Gerhard Petracek [mailto:gpetracek@apache.org]
>> Sent: Thursday, June 1, 2017 1:21 PM
>> To: users@deltaspike.apache.org
>> Subject: Re: Performance of DeltaSpike Data
>>
>> @johannes:
>> as mentioned yesterday you have to move EntityTransaction#begin and
>> EntityTransaction#commit into the for-loop.
>>
>> regards,
>> gerhard
>>
>>
>>
>> 2017-06-01 12:58 GMT+02:00 Thomas Andraschko
>> <andraschko.thomas@gmail.com
>> >:
>>
>> > Hi,
>> >
>> > ~1 year ago i did many optimizations in the data module and AFAIR
>> > DS Data was only a little bit slower.
>> > After i compared my testcase with a benchmark from a user, the big
>> > different came from the transaction handling which was different in
>> > both testcases.
>> >
>> > Regards,
>> > Thomas
>> >
>> > 2017-06-01 12:33 GMT+02:00 Gerhard Petracek <gp...@apache.org>:
>> >
>> > > hi johannes,
>> > >
>> > > after refactoring your initial code to ds-test-control i saw e.g.
>> > > ~6s vs 7,5s for 2560 iterations.
>> > > i'll compare my local version with your new version (mentioned in
>> > > your mail).
>> > >
>> > > regards,
>> > > gerhard
>> > >
>> > >
>> > >
>> > > 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
>> > >
>> > > > Hi,
>> > > >
>> > > > My company is thinking about using DeltaSpike Data. But before
>> > > > we integrate this into our development I was asked to prepare
>> > > > some
>> > > benchmarks,
>> > > > comparing the usage of DeltaSpike Data with the usage of a
>> > > > plain EntityManager.
>> > > > I prepared some benchmarks and I was surprised that there is a
>> > > > big difference in the write performance. I already got some
>> > > > hints in the
>> > > delta
>> > > > spike irc channel, but the performance is still bad.
>> > > > Based on a template from os890 I implemented my tests and
>> > > > prepared a github project.
>> > > > https://github.com/johannesschaefer/javaee_jsf_
>> > cdi_jpa_data_ds_project_
>> > > > template
>> > > > Basically I'm talking about this test:
>> > > > https://github.com/johannesschaefer/javaee_jsf_
>> > cdi_jpa_data_ds_project_
>> > > > template/blob/master/src/test/java/de/psi/metals/futurelab/
>> > > > repo/benchmark/SaveTest.java
>> > > >
>> > > > It just saves an entity into a DB in a loop. Depending of the
>> > > > number of iterations the difference is quite big.
>> > > >
>> > > > SaveTest
>> > > > ____________________________________________________________
>> > > > ____________________________________________________________
>> > > > _____________________________
>> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
>> |
>> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
>> iter
>> > > > 10240 |
>> > > > |===========================================================
>> > > > ============================================================
>> > > > =============================|
>> > > > | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639|
>> > > > | DS| 0.043319612|
>> > > > 0.281807839| 1.308948835| 1.370535533| 8.186996818|
>> > > > 20.920141274| 93.631768475|
>> > > > | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958|
>> > > > | EM| 0.028243393|
>> > > > 0.035484616| 0.038600595| 0.088904458| 0.339158674| 0.745850523
>> > > > |
>> > > > 0.853543234 |
>> > > >
>> > > > Also the difference between a run with 5120 and 10240 iteration
>> > > > is not just the double amount of time, it is more than 4 times more.
>> > > >
>> > > > Do you have some hints to me what I'm doing wrong there?
>> > > >
>> > > > Regards
>> > > > Johannes
>> > > >
>> > > >
>> > > >
>> > >
>> >
>>
>
>
Re: Performance of DeltaSpike Data
Posted by Thomas Andraschko <an...@gmail.com>.
I did a major improvement and in my tests, both plain JPA and DS Data are
now very similar.
Would be great if you could provide the new numbers.
2017-06-07 14:33 GMT+02:00 Thomas Andraschko <an...@gmail.com>:
> Hi,
>
> could you please try to run your test again against the github master?
> I already did a small improvement and refactored a little bit on the
> weekend.
>
> 2017-06-06 8:54 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
>
>> Hi,
>>
>> So after the a long weekend, I'm back with my results.
>> For the write, findByPK and findAll tests I get now good numbers.
>> See:
>> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
>> ds_project_template/blob/master/src/test/java/de/psi/
>> metals/futurelab/repo/benchmark/SaveTest.java
>> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
>> ds_project_template/blob/master/src/test/java/de/psi/
>> metals/futurelab/repo/benchmark/ReadTest.java
>> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
>> ds_project_template/blob/master/src/test/java/de/psi/
>> metals/futurelab/repo/benchmark/ReadAllTest.java
>>
>> The difference between delta spike and plain EM are just a few percent,
>> in both directions ;-) .
>>
>> But I wrote a new test case were I try to find entities by an query.
>> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_
>> ds_project_template/blob/master/src/test/java/de/psi/
>> metals/futurelab/repo/benchmark/ReadQueryTest.java
>> So I compare
>> TypedQuery< Material > query = eml.createQuery(
>> "SELECT m FROM Material m WHERE grade = :grade AND width
>> = :width AND thickness = :thickness",
>> Material.class );
>> query.setParameter( "grade", "AAA" );
>> query.setParameter( "width", 5 );
>> query.setParameter( "thickness", 5. );
>> List< Material > mats = query.getResultList();
>>
>> to
>> List< Material > mats = matRepo.findByGradeAndWidthAndThickness( "AAA",
>> 5, 5. );
>>
>> Here again the difference is quite high.
>> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
>> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
>> 10240 |
>> |===========================================================
>> ============================================================
>> =============================|
>> | DS| 0.03988012 | 0.151870613| 0.144881044| 0.270389952| 0.526700787|
>> 1.023574545| 1.806960223| 3.426772405| 6.969935385| 13.963582543|
>> 26.785764953|
>> | EM| 0.010984804| 0.021940339| 0.059921297| 0.087386918| 0.171045079|
>> 0.375059016| 0.747171594| 1.560946145| 2.968347174| 6.446844753 |
>> 12.361550486|
>>
>> So as you can see the DeltaSpike implementation needs at least the double
>> amount of time.
>>
>> Any hints for improving the performance?
>>
>> Regards,
>> Johannes
>>
>> -----Original Message-----
>> From: Schäfer, Johannes [mailto:jschaefer@psi.de]
>> Sent: Thursday, June 1, 2017 2:27 PM
>> To: users@deltaspike.apache.org
>> Subject: RE: Performance of DeltaSpike Data
>>
>> Right. Copy and paste error.
>> I added also a flush to the EM test.
>> Now I have similar numbers.
>> ____________________________________________________________
>> ____________________________________________________________
>> ______________________________
>> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
>> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
>> 10240 |
>> |=======================================================================
>> |=======================================================================
>> |=======|
>> | DS| 0.001588214| 0.004130191| 0.007351854| 0.014062036| 0.048373222|
>> | DS| 0.593463008| 0.741351593| 1.697058004| 6.049719288| 34.101109279|
>> | DS| 101.589077365|
>> | EM| 0.001385601| 0.002662861| 0.004092937| 0.108730649| 0.046299193|
>> | EM| 0.106900289| 0.461147505| 1.688040769| 5.960683928| 25.604583163|
>> | EM| 106.688041149|
>>
>> It's a little bit strange for me, why I have to compare
>> EntityPersistenceRepository.save with a em.persist + em.flush. I would
>> expect that an simple EntityPersistenceRepository.save don't have a
>> flush (there is a separate EntityPersistenceRepository.saveAndFlush).
>>
>> When I do a run with EntityPersistenceRepository.saveAndFlush I get the
>> following numbers.
>> ____________________________________________________________
>> ____________________________________________________________
>> ______________________________
>> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
>> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
>> 10240 |
>> |=======================================================================
>> |=======================================================================
>> |=======|
>> | DS| 0.001703015| 0.003457728| 0.008079817| 0.019099994| 0.053865065|
>> | DS| 0.940319597| 0.643245399| 2.292716685| 9.902395587| 40.84301017 |
>> | DS| 172.179435413|
>> | EM| 0.001677545| 0.004168205| 0.005779986| 0.014491211| 0.031066334|
>> | EM| 0.110747277| 0.4051742 | 1.925682412| 5.842606084| 23.540393571|
>> | EM| 132.817886521|
>>
>> So I have the feeling that there is still something wrong.
>>
>> Thanks to Gerhard for his additional hints.
>> I committed all changes to the github repo.
>>
>> Regards,
>> Johannes
>>
>> -----Original Message-----
>> From: Gerhard Petracek [mailto:gpetracek@apache.org]
>> Sent: Thursday, June 1, 2017 1:21 PM
>> To: users@deltaspike.apache.org
>> Subject: Re: Performance of DeltaSpike Data
>>
>> @johannes:
>> as mentioned yesterday you have to move EntityTransaction#begin and
>> EntityTransaction#commit into the for-loop.
>>
>> regards,
>> gerhard
>>
>>
>>
>> 2017-06-01 12:58 GMT+02:00 Thomas Andraschko <andraschko.thomas@gmail.com
>> >:
>>
>> > Hi,
>> >
>> > ~1 year ago i did many optimizations in the data module and AFAIR DS
>> > Data was only a little bit slower.
>> > After i compared my testcase with a benchmark from a user, the big
>> > different came from the transaction handling which was different in
>> > both testcases.
>> >
>> > Regards,
>> > Thomas
>> >
>> > 2017-06-01 12:33 GMT+02:00 Gerhard Petracek <gp...@apache.org>:
>> >
>> > > hi johannes,
>> > >
>> > > after refactoring your initial code to ds-test-control i saw e.g.
>> > > ~6s vs 7,5s for 2560 iterations.
>> > > i'll compare my local version with your new version (mentioned in
>> > > your mail).
>> > >
>> > > regards,
>> > > gerhard
>> > >
>> > >
>> > >
>> > > 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
>> > >
>> > > > Hi,
>> > > >
>> > > > My company is thinking about using DeltaSpike Data. But before we
>> > > > integrate this into our development I was asked to prepare some
>> > > benchmarks,
>> > > > comparing the usage of DeltaSpike Data with the usage of a plain
>> > > > EntityManager.
>> > > > I prepared some benchmarks and I was surprised that there is a big
>> > > > difference in the write performance. I already got some hints in
>> > > > the
>> > > delta
>> > > > spike irc channel, but the performance is still bad.
>> > > > Based on a template from os890 I implemented my tests and prepared
>> > > > a github project.
>> > > > https://github.com/johannesschaefer/javaee_jsf_
>> > cdi_jpa_data_ds_project_
>> > > > template
>> > > > Basically I'm talking about this test:
>> > > > https://github.com/johannesschaefer/javaee_jsf_
>> > cdi_jpa_data_ds_project_
>> > > > template/blob/master/src/test/java/de/psi/metals/futurelab/
>> > > > repo/benchmark/SaveTest.java
>> > > >
>> > > > It just saves an entity into a DB in a loop. Depending of the
>> > > > number of iterations the difference is quite big.
>> > > >
>> > > > SaveTest
>> > > > ____________________________________________________________
>> > > > ____________________________________________________________
>> > > > _____________________________
>> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
>> |
>> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
>> iter
>> > > > 10240 |
>> > > > |===========================================================
>> > > > ============================================================
>> > > > =============================|
>> > > > | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639|
>> > > > | DS| 0.043319612|
>> > > > 0.281807839| 1.308948835| 1.370535533| 8.186996818| 20.920141274|
>> > > > 93.631768475|
>> > > > | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958|
>> > > > | EM| 0.028243393|
>> > > > 0.035484616| 0.038600595| 0.088904458| 0.339158674| 0.745850523 |
>> > > > 0.853543234 |
>> > > >
>> > > > Also the difference between a run with 5120 and 10240 iteration is
>> > > > not just the double amount of time, it is more than 4 times more.
>> > > >
>> > > > Do you have some hints to me what I'm doing wrong there?
>> > > >
>> > > > Regards
>> > > > Johannes
>> > > >
>> > > >
>> > > >
>> > >
>> >
>>
>
>
Re: Performance of DeltaSpike Data
Posted by Thomas Andraschko <an...@gmail.com>.
Hi,
could you please try to run your test again against the github master?
I already did a small improvement and refactored a little bit on the
weekend.
2017-06-06 8:54 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> Hi,
>
> So after the a long weekend, I'm back with my results.
> For the write, findByPK and findAll tests I get now good numbers.
> See:
> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ds_project_
> template/blob/master/src/test/java/de/psi/metals/futurelab/
> repo/benchmark/SaveTest.java
> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ds_project_
> template/blob/master/src/test/java/de/psi/metals/futurelab/
> repo/benchmark/ReadTest.java
> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ds_project_
> template/blob/master/src/test/java/de/psi/metals/futurelab/
> repo/benchmark/ReadAllTest.java
>
> The difference between delta spike and plain EM are just a few percent, in
> both directions ;-) .
>
> But I wrote a new test case were I try to find entities by an query.
> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ds_project_
> template/blob/master/src/test/java/de/psi/metals/futurelab/
> repo/benchmark/ReadQueryTest.java
> So I compare
> TypedQuery< Material > query = eml.createQuery(
> "SELECT m FROM Material m WHERE grade = :grade AND width =
> :width AND thickness = :thickness",
> Material.class );
> query.setParameter( "grade", "AAA" );
> query.setParameter( "width", 5 );
> query.setParameter( "thickness", 5. );
> List< Material > mats = query.getResultList();
>
> to
> List< Material > mats = matRepo.findByGradeAndWidthAndThickness( "AAA",
> 5, 5. );
>
> Here again the difference is quite high.
> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> 10240 |
> |===========================================================
> ============================================================
> =============================|
> | DS| 0.03988012 | 0.151870613| 0.144881044| 0.270389952| 0.526700787|
> 1.023574545| 1.806960223| 3.426772405| 6.969935385| 13.963582543|
> 26.785764953|
> | EM| 0.010984804| 0.021940339| 0.059921297| 0.087386918| 0.171045079|
> 0.375059016| 0.747171594| 1.560946145| 2.968347174| 6.446844753 |
> 12.361550486|
>
> So as you can see the DeltaSpike implementation needs at least the double
> amount of time.
>
> Any hints for improving the performance?
>
> Regards,
> Johannes
>
> -----Original Message-----
> From: Schäfer, Johannes [mailto:jschaefer@psi.de]
> Sent: Thursday, June 1, 2017 2:27 PM
> To: users@deltaspike.apache.org
> Subject: RE: Performance of DeltaSpike Data
>
> Right. Copy and paste error.
> I added also a flush to the EM test.
> Now I have similar numbers.
> ____________________________________________________________
> ____________________________________________________________
> ______________________________
> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> 10240 |
> |=======================================================================
> |=======================================================================
> |=======|
> | DS| 0.001588214| 0.004130191| 0.007351854| 0.014062036| 0.048373222|
> | DS| 0.593463008| 0.741351593| 1.697058004| 6.049719288| 34.101109279|
> | DS| 101.589077365|
> | EM| 0.001385601| 0.002662861| 0.004092937| 0.108730649| 0.046299193|
> | EM| 0.106900289| 0.461147505| 1.688040769| 5.960683928| 25.604583163|
> | EM| 106.688041149|
>
> It's a little bit strange for me, why I have to compare
> EntityPersistenceRepository.save with a em.persist + em.flush. I would
> expect that an simple EntityPersistenceRepository.save don't have a flush
> (there is a separate EntityPersistenceRepository.saveAndFlush).
>
> When I do a run with EntityPersistenceRepository.saveAndFlush I get the
> following numbers.
> ____________________________________________________________
> ____________________________________________________________
> ______________________________
> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> 10240 |
> |=======================================================================
> |=======================================================================
> |=======|
> | DS| 0.001703015| 0.003457728| 0.008079817| 0.019099994| 0.053865065|
> | DS| 0.940319597| 0.643245399| 2.292716685| 9.902395587| 40.84301017 |
> | DS| 172.179435413|
> | EM| 0.001677545| 0.004168205| 0.005779986| 0.014491211| 0.031066334|
> | EM| 0.110747277| 0.4051742 | 1.925682412| 5.842606084| 23.540393571|
> | EM| 132.817886521|
>
> So I have the feeling that there is still something wrong.
>
> Thanks to Gerhard for his additional hints.
> I committed all changes to the github repo.
>
> Regards,
> Johannes
>
> -----Original Message-----
> From: Gerhard Petracek [mailto:gpetracek@apache.org]
> Sent: Thursday, June 1, 2017 1:21 PM
> To: users@deltaspike.apache.org
> Subject: Re: Performance of DeltaSpike Data
>
> @johannes:
> as mentioned yesterday you have to move EntityTransaction#begin and
> EntityTransaction#commit into the for-loop.
>
> regards,
> gerhard
>
>
>
> 2017-06-01 12:58 GMT+02:00 Thomas Andraschko <andraschko.thomas@gmail.com
> >:
>
> > Hi,
> >
> > ~1 year ago i did many optimizations in the data module and AFAIR DS
> > Data was only a little bit slower.
> > After i compared my testcase with a benchmark from a user, the big
> > different came from the transaction handling which was different in
> > both testcases.
> >
> > Regards,
> > Thomas
> >
> > 2017-06-01 12:33 GMT+02:00 Gerhard Petracek <gp...@apache.org>:
> >
> > > hi johannes,
> > >
> > > after refactoring your initial code to ds-test-control i saw e.g.
> > > ~6s vs 7,5s for 2560 iterations.
> > > i'll compare my local version with your new version (mentioned in
> > > your mail).
> > >
> > > regards,
> > > gerhard
> > >
> > >
> > >
> > > 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> > >
> > > > Hi,
> > > >
> > > > My company is thinking about using DeltaSpike Data. But before we
> > > > integrate this into our development I was asked to prepare some
> > > benchmarks,
> > > > comparing the usage of DeltaSpike Data with the usage of a plain
> > > > EntityManager.
> > > > I prepared some benchmarks and I was surprised that there is a big
> > > > difference in the write performance. I already got some hints in
> > > > the
> > > delta
> > > > spike irc channel, but the performance is still bad.
> > > > Based on a template from os890 I implemented my tests and prepared
> > > > a github project.
> > > > https://github.com/johannesschaefer/javaee_jsf_
> > cdi_jpa_data_ds_project_
> > > > template
> > > > Basically I'm talking about this test:
> > > > https://github.com/johannesschaefer/javaee_jsf_
> > cdi_jpa_data_ds_project_
> > > > template/blob/master/src/test/java/de/psi/metals/futurelab/
> > > > repo/benchmark/SaveTest.java
> > > >
> > > > It just saves an entity into a DB in a loop. Depending of the
> > > > number of iterations the difference is quite big.
> > > >
> > > > SaveTest
> > > > ____________________________________________________________
> > > > ____________________________________________________________
> > > > _____________________________
> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160
> |
> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 |
> iter
> > > > 10240 |
> > > > |===========================================================
> > > > ============================================================
> > > > =============================|
> > > > | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639|
> > > > | DS| 0.043319612|
> > > > 0.281807839| 1.308948835| 1.370535533| 8.186996818| 20.920141274|
> > > > 93.631768475|
> > > > | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958|
> > > > | EM| 0.028243393|
> > > > 0.035484616| 0.038600595| 0.088904458| 0.339158674| 0.745850523 |
> > > > 0.853543234 |
> > > >
> > > > Also the difference between a run with 5120 and 10240 iteration is
> > > > not just the double amount of time, it is more than 4 times more.
> > > >
> > > > Do you have some hints to me what I'm doing wrong there?
> > > >
> > > > Regards
> > > > Johannes
> > > >
> > > >
> > > >
> > >
> >
>
RE: Performance of DeltaSpike Data
Posted by Schäfer,
Johannes <js...@psi.de>.
Hi,
So after the a long weekend, I'm back with my results.
For the write, findByPK and findAll tests I get now good numbers.
See:
https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ds_project_template/blob/master/src/test/java/de/psi/metals/futurelab/repo/benchmark/SaveTest.java
https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ds_project_template/blob/master/src/test/java/de/psi/metals/futurelab/repo/benchmark/ReadTest.java
https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ds_project_template/blob/master/src/test/java/de/psi/metals/futurelab/repo/benchmark/ReadAllTest.java
The difference between delta spike and plain EM are just a few percent, in both directions ;-) .
But I wrote a new test case were I try to find entities by an query.
https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ds_project_template/blob/master/src/test/java/de/psi/metals/futurelab/repo/benchmark/ReadQueryTest.java
So I compare
TypedQuery< Material > query = eml.createQuery(
"SELECT m FROM Material m WHERE grade = :grade AND width = :width AND thickness = :thickness",
Material.class );
query.setParameter( "grade", "AAA" );
query.setParameter( "width", 5 );
query.setParameter( "thickness", 5. );
List< Material > mats = query.getResultList();
to
List< Material > mats = matRepo.findByGradeAndWidthAndThickness( "AAA", 5, 5. );
Here again the difference is quite high.
| | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 | iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter 10240 |
|====================================================================================================================================================|
| DS| 0.03988012 | 0.151870613| 0.144881044| 0.270389952| 0.526700787| 1.023574545| 1.806960223| 3.426772405| 6.969935385| 13.963582543| 26.785764953|
| EM| 0.010984804| 0.021940339| 0.059921297| 0.087386918| 0.171045079| 0.375059016| 0.747171594| 1.560946145| 2.968347174| 6.446844753 | 12.361550486|
So as you can see the DeltaSpike implementation needs at least the double amount of time.
Any hints for improving the performance?
Regards,
Johannes
-----Original Message-----
From: Schäfer, Johannes [mailto:jschaefer@psi.de]
Sent: Thursday, June 1, 2017 2:27 PM
To: users@deltaspike.apache.org
Subject: RE: Performance of DeltaSpike Data
Right. Copy and paste error.
I added also a flush to the EM test.
Now I have similar numbers.
______________________________________________________________________________________________________________________________________________________
| | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 | iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter 10240 |
|=======================================================================
|=======================================================================
|=======|
| DS| 0.001588214| 0.004130191| 0.007351854| 0.014062036| 0.048373222|
| DS| 0.593463008| 0.741351593| 1.697058004| 6.049719288| 34.101109279|
| DS| 101.589077365|
| EM| 0.001385601| 0.002662861| 0.004092937| 0.108730649| 0.046299193|
| EM| 0.106900289| 0.461147505| 1.688040769| 5.960683928| 25.604583163|
| EM| 106.688041149|
It's a little bit strange for me, why I have to compare EntityPersistenceRepository.save with a em.persist + em.flush. I would expect that an simple EntityPersistenceRepository.save don't have a flush (there is a separate EntityPersistenceRepository.saveAndFlush).
When I do a run with EntityPersistenceRepository.saveAndFlush I get the following numbers.
______________________________________________________________________________________________________________________________________________________
| | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 | iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter 10240 |
|=======================================================================
|=======================================================================
|=======|
| DS| 0.001703015| 0.003457728| 0.008079817| 0.019099994| 0.053865065|
| DS| 0.940319597| 0.643245399| 2.292716685| 9.902395587| 40.84301017 |
| DS| 172.179435413|
| EM| 0.001677545| 0.004168205| 0.005779986| 0.014491211| 0.031066334|
| EM| 0.110747277| 0.4051742 | 1.925682412| 5.842606084| 23.540393571|
| EM| 132.817886521|
So I have the feeling that there is still something wrong.
Thanks to Gerhard for his additional hints.
I committed all changes to the github repo.
Regards,
Johannes
-----Original Message-----
From: Gerhard Petracek [mailto:gpetracek@apache.org]
Sent: Thursday, June 1, 2017 1:21 PM
To: users@deltaspike.apache.org
Subject: Re: Performance of DeltaSpike Data
@johannes:
as mentioned yesterday you have to move EntityTransaction#begin and EntityTransaction#commit into the for-loop.
regards,
gerhard
2017-06-01 12:58 GMT+02:00 Thomas Andraschko <an...@gmail.com>:
> Hi,
>
> ~1 year ago i did many optimizations in the data module and AFAIR DS
> Data was only a little bit slower.
> After i compared my testcase with a benchmark from a user, the big
> different came from the transaction handling which was different in
> both testcases.
>
> Regards,
> Thomas
>
> 2017-06-01 12:33 GMT+02:00 Gerhard Petracek <gp...@apache.org>:
>
> > hi johannes,
> >
> > after refactoring your initial code to ds-test-control i saw e.g.
> > ~6s vs 7,5s for 2560 iterations.
> > i'll compare my local version with your new version (mentioned in
> > your mail).
> >
> > regards,
> > gerhard
> >
> >
> >
> > 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> >
> > > Hi,
> > >
> > > My company is thinking about using DeltaSpike Data. But before we
> > > integrate this into our development I was asked to prepare some
> > benchmarks,
> > > comparing the usage of DeltaSpike Data with the usage of a plain
> > > EntityManager.
> > > I prepared some benchmarks and I was surprised that there is a big
> > > difference in the write performance. I already got some hints in
> > > the
> > delta
> > > spike irc channel, but the performance is still bad.
> > > Based on a template from os890 I implemented my tests and prepared
> > > a github project.
> > > https://github.com/johannesschaefer/javaee_jsf_
> cdi_jpa_data_ds_project_
> > > template
> > > Basically I'm talking about this test:
> > > https://github.com/johannesschaefer/javaee_jsf_
> cdi_jpa_data_ds_project_
> > > template/blob/master/src/test/java/de/psi/metals/futurelab/
> > > repo/benchmark/SaveTest.java
> > >
> > > It just saves an entity into a DB in a loop. Depending of the
> > > number of iterations the difference is quite big.
> > >
> > > SaveTest
> > > ____________________________________________________________
> > > ____________________________________________________________
> > > _____________________________
> > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > > 10240 |
> > > |===========================================================
> > > ============================================================
> > > =============================|
> > > | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639|
> > > | DS| 0.043319612|
> > > 0.281807839| 1.308948835| 1.370535533| 8.186996818| 20.920141274|
> > > 93.631768475|
> > > | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958|
> > > | EM| 0.028243393|
> > > 0.035484616| 0.038600595| 0.088904458| 0.339158674| 0.745850523 |
> > > 0.853543234 |
> > >
> > > Also the difference between a run with 5120 and 10240 iteration is
> > > not just the double amount of time, it is more than 4 times more.
> > >
> > > Do you have some hints to me what I'm doing wrong there?
> > >
> > > Regards
> > > Johannes
> > >
> > >
> > >
> >
>
RE: Performance of DeltaSpike Data
Posted by Schäfer,
Johannes <js...@psi.de>.
Right. Copy and paste error.
I added also a flush to the EM test.
Now I have similar numbers.
______________________________________________________________________________________________________________________________________________________
| | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 | iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter 10240 |
|=====================================================================================================================================================|
| DS| 0.001588214| 0.004130191| 0.007351854| 0.014062036| 0.048373222| 0.593463008| 0.741351593| 1.697058004| 6.049719288| 34.101109279| 101.589077365|
| EM| 0.001385601| 0.002662861| 0.004092937| 0.108730649| 0.046299193| 0.106900289| 0.461147505| 1.688040769| 5.960683928| 25.604583163| 106.688041149|
It's a little bit strange for me, why I have to compare EntityPersistenceRepository.save with a em.persist + em.flush. I would expect that an simple EntityPersistenceRepository.save don't have a flush (there is a separate EntityPersistenceRepository.saveAndFlush).
When I do a run with EntityPersistenceRepository.saveAndFlush I get the following numbers.
______________________________________________________________________________________________________________________________________________________
| | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 | iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter 10240 |
|=====================================================================================================================================================|
| DS| 0.001703015| 0.003457728| 0.008079817| 0.019099994| 0.053865065| 0.940319597| 0.643245399| 2.292716685| 9.902395587| 40.84301017 | 172.179435413|
| EM| 0.001677545| 0.004168205| 0.005779986| 0.014491211| 0.031066334| 0.110747277| 0.4051742 | 1.925682412| 5.842606084| 23.540393571| 132.817886521|
So I have the feeling that there is still something wrong.
Thanks to Gerhard for his additional hints.
I committed all changes to the github repo.
Regards,
Johannes
-----Original Message-----
From: Gerhard Petracek [mailto:gpetracek@apache.org]
Sent: Thursday, June 1, 2017 1:21 PM
To: users@deltaspike.apache.org
Subject: Re: Performance of DeltaSpike Data
@johannes:
as mentioned yesterday you have to move EntityTransaction#begin and EntityTransaction#commit into the for-loop.
regards,
gerhard
2017-06-01 12:58 GMT+02:00 Thomas Andraschko <an...@gmail.com>:
> Hi,
>
> ~1 year ago i did many optimizations in the data module and AFAIR DS
> Data was only a little bit slower.
> After i compared my testcase with a benchmark from a user, the big
> different came from the transaction handling which was different in
> both testcases.
>
> Regards,
> Thomas
>
> 2017-06-01 12:33 GMT+02:00 Gerhard Petracek <gp...@apache.org>:
>
> > hi johannes,
> >
> > after refactoring your initial code to ds-test-control i saw e.g.
> > ~6s vs 7,5s for 2560 iterations.
> > i'll compare my local version with your new version (mentioned in
> > your mail).
> >
> > regards,
> > gerhard
> >
> >
> >
> > 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> >
> > > Hi,
> > >
> > > My company is thinking about using DeltaSpike Data. But before we
> > > integrate this into our development I was asked to prepare some
> > benchmarks,
> > > comparing the usage of DeltaSpike Data with the usage of a plain
> > > EntityManager.
> > > I prepared some benchmarks and I was surprised that there is a big
> > > difference in the write performance. I already got some hints in
> > > the
> > delta
> > > spike irc channel, but the performance is still bad.
> > > Based on a template from os890 I implemented my tests and prepared
> > > a github project.
> > > https://github.com/johannesschaefer/javaee_jsf_
> cdi_jpa_data_ds_project_
> > > template
> > > Basically I'm talking about this test:
> > > https://github.com/johannesschaefer/javaee_jsf_
> cdi_jpa_data_ds_project_
> > > template/blob/master/src/test/java/de/psi/metals/futurelab/
> > > repo/benchmark/SaveTest.java
> > >
> > > It just saves an entity into a DB in a loop. Depending of the
> > > number of iterations the difference is quite big.
> > >
> > > SaveTest
> > > ____________________________________________________________
> > > ____________________________________________________________
> > > _____________________________
> > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > > 10240 |
> > > |===========================================================
> > > ============================================================
> > > =============================|
> > > | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639|
> > > | DS| 0.043319612|
> > > 0.281807839| 1.308948835| 1.370535533| 8.186996818| 20.920141274|
> > > 93.631768475|
> > > | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958|
> > > | EM| 0.028243393|
> > > 0.035484616| 0.038600595| 0.088904458| 0.339158674| 0.745850523 |
> > > 0.853543234 |
> > >
> > > Also the difference between a run with 5120 and 10240 iteration is
> > > not just the double amount of time, it is more than 4 times more.
> > >
> > > Do you have some hints to me what I'm doing wrong there?
> > >
> > > Regards
> > > Johannes
> > >
> > >
> > >
> >
>
Re: Performance of DeltaSpike Data
Posted by Gerhard Petracek <gp...@apache.org>.
@johannes:
as mentioned yesterday you have to move EntityTransaction#begin and
EntityTransaction#commit into the for-loop.
regards,
gerhard
2017-06-01 12:58 GMT+02:00 Thomas Andraschko <an...@gmail.com>:
> Hi,
>
> ~1 year ago i did many optimizations in the data module and AFAIR DS Data
> was only a little bit slower.
> After i compared my testcase with a benchmark from a user, the big
> different came from the transaction handling which was different in both
> testcases.
>
> Regards,
> Thomas
>
> 2017-06-01 12:33 GMT+02:00 Gerhard Petracek <gp...@apache.org>:
>
> > hi johannes,
> >
> > after refactoring your initial code to ds-test-control i saw e.g. ~6s vs
> > 7,5s for 2560 iterations.
> > i'll compare my local version with your new version (mentioned in your
> > mail).
> >
> > regards,
> > gerhard
> >
> >
> >
> > 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> >
> > > Hi,
> > >
> > > My company is thinking about using DeltaSpike Data. But before we
> > > integrate this into our development I was asked to prepare some
> > benchmarks,
> > > comparing the usage of DeltaSpike Data with the usage of a plain
> > > EntityManager.
> > > I prepared some benchmarks and I was surprised that there is a big
> > > difference in the write performance. I already got some hints in the
> > delta
> > > spike irc channel, but the performance is still bad.
> > > Based on a template from os890 I implemented my tests and prepared a
> > > github project.
> > > https://github.com/johannesschaefer/javaee_jsf_
> cdi_jpa_data_ds_project_
> > > template
> > > Basically I'm talking about this test:
> > > https://github.com/johannesschaefer/javaee_jsf_
> cdi_jpa_data_ds_project_
> > > template/blob/master/src/test/java/de/psi/metals/futurelab/
> > > repo/benchmark/SaveTest.java
> > >
> > > It just saves an entity into a DB in a loop. Depending of the number of
> > > iterations the difference is quite big.
> > >
> > > SaveTest
> > > ____________________________________________________________
> > > ____________________________________________________________
> > > _____________________________
> > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > > 10240 |
> > > |===========================================================
> > > ============================================================
> > > =============================|
> > > | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639| 0.043319612|
> > > 0.281807839| 1.308948835| 1.370535533| 8.186996818| 20.920141274|
> > > 93.631768475|
> > > | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958| 0.028243393|
> > > 0.035484616| 0.038600595| 0.088904458| 0.339158674| 0.745850523 |
> > > 0.853543234 |
> > >
> > > Also the difference between a run with 5120 and 10240 iteration is not
> > > just the double amount of time, it is more than 4 times more.
> > >
> > > Do you have some hints to me what I'm doing wrong there?
> > >
> > > Regards
> > > Johannes
> > >
> > >
> > >
> >
>
Re: Performance of DeltaSpike Data
Posted by Thomas Andraschko <an...@gmail.com>.
Hi,
~1 year ago i did many optimizations in the data module and AFAIR DS Data
was only a little bit slower.
After i compared my testcase with a benchmark from a user, the big
different came from the transaction handling which was different in both
testcases.
Regards,
Thomas
2017-06-01 12:33 GMT+02:00 Gerhard Petracek <gp...@apache.org>:
> hi johannes,
>
> after refactoring your initial code to ds-test-control i saw e.g. ~6s vs
> 7,5s for 2560 iterations.
> i'll compare my local version with your new version (mentioned in your
> mail).
>
> regards,
> gerhard
>
>
>
> 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
>
> > Hi,
> >
> > My company is thinking about using DeltaSpike Data. But before we
> > integrate this into our development I was asked to prepare some
> benchmarks,
> > comparing the usage of DeltaSpike Data with the usage of a plain
> > EntityManager.
> > I prepared some benchmarks and I was surprised that there is a big
> > difference in the write performance. I already got some hints in the
> delta
> > spike irc channel, but the performance is still bad.
> > Based on a template from os890 I implemented my tests and prepared a
> > github project.
> > https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ds_project_
> > template
> > Basically I'm talking about this test:
> > https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ds_project_
> > template/blob/master/src/test/java/de/psi/metals/futurelab/
> > repo/benchmark/SaveTest.java
> >
> > It just saves an entity into a DB in a loop. Depending of the number of
> > iterations the difference is quite big.
> >
> > SaveTest
> > ____________________________________________________________
> > ____________________________________________________________
> > _____________________________
> > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> > 10240 |
> > |===========================================================
> > ============================================================
> > =============================|
> > | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639| 0.043319612|
> > 0.281807839| 1.308948835| 1.370535533| 8.186996818| 20.920141274|
> > 93.631768475|
> > | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958| 0.028243393|
> > 0.035484616| 0.038600595| 0.088904458| 0.339158674| 0.745850523 |
> > 0.853543234 |
> >
> > Also the difference between a run with 5120 and 10240 iteration is not
> > just the double amount of time, it is more than 4 times more.
> >
> > Do you have some hints to me what I'm doing wrong there?
> >
> > Regards
> > Johannes
> >
> >
> >
>
Re: Performance of DeltaSpike Data
Posted by Gerhard Petracek <gp...@apache.org>.
hi johannes,
after refactoring your initial code to ds-test-control i saw e.g. ~6s vs
7,5s for 2560 iterations.
i'll compare my local version with your new version (mentioned in your
mail).
regards,
gerhard
2017-06-01 11:35 GMT+02:00 Schäfer, Johannes <js...@psi.de>:
> Hi,
>
> My company is thinking about using DeltaSpike Data. But before we
> integrate this into our development I was asked to prepare some benchmarks,
> comparing the usage of DeltaSpike Data with the usage of a plain
> EntityManager.
> I prepared some benchmarks and I was surprised that there is a big
> difference in the write performance. I already got some hints in the delta
> spike irc channel, but the performance is still bad.
> Based on a template from os890 I implemented my tests and prepared a
> github project.
> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ds_project_
> template
> Basically I'm talking about this test:
> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ds_project_
> template/blob/master/src/test/java/de/psi/metals/futurelab/
> repo/benchmark/SaveTest.java
>
> It just saves an entity into a DB in a loop. Depending of the number of
> iterations the difference is quite big.
>
> SaveTest
> ____________________________________________________________
> ____________________________________________________________
> _____________________________
> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 |
> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter
> 10240 |
> |===========================================================
> ============================================================
> =============================|
> | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639| 0.043319612|
> 0.281807839| 1.308948835| 1.370535533| 8.186996818| 20.920141274|
> 93.631768475|
> | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958| 0.028243393|
> 0.035484616| 0.038600595| 0.088904458| 0.339158674| 0.745850523 |
> 0.853543234 |
>
> Also the difference between a run with 5120 and 10240 iteration is not
> just the double amount of time, it is more than 4 times more.
>
> Do you have some hints to me what I'm doing wrong there?
>
> Regards
> Johannes
>
>
>