You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@openjpa.apache.org by Janos Haber <bo...@gmail.com> on 2008/02/07 14:08:22 UTC
insert 500k row
Hi!
I try to insert 500.000 row to the table (for testing). When my code
run the java eat 100% cpu (if I use hibernate it's 1-10%, but I want
to use openjpa).
Anybody know why?
testDao.persist simply call EntityManager.persist.
Cow
ps:
My code:
public final static long repeat = 50 * 10000;
...
@org.junit.Test
public void fieldNumber() {
logger.info("Create new datas");
for (int i = 0; i < repeat; i++) {
Test t = new Test();
t.setFieldA1("Field test a1 = " + i);
t.setFieldA2("Field test a2 = " + i);
t.setFieldA3("Field test a3 = " + i);
t.setFieldA4("Field test a4 = " + i);
t.setFieldA5("Field test a5 = " + i);
t.setFieldA6("Field test a6 = " + i);
t.setFieldA7("Field test a7 = " + i);
t.setFieldA8("Field test a8 = " + i);
t.setTestName(i + " name field");
t.setTestBool((i % 2 > 0) ? true : false);
t.setTestDate(new Date());
t.setTestLong(i);
t.setTestString("String field");
testDao.persist(t);
if (i % 1000 == 0) {
logger.info("Row:"+i);
//testDao.flush();
//testDao.clear();
}
}
long num = (Long) testDao
.findSingleObject("select COUNT(test) from Test test");
Assert.assertEquals(
"Too bad, the num is " + num + " and not " + repeat, repeat,
num);
}
Re: insert 500k row
Posted by Patrick Linskey <pl...@gmail.com>.
Hi,
For OpenJPA, you'll probably get the best behavior by putting the
flush() in but taking the clear() out.
Do you have any stack traces during the operations? How long do the
two runs take to execute?
-Patrick
On Feb 7, 2008 5:38 AM, Janos Haber <bo...@gmail.com> wrote:
> Yeah!
> Same source, only the library and Dialect changed (spring 2.5 framework)
>
> On Feb 7, 2008 2:12 PM, DOMINGUEZ Felipe
>
> <Fe...@eurocontrol.int> wrote:
> > Hello.
> >
> > I see you have commented out the flush() lines. Is it commented when you
> > use Hibernate too?
> >
> > Cheers
> >
> > Felipe
> >
> >
> > -----Original Message-----
> > From: Janos Haber [mailto:boci.list@gmail.com]
> > Sent: Thursday 07 February 2008 14:08
> > To: users@openjpa.apache.org
> > Subject: insert 500k row
> >
> > Hi!
> >
> > I try to insert 500.000 row to the table (for testing). When my code run
> > the java eat 100% cpu (if I use hibernate it's 1-10%, but I want to use
> > openjpa).
> >
> > Anybody know why?
> > testDao.persist simply call EntityManager.persist.
> >
> > Cow
> >
> > ps:
> >
> > My code:
> > public final static long repeat = 50 * 10000; ...
> >
> > @org.junit.Test
> > public void fieldNumber() {
> > logger.info("Create new datas");
> > for (int i = 0; i < repeat; i++) {
> > Test t = new Test();
> > t.setFieldA1("Field test a1 = " + i);
> > t.setFieldA2("Field test a2 = " + i);
> > t.setFieldA3("Field test a3 = " + i);
> > t.setFieldA4("Field test a4 = " + i);
> > t.setFieldA5("Field test a5 = " + i);
> > t.setFieldA6("Field test a6 = " + i);
> > t.setFieldA7("Field test a7 = " + i);
> > t.setFieldA8("Field test a8 = " + i);
> > t.setTestName(i + " name field");
> > t.setTestBool((i % 2 > 0) ? true : false);
> > t.setTestDate(new Date());
> > t.setTestLong(i);
> > t.setTestString("String field");
> > testDao.persist(t);
> > if (i % 1000 == 0) {
> > logger.info("Row:"+i);
> > //testDao.flush();
> > //testDao.clear();
> > }
> > }
> > long num = (Long) testDao
> > .findSingleObject("select COUNT(test)
> > from Test test");
> > Assert.assertEquals(
> > "Too bad, the num is " + num + " and not
> > " + repeat, repeat,
> > num);
> > }
> > ____
> >
> > This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.
> >
> > Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.
> >
> > Any views expressed in this message are those of the sender.
> >
> >
>
--
Patrick Linskey
202 669 5907
Re: insert 500k row
Posted by Janos Haber <bo...@gmail.com>.
Yeah!
Same source, only the library and Dialect changed (spring 2.5 framework)
On Feb 7, 2008 2:12 PM, DOMINGUEZ Felipe
<Fe...@eurocontrol.int> wrote:
> Hello.
>
> I see you have commented out the flush() lines. Is it commented when you
> use Hibernate too?
>
> Cheers
>
> Felipe
>
>
> -----Original Message-----
> From: Janos Haber [mailto:boci.list@gmail.com]
> Sent: Thursday 07 February 2008 14:08
> To: users@openjpa.apache.org
> Subject: insert 500k row
>
> Hi!
>
> I try to insert 500.000 row to the table (for testing). When my code run
> the java eat 100% cpu (if I use hibernate it's 1-10%, but I want to use
> openjpa).
>
> Anybody know why?
> testDao.persist simply call EntityManager.persist.
>
> Cow
>
> ps:
>
> My code:
> public final static long repeat = 50 * 10000; ...
>
> @org.junit.Test
> public void fieldNumber() {
> logger.info("Create new datas");
> for (int i = 0; i < repeat; i++) {
> Test t = new Test();
> t.setFieldA1("Field test a1 = " + i);
> t.setFieldA2("Field test a2 = " + i);
> t.setFieldA3("Field test a3 = " + i);
> t.setFieldA4("Field test a4 = " + i);
> t.setFieldA5("Field test a5 = " + i);
> t.setFieldA6("Field test a6 = " + i);
> t.setFieldA7("Field test a7 = " + i);
> t.setFieldA8("Field test a8 = " + i);
> t.setTestName(i + " name field");
> t.setTestBool((i % 2 > 0) ? true : false);
> t.setTestDate(new Date());
> t.setTestLong(i);
> t.setTestString("String field");
> testDao.persist(t);
> if (i % 1000 == 0) {
> logger.info("Row:"+i);
> //testDao.flush();
> //testDao.clear();
> }
> }
> long num = (Long) testDao
> .findSingleObject("select COUNT(test)
> from Test test");
> Assert.assertEquals(
> "Too bad, the num is " + num + " and not
> " + repeat, repeat,
> num);
> }
> ____
>
> This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.
>
> Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.
>
> Any views expressed in this message are those of the sender.
>
>
RE: insert 500k row
Posted by DOMINGUEZ Felipe <Fe...@eurocontrol.int>.
Hello.
I see you have commented out the flush() lines. Is it commented when you
use Hibernate too?
Cheers
Felipe
-----Original Message-----
From: Janos Haber [mailto:boci.list@gmail.com]
Sent: Thursday 07 February 2008 14:08
To: users@openjpa.apache.org
Subject: insert 500k row
Hi!
I try to insert 500.000 row to the table (for testing). When my code run
the java eat 100% cpu (if I use hibernate it's 1-10%, but I want to use
openjpa).
Anybody know why?
testDao.persist simply call EntityManager.persist.
Cow
ps:
My code:
public final static long repeat = 50 * 10000; ...
@org.junit.Test
public void fieldNumber() {
logger.info("Create new datas");
for (int i = 0; i < repeat; i++) {
Test t = new Test();
t.setFieldA1("Field test a1 = " + i);
t.setFieldA2("Field test a2 = " + i);
t.setFieldA3("Field test a3 = " + i);
t.setFieldA4("Field test a4 = " + i);
t.setFieldA5("Field test a5 = " + i);
t.setFieldA6("Field test a6 = " + i);
t.setFieldA7("Field test a7 = " + i);
t.setFieldA8("Field test a8 = " + i);
t.setTestName(i + " name field");
t.setTestBool((i % 2 > 0) ? true : false);
t.setTestDate(new Date());
t.setTestLong(i);
t.setTestString("String field");
testDao.persist(t);
if (i % 1000 == 0) {
logger.info("Row:"+i);
//testDao.flush();
//testDao.clear();
}
}
long num = (Long) testDao
.findSingleObject("select COUNT(test)
from Test test");
Assert.assertEquals(
"Too bad, the num is " + num + " and not
" + repeat, repeat,
num);
}
____
This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.
Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.
Any views expressed in this message are those of the sender.
Re: insert 500k row
Posted by Janos Haber <bo...@gmail.com>.
Another funny result.
If I send flush and clear on i % 1000 == 0 (array start from 0 so
element number 1001) I get stack overflow,
if I use i % 999 == 0 It's slow, but WORK!
I think the data cache is overflow(!!!) (from docs: "OpenJPA's default
implementation maintains a map of object ids to cache data. By
default, 1000 elements are kept in cache. When the cache overflows,
random entries are evicted. The maximum cache size can be adjusted by
setting the CacheSize property in your plugin string - see below for
an example. Objects that are pinned into the cache are not counted
when determining if the cache size exceeds its maximum size.")
So Why drop a stack overflow , not evict the entries...
cow
On Feb 8, 2008 11:23 AM, Janos Haber <bo...@gmail.com> wrote:
> Ok. Eat 100% CPU, no prob, but why not work the persistance without
> enhancer? Example I not want to use maven or ant, so I can't integrate
> to eclipse build, and junit run out of the box. Why stackoverflow? I
> think bad idea the enhancer, hibernate can work without javaagent, ok
> slower, but work. Ok JPA going slower if I not use enhancer , but NOT
> WORK. Oh! and if I add 1000 row(!!! not 200k) I get same stack
> overflow error...
>
> ps: Why not work the lazy loading without enhancer? in hibernate work,
> why not work here (no prob, if slower than enhancer "version").
>
> Cow
>
>
> On Feb 8, 2008 10:50 AM, Bruce Beaumont
>
> <br...@ucs-solutions.co.za> wrote:
> > Hi
> >
> > A observation from something I have experienced with OpenJPA.
> >
> > From your log files see the second sentence. (Someone should change this to
> > something like totally consume all CPU resources)
> >
> > -- Log snip
> > Creating subclass for "[class hu.dk.test.data.model.Test]". This means that
> > your application will be less efficient and will consume more memory than it
> > would if you ran the OpenJPA enhancer. Additionally, lazy loading will not be
> > available for one-to-one and many-to-one persistent attributes in types using
> > field access; they will be loaded eagerly instead.
> >
> >
> >
> > If you do not manually run the enhancer but rely on auto-enhancing I have seen
> > absolutely abysmal performance and 4 cores at 100 % utilization. If I run the
> > enhancer manually the performance problems go away..
> >
> >
> > Bruce
> >
> >
> >
> > On Friday 08 February 2008 10:45:39 Janos Haber wrote:
> > > Here is the Test result, Application running flags: -Xmx780m -Xms780m
> > >
> > > persist {
> > > OpenJPA without flush and clear: http://pastebin.com/f3c9492c9
> > > OpenJPA flush and clear after every 10k row: http://pastebin.com/f5ad5ef3e
> > > OpenJPA flush and clear after every 1k row: http://pastebin.com/f1f774239
> > >
> > > Hibernate: http://pastebin.com/f7ac1399e
> > > }
> > >
> > > merge {
> > > OpenJPA without flush and clear: http://pastebin.com/f2f9625d0
> > > OpenJPA flush and clear after every 10k row: http://pastebin.com/f3133d7e3
> > > OpenJPA flush and clear after every 1k row: http://pastebin.com/f33d61458
> > > OpenJPA flush and clear after every row (I can't wait for the finish,
> > > but you see the times) : http://pastebin.com/f41d84668
> > >
> > > Hibernate (without flush and clear, can't wait for finish):
> > > http://pastebin.com/f33ee87b4
> > > }
> > >
> > > So, the OpenJPA is faster if I use flush and clear, but NEVER run well
> > > (all persist failed (!!!), maybe self called method (?!?!), but the
> > > Test class not self referenced..., merge failed too except (maybe) if
> > > I use flush after every(!!!!) row, if I use 10k merge and one flush
> > > openJPA insert one row to the db, not 10.000 ).
> > >
> > > Hibernate slower (usual) yeah, but passed all test well.
> > >
> > > I know this is my fault, but what's the problem with my code?
> > >
> > > cow
> > >
> > > On Feb 7, 2008 11:14 PM, Janos Haber <bo...@gmail.com> wrote:
> > > > OpenJPA takes 100% CPU and after the row 60k going to very
> > > > slooooooooow. Now I can't tell correct values, tomorrow maybe I create
> > > > a test, but if Hibernate create 200k row about 3min, OpenJPA create
> > > > 200k row about..... 40 min. But tomorrow I create a test..
> > > >
> > > > Cow
> > > >
> > > > On Feb 7, 2008 5:35 PM, Marc Prud'hommeaux <mp...@apache.org> wrote:
> > > > > What are the respective execution times? I.e., if OpenJPA takes 100%
> > > > > of the CPU but takes 10 seconds, and Hibernate takes 10% of the CPU
> > > > > but takes 100 seconds, that might indicate that Hibernate is waiting
> > > > > on the database for some operations (like checking for the existence
> > > > > of an instance).
> > > > >
> > > > > On Feb 7, 2008, at 5:08 AM, Janos Haber wrote:
> > > > > > Hi!
> > > > > >
> > > > > > I try to insert 500.000 row to the table (for testing). When my code
> > > > > > run the java eat 100% cpu (if I use hibernate it's 1-10%, but I want
> > > > > > to use openjpa).
> > > > > >
> > > > > > Anybody know why?
> > > > > > testDao.persist simply call EntityManager.persist.
> > > > > >
> > > > > > Cow
> > > > > >
> > > > > > ps:
> > > > > >
> > > > > > My code:
> > > > > > public final static long repeat = 50 * 10000;
> > > > > > ...
> > > > > >
> > > > > > @org.junit.Test
> > > > > > public void fieldNumber() {
> > > > > > logger.info("Create new datas");
> > > > > > for (int i = 0; i < repeat; i++) {
> > > > > > Test t = new Test();
> > > > > > t.setFieldA1("Field test a1 = " + i);
> > > > > > t.setFieldA2("Field test a2 = " + i);
> > > > > > t.setFieldA3("Field test a3 = " + i);
> > > > > > t.setFieldA4("Field test a4 = " + i);
> > > > > > t.setFieldA5("Field test a5 = " + i);
> > > > > > t.setFieldA6("Field test a6 = " + i);
> > > > > > t.setFieldA7("Field test a7 = " + i);
> > > > > > t.setFieldA8("Field test a8 = " + i);
> > > > > > t.setTestName(i + " name field");
> > > > > > t.setTestBool((i % 2 > 0) ? true : false);
> > > > > > t.setTestDate(new Date());
> > > > > > t.setTestLong(i);
> > > > > > t.setTestString("String field");
> > > > > > testDao.persist(t);
> > > > > > if (i % 1000 == 0) {
> > > > > > logger.info("Row:"+i);
> > > > > > //testDao.flush();
> > > > > > //testDao.clear();
> > > > > > }
> > > > > > }
> > > > > > long num = (Long) testDao
> > > > > > .findSingleObject("select COUNT(test)
> > > > > > from Test test"); Assert.assertEquals(
> > > > > > "Too bad, the num is " + num + " and
> > > > > > not " + repeat, repeat, num);
> > > > > > }
> >
> >
> >
>
Extended persistent context and updating non changed entities
Posted by DOMINGUEZ Felipe <Fe...@eurocontrol.int>.
Hello.
I am experimenting with OpenJPA and I find some unexpected behaviour.
If I start a transaction, I read an entity from the data base and I
commit the transaction with out changing the entity, this entity gets
updated back to the database.
It this is a problem to me. I plant to read may entities to just update
one of them, but with this behaviour all of them will be updated, which
causes un necessary calls to the database and a performance issue.
As well I see that EntityManagers obtained through the
EntityManagerFactory have an extended persistence context. How can I
change OpenJPA so that I can have transaction persistence context. I
cannot find any property to set it, I only see property
openjpa.NontransactionalWrite but it does not have any effect at all,
either set to true of false.
In the following code I would prefer to have B41 not to be committed but
it is.
EntityManagerFactory entityManagerFactory = Persistence
.createEntityManagerFactory("asdfsa");
EntityManager em = entityManagerFactory.createEntityManager();
EntityTransaction userTransaction = em.getTransaction();
Bulletin b41 = em.find(Bulletin.class, 41);
b41.setBBB("B41");
userTransaction.begin();
Bulletin b42 = em.find(Bulletin.class, 42);
b42.setBBB("B42");
userTransaction.commit();
em.close();
entityManagerFactory.close();
Thanks
Felipe
____
This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.
Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.
Any views expressed in this message are those of the sender.
Re: insert 500k row
Posted by Janos Haber <bo...@gmail.com>.
Ok. Eat 100% CPU, no prob, but why not work the persistance without
enhancer? Example I not want to use maven or ant, so I can't integrate
to eclipse build, and junit run out of the box. Why stackoverflow? I
think bad idea the enhancer, hibernate can work without javaagent, ok
slower, but work. Ok JPA going slower if I not use enhancer , but NOT
WORK. Oh! and if I add 1000 row(!!! not 200k) I get same stack
overflow error...
ps: Why not work the lazy loading without enhancer? in hibernate work,
why not work here (no prob, if slower than enhancer "version").
Cow
On Feb 8, 2008 10:50 AM, Bruce Beaumont
<br...@ucs-solutions.co.za> wrote:
> Hi
>
> A observation from something I have experienced with OpenJPA.
>
> From your log files see the second sentence. (Someone should change this to
> something like totally consume all CPU resources)
>
> -- Log snip
> Creating subclass for "[class hu.dk.test.data.model.Test]". This means that
> your application will be less efficient and will consume more memory than it
> would if you ran the OpenJPA enhancer. Additionally, lazy loading will not be
> available for one-to-one and many-to-one persistent attributes in types using
> field access; they will be loaded eagerly instead.
>
>
>
> If you do not manually run the enhancer but rely on auto-enhancing I have seen
> absolutely abysmal performance and 4 cores at 100 % utilization. If I run the
> enhancer manually the performance problems go away..
>
>
> Bruce
>
>
>
> On Friday 08 February 2008 10:45:39 Janos Haber wrote:
> > Here is the Test result, Application running flags: -Xmx780m -Xms780m
> >
> > persist {
> > OpenJPA without flush and clear: http://pastebin.com/f3c9492c9
> > OpenJPA flush and clear after every 10k row: http://pastebin.com/f5ad5ef3e
> > OpenJPA flush and clear after every 1k row: http://pastebin.com/f1f774239
> >
> > Hibernate: http://pastebin.com/f7ac1399e
> > }
> >
> > merge {
> > OpenJPA without flush and clear: http://pastebin.com/f2f9625d0
> > OpenJPA flush and clear after every 10k row: http://pastebin.com/f3133d7e3
> > OpenJPA flush and clear after every 1k row: http://pastebin.com/f33d61458
> > OpenJPA flush and clear after every row (I can't wait for the finish,
> > but you see the times) : http://pastebin.com/f41d84668
> >
> > Hibernate (without flush and clear, can't wait for finish):
> > http://pastebin.com/f33ee87b4
> > }
> >
> > So, the OpenJPA is faster if I use flush and clear, but NEVER run well
> > (all persist failed (!!!), maybe self called method (?!?!), but the
> > Test class not self referenced..., merge failed too except (maybe) if
> > I use flush after every(!!!!) row, if I use 10k merge and one flush
> > openJPA insert one row to the db, not 10.000 ).
> >
> > Hibernate slower (usual) yeah, but passed all test well.
> >
> > I know this is my fault, but what's the problem with my code?
> >
> > cow
> >
> > On Feb 7, 2008 11:14 PM, Janos Haber <bo...@gmail.com> wrote:
> > > OpenJPA takes 100% CPU and after the row 60k going to very
> > > slooooooooow. Now I can't tell correct values, tomorrow maybe I create
> > > a test, but if Hibernate create 200k row about 3min, OpenJPA create
> > > 200k row about..... 40 min. But tomorrow I create a test..
> > >
> > > Cow
> > >
> > > On Feb 7, 2008 5:35 PM, Marc Prud'hommeaux <mp...@apache.org> wrote:
> > > > What are the respective execution times? I.e., if OpenJPA takes 100%
> > > > of the CPU but takes 10 seconds, and Hibernate takes 10% of the CPU
> > > > but takes 100 seconds, that might indicate that Hibernate is waiting
> > > > on the database for some operations (like checking for the existence
> > > > of an instance).
> > > >
> > > > On Feb 7, 2008, at 5:08 AM, Janos Haber wrote:
> > > > > Hi!
> > > > >
> > > > > I try to insert 500.000 row to the table (for testing). When my code
> > > > > run the java eat 100% cpu (if I use hibernate it's 1-10%, but I want
> > > > > to use openjpa).
> > > > >
> > > > > Anybody know why?
> > > > > testDao.persist simply call EntityManager.persist.
> > > > >
> > > > > Cow
> > > > >
> > > > > ps:
> > > > >
> > > > > My code:
> > > > > public final static long repeat = 50 * 10000;
> > > > > ...
> > > > >
> > > > > @org.junit.Test
> > > > > public void fieldNumber() {
> > > > > logger.info("Create new datas");
> > > > > for (int i = 0; i < repeat; i++) {
> > > > > Test t = new Test();
> > > > > t.setFieldA1("Field test a1 = " + i);
> > > > > t.setFieldA2("Field test a2 = " + i);
> > > > > t.setFieldA3("Field test a3 = " + i);
> > > > > t.setFieldA4("Field test a4 = " + i);
> > > > > t.setFieldA5("Field test a5 = " + i);
> > > > > t.setFieldA6("Field test a6 = " + i);
> > > > > t.setFieldA7("Field test a7 = " + i);
> > > > > t.setFieldA8("Field test a8 = " + i);
> > > > > t.setTestName(i + " name field");
> > > > > t.setTestBool((i % 2 > 0) ? true : false);
> > > > > t.setTestDate(new Date());
> > > > > t.setTestLong(i);
> > > > > t.setTestString("String field");
> > > > > testDao.persist(t);
> > > > > if (i % 1000 == 0) {
> > > > > logger.info("Row:"+i);
> > > > > //testDao.flush();
> > > > > //testDao.clear();
> > > > > }
> > > > > }
> > > > > long num = (Long) testDao
> > > > > .findSingleObject("select COUNT(test)
> > > > > from Test test"); Assert.assertEquals(
> > > > > "Too bad, the num is " + num + " and
> > > > > not " + repeat, repeat, num);
> > > > > }
>
>
>
Re: insert 500k row
Posted by Bruce Beaumont <br...@ucs-solutions.co.za>.
Hi
A observation from something I have experienced with OpenJPA.
From your log files see the second sentence. (Someone should change this to
something like totally consume all CPU resources)
-- Log snip
Creating subclass for "[class hu.dk.test.data.model.Test]". This means that
your application will be less efficient and will consume more memory than it
would if you ran the OpenJPA enhancer. Additionally, lazy loading will not be
available for one-to-one and many-to-one persistent attributes in types using
field access; they will be loaded eagerly instead.
If you do not manually run the enhancer but rely on auto-enhancing I have seen
absolutely abysmal performance and 4 cores at 100 % utilization. If I run the
enhancer manually the performance problems go away..
Bruce
On Friday 08 February 2008 10:45:39 Janos Haber wrote:
> Here is the Test result, Application running flags: -Xmx780m -Xms780m
>
> persist {
> OpenJPA without flush and clear: http://pastebin.com/f3c9492c9
> OpenJPA flush and clear after every 10k row: http://pastebin.com/f5ad5ef3e
> OpenJPA flush and clear after every 1k row: http://pastebin.com/f1f774239
>
> Hibernate: http://pastebin.com/f7ac1399e
> }
>
> merge {
> OpenJPA without flush and clear: http://pastebin.com/f2f9625d0
> OpenJPA flush and clear after every 10k row: http://pastebin.com/f3133d7e3
> OpenJPA flush and clear after every 1k row: http://pastebin.com/f33d61458
> OpenJPA flush and clear after every row (I can't wait for the finish,
> but you see the times) : http://pastebin.com/f41d84668
>
> Hibernate (without flush and clear, can't wait for finish):
> http://pastebin.com/f33ee87b4
> }
>
> So, the OpenJPA is faster if I use flush and clear, but NEVER run well
> (all persist failed (!!!), maybe self called method (?!?!), but the
> Test class not self referenced..., merge failed too except (maybe) if
> I use flush after every(!!!!) row, if I use 10k merge and one flush
> openJPA insert one row to the db, not 10.000 ).
>
> Hibernate slower (usual) yeah, but passed all test well.
>
> I know this is my fault, but what's the problem with my code?
>
> cow
>
> On Feb 7, 2008 11:14 PM, Janos Haber <bo...@gmail.com> wrote:
> > OpenJPA takes 100% CPU and after the row 60k going to very
> > slooooooooow. Now I can't tell correct values, tomorrow maybe I create
> > a test, but if Hibernate create 200k row about 3min, OpenJPA create
> > 200k row about..... 40 min. But tomorrow I create a test..
> >
> > Cow
> >
> > On Feb 7, 2008 5:35 PM, Marc Prud'hommeaux <mp...@apache.org> wrote:
> > > What are the respective execution times? I.e., if OpenJPA takes 100%
> > > of the CPU but takes 10 seconds, and Hibernate takes 10% of the CPU
> > > but takes 100 seconds, that might indicate that Hibernate is waiting
> > > on the database for some operations (like checking for the existence
> > > of an instance).
> > >
> > > On Feb 7, 2008, at 5:08 AM, Janos Haber wrote:
> > > > Hi!
> > > >
> > > > I try to insert 500.000 row to the table (for testing). When my code
> > > > run the java eat 100% cpu (if I use hibernate it's 1-10%, but I want
> > > > to use openjpa).
> > > >
> > > > Anybody know why?
> > > > testDao.persist simply call EntityManager.persist.
> > > >
> > > > Cow
> > > >
> > > > ps:
> > > >
> > > > My code:
> > > > public final static long repeat = 50 * 10000;
> > > > ...
> > > >
> > > > @org.junit.Test
> > > > public void fieldNumber() {
> > > > logger.info("Create new datas");
> > > > for (int i = 0; i < repeat; i++) {
> > > > Test t = new Test();
> > > > t.setFieldA1("Field test a1 = " + i);
> > > > t.setFieldA2("Field test a2 = " + i);
> > > > t.setFieldA3("Field test a3 = " + i);
> > > > t.setFieldA4("Field test a4 = " + i);
> > > > t.setFieldA5("Field test a5 = " + i);
> > > > t.setFieldA6("Field test a6 = " + i);
> > > > t.setFieldA7("Field test a7 = " + i);
> > > > t.setFieldA8("Field test a8 = " + i);
> > > > t.setTestName(i + " name field");
> > > > t.setTestBool((i % 2 > 0) ? true : false);
> > > > t.setTestDate(new Date());
> > > > t.setTestLong(i);
> > > > t.setTestString("String field");
> > > > testDao.persist(t);
> > > > if (i % 1000 == 0) {
> > > > logger.info("Row:"+i);
> > > > //testDao.flush();
> > > > //testDao.clear();
> > > > }
> > > > }
> > > > long num = (Long) testDao
> > > > .findSingleObject("select COUNT(test)
> > > > from Test test"); Assert.assertEquals(
> > > > "Too bad, the num is " + num + " and
> > > > not " + repeat, repeat, num);
> > > > }
Re: insert 500k row
Posted by Janos Haber <bo...@gmail.com>.
Here is the Test result, Application running flags: -Xmx780m -Xms780m
persist {
OpenJPA without flush and clear: http://pastebin.com/f3c9492c9
OpenJPA flush and clear after every 10k row: http://pastebin.com/f5ad5ef3e
OpenJPA flush and clear after every 1k row: http://pastebin.com/f1f774239
Hibernate: http://pastebin.com/f7ac1399e
}
merge {
OpenJPA without flush and clear: http://pastebin.com/f2f9625d0
OpenJPA flush and clear after every 10k row: http://pastebin.com/f3133d7e3
OpenJPA flush and clear after every 1k row: http://pastebin.com/f33d61458
OpenJPA flush and clear after every row (I can't wait for the finish,
but you see the times) : http://pastebin.com/f41d84668
Hibernate (without flush and clear, can't wait for finish):
http://pastebin.com/f33ee87b4
}
So, the OpenJPA is faster if I use flush and clear, but NEVER run well
(all persist failed (!!!), maybe self called method (?!?!), but the
Test class not self referenced..., merge failed too except (maybe) if
I use flush after every(!!!!) row, if I use 10k merge and one flush
openJPA insert one row to the db, not 10.000 ).
Hibernate slower (usual) yeah, but passed all test well.
I know this is my fault, but what's the problem with my code?
cow
On Feb 7, 2008 11:14 PM, Janos Haber <bo...@gmail.com> wrote:
> OpenJPA takes 100% CPU and after the row 60k going to very
> slooooooooow. Now I can't tell correct values, tomorrow maybe I create
> a test, but if Hibernate create 200k row about 3min, OpenJPA create
> 200k row about..... 40 min. But tomorrow I create a test..
>
> Cow
>
>
>
> On Feb 7, 2008 5:35 PM, Marc Prud'hommeaux <mp...@apache.org> wrote:
> >
> > What are the respective execution times? I.e., if OpenJPA takes 100%
> > of the CPU but takes 10 seconds, and Hibernate takes 10% of the CPU
> > but takes 100 seconds, that might indicate that Hibernate is waiting
> > on the database for some operations (like checking for the existence
> > of an instance).
> >
> >
> >
> > On Feb 7, 2008, at 5:08 AM, Janos Haber wrote:
> >
> > > Hi!
> > >
> > > I try to insert 500.000 row to the table (for testing). When my code
> > > run the java eat 100% cpu (if I use hibernate it's 1-10%, but I want
> > > to use openjpa).
> > >
> > > Anybody know why?
> > > testDao.persist simply call EntityManager.persist.
> > >
> > > Cow
> > >
> > > ps:
> > >
> > > My code:
> > > public final static long repeat = 50 * 10000;
> > > ...
> > >
> > > @org.junit.Test
> > > public void fieldNumber() {
> > > logger.info("Create new datas");
> > > for (int i = 0; i < repeat; i++) {
> > > Test t = new Test();
> > > t.setFieldA1("Field test a1 = " + i);
> > > t.setFieldA2("Field test a2 = " + i);
> > > t.setFieldA3("Field test a3 = " + i);
> > > t.setFieldA4("Field test a4 = " + i);
> > > t.setFieldA5("Field test a5 = " + i);
> > > t.setFieldA6("Field test a6 = " + i);
> > > t.setFieldA7("Field test a7 = " + i);
> > > t.setFieldA8("Field test a8 = " + i);
> > > t.setTestName(i + " name field");
> > > t.setTestBool((i % 2 > 0) ? true : false);
> > > t.setTestDate(new Date());
> > > t.setTestLong(i);
> > > t.setTestString("String field");
> > > testDao.persist(t);
> > > if (i % 1000 == 0) {
> > > logger.info("Row:"+i);
> > > //testDao.flush();
> > > //testDao.clear();
> > > }
> > > }
> > > long num = (Long) testDao
> > > .findSingleObject("select COUNT(test) from Test test");
> > > Assert.assertEquals(
> > > "Too bad, the num is " + num + " and not " + repeat, repeat,
> > > num);
> > > }
> >
> >
>
Re: insert 500k row
Posted by Janos Haber <bo...@gmail.com>.
OpenJPA takes 100% CPU and after the row 60k going to very
slooooooooow. Now I can't tell correct values, tomorrow maybe I create
a test, but if Hibernate create 200k row about 3min, OpenJPA create
200k row about..... 40 min. But tomorrow I create a test..
Cow
On Feb 7, 2008 5:35 PM, Marc Prud'hommeaux <mp...@apache.org> wrote:
>
> What are the respective execution times? I.e., if OpenJPA takes 100%
> of the CPU but takes 10 seconds, and Hibernate takes 10% of the CPU
> but takes 100 seconds, that might indicate that Hibernate is waiting
> on the database for some operations (like checking for the existence
> of an instance).
>
>
>
> On Feb 7, 2008, at 5:08 AM, Janos Haber wrote:
>
> > Hi!
> >
> > I try to insert 500.000 row to the table (for testing). When my code
> > run the java eat 100% cpu (if I use hibernate it's 1-10%, but I want
> > to use openjpa).
> >
> > Anybody know why?
> > testDao.persist simply call EntityManager.persist.
> >
> > Cow
> >
> > ps:
> >
> > My code:
> > public final static long repeat = 50 * 10000;
> > ...
> >
> > @org.junit.Test
> > public void fieldNumber() {
> > logger.info("Create new datas");
> > for (int i = 0; i < repeat; i++) {
> > Test t = new Test();
> > t.setFieldA1("Field test a1 = " + i);
> > t.setFieldA2("Field test a2 = " + i);
> > t.setFieldA3("Field test a3 = " + i);
> > t.setFieldA4("Field test a4 = " + i);
> > t.setFieldA5("Field test a5 = " + i);
> > t.setFieldA6("Field test a6 = " + i);
> > t.setFieldA7("Field test a7 = " + i);
> > t.setFieldA8("Field test a8 = " + i);
> > t.setTestName(i + " name field");
> > t.setTestBool((i % 2 > 0) ? true : false);
> > t.setTestDate(new Date());
> > t.setTestLong(i);
> > t.setTestString("String field");
> > testDao.persist(t);
> > if (i % 1000 == 0) {
> > logger.info("Row:"+i);
> > //testDao.flush();
> > //testDao.clear();
> > }
> > }
> > long num = (Long) testDao
> > .findSingleObject("select COUNT(test) from Test test");
> > Assert.assertEquals(
> > "Too bad, the num is " + num + " and not " + repeat, repeat,
> > num);
> > }
>
>
Re: insert 500k row
Posted by Marc Prud'hommeaux <mp...@apache.org>.
What are the respective execution times? I.e., if OpenJPA takes 100%
of the CPU but takes 10 seconds, and Hibernate takes 10% of the CPU
but takes 100 seconds, that might indicate that Hibernate is waiting
on the database for some operations (like checking for the existence
of an instance).
On Feb 7, 2008, at 5:08 AM, Janos Haber wrote:
> Hi!
>
> I try to insert 500.000 row to the table (for testing). When my code
> run the java eat 100% cpu (if I use hibernate it's 1-10%, but I want
> to use openjpa).
>
> Anybody know why?
> testDao.persist simply call EntityManager.persist.
>
> Cow
>
> ps:
>
> My code:
> public final static long repeat = 50 * 10000;
> ...
>
> @org.junit.Test
> public void fieldNumber() {
> logger.info("Create new datas");
> for (int i = 0; i < repeat; i++) {
> Test t = new Test();
> t.setFieldA1("Field test a1 = " + i);
> t.setFieldA2("Field test a2 = " + i);
> t.setFieldA3("Field test a3 = " + i);
> t.setFieldA4("Field test a4 = " + i);
> t.setFieldA5("Field test a5 = " + i);
> t.setFieldA6("Field test a6 = " + i);
> t.setFieldA7("Field test a7 = " + i);
> t.setFieldA8("Field test a8 = " + i);
> t.setTestName(i + " name field");
> t.setTestBool((i % 2 > 0) ? true : false);
> t.setTestDate(new Date());
> t.setTestLong(i);
> t.setTestString("String field");
> testDao.persist(t);
> if (i % 1000 == 0) {
> logger.info("Row:"+i);
> //testDao.flush();
> //testDao.clear();
> }
> }
> long num = (Long) testDao
> .findSingleObject("select COUNT(test) from Test test");
> Assert.assertEquals(
> "Too bad, the num is " + num + " and not " + repeat, repeat,
> num);
> }