You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@openjpa.apache.org by ssso <si...@yahoo.com> on 2010/05/04 01:42:40 UTC
Slice: OutOfMemoryError shortly after pounding 1000+ threads to the
system.
I have a very simple spring web app, inserting an order into the database.
After pounding 500+ threads or so, I got an OutOfMemoryError:
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:597)
at
java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadPoolExecutor.java:703)
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:652)
at
java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:92)
at
org.apache.openjpa.slice.jdbc.DistributedJDBCStoreManager.flush(DistributedJDBCStoreManager.java:290)
at
org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreManager.java:131)
at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2139)
at
org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2037)
at
org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:1955)
at
org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime.java:81)
at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1479)
at
org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:925)
at
org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:559)
at
org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:467)
at
org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:754)
at
org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)
at
org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:375)
at
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:120)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at $Proxy17.merge(Unknown Source)
at
com.shutterfly.sfweb.InsertController.handleRequest(InsertController.java:45)
at
org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:771)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:716)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
at
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:549)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
My question about the DistributedJDBCStoreManager.flush(). Does it really
have to create a thread pool every single time a row is committed? Also,
from the profiler it seems like the threads in the pool are not necessarily
used.
Thanks,
Simon
--
View this message in context: http://openjpa.208410.n2.nabble.com/Slice-OutOfMemoryError-shortly-after-pounding-1000-threads-to-the-system-tp5000822p5000822.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Slice: OutOfMemoryError shortly after pounding 1000+ threads to
the system.
Posted by Rick Curtis <cu...@gmail.com>.
Was there a heapdump generated? If so, did you look at it to see what is
taking up all of the memory?
On Mon, May 3, 2010 at 6:42 PM, ssso <si...@yahoo.com> wrote:
>
> I have a very simple spring web app, inserting an order into the database.
>
> After pounding 500+ threads or so, I got an OutOfMemoryError:
>
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:597)
> at
>
> java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadPoolExecutor.java:703)
> at
>
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:652)
> at
>
> java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:92)
> at
>
> org.apache.openjpa.slice.jdbc.DistributedJDBCStoreManager.flush(DistributedJDBCStoreManager.java:290)
> at
>
> org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreManager.java:131)
> at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2139)
> at
> org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2037)
> at
> org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:1955)
> at
>
> org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime.java:81)
> at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1479)
> at
>
> org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:925)
> at
>
> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:559)
> at
>
> org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:467)
> at
>
> org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:754)
> at
>
> org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)
> at
>
> org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:375)
> at
>
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:120)
> at
>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at
>
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
> at $Proxy17.merge(Unknown Source)
> at
>
> com.shutterfly.sfweb.InsertController.handleRequest(InsertController.java:45)
> at
>
> org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
> at
>
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:771)
> at
>
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:716)
> at
>
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
> at
>
> org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:549)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> at
>
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
>
>
> My question about the DistributedJDBCStoreManager.flush(). Does it really
> have to create a thread pool every single time a row is committed? Also,
> from the profiler it seems like the threads in the pool are not necessarily
> used.
>
> Thanks,
> Simon
>
> --
> View this message in context:
> http://openjpa.208410.n2.nabble.com/Slice-OutOfMemoryError-shortly-after-pounding-1000-threads-to-the-system-tp5000822p5000822.html
> Sent from the OpenJPA Users mailing list archive at Nabble.com.
>
--
Thanks,
Rick
Re: Spring 3.0.2 + OpenJPA 2.0 Slice: OutOfMemoryError shortly
after pounding 1000+ threads to the system.
Posted by ssso <si...@yahoo.com>.
Just had another round of testing, the same program, but *TURNED OFF* openjpa
slice.
2 boxes, 10 concurrent users, hit the program 500 times.
No problem. Everything went through. Memory very healthy. Perm Space
about 34MB. No leak.
--
View this message in context: http://openjpa.208410.n2.nabble.com/Spring-3-0-2-OpenJPA-2-0-Slice-OutOfMemoryError-shortly-after-pounding-1000-threads-to-the-system-tp5000822p5000939.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
RE: Slice: OutOfMemoryError shortly after pounding 1000+ threads to the system.
Posted by C N Davies <cn...@cndavies.com>.
I had this issue a lot on OpenJPA 1.2.2, I found that doing a clear after
committing a specified number of record kept things going.
Chris
-----Original Message-----
From: ssso [mailto:simonso@yahoo.com]
Sent: Tuesday, 4 May 2010 9:43 AM
To: users@openjpa.apache.org
Subject: Slice: OutOfMemoryError shortly after pounding 1000+ threads to the
system.
I have a very simple spring web app, inserting an order into the database.
After pounding 500+ threads or so, I got an OutOfMemoryError:
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:597)
at
java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadPoolExe
cutor.java:703)
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:652)
at
java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.
java:92)
at
org.apache.openjpa.slice.jdbc.DistributedJDBCStoreManager.flush(DistributedJ
DBCStoreManager.java:290)
at
org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreManage
r.java:131)
at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2139)
at
org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2037)
at
org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:1955)
at
org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime.jav
a:81)
at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1479)
at
org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:925)
at
org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.ja
va:559)
at
org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionMan
ager.java:467)
at
org.springframework.transaction.support.AbstractPlatformTransactionManager.p
rocessCommit(AbstractPlatformTransactionManager.java:754)
at
org.springframework.transaction.support.AbstractPlatformTransactionManager.c
ommit(AbstractPlatformTransactionManager.java:723)
at
org.springframework.transaction.interceptor.TransactionAspectSupport.commitT
ransactionAfterReturning(TransactionAspectSupport.java:375)
at
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(Tr
ansactionInterceptor.java:120)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(Reflect
iveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopPro
xy.java:202)
at $Proxy17.merge(Unknown Source)
at
com.shutterfly.sfweb.InsertController.handleRequest(InsertController.java:45
)
at
org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(Si
mpleControllerHandlerAdapter.java:48)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServl
et.java:771)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServle
t.java:716)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkSer
vlet.java:644)
at
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java
:549)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:290)
My question about the DistributedJDBCStoreManager.flush(). Does it really
have to create a thread pool every single time a row is committed? Also,
from the profiler it seems like the threads in the pool are not necessarily
used.
Thanks,
Simon
--
View this message in context:
http://openjpa.208410.n2.nabble.com/Slice-OutOfMemoryError-shortly-after-pou
nding-1000-threads-to-the-system-tp5000822p5000822.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: AW: AW: Slice: OutOfMemoryError shortly after pounding 1000+
threads to the system.
Posted by Pinaki Poddar <pp...@apache.org>.
Hi Mark,
> My problem was that I did not find a way to properly reproduce the String
> NULL -> "" bug with derby, so I thought it's easier to show the bug in a
> small project on it's own.
That's OK. A recent change [1] was committed on trunk. If you please, you
can
a) verify if I have faithfully reproduced the test scenario
b) try that revision for the original unique constraint on null/empty String
on a database of your choice.
[1]
http://openjpa.208410.n2.nabble.com/svn-commit-r941362-in-openjpa-trunk-openjpa-jdbc-src-main-java-org-apache-openjpa-jdbc-schema-openjp-td5009752.html#a5009752
-----
Pinaki
--
View this message in context: http://openjpa.208410.n2.nabble.com/Spring-3-0-2-OpenJPA-2-0-Slice-OutOfMemoryError-shortly-after-pounding-1000-threads-to-the-system-tp5000822p5014474.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: AW: AW: Slice: OutOfMemoryError shortly after pounding 1000+ threads to the system.
Posted by Mark Struberg <st...@yahoo.de>.
Hi Pinaki!
Yes, there is a ulimit -n command, but most userers lack of the rights to change it at runtime ;)
So it's better to configure it properly in limits.conf.
And thanks for the test-howto!
My problem was that I did not find a way to properly reproduce the String NULL -> "" bug with derby, so I thought it's easier to show the bug in a small project on it's own.
LieGrue,
strub
--- Pinaki Poddar <pp...@apache.org> schrieb am Do, 6.5.2010:
> Von: Pinaki Poddar <pp...@apache.org>
> Betreff: Re: AW: AW: Slice: OutOfMemoryError shortly after pounding 1000+ threads to the system.
> An: users@openjpa.apache.org
> Datum: Donnerstag, 6. Mai, 2010 01:04 Uhr
>
> Hi Mark,
>
> 1. Thanks for contributing the test case. I had
> cannibalized the test
> scenario in an existing test case (TestBasicFieldNullity).
> Further instructions on developing tests for OpenJPA
> can be found here
> [1].
>
> 2. We will add the file limit instruction for the
> unixoid users. There used
> to be a
> $ ulimit -n <file limit>
> command in unix -- is not it available in Linux?
>
>
> [1] http://openjpa.apache.org/writing-test-cases-for-openjpa.html
>
> -----
> Pinaki
> --
> View this message in context: http://openjpa.208410.n2.nabble.com/Spring-3-0-2-OpenJPA-2-0-Slice-OutOfMemoryError-shortly-after-pounding-1000-threads-to-the-system-tp5000822p5011556.html
> Sent from the OpenJPA Users mailing list archive at
> Nabble.com.
>
Re: AW: AW: Slice: OutOfMemoryError shortly after pounding 1000+
threads to the system.
Posted by Pinaki Poddar <pp...@apache.org>.
Hi Mark,
1. Thanks for contributing the test case. I had cannibalized the test
scenario in an existing test case (TestBasicFieldNullity).
Further instructions on developing tests for OpenJPA can be found here
[1].
2. We will add the file limit instruction for the unixoid users. There used
to be a
$ ulimit -n <file limit>
command in unix -- is not it available in Linux?
[1] http://openjpa.apache.org/writing-test-cases-for-openjpa.html
-----
Pinaki
--
View this message in context: http://openjpa.208410.n2.nabble.com/Spring-3-0-2-OpenJPA-2-0-Slice-OutOfMemoryError-shortly-after-pounding-1000-threads-to-the-system-tp5000822p5011556.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
AW: AW: Slice: OutOfMemoryError shortly after pounding 1000+ threads to the system.
Posted by Mark Struberg <st...@yahoo.de>.
Btw, what I miss in the wiki section is the pretty common ulimit error on *nix.
Most unixoid systems (like Linux) have a standard quota of 1024 file handles per process. This is not enough for building OpenJPA on a Linux box and you will most likely get funny IO exceptions.
You might have to edit
/etc/security/limits.conf
and add the following (or adequat) lines to increase the ulimit:
[insertyourusername] hard nofile 4096
[insertyourusername] soft nofile 4096
LieGrue,
strub
----- Ursprüngliche Mail ----
> Von: Pinaki Poddar <pp...@apache.org>
> An: users@openjpa.apache.org
> Gesendet: Mittwoch, den 5. Mai 2010, 1:07:38 Uhr
> Betreff: Re: AW: Slice: OutOfMemoryError shortly after pounding 1000+ threads to the system.
>
>
Hi Simon,
New Revision: 941084 [1] changes the thread pool usage
> in Slice.
Also has other assorted changes. The changes are generally
> available on
nightly snapshot builds [2]
But you can also build
> OpenJPA locally [3].
[1]
> href="http://openjpa.208410.n2.nabble.com/svn-commit-r941084-in-openjpa-trunk-openjpa-slice-src-main-java-org-apache-openjpa-slice-main-java-o-td5006281.html#a5006281"
> target=_blank
> >http://openjpa.208410.n2.nabble.com/svn-commit-r941084-in-openjpa-trunk-openjpa-slice-src-main-java-org-apache-openjpa-slice-main-java-o-td5006281.html#a5006281
[2]
>
> >http://openjpa.apache.org/downloads.html
[3]
> href="http://openjpa.apache.org/building.html#Building-CommandLineBuilds"
> target=_blank
> >http://openjpa.apache.org/building.html#Building-CommandLineBuilds
-----
Pinaki
>
--
View this message in context:
> href="http://openjpa.208410.n2.nabble.com/Spring-3-0-2-OpenJPA-2-0-Slice-OutOfMemoryError-shortly-after-pounding-1000-threads-to-the-system-tp5000822p5006294.html"
> target=_blank
> >http://openjpa.208410.n2.nabble.com/Spring-3-0-2-OpenJPA-2-0-Slice-OutOfMemoryError-shortly-after-pounding-1000-threads-to-the-system-tp5000822p5006294.html
Sent
> from the OpenJPA Users mailing list archive at Nabble.com.
Re: AW: Slice: OutOfMemoryError shortly after pounding 1000+
threads to the system.
Posted by Pinaki Poddar <pp...@apache.org>.
Hi Simon,
New Revision: 941084 [1] changes the thread pool usage in Slice.
Also has other assorted changes. The changes are generally available on
nightly snapshot builds [2]
But you can also build OpenJPA locally [3].
[1]
http://openjpa.208410.n2.nabble.com/svn-commit-r941084-in-openjpa-trunk-openjpa-slice-src-main-java-org-apache-openjpa-slice-main-java-o-td5006281.html#a5006281
[2] http://openjpa.apache.org/downloads.html
[3] http://openjpa.apache.org/building.html#Building-CommandLineBuilds
-----
Pinaki
--
View this message in context: http://openjpa.208410.n2.nabble.com/Spring-3-0-2-OpenJPA-2-0-Slice-OutOfMemoryError-shortly-after-pounding-1000-threads-to-the-system-tp5000822p5006294.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: AW: Slice: OutOfMemoryError shortly after pounding 1000+
threads to the system.
Posted by Pinaki Poddar <pp...@apache.org>.
> 1 million order inserts (Order - Customer tables, many to one, Order
sharded to 3 slices, customer
> replicated. Data cache 1000 customers). No sweat. No leak.
Good.
Really appreciate your interest in Slice and your contribution to improve
its quality/robustness.
Here are few tips that will exercise the 'weak' points
a) update instances that are @Replicated e.g. Customer, occasionally.
b) use queries that uses other persistent entity as parameter. For example,
EntityManager em = ...;
Customer customer = em.find(Customer.class, 12345);
String jpql = "select o from PurchaseOrder o where o.customer=:c order
by o.placedDate";
List<PurchaseOrder> orders = em.createQuery(jpql, PurchaseOrder.class)
.setParameter("c",
customer)
.getResultList();
And, of course, it will be great to see the same test application running on
a single database.
-----
Pinaki
--
View this message in context: http://openjpa.208410.n2.nabble.com/Spring-3-0-2-OpenJPA-2-0-Slice-OutOfMemoryError-shortly-after-pounding-1000-threads-to-the-system-tp5000822p5015708.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: AW: Slice: OutOfMemoryError shortly after pounding 1000+
threads to the system.
Posted by ssso <si...@yahoo.com>.
Hi Pinaki,
Sorry didn't get back to you earlier.
1 million order inserts (Order - Customer tables, many to one, Order sharded
to 3 slices, customer replicated. Data cache 1000 customers). No sweat.
No leak.
I used the 2.1.0-SNAPSHOT.
Your fix is awesome!
When I have a chance, I'll shoot you an e-mail to discuss the results, and
how I can refine my test to get even more mileage out of the test.
Cheers,
Simon
--
View this message in context: http://openjpa.208410.n2.nabble.com/Spring-3-0-2-OpenJPA-2-0-Slice-OutOfMemoryError-shortly-after-pounding-1000-threads-to-the-system-tp5000822p5015506.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: AW: Slice: OutOfMemoryError shortly after pounding 1000+
threads to the system.
Posted by Pinaki Poddar <pp...@apache.org>.
Hi Simon,
Did you get a chance to re-run the multi-threaded stress test with the
latest changes [1]?
[1]
http://openjpa.208410.n2.nabble.com/svn-commit-r941084-in-openjpa-trunk-openjpa-slice-src-main-java-org-apache-openjpa-slice-main-java-o-td5006281.html#a5006281
-----
Pinaki
--
View this message in context: http://openjpa.208410.n2.nabble.com/Spring-3-0-2-OpenJPA-2-0-Slice-OutOfMemoryError-shortly-after-pounding-1000-threads-to-the-system-tp5000822p5014524.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: AW: Slice: OutOfMemoryError shortly after pounding 1000+
threads to the system.
Posted by ssso <si...@yahoo.com>.
Hi Pinaki
I try and finally{} the block with threadPool.shutdown().
That seems to work. 100k transaction no problem now.
You nail it right on with the excessive pool creation in every flush().
I am not sure CachedThreadPool would have solved the problem. You still
have more flush() call coming along the way -- then we need to define
RejectExecutionHandler. We can't abort and we can't discard.
Since the pool is gonna go out of scope by the time flush() is done, we
probably need to shut it down before going out of scope (so that expired
thread no longer hang around).
I will keep on stressing the stack and see if there are more problems.
Cheers,
Simon
--
View this message in context: http://openjpa.208410.n2.nabble.com/Spring-3-0-2-OpenJPA-2-0-Slice-OutOfMemoryError-shortly-after-pounding-1000-threads-to-the-system-tp5000822p5005891.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: AW: Slice: OutOfMemoryError shortly after pounding 1000+
threads to the system.
Posted by Pinaki Poddar <pp...@apache.org>.
Hi Simon,
This error is due to bad choice of thread pool.
java.util.concurrent package does have powerful thread pools and we should
use them effectively.
I have opened a new JIRA issue [1] to change the thread pool usage.
And yes, we do not need to create a pool for every flush.
I will fix both the outage and repeated creation of thread pool.
[1] https://issues.apache.org/jira/browse/OPENJPA-1648
-----
Pinaki
--
View this message in context: http://openjpa.208410.n2.nabble.com/Spring-3-0-2-OpenJPA-2-0-Slice-OutOfMemoryError-shortly-after-pounding-1000-threads-to-the-system-tp5000822p5005774.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
AW: Slice: OutOfMemoryError shortly after pounding 1000+ threads to the system.
Posted by Mark Struberg <st...@yahoo.de>.
which VM are you running? 32 bit or 64 bit? Because each thread you start eats up memory - even if it doesn't use it.
LieGrue,
strub
----- Ursprüngliche Mail ----
> Von: ssso <si...@yahoo.com>
> An: users@openjpa.apache.org
> Gesendet: Dienstag, den 4. Mai 2010, 1:42:40 Uhr
> Betreff: Slice: OutOfMemoryError shortly after pounding 1000+ threads to the system.
>
>
I have a very simple spring web app, inserting an order into the
> database.
After pounding 500+ threads or so, I got an
> OutOfMemoryError:
java.lang.OutOfMemoryError: unable to create new native
> thread
at java.lang.Thread.start0(Native
> Method)
at
> java.lang.Thread.start(Thread.java:597)
> at
java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadPoolExecutor.java:703)
>
> at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:652)
>
> at
java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:92)
>
> at
org.apache.openjpa.slice.jdbc.DistributedJDBCStoreManager.flush(DistributedJDBCStoreManager.java:290)
>
> at
org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreManager.java:131)
> at
> org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2139)
>
> at
org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2037)
>
> at
org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:1955)
>
> at
org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime.java:81)
> at
> org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1479)
>
> at
org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:925)
>
> at
org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:559)
>
> at
org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:467)
>
> at
org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:754)
>
> at
org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)
>
> at
org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:375)
>
> at
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:120)
>
> at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>
> at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
> at $Proxy17.merge(Unknown Source)
>
> at
com.shutterfly.sfweb.InsertController.handleRequest(InsertController.java:45)
>
> at
org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
>
> at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:771)
>
> at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:716)
>
> at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
>
> at
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:549)
> at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>
> at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
My
> question about the DistributedJDBCStoreManager.flush(). Does it
> really
have to create a thread pool every single time a row is
> committed? Also,
from the profiler it seems like the threads in the
> pool are not necessarily
used.
Thanks,
Simon
--
View
> this message in context:
> href="http://openjpa.208410.n2.nabble.com/Slice-OutOfMemoryError-shortly-after-pounding-1000-threads-to-the-system-tp5000822p5000822.html"
> target=_blank
> >http://openjpa.208410.n2.nabble.com/Slice-OutOfMemoryError-shortly-after-pounding-1000-threads-to-the-system-tp5000822p5000822.html
Sent
> from the OpenJPA Users mailing list archive at Nabble.com.