You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by James Carman <ja...@carmanconsulting.com> on 2013/10/10 12:28:32 UTC
Build failures...
There are quite a few broken tests. Jetty seems to be hosed. JPA has a
failed test. CDI has issues. It looks like JMS and FTP even decided to
join the party. This has been going on for a while now. This particular
job hasn't been "green" since 9/29. Should we stop working other issues
and get ourselves whole again?
https://builds.apache.org/job/Camel.trunk.fulltest/1561/
Re: Build failures...
Posted by Claus Ibsen <cl...@gmail.com>.
Nice digging Babak.
On Sat, Oct 19, 2013 at 2:30 AM, Babak Vahdat
<ba...@swissonline.ch> wrote:
> Christian,
>
> I commited fix for this both on the master as well as the 2.12.x branch.
> Would you please give it another try to see if you can still reproduce the
> test failures you mentioned by your previous posts?
>
> Babak
>
>
> Christian Mueller wrote
>> Really good catch!
>>
>> Best,
>>
>> Christian
>> -----------------
>>
>> Software Integration Specialist
>>
>> Apache Member
>> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
>> Apache Incubator PMC Member
>>
>> https://www.linkedin.com/pub/christian-mueller/11/551/642
>>
>>
>> On Fri, Oct 18, 2013 at 11:47 PM, Babak Vahdat
>> <
>
>> babak.vahdat@
>
>> >wrote:
>>
>>> The root cause of the problem for the failing JPA test is CAMEL-5932
>>> through
>>> which we started sharing the *same* EntityManager object across
>>> JpaEndpoint,
>>> all it's created producers and consumers which is wrong! JPA 2.0 Spec,
>>> section 7.2:
>>>
>>> An entity manager must not be shared among multiple concurrently
>>> executing
>>> threads, as the entity manager and persistence context are not required
>>> to
>>> be threadsafe. Entity managers must only be accessed in a single-threaded
>>> manner.
>>>
>>> That said using OpenJPA you could make use of a proprietary property to
>>> make
>>> OpenJPA EntityManager-impl thread safe:
>>>
>>>
>> <property name="openjpa.Multithreaded" value="true"/>
>>>
>>> But even then, what about Camel users whose JPA provider is Eclipselink
>>> or
>>> Hibernate-EntityManager.
>>>
>>> I logged a ticket to get this fixed and hopfully the JPA test being
>>> mentioned here as well:
>>>
>>> https://issues.apache.org/jira/browse/CAMEL-6874
>>>
>>> Also there're also some aimless lines of code inside
>>> JpaProducer#process()
>>> method which I will clean up as well.
>>>
>>> Babak
>>>
>>>
>>> Babak Vahdat wrote
>>> > Yeah you're right the test failed again on the CI-Server as well.
>>> There're
>>> > also couple of unresolved concurrency/deadlock related JIRA tickets by
>>> > OpenJPA going into the exactly same direction:
>>> >
>>> > https://issues.apache.org/jira/browse/OPENJPA-1716
>>> >
>>> > Babak
>>> > Christian Mueller wrote
>>> >> It's still failing sometimes on my machine: :-(
>>> >>
>>> >> Results :
>>> >> Failed tests:
>>> >>
>>> >>
>>> JpaProducerConcurrentTest.testConcurrentProducers:45->doSendMessages:66->CamelTestSupport.assertMockEndpointsSatisfied:682
>>> >> mock://result Received message count. Expected: <10> but was: <8>
>>> >>
>>> >> Tests run: 49, Failures: 1, Errors: 0, Skipped: 0
>>> >>
>>> >> [INFO]
>>> >>
>>> ------------------------------------------------------------------------
>>> >> [INFO] BUILD FAILURE
>>> >> [INFO]
>>> >>
>>> ------------------------------------------------------------------------
>>> >>
>>> >>
>>> >> Best,
>>> >> Christian
>>> >> -----------------
>>> >>
>>> >> Software Integration Specialist
>>> >>
>>> >> Apache Member
>>> >> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
>>> >> Apache Incubator PMC Member
>>> >>
>>> >> https://www.linkedin.com/pub/christian-mueller/11/551/642
>>> >>
>>> >>
>>> >> On Sat, Oct 12, 2013 at 11:07 PM, Babak Vahdat
>>> >> <
>>>
>>> >> babak.vahdat@
>>>
>>> >> >wrote:
>>> >>
>>> >>> Yeah indeed a deadlock inside the OpenJPA code is the root cause of
>>> the
>>> >>> problem (org.apache.openjpa.kernel.BrokerImpl#lock()). I just
>>> commited
>>> a
>>> >>> fix
>>> >>> regarding this, so let's keep an eye on this test by the next builds.
>>> >>> Following a screenshot of the proven deadlock through jconsole:
>>> >>>
>>> >>> <
>>> >>>
>>> http://camel.465427.n5.nabble.com/file/n5741448/Screen_Shot_2013-10-12_at_11.02.58_PM.png
>>> >>> >
>>> >>>
>>> >>> You can enforce the deadlock if you would make use of a high log
>>> level
>>> >>> (log4j.properties), e.g. TRACE instead of just INFO.
>>> >>>
>>> >>> Babak
>>> >>>
>>> >>>
>>> >>> Claus Ibsen-2 wrote
>>> >>> > It could be due an OpenJPA upgrade. These tests used to work better
>>> in
>>> >>> the
>>> >>> > past.
>>> >>> >
>>> >>> > On Sat, Oct 12, 2013 at 10:05 AM, Babak Vahdat
>>> >>> > <
>>> >>>
>>> >>> > babak.vahdat@
>>> >>>
>>> >>> > > wrote:
>>> >>> >> The test failed again on the CI-Server. I will look into this in
>>> the
>>> >>> next
>>> >>> >> days to see if I can find the root cause as on my box I can't
>>> >>> reproduce
>>> >>> >> this.
>>> >>> >>
>>> >>> >> Babak
>>> >>> >>
>>> >>> >>
>>> >>> >> Babak Vahdat wrote
>>> >>> >>> Hi
>>> >>> >>>
>>> >>> >>> @Christian I believe I've fixed the JPA test now, could you
>>> please
>>> >>> check
>>> >>> >>> to see if you can still reproduce the same exception you
>>> mentioned
>>> >>> >>> below?
>>> >>> >>>
>>> >>> >>> Babak
>>> >>> >>> Christian Mueller wrote
>>> >>> >>>> Today, it looks much better [1]. We are trying to fix the unit
>>> >>> tests
>>> >>> >>>> which
>>> >>> >>>> are failing from time to time on slower boxes. But this isn't so
>>> >>> easy
>>> >>> >>>> for
>>> >>> >>>> all of them...
>>> >>> >>>>
>>> >>> >>>> I had a look at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.processor.jpa.JpaProducerConcurrentTest.testConcurrentProducers.
>>> >>> >>>> A few times I get the following exception which results in
>>> missing
>>> >>> >>>> exchanges in the mock:result endpoint:
>>> >>> >>>>
>>> >>> >>>> 2013-10-11 14:01:10,623 [pool-2-thread-2] ERROR
>>> >>> >>>> DefaultErrorHandler - Failed delivery for (MessageId:
>>> >>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4 on
>>> >>> >>>> ExchangeId:
>>> >>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9).
>>> >>> Exhausted
>>> >>> >>>> after delivery attempt: 1 caught:
>>> >>> >>>> org.springframework.dao.InvalidDataAccessApiUsageException: This
>>> >>> >>>> operation
>>> >>> >>>> cannot be performed while a Transaction is active.; nested
>>> >>> exception
>>> >>> is
>>> >>> >>>>
>>> >>> >
>>> >>
>> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>>> >>> >>>> org.apache.openjpa.persistence.InvalidStateException: This
>>> >>> operation
>>> >>> >>>> cannot
>>> >>> >>>> be performed while a Transaction is active.
>>> >>> >>>>
>>> >>> >>>> Message History
>>> >>> >>>>
>>> >>>
>>> ---------------------------------------------------------------------------------------------------------------------------------------
>>> >>> >>>> RouteId ProcessorId
>>> >>> >>>> Processor
>>> >>> >>>> Elapsed (ms)
>>> >>> >>>> [route1 ] [route1 ]
>>> >>> >>>> [direct://start
>>> >>> >>>> ] [ 68]
>>> >>> >>>> [route1 ] [to1 ]
>>> >>> >>>> [jpa://org.apache.camel.examples.SendEmail
>>> >>> >>>> ] [ 67]
>>> >>> >>>>
>>> >>> >>>> Exchange
>>> >>> >>>>
>>> >>>
>>> ---------------------------------------------------------------------------------------------------------------------------------------
>>> >>> >>>> Exchange[
>>> >>> >>>> Id
>>> >>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9
>>> >>> >>>> ExchangePattern InOnly
>>> >>> >>>> Headers
>>> >>> >>>>
>>> >>>
>>> {breadcrumbId=ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4,
>>> >>> >>>>
>>> >>>
>>> CamelEntityManager=org.apache.openjpa.persistence.EntityManagerImpl@6e453dd5
>>> >>> ,
>>> >>> >>>> CamelRedelivered=false, CamelRedeliveryCounter=0}
>>> >>> >>>> BodyType org.apache.camel.examples.SendEmail
>>> >>> >>>> Body SendEmail[id: 102 address: null]
>>> >>> >>>> ]
>>> >>> >>>>
>>> >>> >>>> Stacktrace
>>> >>> >>>>
>>> >>>
>>> ---------------------------------------------------------------------------------------------------------------------------------------
>>> >>> >>>> org.springframework.dao.InvalidDataAccessApiUsageException: This
>>> >>> >>>> operation
>>> >>> >>>> cannot be performed while a Transaction is active.; nested
>>> >>> exception
>>> >>> is
>>> >>> >>>>
>>> >>> >
>>> >>
>> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>>> >>> >>>> org.apache.openjpa.persistence.InvalidStateException: This
>>> >>> operation
>>> >>> >>>> cannot
>>> >>> >>>> be performed while a Transaction is active.
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:293)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.convertException(ExtendedEntityManagerCreator.java:501)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.afterCommit(ExtendedEntityManagerCreator.java:482)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCommit(TransactionSynchronizationUtils.java:133)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.springframework.transaction.support.TransactionSynchronizationUtils.triggerAfterCommit(TransactionSynchronizationUtils.java:121)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCommit(AbstractPlatformTransactionManager.java:951)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:797)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:148)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.component.jpa.JpaProducer.process(JpaProducer.java:53)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:132)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:307)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:127)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>>> >>> >>>> at
>>> >>> org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
>>> >>> >>>> at
>>> >>> org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:56)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.processor.UnitOfWorkProducer.process(UnitOfWorkProducer.java:73)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:378)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:1)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:242)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:346)
>>> >>> >>>> at
>>> >>> org.apache.camel.impl.ProducerCache.send(ProducerCache.java:184)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:124)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:137)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:144)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.processor.jpa.JpaProducerConcurrentTest$1.call(JpaProducerConcurrentTest.java:58)
>>> >>> >>>> at
>>> >>> >>>>
>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>> >>> >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>> >>> >>>> at java.lang.Thread.run(Thread.java:680)
>>> >>> >>>> Caused by:
>>> >>> >>>>
>>> >>> >
>>> >>
>> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>>> >>> >>>> org.apache.openjpa.persistence.InvalidStateException: This
>>> >>> operation
>>> >>> >>>> cannot
>>> >>> >>>> be performed while a Transaction is active.
>>> >>> >>>> at
>>> >>> org.apache.openjpa.kernel.BrokerImpl.begin(BrokerImpl.java:1369)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.openjpa.kernel.DelegatingBroker.begin(DelegatingBroker.java:925)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.openjpa.persistence.EntityManagerImpl.begin(EntityManagerImpl.java:565)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.enlistInCurrentTransaction(ExtendedEntityManagerCreator.java:422)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.doJoinTransaction(ExtendedEntityManagerCreator.java:399)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.invoke(ExtendedEntityManagerCreator.java:351)
>>> >>> >>>> at com.sun.proxy.$Proxy18.joinTransaction(Unknown Source)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.apache.camel.component.jpa.JpaProducer$1.doInTransaction(JpaProducer.java:55)
>>> >>> >>>> at
>>> >>> >>>>
>>> >>>
>>> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:131)
>>> >>> >>>> ... 27 more
>>> >>> >>>>
>>> >>> >>>> which is throwing from the line 55 in
>>> >>> >>>> org.apache.camel.component.jpa.JpaProducer
>>> >>> >>>> entityManager.joinTransaction();
>>> >>> >>>>
>>> >>> >>>> I think this is a concurrency issue we have to investigate (not
>>> >>> sure
>>> >>> >>>> whether the issue is in Camel, Spring, OpenJPA, ...). Will open
>>> a
>>> >>> JIRA
>>> >>> >>>> for
>>> >>> >>>> it.
>>> >>> >>>>
>>> >>> >>>> [1] https://builds.apache.org/job/Camel.trunk.fulltest/1562/
>>> >>> >>>>
>>> >>> >>>> Best,
>>> >>> >>>>
>>> >>> >>>> Christian
>>> >>> >>>> -----------------
>>> >>> >>>>
>>> >>> >>>> Software Integration Specialist
>>> >>> >>>>
>>> >>> >>>> Apache Member
>>> >>> >>>> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel
>>> >>> committer
>>> >>> >>>> Apache Incubator PMC Member
>>> >>> >>>>
>>> >>> >>>> https://www.linkedin.com/pub/christian-mueller/11/551/642
>>> >>> >>>>
>>> >>> >>>>
>>> >>> >>>> On Thu, Oct 10, 2013 at 12:28 PM, James Carman
>>> >>> >>>> <
>>> >>> >>
>>> >>> >>>> james@
>>> >>> >>
>>> >>> >>>> >wrote:
>>> >>> >>>>
>>> >>> >>>>> There are quite a few broken tests. Jetty seems to be hosed.
>>> JPA
>>> >>> has
>>> >>> >>>>> a
>>> >>> >>>>> failed test. CDI has issues. It looks like JMS and FTP even
>>> >>> decided
>>> >>> >>>>> to
>>> >>> >>>>> join the party. This has been going on for a while now. This
>>> >>> >>>>> particular
>>> >>> >>>>> job hasn't been "green" since 9/29. Should we stop working
>>> other
>>> >>> >>>>> issues
>>> >>> >>>>> and get ourselves whole again?
>>> >>> >>>>>
>>> >>> >>>>> https://builds.apache.org/job/Camel.trunk.fulltest/1561/
>>> >>> >>>>>
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >> --
>>> >>> >> View this message in context:
>>> >>> >>
>>> >>>
>>> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741403.html
>>> >>> >> Sent from the Camel Development mailing list archive at
>>> Nabble.com.
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> > --
>>> >>> > Claus Ibsen
>>> >>> > -----------------
>>> >>> > Red Hat, Inc.
>>> >>> > Email:
>>> >>>
>>> >>> > cibsen@
>>> >>>
>>> >>> > Twitter: davsclaus
>>> >>> > Blog: http://davsclaus.com
>>> >>> > Author of Camel in Action: http://www.manning.com/ibsen
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> View this message in context:
>>> >>>
>>> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741448.html
>>> >>> Sent from the Camel Development mailing list archive at Nabble.com.
>>> >>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741864.html
>>> Sent from the Camel Development mailing list archive at Nabble.com.
>>>
>
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741876.html
> Sent from the Camel Development mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Build failures...
Posted by Babak Vahdat <ba...@swissonline.ch>.
CAMEL-5932 also broke one of our distribution examples, so logged another
ticket to get that fixed as well:
https://issues.apache.org/jira/browse/CAMEL-6877
Babak
Am 20.10.13 21:59 schrieb "Christian Müller" unter
<ch...@gmail.com>:
>It works now on my machine.
>Thanks for taking care of it.
>
>Best,
>
>Christian
>-----------------
>
>Software Integration Specialist
>
>Apache Member
>V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
>Apache Incubator PMC Member
>
>https://www.linkedin.com/pub/christian-mueller/11/551/642
>
>
>On Sat, Oct 19, 2013 at 2:30 AM, Babak Vahdat
><ba...@swissonline.ch>wrote:
>
>> Christian,
>>
>> I commited fix for this both on the master as well as the 2.12.x branch.
>> Would you please give it another try to see if you can still reproduce
>>the
>> test failures you mentioned by your previous posts?
>>
>> Babak
>>
>>
>> Christian Mueller wrote
>> > Really good catch!
>> >
>> > Best,
>> >
>> > Christian
>> > -----------------
>> >
>> > Software Integration Specialist
>> >
>> > Apache Member
>> > V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
>> > Apache Incubator PMC Member
>> >
>> > https://www.linkedin.com/pub/christian-mueller/11/551/642
>> >
>> >
>> > On Fri, Oct 18, 2013 at 11:47 PM, Babak Vahdat
>> > <
>>
>> > babak.vahdat@
>>
>> > >wrote:
>> >
>> >> The root cause of the problem for the failing JPA test is CAMEL-5932
>> >> through
>> >> which we started sharing the *same* EntityManager object across
>> >> JpaEndpoint,
>> >> all it's created producers and consumers which is wrong! JPA 2.0
>>Spec,
>> >> section 7.2:
>> >>
>> >> An entity manager must not be shared among multiple concurrently
>> >> executing
>> >> threads, as the entity manager and persistence context are not
>>required
>> >> to
>> >> be threadsafe. Entity managers must only be accessed in a
>> single-threaded
>> >> manner.
>> >>
>> >> That said using OpenJPA you could make use of a proprietary property
>>to
>> >> make
>> >> OpenJPA EntityManager-impl thread safe:
>> >>
>> >>
>> > <property name="openjpa.Multithreaded" value="true"/>
>> >>
>> >> But even then, what about Camel users whose JPA provider is
>>Eclipselink
>> >> or
>> >> Hibernate-EntityManager.
>> >>
>> >> I logged a ticket to get this fixed and hopfully the JPA test being
>> >> mentioned here as well:
>> >>
>> >> https://issues.apache.org/jira/browse/CAMEL-6874
>> >>
>> >> Also there're also some aimless lines of code inside
>> >> JpaProducer#process()
>> >> method which I will clean up as well.
>> >>
>> >> Babak
>> >>
>> >>
>> >> Babak Vahdat wrote
>> >> > Yeah you're right the test failed again on the CI-Server as well.
>> >> There're
>> >> > also couple of unresolved concurrency/deadlock related JIRA
>>tickets by
>> >> > OpenJPA going into the exactly same direction:
>> >> >
>> >> > https://issues.apache.org/jira/browse/OPENJPA-1716
>> >> >
>> >> > Babak
>> >> > Christian Mueller wrote
>> >> >> It's still failing sometimes on my machine: :-(
>> >> >>
>> >> >> Results :
>> >> >> Failed tests:
>> >> >>
>> >> >>
>> >>
>>
>>JpaProducerConcurrentTest.testConcurrentProducers:45->doSendMessages:66->
>>CamelTestSupport.assertMockEndpointsSatisfied:682
>> >> >> mock://result Received message count. Expected: <10> but was: <8>
>> >> >>
>> >> >> Tests run: 49, Failures: 1, Errors: 0, Skipped: 0
>> >> >>
>> >> >> [INFO]
>> >> >>
>> >>
>>------------------------------------------------------------------------
>> >> >> [INFO] BUILD FAILURE
>> >> >> [INFO]
>> >> >>
>> >>
>>------------------------------------------------------------------------
>> >> >>
>> >> >>
>> >> >> Best,
>> >> >> Christian
>> >> >> -----------------
>> >> >>
>> >> >> Software Integration Specialist
>> >> >>
>> >> >> Apache Member
>> >> >> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel
>>committer
>> >> >> Apache Incubator PMC Member
>> >> >>
>> >> >> https://www.linkedin.com/pub/christian-mueller/11/551/642
>> >> >>
>> >> >>
>> >> >> On Sat, Oct 12, 2013 at 11:07 PM, Babak Vahdat
>> >> >> <
>> >>
>> >> >> babak.vahdat@
>> >>
>> >> >> >wrote:
>> >> >>
>> >> >>> Yeah indeed a deadlock inside the OpenJPA code is the root cause
>>of
>> >> the
>> >> >>> problem (org.apache.openjpa.kernel.BrokerImpl#lock()). I just
>> >> commited
>> >> a
>> >> >>> fix
>> >> >>> regarding this, so let's keep an eye on this test by the next
>> builds.
>> >> >>> Following a screenshot of the proven deadlock through jconsole:
>> >> >>>
>> >> >>> <
>> >> >>>
>> >>
>>
>>http://camel.465427.n5.nabble.com/file/n5741448/Screen_Shot_2013-10-12_at
>>_11.02.58_PM.png
>> >> >>> >
>> >> >>>
>> >> >>> You can enforce the deadlock if you would make use of a high log
>> >> level
>> >> >>> (log4j.properties), e.g. TRACE instead of just INFO.
>> >> >>>
>> >> >>> Babak
>> >> >>>
>> >> >>>
>> >> >>> Claus Ibsen-2 wrote
>> >> >>> > It could be due an OpenJPA upgrade. These tests used to work
>> better
>> >> in
>> >> >>> the
>> >> >>> > past.
>> >> >>> >
>> >> >>> > On Sat, Oct 12, 2013 at 10:05 AM, Babak Vahdat
>> >> >>> > <
>> >> >>>
>> >> >>> > babak.vahdat@
>> >> >>>
>> >> >>> > > wrote:
>> >> >>> >> The test failed again on the CI-Server. I will look into this
>>in
>> >> the
>> >> >>> next
>> >> >>> >> days to see if I can find the root cause as on my box I can't
>> >> >>> reproduce
>> >> >>> >> this.
>> >> >>> >>
>> >> >>> >> Babak
>> >> >>> >>
>> >> >>> >>
>> >> >>> >> Babak Vahdat wrote
>> >> >>> >>> Hi
>> >> >>> >>>
>> >> >>> >>> @Christian I believe I've fixed the JPA test now, could you
>> >> please
>> >> >>> check
>> >> >>> >>> to see if you can still reproduce the same exception you
>> >> mentioned
>> >> >>> >>> below?
>> >> >>> >>>
>> >> >>> >>> Babak
>> >> >>> >>> Christian Mueller wrote
>> >> >>> >>>> Today, it looks much better [1]. We are trying to fix the
>>unit
>> >> >>> tests
>> >> >>> >>>> which
>> >> >>> >>>> are failing from time to time on slower boxes. But this
>>isn't
>> so
>> >> >>> easy
>> >> >>> >>>> for
>> >> >>> >>>> all of them...
>> >> >>> >>>>
>> >> >>> >>>> I had a look at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.apache.camel.processor.jpa.JpaProducerConcurrentTest.testConcurrentPr
>>oducers.
>> >> >>> >>>> A few times I get the following exception which results in
>> >> missing
>> >> >>> >>>> exchanges in the mock:result endpoint:
>> >> >>> >>>>
>> >> >>> >>>> 2013-10-11 14:01:10,623 [pool-2-thread-2] ERROR
>> >> >>> >>>> DefaultErrorHandler - Failed delivery for
>> (MessageId:
>> >> >>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4
>>on
>> >> >>> >>>> ExchangeId:
>> >> >>> >>>>
>>ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9).
>> >> >>> Exhausted
>> >> >>> >>>> after delivery attempt: 1 caught:
>> >> >>> >>>> org.springframework.dao.InvalidDataAccessApiUsageException:
>> This
>> >> >>> >>>> operation
>> >> >>> >>>> cannot be performed while a Transaction is active.; nested
>> >> >>> exception
>> >> >>> is
>> >> >>> >>>>
>> >> >>> >
>> >> >>
>> > <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>> >> >>> >>>> org.apache.openjpa.persistence.InvalidStateException: This
>> >> >>> operation
>> >> >>> >>>> cannot
>> >> >>> >>>> be performed while a Transaction is active.
>> >> >>> >>>>
>> >> >>> >>>> Message History
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>-------------------------------------------------------------------------
>>--------------------------------------------------------------
>> >> >>> >>>> RouteId ProcessorId
>> >> >>> >>>> Processor
>> >> >>> >>>> Elapsed (ms)
>> >> >>> >>>> [route1 ] [route1 ]
>> >> >>> >>>> [direct://start
>> >> >>> >>>> ] [ 68]
>> >> >>> >>>> [route1 ] [to1 ]
>> >> >>> >>>> [jpa://org.apache.camel.examples.SendEmail
>> >> >>> >>>> ] [ 67]
>> >> >>> >>>>
>> >> >>> >>>> Exchange
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>-------------------------------------------------------------------------
>>--------------------------------------------------------------
>> >> >>> >>>> Exchange[
>> >> >>> >>>> Id
>> >> >>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9
>> >> >>> >>>> ExchangePattern InOnly
>> >> >>> >>>> Headers
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>{breadcrumbId=ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4
>>,
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>CamelEntityManager=org.apache.openjpa.persistence.EntityManagerImpl@6e453
>>dd5
>> >> >>> ,
>> >> >>> >>>> CamelRedelivered=false, CamelRedeliveryCounter=0}
>> >> >>> >>>> BodyType org.apache.camel.examples.SendEmail
>> >> >>> >>>> Body SendEmail[id: 102 address: null]
>> >> >>> >>>> ]
>> >> >>> >>>>
>> >> >>> >>>> Stacktrace
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>-------------------------------------------------------------------------
>>--------------------------------------------------------------
>> >> >>> >>>> org.springframework.dao.InvalidDataAccessApiUsageException:
>> This
>> >> >>> >>>> operation
>> >> >>> >>>> cannot be performed while a Transaction is active.; nested
>> >> >>> exception
>> >> >>> is
>> >> >>> >>>>
>> >> >>> >
>> >> >>
>> > <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>> >> >>> >>>> org.apache.openjpa.persistence.InvalidStateException: This
>> >> >>> operation
>> >> >>> >>>> cannot
>> >> >>> >>>> be performed while a Transaction is active.
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExc
>>eptionIfPossible(EntityManagerFactoryUtils.java:293)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityMa
>>nagerSynchronization.convertException(ExtendedEntityManagerCreator.java:5
>>01)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityMa
>>nagerSynchronization.afterCommit(ExtendedEntityManagerCreator.java:482)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.springframework.transaction.support.TransactionSynchronizationUtils.i
>>nvokeAfterCommit(TransactionSynchronizationUtils.java:133)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.springframework.transaction.support.TransactionSynchronizationUtils.t
>>riggerAfterCommit(TransactionSynchronizationUtils.java:121)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.springframework.transaction.support.AbstractPlatformTransactionManage
>>r.triggerAfterCommit(AbstractPlatformTransactionManager.java:951)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.springframework.transaction.support.AbstractPlatformTransactionManage
>>r.processCommit(AbstractPlatformTransactionManager.java:797)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.springframework.transaction.support.AbstractPlatformTransactionManage
>>r.commit(AbstractPlatformTransactionManager.java:724)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.springframework.transaction.support.TransactionTemplate.execute(Trans
>>actionTemplate.java:148)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>org.apache.camel.component.jpa.JpaProducer.process(JpaProducer.java:53)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProce
>>ssorBridge.process(AsyncProcessorConverterHelper.java:61)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcesso
>>r.java:132)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:
>>307)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>org.apache.camel.processor.SendProcessor.process(SendProcessor.java:127)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryError
>>Handler.java:398)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalPr
>>ocessor.java:191)
>> >> >>> >>>> at
>> >> >>> org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
>> >> >>> >>>> at
>> >> >>> org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalPr
>>ocessor.java:191)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.apache.camel.component.direct.DirectProducer.process(DirectProducer.j
>>ava:56)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalPr
>>ocessor.java:191)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.apache.camel.processor.UnitOfWorkProducer.process(UnitOfWorkProducer.
>>java:73)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:378
>>)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:1)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:242)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:346)
>> >> >>> >>>> at
>> >> >>> org.apache.camel.impl.ProducerCache.send(ProducerCache.java:184)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplat
>>e.java:124)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTem
>>plate.java:137)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTem
>>plate.java:144)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.apache.camel.processor.jpa.JpaProducerConcurrentTest$1.call(JpaProduc
>>erConcurrentTest.java:58)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> >> >>> >>>> at
>>java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor
>>.java:895)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
>>a:918)
>> >> >>> >>>> at java.lang.Thread.run(Thread.java:680)
>> >> >>> >>>> Caused by:
>> >> >>> >>>>
>> >> >>> >
>> >> >>
>> > <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>> >> >>> >>>> org.apache.openjpa.persistence.InvalidStateException: This
>> >> >>> operation
>> >> >>> >>>> cannot
>> >> >>> >>>> be performed while a Transaction is active.
>> >> >>> >>>> at
>> >> >>> org.apache.openjpa.kernel.BrokerImpl.begin(BrokerImpl.java:1369)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.apache.openjpa.kernel.DelegatingBroker.begin(DelegatingBroker.java:92
>>5)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.apache.openjpa.persistence.EntityManagerImpl.begin(EntityManagerImpl.
>>java:565)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityMa
>>nagerInvocationHandler.enlistInCurrentTransaction(ExtendedEntityManagerCr
>>eator.java:422)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityMa
>>nagerInvocationHandler.doJoinTransaction(ExtendedEntityManagerCreator.jav
>>a:399)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityMa
>>nagerInvocationHandler.invoke(ExtendedEntityManagerCreator.java:351)
>> >> >>> >>>> at com.sun.proxy.$Proxy18.joinTransaction(Unknown
>>Source)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.apache.camel.component.jpa.JpaProducer$1.doInTransaction(JpaProducer.
>>java:55)
>> >> >>> >>>> at
>> >> >>> >>>>
>> >> >>>
>> >>
>>
>>org.springframework.transaction.support.TransactionTemplate.execute(Trans
>>actionTemplate.java:131)
>> >> >>> >>>> ... 27 more
>> >> >>> >>>>
>> >> >>> >>>> which is throwing from the line 55 in
>> >> >>> >>>> org.apache.camel.component.jpa.JpaProducer
>> >> >>> >>>> entityManager.joinTransaction();
>> >> >>> >>>>
>> >> >>> >>>> I think this is a concurrency issue we have to investigate
>>(not
>> >> >>> sure
>> >> >>> >>>> whether the issue is in Camel, Spring, OpenJPA, ...). Will
>>open
>> >> a
>> >> >>> JIRA
>> >> >>> >>>> for
>> >> >>> >>>> it.
>> >> >>> >>>>
>> >> >>> >>>> [1] https://builds.apache.org/job/Camel.trunk.fulltest/1562/
>> >> >>> >>>>
>> >> >>> >>>> Best,
>> >> >>> >>>>
>> >> >>> >>>> Christian
>> >> >>> >>>> -----------------
>> >> >>> >>>>
>> >> >>> >>>> Software Integration Specialist
>> >> >>> >>>>
>> >> >>> >>>> Apache Member
>> >> >>> >>>> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel
>> >> >>> committer
>> >> >>> >>>> Apache Incubator PMC Member
>> >> >>> >>>>
>> >> >>> >>>> https://www.linkedin.com/pub/christian-mueller/11/551/642
>> >> >>> >>>>
>> >> >>> >>>>
>> >> >>> >>>> On Thu, Oct 10, 2013 at 12:28 PM, James Carman
>> >> >>> >>>> <
>> >> >>> >>
>> >> >>> >>>> james@
>> >> >>> >>
>> >> >>> >>>> >wrote:
>> >> >>> >>>>
>> >> >>> >>>>> There are quite a few broken tests. Jetty seems to be
>>hosed.
>> >> JPA
>> >> >>> has
>> >> >>> >>>>> a
>> >> >>> >>>>> failed test. CDI has issues. It looks like JMS and FTP
>>even
>> >> >>> decided
>> >> >>> >>>>> to
>> >> >>> >>>>> join the party. This has been going on for a while now.
>>This
>> >> >>> >>>>> particular
>> >> >>> >>>>> job hasn't been "green" since 9/29. Should we stop working
>> >> other
>> >> >>> >>>>> issues
>> >> >>> >>>>> and get ourselves whole again?
>> >> >>> >>>>>
>> >> >>> >>>>> https://builds.apache.org/job/Camel.trunk.fulltest/1561/
>> >> >>> >>>>>
>> >> >>> >>
>> >> >>> >>
>> >> >>> >>
>> >> >>> >>
>> >> >>> >>
>> >> >>> >> --
>> >> >>> >> View this message in context:
>> >> >>> >>
>> >> >>>
>> >>
>>http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741403.html
>> >> >>> >> Sent from the Camel Development mailing list archive at
>> >> Nabble.com.
>> >> >>> >
>> >> >>> >
>> >> >>> >
>> >> >>> > --
>> >> >>> > Claus Ibsen
>> >> >>> > -----------------
>> >> >>> > Red Hat, Inc.
>> >> >>> > Email:
>> >> >>>
>> >> >>> > cibsen@
>> >> >>>
>> >> >>> > Twitter: davsclaus
>> >> >>> > Blog: http://davsclaus.com
>> >> >>> > Author of Camel in Action: http://www.manning.com/ibsen
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> --
>> >> >>> View this message in context:
>> >> >>>
>> >>
>>http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741448.html
>> >> >>> Sent from the Camel Development mailing list archive at
>>Nabble.com.
>> >> >>>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> View this message in context:
>> >>
>>http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741864.html
>> >> Sent from the Camel Development mailing list archive at Nabble.com.
>> >>
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741876.html
>> Sent from the Camel Development mailing list archive at Nabble.com.
>>
Re: Build failures...
Posted by Christian Müller <ch...@gmail.com>.
It works now on my machine.
Thanks for taking care of it.
Best,
Christian
-----------------
Software Integration Specialist
Apache Member
V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
Apache Incubator PMC Member
https://www.linkedin.com/pub/christian-mueller/11/551/642
On Sat, Oct 19, 2013 at 2:30 AM, Babak Vahdat
<ba...@swissonline.ch>wrote:
> Christian,
>
> I commited fix for this both on the master as well as the 2.12.x branch.
> Would you please give it another try to see if you can still reproduce the
> test failures you mentioned by your previous posts?
>
> Babak
>
>
> Christian Mueller wrote
> > Really good catch!
> >
> > Best,
> >
> > Christian
> > -----------------
> >
> > Software Integration Specialist
> >
> > Apache Member
> > V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
> > Apache Incubator PMC Member
> >
> > https://www.linkedin.com/pub/christian-mueller/11/551/642
> >
> >
> > On Fri, Oct 18, 2013 at 11:47 PM, Babak Vahdat
> > <
>
> > babak.vahdat@
>
> > >wrote:
> >
> >> The root cause of the problem for the failing JPA test is CAMEL-5932
> >> through
> >> which we started sharing the *same* EntityManager object across
> >> JpaEndpoint,
> >> all it's created producers and consumers which is wrong! JPA 2.0 Spec,
> >> section 7.2:
> >>
> >> An entity manager must not be shared among multiple concurrently
> >> executing
> >> threads, as the entity manager and persistence context are not required
> >> to
> >> be threadsafe. Entity managers must only be accessed in a
> single-threaded
> >> manner.
> >>
> >> That said using OpenJPA you could make use of a proprietary property to
> >> make
> >> OpenJPA EntityManager-impl thread safe:
> >>
> >>
> > <property name="openjpa.Multithreaded" value="true"/>
> >>
> >> But even then, what about Camel users whose JPA provider is Eclipselink
> >> or
> >> Hibernate-EntityManager.
> >>
> >> I logged a ticket to get this fixed and hopfully the JPA test being
> >> mentioned here as well:
> >>
> >> https://issues.apache.org/jira/browse/CAMEL-6874
> >>
> >> Also there're also some aimless lines of code inside
> >> JpaProducer#process()
> >> method which I will clean up as well.
> >>
> >> Babak
> >>
> >>
> >> Babak Vahdat wrote
> >> > Yeah you're right the test failed again on the CI-Server as well.
> >> There're
> >> > also couple of unresolved concurrency/deadlock related JIRA tickets by
> >> > OpenJPA going into the exactly same direction:
> >> >
> >> > https://issues.apache.org/jira/browse/OPENJPA-1716
> >> >
> >> > Babak
> >> > Christian Mueller wrote
> >> >> It's still failing sometimes on my machine: :-(
> >> >>
> >> >> Results :
> >> >> Failed tests:
> >> >>
> >> >>
> >>
> JpaProducerConcurrentTest.testConcurrentProducers:45->doSendMessages:66->CamelTestSupport.assertMockEndpointsSatisfied:682
> >> >> mock://result Received message count. Expected: <10> but was: <8>
> >> >>
> >> >> Tests run: 49, Failures: 1, Errors: 0, Skipped: 0
> >> >>
> >> >> [INFO]
> >> >>
> >> ------------------------------------------------------------------------
> >> >> [INFO] BUILD FAILURE
> >> >> [INFO]
> >> >>
> >> ------------------------------------------------------------------------
> >> >>
> >> >>
> >> >> Best,
> >> >> Christian
> >> >> -----------------
> >> >>
> >> >> Software Integration Specialist
> >> >>
> >> >> Apache Member
> >> >> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
> >> >> Apache Incubator PMC Member
> >> >>
> >> >> https://www.linkedin.com/pub/christian-mueller/11/551/642
> >> >>
> >> >>
> >> >> On Sat, Oct 12, 2013 at 11:07 PM, Babak Vahdat
> >> >> <
> >>
> >> >> babak.vahdat@
> >>
> >> >> >wrote:
> >> >>
> >> >>> Yeah indeed a deadlock inside the OpenJPA code is the root cause of
> >> the
> >> >>> problem (org.apache.openjpa.kernel.BrokerImpl#lock()). I just
> >> commited
> >> a
> >> >>> fix
> >> >>> regarding this, so let's keep an eye on this test by the next
> builds.
> >> >>> Following a screenshot of the proven deadlock through jconsole:
> >> >>>
> >> >>> <
> >> >>>
> >>
> http://camel.465427.n5.nabble.com/file/n5741448/Screen_Shot_2013-10-12_at_11.02.58_PM.png
> >> >>> >
> >> >>>
> >> >>> You can enforce the deadlock if you would make use of a high log
> >> level
> >> >>> (log4j.properties), e.g. TRACE instead of just INFO.
> >> >>>
> >> >>> Babak
> >> >>>
> >> >>>
> >> >>> Claus Ibsen-2 wrote
> >> >>> > It could be due an OpenJPA upgrade. These tests used to work
> better
> >> in
> >> >>> the
> >> >>> > past.
> >> >>> >
> >> >>> > On Sat, Oct 12, 2013 at 10:05 AM, Babak Vahdat
> >> >>> > <
> >> >>>
> >> >>> > babak.vahdat@
> >> >>>
> >> >>> > > wrote:
> >> >>> >> The test failed again on the CI-Server. I will look into this in
> >> the
> >> >>> next
> >> >>> >> days to see if I can find the root cause as on my box I can't
> >> >>> reproduce
> >> >>> >> this.
> >> >>> >>
> >> >>> >> Babak
> >> >>> >>
> >> >>> >>
> >> >>> >> Babak Vahdat wrote
> >> >>> >>> Hi
> >> >>> >>>
> >> >>> >>> @Christian I believe I've fixed the JPA test now, could you
> >> please
> >> >>> check
> >> >>> >>> to see if you can still reproduce the same exception you
> >> mentioned
> >> >>> >>> below?
> >> >>> >>>
> >> >>> >>> Babak
> >> >>> >>> Christian Mueller wrote
> >> >>> >>>> Today, it looks much better [1]. We are trying to fix the unit
> >> >>> tests
> >> >>> >>>> which
> >> >>> >>>> are failing from time to time on slower boxes. But this isn't
> so
> >> >>> easy
> >> >>> >>>> for
> >> >>> >>>> all of them...
> >> >>> >>>>
> >> >>> >>>> I had a look at
> >> >>> >>>>
> >> >>>
> >>
> org.apache.camel.processor.jpa.JpaProducerConcurrentTest.testConcurrentProducers.
> >> >>> >>>> A few times I get the following exception which results in
> >> missing
> >> >>> >>>> exchanges in the mock:result endpoint:
> >> >>> >>>>
> >> >>> >>>> 2013-10-11 14:01:10,623 [pool-2-thread-2] ERROR
> >> >>> >>>> DefaultErrorHandler - Failed delivery for
> (MessageId:
> >> >>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4 on
> >> >>> >>>> ExchangeId:
> >> >>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9).
> >> >>> Exhausted
> >> >>> >>>> after delivery attempt: 1 caught:
> >> >>> >>>> org.springframework.dao.InvalidDataAccessApiUsageException:
> This
> >> >>> >>>> operation
> >> >>> >>>> cannot be performed while a Transaction is active.; nested
> >> >>> exception
> >> >>> is
> >> >>> >>>>
> >> >>> >
> >> >>
> > <openjpa-2.2.2-r422266:1468616 nonfatal user error>
> >> >>> >>>> org.apache.openjpa.persistence.InvalidStateException: This
> >> >>> operation
> >> >>> >>>> cannot
> >> >>> >>>> be performed while a Transaction is active.
> >> >>> >>>>
> >> >>> >>>> Message History
> >> >>> >>>>
> >> >>>
> >>
> ---------------------------------------------------------------------------------------------------------------------------------------
> >> >>> >>>> RouteId ProcessorId
> >> >>> >>>> Processor
> >> >>> >>>> Elapsed (ms)
> >> >>> >>>> [route1 ] [route1 ]
> >> >>> >>>> [direct://start
> >> >>> >>>> ] [ 68]
> >> >>> >>>> [route1 ] [to1 ]
> >> >>> >>>> [jpa://org.apache.camel.examples.SendEmail
> >> >>> >>>> ] [ 67]
> >> >>> >>>>
> >> >>> >>>> Exchange
> >> >>> >>>>
> >> >>>
> >>
> ---------------------------------------------------------------------------------------------------------------------------------------
> >> >>> >>>> Exchange[
> >> >>> >>>> Id
> >> >>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9
> >> >>> >>>> ExchangePattern InOnly
> >> >>> >>>> Headers
> >> >>> >>>>
> >> >>>
> >>
> {breadcrumbId=ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4,
> >> >>> >>>>
> >> >>>
> >>
> CamelEntityManager=org.apache.openjpa.persistence.EntityManagerImpl@6e453dd5
> >> >>> ,
> >> >>> >>>> CamelRedelivered=false, CamelRedeliveryCounter=0}
> >> >>> >>>> BodyType org.apache.camel.examples.SendEmail
> >> >>> >>>> Body SendEmail[id: 102 address: null]
> >> >>> >>>> ]
> >> >>> >>>>
> >> >>> >>>> Stacktrace
> >> >>> >>>>
> >> >>>
> >>
> ---------------------------------------------------------------------------------------------------------------------------------------
> >> >>> >>>> org.springframework.dao.InvalidDataAccessApiUsageException:
> This
> >> >>> >>>> operation
> >> >>> >>>> cannot be performed while a Transaction is active.; nested
> >> >>> exception
> >> >>> is
> >> >>> >>>>
> >> >>> >
> >> >>
> > <openjpa-2.2.2-r422266:1468616 nonfatal user error>
> >> >>> >>>> org.apache.openjpa.persistence.InvalidStateException: This
> >> >>> operation
> >> >>> >>>> cannot
> >> >>> >>>> be performed while a Transaction is active.
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:293)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.convertException(ExtendedEntityManagerCreator.java:501)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.afterCommit(ExtendedEntityManagerCreator.java:482)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCommit(TransactionSynchronizationUtils.java:133)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.springframework.transaction.support.TransactionSynchronizationUtils.triggerAfterCommit(TransactionSynchronizationUtils.java:121)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCommit(AbstractPlatformTransactionManager.java:951)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:797)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:148)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >> org.apache.camel.component.jpa.JpaProducer.process(JpaProducer.java:53)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:132)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:307)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:127)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
> >> >>> >>>> at
> >> >>> org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
> >> >>> >>>> at
> >> >>> org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:56)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.apache.camel.processor.UnitOfWorkProducer.process(UnitOfWorkProducer.java:73)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:378)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:1)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >> org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:242)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >> org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:346)
> >> >>> >>>> at
> >> >>> org.apache.camel.impl.ProducerCache.send(ProducerCache.java:184)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:124)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:137)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:144)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.apache.camel.processor.jpa.JpaProducerConcurrentTest$1.call(JpaProducerConcurrentTest.java:58)
> >> >>> >>>> at
> >> >>> >>>>
> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> >> >>> >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> >> >>> >>>> at java.lang.Thread.run(Thread.java:680)
> >> >>> >>>> Caused by:
> >> >>> >>>>
> >> >>> >
> >> >>
> > <openjpa-2.2.2-r422266:1468616 nonfatal user error>
> >> >>> >>>> org.apache.openjpa.persistence.InvalidStateException: This
> >> >>> operation
> >> >>> >>>> cannot
> >> >>> >>>> be performed while a Transaction is active.
> >> >>> >>>> at
> >> >>> org.apache.openjpa.kernel.BrokerImpl.begin(BrokerImpl.java:1369)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.apache.openjpa.kernel.DelegatingBroker.begin(DelegatingBroker.java:925)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.apache.openjpa.persistence.EntityManagerImpl.begin(EntityManagerImpl.java:565)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.enlistInCurrentTransaction(ExtendedEntityManagerCreator.java:422)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.doJoinTransaction(ExtendedEntityManagerCreator.java:399)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.invoke(ExtendedEntityManagerCreator.java:351)
> >> >>> >>>> at com.sun.proxy.$Proxy18.joinTransaction(Unknown Source)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.apache.camel.component.jpa.JpaProducer$1.doInTransaction(JpaProducer.java:55)
> >> >>> >>>> at
> >> >>> >>>>
> >> >>>
> >>
> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:131)
> >> >>> >>>> ... 27 more
> >> >>> >>>>
> >> >>> >>>> which is throwing from the line 55 in
> >> >>> >>>> org.apache.camel.component.jpa.JpaProducer
> >> >>> >>>> entityManager.joinTransaction();
> >> >>> >>>>
> >> >>> >>>> I think this is a concurrency issue we have to investigate (not
> >> >>> sure
> >> >>> >>>> whether the issue is in Camel, Spring, OpenJPA, ...). Will open
> >> a
> >> >>> JIRA
> >> >>> >>>> for
> >> >>> >>>> it.
> >> >>> >>>>
> >> >>> >>>> [1] https://builds.apache.org/job/Camel.trunk.fulltest/1562/
> >> >>> >>>>
> >> >>> >>>> Best,
> >> >>> >>>>
> >> >>> >>>> Christian
> >> >>> >>>> -----------------
> >> >>> >>>>
> >> >>> >>>> Software Integration Specialist
> >> >>> >>>>
> >> >>> >>>> Apache Member
> >> >>> >>>> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel
> >> >>> committer
> >> >>> >>>> Apache Incubator PMC Member
> >> >>> >>>>
> >> >>> >>>> https://www.linkedin.com/pub/christian-mueller/11/551/642
> >> >>> >>>>
> >> >>> >>>>
> >> >>> >>>> On Thu, Oct 10, 2013 at 12:28 PM, James Carman
> >> >>> >>>> <
> >> >>> >>
> >> >>> >>>> james@
> >> >>> >>
> >> >>> >>>> >wrote:
> >> >>> >>>>
> >> >>> >>>>> There are quite a few broken tests. Jetty seems to be hosed.
> >> JPA
> >> >>> has
> >> >>> >>>>> a
> >> >>> >>>>> failed test. CDI has issues. It looks like JMS and FTP even
> >> >>> decided
> >> >>> >>>>> to
> >> >>> >>>>> join the party. This has been going on for a while now. This
> >> >>> >>>>> particular
> >> >>> >>>>> job hasn't been "green" since 9/29. Should we stop working
> >> other
> >> >>> >>>>> issues
> >> >>> >>>>> and get ourselves whole again?
> >> >>> >>>>>
> >> >>> >>>>> https://builds.apache.org/job/Camel.trunk.fulltest/1561/
> >> >>> >>>>>
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >> --
> >> >>> >> View this message in context:
> >> >>> >>
> >> >>>
> >> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741403.html
> >> >>> >> Sent from the Camel Development mailing list archive at
> >> Nabble.com.
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> > --
> >> >>> > Claus Ibsen
> >> >>> > -----------------
> >> >>> > Red Hat, Inc.
> >> >>> > Email:
> >> >>>
> >> >>> > cibsen@
> >> >>>
> >> >>> > Twitter: davsclaus
> >> >>> > Blog: http://davsclaus.com
> >> >>> > Author of Camel in Action: http://www.manning.com/ibsen
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> View this message in context:
> >> >>>
> >> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741448.html
> >> >>> Sent from the Camel Development mailing list archive at Nabble.com.
> >> >>>
> >>
> >>
> >>
> >>
> >>
> >> --
> >> View this message in context:
> >> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741864.html
> >> Sent from the Camel Development mailing list archive at Nabble.com.
> >>
>
>
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741876.html
> Sent from the Camel Development mailing list archive at Nabble.com.
>
Re: Build failures...
Posted by Babak Vahdat <ba...@swissonline.ch>.
Christian,
I commited fix for this both on the master as well as the 2.12.x branch.
Would you please give it another try to see if you can still reproduce the
test failures you mentioned by your previous posts?
Babak
Christian Mueller wrote
> Really good catch!
>
> Best,
>
> Christian
> -----------------
>
> Software Integration Specialist
>
> Apache Member
> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
> Apache Incubator PMC Member
>
> https://www.linkedin.com/pub/christian-mueller/11/551/642
>
>
> On Fri, Oct 18, 2013 at 11:47 PM, Babak Vahdat
> <
> babak.vahdat@
> >wrote:
>
>> The root cause of the problem for the failing JPA test is CAMEL-5932
>> through
>> which we started sharing the *same* EntityManager object across
>> JpaEndpoint,
>> all it's created producers and consumers which is wrong! JPA 2.0 Spec,
>> section 7.2:
>>
>> An entity manager must not be shared among multiple concurrently
>> executing
>> threads, as the entity manager and persistence context are not required
>> to
>> be threadsafe. Entity managers must only be accessed in a single-threaded
>> manner.
>>
>> That said using OpenJPA you could make use of a proprietary property to
>> make
>> OpenJPA EntityManager-impl thread safe:
>>
>>
> <property name="openjpa.Multithreaded" value="true"/>
>>
>> But even then, what about Camel users whose JPA provider is Eclipselink
>> or
>> Hibernate-EntityManager.
>>
>> I logged a ticket to get this fixed and hopfully the JPA test being
>> mentioned here as well:
>>
>> https://issues.apache.org/jira/browse/CAMEL-6874
>>
>> Also there're also some aimless lines of code inside
>> JpaProducer#process()
>> method which I will clean up as well.
>>
>> Babak
>>
>>
>> Babak Vahdat wrote
>> > Yeah you're right the test failed again on the CI-Server as well.
>> There're
>> > also couple of unresolved concurrency/deadlock related JIRA tickets by
>> > OpenJPA going into the exactly same direction:
>> >
>> > https://issues.apache.org/jira/browse/OPENJPA-1716
>> >
>> > Babak
>> > Christian Mueller wrote
>> >> It's still failing sometimes on my machine: :-(
>> >>
>> >> Results :
>> >> Failed tests:
>> >>
>> >>
>> JpaProducerConcurrentTest.testConcurrentProducers:45->doSendMessages:66->CamelTestSupport.assertMockEndpointsSatisfied:682
>> >> mock://result Received message count. Expected: <10> but was: <8>
>> >>
>> >> Tests run: 49, Failures: 1, Errors: 0, Skipped: 0
>> >>
>> >> [INFO]
>> >>
>> ------------------------------------------------------------------------
>> >> [INFO] BUILD FAILURE
>> >> [INFO]
>> >>
>> ------------------------------------------------------------------------
>> >>
>> >>
>> >> Best,
>> >> Christian
>> >> -----------------
>> >>
>> >> Software Integration Specialist
>> >>
>> >> Apache Member
>> >> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
>> >> Apache Incubator PMC Member
>> >>
>> >> https://www.linkedin.com/pub/christian-mueller/11/551/642
>> >>
>> >>
>> >> On Sat, Oct 12, 2013 at 11:07 PM, Babak Vahdat
>> >> <
>>
>> >> babak.vahdat@
>>
>> >> >wrote:
>> >>
>> >>> Yeah indeed a deadlock inside the OpenJPA code is the root cause of
>> the
>> >>> problem (org.apache.openjpa.kernel.BrokerImpl#lock()). I just
>> commited
>> a
>> >>> fix
>> >>> regarding this, so let's keep an eye on this test by the next builds.
>> >>> Following a screenshot of the proven deadlock through jconsole:
>> >>>
>> >>> <
>> >>>
>> http://camel.465427.n5.nabble.com/file/n5741448/Screen_Shot_2013-10-12_at_11.02.58_PM.png
>> >>> >
>> >>>
>> >>> You can enforce the deadlock if you would make use of a high log
>> level
>> >>> (log4j.properties), e.g. TRACE instead of just INFO.
>> >>>
>> >>> Babak
>> >>>
>> >>>
>> >>> Claus Ibsen-2 wrote
>> >>> > It could be due an OpenJPA upgrade. These tests used to work better
>> in
>> >>> the
>> >>> > past.
>> >>> >
>> >>> > On Sat, Oct 12, 2013 at 10:05 AM, Babak Vahdat
>> >>> > <
>> >>>
>> >>> > babak.vahdat@
>> >>>
>> >>> > > wrote:
>> >>> >> The test failed again on the CI-Server. I will look into this in
>> the
>> >>> next
>> >>> >> days to see if I can find the root cause as on my box I can't
>> >>> reproduce
>> >>> >> this.
>> >>> >>
>> >>> >> Babak
>> >>> >>
>> >>> >>
>> >>> >> Babak Vahdat wrote
>> >>> >>> Hi
>> >>> >>>
>> >>> >>> @Christian I believe I've fixed the JPA test now, could you
>> please
>> >>> check
>> >>> >>> to see if you can still reproduce the same exception you
>> mentioned
>> >>> >>> below?
>> >>> >>>
>> >>> >>> Babak
>> >>> >>> Christian Mueller wrote
>> >>> >>>> Today, it looks much better [1]. We are trying to fix the unit
>> >>> tests
>> >>> >>>> which
>> >>> >>>> are failing from time to time on slower boxes. But this isn't so
>> >>> easy
>> >>> >>>> for
>> >>> >>>> all of them...
>> >>> >>>>
>> >>> >>>> I had a look at
>> >>> >>>>
>> >>>
>> org.apache.camel.processor.jpa.JpaProducerConcurrentTest.testConcurrentProducers.
>> >>> >>>> A few times I get the following exception which results in
>> missing
>> >>> >>>> exchanges in the mock:result endpoint:
>> >>> >>>>
>> >>> >>>> 2013-10-11 14:01:10,623 [pool-2-thread-2] ERROR
>> >>> >>>> DefaultErrorHandler - Failed delivery for (MessageId:
>> >>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4 on
>> >>> >>>> ExchangeId:
>> >>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9).
>> >>> Exhausted
>> >>> >>>> after delivery attempt: 1 caught:
>> >>> >>>> org.springframework.dao.InvalidDataAccessApiUsageException: This
>> >>> >>>> operation
>> >>> >>>> cannot be performed while a Transaction is active.; nested
>> >>> exception
>> >>> is
>> >>> >>>>
>> >>> >
>> >>
> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>> >>> >>>> org.apache.openjpa.persistence.InvalidStateException: This
>> >>> operation
>> >>> >>>> cannot
>> >>> >>>> be performed while a Transaction is active.
>> >>> >>>>
>> >>> >>>> Message History
>> >>> >>>>
>> >>>
>> ---------------------------------------------------------------------------------------------------------------------------------------
>> >>> >>>> RouteId ProcessorId
>> >>> >>>> Processor
>> >>> >>>> Elapsed (ms)
>> >>> >>>> [route1 ] [route1 ]
>> >>> >>>> [direct://start
>> >>> >>>> ] [ 68]
>> >>> >>>> [route1 ] [to1 ]
>> >>> >>>> [jpa://org.apache.camel.examples.SendEmail
>> >>> >>>> ] [ 67]
>> >>> >>>>
>> >>> >>>> Exchange
>> >>> >>>>
>> >>>
>> ---------------------------------------------------------------------------------------------------------------------------------------
>> >>> >>>> Exchange[
>> >>> >>>> Id
>> >>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9
>> >>> >>>> ExchangePattern InOnly
>> >>> >>>> Headers
>> >>> >>>>
>> >>>
>> {breadcrumbId=ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4,
>> >>> >>>>
>> >>>
>> CamelEntityManager=org.apache.openjpa.persistence.EntityManagerImpl@6e453dd5
>> >>> ,
>> >>> >>>> CamelRedelivered=false, CamelRedeliveryCounter=0}
>> >>> >>>> BodyType org.apache.camel.examples.SendEmail
>> >>> >>>> Body SendEmail[id: 102 address: null]
>> >>> >>>> ]
>> >>> >>>>
>> >>> >>>> Stacktrace
>> >>> >>>>
>> >>>
>> ---------------------------------------------------------------------------------------------------------------------------------------
>> >>> >>>> org.springframework.dao.InvalidDataAccessApiUsageException: This
>> >>> >>>> operation
>> >>> >>>> cannot be performed while a Transaction is active.; nested
>> >>> exception
>> >>> is
>> >>> >>>>
>> >>> >
>> >>
> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>> >>> >>>> org.apache.openjpa.persistence.InvalidStateException: This
>> >>> operation
>> >>> >>>> cannot
>> >>> >>>> be performed while a Transaction is active.
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:293)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.convertException(ExtendedEntityManagerCreator.java:501)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.afterCommit(ExtendedEntityManagerCreator.java:482)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCommit(TransactionSynchronizationUtils.java:133)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.springframework.transaction.support.TransactionSynchronizationUtils.triggerAfterCommit(TransactionSynchronizationUtils.java:121)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCommit(AbstractPlatformTransactionManager.java:951)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:797)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:148)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.component.jpa.JpaProducer.process(JpaProducer.java:53)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:132)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:307)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:127)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>> >>> >>>> at
>> >>> org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
>> >>> >>>> at
>> >>> org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:56)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.processor.UnitOfWorkProducer.process(UnitOfWorkProducer.java:73)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:378)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:1)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:242)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:346)
>> >>> >>>> at
>> >>> org.apache.camel.impl.ProducerCache.send(ProducerCache.java:184)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:124)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:137)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:144)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.processor.jpa.JpaProducerConcurrentTest$1.call(JpaProducerConcurrentTest.java:58)
>> >>> >>>> at
>> >>> >>>>
>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> >>> >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>> >>> >>>> at java.lang.Thread.run(Thread.java:680)
>> >>> >>>> Caused by:
>> >>> >>>>
>> >>> >
>> >>
> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>> >>> >>>> org.apache.openjpa.persistence.InvalidStateException: This
>> >>> operation
>> >>> >>>> cannot
>> >>> >>>> be performed while a Transaction is active.
>> >>> >>>> at
>> >>> org.apache.openjpa.kernel.BrokerImpl.begin(BrokerImpl.java:1369)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.openjpa.kernel.DelegatingBroker.begin(DelegatingBroker.java:925)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.openjpa.persistence.EntityManagerImpl.begin(EntityManagerImpl.java:565)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.enlistInCurrentTransaction(ExtendedEntityManagerCreator.java:422)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.doJoinTransaction(ExtendedEntityManagerCreator.java:399)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.invoke(ExtendedEntityManagerCreator.java:351)
>> >>> >>>> at com.sun.proxy.$Proxy18.joinTransaction(Unknown Source)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.apache.camel.component.jpa.JpaProducer$1.doInTransaction(JpaProducer.java:55)
>> >>> >>>> at
>> >>> >>>>
>> >>>
>> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:131)
>> >>> >>>> ... 27 more
>> >>> >>>>
>> >>> >>>> which is throwing from the line 55 in
>> >>> >>>> org.apache.camel.component.jpa.JpaProducer
>> >>> >>>> entityManager.joinTransaction();
>> >>> >>>>
>> >>> >>>> I think this is a concurrency issue we have to investigate (not
>> >>> sure
>> >>> >>>> whether the issue is in Camel, Spring, OpenJPA, ...). Will open
>> a
>> >>> JIRA
>> >>> >>>> for
>> >>> >>>> it.
>> >>> >>>>
>> >>> >>>> [1] https://builds.apache.org/job/Camel.trunk.fulltest/1562/
>> >>> >>>>
>> >>> >>>> Best,
>> >>> >>>>
>> >>> >>>> Christian
>> >>> >>>> -----------------
>> >>> >>>>
>> >>> >>>> Software Integration Specialist
>> >>> >>>>
>> >>> >>>> Apache Member
>> >>> >>>> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel
>> >>> committer
>> >>> >>>> Apache Incubator PMC Member
>> >>> >>>>
>> >>> >>>> https://www.linkedin.com/pub/christian-mueller/11/551/642
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> On Thu, Oct 10, 2013 at 12:28 PM, James Carman
>> >>> >>>> <
>> >>> >>
>> >>> >>>> james@
>> >>> >>
>> >>> >>>> >wrote:
>> >>> >>>>
>> >>> >>>>> There are quite a few broken tests. Jetty seems to be hosed.
>> JPA
>> >>> has
>> >>> >>>>> a
>> >>> >>>>> failed test. CDI has issues. It looks like JMS and FTP even
>> >>> decided
>> >>> >>>>> to
>> >>> >>>>> join the party. This has been going on for a while now. This
>> >>> >>>>> particular
>> >>> >>>>> job hasn't been "green" since 9/29. Should we stop working
>> other
>> >>> >>>>> issues
>> >>> >>>>> and get ourselves whole again?
>> >>> >>>>>
>> >>> >>>>> https://builds.apache.org/job/Camel.trunk.fulltest/1561/
>> >>> >>>>>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> --
>> >>> >> View this message in context:
>> >>> >>
>> >>>
>> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741403.html
>> >>> >> Sent from the Camel Development mailing list archive at
>> Nabble.com.
>> >>> >
>> >>> >
>> >>> >
>> >>> > --
>> >>> > Claus Ibsen
>> >>> > -----------------
>> >>> > Red Hat, Inc.
>> >>> > Email:
>> >>>
>> >>> > cibsen@
>> >>>
>> >>> > Twitter: davsclaus
>> >>> > Blog: http://davsclaus.com
>> >>> > Author of Camel in Action: http://www.manning.com/ibsen
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> View this message in context:
>> >>>
>> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741448.html
>> >>> Sent from the Camel Development mailing list archive at Nabble.com.
>> >>>
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741864.html
>> Sent from the Camel Development mailing list archive at Nabble.com.
>>
--
View this message in context: http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741876.html
Sent from the Camel Development mailing list archive at Nabble.com.
Re: Build failures...
Posted by Christian Müller <ch...@gmail.com>.
Really good catch!
Best,
Christian
-----------------
Software Integration Specialist
Apache Member
V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
Apache Incubator PMC Member
https://www.linkedin.com/pub/christian-mueller/11/551/642
On Fri, Oct 18, 2013 at 11:47 PM, Babak Vahdat
<ba...@swissonline.ch>wrote:
> The root cause of the problem for the failing JPA test is CAMEL-5932
> through
> which we started sharing the *same* EntityManager object across
> JpaEndpoint,
> all it's created producers and consumers which is wrong! JPA 2.0 Spec,
> section 7.2:
>
> An entity manager must not be shared among multiple concurrently executing
> threads, as the entity manager and persistence context are not required to
> be threadsafe. Entity managers must only be accessed in a single-threaded
> manner.
>
> That said using OpenJPA you could make use of a proprietary property to
> make
> OpenJPA EntityManager-impl thread safe:
>
> <property name="openjpa.Multithreaded" value="true"/>
>
> But even then, what about Camel users whose JPA provider is Eclipselink or
> Hibernate-EntityManager.
>
> I logged a ticket to get this fixed and hopfully the JPA test being
> mentioned here as well:
>
> https://issues.apache.org/jira/browse/CAMEL-6874
>
> Also there're also some aimless lines of code inside JpaProducer#process()
> method which I will clean up as well.
>
> Babak
>
>
> Babak Vahdat wrote
> > Yeah you're right the test failed again on the CI-Server as well.
> There're
> > also couple of unresolved concurrency/deadlock related JIRA tickets by
> > OpenJPA going into the exactly same direction:
> >
> > https://issues.apache.org/jira/browse/OPENJPA-1716
> >
> > Babak
> > Christian Mueller wrote
> >> It's still failing sometimes on my machine: :-(
> >>
> >> Results :
> >> Failed tests:
> >>
> >>
> JpaProducerConcurrentTest.testConcurrentProducers:45->doSendMessages:66->CamelTestSupport.assertMockEndpointsSatisfied:682
> >> mock://result Received message count. Expected: <10> but was: <8>
> >>
> >> Tests run: 49, Failures: 1, Errors: 0, Skipped: 0
> >>
> >> [INFO]
> >> ------------------------------------------------------------------------
> >> [INFO] BUILD FAILURE
> >> [INFO]
> >> ------------------------------------------------------------------------
> >>
> >>
> >> Best,
> >> Christian
> >> -----------------
> >>
> >> Software Integration Specialist
> >>
> >> Apache Member
> >> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
> >> Apache Incubator PMC Member
> >>
> >> https://www.linkedin.com/pub/christian-mueller/11/551/642
> >>
> >>
> >> On Sat, Oct 12, 2013 at 11:07 PM, Babak Vahdat
> >> <
>
> >> babak.vahdat@
>
> >> >wrote:
> >>
> >>> Yeah indeed a deadlock inside the OpenJPA code is the root cause of the
> >>> problem (org.apache.openjpa.kernel.BrokerImpl#lock()). I just commited
> a
> >>> fix
> >>> regarding this, so let's keep an eye on this test by the next builds.
> >>> Following a screenshot of the proven deadlock through jconsole:
> >>>
> >>> <
> >>>
> http://camel.465427.n5.nabble.com/file/n5741448/Screen_Shot_2013-10-12_at_11.02.58_PM.png
> >>> >
> >>>
> >>> You can enforce the deadlock if you would make use of a high log level
> >>> (log4j.properties), e.g. TRACE instead of just INFO.
> >>>
> >>> Babak
> >>>
> >>>
> >>> Claus Ibsen-2 wrote
> >>> > It could be due an OpenJPA upgrade. These tests used to work better
> in
> >>> the
> >>> > past.
> >>> >
> >>> > On Sat, Oct 12, 2013 at 10:05 AM, Babak Vahdat
> >>> > <
> >>>
> >>> > babak.vahdat@
> >>>
> >>> > > wrote:
> >>> >> The test failed again on the CI-Server. I will look into this in the
> >>> next
> >>> >> days to see if I can find the root cause as on my box I can't
> >>> reproduce
> >>> >> this.
> >>> >>
> >>> >> Babak
> >>> >>
> >>> >>
> >>> >> Babak Vahdat wrote
> >>> >>> Hi
> >>> >>>
> >>> >>> @Christian I believe I've fixed the JPA test now, could you please
> >>> check
> >>> >>> to see if you can still reproduce the same exception you mentioned
> >>> >>> below?
> >>> >>>
> >>> >>> Babak
> >>> >>> Christian Mueller wrote
> >>> >>>> Today, it looks much better [1]. We are trying to fix the unit
> >>> tests
> >>> >>>> which
> >>> >>>> are failing from time to time on slower boxes. But this isn't so
> >>> easy
> >>> >>>> for
> >>> >>>> all of them...
> >>> >>>>
> >>> >>>> I had a look at
> >>> >>>>
> >>>
> org.apache.camel.processor.jpa.JpaProducerConcurrentTest.testConcurrentProducers.
> >>> >>>> A few times I get the following exception which results in missing
> >>> >>>> exchanges in the mock:result endpoint:
> >>> >>>>
> >>> >>>> 2013-10-11 14:01:10,623 [pool-2-thread-2] ERROR
> >>> >>>> DefaultErrorHandler - Failed delivery for (MessageId:
> >>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4 on
> >>> >>>> ExchangeId:
> >>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9).
> >>> Exhausted
> >>> >>>> after delivery attempt: 1 caught:
> >>> >>>> org.springframework.dao.InvalidDataAccessApiUsageException: This
> >>> >>>> operation
> >>> >>>> cannot be performed while a Transaction is active.; nested
> >>> exception
> >>> is
> >>> >>>>
> >>> >
> >> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
> >>> >>>> org.apache.openjpa.persistence.InvalidStateException: This
> >>> operation
> >>> >>>> cannot
> >>> >>>> be performed while a Transaction is active.
> >>> >>>>
> >>> >>>> Message History
> >>> >>>>
> >>>
> ---------------------------------------------------------------------------------------------------------------------------------------
> >>> >>>> RouteId ProcessorId
> >>> >>>> Processor
> >>> >>>> Elapsed (ms)
> >>> >>>> [route1 ] [route1 ]
> >>> >>>> [direct://start
> >>> >>>> ] [ 68]
> >>> >>>> [route1 ] [to1 ]
> >>> >>>> [jpa://org.apache.camel.examples.SendEmail
> >>> >>>> ] [ 67]
> >>> >>>>
> >>> >>>> Exchange
> >>> >>>>
> >>>
> ---------------------------------------------------------------------------------------------------------------------------------------
> >>> >>>> Exchange[
> >>> >>>> Id
> >>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9
> >>> >>>> ExchangePattern InOnly
> >>> >>>> Headers
> >>> >>>>
> >>>
> {breadcrumbId=ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4,
> >>> >>>>
> >>>
> CamelEntityManager=org.apache.openjpa.persistence.EntityManagerImpl@6e453dd5
> >>> ,
> >>> >>>> CamelRedelivered=false, CamelRedeliveryCounter=0}
> >>> >>>> BodyType org.apache.camel.examples.SendEmail
> >>> >>>> Body SendEmail[id: 102 address: null]
> >>> >>>> ]
> >>> >>>>
> >>> >>>> Stacktrace
> >>> >>>>
> >>>
> ---------------------------------------------------------------------------------------------------------------------------------------
> >>> >>>> org.springframework.dao.InvalidDataAccessApiUsageException: This
> >>> >>>> operation
> >>> >>>> cannot be performed while a Transaction is active.; nested
> >>> exception
> >>> is
> >>> >>>>
> >>> >
> >> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
> >>> >>>> org.apache.openjpa.persistence.InvalidStateException: This
> >>> operation
> >>> >>>> cannot
> >>> >>>> be performed while a Transaction is active.
> >>> >>>> at
> >>> >>>>
> >>>
> org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:293)
> >>> >>>> at
> >>> >>>>
> >>>
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.convertException(ExtendedEntityManagerCreator.java:501)
> >>> >>>> at
> >>> >>>>
> >>>
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.afterCommit(ExtendedEntityManagerCreator.java:482)
> >>> >>>> at
> >>> >>>>
> >>>
> org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCommit(TransactionSynchronizationUtils.java:133)
> >>> >>>> at
> >>> >>>>
> >>>
> org.springframework.transaction.support.TransactionSynchronizationUtils.triggerAfterCommit(TransactionSynchronizationUtils.java:121)
> >>> >>>> at
> >>> >>>>
> >>>
> org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCommit(AbstractPlatformTransactionManager.java:951)
> >>> >>>> at
> >>> >>>>
> >>>
> org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:797)
> >>> >>>> at
> >>> >>>>
> >>>
> org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724)
> >>> >>>> at
> >>> >>>>
> >>>
> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:148)
> >>> >>>> at
> >>> >>>>
> >>> org.apache.camel.component.jpa.JpaProducer.process(JpaProducer.java:53)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:132)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:307)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:127)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
> >>> >>>> at
> >>> org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
> >>> >>>> at
> >>> org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:56)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.camel.processor.UnitOfWorkProducer.process(UnitOfWorkProducer.java:73)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:378)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:1)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:242)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:346)
> >>> >>>> at
> >>> org.apache.camel.impl.ProducerCache.send(ProducerCache.java:184)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:124)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:137)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:144)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.camel.processor.jpa.JpaProducerConcurrentTest$1.call(JpaProducerConcurrentTest.java:58)
> >>> >>>> at
> >>> >>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> >>> >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> >>> >>>> at
> >>> >>>>
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> >>> >>>> at
> >>> >>>>
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> >>> >>>> at java.lang.Thread.run(Thread.java:680)
> >>> >>>> Caused by:
> >>> >>>>
> >>> >
> >> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
> >>> >>>> org.apache.openjpa.persistence.InvalidStateException: This
> >>> operation
> >>> >>>> cannot
> >>> >>>> be performed while a Transaction is active.
> >>> >>>> at
> >>> org.apache.openjpa.kernel.BrokerImpl.begin(BrokerImpl.java:1369)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.openjpa.kernel.DelegatingBroker.begin(DelegatingBroker.java:925)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.openjpa.persistence.EntityManagerImpl.begin(EntityManagerImpl.java:565)
> >>> >>>> at
> >>> >>>>
> >>>
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.enlistInCurrentTransaction(ExtendedEntityManagerCreator.java:422)
> >>> >>>> at
> >>> >>>>
> >>>
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.doJoinTransaction(ExtendedEntityManagerCreator.java:399)
> >>> >>>> at
> >>> >>>>
> >>>
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.invoke(ExtendedEntityManagerCreator.java:351)
> >>> >>>> at com.sun.proxy.$Proxy18.joinTransaction(Unknown Source)
> >>> >>>> at
> >>> >>>>
> >>>
> org.apache.camel.component.jpa.JpaProducer$1.doInTransaction(JpaProducer.java:55)
> >>> >>>> at
> >>> >>>>
> >>>
> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:131)
> >>> >>>> ... 27 more
> >>> >>>>
> >>> >>>> which is throwing from the line 55 in
> >>> >>>> org.apache.camel.component.jpa.JpaProducer
> >>> >>>> entityManager.joinTransaction();
> >>> >>>>
> >>> >>>> I think this is a concurrency issue we have to investigate (not
> >>> sure
> >>> >>>> whether the issue is in Camel, Spring, OpenJPA, ...). Will open a
> >>> JIRA
> >>> >>>> for
> >>> >>>> it.
> >>> >>>>
> >>> >>>> [1] https://builds.apache.org/job/Camel.trunk.fulltest/1562/
> >>> >>>>
> >>> >>>> Best,
> >>> >>>>
> >>> >>>> Christian
> >>> >>>> -----------------
> >>> >>>>
> >>> >>>> Software Integration Specialist
> >>> >>>>
> >>> >>>> Apache Member
> >>> >>>> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel
> >>> committer
> >>> >>>> Apache Incubator PMC Member
> >>> >>>>
> >>> >>>> https://www.linkedin.com/pub/christian-mueller/11/551/642
> >>> >>>>
> >>> >>>>
> >>> >>>> On Thu, Oct 10, 2013 at 12:28 PM, James Carman
> >>> >>>> <
> >>> >>
> >>> >>>> james@
> >>> >>
> >>> >>>> >wrote:
> >>> >>>>
> >>> >>>>> There are quite a few broken tests. Jetty seems to be hosed.
> JPA
> >>> has
> >>> >>>>> a
> >>> >>>>> failed test. CDI has issues. It looks like JMS and FTP even
> >>> decided
> >>> >>>>> to
> >>> >>>>> join the party. This has been going on for a while now. This
> >>> >>>>> particular
> >>> >>>>> job hasn't been "green" since 9/29. Should we stop working other
> >>> >>>>> issues
> >>> >>>>> and get ourselves whole again?
> >>> >>>>>
> >>> >>>>> https://builds.apache.org/job/Camel.trunk.fulltest/1561/
> >>> >>>>>
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >> --
> >>> >> View this message in context:
> >>> >>
> >>>
> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741403.html
> >>> >> Sent from the Camel Development mailing list archive at Nabble.com.
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Claus Ibsen
> >>> > -----------------
> >>> > Red Hat, Inc.
> >>> > Email:
> >>>
> >>> > cibsen@
> >>>
> >>> > Twitter: davsclaus
> >>> > Blog: http://davsclaus.com
> >>> > Author of Camel in Action: http://www.manning.com/ibsen
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> View this message in context:
> >>>
> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741448.html
> >>> Sent from the Camel Development mailing list archive at Nabble.com.
> >>>
>
>
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741864.html
> Sent from the Camel Development mailing list archive at Nabble.com.
>
Re: Build failures...
Posted by Babak Vahdat <ba...@swissonline.ch>.
The root cause of the problem for the failing JPA test is CAMEL-5932 through
which we started sharing the *same* EntityManager object across JpaEndpoint,
all it's created producers and consumers which is wrong! JPA 2.0 Spec,
section 7.2:
An entity manager must not be shared among multiple concurrently executing
threads, as the entity manager and persistence context are not required to
be threadsafe. Entity managers must only be accessed in a single-threaded
manner.
That said using OpenJPA you could make use of a proprietary property to make
OpenJPA EntityManager-impl thread safe:
<property name="openjpa.Multithreaded" value="true"/>
But even then, what about Camel users whose JPA provider is Eclipselink or
Hibernate-EntityManager.
I logged a ticket to get this fixed and hopfully the JPA test being
mentioned here as well:
https://issues.apache.org/jira/browse/CAMEL-6874
Also there're also some aimless lines of code inside JpaProducer#process()
method which I will clean up as well.
Babak
Babak Vahdat wrote
> Yeah you're right the test failed again on the CI-Server as well. There're
> also couple of unresolved concurrency/deadlock related JIRA tickets by
> OpenJPA going into the exactly same direction:
>
> https://issues.apache.org/jira/browse/OPENJPA-1716
>
> Babak
> Christian Mueller wrote
>> It's still failing sometimes on my machine: :-(
>>
>> Results :
>> Failed tests:
>>
>> JpaProducerConcurrentTest.testConcurrentProducers:45->doSendMessages:66->CamelTestSupport.assertMockEndpointsSatisfied:682
>> mock://result Received message count. Expected: <10> but was: <8>
>>
>> Tests run: 49, Failures: 1, Errors: 0, Skipped: 0
>>
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] BUILD FAILURE
>> [INFO]
>> ------------------------------------------------------------------------
>>
>>
>> Best,
>> Christian
>> -----------------
>>
>> Software Integration Specialist
>>
>> Apache Member
>> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
>> Apache Incubator PMC Member
>>
>> https://www.linkedin.com/pub/christian-mueller/11/551/642
>>
>>
>> On Sat, Oct 12, 2013 at 11:07 PM, Babak Vahdat
>> <
>> babak.vahdat@
>> >wrote:
>>
>>> Yeah indeed a deadlock inside the OpenJPA code is the root cause of the
>>> problem (org.apache.openjpa.kernel.BrokerImpl#lock()). I just commited a
>>> fix
>>> regarding this, so let's keep an eye on this test by the next builds.
>>> Following a screenshot of the proven deadlock through jconsole:
>>>
>>> <
>>> http://camel.465427.n5.nabble.com/file/n5741448/Screen_Shot_2013-10-12_at_11.02.58_PM.png
>>> >
>>>
>>> You can enforce the deadlock if you would make use of a high log level
>>> (log4j.properties), e.g. TRACE instead of just INFO.
>>>
>>> Babak
>>>
>>>
>>> Claus Ibsen-2 wrote
>>> > It could be due an OpenJPA upgrade. These tests used to work better in
>>> the
>>> > past.
>>> >
>>> > On Sat, Oct 12, 2013 at 10:05 AM, Babak Vahdat
>>> > <
>>>
>>> > babak.vahdat@
>>>
>>> > > wrote:
>>> >> The test failed again on the CI-Server. I will look into this in the
>>> next
>>> >> days to see if I can find the root cause as on my box I can't
>>> reproduce
>>> >> this.
>>> >>
>>> >> Babak
>>> >>
>>> >>
>>> >> Babak Vahdat wrote
>>> >>> Hi
>>> >>>
>>> >>> @Christian I believe I've fixed the JPA test now, could you please
>>> check
>>> >>> to see if you can still reproduce the same exception you mentioned
>>> >>> below?
>>> >>>
>>> >>> Babak
>>> >>> Christian Mueller wrote
>>> >>>> Today, it looks much better [1]. We are trying to fix the unit
>>> tests
>>> >>>> which
>>> >>>> are failing from time to time on slower boxes. But this isn't so
>>> easy
>>> >>>> for
>>> >>>> all of them...
>>> >>>>
>>> >>>> I had a look at
>>> >>>>
>>> org.apache.camel.processor.jpa.JpaProducerConcurrentTest.testConcurrentProducers.
>>> >>>> A few times I get the following exception which results in missing
>>> >>>> exchanges in the mock:result endpoint:
>>> >>>>
>>> >>>> 2013-10-11 14:01:10,623 [pool-2-thread-2] ERROR
>>> >>>> DefaultErrorHandler - Failed delivery for (MessageId:
>>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4 on
>>> >>>> ExchangeId:
>>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9).
>>> Exhausted
>>> >>>> after delivery attempt: 1 caught:
>>> >>>> org.springframework.dao.InvalidDataAccessApiUsageException: This
>>> >>>> operation
>>> >>>> cannot be performed while a Transaction is active.; nested
>>> exception
>>> is
>>> >>>>
>>> >
>> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>>> >>>> org.apache.openjpa.persistence.InvalidStateException: This
>>> operation
>>> >>>> cannot
>>> >>>> be performed while a Transaction is active.
>>> >>>>
>>> >>>> Message History
>>> >>>>
>>> ---------------------------------------------------------------------------------------------------------------------------------------
>>> >>>> RouteId ProcessorId
>>> >>>> Processor
>>> >>>> Elapsed (ms)
>>> >>>> [route1 ] [route1 ]
>>> >>>> [direct://start
>>> >>>> ] [ 68]
>>> >>>> [route1 ] [to1 ]
>>> >>>> [jpa://org.apache.camel.examples.SendEmail
>>> >>>> ] [ 67]
>>> >>>>
>>> >>>> Exchange
>>> >>>>
>>> ---------------------------------------------------------------------------------------------------------------------------------------
>>> >>>> Exchange[
>>> >>>> Id
>>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9
>>> >>>> ExchangePattern InOnly
>>> >>>> Headers
>>> >>>>
>>> {breadcrumbId=ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4,
>>> >>>>
>>> CamelEntityManager=org.apache.openjpa.persistence.EntityManagerImpl@6e453dd5
>>> ,
>>> >>>> CamelRedelivered=false, CamelRedeliveryCounter=0}
>>> >>>> BodyType org.apache.camel.examples.SendEmail
>>> >>>> Body SendEmail[id: 102 address: null]
>>> >>>> ]
>>> >>>>
>>> >>>> Stacktrace
>>> >>>>
>>> ---------------------------------------------------------------------------------------------------------------------------------------
>>> >>>> org.springframework.dao.InvalidDataAccessApiUsageException: This
>>> >>>> operation
>>> >>>> cannot be performed while a Transaction is active.; nested
>>> exception
>>> is
>>> >>>>
>>> >
>> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>>> >>>> org.apache.openjpa.persistence.InvalidStateException: This
>>> operation
>>> >>>> cannot
>>> >>>> be performed while a Transaction is active.
>>> >>>> at
>>> >>>>
>>> org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:293)
>>> >>>> at
>>> >>>>
>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.convertException(ExtendedEntityManagerCreator.java:501)
>>> >>>> at
>>> >>>>
>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.afterCommit(ExtendedEntityManagerCreator.java:482)
>>> >>>> at
>>> >>>>
>>> org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCommit(TransactionSynchronizationUtils.java:133)
>>> >>>> at
>>> >>>>
>>> org.springframework.transaction.support.TransactionSynchronizationUtils.triggerAfterCommit(TransactionSynchronizationUtils.java:121)
>>> >>>> at
>>> >>>>
>>> org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCommit(AbstractPlatformTransactionManager.java:951)
>>> >>>> at
>>> >>>>
>>> org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:797)
>>> >>>> at
>>> >>>>
>>> org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724)
>>> >>>> at
>>> >>>>
>>> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:148)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.component.jpa.JpaProducer.process(JpaProducer.java:53)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:132)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:307)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:127)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>>> >>>> at
>>> org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
>>> >>>> at
>>> org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:56)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.processor.UnitOfWorkProducer.process(UnitOfWorkProducer.java:73)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:378)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:1)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:242)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:346)
>>> >>>> at
>>> org.apache.camel.impl.ProducerCache.send(ProducerCache.java:184)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:124)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:137)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:144)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.processor.jpa.JpaProducerConcurrentTest$1.call(JpaProducerConcurrentTest.java:58)
>>> >>>> at
>>> >>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>> >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>> >>>> at
>>> >>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>> >>>> at
>>> >>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>> >>>> at java.lang.Thread.run(Thread.java:680)
>>> >>>> Caused by:
>>> >>>>
>>> >
>> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>>> >>>> org.apache.openjpa.persistence.InvalidStateException: This
>>> operation
>>> >>>> cannot
>>> >>>> be performed while a Transaction is active.
>>> >>>> at
>>> org.apache.openjpa.kernel.BrokerImpl.begin(BrokerImpl.java:1369)
>>> >>>> at
>>> >>>>
>>> org.apache.openjpa.kernel.DelegatingBroker.begin(DelegatingBroker.java:925)
>>> >>>> at
>>> >>>>
>>> org.apache.openjpa.persistence.EntityManagerImpl.begin(EntityManagerImpl.java:565)
>>> >>>> at
>>> >>>>
>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.enlistInCurrentTransaction(ExtendedEntityManagerCreator.java:422)
>>> >>>> at
>>> >>>>
>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.doJoinTransaction(ExtendedEntityManagerCreator.java:399)
>>> >>>> at
>>> >>>>
>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.invoke(ExtendedEntityManagerCreator.java:351)
>>> >>>> at com.sun.proxy.$Proxy18.joinTransaction(Unknown Source)
>>> >>>> at
>>> >>>>
>>> org.apache.camel.component.jpa.JpaProducer$1.doInTransaction(JpaProducer.java:55)
>>> >>>> at
>>> >>>>
>>> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:131)
>>> >>>> ... 27 more
>>> >>>>
>>> >>>> which is throwing from the line 55 in
>>> >>>> org.apache.camel.component.jpa.JpaProducer
>>> >>>> entityManager.joinTransaction();
>>> >>>>
>>> >>>> I think this is a concurrency issue we have to investigate (not
>>> sure
>>> >>>> whether the issue is in Camel, Spring, OpenJPA, ...). Will open a
>>> JIRA
>>> >>>> for
>>> >>>> it.
>>> >>>>
>>> >>>> [1] https://builds.apache.org/job/Camel.trunk.fulltest/1562/
>>> >>>>
>>> >>>> Best,
>>> >>>>
>>> >>>> Christian
>>> >>>> -----------------
>>> >>>>
>>> >>>> Software Integration Specialist
>>> >>>>
>>> >>>> Apache Member
>>> >>>> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel
>>> committer
>>> >>>> Apache Incubator PMC Member
>>> >>>>
>>> >>>> https://www.linkedin.com/pub/christian-mueller/11/551/642
>>> >>>>
>>> >>>>
>>> >>>> On Thu, Oct 10, 2013 at 12:28 PM, James Carman
>>> >>>> <
>>> >>
>>> >>>> james@
>>> >>
>>> >>>> >wrote:
>>> >>>>
>>> >>>>> There are quite a few broken tests. Jetty seems to be hosed. JPA
>>> has
>>> >>>>> a
>>> >>>>> failed test. CDI has issues. It looks like JMS and FTP even
>>> decided
>>> >>>>> to
>>> >>>>> join the party. This has been going on for a while now. This
>>> >>>>> particular
>>> >>>>> job hasn't been "green" since 9/29. Should we stop working other
>>> >>>>> issues
>>> >>>>> and get ourselves whole again?
>>> >>>>>
>>> >>>>> https://builds.apache.org/job/Camel.trunk.fulltest/1561/
>>> >>>>>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> View this message in context:
>>> >>
>>> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741403.html
>>> >> Sent from the Camel Development mailing list archive at Nabble.com.
>>> >
>>> >
>>> >
>>> > --
>>> > Claus Ibsen
>>> > -----------------
>>> > Red Hat, Inc.
>>> > Email:
>>>
>>> > cibsen@
>>>
>>> > Twitter: davsclaus
>>> > Blog: http://davsclaus.com
>>> > Author of Camel in Action: http://www.manning.com/ibsen
>>>
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741448.html
>>> Sent from the Camel Development mailing list archive at Nabble.com.
>>>
--
View this message in context: http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741864.html
Sent from the Camel Development mailing list archive at Nabble.com.
Re: Build failures...
Posted by Babak Vahdat <ba...@swissonline.ch>.
Yeah you're right the test failed again on the CI-Server as well. There're
also couple of unresolved concurrency/deadlock related JIRA tickets by
OpenJPA going into the exactly same direction:
https://issues.apache.org/jira/browse/OPENJPA-1716
Babak
Christian Mueller wrote
> It's still failing sometimes on my machine: :-(
>
> Results :
> Failed tests:
>
> JpaProducerConcurrentTest.testConcurrentProducers:45->doSendMessages:66->CamelTestSupport.assertMockEndpointsSatisfied:682
> mock://result Received message count. Expected: <10> but was: <8>
>
> Tests run: 49, Failures: 1, Errors: 0, Skipped: 0
>
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
>
>
> Best,
> Christian
> -----------------
>
> Software Integration Specialist
>
> Apache Member
> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
> Apache Incubator PMC Member
>
> https://www.linkedin.com/pub/christian-mueller/11/551/642
>
>
> On Sat, Oct 12, 2013 at 11:07 PM, Babak Vahdat
> <
> babak.vahdat@
> >wrote:
>
>> Yeah indeed a deadlock inside the OpenJPA code is the root cause of the
>> problem (org.apache.openjpa.kernel.BrokerImpl#lock()). I just commited a
>> fix
>> regarding this, so let's keep an eye on this test by the next builds.
>> Following a screenshot of the proven deadlock through jconsole:
>>
>> <
>> http://camel.465427.n5.nabble.com/file/n5741448/Screen_Shot_2013-10-12_at_11.02.58_PM.png
>> >
>>
>> You can enforce the deadlock if you would make use of a high log level
>> (log4j.properties), e.g. TRACE instead of just INFO.
>>
>> Babak
>>
>>
>> Claus Ibsen-2 wrote
>> > It could be due an OpenJPA upgrade. These tests used to work better in
>> the
>> > past.
>> >
>> > On Sat, Oct 12, 2013 at 10:05 AM, Babak Vahdat
>> > <
>>
>> > babak.vahdat@
>>
>> > > wrote:
>> >> The test failed again on the CI-Server. I will look into this in the
>> next
>> >> days to see if I can find the root cause as on my box I can't
>> reproduce
>> >> this.
>> >>
>> >> Babak
>> >>
>> >>
>> >> Babak Vahdat wrote
>> >>> Hi
>> >>>
>> >>> @Christian I believe I've fixed the JPA test now, could you please
>> check
>> >>> to see if you can still reproduce the same exception you mentioned
>> >>> below?
>> >>>
>> >>> Babak
>> >>> Christian Mueller wrote
>> >>>> Today, it looks much better [1]. We are trying to fix the unit tests
>> >>>> which
>> >>>> are failing from time to time on slower boxes. But this isn't so
>> easy
>> >>>> for
>> >>>> all of them...
>> >>>>
>> >>>> I had a look at
>> >>>>
>> org.apache.camel.processor.jpa.JpaProducerConcurrentTest.testConcurrentProducers.
>> >>>> A few times I get the following exception which results in missing
>> >>>> exchanges in the mock:result endpoint:
>> >>>>
>> >>>> 2013-10-11 14:01:10,623 [pool-2-thread-2] ERROR
>> >>>> DefaultErrorHandler - Failed delivery for (MessageId:
>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4 on
>> >>>> ExchangeId:
>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9).
>> Exhausted
>> >>>> after delivery attempt: 1 caught:
>> >>>> org.springframework.dao.InvalidDataAccessApiUsageException: This
>> >>>> operation
>> >>>> cannot be performed while a Transaction is active.; nested exception
>> is
>> >>>>
>> >
> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>> >>>> org.apache.openjpa.persistence.InvalidStateException: This operation
>> >>>> cannot
>> >>>> be performed while a Transaction is active.
>> >>>>
>> >>>> Message History
>> >>>>
>> ---------------------------------------------------------------------------------------------------------------------------------------
>> >>>> RouteId ProcessorId
>> >>>> Processor
>> >>>> Elapsed (ms)
>> >>>> [route1 ] [route1 ]
>> >>>> [direct://start
>> >>>> ] [ 68]
>> >>>> [route1 ] [to1 ]
>> >>>> [jpa://org.apache.camel.examples.SendEmail
>> >>>> ] [ 67]
>> >>>>
>> >>>> Exchange
>> >>>>
>> ---------------------------------------------------------------------------------------------------------------------------------------
>> >>>> Exchange[
>> >>>> Id
>> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9
>> >>>> ExchangePattern InOnly
>> >>>> Headers
>> >>>>
>> {breadcrumbId=ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4,
>> >>>>
>> CamelEntityManager=org.apache.openjpa.persistence.EntityManagerImpl@6e453dd5
>> ,
>> >>>> CamelRedelivered=false, CamelRedeliveryCounter=0}
>> >>>> BodyType org.apache.camel.examples.SendEmail
>> >>>> Body SendEmail[id: 102 address: null]
>> >>>> ]
>> >>>>
>> >>>> Stacktrace
>> >>>>
>> ---------------------------------------------------------------------------------------------------------------------------------------
>> >>>> org.springframework.dao.InvalidDataAccessApiUsageException: This
>> >>>> operation
>> >>>> cannot be performed while a Transaction is active.; nested exception
>> is
>> >>>>
>> >
> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>> >>>> org.apache.openjpa.persistence.InvalidStateException: This operation
>> >>>> cannot
>> >>>> be performed while a Transaction is active.
>> >>>> at
>> >>>>
>> org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:293)
>> >>>> at
>> >>>>
>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.convertException(ExtendedEntityManagerCreator.java:501)
>> >>>> at
>> >>>>
>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.afterCommit(ExtendedEntityManagerCreator.java:482)
>> >>>> at
>> >>>>
>> org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCommit(TransactionSynchronizationUtils.java:133)
>> >>>> at
>> >>>>
>> org.springframework.transaction.support.TransactionSynchronizationUtils.triggerAfterCommit(TransactionSynchronizationUtils.java:121)
>> >>>> at
>> >>>>
>> org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCommit(AbstractPlatformTransactionManager.java:951)
>> >>>> at
>> >>>>
>> org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:797)
>> >>>> at
>> >>>>
>> org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724)
>> >>>> at
>> >>>>
>> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:148)
>> >>>> at
>> >>>>
>> org.apache.camel.component.jpa.JpaProducer.process(JpaProducer.java:53)
>> >>>> at
>> >>>>
>> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
>> >>>> at
>> >>>>
>> org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:132)
>> >>>> at
>> >>>>
>> org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:307)
>> >>>> at
>> >>>>
>> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:127)
>> >>>> at
>> >>>>
>> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)
>> >>>> at
>> >>>>
>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>> >>>> at
>> org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
>> >>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
>> >>>> at
>> >>>>
>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>> >>>> at
>> >>>>
>> org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:56)
>> >>>> at
>> >>>>
>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>> >>>> at
>> >>>>
>> org.apache.camel.processor.UnitOfWorkProducer.process(UnitOfWorkProducer.java:73)
>> >>>> at
>> >>>>
>> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:378)
>> >>>> at
>> >>>>
>> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:1)
>> >>>> at
>> >>>>
>> org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:242)
>> >>>> at
>> >>>>
>> org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:346)
>> >>>> at
>> org.apache.camel.impl.ProducerCache.send(ProducerCache.java:184)
>> >>>> at
>> >>>>
>> org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:124)
>> >>>> at
>> >>>>
>> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:137)
>> >>>> at
>> >>>>
>> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:144)
>> >>>> at
>> >>>>
>> org.apache.camel.processor.jpa.JpaProducerConcurrentTest$1.call(JpaProducerConcurrentTest.java:58)
>> >>>> at
>> >>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> >>>> at
>> >>>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>> >>>> at
>> >>>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>> >>>> at java.lang.Thread.run(Thread.java:680)
>> >>>> Caused by:
>> >>>>
>> >
> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>> >>>> org.apache.openjpa.persistence.InvalidStateException: This operation
>> >>>> cannot
>> >>>> be performed while a Transaction is active.
>> >>>> at
>> org.apache.openjpa.kernel.BrokerImpl.begin(BrokerImpl.java:1369)
>> >>>> at
>> >>>>
>> org.apache.openjpa.kernel.DelegatingBroker.begin(DelegatingBroker.java:925)
>> >>>> at
>> >>>>
>> org.apache.openjpa.persistence.EntityManagerImpl.begin(EntityManagerImpl.java:565)
>> >>>> at
>> >>>>
>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.enlistInCurrentTransaction(ExtendedEntityManagerCreator.java:422)
>> >>>> at
>> >>>>
>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.doJoinTransaction(ExtendedEntityManagerCreator.java:399)
>> >>>> at
>> >>>>
>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.invoke(ExtendedEntityManagerCreator.java:351)
>> >>>> at com.sun.proxy.$Proxy18.joinTransaction(Unknown Source)
>> >>>> at
>> >>>>
>> org.apache.camel.component.jpa.JpaProducer$1.doInTransaction(JpaProducer.java:55)
>> >>>> at
>> >>>>
>> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:131)
>> >>>> ... 27 more
>> >>>>
>> >>>> which is throwing from the line 55 in
>> >>>> org.apache.camel.component.jpa.JpaProducer
>> >>>> entityManager.joinTransaction();
>> >>>>
>> >>>> I think this is a concurrency issue we have to investigate (not sure
>> >>>> whether the issue is in Camel, Spring, OpenJPA, ...). Will open a
>> JIRA
>> >>>> for
>> >>>> it.
>> >>>>
>> >>>> [1] https://builds.apache.org/job/Camel.trunk.fulltest/1562/
>> >>>>
>> >>>> Best,
>> >>>>
>> >>>> Christian
>> >>>> -----------------
>> >>>>
>> >>>> Software Integration Specialist
>> >>>>
>> >>>> Apache Member
>> >>>> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
>> >>>> Apache Incubator PMC Member
>> >>>>
>> >>>> https://www.linkedin.com/pub/christian-mueller/11/551/642
>> >>>>
>> >>>>
>> >>>> On Thu, Oct 10, 2013 at 12:28 PM, James Carman
>> >>>> <
>> >>
>> >>>> james@
>> >>
>> >>>> >wrote:
>> >>>>
>> >>>>> There are quite a few broken tests. Jetty seems to be hosed. JPA
>> has
>> >>>>> a
>> >>>>> failed test. CDI has issues. It looks like JMS and FTP even
>> decided
>> >>>>> to
>> >>>>> join the party. This has been going on for a while now. This
>> >>>>> particular
>> >>>>> job hasn't been "green" since 9/29. Should we stop working other
>> >>>>> issues
>> >>>>> and get ourselves whole again?
>> >>>>>
>> >>>>> https://builds.apache.org/job/Camel.trunk.fulltest/1561/
>> >>>>>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> View this message in context:
>> >>
>> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741403.html
>> >> Sent from the Camel Development mailing list archive at Nabble.com.
>> >
>> >
>> >
>> > --
>> > Claus Ibsen
>> > -----------------
>> > Red Hat, Inc.
>> > Email:
>>
>> > cibsen@
>>
>> > Twitter: davsclaus
>> > Blog: http://davsclaus.com
>> > Author of Camel in Action: http://www.manning.com/ibsen
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741448.html
>> Sent from the Camel Development mailing list archive at Nabble.com.
>>
--
View this message in context: http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741510.html
Sent from the Camel Development mailing list archive at Nabble.com.
Re: Build failures...
Posted by Christian Müller <ch...@gmail.com>.
It's still failing sometimes on my machine: :-(
Results :
Failed tests:
JpaProducerConcurrentTest.testConcurrentProducers:45->doSendMessages:66->CamelTestSupport.assertMockEndpointsSatisfied:682
mock://result Received message count. Expected: <10> but was: <8>
Tests run: 49, Failures: 1, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO]
------------------------------------------------------------------------
Best,
Christian
-----------------
Software Integration Specialist
Apache Member
V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
Apache Incubator PMC Member
https://www.linkedin.com/pub/christian-mueller/11/551/642
On Sat, Oct 12, 2013 at 11:07 PM, Babak Vahdat
<ba...@swissonline.ch>wrote:
> Yeah indeed a deadlock inside the OpenJPA code is the root cause of the
> problem (org.apache.openjpa.kernel.BrokerImpl#lock()). I just commited a
> fix
> regarding this, so let's keep an eye on this test by the next builds.
> Following a screenshot of the proven deadlock through jconsole:
>
> <
> http://camel.465427.n5.nabble.com/file/n5741448/Screen_Shot_2013-10-12_at_11.02.58_PM.png
> >
>
> You can enforce the deadlock if you would make use of a high log level
> (log4j.properties), e.g. TRACE instead of just INFO.
>
> Babak
>
>
> Claus Ibsen-2 wrote
> > It could be due an OpenJPA upgrade. These tests used to work better in
> the
> > past.
> >
> > On Sat, Oct 12, 2013 at 10:05 AM, Babak Vahdat
> > <
>
> > babak.vahdat@
>
> > > wrote:
> >> The test failed again on the CI-Server. I will look into this in the
> next
> >> days to see if I can find the root cause as on my box I can't reproduce
> >> this.
> >>
> >> Babak
> >>
> >>
> >> Babak Vahdat wrote
> >>> Hi
> >>>
> >>> @Christian I believe I've fixed the JPA test now, could you please
> check
> >>> to see if you can still reproduce the same exception you mentioned
> >>> below?
> >>>
> >>> Babak
> >>> Christian Mueller wrote
> >>>> Today, it looks much better [1]. We are trying to fix the unit tests
> >>>> which
> >>>> are failing from time to time on slower boxes. But this isn't so easy
> >>>> for
> >>>> all of them...
> >>>>
> >>>> I had a look at
> >>>>
> org.apache.camel.processor.jpa.JpaProducerConcurrentTest.testConcurrentProducers.
> >>>> A few times I get the following exception which results in missing
> >>>> exchanges in the mock:result endpoint:
> >>>>
> >>>> 2013-10-11 14:01:10,623 [pool-2-thread-2] ERROR
> >>>> DefaultErrorHandler - Failed delivery for (MessageId:
> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4 on
> >>>> ExchangeId:
> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9).
> Exhausted
> >>>> after delivery attempt: 1 caught:
> >>>> org.springframework.dao.InvalidDataAccessApiUsageException: This
> >>>> operation
> >>>> cannot be performed while a Transaction is active.; nested exception
> is
> >>>>
> > <openjpa-2.2.2-r422266:1468616 nonfatal user error>
> >>>> org.apache.openjpa.persistence.InvalidStateException: This operation
> >>>> cannot
> >>>> be performed while a Transaction is active.
> >>>>
> >>>> Message History
> >>>>
> ---------------------------------------------------------------------------------------------------------------------------------------
> >>>> RouteId ProcessorId
> >>>> Processor
> >>>> Elapsed (ms)
> >>>> [route1 ] [route1 ]
> >>>> [direct://start
> >>>> ] [ 68]
> >>>> [route1 ] [to1 ]
> >>>> [jpa://org.apache.camel.examples.SendEmail
> >>>> ] [ 67]
> >>>>
> >>>> Exchange
> >>>>
> ---------------------------------------------------------------------------------------------------------------------------------------
> >>>> Exchange[
> >>>> Id
> >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9
> >>>> ExchangePattern InOnly
> >>>> Headers
> >>>>
> {breadcrumbId=ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4,
> >>>>
> CamelEntityManager=org.apache.openjpa.persistence.EntityManagerImpl@6e453dd5
> ,
> >>>> CamelRedelivered=false, CamelRedeliveryCounter=0}
> >>>> BodyType org.apache.camel.examples.SendEmail
> >>>> Body SendEmail[id: 102 address: null]
> >>>> ]
> >>>>
> >>>> Stacktrace
> >>>>
> ---------------------------------------------------------------------------------------------------------------------------------------
> >>>> org.springframework.dao.InvalidDataAccessApiUsageException: This
> >>>> operation
> >>>> cannot be performed while a Transaction is active.; nested exception
> is
> >>>>
> > <openjpa-2.2.2-r422266:1468616 nonfatal user error>
> >>>> org.apache.openjpa.persistence.InvalidStateException: This operation
> >>>> cannot
> >>>> be performed while a Transaction is active.
> >>>> at
> >>>>
> org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:293)
> >>>> at
> >>>>
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.convertException(ExtendedEntityManagerCreator.java:501)
> >>>> at
> >>>>
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.afterCommit(ExtendedEntityManagerCreator.java:482)
> >>>> at
> >>>>
> org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCommit(TransactionSynchronizationUtils.java:133)
> >>>> at
> >>>>
> org.springframework.transaction.support.TransactionSynchronizationUtils.triggerAfterCommit(TransactionSynchronizationUtils.java:121)
> >>>> at
> >>>>
> org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCommit(AbstractPlatformTransactionManager.java:951)
> >>>> at
> >>>>
> org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:797)
> >>>> at
> >>>>
> org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724)
> >>>> at
> >>>>
> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:148)
> >>>> at
> >>>>
> org.apache.camel.component.jpa.JpaProducer.process(JpaProducer.java:53)
> >>>> at
> >>>>
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> >>>> at
> >>>>
> org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:132)
> >>>> at
> >>>>
> org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:307)
> >>>> at
> >>>>
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:127)
> >>>> at
> >>>>
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)
> >>>> at
> >>>>
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
> >>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
> >>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
> >>>> at
> >>>>
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
> >>>> at
> >>>>
> org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:56)
> >>>> at
> >>>>
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
> >>>> at
> >>>>
> org.apache.camel.processor.UnitOfWorkProducer.process(UnitOfWorkProducer.java:73)
> >>>> at
> >>>>
> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:378)
> >>>> at
> >>>>
> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:1)
> >>>> at
> >>>>
> org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:242)
> >>>> at
> >>>>
> org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:346)
> >>>> at
> org.apache.camel.impl.ProducerCache.send(ProducerCache.java:184)
> >>>> at
> >>>>
> org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:124)
> >>>> at
> >>>>
> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:137)
> >>>> at
> >>>>
> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:144)
> >>>> at
> >>>>
> org.apache.camel.processor.jpa.JpaProducerConcurrentTest$1.call(JpaProducerConcurrentTest.java:58)
> >>>> at
> >>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> >>>> at
> >>>>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> >>>> at
> >>>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> >>>> at java.lang.Thread.run(Thread.java:680)
> >>>> Caused by:
> >>>>
> > <openjpa-2.2.2-r422266:1468616 nonfatal user error>
> >>>> org.apache.openjpa.persistence.InvalidStateException: This operation
> >>>> cannot
> >>>> be performed while a Transaction is active.
> >>>> at
> org.apache.openjpa.kernel.BrokerImpl.begin(BrokerImpl.java:1369)
> >>>> at
> >>>>
> org.apache.openjpa.kernel.DelegatingBroker.begin(DelegatingBroker.java:925)
> >>>> at
> >>>>
> org.apache.openjpa.persistence.EntityManagerImpl.begin(EntityManagerImpl.java:565)
> >>>> at
> >>>>
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.enlistInCurrentTransaction(ExtendedEntityManagerCreator.java:422)
> >>>> at
> >>>>
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.doJoinTransaction(ExtendedEntityManagerCreator.java:399)
> >>>> at
> >>>>
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.invoke(ExtendedEntityManagerCreator.java:351)
> >>>> at com.sun.proxy.$Proxy18.joinTransaction(Unknown Source)
> >>>> at
> >>>>
> org.apache.camel.component.jpa.JpaProducer$1.doInTransaction(JpaProducer.java:55)
> >>>> at
> >>>>
> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:131)
> >>>> ... 27 more
> >>>>
> >>>> which is throwing from the line 55 in
> >>>> org.apache.camel.component.jpa.JpaProducer
> >>>> entityManager.joinTransaction();
> >>>>
> >>>> I think this is a concurrency issue we have to investigate (not sure
> >>>> whether the issue is in Camel, Spring, OpenJPA, ...). Will open a JIRA
> >>>> for
> >>>> it.
> >>>>
> >>>> [1] https://builds.apache.org/job/Camel.trunk.fulltest/1562/
> >>>>
> >>>> Best,
> >>>>
> >>>> Christian
> >>>> -----------------
> >>>>
> >>>> Software Integration Specialist
> >>>>
> >>>> Apache Member
> >>>> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
> >>>> Apache Incubator PMC Member
> >>>>
> >>>> https://www.linkedin.com/pub/christian-mueller/11/551/642
> >>>>
> >>>>
> >>>> On Thu, Oct 10, 2013 at 12:28 PM, James Carman
> >>>> <
> >>
> >>>> james@
> >>
> >>>> >wrote:
> >>>>
> >>>>> There are quite a few broken tests. Jetty seems to be hosed. JPA
> has
> >>>>> a
> >>>>> failed test. CDI has issues. It looks like JMS and FTP even decided
> >>>>> to
> >>>>> join the party. This has been going on for a while now. This
> >>>>> particular
> >>>>> job hasn't been "green" since 9/29. Should we stop working other
> >>>>> issues
> >>>>> and get ourselves whole again?
> >>>>>
> >>>>> https://builds.apache.org/job/Camel.trunk.fulltest/1561/
> >>>>>
> >>
> >>
> >>
> >>
> >>
> >> --
> >> View this message in context:
> >> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741403.html
> >> Sent from the Camel Development mailing list archive at Nabble.com.
> >
> >
> >
> > --
> > Claus Ibsen
> > -----------------
> > Red Hat, Inc.
> > Email:
>
> > cibsen@
>
> > Twitter: davsclaus
> > Blog: http://davsclaus.com
> > Author of Camel in Action: http://www.manning.com/ibsen
>
>
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741448.html
> Sent from the Camel Development mailing list archive at Nabble.com.
>
Re: Build failures...
Posted by Babak Vahdat <ba...@swissonline.ch>.
Yeah indeed a deadlock inside the OpenJPA code is the root cause of the
problem (org.apache.openjpa.kernel.BrokerImpl#lock()). I just commited a fix
regarding this, so let's keep an eye on this test by the next builds.
Following a screenshot of the proven deadlock through jconsole:
<http://camel.465427.n5.nabble.com/file/n5741448/Screen_Shot_2013-10-12_at_11.02.58_PM.png>
You can enforce the deadlock if you would make use of a high log level
(log4j.properties), e.g. TRACE instead of just INFO.
Babak
Claus Ibsen-2 wrote
> It could be due an OpenJPA upgrade. These tests used to work better in the
> past.
>
> On Sat, Oct 12, 2013 at 10:05 AM, Babak Vahdat
> <
> babak.vahdat@
> > wrote:
>> The test failed again on the CI-Server. I will look into this in the next
>> days to see if I can find the root cause as on my box I can't reproduce
>> this.
>>
>> Babak
>>
>>
>> Babak Vahdat wrote
>>> Hi
>>>
>>> @Christian I believe I've fixed the JPA test now, could you please check
>>> to see if you can still reproduce the same exception you mentioned
>>> below?
>>>
>>> Babak
>>> Christian Mueller wrote
>>>> Today, it looks much better [1]. We are trying to fix the unit tests
>>>> which
>>>> are failing from time to time on slower boxes. But this isn't so easy
>>>> for
>>>> all of them...
>>>>
>>>> I had a look at
>>>> org.apache.camel.processor.jpa.JpaProducerConcurrentTest.testConcurrentProducers.
>>>> A few times I get the following exception which results in missing
>>>> exchanges in the mock:result endpoint:
>>>>
>>>> 2013-10-11 14:01:10,623 [pool-2-thread-2] ERROR
>>>> DefaultErrorHandler - Failed delivery for (MessageId:
>>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4 on
>>>> ExchangeId:
>>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9). Exhausted
>>>> after delivery attempt: 1 caught:
>>>> org.springframework.dao.InvalidDataAccessApiUsageException: This
>>>> operation
>>>> cannot be performed while a Transaction is active.; nested exception is
>>>>
> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>>>> org.apache.openjpa.persistence.InvalidStateException: This operation
>>>> cannot
>>>> be performed while a Transaction is active.
>>>>
>>>> Message History
>>>> ---------------------------------------------------------------------------------------------------------------------------------------
>>>> RouteId ProcessorId
>>>> Processor
>>>> Elapsed (ms)
>>>> [route1 ] [route1 ]
>>>> [direct://start
>>>> ] [ 68]
>>>> [route1 ] [to1 ]
>>>> [jpa://org.apache.camel.examples.SendEmail
>>>> ] [ 67]
>>>>
>>>> Exchange
>>>> ---------------------------------------------------------------------------------------------------------------------------------------
>>>> Exchange[
>>>> Id
>>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9
>>>> ExchangePattern InOnly
>>>> Headers
>>>> {breadcrumbId=ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4,
>>>> CamelEntityManager=org.apache.openjpa.persistence.EntityManagerImpl@6e453dd5,
>>>> CamelRedelivered=false, CamelRedeliveryCounter=0}
>>>> BodyType org.apache.camel.examples.SendEmail
>>>> Body SendEmail[id: 102 address: null]
>>>> ]
>>>>
>>>> Stacktrace
>>>> ---------------------------------------------------------------------------------------------------------------------------------------
>>>> org.springframework.dao.InvalidDataAccessApiUsageException: This
>>>> operation
>>>> cannot be performed while a Transaction is active.; nested exception is
>>>>
> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>>>> org.apache.openjpa.persistence.InvalidStateException: This operation
>>>> cannot
>>>> be performed while a Transaction is active.
>>>> at
>>>> org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:293)
>>>> at
>>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.convertException(ExtendedEntityManagerCreator.java:501)
>>>> at
>>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.afterCommit(ExtendedEntityManagerCreator.java:482)
>>>> at
>>>> org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCommit(TransactionSynchronizationUtils.java:133)
>>>> at
>>>> org.springframework.transaction.support.TransactionSynchronizationUtils.triggerAfterCommit(TransactionSynchronizationUtils.java:121)
>>>> at
>>>> org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCommit(AbstractPlatformTransactionManager.java:951)
>>>> at
>>>> org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:797)
>>>> at
>>>> org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724)
>>>> at
>>>> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:148)
>>>> at
>>>> org.apache.camel.component.jpa.JpaProducer.process(JpaProducer.java:53)
>>>> at
>>>> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
>>>> at
>>>> org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:132)
>>>> at
>>>> org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:307)
>>>> at
>>>> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:127)
>>>> at
>>>> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)
>>>> at
>>>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
>>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
>>>> at
>>>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>>>> at
>>>> org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:56)
>>>> at
>>>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>>>> at
>>>> org.apache.camel.processor.UnitOfWorkProducer.process(UnitOfWorkProducer.java:73)
>>>> at
>>>> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:378)
>>>> at
>>>> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:1)
>>>> at
>>>> org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:242)
>>>> at
>>>> org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:346)
>>>> at org.apache.camel.impl.ProducerCache.send(ProducerCache.java:184)
>>>> at
>>>> org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:124)
>>>> at
>>>> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:137)
>>>> at
>>>> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:144)
>>>> at
>>>> org.apache.camel.processor.jpa.JpaProducerConcurrentTest$1.call(JpaProducerConcurrentTest.java:58)
>>>> at
>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>> at java.lang.Thread.run(Thread.java:680)
>>>> Caused by:
>>>>
> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>>>> org.apache.openjpa.persistence.InvalidStateException: This operation
>>>> cannot
>>>> be performed while a Transaction is active.
>>>> at org.apache.openjpa.kernel.BrokerImpl.begin(BrokerImpl.java:1369)
>>>> at
>>>> org.apache.openjpa.kernel.DelegatingBroker.begin(DelegatingBroker.java:925)
>>>> at
>>>> org.apache.openjpa.persistence.EntityManagerImpl.begin(EntityManagerImpl.java:565)
>>>> at
>>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.enlistInCurrentTransaction(ExtendedEntityManagerCreator.java:422)
>>>> at
>>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.doJoinTransaction(ExtendedEntityManagerCreator.java:399)
>>>> at
>>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.invoke(ExtendedEntityManagerCreator.java:351)
>>>> at com.sun.proxy.$Proxy18.joinTransaction(Unknown Source)
>>>> at
>>>> org.apache.camel.component.jpa.JpaProducer$1.doInTransaction(JpaProducer.java:55)
>>>> at
>>>> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:131)
>>>> ... 27 more
>>>>
>>>> which is throwing from the line 55 in
>>>> org.apache.camel.component.jpa.JpaProducer
>>>> entityManager.joinTransaction();
>>>>
>>>> I think this is a concurrency issue we have to investigate (not sure
>>>> whether the issue is in Camel, Spring, OpenJPA, ...). Will open a JIRA
>>>> for
>>>> it.
>>>>
>>>> [1] https://builds.apache.org/job/Camel.trunk.fulltest/1562/
>>>>
>>>> Best,
>>>>
>>>> Christian
>>>> -----------------
>>>>
>>>> Software Integration Specialist
>>>>
>>>> Apache Member
>>>> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
>>>> Apache Incubator PMC Member
>>>>
>>>> https://www.linkedin.com/pub/christian-mueller/11/551/642
>>>>
>>>>
>>>> On Thu, Oct 10, 2013 at 12:28 PM, James Carman
>>>> <
>>
>>>> james@
>>
>>>> >wrote:
>>>>
>>>>> There are quite a few broken tests. Jetty seems to be hosed. JPA has
>>>>> a
>>>>> failed test. CDI has issues. It looks like JMS and FTP even decided
>>>>> to
>>>>> join the party. This has been going on for a while now. This
>>>>> particular
>>>>> job hasn't been "green" since 9/29. Should we stop working other
>>>>> issues
>>>>> and get ourselves whole again?
>>>>>
>>>>> https://builds.apache.org/job/Camel.trunk.fulltest/1561/
>>>>>
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741403.html
>> Sent from the Camel Development mailing list archive at Nabble.com.
>
>
>
> --
> Claus Ibsen
> -----------------
> Red Hat, Inc.
> Email:
> cibsen@
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
--
View this message in context: http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741448.html
Sent from the Camel Development mailing list archive at Nabble.com.
Re: Build failures...
Posted by Claus Ibsen <cl...@gmail.com>.
It could be due an OpenJPA upgrade. These tests used to work better in the past.
On Sat, Oct 12, 2013 at 10:05 AM, Babak Vahdat
<ba...@swissonline.ch> wrote:
> The test failed again on the CI-Server. I will look into this in the next
> days to see if I can find the root cause as on my box I can't reproduce
> this.
>
> Babak
>
>
> Babak Vahdat wrote
>> Hi
>>
>> @Christian I believe I've fixed the JPA test now, could you please check
>> to see if you can still reproduce the same exception you mentioned below?
>>
>> Babak
>> Christian Mueller wrote
>>> Today, it looks much better [1]. We are trying to fix the unit tests
>>> which
>>> are failing from time to time on slower boxes. But this isn't so easy for
>>> all of them...
>>>
>>> I had a look at
>>> org.apache.camel.processor.jpa.JpaProducerConcurrentTest.testConcurrentProducers.
>>> A few times I get the following exception which results in missing
>>> exchanges in the mock:result endpoint:
>>>
>>> 2013-10-11 14:01:10,623 [pool-2-thread-2] ERROR
>>> DefaultErrorHandler - Failed delivery for (MessageId:
>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4 on
>>> ExchangeId:
>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9). Exhausted
>>> after delivery attempt: 1 caught:
>>> org.springframework.dao.InvalidDataAccessApiUsageException: This
>>> operation
>>> cannot be performed while a Transaction is active.; nested exception is
>>> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>>> org.apache.openjpa.persistence.InvalidStateException: This operation
>>> cannot
>>> be performed while a Transaction is active.
>>>
>>> Message History
>>> ---------------------------------------------------------------------------------------------------------------------------------------
>>> RouteId ProcessorId
>>> Processor
>>> Elapsed (ms)
>>> [route1 ] [route1 ]
>>> [direct://start
>>> ] [ 68]
>>> [route1 ] [to1 ]
>>> [jpa://org.apache.camel.examples.SendEmail
>>> ] [ 67]
>>>
>>> Exchange
>>> ---------------------------------------------------------------------------------------------------------------------------------------
>>> Exchange[
>>> Id
>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9
>>> ExchangePattern InOnly
>>> Headers
>>> {breadcrumbId=ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4,
>>> CamelEntityManager=org.apache.openjpa.persistence.EntityManagerImpl@6e453dd5,
>>> CamelRedelivered=false, CamelRedeliveryCounter=0}
>>> BodyType org.apache.camel.examples.SendEmail
>>> Body SendEmail[id: 102 address: null]
>>> ]
>>>
>>> Stacktrace
>>> ---------------------------------------------------------------------------------------------------------------------------------------
>>> org.springframework.dao.InvalidDataAccessApiUsageException: This
>>> operation
>>> cannot be performed while a Transaction is active.; nested exception is
>>> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>>> org.apache.openjpa.persistence.InvalidStateException: This operation
>>> cannot
>>> be performed while a Transaction is active.
>>> at
>>> org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:293)
>>> at
>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.convertException(ExtendedEntityManagerCreator.java:501)
>>> at
>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.afterCommit(ExtendedEntityManagerCreator.java:482)
>>> at
>>> org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCommit(TransactionSynchronizationUtils.java:133)
>>> at
>>> org.springframework.transaction.support.TransactionSynchronizationUtils.triggerAfterCommit(TransactionSynchronizationUtils.java:121)
>>> at
>>> org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCommit(AbstractPlatformTransactionManager.java:951)
>>> at
>>> org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:797)
>>> at
>>> org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724)
>>> at
>>> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:148)
>>> at
>>> org.apache.camel.component.jpa.JpaProducer.process(JpaProducer.java:53)
>>> at
>>> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
>>> at
>>> org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:132)
>>> at
>>> org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:307)
>>> at
>>> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:127)
>>> at
>>> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)
>>> at
>>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
>>> at
>>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>>> at
>>> org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:56)
>>> at
>>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>>> at
>>> org.apache.camel.processor.UnitOfWorkProducer.process(UnitOfWorkProducer.java:73)
>>> at
>>> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:378)
>>> at
>>> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:1)
>>> at
>>> org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:242)
>>> at
>>> org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:346)
>>> at org.apache.camel.impl.ProducerCache.send(ProducerCache.java:184)
>>> at
>>> org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:124)
>>> at
>>> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:137)
>>> at
>>> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:144)
>>> at
>>> org.apache.camel.processor.jpa.JpaProducerConcurrentTest$1.call(JpaProducerConcurrentTest.java:58)
>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>> at java.lang.Thread.run(Thread.java:680)
>>> Caused by:
>>> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>>> org.apache.openjpa.persistence.InvalidStateException: This operation
>>> cannot
>>> be performed while a Transaction is active.
>>> at org.apache.openjpa.kernel.BrokerImpl.begin(BrokerImpl.java:1369)
>>> at
>>> org.apache.openjpa.kernel.DelegatingBroker.begin(DelegatingBroker.java:925)
>>> at
>>> org.apache.openjpa.persistence.EntityManagerImpl.begin(EntityManagerImpl.java:565)
>>> at
>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.enlistInCurrentTransaction(ExtendedEntityManagerCreator.java:422)
>>> at
>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.doJoinTransaction(ExtendedEntityManagerCreator.java:399)
>>> at
>>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.invoke(ExtendedEntityManagerCreator.java:351)
>>> at com.sun.proxy.$Proxy18.joinTransaction(Unknown Source)
>>> at
>>> org.apache.camel.component.jpa.JpaProducer$1.doInTransaction(JpaProducer.java:55)
>>> at
>>> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:131)
>>> ... 27 more
>>>
>>> which is throwing from the line 55 in
>>> org.apache.camel.component.jpa.JpaProducer
>>> entityManager.joinTransaction();
>>>
>>> I think this is a concurrency issue we have to investigate (not sure
>>> whether the issue is in Camel, Spring, OpenJPA, ...). Will open a JIRA
>>> for
>>> it.
>>>
>>> [1] https://builds.apache.org/job/Camel.trunk.fulltest/1562/
>>>
>>> Best,
>>>
>>> Christian
>>> -----------------
>>>
>>> Software Integration Specialist
>>>
>>> Apache Member
>>> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
>>> Apache Incubator PMC Member
>>>
>>> https://www.linkedin.com/pub/christian-mueller/11/551/642
>>>
>>>
>>> On Thu, Oct 10, 2013 at 12:28 PM, James Carman
>>> <
>
>>> james@
>
>>> >wrote:
>>>
>>>> There are quite a few broken tests. Jetty seems to be hosed. JPA has a
>>>> failed test. CDI has issues. It looks like JMS and FTP even decided to
>>>> join the party. This has been going on for a while now. This
>>>> particular
>>>> job hasn't been "green" since 9/29. Should we stop working other issues
>>>> and get ourselves whole again?
>>>>
>>>> https://builds.apache.org/job/Camel.trunk.fulltest/1561/
>>>>
>
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741403.html
> Sent from the Camel Development mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Build failures...
Posted by Babak Vahdat <ba...@swissonline.ch>.
The test failed again on the CI-Server. I will look into this in the next
days to see if I can find the root cause as on my box I can't reproduce
this.
Babak
Babak Vahdat wrote
> Hi
>
> @Christian I believe I've fixed the JPA test now, could you please check
> to see if you can still reproduce the same exception you mentioned below?
>
> Babak
> Christian Mueller wrote
>> Today, it looks much better [1]. We are trying to fix the unit tests
>> which
>> are failing from time to time on slower boxes. But this isn't so easy for
>> all of them...
>>
>> I had a look at
>> org.apache.camel.processor.jpa.JpaProducerConcurrentTest.testConcurrentProducers.
>> A few times I get the following exception which results in missing
>> exchanges in the mock:result endpoint:
>>
>> 2013-10-11 14:01:10,623 [pool-2-thread-2] ERROR
>> DefaultErrorHandler - Failed delivery for (MessageId:
>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4 on
>> ExchangeId:
>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9). Exhausted
>> after delivery attempt: 1 caught:
>> org.springframework.dao.InvalidDataAccessApiUsageException: This
>> operation
>> cannot be performed while a Transaction is active.; nested exception is
>> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>> org.apache.openjpa.persistence.InvalidStateException: This operation
>> cannot
>> be performed while a Transaction is active.
>>
>> Message History
>> ---------------------------------------------------------------------------------------------------------------------------------------
>> RouteId ProcessorId
>> Processor
>> Elapsed (ms)
>> [route1 ] [route1 ]
>> [direct://start
>> ] [ 68]
>> [route1 ] [to1 ]
>> [jpa://org.apache.camel.examples.SendEmail
>> ] [ 67]
>>
>> Exchange
>> ---------------------------------------------------------------------------------------------------------------------------------------
>> Exchange[
>> Id
>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9
>> ExchangePattern InOnly
>> Headers
>> {breadcrumbId=ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4,
>> CamelEntityManager=org.apache.openjpa.persistence.EntityManagerImpl@6e453dd5,
>> CamelRedelivered=false, CamelRedeliveryCounter=0}
>> BodyType org.apache.camel.examples.SendEmail
>> Body SendEmail[id: 102 address: null]
>> ]
>>
>> Stacktrace
>> ---------------------------------------------------------------------------------------------------------------------------------------
>> org.springframework.dao.InvalidDataAccessApiUsageException: This
>> operation
>> cannot be performed while a Transaction is active.; nested exception is
>> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>> org.apache.openjpa.persistence.InvalidStateException: This operation
>> cannot
>> be performed while a Transaction is active.
>> at
>> org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:293)
>> at
>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.convertException(ExtendedEntityManagerCreator.java:501)
>> at
>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.afterCommit(ExtendedEntityManagerCreator.java:482)
>> at
>> org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCommit(TransactionSynchronizationUtils.java:133)
>> at
>> org.springframework.transaction.support.TransactionSynchronizationUtils.triggerAfterCommit(TransactionSynchronizationUtils.java:121)
>> at
>> org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCommit(AbstractPlatformTransactionManager.java:951)
>> at
>> org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:797)
>> at
>> org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724)
>> at
>> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:148)
>> at
>> org.apache.camel.component.jpa.JpaProducer.process(JpaProducer.java:53)
>> at
>> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
>> at
>> org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:132)
>> at
>> org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:307)
>> at
>> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:127)
>> at
>> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)
>> at
>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
>> at
>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>> at
>> org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:56)
>> at
>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>> at
>> org.apache.camel.processor.UnitOfWorkProducer.process(UnitOfWorkProducer.java:73)
>> at
>> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:378)
>> at
>> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:1)
>> at
>> org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:242)
>> at
>> org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:346)
>> at org.apache.camel.impl.ProducerCache.send(ProducerCache.java:184)
>> at
>> org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:124)
>> at
>> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:137)
>> at
>> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:144)
>> at
>> org.apache.camel.processor.jpa.JpaProducerConcurrentTest$1.call(JpaProducerConcurrentTest.java:58)
>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>> at java.lang.Thread.run(Thread.java:680)
>> Caused by:
>> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
>> org.apache.openjpa.persistence.InvalidStateException: This operation
>> cannot
>> be performed while a Transaction is active.
>> at org.apache.openjpa.kernel.BrokerImpl.begin(BrokerImpl.java:1369)
>> at
>> org.apache.openjpa.kernel.DelegatingBroker.begin(DelegatingBroker.java:925)
>> at
>> org.apache.openjpa.persistence.EntityManagerImpl.begin(EntityManagerImpl.java:565)
>> at
>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.enlistInCurrentTransaction(ExtendedEntityManagerCreator.java:422)
>> at
>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.doJoinTransaction(ExtendedEntityManagerCreator.java:399)
>> at
>> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.invoke(ExtendedEntityManagerCreator.java:351)
>> at com.sun.proxy.$Proxy18.joinTransaction(Unknown Source)
>> at
>> org.apache.camel.component.jpa.JpaProducer$1.doInTransaction(JpaProducer.java:55)
>> at
>> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:131)
>> ... 27 more
>>
>> which is throwing from the line 55 in
>> org.apache.camel.component.jpa.JpaProducer
>> entityManager.joinTransaction();
>>
>> I think this is a concurrency issue we have to investigate (not sure
>> whether the issue is in Camel, Spring, OpenJPA, ...). Will open a JIRA
>> for
>> it.
>>
>> [1] https://builds.apache.org/job/Camel.trunk.fulltest/1562/
>>
>> Best,
>>
>> Christian
>> -----------------
>>
>> Software Integration Specialist
>>
>> Apache Member
>> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
>> Apache Incubator PMC Member
>>
>> https://www.linkedin.com/pub/christian-mueller/11/551/642
>>
>>
>> On Thu, Oct 10, 2013 at 12:28 PM, James Carman
>> <
>> james@
>> >wrote:
>>
>>> There are quite a few broken tests. Jetty seems to be hosed. JPA has a
>>> failed test. CDI has issues. It looks like JMS and FTP even decided to
>>> join the party. This has been going on for a while now. This
>>> particular
>>> job hasn't been "green" since 9/29. Should we stop working other issues
>>> and get ourselves whole again?
>>>
>>> https://builds.apache.org/job/Camel.trunk.fulltest/1561/
>>>
--
View this message in context: http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741403.html
Sent from the Camel Development mailing list archive at Nabble.com.
Re: Build failures...
Posted by Babak Vahdat <ba...@swissonline.ch>.
Hi
@Christian I believe I've fixed the JPA test now, could you please check to
see if you can still reproduce the same exception you mentioned below?
Babak
Christian Mueller wrote
> Today, it looks much better [1]. We are trying to fix the unit tests which
> are failing from time to time on slower boxes. But this isn't so easy for
> all of them...
>
> I had a look at
> org.apache.camel.processor.jpa.JpaProducerConcurrentTest.testConcurrentProducers.
> A few times I get the following exception which results in missing
> exchanges in the mock:result endpoint:
>
> 2013-10-11 14:01:10,623 [pool-2-thread-2] ERROR
> DefaultErrorHandler - Failed delivery for (MessageId:
> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4 on ExchangeId:
> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9). Exhausted
> after delivery attempt: 1 caught:
> org.springframework.dao.InvalidDataAccessApiUsageException: This operation
> cannot be performed while a Transaction is active.; nested exception is
> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
> org.apache.openjpa.persistence.InvalidStateException: This operation
> cannot
> be performed while a Transaction is active.
>
> Message History
> ---------------------------------------------------------------------------------------------------------------------------------------
> RouteId ProcessorId
> Processor
> Elapsed (ms)
> [route1 ] [route1 ]
> [direct://start
> ] [ 68]
> [route1 ] [to1 ]
> [jpa://org.apache.camel.examples.SendEmail
> ] [ 67]
>
> Exchange
> ---------------------------------------------------------------------------------------------------------------------------------------
> Exchange[
> Id
> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9
> ExchangePattern InOnly
> Headers
> {breadcrumbId=ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4,
> CamelEntityManager=org.apache.openjpa.persistence.EntityManagerImpl@6e453dd5,
> CamelRedelivered=false, CamelRedeliveryCounter=0}
> BodyType org.apache.camel.examples.SendEmail
> Body SendEmail[id: 102 address: null]
> ]
>
> Stacktrace
> ---------------------------------------------------------------------------------------------------------------------------------------
> org.springframework.dao.InvalidDataAccessApiUsageException: This operation
> cannot be performed while a Transaction is active.; nested exception is
> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
> org.apache.openjpa.persistence.InvalidStateException: This operation
> cannot
> be performed while a Transaction is active.
> at
> org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:293)
> at
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.convertException(ExtendedEntityManagerCreator.java:501)
> at
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.afterCommit(ExtendedEntityManagerCreator.java:482)
> at
> org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCommit(TransactionSynchronizationUtils.java:133)
> at
> org.springframework.transaction.support.TransactionSynchronizationUtils.triggerAfterCommit(TransactionSynchronizationUtils.java:121)
> at
> org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCommit(AbstractPlatformTransactionManager.java:951)
> at
> org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:797)
> at
> org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724)
> at
> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:148)
> at
> org.apache.camel.component.jpa.JpaProducer.process(JpaProducer.java:53)
> at
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> at
> org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:132)
> at
> org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:307)
> at
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:127)
> at
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)
> at
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
> at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
> at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
> at
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
> at
> org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:56)
> at
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
> at
> org.apache.camel.processor.UnitOfWorkProducer.process(UnitOfWorkProducer.java:73)
> at
> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:378)
> at
> org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:1)
> at
> org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:242)
> at
> org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:346)
> at org.apache.camel.impl.ProducerCache.send(ProducerCache.java:184)
> at
> org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:124)
> at
> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:137)
> at
> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:144)
> at
> org.apache.camel.processor.jpa.JpaProducerConcurrentTest$1.call(JpaProducerConcurrentTest.java:58)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:680)
> Caused by:
> <openjpa-2.2.2-r422266:1468616 nonfatal user error>
> org.apache.openjpa.persistence.InvalidStateException: This operation
> cannot
> be performed while a Transaction is active.
> at org.apache.openjpa.kernel.BrokerImpl.begin(BrokerImpl.java:1369)
> at
> org.apache.openjpa.kernel.DelegatingBroker.begin(DelegatingBroker.java:925)
> at
> org.apache.openjpa.persistence.EntityManagerImpl.begin(EntityManagerImpl.java:565)
> at
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.enlistInCurrentTransaction(ExtendedEntityManagerCreator.java:422)
> at
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.doJoinTransaction(ExtendedEntityManagerCreator.java:399)
> at
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.invoke(ExtendedEntityManagerCreator.java:351)
> at com.sun.proxy.$Proxy18.joinTransaction(Unknown Source)
> at
> org.apache.camel.component.jpa.JpaProducer$1.doInTransaction(JpaProducer.java:55)
> at
> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:131)
> ... 27 more
>
> which is throwing from the line 55 in
> org.apache.camel.component.jpa.JpaProducer
> entityManager.joinTransaction();
>
> I think this is a concurrency issue we have to investigate (not sure
> whether the issue is in Camel, Spring, OpenJPA, ...). Will open a JIRA for
> it.
>
> [1] https://builds.apache.org/job/Camel.trunk.fulltest/1562/
>
> Best,
>
> Christian
> -----------------
>
> Software Integration Specialist
>
> Apache Member
> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
> Apache Incubator PMC Member
>
> https://www.linkedin.com/pub/christian-mueller/11/551/642
>
>
> On Thu, Oct 10, 2013 at 12:28 PM, James Carman
> <
> james@
> >wrote:
>
>> There are quite a few broken tests. Jetty seems to be hosed. JPA has a
>> failed test. CDI has issues. It looks like JMS and FTP even decided to
>> join the party. This has been going on for a while now. This particular
>> job hasn't been "green" since 9/29. Should we stop working other issues
>> and get ourselves whole again?
>>
>> https://builds.apache.org/job/Camel.trunk.fulltest/1561/
>>
--
View this message in context: http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741367.html
Sent from the Camel Development mailing list archive at Nabble.com.
Re: Build failures...
Posted by Christian Müller <ch...@gmail.com>.
Today, it looks much better [1]. We are trying to fix the unit tests which
are failing from time to time on slower boxes. But this isn't so easy for
all of them...
I had a look at
org.apache.camel.processor.jpa.JpaProducerConcurrentTest.testConcurrentProducers.
A few times I get the following exception which results in missing
exchanges in the mock:result endpoint:
2013-10-11 14:01:10,623 [pool-2-thread-2] ERROR
DefaultErrorHandler - Failed delivery for (MessageId:
ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4 on ExchangeId:
ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9). Exhausted
after delivery attempt: 1 caught:
org.springframework.dao.InvalidDataAccessApiUsageException: This operation
cannot be performed while a Transaction is active.; nested exception is
<openjpa-2.2.2-r422266:1468616 nonfatal user error>
org.apache.openjpa.persistence.InvalidStateException: This operation cannot
be performed while a Transaction is active.
Message History
---------------------------------------------------------------------------------------------------------------------------------------
RouteId ProcessorId
Processor
Elapsed (ms)
[route1 ] [route1 ]
[direct://start
] [ 68]
[route1 ] [to1 ]
[jpa://org.apache.camel.examples.SendEmail
] [ 67]
Exchange
---------------------------------------------------------------------------------------------------------------------------------------
Exchange[
Id
ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9
ExchangePattern InOnly
Headers
{breadcrumbId=ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4,
CamelEntityManager=org.apache.openjpa.persistence.EntityManagerImpl@6e453dd5,
CamelRedelivered=false, CamelRedeliveryCounter=0}
BodyType org.apache.camel.examples.SendEmail
Body SendEmail[id: 102 address: null]
]
Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
org.springframework.dao.InvalidDataAccessApiUsageException: This operation
cannot be performed while a Transaction is active.; nested exception is
<openjpa-2.2.2-r422266:1468616 nonfatal user error>
org.apache.openjpa.persistence.InvalidStateException: This operation cannot
be performed while a Transaction is active.
at
org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:293)
at
org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.convertException(ExtendedEntityManagerCreator.java:501)
at
org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.afterCommit(ExtendedEntityManagerCreator.java:482)
at
org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCommit(TransactionSynchronizationUtils.java:133)
at
org.springframework.transaction.support.TransactionSynchronizationUtils.triggerAfterCommit(TransactionSynchronizationUtils.java:121)
at
org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCommit(AbstractPlatformTransactionManager.java:951)
at
org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:797)
at
org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724)
at
org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:148)
at
org.apache.camel.component.jpa.JpaProducer.process(JpaProducer.java:53)
at
org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
at
org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:132)
at
org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:307)
at
org.apache.camel.processor.SendProcessor.process(SendProcessor.java:127)
at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)
at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at
org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:56)
at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at
org.apache.camel.processor.UnitOfWorkProducer.process(UnitOfWorkProducer.java:73)
at
org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:378)
at
org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:1)
at
org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:242)
at
org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:346)
at org.apache.camel.impl.ProducerCache.send(ProducerCache.java:184)
at
org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:124)
at
org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:137)
at
org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:144)
at
org.apache.camel.processor.jpa.JpaProducerConcurrentTest$1.call(JpaProducerConcurrentTest.java:58)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:680)
Caused by: <openjpa-2.2.2-r422266:1468616 nonfatal user error>
org.apache.openjpa.persistence.InvalidStateException: This operation cannot
be performed while a Transaction is active.
at org.apache.openjpa.kernel.BrokerImpl.begin(BrokerImpl.java:1369)
at
org.apache.openjpa.kernel.DelegatingBroker.begin(DelegatingBroker.java:925)
at
org.apache.openjpa.persistence.EntityManagerImpl.begin(EntityManagerImpl.java:565)
at
org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.enlistInCurrentTransaction(ExtendedEntityManagerCreator.java:422)
at
org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.doJoinTransaction(ExtendedEntityManagerCreator.java:399)
at
org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.invoke(ExtendedEntityManagerCreator.java:351)
at com.sun.proxy.$Proxy18.joinTransaction(Unknown Source)
at
org.apache.camel.component.jpa.JpaProducer$1.doInTransaction(JpaProducer.java:55)
at
org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:131)
... 27 more
which is throwing from the line 55 in
org.apache.camel.component.jpa.JpaProducer
entityManager.joinTransaction();
I think this is a concurrency issue we have to investigate (not sure
whether the issue is in Camel, Spring, OpenJPA, ...). Will open a JIRA for
it.
[1] https://builds.apache.org/job/Camel.trunk.fulltest/1562/
Best,
Christian
-----------------
Software Integration Specialist
Apache Member
V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer
Apache Incubator PMC Member
https://www.linkedin.com/pub/christian-mueller/11/551/642
On Thu, Oct 10, 2013 at 12:28 PM, James Carman
<ja...@carmanconsulting.com>wrote:
> There are quite a few broken tests. Jetty seems to be hosed. JPA has a
> failed test. CDI has issues. It looks like JMS and FTP even decided to
> join the party. This has been going on for a while now. This particular
> job hasn't been "green" since 9/29. Should we stop working other issues
> and get ourselves whole again?
>
> https://builds.apache.org/job/Camel.trunk.fulltest/1561/
>