You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-user@db.apache.org by Johan Hoogenboezem <ho...@worldonline.co.za> on 2005/10/05 14:43:13 UTC

RE: Derby XA, Hibernate 2.1.6, WebSphere 5.1 - Server returned XAER_NOTA at commit time

Hi Stanley
Thank you for your suggestion. I had a look at the link you sent me and made
sure I had the latest fixpack (for WSAD 5.1.2)loaded. It had no effect on
the problem. 
I feel I should point out that the error you latched on to was one of the
last ones listed. In my experience the first error in a cascade of errors is
usually the important one:

>[9/13/05 8:23:49:819 CAT] 1e59f386 WSRdbXaResour E DSRA0304E:  XAException
>occurred. XAException contents and details are: The cause is
:
>org.apache.derby.client.am.SqlException: Error executing a
>XAResource.commit(), Server returned XAER_NOTA.
>
>[9/13/05 8:23:49:819 CAT] 1e59f386 WSRdbXaResour E DSRA0302E:  XAException
>occurred.  Error code is: XAER_NOTA.  Exception is: XAER_NOTA : Error
>executing a XAResource.commit(), Server returned XAER_NOTA
>
>[9/13/05 8:23:49:889 CAT] 1e59f386 XATransaction E J2CA0027E: An exception
>occurred while invoking commit on an XA Resource Adapter from dataSource
>jdbc/derby/scvwebdev, within transaction ID {XID: formatId(57415344),
>gtrid_length(39), bqual_length(28),
>data(0000000000000002000000044c4c40cbd49eaa1530e4b7146f080d8853876c7a736572
7
>66572314c4c40cbd49eaa1530e4b7146f080d8853876c7a000000048333bb35)}:
>org.apache.derby.client.am.XaException: XAER_NOTA : Error executing a
>XAResource.commit(), Server returned XAER_NOTA
>	at
>org.apache.derby.client.net.NetXAResource.throwXAException(Unknown Source)
>	at org.apache.derby.client.net.NetXAResource.commit(Unknown Source)
>	at
>...
> etc. etc. etc.

The later errors are usually side effects which go away when the real
problem is solved. From analyzing the javadoc of the standard JTA api's it
is clear that if the "destroy" method is called when a transaction is not in
an "idle" state, then further errors will be generated. The container calls
the destroy method, even though the commit resulted in an error, so further
errors are almost assured.

In any case, I tried to enable tracing as you suggested - and quickly got
lost in the quagmire of log files and more log files. I am completely lost
here as I have no documentation that can explain to me what the trace means.
I can provide the trace log if anyone wants it, though.

I then resorted to re-building Derby with some debug statements. I was
flying blind here, too, which is what one does when desperate. I also get
nowhere, having no knowledge of the inner workings of a JTA implementation.

So, it seems I'm stuck.

For me, it seems to be a pretty simple matter of a derby developer
(preferably the guy/girl who worked on the XA stuff :-)) to get together
with an IBM WebSphere (v5.1.x) guy (IBM donates development time to Derby,
right?), and then for this elite team of two to knock up a quick example of
responding to a JMS message and accessing a Derby datasource suitably
embedded in a Hibernate session. I can provide some code examples that will
help reproduce the problem. I've invested a lot of time in getting
Hibernate, Session beans, MDBs and datasources to co-exist peacefully (at
least for DB2). So at least that was worth the effort.

I'd be very grateful if someone can help with this. I would really like to
use Derby exclusively as my development rdbms. I rather like the idea of a
RDBMS written in Java.
Regards
Johan

-----Original Message-----
From: Stanley Bradbury [mailto:Stan.Bradbury@gmail.com] 
Sent: Wednesday, September 21, 2005 2:09 AM
To: Derby Discussion
Subject: Re: Derby XA, Hibernate 2.1.6, WebSphere 5.1 - Server returned
XAER_NOTA at commit time


Hi Johan -
I had hoped to find more for you on either the hibernate or the 
WebSphere site but only found one reference to this error.  This problem 
report suggests that a 'fix' may be available from IBM (this is a Tivoli 
product  but Tivoli uses WebSpere and this did happen in the same class 
indicated in your trace:   
   com.ibm.ws.rsadapter.spi.WSRdbManagedConnection).

 It also sounds like the problem was related to threads not being in 
sync (..use more thread-safe functions).  I think the next step in 
resolving this would be to turn on tracing in WebSphere to see if  the 
additional information indicates why there is still an open 
transaction.  Here is the URL to review  the description problem 
description.

http://www-1.ibm.com/support/docview.wss?rs=493&context=SWK90&uid=swg1IY7188
6



RE: Derby XA, Hibernate 2.1.6, WebSphere 5.1 - Server returned XAER_NOTA at commit time

Posted by Johan Hoogenboezem <ho...@worldonline.co.za>.
No, the snapshot didn't help...

-----Original Message-----
From: Stanley Bradbury [mailto:Stan.Bradbury@gmail.com] 
Sent: Monday, October 17, 2005 12:19 AM
To: Johan Hoogenboezem
Cc: derby-user@db.apache.org
Subject: Re: Derby XA, Hibernate 2.1.6, WebSphere 5.1 - Server returned
XAER_NOTA at commit time


Johan Hoogenboezem wrote:

>Hi Stanley
>Thank you for your suggestion. I had a look at the link you sent me and
made
>sure I had the latest fixpack (for WSAD 5.1.2)loaded. It had no effect on
>the problem. 
>I feel I should point out that the error you latched on to was one of the
>last ones listed. In my experience the first error in a cascade of errors
is
>usually the important one:
>
>  
>
>>[9/13/05 8:23:49:819 CAT] 1e59f386 WSRdbXaResour E DSRA0304E:  XAException
>>occurred. XAException contents and details are: The cause is
>>    
>>
>:
>  
>
>>org.apache.derby.client.am.SqlException: Error executing a
>>XAResource.commit(), Server returned XAER_NOTA.
>>
>>[9/13/05 8:23:49:819 CAT] 1e59f386 WSRdbXaResour E DSRA0302E:  XAException
>>occurred.  Error code is: XAER_NOTA.  Exception is: XAER_NOTA : Error
>>executing a XAResource.commit(), Server returned XAER_NOTA
>>
>>[9/13/05 8:23:49:889 CAT] 1e59f386 XATransaction E J2CA0027E: An exception
>>occurred while invoking commit on an XA Resource Adapter from dataSource
>>jdbc/derby/scvwebdev, within transaction ID {XID: formatId(57415344),
>>gtrid_length(39), bqual_length(28),
>>data(0000000000000002000000044c4c40cbd49eaa1530e4b7146f080d8853876c7a73657
2
>>    
>>
>7
>  
>
>>66572314c4c40cbd49eaa1530e4b7146f080d8853876c7a000000048333bb35)}:
>>org.apache.derby.client.am.XaException: XAER_NOTA : Error executing a
>>XAResource.commit(), Server returned XAER_NOTA
>>	at
>>org.apache.derby.client.net.NetXAResource.throwXAException(Unknown Source)
>>	at org.apache.derby.client.net.NetXAResource.commit(Unknown Source)
>>	at
>>...
>>etc. etc. etc.
>>    
>>
>
>The later errors are usually side effects which go away when the real
>problem is solved. From analyzing the javadoc of the standard JTA api's it
>is clear that if the "destroy" method is called when a transaction is not
in
>an "idle" state, then further errors will be generated. The container calls
>the destroy method, even though the commit resulted in an error, so further
>errors are almost assured.
>
>In any case, I tried to enable tracing as you suggested - and quickly got
>lost in the quagmire of log files and more log files. I am completely lost
>here as I have no documentation that can explain to me what the trace
means.
>I can provide the trace log if anyone wants it, though.
>
>I then resorted to re-building Derby with some debug statements. I was
>flying blind here, too, which is what one does when desperate. I also get
>nowhere, having no knowledge of the inner workings of a JTA implementation.
>
>So, it seems I'm stuck.
>
>For me, it seems to be a pretty simple matter of a derby developer
>(preferably the guy/girl who worked on the XA stuff :-)) to get together
>with an IBM WebSphere (v5.1.x) guy (IBM donates development time to Derby,
>right?), and then for this elite team of two to knock up a quick example of
>responding to a JMS message and accessing a Derby datasource suitably
>embedded in a Hibernate session. I can provide some code examples that will
>help reproduce the problem. I've invested a lot of time in getting
>Hibernate, Session beans, MDBs and datasources to co-exist peacefully (at
>least for DB2). So at least that was worth the effort.
>
>I'd be very grateful if someone can help with this. I would really like to
>use Derby exclusively as my development rdbms. I rather like the idea of a
>RDBMS written in Java.
>Regards
>Johan
>
>-----Original Message-----
>From: Stanley Bradbury [mailto:Stan.Bradbury@gmail.com] 
>Sent: Wednesday, September 21, 2005 2:09 AM
>To: Derby Discussion
>Subject: Re: Derby XA, Hibernate 2.1.6, WebSphere 5.1 - Server returned
>XAER_NOTA at commit time
>
>
>Hi Johan -
>I had hoped to find more for you on either the hibernate or the 
>WebSphere site but only found one reference to this error.  This problem 
>report suggests that a 'fix' may be available from IBM (this is a Tivoli 
>product  but Tivoli uses WebSpere and this did happen in the same class 
>indicated in your trace:   
>   com.ibm.ws.rsadapter.spi.WSRdbManagedConnection).
>
> It also sounds like the problem was related to threads not being in 
>sync (..use more thread-safe functions).  I think the next step in 
>resolving this would be to turn on tracing in WebSphere to see if  the 
>additional information indicates why there is still an open 
>transaction.  Here is the URL to review  the description problem 
>description.
>
>http://www-1.ibm.com/support/docview.wss?rs=493&context=SWK90&uid=swg1IY718
8
>6
>
>  
>
Hi Johan -

I'm not sure where to start so will say I can understand your 
frustration and agree that bringing together an elite team from Derby, 
Hibernate and WebSphere development would be the quickest way to achieve 
a resolution to this issue.  In my experience this is not a simple thing 
to achieve and next to impossible without a Support contract for at 
least one of the products.  In this case the main product involved is 
the transaction manager (TM) within WAS so this is probably the key 
expertise needed - can you initiate a Support case with WAS support?  
The TM is the main component and does appear to be reporting a problem 
(Cannot call 'cleanup' on a ManagedConnection while it is still in a 
transaction) - the question being 'who is calling cleanup and why?'.   I 
believe that obtaining traces from the TM is required to resolve this 
issue - if nothing else WAS support can help in the selection of the 
proper trace flags and in understanding the potentially copius output. 

Regarding the trace information I recommend setting transaction tracing, 
if possible, and searching for occurances of the classes involved in the 
exceptions.  In this case I would be looking for :

  com.ibm.ws.rsadapter.spi.WSRdbManagedConnectionImpl.cleanupTransactions
  com.ibm.ejs.j2c.MCWrapper.cleanup
  com.ibm.ejs.j2c.MCWrapper
  com.ibm.ejs.j2c.poolmanager.FreePool.returnToFreePool

 This is because the "Cannot call 'cleanup' on a ManagedConnection while 
it is still in a transaction" message is listed prior to the "XAER_NOTA" 
and so I think preceeds it .  My assumption is that the logging system 
reports error in the order they occur and the order they are listed is 
more significant than the timevalue when the timevalues are close.  Are 
your reading this as the 'XAER' came before the 'cleanup'?  I agree that 
what comes first is usually the more significant.

Lastly, I talked to a developer that worked on the Derby ClientDriver XA 
implementation and she made the following observations.

    The driver has been tested with WAS 6 and is working fine in those 
tests.  WAS 5.1 was never tested with the Derby Client Driver - all 
testing was with Cloudscape 5.1 - she was wondering if you could move 
your system to WAS 6 to see if the problem happens with the current 
WebSphere codeline.   She also mentioned some XA fixes that went into 
the Derby codeline recently and you might try a recent nightly build to 
see it helps with this problem.

Hope this helps.




RE: Derby XA, Hibernate 2.1.6, WebSphere 5.1 - Server returned XAER_NOTA at commit time

Posted by Johan Hoogenboezem <ho...@worldonline.co.za>.
Thank you for your contribution. Especially the links. Stanley suggested I
try WAS 6 as apparently that works with Derby and XA. The tracing I
mentioned was the WAS tracing. I'll certainly try the Derby tracing too. I
agree it would be ideal to reproduce the error using Derby only. However, I
should mention I found no errors with WAS and Derby and XA as long as the
distributed transaction spanned databases only. For example, I successfully
committed and rolled back changes across two Derby databases in one
transaction. The error appears when one of the XA partners is JMS (WebSphere
MQ). Before you jump to conclusions, it works fine with DB2 and MaxDB, so it
seems as if WebSphere and/or WebSphere MQ are not at fault here.

Johan

-----Original Message-----
From: Kathey Marsden [mailto:kmarsdenderby@sbcglobal.net] 
Sent: Monday, October 17, 2005 6:02 PM
To: Derby Discussion
Cc: 'Stanley Bradbury'
Subject: Re: Derby XA, Hibernate 2.1.6, WebSphere 5.1 - Server returned
XAER_NOTA at commit time


Johan Hoogenboezem wrote:

>Hi Stanley
>1. I will try to initiate a support case, but in my experience they drag
>their feet if they think the problem does not lie with WAS. They are also
>very likely to tell me to move to WAS 6.
>2. Like I said before, the trace logs are meaningless to me. And you are
>mistaken about the cleanup happening before XAERR_NOTA. Look at my first
>email.
>3. I'll try to get hold of WAS 6. I won't be able to pursue this avenue if
>it cannot co-exist with WAS 5, since it takes me more than a day to set up
>everything we require in WAS 5.
>  
>
I haven't been following this thread, but  I don' think the Derby client
driver is shipped with WAS 6, and I could be wrong but I  don't think
there is a released  version of WAS  that includes the client driver, so
you are a bit of a trail blazer here.  That said,  if there is a client
XA bug it would be really good to track it down.  Probably the best bet
is for you to try to get a reproducible case that includes just Derby 
that you can submit in Jira. . One thing  that might help you here is 
the client tracing.  You can turn this on by using the setTraceFile
method on the ClientDataSource. If these are the tracelogs that you
talked about being meaningless to you, then please take a closer look
and come up with some specific quetsions about the tracing and I am sure
folks here on the list would be happy to answer your questions.  The
tracing is documented at:
http://db.apache.org/derby/docs/10.1/adminguide/cadminappsclienttracing.html
.
You could turn off  TRACE_PROTOCOL_FLOWS to reduce the trace volume of
the trace output and make it more manageable.

If you figure out the problem sequence of XA calls you can either write
a standalone java program or use the internal ij xa scripting language
that is used for the xa tests to write a reproducible case that you can
submit.

Good XA references if you are new to XA are:

The Sun Java Transaction API specification
http://java.sun.com/products/jta/index.html

The XA+ specification.
http://www.opengroup.org/publications/catalog/s423.htm


Kathey











Re: Derby XA, Hibernate 2.1.6, WebSphere 5.1 - Server returned XAER_NOTA at commit time

Posted by Kathey Marsden <km...@sbcglobal.net>.
Johan Hoogenboezem wrote:

>Hi Stanley
>1. I will try to initiate a support case, but in my experience they drag
>their feet if they think the problem does not lie with WAS. They are also
>very likely to tell me to move to WAS 6.
>2. Like I said before, the trace logs are meaningless to me. And you are
>mistaken about the cleanup happening before XAERR_NOTA. Look at my first
>email.
>3. I'll try to get hold of WAS 6. I won't be able to pursue this avenue if
>it cannot co-exist with WAS 5, since it takes me more than a day to set up
>everything we require in WAS 5.
>  
>
I haven't been following this thread, but  I don' think the Derby client
driver is shipped with WAS 6, and I could be wrong but I  don't think
there is a released  version of WAS  that includes the client driver, so
you are a bit of a trail blazer here.  That said,  if there is a client
XA bug it would be really good to track it down.  Probably the best bet
is for you to try to get a reproducible case that includes just Derby 
that you can submit in Jira. . One thing  that might help you here is 
the client tracing.  You can turn this on by using the setTraceFile
method on the ClientDataSource. If these are the tracelogs that you
talked about being meaningless to you, then please take a closer look
and come up with some specific quetsions about the tracing and I am sure
folks here on the list would be happy to answer your questions.  The
tracing is documented at:
http://db.apache.org/derby/docs/10.1/adminguide/cadminappsclienttracing.html.
You could turn off  TRACE_PROTOCOL_FLOWS to reduce the trace volume of
the trace output and make it more manageable.

If you figure out the problem sequence of XA calls you can either write
a standalone java program or use the internal ij xa scripting language
that is used for the xa tests to write a reproducible case that you can
submit.

Good XA references if you are new to XA are:

The Sun Java Transaction API specification
http://java.sun.com/products/jta/index.html

The XA+ specification.
http://www.opengroup.org/publications/catalog/s423.htm


Kathey









RE: Derby XA, Hibernate 2.1.6, WebSphere 5.1 - Server returned XAER_NOTA at commit time

Posted by Johan Hoogenboezem <ho...@worldonline.co.za>.
Hi Stanley
1. I will try to initiate a support case, but in my experience they drag
their feet if they think the problem does not lie with WAS. They are also
very likely to tell me to move to WAS 6.
2. Like I said before, the trace logs are meaningless to me. And you are
mistaken about the cleanup happening before XAERR_NOTA. Look at my first
email.
3. I'll try to get hold of WAS 6. I won't be able to pursue this avenue if
it cannot co-exist with WAS 5, since it takes me more than a day to set up
everything we require in WAS 5.
4. I'll try the latest Derby nightly build first. 
Regards
Johan


-----Original Message-----
From: Stanley Bradbury [mailto:Stan.Bradbury@gmail.com] 
Sent: Monday, October 17, 2005 12:19 AM
To: Johan Hoogenboezem
Cc: derby-user@db.apache.org
Subject: Re: Derby XA, Hibernate 2.1.6, WebSphere 5.1 - Server returned
XAER_NOTA at commit time


Johan Hoogenboezem wrote:

>Hi Stanley
>Thank you for your suggestion. I had a look at the link you sent me and
made
>sure I had the latest fixpack (for WSAD 5.1.2)loaded. It had no effect on
>the problem. 
>I feel I should point out that the error you latched on to was one of the
>last ones listed. In my experience the first error in a cascade of errors
is
>usually the important one:
>
>  
>
>>[9/13/05 8:23:49:819 CAT] 1e59f386 WSRdbXaResour E DSRA0304E:  XAException
>>occurred. XAException contents and details are: The cause is
>>    
>>
>:
>  
>
>>org.apache.derby.client.am.SqlException: Error executing a
>>XAResource.commit(), Server returned XAER_NOTA.
>>
>>[9/13/05 8:23:49:819 CAT] 1e59f386 WSRdbXaResour E DSRA0302E:  XAException
>>occurred.  Error code is: XAER_NOTA.  Exception is: XAER_NOTA : Error
>>executing a XAResource.commit(), Server returned XAER_NOTA
>>
>>[9/13/05 8:23:49:889 CAT] 1e59f386 XATransaction E J2CA0027E: An exception
>>occurred while invoking commit on an XA Resource Adapter from dataSource
>>jdbc/derby/scvwebdev, within transaction ID {XID: formatId(57415344),
>>gtrid_length(39), bqual_length(28),
>>data(0000000000000002000000044c4c40cbd49eaa1530e4b7146f080d8853876c7a73657
2
>>    
>>
>7
>  
>
>>66572314c4c40cbd49eaa1530e4b7146f080d8853876c7a000000048333bb35)}:
>>org.apache.derby.client.am.XaException: XAER_NOTA : Error executing a
>>XAResource.commit(), Server returned XAER_NOTA
>>	at
>>org.apache.derby.client.net.NetXAResource.throwXAException(Unknown Source)
>>	at org.apache.derby.client.net.NetXAResource.commit(Unknown Source)
>>	at
>>...
>>etc. etc. etc.
>>    
>>
>
>The later errors are usually side effects which go away when the real
>problem is solved. From analyzing the javadoc of the standard JTA api's it
>is clear that if the "destroy" method is called when a transaction is not
in
>an "idle" state, then further errors will be generated. The container calls
>the destroy method, even though the commit resulted in an error, so further
>errors are almost assured.
>
>In any case, I tried to enable tracing as you suggested - and quickly got
>lost in the quagmire of log files and more log files. I am completely lost
>here as I have no documentation that can explain to me what the trace
means.
>I can provide the trace log if anyone wants it, though.
>
>I then resorted to re-building Derby with some debug statements. I was
>flying blind here, too, which is what one does when desperate. I also get
>nowhere, having no knowledge of the inner workings of a JTA implementation.
>
>So, it seems I'm stuck.
>
>For me, it seems to be a pretty simple matter of a derby developer
>(preferably the guy/girl who worked on the XA stuff :-)) to get together
>with an IBM WebSphere (v5.1.x) guy (IBM donates development time to Derby,
>right?), and then for this elite team of two to knock up a quick example of
>responding to a JMS message and accessing a Derby datasource suitably
>embedded in a Hibernate session. I can provide some code examples that will
>help reproduce the problem. I've invested a lot of time in getting
>Hibernate, Session beans, MDBs and datasources to co-exist peacefully (at
>least for DB2). So at least that was worth the effort.
>
>I'd be very grateful if someone can help with this. I would really like to
>use Derby exclusively as my development rdbms. I rather like the idea of a
>RDBMS written in Java.
>Regards
>Johan
>
>-----Original Message-----
>From: Stanley Bradbury [mailto:Stan.Bradbury@gmail.com] 
>Sent: Wednesday, September 21, 2005 2:09 AM
>To: Derby Discussion
>Subject: Re: Derby XA, Hibernate 2.1.6, WebSphere 5.1 - Server returned
>XAER_NOTA at commit time
>
>
>Hi Johan -
>I had hoped to find more for you on either the hibernate or the 
>WebSphere site but only found one reference to this error.  This problem 
>report suggests that a 'fix' may be available from IBM (this is a Tivoli 
>product  but Tivoli uses WebSpere and this did happen in the same class 
>indicated in your trace:   
>   com.ibm.ws.rsadapter.spi.WSRdbManagedConnection).
>
> It also sounds like the problem was related to threads not being in 
>sync (..use more thread-safe functions).  I think the next step in 
>resolving this would be to turn on tracing in WebSphere to see if  the 
>additional information indicates why there is still an open 
>transaction.  Here is the URL to review  the description problem 
>description.
>
>http://www-1.ibm.com/support/docview.wss?rs=493&context=SWK90&uid=swg1IY718
8
>6
>
>  
>
Hi Johan -

I'm not sure where to start so will say I can understand your 
frustration and agree that bringing together an elite team from Derby, 
Hibernate and WebSphere development would be the quickest way to achieve 
a resolution to this issue.  In my experience this is not a simple thing 
to achieve and next to impossible without a Support contract for at 
least one of the products.  In this case the main product involved is 
the transaction manager (TM) within WAS so this is probably the key 
expertise needed - can you initiate a Support case with WAS support?  
The TM is the main component and does appear to be reporting a problem 
(Cannot call 'cleanup' on a ManagedConnection while it is still in a 
transaction) - the question being 'who is calling cleanup and why?'.   I 
believe that obtaining traces from the TM is required to resolve this 
issue - if nothing else WAS support can help in the selection of the 
proper trace flags and in understanding the potentially copius output. 

Regarding the trace information I recommend setting transaction tracing, 
if possible, and searching for occurances of the classes involved in the 
exceptions.  In this case I would be looking for :

  com.ibm.ws.rsadapter.spi.WSRdbManagedConnectionImpl.cleanupTransactions
  com.ibm.ejs.j2c.MCWrapper.cleanup
  com.ibm.ejs.j2c.MCWrapper
  com.ibm.ejs.j2c.poolmanager.FreePool.returnToFreePool

 This is because the "Cannot call 'cleanup' on a ManagedConnection while 
it is still in a transaction" message is listed prior to the "XAER_NOTA" 
and so I think preceeds it .  My assumption is that the logging system 
reports error in the order they occur and the order they are listed is 
more significant than the timevalue when the timevalues are close.  Are 
your reading this as the 'XAER' came before the 'cleanup'?  I agree that 
what comes first is usually the more significant.

Lastly, I talked to a developer that worked on the Derby ClientDriver XA 
implementation and she made the following observations.

    The driver has been tested with WAS 6 and is working fine in those 
tests.  WAS 5.1 was never tested with the Derby Client Driver - all 
testing was with Cloudscape 5.1 - she was wondering if you could move 
your system to WAS 6 to see if the problem happens with the current 
WebSphere codeline.   She also mentioned some XA fixes that went into 
the Derby codeline recently and you might try a recent nightly build to 
see it helps with this problem.

Hope this helps.




Re: Derby XA, Hibernate 2.1.6, WebSphere 5.1 - Server returned XAER_NOTA at commit time

Posted by Stanley Bradbury <St...@gmail.com>.
Johan Hoogenboezem wrote:

>Hi Stanley
>Thank you for your suggestion. I had a look at the link you sent me and made
>sure I had the latest fixpack (for WSAD 5.1.2)loaded. It had no effect on
>the problem. 
>I feel I should point out that the error you latched on to was one of the
>last ones listed. In my experience the first error in a cascade of errors is
>usually the important one:
>
>  
>
>>[9/13/05 8:23:49:819 CAT] 1e59f386 WSRdbXaResour E DSRA0304E:  XAException
>>occurred. XAException contents and details are: The cause is
>>    
>>
>:
>  
>
>>org.apache.derby.client.am.SqlException: Error executing a
>>XAResource.commit(), Server returned XAER_NOTA.
>>
>>[9/13/05 8:23:49:819 CAT] 1e59f386 WSRdbXaResour E DSRA0302E:  XAException
>>occurred.  Error code is: XAER_NOTA.  Exception is: XAER_NOTA : Error
>>executing a XAResource.commit(), Server returned XAER_NOTA
>>
>>[9/13/05 8:23:49:889 CAT] 1e59f386 XATransaction E J2CA0027E: An exception
>>occurred while invoking commit on an XA Resource Adapter from dataSource
>>jdbc/derby/scvwebdev, within transaction ID {XID: formatId(57415344),
>>gtrid_length(39), bqual_length(28),
>>data(0000000000000002000000044c4c40cbd49eaa1530e4b7146f080d8853876c7a736572
>>    
>>
>7
>  
>
>>66572314c4c40cbd49eaa1530e4b7146f080d8853876c7a000000048333bb35)}:
>>org.apache.derby.client.am.XaException: XAER_NOTA : Error executing a
>>XAResource.commit(), Server returned XAER_NOTA
>>	at
>>org.apache.derby.client.net.NetXAResource.throwXAException(Unknown Source)
>>	at org.apache.derby.client.net.NetXAResource.commit(Unknown Source)
>>	at
>>...
>>etc. etc. etc.
>>    
>>
>
>The later errors are usually side effects which go away when the real
>problem is solved. From analyzing the javadoc of the standard JTA api's it
>is clear that if the "destroy" method is called when a transaction is not in
>an "idle" state, then further errors will be generated. The container calls
>the destroy method, even though the commit resulted in an error, so further
>errors are almost assured.
>
>In any case, I tried to enable tracing as you suggested - and quickly got
>lost in the quagmire of log files and more log files. I am completely lost
>here as I have no documentation that can explain to me what the trace means.
>I can provide the trace log if anyone wants it, though.
>
>I then resorted to re-building Derby with some debug statements. I was
>flying blind here, too, which is what one does when desperate. I also get
>nowhere, having no knowledge of the inner workings of a JTA implementation.
>
>So, it seems I'm stuck.
>
>For me, it seems to be a pretty simple matter of a derby developer
>(preferably the guy/girl who worked on the XA stuff :-)) to get together
>with an IBM WebSphere (v5.1.x) guy (IBM donates development time to Derby,
>right?), and then for this elite team of two to knock up a quick example of
>responding to a JMS message and accessing a Derby datasource suitably
>embedded in a Hibernate session. I can provide some code examples that will
>help reproduce the problem. I've invested a lot of time in getting
>Hibernate, Session beans, MDBs and datasources to co-exist peacefully (at
>least for DB2). So at least that was worth the effort.
>
>I'd be very grateful if someone can help with this. I would really like to
>use Derby exclusively as my development rdbms. I rather like the idea of a
>RDBMS written in Java.
>Regards
>Johan
>
>-----Original Message-----
>From: Stanley Bradbury [mailto:Stan.Bradbury@gmail.com] 
>Sent: Wednesday, September 21, 2005 2:09 AM
>To: Derby Discussion
>Subject: Re: Derby XA, Hibernate 2.1.6, WebSphere 5.1 - Server returned
>XAER_NOTA at commit time
>
>
>Hi Johan -
>I had hoped to find more for you on either the hibernate or the 
>WebSphere site but only found one reference to this error.  This problem 
>report suggests that a 'fix' may be available from IBM (this is a Tivoli 
>product  but Tivoli uses WebSpere and this did happen in the same class 
>indicated in your trace:   
>   com.ibm.ws.rsadapter.spi.WSRdbManagedConnection).
>
> It also sounds like the problem was related to threads not being in 
>sync (..use more thread-safe functions).  I think the next step in 
>resolving this would be to turn on tracing in WebSphere to see if  the 
>additional information indicates why there is still an open 
>transaction.  Here is the URL to review  the description problem 
>description.
>
>http://www-1.ibm.com/support/docview.wss?rs=493&context=SWK90&uid=swg1IY7188
>6
>
>  
>
Hi Johan -

I'm not sure where to start so will say I can understand your 
frustration and agree that bringing together an elite team from Derby, 
Hibernate and WebSphere development would be the quickest way to achieve 
a resolution to this issue.  In my experience this is not a simple thing 
to achieve and next to impossible without a Support contract for at 
least one of the products.  In this case the main product involved is 
the transaction manager (TM) within WAS so this is probably the key 
expertise needed - can you initiate a Support case with WAS support?  
The TM is the main component and does appear to be reporting a problem 
(Cannot call 'cleanup' on a ManagedConnection while it is still in a 
transaction) - the question being 'who is calling cleanup and why?'.   I 
believe that obtaining traces from the TM is required to resolve this 
issue - if nothing else WAS support can help in the selection of the 
proper trace flags and in understanding the potentially copius output. 

Regarding the trace information I recommend setting transaction tracing, 
if possible, and searching for occurances of the classes involved in the 
exceptions.  In this case I would be looking for :

  com.ibm.ws.rsadapter.spi.WSRdbManagedConnectionImpl.cleanupTransactions
  com.ibm.ejs.j2c.MCWrapper.cleanup
  com.ibm.ejs.j2c.MCWrapper
  com.ibm.ejs.j2c.poolmanager.FreePool.returnToFreePool

 This is because the "Cannot call 'cleanup' on a ManagedConnection while 
it is still in a transaction" message is listed prior to the "XAER_NOTA" 
and so I think preceeds it .  My assumption is that the logging system 
reports error in the order they occur and the order they are listed is 
more significant than the timevalue when the timevalues are close.  Are 
your reading this as the 'XAER' came before the 'cleanup'?  I agree that 
what comes first is usually the more significant.

Lastly, I talked to a developer that worked on the Derby ClientDriver XA 
implementation and she made the following observations.

    The driver has been tested with WAS 6 and is working fine in those 
tests.  WAS 5.1 was never tested with the Derby Client Driver - all 
testing was with Cloudscape 5.1 - she was wondering if you could move 
your system to WAS 6 to see if the problem happens with the current 
WebSphere codeline.   She also mentioned some XA fixes that went into 
the Derby codeline recently and you might try a recent nightly build to 
see it helps with this problem.

Hope this helps.