Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?

[Ma...@wellsfargo.com]

I am in the process of refactoring an existing JPA/Eclipselink-based application to introduce Eclipselink's partitioning support for database sharding. My current application runs quite well with pooled jdbc connections via boncep.

However, with the sharded model there will be a number of entity types that are Replicated to all shards and, as such, eclipselink will be doing writes that span multiple jdbc connections. Per the eclipselink recommendations for paritioning I would like to introduce JTA in order to enforce integrity across commits that hit multiple partitions.

It seems like the ops4j-pax-jdbc bundles are designed to support XA/JTA and it seems like there is a lot of XA support in the various aries bundles (aries.transaction. and aries.jpa.). However, I'm not sure of what combination of these bundles+configuration will provide what I am after. Since many of these bundles seem to be designed/documented around general JTA usage - like performing a transaction that spans JMS and JDBC - I'm not confident if that model works with my particular use case.

To further add to this, I would like for my JDBC connections to be pooled for performance. ops4j-pax-jdbc-pool-dbcp2 seems like a logical choice, but again I'm not sure if it will play nicely with the transaction management features from aries.

I think I have a working model of this, but I'm rather new to XA/JTA especially from OSGi so I want to make sure I'm getting this correct. So, I guess my questions are:

1. Can ops4j-pax-jdbc-pool and it's dbcp2 sibling work with aries-jpa/transaction to pool and auto-enlist XADataSources?

2. If not, will ops4j-pax-jdbc-pool-aries work? And if so, can its pooling semantics be configured? E.g. minIdle, maxIdle, etc.

3. If neither of the above works, is there a recommended set of bundles to accomplish pooled XADataSources with eclipselink+JTA?


Thanks,


Matthew Pitts
 
Developer
Security Solutions Design & Automation
 
Wells Fargo Bank | Tel 336.608.3332 | Cell 336.202.3913 | Kernersville, NC | MAC D9693-010
matthew.w.pitts@wellsfargo.com
 

RE: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?

[Ma...@wellsfargo.com]

Ok, maybe I'm doing more than I need to do there. I do have the EL adapter bundle, but I guess for some reason I thought that I needed to do the target-server bit also. I will see if my results are any different if I remove the target-server declaration.

Thanks,
-matt

-----Original Message-----
From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of Christian Schneider
Sent: Friday, October 09, 2015 8:59 AM
To: Pitts, Matthew W.
Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?

Oh .. I guess you are kind of doing the right thing but there is an
existing module for it.

See
https://github.com/apache/aries/tree/trunk/jpa/jpa-container-eclipselink-adapter.

The problem is that eclipselink unlike openjpa and hibernate is not OSGi
ready per se. Kind of strange as they are OSGi in eclipse all over the
place.
So aries jpa has an adapter for eclipselink that does the necessary steps.

So the solution is to simply install the adapter. Then you own bundles
do not have to do anything special.

In karaf you just need to install the eclipselink feature.

Christian



On 09.10.2015 14:22, Matthew.W.Pitts@wellsfargo.com wrote:
> I don't know the exact reference, but I did this based on some examples I
> had seen. The aries.jpa.eclipselink.adapter bundle comes with classes
> (OSGiTSWrapper and OSGiTSServer) and I had seen some examples of linking a
> persistence unit to use them via the 'eclipselink.target-server' property.
>
> So, are you saying this doesn't need to be done at all? If not, then how
> does EL become aware of the transaction manager?
>
> Is it possible that because I am doing this *and* using eclipselink's own
> Partitioning/Replication support that I am indeed getting XA-enlisted
> datasources without the aries wrappers?
>
> Thanks again for all your help.
>
> v/r
> -matt
>
> -----Original Message-----
> From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
> Christian Schneider
> Sent: Friday, October 09, 2015 4:00 AM
> To: Pitts, Matthew W.
> Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink
> Partitioning?
>
> I totally did not get why you need the code below.
> When you use Aries JPA then Eclipselink should be working with the
> TransactionManager without any special code from your side.
>
> It should also not be necessary that you use any Aries dependencies in
> your user bundles. The only case where it is needed is when you want to
> use the JpaTemplate with DS.
>
> Christian
>
>
>
> On 08.10.2015 22:12, Matthew.W.Pitts@wellsfargo.com wrote:
>> I'll see if I can put together a sample app replicating this. As for the
>> eclipselink enlistment, if it helps to know, I am making EL 'aware' of the
>> OSGI TransactionManager by specifying a 'target-platform' that acquires
> the
>> service, like in [1]. I think this is pretty standard practice though with
>> aries-jpa. I just do it from my own classes so that my actual persistence
>> unit bundles don't have to import aries-specific classes.
>>
>> [1]
>> public class OSGiTSWrapper extends JTATransactionController
>> {
>>
>>     @Override
>>     protected TransactionManager acquireTransactionManager() throws
> Exception
>>     {
>>       BundleContext ctx =
>> FrameworkUtil.getBundle(OSGiTSWrapper.class).getBundleContext();
>>
>>       if (ctx != null)
>>       {
>>         ServiceReference ref =
>> ctx.getServiceReference(TransactionManager.class.getName());
>>
>>         if (ref != null)
>>         {
>>           TransactionManager manager = (TransactionManager)
>> ctx.getService(ref);
>>           return manager;
>>         }
>>       }
>>
>>       return super.acquireTransactionManager();
>>     }
>>
>> }
>>


--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com

RE: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?

[Ma...@wellsfargo.com]

I don't know the exact reference, but I did this based on some examples I
had seen. The aries.jpa.eclipselink.adapter bundle comes with classes
(OSGiTSWrapper and OSGiTSServer) and I had seen some examples of linking a
persistence unit to use them via the 'eclipselink.target-server' property.

So, are you saying this doesn't need to be done at all? If not, then how
does EL become aware of the transaction manager?

Is it possible that because I am doing this *and* using eclipselink's own
Partitioning/Replication support that I am indeed getting XA-enlisted
datasources without the aries wrappers?

Thanks again for all your help.

v/r
-matt

-----Original Message-----
From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
Christian Schneider
Sent: Friday, October 09, 2015 4:00 AM
To: Pitts, Matthew W.
Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink
Partitioning?

I totally did not get why you need the code below.
When you use Aries JPA then Eclipselink should be working with the 
TransactionManager without any special code from your side.

It should also not be necessary that you use any Aries dependencies in 
your user bundles. The only case where it is needed is when you want to 
use the JpaTemplate with DS.

Christian



On 08.10.2015 22:12, Matthew.W.Pitts@wellsfargo.com wrote:
> I'll see if I can put together a sample app replicating this. As for the
> eclipselink enlistment, if it helps to know, I am making EL 'aware' of the
> OSGI TransactionManager by specifying a 'target-platform' that acquires
the
> service, like in [1]. I think this is pretty standard practice though with
> aries-jpa. I just do it from my own classes so that my actual persistence
> unit bundles don't have to import aries-specific classes.
>
> [1]
> public class OSGiTSWrapper extends JTATransactionController
> {
>
>    @Override
>    protected TransactionManager acquireTransactionManager() throws
Exception
>    {
>      BundleContext ctx =
> FrameworkUtil.getBundle(OSGiTSWrapper.class).getBundleContext();
>
>      if (ctx != null)
>      {
>        ServiceReference ref =
> ctx.getServiceReference(TransactionManager.class.getName());
>
>        if (ref != null)
>        {
>          TransactionManager manager = (TransactionManager)
> ctx.getService(ref);
>          return manager;
>        }
>      }
>
>      return super.acquireTransactionManager();
>    }
>
> }
>

-- 
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com

RE: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?

[Ma...@wellsfargo.com]

I'll see if I can put together a sample app replicating this. As for the
eclipselink enlistment, if it helps to know, I am making EL 'aware' of the
OSGI TransactionManager by specifying a 'target-platform' that acquires the
service, like in [1]. I think this is pretty standard practice though with
aries-jpa. I just do it from my own classes so that my actual persistence
unit bundles don't have to import aries-specific classes.

[1]
public class OSGiTSWrapper extends JTATransactionController
{

  @Override
  protected TransactionManager acquireTransactionManager() throws Exception
  {
    BundleContext ctx =
FrameworkUtil.getBundle(OSGiTSWrapper.class).getBundleContext();

    if (ctx != null)
    {
      ServiceReference ref =
ctx.getServiceReference(TransactionManager.class.getName());

      if (ref != null)
      {
        TransactionManager manager = (TransactionManager)
ctx.getService(ref);
        return manager;
      }
    }

    return super.acquireTransactionManager();
  }

}
-----Original Message-----
From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
Christian Schneider
Sent: Wednesday, October 07, 2015 12:55 AM
To: user@aries.apache.org
Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink
Partitioning?

This is a complicated theme which is difficult to diagnose just from the 
outside.

I guess your problem already occurs when you have only one connection. 
Can you create a small example at github
that shows how you setup your application?

Using the non wrapped connections can work if some other layer is doing 
the enlistment (e.g. eclipselink) but I am not sure.
I am still wondering why I do not see the tranql problems in my tests.

Christian

Am 07.10.2015 um 00:15 schrieb Matthew.W.Pitts@wellsfargo.com:
> Once again, thanks very much for the input and sorry for the late
> response....
>
> Here's where I am at... I was pushing forward with just using the aries
> pooling bundle, however it seemed to *not* do what I would normally expect
> with connections after a query/exception problem. It would leave them in
the
> pool and allow for their reuse by subsequent queries. This would then lead
> to 'Connection already closed' or 'communication link failure' type
messages
> in subsequent JPA operations unrelated to the previous problem.
>
> With this, I tried once again to work with the PAX dbcp2 bundle and ran
into
> the same problems I had before with the tranql classes calling methods on
> the wrapped connection that DBCP2 did not like. I went so far as to
comment
> some of the parts of the tranql ManagedJDBCConnection class to prevent it
> calling down into the DBCP2 connection. When this happened, my
> queries/transactions would continue on but then fail during another method
> call (like #commit) where the DBCP2 connection would throw an exception
> complaining about not being able to perform certain operations when
> participating in a managed transaction. So, it seemed to me like my DBCP2
> connections were already XA aware/enlisted (or whatever you call it) and
> that the higher-level stuff in aries-jdbc/tranql was incompatible with
them.
>
> So, here is where it gets really interesting... thinking about this
problem
> I wondered if maybe I didn't even need the aries-wrapped datasources. So,
I
> changed my eclipselink/jpa JNDI lookups to be !(aries.managed=true), to
> hopefully force eclipselink to use the DBCP2 XA datasource services
> directly. Well, voila... this seems to not only work, but still provide
the
> xa/jta management as well. To test this I created a conflicting entity row
> in only one of my three eclipselink partitions or 'nodes'. Attempting to
> persist the corresponding entity in osgi via my jpa classes failed with
the
> expected exception indicating a conflicting row *and* inspecting the
> database nodes showed that the only node with the row was the original
node
> where it was manually inserted for the test. So, I'm assuming this is an
> indicator that the XA stuff ran as expected and prevented the insert from
> propagating to other nodes because the coordinated transaction was rolled
> back.
>
> So, my question is - does this setup make sense? To have the aries
> transaction manager/bundles in-place but not actually use the
aries-managed
> datasources? Perhaps there are facilities in the pax dbcp2 pool bundle
that
> does the auto-enlistment already? Perhaps I have no idea what I am talking
> about?
>
> Any guidance is greatly appreciated. Thanks.
>
> -matt
>
> -----Original Message-----
> From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
> Christian Schneider
> Sent: Wednesday, September 23, 2015 1:53 AM
> To: user@aries.apache.org
> Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink
> Partitioning?
>
> Not sure why you hit the problem with dbcp2 and eclipselink. I have some
> itests running
> with eclipselink and they showed no problems. Will look into what
> versions I used exactly.
>
> I am not very familiar with the aries pooling but it should work fine.
>
> Christian
>
> Am 23.09.2015 um 02:48 schrieb Matthew.W.Pitts@wellsfargo.com:
>> Ok, thanks so much for the suggestions.
>>
>> It is interesting to me that this isn't something someone else has seen
>> since I feel like I am using the pax-jdbc and aries bundles in a
>> straightforward way.
>>
>> Do you have a recommendation of using the aries pool bundle vs. the dbcp2
>> one? I'm not set on either one, I just need it to work, have some
>> tweakability and perform well enough. I had been leaning towards dbcp2
> since
>> it seems like a reasonable, actively developed pool implementation that I
>> have used elsewhere, but since it doesn't currently work for me I'm fine
>> using aries if it works and performs.
>>
>> It seems to me that the aries pooling basically delegates back to the
>> Geronimo classes, which I would suspect are a decent pool implementation
>> themselves having originated as part of that project. But I have no
>> experience with them, so my confidence is low.
>>
>> Obviously, I'll do some performance tests when all this comes together
> more,
>> but recommended best practices or known working bundle combinations are
>> always a good starting point. I'm still looking through your earlier
>> references and all the itest stuff to try to piece together a logical
> bundle
>> set, but it's slow for me as I am not very familiar with PaxExam.
>>
>> Thanks,
>> -matt
>>
>> -----Original Message-----
>> From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
>> Christian Schneider
>> Sent: Tuesday, September 22, 2015 4:04 PM
>> To: user@aries.apache.org
>> Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink
>> Partitioning?
>>
>> Honestly I have no idea why tranql sets the autocommit. I would not
>> expect that it needs to be tweaked.
>>
>> I can imagine that dbcp2 does not allow to change autocommit for XA
>> transactions as the commit should
>> only be done when the XA transaction commits. Not sure about this either
>> though.
>>
>> I propose you ask this on the apache commons list or open an issue for
>> dbcp2. It definitely seems to be an issue that
>> can also happen when just configuring dbcp2 manually and then use tranql
>> on it.
>>
>> Christian
>>
>> Am 22.09.2015 um 20:19 schrieb Matthew.W.Pitts@wellsfargo.com:
>>> [1] is a snippet from the stacktrace I get when using
pax-jdbc-pool-dbcp2
>>> instead of -aries. As you can see,
>>> ManagedJDBCConnection#localTransactionStart appears to call
setAutoCommit
>> on
>>> the connection class, which the DBCP2 impl short circuits at [2].
>>>
>>> This same exact environment seems to run fine with the -aries pool
> bundle.
>> I
>>> am running this with a customized transactions-jdbc bundle checked-out
>> from
>>> SVN. But, the only change I made was the pull in all the tranql source
to
>>> fix issues with the two-arg form of getConnection in some of those
>> classes.
>>> Aside from that it's unchanged and works fine with the -aries pooling
>>> bundle.
>>>
>>> Any suggestions are greatly appreciated.
>>>
>>> [1]
>>> Caused by: javax.resource.spi.LocalTransactionException: Unable to
> disable
>>> autoCommit
>>> 	at
>>>
>
org.tranql.connector.jdbc.ManagedJDBCConnection.localTransactionStart(Manage
>>> dJDBCConnection.java:83)
>>> ~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
>>> 	at
>>>
>
org.tranql.connector.AbstractManagedConnection$LocalTransactionImpl.begin(Ab
>>> stractManagedConnection.java:195)
>>> ~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
>>> 	at
>>>
>
org.apache.geronimo.connector.outbound.LocalXAResource.start(LocalXAResource
>>> .java:107) ~[geronimo-connector-3.1.3.jar:3.1.3]
>>> 	... 76 common frames omitted
>>> Caused by: java.sql.SQLException: Auto-commit can not be set while
>> enrolled
>>> in a transaction
>>> 	at
>>>
>
org.apache.commons.dbcp2.managed.ManagedConnection.setAutoCommit(ManagedConn
>>> ection.java:223) ~[na:na]
>>> 	at
>>>
>
org.tranql.connector.jdbc.ManagedJDBCConnection.localTransactionStart(Manage
>>> dJDBCConnection.java:81)
>>> ~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
>>> 	... 78 common frames omitted
>>>
>>> [2]
>>> public void setAutoCommit(boolean autoCommit) throws SQLException {
>>>        if (transactionContext != null) {
>>>     
>>> throw new SQLException("Auto-commit can not be set while enrolled in a
>> trans
>>> action");
>>>        }
>>>        super.setAutoCommit(autoCommit);
>>> }
>>>
>>>
>>> -----Original Message-----
>>> From: Matthew.W.Pitts@wellsfargo.com
>> [mailto:Matthew.W.Pitts@wellsfargo.com]
>>> Sent: Tuesday, September 22, 2015 11:14 AM
>>> To: user@aries.apache.org
>>> Subject: RE: Bundle set recommendations for JPA+JTA+Eclipselink
>>> Partitioning?
>>>
>>> Thanks very much for the references, I will definitely be looking over
>> them.
>>> I was hoping the dbcp2 bundle would allow for the same XA capability, as
> I
>>> would prefer dbcp2 pooling since it seems fairly analogous to my current
>>> pooling - bonecp. However, I did some testing with dbcp2 and got
>> exceptions
>>> during runtime when code from base datasource wrappers (from tranql
> maybe)
>>> was attempting to set the auto-commit flag and the DBCP datasource
> wrapper
>>> class was throwing an exception about setting auto-commit. I can't
>> remember
>>> exactly the code flow but I'll try to test this again and see about
>> getting
>>> a stacktrace. Does any of this sound familiar? Should I be able to
simply
>>> replace pax-jdbc-pool-aries with pax-jdbc-pool-dbcp2 in my runtime?
Maybe
>> it
>>> was something in my jndi references not getting the aries-managed
>> datasource
>>> services.
>>>
>>> As for the blueprint/DS question, I will be largely using DS to inject
>>> app-specific API that gives my code access to the
>> JpaTemplate/EntityManager
>>> classes. I hope to move toward blueprint, but that will be another
> effort.
>>> Thanks again,
>>> -matt
>>>
>>> -----Original Message-----
>>> From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
>>> Christian Schneider
>>> Sent: Tuesday, September 22, 2015 10:24 AM
>>> To: user@aries.apache.org
>>> Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink
>>> Partitioning?
>>>
>>> Hi Matthew,
>>>
>>> pax-jdbc-pool-dbcp2 as well as pax-jdbc-pool-aries should both be able
>>> to provide XA enlisting poolable DataSources.
>>> Aries JPA and Transaction actually use the above in their integration
>>> tests. So they should work together nicely.
>>>
>>> I got two tutorials that both use XA transactions with Aries:
>>>
>
http://liquid-reality.de/display/liquid/2015/06/30/Apache+Karaf+Tutorial+par
>>> t+10+-+Declarative+services
>>>
>
http://liquid-reality.de/display/liquid/2015/03/05/Apache+Karaf+Tutorial+Par
>>> t+9+-+Annotation+based+blueprint+and+JPA
>>>
>>> Btw. Do you plan to use blueprint or DS?
>>>
>>> Christian
>>>
>>> On 03.09.2015 22:51, Matthew.W.Pitts@wellsfargo.com wrote:
>>>> Ok, maybe I can answer my own questions here, but if I could get some
>>>> confirmation that would be great. I seem to have a working system
> running
>>>> with ops4j-pax-jdbc-pool-aries rather than dbcp2.
>>>>
>>>> Digging into the code in org.apache.aries.transaction.jdbc it looks
like
>>>> RecoverableDataSource is the wrapper for my XADataSources and it does
>>> indeed
>>>> appear to support passing of pooling options onto
>> ConnectionManagerFactory
>>>> which itself appears to use the pooling capabilities that come from the
>>>> Geronimo bundles. Does all this sound correct so far?
>>>>
>>>> So, it's not dbcp2, but it's pooling nonetheless and I can adjust
>>>> preferences by passing the appropriate bean options to my pax-jdbc
>> service
>>>> with the 'pool.' prefix which will in-turn send those on to the
>>>> RecoverableDataSource instance. Sound ok still?
>>>>
>>>> And finally, my eclipselink JTA datasource JNDI lookups are formatted
>> with
>>>> filters for the 'aries.managed' property [1] to ensure that I get the
>>> aries
>>>> auto-enlisting DataSources.
>>>>
>>>> So far, this is running my existing code ok with only two changes I had
>> to
>>>> make:
>>>>
>>>> 1. Fixed ARIES-1171, but actually in the Tranql code as I was getting
>>>> connection errors and didn't see a logical place to fix it from an
aries
>>>> class (using latest aries-transaction-jdbc from SVN). This required
>>> bringing
>>>> all the tranql source into my aries bundle and just removing the
> external
>>>> dependency. I didn't like doing that, but since that code hasn't
changed
>>> in
>>>> a while and I'm still prototyping all this it was the path of least
>>>> resistance.
>>>>
>>>> 2. Changed RecoverableDataSource to not use empty strings by default
for
>>>> username and password. Which seemed to be similar/related to the above
>> and
>>> I
>>>> posted as ARIES-1376.
>>>>
>>>> Thoughts and criticisms welcome. Thanks!
>>>>
>>>> [1]
>>>>
>
osgi:service/javax.sql.DataSource/(&(osgi.jndi.service.name=my-app-node1)(ar
>>>> ies.managed=true))
>>>>
>>>> -----Original Message-----
>>>> From: Matthew.W.Pitts@wellsfargo.com
>>> [mailto:Matthew.W.Pitts@wellsfargo.com]
>>>> Sent: Thursday, September 03, 2015 10:58 AM
>>>> To: user@aries.apache.org
>>>> Subject: Bundle set recommendations for JPA+JTA+Eclipselink
> Partitioning?
>>>> I am in the process of refactoring an existing JPA/Eclipselink-based
>>>> application to introduce Eclipselink's partitioning support for
database
>>>> sharding. My current application runs quite well with pooled jdbc
>>>> connections via boncep.
>>>>
>>>> However, with the sharded model there will be a number of entity types
>>> that
>>>> are Replicated to all shards and, as such, eclipselink will be doing
>>> writes
>>>> that span multiple jdbc connections. Per the eclipselink
recommendations
>>> for
>>>> paritioning I would like to introduce JTA in order to enforce integrity
>>>> across commits that hit multiple partitions.
>>>>
>>>> It seems like the ops4j-pax-jdbc bundles are designed to support XA/JTA
>>> and
>>>> it seems like there is a lot of XA support in the various aries bundles
>>>> (aries.transaction. and aries.jpa.). However, I'm not sure of what
>>>> combination of these bundles+configuration will provide what I am
after.
>>>> Since many of these bundles seem to be designed/documented around
> general
>>>> JTA usage - like performing a transaction that spans JMS and JDBC - I'm
>>> not
>>>> confident if that model works with my particular use case.
>>>>
>>>> To further add to this, I would like for my JDBC connections to be
> pooled
>>>> for performance. ops4j-pax-jdbc-pool-dbcp2 seems like a logical choice,
>>> but
>>>> again I'm not sure if it will play nicely with the transaction
> management
>>>> features from aries.
>>>>
>>>> I think I have a working model of this, but I'm rather new to XA/JTA
>>>> especially from OSGi so I want to make sure I'm getting this correct.
> So,
>>> I
>>>> guess my questions are:
>>>>
>>>> 1. Can ops4j-pax-jdbc-pool and it's dbcp2 sibling work with
>>>> aries-jpa/transaction to pool and auto-enlist XADataSources?
>>>>
>>>> 2. If not, will ops4j-pax-jdbc-pool-aries work? And if so, can its
>> pooling
>>>> semantics be configured? E.g. minIdle, maxIdle, etc.
>>>>
>>>> 3. If neither of the above works, is there a recommended set of bundles
>> to
>>>> accomplish pooled XADataSources with eclipselink+JTA?
>>>>
>>>>
>>>> Thanks,
>>>>
>>>>
>>>> Matthew Pitts
>>>>      
>>>> Developer
>>>> Security Solutions Design & Automation
>>>>      
>>>> Wells Fargo Bank | Tel 336.608.3332 | Cell 336.202.3913 | Kernersville,
>> NC
>>> |
>>>> MAC D9693-010
>>>> matthew.w.pitts@wellsfargo.com
>>>>      
>>>>

Re: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?

[Christian Schneider <ch...@die-schneider.net>]

This is a complicated theme which is difficult to diagnose just from the 
outside.

I guess your problem already occurs when you have only one connection. 
Can you create a small example at github
that shows how you setup your application?

Using the non wrapped connections can work if some other layer is doing 
the enlistment (e.g. eclipselink) but I am not sure.
I am still wondering why I do not see the tranql problems in my tests.

Christian

Am 07.10.2015 um 00:15 schrieb Matthew.W.Pitts@wellsfargo.com:
> Once again, thanks very much for the input and sorry for the late
> response....
>
> Here's where I am at... I was pushing forward with just using the aries
> pooling bundle, however it seemed to *not* do what I would normally expect
> with connections after a query/exception problem. It would leave them in the
> pool and allow for their reuse by subsequent queries. This would then lead
> to 'Connection already closed' or 'communication link failure' type messages
> in subsequent JPA operations unrelated to the previous problem.
>
> With this, I tried once again to work with the PAX dbcp2 bundle and ran into
> the same problems I had before with the tranql classes calling methods on
> the wrapped connection that DBCP2 did not like. I went so far as to comment
> some of the parts of the tranql ManagedJDBCConnection class to prevent it
> calling down into the DBCP2 connection. When this happened, my
> queries/transactions would continue on but then fail during another method
> call (like #commit) where the DBCP2 connection would throw an exception
> complaining about not being able to perform certain operations when
> participating in a managed transaction. So, it seemed to me like my DBCP2
> connections were already XA aware/enlisted (or whatever you call it) and
> that the higher-level stuff in aries-jdbc/tranql was incompatible with them.
>
> So, here is where it gets really interesting... thinking about this problem
> I wondered if maybe I didn't even need the aries-wrapped datasources. So, I
> changed my eclipselink/jpa JNDI lookups to be !(aries.managed=true), to
> hopefully force eclipselink to use the DBCP2 XA datasource services
> directly. Well, voila... this seems to not only work, but still provide the
> xa/jta management as well. To test this I created a conflicting entity row
> in only one of my three eclipselink partitions or 'nodes'. Attempting to
> persist the corresponding entity in osgi via my jpa classes failed with the
> expected exception indicating a conflicting row *and* inspecting the
> database nodes showed that the only node with the row was the original node
> where it was manually inserted for the test. So, I'm assuming this is an
> indicator that the XA stuff ran as expected and prevented the insert from
> propagating to other nodes because the coordinated transaction was rolled
> back.
>
> So, my question is - does this setup make sense? To have the aries
> transaction manager/bundles in-place but not actually use the aries-managed
> datasources? Perhaps there are facilities in the pax dbcp2 pool bundle that
> does the auto-enlistment already? Perhaps I have no idea what I am talking
> about?
>
> Any guidance is greatly appreciated. Thanks.
>
> -matt
>
> -----Original Message-----
> From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
> Christian Schneider
> Sent: Wednesday, September 23, 2015 1:53 AM
> To: user@aries.apache.org
> Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink
> Partitioning?
>
> Not sure why you hit the problem with dbcp2 and eclipselink. I have some
> itests running
> with eclipselink and they showed no problems. Will look into what
> versions I used exactly.
>
> I am not very familiar with the aries pooling but it should work fine.
>
> Christian
>
> Am 23.09.2015 um 02:48 schrieb Matthew.W.Pitts@wellsfargo.com:
>> Ok, thanks so much for the suggestions.
>>
>> It is interesting to me that this isn't something someone else has seen
>> since I feel like I am using the pax-jdbc and aries bundles in a
>> straightforward way.
>>
>> Do you have a recommendation of using the aries pool bundle vs. the dbcp2
>> one? I'm not set on either one, I just need it to work, have some
>> tweakability and perform well enough. I had been leaning towards dbcp2
> since
>> it seems like a reasonable, actively developed pool implementation that I
>> have used elsewhere, but since it doesn't currently work for me I'm fine
>> using aries if it works and performs.
>>
>> It seems to me that the aries pooling basically delegates back to the
>> Geronimo classes, which I would suspect are a decent pool implementation
>> themselves having originated as part of that project. But I have no
>> experience with them, so my confidence is low.
>>
>> Obviously, I'll do some performance tests when all this comes together
> more,
>> but recommended best practices or known working bundle combinations are
>> always a good starting point. I'm still looking through your earlier
>> references and all the itest stuff to try to piece together a logical
> bundle
>> set, but it's slow for me as I am not very familiar with PaxExam.
>>
>> Thanks,
>> -matt
>>
>> -----Original Message-----
>> From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
>> Christian Schneider
>> Sent: Tuesday, September 22, 2015 4:04 PM
>> To: user@aries.apache.org
>> Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink
>> Partitioning?
>>
>> Honestly I have no idea why tranql sets the autocommit. I would not
>> expect that it needs to be tweaked.
>>
>> I can imagine that dbcp2 does not allow to change autocommit for XA
>> transactions as the commit should
>> only be done when the XA transaction commits. Not sure about this either
>> though.
>>
>> I propose you ask this on the apache commons list or open an issue for
>> dbcp2. It definitely seems to be an issue that
>> can also happen when just configuring dbcp2 manually and then use tranql
>> on it.
>>
>> Christian
>>
>> Am 22.09.2015 um 20:19 schrieb Matthew.W.Pitts@wellsfargo.com:
>>> [1] is a snippet from the stacktrace I get when using pax-jdbc-pool-dbcp2
>>> instead of -aries. As you can see,
>>> ManagedJDBCConnection#localTransactionStart appears to call setAutoCommit
>> on
>>> the connection class, which the DBCP2 impl short circuits at [2].
>>>
>>> This same exact environment seems to run fine with the -aries pool
> bundle.
>> I
>>> am running this with a customized transactions-jdbc bundle checked-out
>> from
>>> SVN. But, the only change I made was the pull in all the tranql source to
>>> fix issues with the two-arg form of getConnection in some of those
>> classes.
>>> Aside from that it's unchanged and works fine with the -aries pooling
>>> bundle.
>>>
>>> Any suggestions are greatly appreciated.
>>>
>>> [1]
>>> Caused by: javax.resource.spi.LocalTransactionException: Unable to
> disable
>>> autoCommit
>>> 	at
>>>
> org.tranql.connector.jdbc.ManagedJDBCConnection.localTransactionStart(Manage
>>> dJDBCConnection.java:83)
>>> ~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
>>> 	at
>>>
> org.tranql.connector.AbstractManagedConnection$LocalTransactionImpl.begin(Ab
>>> stractManagedConnection.java:195)
>>> ~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
>>> 	at
>>>
> org.apache.geronimo.connector.outbound.LocalXAResource.start(LocalXAResource
>>> .java:107) ~[geronimo-connector-3.1.3.jar:3.1.3]
>>> 	... 76 common frames omitted
>>> Caused by: java.sql.SQLException: Auto-commit can not be set while
>> enrolled
>>> in a transaction
>>> 	at
>>>
> org.apache.commons.dbcp2.managed.ManagedConnection.setAutoCommit(ManagedConn
>>> ection.java:223) ~[na:na]
>>> 	at
>>>
> org.tranql.connector.jdbc.ManagedJDBCConnection.localTransactionStart(Manage
>>> dJDBCConnection.java:81)
>>> ~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
>>> 	... 78 common frames omitted
>>>
>>> [2]
>>> public void setAutoCommit(boolean autoCommit) throws SQLException {
>>>        if (transactionContext != null) {
>>>     
>>> throw new SQLException("Auto-commit can not be set while enrolled in a
>> trans
>>> action");
>>>        }
>>>        super.setAutoCommit(autoCommit);
>>> }
>>>
>>>
>>> -----Original Message-----
>>> From: Matthew.W.Pitts@wellsfargo.com
>> [mailto:Matthew.W.Pitts@wellsfargo.com]
>>> Sent: Tuesday, September 22, 2015 11:14 AM
>>> To: user@aries.apache.org
>>> Subject: RE: Bundle set recommendations for JPA+JTA+Eclipselink
>>> Partitioning?
>>>
>>> Thanks very much for the references, I will definitely be looking over
>> them.
>>> I was hoping the dbcp2 bundle would allow for the same XA capability, as
> I
>>> would prefer dbcp2 pooling since it seems fairly analogous to my current
>>> pooling - bonecp. However, I did some testing with dbcp2 and got
>> exceptions
>>> during runtime when code from base datasource wrappers (from tranql
> maybe)
>>> was attempting to set the auto-commit flag and the DBCP datasource
> wrapper
>>> class was throwing an exception about setting auto-commit. I can't
>> remember
>>> exactly the code flow but I'll try to test this again and see about
>> getting
>>> a stacktrace. Does any of this sound familiar? Should I be able to simply
>>> replace pax-jdbc-pool-aries with pax-jdbc-pool-dbcp2 in my runtime? Maybe
>> it
>>> was something in my jndi references not getting the aries-managed
>> datasource
>>> services.
>>>
>>> As for the blueprint/DS question, I will be largely using DS to inject
>>> app-specific API that gives my code access to the
>> JpaTemplate/EntityManager
>>> classes. I hope to move toward blueprint, but that will be another
> effort.
>>> Thanks again,
>>> -matt
>>>
>>> -----Original Message-----
>>> From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
>>> Christian Schneider
>>> Sent: Tuesday, September 22, 2015 10:24 AM
>>> To: user@aries.apache.org
>>> Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink
>>> Partitioning?
>>>
>>> Hi Matthew,
>>>
>>> pax-jdbc-pool-dbcp2 as well as pax-jdbc-pool-aries should both be able
>>> to provide XA enlisting poolable DataSources.
>>> Aries JPA and Transaction actually use the above in their integration
>>> tests. So they should work together nicely.
>>>
>>> I got two tutorials that both use XA transactions with Aries:
>>>
> http://liquid-reality.de/display/liquid/2015/06/30/Apache+Karaf+Tutorial+par
>>> t+10+-+Declarative+services
>>>
> http://liquid-reality.de/display/liquid/2015/03/05/Apache+Karaf+Tutorial+Par
>>> t+9+-+Annotation+based+blueprint+and+JPA
>>>
>>> Btw. Do you plan to use blueprint or DS?
>>>
>>> Christian
>>>
>>> On 03.09.2015 22:51, Matthew.W.Pitts@wellsfargo.com wrote:
>>>> Ok, maybe I can answer my own questions here, but if I could get some
>>>> confirmation that would be great. I seem to have a working system
> running
>>>> with ops4j-pax-jdbc-pool-aries rather than dbcp2.
>>>>
>>>> Digging into the code in org.apache.aries.transaction.jdbc it looks like
>>>> RecoverableDataSource is the wrapper for my XADataSources and it does
>>> indeed
>>>> appear to support passing of pooling options onto
>> ConnectionManagerFactory
>>>> which itself appears to use the pooling capabilities that come from the
>>>> Geronimo bundles. Does all this sound correct so far?
>>>>
>>>> So, it's not dbcp2, but it's pooling nonetheless and I can adjust
>>>> preferences by passing the appropriate bean options to my pax-jdbc
>> service
>>>> with the 'pool.' prefix which will in-turn send those on to the
>>>> RecoverableDataSource instance. Sound ok still?
>>>>
>>>> And finally, my eclipselink JTA datasource JNDI lookups are formatted
>> with
>>>> filters for the 'aries.managed' property [1] to ensure that I get the
>>> aries
>>>> auto-enlisting DataSources.
>>>>
>>>> So far, this is running my existing code ok with only two changes I had
>> to
>>>> make:
>>>>
>>>> 1. Fixed ARIES-1171, but actually in the Tranql code as I was getting
>>>> connection errors and didn't see a logical place to fix it from an aries
>>>> class (using latest aries-transaction-jdbc from SVN). This required
>>> bringing
>>>> all the tranql source into my aries bundle and just removing the
> external
>>>> dependency. I didn't like doing that, but since that code hasn't changed
>>> in
>>>> a while and I'm still prototyping all this it was the path of least
>>>> resistance.
>>>>
>>>> 2. Changed RecoverableDataSource to not use empty strings by default for
>>>> username and password. Which seemed to be similar/related to the above
>> and
>>> I
>>>> posted as ARIES-1376.
>>>>
>>>> Thoughts and criticisms welcome. Thanks!
>>>>
>>>> [1]
>>>>
> osgi:service/javax.sql.DataSource/(&(osgi.jndi.service.name=my-app-node1)(ar
>>>> ies.managed=true))
>>>>
>>>> -----Original Message-----
>>>> From: Matthew.W.Pitts@wellsfargo.com
>>> [mailto:Matthew.W.Pitts@wellsfargo.com]
>>>> Sent: Thursday, September 03, 2015 10:58 AM
>>>> To: user@aries.apache.org
>>>> Subject: Bundle set recommendations for JPA+JTA+Eclipselink
> Partitioning?
>>>> I am in the process of refactoring an existing JPA/Eclipselink-based
>>>> application to introduce Eclipselink's partitioning support for database
>>>> sharding. My current application runs quite well with pooled jdbc
>>>> connections via boncep.
>>>>
>>>> However, with the sharded model there will be a number of entity types
>>> that
>>>> are Replicated to all shards and, as such, eclipselink will be doing
>>> writes
>>>> that span multiple jdbc connections. Per the eclipselink recommendations
>>> for
>>>> paritioning I would like to introduce JTA in order to enforce integrity
>>>> across commits that hit multiple partitions.
>>>>
>>>> It seems like the ops4j-pax-jdbc bundles are designed to support XA/JTA
>>> and
>>>> it seems like there is a lot of XA support in the various aries bundles
>>>> (aries.transaction. and aries.jpa.). However, I'm not sure of what
>>>> combination of these bundles+configuration will provide what I am after.
>>>> Since many of these bundles seem to be designed/documented around
> general
>>>> JTA usage - like performing a transaction that spans JMS and JDBC - I'm
>>> not
>>>> confident if that model works with my particular use case.
>>>>
>>>> To further add to this, I would like for my JDBC connections to be
> pooled
>>>> for performance. ops4j-pax-jdbc-pool-dbcp2 seems like a logical choice,
>>> but
>>>> again I'm not sure if it will play nicely with the transaction
> management
>>>> features from aries.
>>>>
>>>> I think I have a working model of this, but I'm rather new to XA/JTA
>>>> especially from OSGi so I want to make sure I'm getting this correct.
> So,
>>> I
>>>> guess my questions are:
>>>>
>>>> 1. Can ops4j-pax-jdbc-pool and it's dbcp2 sibling work with
>>>> aries-jpa/transaction to pool and auto-enlist XADataSources?
>>>>
>>>> 2. If not, will ops4j-pax-jdbc-pool-aries work? And if so, can its
>> pooling
>>>> semantics be configured? E.g. minIdle, maxIdle, etc.
>>>>
>>>> 3. If neither of the above works, is there a recommended set of bundles
>> to
>>>> accomplish pooled XADataSources with eclipselink+JTA?
>>>>
>>>>
>>>> Thanks,
>>>>
>>>>
>>>> Matthew Pitts
>>>>      
>>>> Developer
>>>> Security Solutions Design & Automation
>>>>      
>>>> Wells Fargo Bank | Tel 336.608.3332 | Cell 336.202.3913 | Kernersville,
>> NC
>>> |
>>>> MAC D9693-010
>>>> matthew.w.pitts@wellsfargo.com
>>>>      
>>>>

RE: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?

[Ma...@wellsfargo.com]

Once again, thanks very much for the input and sorry for the late
response....

Here's where I am at... I was pushing forward with just using the aries
pooling bundle, however it seemed to *not* do what I would normally expect
with connections after a query/exception problem. It would leave them in the
pool and allow for their reuse by subsequent queries. This would then lead
to 'Connection already closed' or 'communication link failure' type messages
in subsequent JPA operations unrelated to the previous problem.

With this, I tried once again to work with the PAX dbcp2 bundle and ran into
the same problems I had before with the tranql classes calling methods on
the wrapped connection that DBCP2 did not like. I went so far as to comment
some of the parts of the tranql ManagedJDBCConnection class to prevent it
calling down into the DBCP2 connection. When this happened, my
queries/transactions would continue on but then fail during another method
call (like #commit) where the DBCP2 connection would throw an exception
complaining about not being able to perform certain operations when
participating in a managed transaction. So, it seemed to me like my DBCP2
connections were already XA aware/enlisted (or whatever you call it) and
that the higher-level stuff in aries-jdbc/tranql was incompatible with them.

So, here is where it gets really interesting... thinking about this problem
I wondered if maybe I didn't even need the aries-wrapped datasources. So, I
changed my eclipselink/jpa JNDI lookups to be !(aries.managed=true), to
hopefully force eclipselink to use the DBCP2 XA datasource services
directly. Well, voila... this seems to not only work, but still provide the
xa/jta management as well. To test this I created a conflicting entity row
in only one of my three eclipselink partitions or 'nodes'. Attempting to
persist the corresponding entity in osgi via my jpa classes failed with the
expected exception indicating a conflicting row *and* inspecting the
database nodes showed that the only node with the row was the original node
where it was manually inserted for the test. So, I'm assuming this is an
indicator that the XA stuff ran as expected and prevented the insert from
propagating to other nodes because the coordinated transaction was rolled
back.

So, my question is - does this setup make sense? To have the aries
transaction manager/bundles in-place but not actually use the aries-managed
datasources? Perhaps there are facilities in the pax dbcp2 pool bundle that
does the auto-enlistment already? Perhaps I have no idea what I am talking
about?

Any guidance is greatly appreciated. Thanks.

-matt

-----Original Message-----
From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
Christian Schneider
Sent: Wednesday, September 23, 2015 1:53 AM
To: user@aries.apache.org
Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink
Partitioning?

Not sure why you hit the problem with dbcp2 and eclipselink. I have some 
itests running
with eclipselink and they showed no problems. Will look into what 
versions I used exactly.

I am not very familiar with the aries pooling but it should work fine.

Christian

Am 23.09.2015 um 02:48 schrieb Matthew.W.Pitts@wellsfargo.com:
> Ok, thanks so much for the suggestions.
>
> It is interesting to me that this isn't something someone else has seen
> since I feel like I am using the pax-jdbc and aries bundles in a
> straightforward way.
>
> Do you have a recommendation of using the aries pool bundle vs. the dbcp2
> one? I'm not set on either one, I just need it to work, have some
> tweakability and perform well enough. I had been leaning towards dbcp2
since
> it seems like a reasonable, actively developed pool implementation that I
> have used elsewhere, but since it doesn't currently work for me I'm fine
> using aries if it works and performs.
>
> It seems to me that the aries pooling basically delegates back to the
> Geronimo classes, which I would suspect are a decent pool implementation
> themselves having originated as part of that project. But I have no
> experience with them, so my confidence is low.
>
> Obviously, I'll do some performance tests when all this comes together
more,
> but recommended best practices or known working bundle combinations are
> always a good starting point. I'm still looking through your earlier
> references and all the itest stuff to try to piece together a logical
bundle
> set, but it's slow for me as I am not very familiar with PaxExam.
>
> Thanks,
> -matt
>
> -----Original Message-----
> From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
> Christian Schneider
> Sent: Tuesday, September 22, 2015 4:04 PM
> To: user@aries.apache.org
> Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink
> Partitioning?
>
> Honestly I have no idea why tranql sets the autocommit. I would not
> expect that it needs to be tweaked.
>
> I can imagine that dbcp2 does not allow to change autocommit for XA
> transactions as the commit should
> only be done when the XA transaction commits. Not sure about this either
> though.
>
> I propose you ask this on the apache commons list or open an issue for
> dbcp2. It definitely seems to be an issue that
> can also happen when just configuring dbcp2 manually and then use tranql
> on it.
>
> Christian
>
> Am 22.09.2015 um 20:19 schrieb Matthew.W.Pitts@wellsfargo.com:
>> [1] is a snippet from the stacktrace I get when using pax-jdbc-pool-dbcp2
>> instead of -aries. As you can see,
>> ManagedJDBCConnection#localTransactionStart appears to call setAutoCommit
> on
>> the connection class, which the DBCP2 impl short circuits at [2].
>>
>> This same exact environment seems to run fine with the -aries pool
bundle.
> I
>> am running this with a customized transactions-jdbc bundle checked-out
> from
>> SVN. But, the only change I made was the pull in all the tranql source to
>> fix issues with the two-arg form of getConnection in some of those
> classes.
>> Aside from that it's unchanged and works fine with the -aries pooling
>> bundle.
>>
>> Any suggestions are greatly appreciated.
>>
>> [1]
>> Caused by: javax.resource.spi.LocalTransactionException: Unable to
disable
>> autoCommit
>> 	at
>>
>
org.tranql.connector.jdbc.ManagedJDBCConnection.localTransactionStart(Manage
>> dJDBCConnection.java:83)
>> ~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
>> 	at
>>
>
org.tranql.connector.AbstractManagedConnection$LocalTransactionImpl.begin(Ab
>> stractManagedConnection.java:195)
>> ~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
>> 	at
>>
>
org.apache.geronimo.connector.outbound.LocalXAResource.start(LocalXAResource
>> .java:107) ~[geronimo-connector-3.1.3.jar:3.1.3]
>> 	... 76 common frames omitted
>> Caused by: java.sql.SQLException: Auto-commit can not be set while
> enrolled
>> in a transaction
>> 	at
>>
>
org.apache.commons.dbcp2.managed.ManagedConnection.setAutoCommit(ManagedConn
>> ection.java:223) ~[na:na]
>> 	at
>>
>
org.tranql.connector.jdbc.ManagedJDBCConnection.localTransactionStart(Manage
>> dJDBCConnection.java:81)
>> ~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
>> 	... 78 common frames omitted
>>
>> [2]
>> public void setAutoCommit(boolean autoCommit) throws SQLException {
>>       if (transactionContext != null) {
>>    
>> throw new SQLException("Auto-commit can not be set while enrolled in a
> trans
>> action");
>>       }
>>       super.setAutoCommit(autoCommit);
>> }
>>
>>
>> -----Original Message-----
>> From: Matthew.W.Pitts@wellsfargo.com
> [mailto:Matthew.W.Pitts@wellsfargo.com]
>> Sent: Tuesday, September 22, 2015 11:14 AM
>> To: user@aries.apache.org
>> Subject: RE: Bundle set recommendations for JPA+JTA+Eclipselink
>> Partitioning?
>>
>> Thanks very much for the references, I will definitely be looking over
> them.
>> I was hoping the dbcp2 bundle would allow for the same XA capability, as
I
>> would prefer dbcp2 pooling since it seems fairly analogous to my current
>> pooling - bonecp. However, I did some testing with dbcp2 and got
> exceptions
>> during runtime when code from base datasource wrappers (from tranql
maybe)
>> was attempting to set the auto-commit flag and the DBCP datasource
wrapper
>> class was throwing an exception about setting auto-commit. I can't
> remember
>> exactly the code flow but I'll try to test this again and see about
> getting
>> a stacktrace. Does any of this sound familiar? Should I be able to simply
>> replace pax-jdbc-pool-aries with pax-jdbc-pool-dbcp2 in my runtime? Maybe
> it
>> was something in my jndi references not getting the aries-managed
> datasource
>> services.
>>
>> As for the blueprint/DS question, I will be largely using DS to inject
>> app-specific API that gives my code access to the
> JpaTemplate/EntityManager
>> classes. I hope to move toward blueprint, but that will be another
effort.
>>
>> Thanks again,
>> -matt
>>
>> -----Original Message-----
>> From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
>> Christian Schneider
>> Sent: Tuesday, September 22, 2015 10:24 AM
>> To: user@aries.apache.org
>> Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink
>> Partitioning?
>>
>> Hi Matthew,
>>
>> pax-jdbc-pool-dbcp2 as well as pax-jdbc-pool-aries should both be able
>> to provide XA enlisting poolable DataSources.
>> Aries JPA and Transaction actually use the above in their integration
>> tests. So they should work together nicely.
>>
>> I got two tutorials that both use XA transactions with Aries:
>>
>
http://liquid-reality.de/display/liquid/2015/06/30/Apache+Karaf+Tutorial+par
>> t+10+-+Declarative+services
>>
>
http://liquid-reality.de/display/liquid/2015/03/05/Apache+Karaf+Tutorial+Par
>> t+9+-+Annotation+based+blueprint+and+JPA
>>
>> Btw. Do you plan to use blueprint or DS?
>>
>> Christian
>>
>> On 03.09.2015 22:51, Matthew.W.Pitts@wellsfargo.com wrote:
>>> Ok, maybe I can answer my own questions here, but if I could get some
>>> confirmation that would be great. I seem to have a working system
running
>>> with ops4j-pax-jdbc-pool-aries rather than dbcp2.
>>>
>>> Digging into the code in org.apache.aries.transaction.jdbc it looks like
>>> RecoverableDataSource is the wrapper for my XADataSources and it does
>> indeed
>>> appear to support passing of pooling options onto
> ConnectionManagerFactory
>>> which itself appears to use the pooling capabilities that come from the
>>> Geronimo bundles. Does all this sound correct so far?
>>>
>>> So, it's not dbcp2, but it's pooling nonetheless and I can adjust
>>> preferences by passing the appropriate bean options to my pax-jdbc
> service
>>> with the 'pool.' prefix which will in-turn send those on to the
>>> RecoverableDataSource instance. Sound ok still?
>>>
>>> And finally, my eclipselink JTA datasource JNDI lookups are formatted
> with
>>> filters for the 'aries.managed' property [1] to ensure that I get the
>> aries
>>> auto-enlisting DataSources.
>>>
>>> So far, this is running my existing code ok with only two changes I had
> to
>>> make:
>>>
>>> 1. Fixed ARIES-1171, but actually in the Tranql code as I was getting
>>> connection errors and didn't see a logical place to fix it from an aries
>>> class (using latest aries-transaction-jdbc from SVN). This required
>> bringing
>>> all the tranql source into my aries bundle and just removing the
external
>>> dependency. I didn't like doing that, but since that code hasn't changed
>> in
>>> a while and I'm still prototyping all this it was the path of least
>>> resistance.
>>>
>>> 2. Changed RecoverableDataSource to not use empty strings by default for
>>> username and password. Which seemed to be similar/related to the above
> and
>> I
>>> posted as ARIES-1376.
>>>
>>> Thoughts and criticisms welcome. Thanks!
>>>
>>> [1]
>>>
>
osgi:service/javax.sql.DataSource/(&(osgi.jndi.service.name=my-app-node1)(ar
>>> ies.managed=true))
>>>
>>> -----Original Message-----
>>> From: Matthew.W.Pitts@wellsfargo.com
>> [mailto:Matthew.W.Pitts@wellsfargo.com]
>>> Sent: Thursday, September 03, 2015 10:58 AM
>>> To: user@aries.apache.org
>>> Subject: Bundle set recommendations for JPA+JTA+Eclipselink
Partitioning?
>>>
>>> I am in the process of refactoring an existing JPA/Eclipselink-based
>>> application to introduce Eclipselink's partitioning support for database
>>> sharding. My current application runs quite well with pooled jdbc
>>> connections via boncep.
>>>
>>> However, with the sharded model there will be a number of entity types
>> that
>>> are Replicated to all shards and, as such, eclipselink will be doing
>> writes
>>> that span multiple jdbc connections. Per the eclipselink recommendations
>> for
>>> paritioning I would like to introduce JTA in order to enforce integrity
>>> across commits that hit multiple partitions.
>>>
>>> It seems like the ops4j-pax-jdbc bundles are designed to support XA/JTA
>> and
>>> it seems like there is a lot of XA support in the various aries bundles
>>> (aries.transaction. and aries.jpa.). However, I'm not sure of what
>>> combination of these bundles+configuration will provide what I am after.
>>> Since many of these bundles seem to be designed/documented around
general
>>> JTA usage - like performing a transaction that spans JMS and JDBC - I'm
>> not
>>> confident if that model works with my particular use case.
>>>
>>> To further add to this, I would like for my JDBC connections to be
pooled
>>> for performance. ops4j-pax-jdbc-pool-dbcp2 seems like a logical choice,
>> but
>>> again I'm not sure if it will play nicely with the transaction
management
>>> features from aries.
>>>
>>> I think I have a working model of this, but I'm rather new to XA/JTA
>>> especially from OSGi so I want to make sure I'm getting this correct.
So,
>> I
>>> guess my questions are:
>>>
>>> 1. Can ops4j-pax-jdbc-pool and it's dbcp2 sibling work with
>>> aries-jpa/transaction to pool and auto-enlist XADataSources?
>>>
>>> 2. If not, will ops4j-pax-jdbc-pool-aries work? And if so, can its
> pooling
>>> semantics be configured? E.g. minIdle, maxIdle, etc.
>>>
>>> 3. If neither of the above works, is there a recommended set of bundles
> to
>>> accomplish pooled XADataSources with eclipselink+JTA?
>>>
>>>
>>> Thanks,
>>>
>>>
>>> Matthew Pitts
>>>     
>>> Developer
>>> Security Solutions Design & Automation
>>>     
>>> Wells Fargo Bank | Tel 336.608.3332 | Cell 336.202.3913 | Kernersville,
> NC
>> |
>>> MAC D9693-010
>>> matthew.w.pitts@wellsfargo.com
>>>     
>>>

Re: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?

[Christian Schneider <ch...@die-schneider.net>]

Not sure why you hit the problem with dbcp2 and eclipselink. I have some 
itests running
with eclipselink and they showed no problems. Will look into what 
versions I used exactly.

I am not very familiar with the aries pooling but it should work fine.

Christian

Am 23.09.2015 um 02:48 schrieb Matthew.W.Pitts@wellsfargo.com:
> Ok, thanks so much for the suggestions.
>
> It is interesting to me that this isn't something someone else has seen
> since I feel like I am using the pax-jdbc and aries bundles in a
> straightforward way.
>
> Do you have a recommendation of using the aries pool bundle vs. the dbcp2
> one? I'm not set on either one, I just need it to work, have some
> tweakability and perform well enough. I had been leaning towards dbcp2 since
> it seems like a reasonable, actively developed pool implementation that I
> have used elsewhere, but since it doesn't currently work for me I'm fine
> using aries if it works and performs.
>
> It seems to me that the aries pooling basically delegates back to the
> Geronimo classes, which I would suspect are a decent pool implementation
> themselves having originated as part of that project. But I have no
> experience with them, so my confidence is low.
>
> Obviously, I'll do some performance tests when all this comes together more,
> but recommended best practices or known working bundle combinations are
> always a good starting point. I'm still looking through your earlier
> references and all the itest stuff to try to piece together a logical bundle
> set, but it's slow for me as I am not very familiar with PaxExam.
>
> Thanks,
> -matt
>
> -----Original Message-----
> From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
> Christian Schneider
> Sent: Tuesday, September 22, 2015 4:04 PM
> To: user@aries.apache.org
> Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink
> Partitioning?
>
> Honestly I have no idea why tranql sets the autocommit. I would not
> expect that it needs to be tweaked.
>
> I can imagine that dbcp2 does not allow to change autocommit for XA
> transactions as the commit should
> only be done when the XA transaction commits. Not sure about this either
> though.
>
> I propose you ask this on the apache commons list or open an issue for
> dbcp2. It definitely seems to be an issue that
> can also happen when just configuring dbcp2 manually and then use tranql
> on it.
>
> Christian
>
> Am 22.09.2015 um 20:19 schrieb Matthew.W.Pitts@wellsfargo.com:
>> [1] is a snippet from the stacktrace I get when using pax-jdbc-pool-dbcp2
>> instead of -aries. As you can see,
>> ManagedJDBCConnection#localTransactionStart appears to call setAutoCommit
> on
>> the connection class, which the DBCP2 impl short circuits at [2].
>>
>> This same exact environment seems to run fine with the -aries pool bundle.
> I
>> am running this with a customized transactions-jdbc bundle checked-out
> from
>> SVN. But, the only change I made was the pull in all the tranql source to
>> fix issues with the two-arg form of getConnection in some of those
> classes.
>> Aside from that it's unchanged and works fine with the -aries pooling
>> bundle.
>>
>> Any suggestions are greatly appreciated.
>>
>> [1]
>> Caused by: javax.resource.spi.LocalTransactionException: Unable to disable
>> autoCommit
>> 	at
>>
> org.tranql.connector.jdbc.ManagedJDBCConnection.localTransactionStart(Manage
>> dJDBCConnection.java:83)
>> ~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
>> 	at
>>
> org.tranql.connector.AbstractManagedConnection$LocalTransactionImpl.begin(Ab
>> stractManagedConnection.java:195)
>> ~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
>> 	at
>>
> org.apache.geronimo.connector.outbound.LocalXAResource.start(LocalXAResource
>> .java:107) ~[geronimo-connector-3.1.3.jar:3.1.3]
>> 	... 76 common frames omitted
>> Caused by: java.sql.SQLException: Auto-commit can not be set while
> enrolled
>> in a transaction
>> 	at
>>
> org.apache.commons.dbcp2.managed.ManagedConnection.setAutoCommit(ManagedConn
>> ection.java:223) ~[na:na]
>> 	at
>>
> org.tranql.connector.jdbc.ManagedJDBCConnection.localTransactionStart(Manage
>> dJDBCConnection.java:81)
>> ~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
>> 	... 78 common frames omitted
>>
>> [2]
>> public void setAutoCommit(boolean autoCommit) throws SQLException {
>>       if (transactionContext != null) {
>>    
>> throw new SQLException("Auto-commit can not be set while enrolled in a
> trans
>> action");
>>       }
>>       super.setAutoCommit(autoCommit);
>> }
>>
>>
>> -----Original Message-----
>> From: Matthew.W.Pitts@wellsfargo.com
> [mailto:Matthew.W.Pitts@wellsfargo.com]
>> Sent: Tuesday, September 22, 2015 11:14 AM
>> To: user@aries.apache.org
>> Subject: RE: Bundle set recommendations for JPA+JTA+Eclipselink
>> Partitioning?
>>
>> Thanks very much for the references, I will definitely be looking over
> them.
>> I was hoping the dbcp2 bundle would allow for the same XA capability, as I
>> would prefer dbcp2 pooling since it seems fairly analogous to my current
>> pooling - bonecp. However, I did some testing with dbcp2 and got
> exceptions
>> during runtime when code from base datasource wrappers (from tranql maybe)
>> was attempting to set the auto-commit flag and the DBCP datasource wrapper
>> class was throwing an exception about setting auto-commit. I can't
> remember
>> exactly the code flow but I'll try to test this again and see about
> getting
>> a stacktrace. Does any of this sound familiar? Should I be able to simply
>> replace pax-jdbc-pool-aries with pax-jdbc-pool-dbcp2 in my runtime? Maybe
> it
>> was something in my jndi references not getting the aries-managed
> datasource
>> services.
>>
>> As for the blueprint/DS question, I will be largely using DS to inject
>> app-specific API that gives my code access to the
> JpaTemplate/EntityManager
>> classes. I hope to move toward blueprint, but that will be another effort.
>>
>> Thanks again,
>> -matt
>>
>> -----Original Message-----
>> From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
>> Christian Schneider
>> Sent: Tuesday, September 22, 2015 10:24 AM
>> To: user@aries.apache.org
>> Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink
>> Partitioning?
>>
>> Hi Matthew,
>>
>> pax-jdbc-pool-dbcp2 as well as pax-jdbc-pool-aries should both be able
>> to provide XA enlisting poolable DataSources.
>> Aries JPA and Transaction actually use the above in their integration
>> tests. So they should work together nicely.
>>
>> I got two tutorials that both use XA transactions with Aries:
>>
> http://liquid-reality.de/display/liquid/2015/06/30/Apache+Karaf+Tutorial+par
>> t+10+-+Declarative+services
>>
> http://liquid-reality.de/display/liquid/2015/03/05/Apache+Karaf+Tutorial+Par
>> t+9+-+Annotation+based+blueprint+and+JPA
>>
>> Btw. Do you plan to use blueprint or DS?
>>
>> Christian
>>
>> On 03.09.2015 22:51, Matthew.W.Pitts@wellsfargo.com wrote:
>>> Ok, maybe I can answer my own questions here, but if I could get some
>>> confirmation that would be great. I seem to have a working system running
>>> with ops4j-pax-jdbc-pool-aries rather than dbcp2.
>>>
>>> Digging into the code in org.apache.aries.transaction.jdbc it looks like
>>> RecoverableDataSource is the wrapper for my XADataSources and it does
>> indeed
>>> appear to support passing of pooling options onto
> ConnectionManagerFactory
>>> which itself appears to use the pooling capabilities that come from the
>>> Geronimo bundles. Does all this sound correct so far?
>>>
>>> So, it's not dbcp2, but it's pooling nonetheless and I can adjust
>>> preferences by passing the appropriate bean options to my pax-jdbc
> service
>>> with the 'pool.' prefix which will in-turn send those on to the
>>> RecoverableDataSource instance. Sound ok still?
>>>
>>> And finally, my eclipselink JTA datasource JNDI lookups are formatted
> with
>>> filters for the 'aries.managed' property [1] to ensure that I get the
>> aries
>>> auto-enlisting DataSources.
>>>
>>> So far, this is running my existing code ok with only two changes I had
> to
>>> make:
>>>
>>> 1. Fixed ARIES-1171, but actually in the Tranql code as I was getting
>>> connection errors and didn't see a logical place to fix it from an aries
>>> class (using latest aries-transaction-jdbc from SVN). This required
>> bringing
>>> all the tranql source into my aries bundle and just removing the external
>>> dependency. I didn't like doing that, but since that code hasn't changed
>> in
>>> a while and I'm still prototyping all this it was the path of least
>>> resistance.
>>>
>>> 2. Changed RecoverableDataSource to not use empty strings by default for
>>> username and password. Which seemed to be similar/related to the above
> and
>> I
>>> posted as ARIES-1376.
>>>
>>> Thoughts and criticisms welcome. Thanks!
>>>
>>> [1]
>>>
> osgi:service/javax.sql.DataSource/(&(osgi.jndi.service.name=my-app-node1)(ar
>>> ies.managed=true))
>>>
>>> -----Original Message-----
>>> From: Matthew.W.Pitts@wellsfargo.com
>> [mailto:Matthew.W.Pitts@wellsfargo.com]
>>> Sent: Thursday, September 03, 2015 10:58 AM
>>> To: user@aries.apache.org
>>> Subject: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?
>>>
>>> I am in the process of refactoring an existing JPA/Eclipselink-based
>>> application to introduce Eclipselink's partitioning support for database
>>> sharding. My current application runs quite well with pooled jdbc
>>> connections via boncep.
>>>
>>> However, with the sharded model there will be a number of entity types
>> that
>>> are Replicated to all shards and, as such, eclipselink will be doing
>> writes
>>> that span multiple jdbc connections. Per the eclipselink recommendations
>> for
>>> paritioning I would like to introduce JTA in order to enforce integrity
>>> across commits that hit multiple partitions.
>>>
>>> It seems like the ops4j-pax-jdbc bundles are designed to support XA/JTA
>> and
>>> it seems like there is a lot of XA support in the various aries bundles
>>> (aries.transaction. and aries.jpa.). However, I'm not sure of what
>>> combination of these bundles+configuration will provide what I am after.
>>> Since many of these bundles seem to be designed/documented around general
>>> JTA usage - like performing a transaction that spans JMS and JDBC - I'm
>> not
>>> confident if that model works with my particular use case.
>>>
>>> To further add to this, I would like for my JDBC connections to be pooled
>>> for performance. ops4j-pax-jdbc-pool-dbcp2 seems like a logical choice,
>> but
>>> again I'm not sure if it will play nicely with the transaction management
>>> features from aries.
>>>
>>> I think I have a working model of this, but I'm rather new to XA/JTA
>>> especially from OSGi so I want to make sure I'm getting this correct. So,
>> I
>>> guess my questions are:
>>>
>>> 1. Can ops4j-pax-jdbc-pool and it's dbcp2 sibling work with
>>> aries-jpa/transaction to pool and auto-enlist XADataSources?
>>>
>>> 2. If not, will ops4j-pax-jdbc-pool-aries work? And if so, can its
> pooling
>>> semantics be configured? E.g. minIdle, maxIdle, etc.
>>>
>>> 3. If neither of the above works, is there a recommended set of bundles
> to
>>> accomplish pooled XADataSources with eclipselink+JTA?
>>>
>>>
>>> Thanks,
>>>
>>>
>>> Matthew Pitts
>>>     
>>> Developer
>>> Security Solutions Design & Automation
>>>     
>>> Wells Fargo Bank | Tel 336.608.3332 | Cell 336.202.3913 | Kernersville,
> NC
>> |
>>> MAC D9693-010
>>> matthew.w.pitts@wellsfargo.com
>>>     
>>>

RE: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?

[Ma...@wellsfargo.com]

Ok, thanks so much for the suggestions.

It is interesting to me that this isn't something someone else has seen
since I feel like I am using the pax-jdbc and aries bundles in a
straightforward way.

Do you have a recommendation of using the aries pool bundle vs. the dbcp2
one? I'm not set on either one, I just need it to work, have some
tweakability and perform well enough. I had been leaning towards dbcp2 since
it seems like a reasonable, actively developed pool implementation that I
have used elsewhere, but since it doesn't currently work for me I'm fine
using aries if it works and performs.

It seems to me that the aries pooling basically delegates back to the
Geronimo classes, which I would suspect are a decent pool implementation
themselves having originated as part of that project. But I have no
experience with them, so my confidence is low.

Obviously, I'll do some performance tests when all this comes together more,
but recommended best practices or known working bundle combinations are
always a good starting point. I'm still looking through your earlier
references and all the itest stuff to try to piece together a logical bundle
set, but it's slow for me as I am not very familiar with PaxExam.

Thanks,
-matt

-----Original Message-----
From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
Christian Schneider
Sent: Tuesday, September 22, 2015 4:04 PM
To: user@aries.apache.org
Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink
Partitioning?

Honestly I have no idea why tranql sets the autocommit. I would not 
expect that it needs to be tweaked.

I can imagine that dbcp2 does not allow to change autocommit for XA 
transactions as the commit should
only be done when the XA transaction commits. Not sure about this either 
though.

I propose you ask this on the apache commons list or open an issue for 
dbcp2. It definitely seems to be an issue that
can also happen when just configuring dbcp2 manually and then use tranql 
on it.

Christian

Am 22.09.2015 um 20:19 schrieb Matthew.W.Pitts@wellsfargo.com:
> [1] is a snippet from the stacktrace I get when using pax-jdbc-pool-dbcp2
> instead of -aries. As you can see,
> ManagedJDBCConnection#localTransactionStart appears to call setAutoCommit
on
> the connection class, which the DBCP2 impl short circuits at [2].
>
> This same exact environment seems to run fine with the -aries pool bundle.
I
> am running this with a customized transactions-jdbc bundle checked-out
from
> SVN. But, the only change I made was the pull in all the tranql source to
> fix issues with the two-arg form of getConnection in some of those
classes.
> Aside from that it's unchanged and works fine with the -aries pooling
> bundle.
>
> Any suggestions are greatly appreciated.
>
> [1]
> Caused by: javax.resource.spi.LocalTransactionException: Unable to disable
> autoCommit
> 	at
>
org.tranql.connector.jdbc.ManagedJDBCConnection.localTransactionStart(Manage
> dJDBCConnection.java:83)
> ~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
> 	at
>
org.tranql.connector.AbstractManagedConnection$LocalTransactionImpl.begin(Ab
> stractManagedConnection.java:195)
> ~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
> 	at
>
org.apache.geronimo.connector.outbound.LocalXAResource.start(LocalXAResource
> .java:107) ~[geronimo-connector-3.1.3.jar:3.1.3]
> 	... 76 common frames omitted
> Caused by: java.sql.SQLException: Auto-commit can not be set while
enrolled
> in a transaction
> 	at
>
org.apache.commons.dbcp2.managed.ManagedConnection.setAutoCommit(ManagedConn
> ection.java:223) ~[na:na]
> 	at
>
org.tranql.connector.jdbc.ManagedJDBCConnection.localTransactionStart(Manage
> dJDBCConnection.java:81)
> ~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
> 	... 78 common frames omitted
>
> [2]
> public void setAutoCommit(boolean autoCommit) throws SQLException {
>      if (transactionContext != null) {
>   
> throw new SQLException("Auto-commit can not be set while enrolled in a
trans
> action");
>      }
>      super.setAutoCommit(autoCommit);
> }
>
>
> -----Original Message-----
> From: Matthew.W.Pitts@wellsfargo.com
[mailto:Matthew.W.Pitts@wellsfargo.com]
>
> Sent: Tuesday, September 22, 2015 11:14 AM
> To: user@aries.apache.org
> Subject: RE: Bundle set recommendations for JPA+JTA+Eclipselink
> Partitioning?
>
> Thanks very much for the references, I will definitely be looking over
them.
>
> I was hoping the dbcp2 bundle would allow for the same XA capability, as I
> would prefer dbcp2 pooling since it seems fairly analogous to my current
> pooling - bonecp. However, I did some testing with dbcp2 and got
exceptions
> during runtime when code from base datasource wrappers (from tranql maybe)
> was attempting to set the auto-commit flag and the DBCP datasource wrapper
> class was throwing an exception about setting auto-commit. I can't
remember
> exactly the code flow but I'll try to test this again and see about
getting
> a stacktrace. Does any of this sound familiar? Should I be able to simply
> replace pax-jdbc-pool-aries with pax-jdbc-pool-dbcp2 in my runtime? Maybe
it
> was something in my jndi references not getting the aries-managed
datasource
> services.
>
> As for the blueprint/DS question, I will be largely using DS to inject
> app-specific API that gives my code access to the
JpaTemplate/EntityManager
> classes. I hope to move toward blueprint, but that will be another effort.
>
> Thanks again,
> -matt
>
> -----Original Message-----
> From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
> Christian Schneider
> Sent: Tuesday, September 22, 2015 10:24 AM
> To: user@aries.apache.org
> Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink
> Partitioning?
>
> Hi Matthew,
>
> pax-jdbc-pool-dbcp2 as well as pax-jdbc-pool-aries should both be able
> to provide XA enlisting poolable DataSources.
> Aries JPA and Transaction actually use the above in their integration
> tests. So they should work together nicely.
>
> I got two tutorials that both use XA transactions with Aries:
>
http://liquid-reality.de/display/liquid/2015/06/30/Apache+Karaf+Tutorial+par
> t+10+-+Declarative+services
>
http://liquid-reality.de/display/liquid/2015/03/05/Apache+Karaf+Tutorial+Par
> t+9+-+Annotation+based+blueprint+and+JPA
>
> Btw. Do you plan to use blueprint or DS?
>
> Christian
>
> On 03.09.2015 22:51, Matthew.W.Pitts@wellsfargo.com wrote:
>> Ok, maybe I can answer my own questions here, but if I could get some
>> confirmation that would be great. I seem to have a working system running
>> with ops4j-pax-jdbc-pool-aries rather than dbcp2.
>>
>> Digging into the code in org.apache.aries.transaction.jdbc it looks like
>> RecoverableDataSource is the wrapper for my XADataSources and it does
> indeed
>> appear to support passing of pooling options onto
ConnectionManagerFactory
>> which itself appears to use the pooling capabilities that come from the
>> Geronimo bundles. Does all this sound correct so far?
>>
>> So, it's not dbcp2, but it's pooling nonetheless and I can adjust
>> preferences by passing the appropriate bean options to my pax-jdbc
service
>> with the 'pool.' prefix which will in-turn send those on to the
>> RecoverableDataSource instance. Sound ok still?
>>
>> And finally, my eclipselink JTA datasource JNDI lookups are formatted
with
>> filters for the 'aries.managed' property [1] to ensure that I get the
> aries
>> auto-enlisting DataSources.
>>
>> So far, this is running my existing code ok with only two changes I had
to
>> make:
>>
>> 1. Fixed ARIES-1171, but actually in the Tranql code as I was getting
>> connection errors and didn't see a logical place to fix it from an aries
>> class (using latest aries-transaction-jdbc from SVN). This required
> bringing
>> all the tranql source into my aries bundle and just removing the external
>> dependency. I didn't like doing that, but since that code hasn't changed
> in
>> a while and I'm still prototyping all this it was the path of least
>> resistance.
>>
>> 2. Changed RecoverableDataSource to not use empty strings by default for
>> username and password. Which seemed to be similar/related to the above
and
> I
>> posted as ARIES-1376.
>>
>> Thoughts and criticisms welcome. Thanks!
>>
>> [1]
>>
>
osgi:service/javax.sql.DataSource/(&(osgi.jndi.service.name=my-app-node1)(ar
>> ies.managed=true))
>>
>> -----Original Message-----
>> From: Matthew.W.Pitts@wellsfargo.com
> [mailto:Matthew.W.Pitts@wellsfargo.com]
>> Sent: Thursday, September 03, 2015 10:58 AM
>> To: user@aries.apache.org
>> Subject: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?
>>
>> I am in the process of refactoring an existing JPA/Eclipselink-based
>> application to introduce Eclipselink's partitioning support for database
>> sharding. My current application runs quite well with pooled jdbc
>> connections via boncep.
>>
>> However, with the sharded model there will be a number of entity types
> that
>> are Replicated to all shards and, as such, eclipselink will be doing
> writes
>> that span multiple jdbc connections. Per the eclipselink recommendations
> for
>> paritioning I would like to introduce JTA in order to enforce integrity
>> across commits that hit multiple partitions.
>>
>> It seems like the ops4j-pax-jdbc bundles are designed to support XA/JTA
> and
>> it seems like there is a lot of XA support in the various aries bundles
>> (aries.transaction. and aries.jpa.). However, I'm not sure of what
>> combination of these bundles+configuration will provide what I am after.
>> Since many of these bundles seem to be designed/documented around general
>> JTA usage - like performing a transaction that spans JMS and JDBC - I'm
> not
>> confident if that model works with my particular use case.
>>
>> To further add to this, I would like for my JDBC connections to be pooled
>> for performance. ops4j-pax-jdbc-pool-dbcp2 seems like a logical choice,
> but
>> again I'm not sure if it will play nicely with the transaction management
>> features from aries.
>>
>> I think I have a working model of this, but I'm rather new to XA/JTA
>> especially from OSGi so I want to make sure I'm getting this correct. So,
> I
>> guess my questions are:
>>
>> 1. Can ops4j-pax-jdbc-pool and it's dbcp2 sibling work with
>> aries-jpa/transaction to pool and auto-enlist XADataSources?
>>
>> 2. If not, will ops4j-pax-jdbc-pool-aries work? And if so, can its
pooling
>> semantics be configured? E.g. minIdle, maxIdle, etc.
>>
>> 3. If neither of the above works, is there a recommended set of bundles
to
>> accomplish pooled XADataSources with eclipselink+JTA?
>>
>>
>> Thanks,
>>
>>
>> Matthew Pitts
>>    
>> Developer
>> Security Solutions Design & Automation
>>    
>> Wells Fargo Bank | Tel 336.608.3332 | Cell 336.202.3913 | Kernersville,
NC
> |
>> MAC D9693-010
>> matthew.w.pitts@wellsfargo.com
>>    
>>
>

Re: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?

[Christian Schneider <ch...@die-schneider.net>]

Honestly I have no idea why tranql sets the autocommit. I would not 
expect that it needs to be tweaked.

I can imagine that dbcp2 does not allow to change autocommit for XA 
transactions as the commit should
only be done when the XA transaction commits. Not sure about this either 
though.

I propose you ask this on the apache commons list or open an issue for 
dbcp2. It definitely seems to be an issue that
can also happen when just configuring dbcp2 manually and then use tranql 
on it.

Christian

Am 22.09.2015 um 20:19 schrieb Matthew.W.Pitts@wellsfargo.com:
> [1] is a snippet from the stacktrace I get when using pax-jdbc-pool-dbcp2
> instead of -aries. As you can see,
> ManagedJDBCConnection#localTransactionStart appears to call setAutoCommit on
> the connection class, which the DBCP2 impl short circuits at [2].
>
> This same exact environment seems to run fine with the -aries pool bundle. I
> am running this with a customized transactions-jdbc bundle checked-out from
> SVN. But, the only change I made was the pull in all the tranql source to
> fix issues with the two-arg form of getConnection in some of those classes.
> Aside from that it's unchanged and works fine with the -aries pooling
> bundle.
>
> Any suggestions are greatly appreciated.
>
> [1]
> Caused by: javax.resource.spi.LocalTransactionException: Unable to disable
> autoCommit
> 	at
> org.tranql.connector.jdbc.ManagedJDBCConnection.localTransactionStart(Manage
> dJDBCConnection.java:83)
> ~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
> 	at
> org.tranql.connector.AbstractManagedConnection$LocalTransactionImpl.begin(Ab
> stractManagedConnection.java:195)
> ~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
> 	at
> org.apache.geronimo.connector.outbound.LocalXAResource.start(LocalXAResource
> .java:107) ~[geronimo-connector-3.1.3.jar:3.1.3]
> 	... 76 common frames omitted
> Caused by: java.sql.SQLException: Auto-commit can not be set while enrolled
> in a transaction
> 	at
> org.apache.commons.dbcp2.managed.ManagedConnection.setAutoCommit(ManagedConn
> ection.java:223) ~[na:na]
> 	at
> org.tranql.connector.jdbc.ManagedJDBCConnection.localTransactionStart(Manage
> dJDBCConnection.java:81)
> ~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
> 	... 78 common frames omitted
>
> [2]
> public void setAutoCommit(boolean autoCommit) throws SQLException {
>      if (transactionContext != null) {
>   
> throw new SQLException("Auto-commit can not be set while enrolled in a trans
> action");
>      }
>      super.setAutoCommit(autoCommit);
> }
>
>
> -----Original Message-----
> From: Matthew.W.Pitts@wellsfargo.com [mailto:Matthew.W.Pitts@wellsfargo.com]
>
> Sent: Tuesday, September 22, 2015 11:14 AM
> To: user@aries.apache.org
> Subject: RE: Bundle set recommendations for JPA+JTA+Eclipselink
> Partitioning?
>
> Thanks very much for the references, I will definitely be looking over them.
>
> I was hoping the dbcp2 bundle would allow for the same XA capability, as I
> would prefer dbcp2 pooling since it seems fairly analogous to my current
> pooling - bonecp. However, I did some testing with dbcp2 and got exceptions
> during runtime when code from base datasource wrappers (from tranql maybe)
> was attempting to set the auto-commit flag and the DBCP datasource wrapper
> class was throwing an exception about setting auto-commit. I can't remember
> exactly the code flow but I'll try to test this again and see about getting
> a stacktrace. Does any of this sound familiar? Should I be able to simply
> replace pax-jdbc-pool-aries with pax-jdbc-pool-dbcp2 in my runtime? Maybe it
> was something in my jndi references not getting the aries-managed datasource
> services.
>
> As for the blueprint/DS question, I will be largely using DS to inject
> app-specific API that gives my code access to the JpaTemplate/EntityManager
> classes. I hope to move toward blueprint, but that will be another effort.
>
> Thanks again,
> -matt
>
> -----Original Message-----
> From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
> Christian Schneider
> Sent: Tuesday, September 22, 2015 10:24 AM
> To: user@aries.apache.org
> Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink
> Partitioning?
>
> Hi Matthew,
>
> pax-jdbc-pool-dbcp2 as well as pax-jdbc-pool-aries should both be able
> to provide XA enlisting poolable DataSources.
> Aries JPA and Transaction actually use the above in their integration
> tests. So they should work together nicely.
>
> I got two tutorials that both use XA transactions with Aries:
> http://liquid-reality.de/display/liquid/2015/06/30/Apache+Karaf+Tutorial+par
> t+10+-+Declarative+services
> http://liquid-reality.de/display/liquid/2015/03/05/Apache+Karaf+Tutorial+Par
> t+9+-+Annotation+based+blueprint+and+JPA
>
> Btw. Do you plan to use blueprint or DS?
>
> Christian
>
> On 03.09.2015 22:51, Matthew.W.Pitts@wellsfargo.com wrote:
>> Ok, maybe I can answer my own questions here, but if I could get some
>> confirmation that would be great. I seem to have a working system running
>> with ops4j-pax-jdbc-pool-aries rather than dbcp2.
>>
>> Digging into the code in org.apache.aries.transaction.jdbc it looks like
>> RecoverableDataSource is the wrapper for my XADataSources and it does
> indeed
>> appear to support passing of pooling options onto ConnectionManagerFactory
>> which itself appears to use the pooling capabilities that come from the
>> Geronimo bundles. Does all this sound correct so far?
>>
>> So, it's not dbcp2, but it's pooling nonetheless and I can adjust
>> preferences by passing the appropriate bean options to my pax-jdbc service
>> with the 'pool.' prefix which will in-turn send those on to the
>> RecoverableDataSource instance. Sound ok still?
>>
>> And finally, my eclipselink JTA datasource JNDI lookups are formatted with
>> filters for the 'aries.managed' property [1] to ensure that I get the
> aries
>> auto-enlisting DataSources.
>>
>> So far, this is running my existing code ok with only two changes I had to
>> make:
>>
>> 1. Fixed ARIES-1171, but actually in the Tranql code as I was getting
>> connection errors and didn't see a logical place to fix it from an aries
>> class (using latest aries-transaction-jdbc from SVN). This required
> bringing
>> all the tranql source into my aries bundle and just removing the external
>> dependency. I didn't like doing that, but since that code hasn't changed
> in
>> a while and I'm still prototyping all this it was the path of least
>> resistance.
>>
>> 2. Changed RecoverableDataSource to not use empty strings by default for
>> username and password. Which seemed to be similar/related to the above and
> I
>> posted as ARIES-1376.
>>
>> Thoughts and criticisms welcome. Thanks!
>>
>> [1]
>>
> osgi:service/javax.sql.DataSource/(&(osgi.jndi.service.name=my-app-node1)(ar
>> ies.managed=true))
>>
>> -----Original Message-----
>> From: Matthew.W.Pitts@wellsfargo.com
> [mailto:Matthew.W.Pitts@wellsfargo.com]
>> Sent: Thursday, September 03, 2015 10:58 AM
>> To: user@aries.apache.org
>> Subject: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?
>>
>> I am in the process of refactoring an existing JPA/Eclipselink-based
>> application to introduce Eclipselink's partitioning support for database
>> sharding. My current application runs quite well with pooled jdbc
>> connections via boncep.
>>
>> However, with the sharded model there will be a number of entity types
> that
>> are Replicated to all shards and, as such, eclipselink will be doing
> writes
>> that span multiple jdbc connections. Per the eclipselink recommendations
> for
>> paritioning I would like to introduce JTA in order to enforce integrity
>> across commits that hit multiple partitions.
>>
>> It seems like the ops4j-pax-jdbc bundles are designed to support XA/JTA
> and
>> it seems like there is a lot of XA support in the various aries bundles
>> (aries.transaction. and aries.jpa.). However, I'm not sure of what
>> combination of these bundles+configuration will provide what I am after.
>> Since many of these bundles seem to be designed/documented around general
>> JTA usage - like performing a transaction that spans JMS and JDBC - I'm
> not
>> confident if that model works with my particular use case.
>>
>> To further add to this, I would like for my JDBC connections to be pooled
>> for performance. ops4j-pax-jdbc-pool-dbcp2 seems like a logical choice,
> but
>> again I'm not sure if it will play nicely with the transaction management
>> features from aries.
>>
>> I think I have a working model of this, but I'm rather new to XA/JTA
>> especially from OSGi so I want to make sure I'm getting this correct. So,
> I
>> guess my questions are:
>>
>> 1. Can ops4j-pax-jdbc-pool and it's dbcp2 sibling work with
>> aries-jpa/transaction to pool and auto-enlist XADataSources?
>>
>> 2. If not, will ops4j-pax-jdbc-pool-aries work? And if so, can its pooling
>> semantics be configured? E.g. minIdle, maxIdle, etc.
>>
>> 3. If neither of the above works, is there a recommended set of bundles to
>> accomplish pooled XADataSources with eclipselink+JTA?
>>
>>
>> Thanks,
>>
>>
>> Matthew Pitts
>>    
>> Developer
>> Security Solutions Design & Automation
>>    
>> Wells Fargo Bank | Tel 336.608.3332 | Cell 336.202.3913 | Kernersville, NC
> |
>> MAC D9693-010
>> matthew.w.pitts@wellsfargo.com
>>    
>>
>

RE: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?

[Ma...@wellsfargo.com]

[1] is a snippet from the stacktrace I get when using pax-jdbc-pool-dbcp2
instead of -aries. As you can see,
ManagedJDBCConnection#localTransactionStart appears to call setAutoCommit on
the connection class, which the DBCP2 impl short circuits at [2].

This same exact environment seems to run fine with the -aries pool bundle. I
am running this with a customized transactions-jdbc bundle checked-out from
SVN. But, the only change I made was the pull in all the tranql source to
fix issues with the two-arg form of getConnection in some of those classes.
Aside from that it's unchanged and works fine with the -aries pooling
bundle.

Any suggestions are greatly appreciated.

[1]
Caused by: javax.resource.spi.LocalTransactionException: Unable to disable
autoCommit
	at
org.tranql.connector.jdbc.ManagedJDBCConnection.localTransactionStart(Manage
dJDBCConnection.java:83)
~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
	at
org.tranql.connector.AbstractManagedConnection$LocalTransactionImpl.begin(Ab
stractManagedConnection.java:195)
~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
	at
org.apache.geronimo.connector.outbound.LocalXAResource.start(LocalXAResource
.java:107) ~[geronimo-connector-3.1.3.jar:3.1.3]
	... 76 common frames omitted
Caused by: java.sql.SQLException: Auto-commit can not be set while enrolled
in a transaction
	at
org.apache.commons.dbcp2.managed.ManagedConnection.setAutoCommit(ManagedConn
ection.java:223) ~[na:na]
	at
org.tranql.connector.jdbc.ManagedJDBCConnection.localTransactionStart(Manage
dJDBCConnection.java:81)
~[org.apache.aries.transaction.jdbc-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
	... 78 common frames omitted

[2]
public void setAutoCommit(boolean autoCommit) throws SQLException {
    if (transactionContext != null) {
 
throw new SQLException("Auto-commit can not be set while enrolled in a trans
action");
    }
    super.setAutoCommit(autoCommit);
}


-----Original Message-----
From: Matthew.W.Pitts@wellsfargo.com [mailto:Matthew.W.Pitts@wellsfargo.com]

Sent: Tuesday, September 22, 2015 11:14 AM
To: user@aries.apache.org
Subject: RE: Bundle set recommendations for JPA+JTA+Eclipselink
Partitioning?

Thanks very much for the references, I will definitely be looking over them.

I was hoping the dbcp2 bundle would allow for the same XA capability, as I
would prefer dbcp2 pooling since it seems fairly analogous to my current
pooling - bonecp. However, I did some testing with dbcp2 and got exceptions
during runtime when code from base datasource wrappers (from tranql maybe)
was attempting to set the auto-commit flag and the DBCP datasource wrapper
class was throwing an exception about setting auto-commit. I can't remember
exactly the code flow but I'll try to test this again and see about getting
a stacktrace. Does any of this sound familiar? Should I be able to simply
replace pax-jdbc-pool-aries with pax-jdbc-pool-dbcp2 in my runtime? Maybe it
was something in my jndi references not getting the aries-managed datasource
services.

As for the blueprint/DS question, I will be largely using DS to inject
app-specific API that gives my code access to the JpaTemplate/EntityManager
classes. I hope to move toward blueprint, but that will be another effort.

Thanks again,
-matt

-----Original Message-----
From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
Christian Schneider
Sent: Tuesday, September 22, 2015 10:24 AM
To: user@aries.apache.org
Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink
Partitioning?

Hi Matthew,

pax-jdbc-pool-dbcp2 as well as pax-jdbc-pool-aries should both be able 
to provide XA enlisting poolable DataSources.
Aries JPA and Transaction actually use the above in their integration 
tests. So they should work together nicely.

I got two tutorials that both use XA transactions with Aries:
http://liquid-reality.de/display/liquid/2015/06/30/Apache+Karaf+Tutorial+par
t+10+-+Declarative+services
http://liquid-reality.de/display/liquid/2015/03/05/Apache+Karaf+Tutorial+Par
t+9+-+Annotation+based+blueprint+and+JPA

Btw. Do you plan to use blueprint or DS?

Christian

On 03.09.2015 22:51, Matthew.W.Pitts@wellsfargo.com wrote:
> Ok, maybe I can answer my own questions here, but if I could get some
> confirmation that would be great. I seem to have a working system running
> with ops4j-pax-jdbc-pool-aries rather than dbcp2.
>
> Digging into the code in org.apache.aries.transaction.jdbc it looks like
> RecoverableDataSource is the wrapper for my XADataSources and it does
indeed
> appear to support passing of pooling options onto ConnectionManagerFactory
> which itself appears to use the pooling capabilities that come from the
> Geronimo bundles. Does all this sound correct so far?
>
> So, it's not dbcp2, but it's pooling nonetheless and I can adjust
> preferences by passing the appropriate bean options to my pax-jdbc service
> with the 'pool.' prefix which will in-turn send those on to the
> RecoverableDataSource instance. Sound ok still?
>
> And finally, my eclipselink JTA datasource JNDI lookups are formatted with
> filters for the 'aries.managed' property [1] to ensure that I get the
aries
> auto-enlisting DataSources.
>
> So far, this is running my existing code ok with only two changes I had to
> make:
>
> 1. Fixed ARIES-1171, but actually in the Tranql code as I was getting
> connection errors and didn't see a logical place to fix it from an aries
> class (using latest aries-transaction-jdbc from SVN). This required
bringing
> all the tranql source into my aries bundle and just removing the external
> dependency. I didn't like doing that, but since that code hasn't changed
in
> a while and I'm still prototyping all this it was the path of least
> resistance.
>
> 2. Changed RecoverableDataSource to not use empty strings by default for
> username and password. Which seemed to be similar/related to the above and
I
> posted as ARIES-1376.
>
> Thoughts and criticisms welcome. Thanks!
>
> [1]
>
osgi:service/javax.sql.DataSource/(&(osgi.jndi.service.name=my-app-node1)(ar
> ies.managed=true))
>
> -----Original Message-----
> From: Matthew.W.Pitts@wellsfargo.com
[mailto:Matthew.W.Pitts@wellsfargo.com]
>
> Sent: Thursday, September 03, 2015 10:58 AM
> To: user@aries.apache.org
> Subject: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?
>
> I am in the process of refactoring an existing JPA/Eclipselink-based
> application to introduce Eclipselink's partitioning support for database
> sharding. My current application runs quite well with pooled jdbc
> connections via boncep.
>
> However, with the sharded model there will be a number of entity types
that
> are Replicated to all shards and, as such, eclipselink will be doing
writes
> that span multiple jdbc connections. Per the eclipselink recommendations
for
> paritioning I would like to introduce JTA in order to enforce integrity
> across commits that hit multiple partitions.
>
> It seems like the ops4j-pax-jdbc bundles are designed to support XA/JTA
and
> it seems like there is a lot of XA support in the various aries bundles
> (aries.transaction. and aries.jpa.). However, I'm not sure of what
> combination of these bundles+configuration will provide what I am after.
> Since many of these bundles seem to be designed/documented around general
> JTA usage - like performing a transaction that spans JMS and JDBC - I'm
not
> confident if that model works with my particular use case.
>
> To further add to this, I would like for my JDBC connections to be pooled
> for performance. ops4j-pax-jdbc-pool-dbcp2 seems like a logical choice,
but
> again I'm not sure if it will play nicely with the transaction management
> features from aries.
>
> I think I have a working model of this, but I'm rather new to XA/JTA
> especially from OSGi so I want to make sure I'm getting this correct. So,
I
> guess my questions are:
>
> 1. Can ops4j-pax-jdbc-pool and it's dbcp2 sibling work with
> aries-jpa/transaction to pool and auto-enlist XADataSources?
>
> 2. If not, will ops4j-pax-jdbc-pool-aries work? And if so, can its pooling
> semantics be configured? E.g. minIdle, maxIdle, etc.
>
> 3. If neither of the above works, is there a recommended set of bundles to
> accomplish pooled XADataSources with eclipselink+JTA?
>
>
> Thanks,
>
>
> Matthew Pitts
>   
> Developer
> Security Solutions Design & Automation
>   
> Wells Fargo Bank | Tel 336.608.3332 | Cell 336.202.3913 | Kernersville, NC
|
> MAC D9693-010
> matthew.w.pitts@wellsfargo.com
>   
>


-- 
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com

RE: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?

[Ma...@wellsfargo.com]

Thanks very much for the references, I will definitely be looking over them.

I was hoping the dbcp2 bundle would allow for the same XA capability, as I
would prefer dbcp2 pooling since it seems fairly analogous to my current
pooling - bonecp. However, I did some testing with dbcp2 and got exceptions
during runtime when code from base datasource wrappers (from tranql maybe)
was attempting to set the auto-commit flag and the DBCP datasource wrapper
class was throwing an exception about setting auto-commit. I can't remember
exactly the code flow but I'll try to test this again and see about getting
a stacktrace. Does any of this sound familiar? Should I be able to simply
replace pax-jdbc-pool-aries with pax-jdbc-pool-dbcp2 in my runtime? Maybe it
was something in my jndi references not getting the aries-managed datasource
services.

As for the blueprint/DS question, I will be largely using DS to inject
app-specific API that gives my code access to the JpaTemplate/EntityManager
classes. I hope to move toward blueprint, but that will be another effort.

Thanks again,
-matt

-----Original Message-----
From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
Christian Schneider
Sent: Tuesday, September 22, 2015 10:24 AM
To: user@aries.apache.org
Subject: Re: Bundle set recommendations for JPA+JTA+Eclipselink
Partitioning?

Hi Matthew,

pax-jdbc-pool-dbcp2 as well as pax-jdbc-pool-aries should both be able 
to provide XA enlisting poolable DataSources.
Aries JPA and Transaction actually use the above in their integration 
tests. So they should work together nicely.

I got two tutorials that both use XA transactions with Aries:
http://liquid-reality.de/display/liquid/2015/06/30/Apache+Karaf+Tutorial+par
t+10+-+Declarative+services
http://liquid-reality.de/display/liquid/2015/03/05/Apache+Karaf+Tutorial+Par
t+9+-+Annotation+based+blueprint+and+JPA

Btw. Do you plan to use blueprint or DS?

Christian

On 03.09.2015 22:51, Matthew.W.Pitts@wellsfargo.com wrote:
> Ok, maybe I can answer my own questions here, but if I could get some
> confirmation that would be great. I seem to have a working system running
> with ops4j-pax-jdbc-pool-aries rather than dbcp2.
>
> Digging into the code in org.apache.aries.transaction.jdbc it looks like
> RecoverableDataSource is the wrapper for my XADataSources and it does
indeed
> appear to support passing of pooling options onto ConnectionManagerFactory
> which itself appears to use the pooling capabilities that come from the
> Geronimo bundles. Does all this sound correct so far?
>
> So, it's not dbcp2, but it's pooling nonetheless and I can adjust
> preferences by passing the appropriate bean options to my pax-jdbc service
> with the 'pool.' prefix which will in-turn send those on to the
> RecoverableDataSource instance. Sound ok still?
>
> And finally, my eclipselink JTA datasource JNDI lookups are formatted with
> filters for the 'aries.managed' property [1] to ensure that I get the
aries
> auto-enlisting DataSources.
>
> So far, this is running my existing code ok with only two changes I had to
> make:
>
> 1. Fixed ARIES-1171, but actually in the Tranql code as I was getting
> connection errors and didn't see a logical place to fix it from an aries
> class (using latest aries-transaction-jdbc from SVN). This required
bringing
> all the tranql source into my aries bundle and just removing the external
> dependency. I didn't like doing that, but since that code hasn't changed
in
> a while and I'm still prototyping all this it was the path of least
> resistance.
>
> 2. Changed RecoverableDataSource to not use empty strings by default for
> username and password. Which seemed to be similar/related to the above and
I
> posted as ARIES-1376.
>
> Thoughts and criticisms welcome. Thanks!
>
> [1]
>
osgi:service/javax.sql.DataSource/(&(osgi.jndi.service.name=my-app-node1)(ar
> ies.managed=true))
>
> -----Original Message-----
> From: Matthew.W.Pitts@wellsfargo.com
[mailto:Matthew.W.Pitts@wellsfargo.com]
>
> Sent: Thursday, September 03, 2015 10:58 AM
> To: user@aries.apache.org
> Subject: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?
>
> I am in the process of refactoring an existing JPA/Eclipselink-based
> application to introduce Eclipselink's partitioning support for database
> sharding. My current application runs quite well with pooled jdbc
> connections via boncep.
>
> However, with the sharded model there will be a number of entity types
that
> are Replicated to all shards and, as such, eclipselink will be doing
writes
> that span multiple jdbc connections. Per the eclipselink recommendations
for
> paritioning I would like to introduce JTA in order to enforce integrity
> across commits that hit multiple partitions.
>
> It seems like the ops4j-pax-jdbc bundles are designed to support XA/JTA
and
> it seems like there is a lot of XA support in the various aries bundles
> (aries.transaction. and aries.jpa.). However, I'm not sure of what
> combination of these bundles+configuration will provide what I am after.
> Since many of these bundles seem to be designed/documented around general
> JTA usage - like performing a transaction that spans JMS and JDBC - I'm
not
> confident if that model works with my particular use case.
>
> To further add to this, I would like for my JDBC connections to be pooled
> for performance. ops4j-pax-jdbc-pool-dbcp2 seems like a logical choice,
but
> again I'm not sure if it will play nicely with the transaction management
> features from aries.
>
> I think I have a working model of this, but I'm rather new to XA/JTA
> especially from OSGi so I want to make sure I'm getting this correct. So,
I
> guess my questions are:
>
> 1. Can ops4j-pax-jdbc-pool and it's dbcp2 sibling work with
> aries-jpa/transaction to pool and auto-enlist XADataSources?
>
> 2. If not, will ops4j-pax-jdbc-pool-aries work? And if so, can its pooling
> semantics be configured? E.g. minIdle, maxIdle, etc.
>
> 3. If neither of the above works, is there a recommended set of bundles to
> accomplish pooled XADataSources with eclipselink+JTA?
>
>
> Thanks,
>
>
> Matthew Pitts
>   
> Developer
> Security Solutions Design & Automation
>   
> Wells Fargo Bank | Tel 336.608.3332 | Cell 336.202.3913 | Kernersville, NC
|
> MAC D9693-010
> matthew.w.pitts@wellsfargo.com
>   
>


-- 
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com

Re: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?

[Christian Schneider <ch...@die-schneider.net>]

Hi Matthew,

pax-jdbc-pool-dbcp2 as well as pax-jdbc-pool-aries should both be able 
to provide XA enlisting poolable DataSources.
Aries JPA and Transaction actually use the above in their integration 
tests. So they should work together nicely.

I got two tutorials that both use XA transactions with Aries:
http://liquid-reality.de/display/liquid/2015/06/30/Apache+Karaf+Tutorial+part+10+-+Declarative+services
http://liquid-reality.de/display/liquid/2015/03/05/Apache+Karaf+Tutorial+Part+9+-+Annotation+based+blueprint+and+JPA

Btw. Do you plan to use blueprint or DS?

Christian

On 03.09.2015 22:51, Matthew.W.Pitts@wellsfargo.com wrote:
> Ok, maybe I can answer my own questions here, but if I could get some
> confirmation that would be great. I seem to have a working system running
> with ops4j-pax-jdbc-pool-aries rather than dbcp2.
>
> Digging into the code in org.apache.aries.transaction.jdbc it looks like
> RecoverableDataSource is the wrapper for my XADataSources and it does indeed
> appear to support passing of pooling options onto ConnectionManagerFactory
> which itself appears to use the pooling capabilities that come from the
> Geronimo bundles. Does all this sound correct so far?
>
> So, it's not dbcp2, but it's pooling nonetheless and I can adjust
> preferences by passing the appropriate bean options to my pax-jdbc service
> with the 'pool.' prefix which will in-turn send those on to the
> RecoverableDataSource instance. Sound ok still?
>
> And finally, my eclipselink JTA datasource JNDI lookups are formatted with
> filters for the 'aries.managed' property [1] to ensure that I get the aries
> auto-enlisting DataSources.
>
> So far, this is running my existing code ok with only two changes I had to
> make:
>
> 1. Fixed ARIES-1171, but actually in the Tranql code as I was getting
> connection errors and didn't see a logical place to fix it from an aries
> class (using latest aries-transaction-jdbc from SVN). This required bringing
> all the tranql source into my aries bundle and just removing the external
> dependency. I didn't like doing that, but since that code hasn't changed in
> a while and I'm still prototyping all this it was the path of least
> resistance.
>
> 2. Changed RecoverableDataSource to not use empty strings by default for
> username and password. Which seemed to be similar/related to the above and I
> posted as ARIES-1376.
>
> Thoughts and criticisms welcome. Thanks!
>
> [1]
> osgi:service/javax.sql.DataSource/(&(osgi.jndi.service.name=my-app-node1)(ar
> ies.managed=true))
>
> -----Original Message-----
> From: Matthew.W.Pitts@wellsfargo.com [mailto:Matthew.W.Pitts@wellsfargo.com]
>
> Sent: Thursday, September 03, 2015 10:58 AM
> To: user@aries.apache.org
> Subject: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?
>
> I am in the process of refactoring an existing JPA/Eclipselink-based
> application to introduce Eclipselink's partitioning support for database
> sharding. My current application runs quite well with pooled jdbc
> connections via boncep.
>
> However, with the sharded model there will be a number of entity types that
> are Replicated to all shards and, as such, eclipselink will be doing writes
> that span multiple jdbc connections. Per the eclipselink recommendations for
> paritioning I would like to introduce JTA in order to enforce integrity
> across commits that hit multiple partitions.
>
> It seems like the ops4j-pax-jdbc bundles are designed to support XA/JTA and
> it seems like there is a lot of XA support in the various aries bundles
> (aries.transaction. and aries.jpa.). However, I'm not sure of what
> combination of these bundles+configuration will provide what I am after.
> Since many of these bundles seem to be designed/documented around general
> JTA usage - like performing a transaction that spans JMS and JDBC - I'm not
> confident if that model works with my particular use case.
>
> To further add to this, I would like for my JDBC connections to be pooled
> for performance. ops4j-pax-jdbc-pool-dbcp2 seems like a logical choice, but
> again I'm not sure if it will play nicely with the transaction management
> features from aries.
>
> I think I have a working model of this, but I'm rather new to XA/JTA
> especially from OSGi so I want to make sure I'm getting this correct. So, I
> guess my questions are:
>
> 1. Can ops4j-pax-jdbc-pool and it's dbcp2 sibling work with
> aries-jpa/transaction to pool and auto-enlist XADataSources?
>
> 2. If not, will ops4j-pax-jdbc-pool-aries work? And if so, can its pooling
> semantics be configured? E.g. minIdle, maxIdle, etc.
>
> 3. If neither of the above works, is there a recommended set of bundles to
> accomplish pooled XADataSources with eclipselink+JTA?
>
>
> Thanks,
>
>
> Matthew Pitts
>   
> Developer
> Security Solutions Design & Automation
>   
> Wells Fargo Bank | Tel 336.608.3332 | Cell 336.202.3913 | Kernersville, NC |
> MAC D9693-010
> matthew.w.pitts@wellsfargo.com
>   
>


-- 
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com

Re: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?

[Charlie Mordant <cm...@gmail.com>]

Hi Matthew,

I've got a working sample of OpenJPA/Pax-JDBC-pool-Aries/Aries JPA
(1)/Aries JTA/ActiveMQ/Karaf 3.0.3 here:
https://github.com/OsgiliathEnterprise/net.osgiliath.parent/tree/master/net.osgiliath.samples

Feel free to use the archetypes, features, reuse parts, or contribute!

Best regards,
Charlie

2015-09-03 22:51 GMT+02:00 <Ma...@wellsfargo.com>:

> Ok, maybe I can answer my own questions here, but if I could get some
> confirmation that would be great. I seem to have a working system running
> with ops4j-pax-jdbc-pool-aries rather than dbcp2.
>
> Digging into the code in org.apache.aries.transaction.jdbc it looks like
> RecoverableDataSource is the wrapper for my XADataSources and it does
> indeed
> appear to support passing of pooling options onto ConnectionManagerFactory
> which itself appears to use the pooling capabilities that come from the
> Geronimo bundles. Does all this sound correct so far?
>
> So, it's not dbcp2, but it's pooling nonetheless and I can adjust
> preferences by passing the appropriate bean options to my pax-jdbc service
> with the 'pool.' prefix which will in-turn send those on to the
> RecoverableDataSource instance. Sound ok still?
>
> And finally, my eclipselink JTA datasource JNDI lookups are formatted with
> filters for the 'aries.managed' property [1] to ensure that I get the aries
> auto-enlisting DataSources.
>
> So far, this is running my existing code ok with only two changes I had to
> make:
>
> 1. Fixed ARIES-1171, but actually in the Tranql code as I was getting
> connection errors and didn't see a logical place to fix it from an aries
> class (using latest aries-transaction-jdbc from SVN). This required
> bringing
> all the tranql source into my aries bundle and just removing the external
> dependency. I didn't like doing that, but since that code hasn't changed in
> a while and I'm still prototyping all this it was the path of least
> resistance.
>
> 2. Changed RecoverableDataSource to not use empty strings by default for
> username and password. Which seemed to be similar/related to the above and
> I
> posted as ARIES-1376.
>
> Thoughts and criticisms welcome. Thanks!
>
> [1]
> osgi:service/javax.sql.DataSource/(&(osgi.jndi.service.name
> =my-app-node1)(ar
> ies.managed=true))
>
> -----Original Message-----
> From: Matthew.W.Pitts@wellsfargo.com [mailto:
> Matthew.W.Pitts@wellsfargo.com]
>
> Sent: Thursday, September 03, 2015 10:58 AM
> To: user@aries.apache.org
> Subject: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?
>
> I am in the process of refactoring an existing JPA/Eclipselink-based
> application to introduce Eclipselink's partitioning support for database
> sharding. My current application runs quite well with pooled jdbc
> connections via boncep.
>
> However, with the sharded model there will be a number of entity types that
> are Replicated to all shards and, as such, eclipselink will be doing writes
> that span multiple jdbc connections. Per the eclipselink recommendations
> for
> paritioning I would like to introduce JTA in order to enforce integrity
> across commits that hit multiple partitions.
>
> It seems like the ops4j-pax-jdbc bundles are designed to support XA/JTA and
> it seems like there is a lot of XA support in the various aries bundles
> (aries.transaction. and aries.jpa.). However, I'm not sure of what
> combination of these bundles+configuration will provide what I am after.
> Since many of these bundles seem to be designed/documented around general
> JTA usage - like performing a transaction that spans JMS and JDBC - I'm not
> confident if that model works with my particular use case.
>
> To further add to this, I would like for my JDBC connections to be pooled
> for performance. ops4j-pax-jdbc-pool-dbcp2 seems like a logical choice, but
> again I'm not sure if it will play nicely with the transaction management
> features from aries.
>
> I think I have a working model of this, but I'm rather new to XA/JTA
> especially from OSGi so I want to make sure I'm getting this correct. So, I
> guess my questions are:
>
> 1. Can ops4j-pax-jdbc-pool and it's dbcp2 sibling work with
> aries-jpa/transaction to pool and auto-enlist XADataSources?
>
> 2. If not, will ops4j-pax-jdbc-pool-aries work? And if so, can its pooling
> semantics be configured? E.g. minIdle, maxIdle, etc.
>
> 3. If neither of the above works, is there a recommended set of bundles to
> accomplish pooled XADataSources with eclipselink+JTA?
>
>
> Thanks,
>
>
> Matthew Pitts
>
> Developer
> Security Solutions Design & Automation
>
> Wells Fargo Bank | Tel 336.608.3332 | Cell 336.202.3913 | Kernersville,
> NC |
> MAC D9693-010
> matthew.w.pitts@wellsfargo.com
>
>
>


-- 
Charlie Mordant

Full OSGI/EE stack made with Karaf:
https://github.com/OsgiliathEnterprise/net.osgiliath.parent

RE: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?

[Ma...@wellsfargo.com]

Ok, maybe I can answer my own questions here, but if I could get some
confirmation that would be great. I seem to have a working system running
with ops4j-pax-jdbc-pool-aries rather than dbcp2. 

Digging into the code in org.apache.aries.transaction.jdbc it looks like
RecoverableDataSource is the wrapper for my XADataSources and it does indeed
appear to support passing of pooling options onto ConnectionManagerFactory
which itself appears to use the pooling capabilities that come from the
Geronimo bundles. Does all this sound correct so far?

So, it's not dbcp2, but it's pooling nonetheless and I can adjust
preferences by passing the appropriate bean options to my pax-jdbc service
with the 'pool.' prefix which will in-turn send those on to the
RecoverableDataSource instance. Sound ok still?

And finally, my eclipselink JTA datasource JNDI lookups are formatted with
filters for the 'aries.managed' property [1] to ensure that I get the aries
auto-enlisting DataSources.

So far, this is running my existing code ok with only two changes I had to
make:

1. Fixed ARIES-1171, but actually in the Tranql code as I was getting
connection errors and didn't see a logical place to fix it from an aries
class (using latest aries-transaction-jdbc from SVN). This required bringing
all the tranql source into my aries bundle and just removing the external
dependency. I didn't like doing that, but since that code hasn't changed in
a while and I'm still prototyping all this it was the path of least
resistance.

2. Changed RecoverableDataSource to not use empty strings by default for
username and password. Which seemed to be similar/related to the above and I
posted as ARIES-1376. 

Thoughts and criticisms welcome. Thanks!

[1]
osgi:service/javax.sql.DataSource/(&(osgi.jndi.service.name=my-app-node1)(ar
ies.managed=true))

-----Original Message-----
From: Matthew.W.Pitts@wellsfargo.com [mailto:Matthew.W.Pitts@wellsfargo.com]

Sent: Thursday, September 03, 2015 10:58 AM
To: user@aries.apache.org
Subject: Bundle set recommendations for JPA+JTA+Eclipselink Partitioning?

I am in the process of refactoring an existing JPA/Eclipselink-based
application to introduce Eclipselink's partitioning support for database
sharding. My current application runs quite well with pooled jdbc
connections via boncep.

However, with the sharded model there will be a number of entity types that
are Replicated to all shards and, as such, eclipselink will be doing writes
that span multiple jdbc connections. Per the eclipselink recommendations for
paritioning I would like to introduce JTA in order to enforce integrity
across commits that hit multiple partitions.

It seems like the ops4j-pax-jdbc bundles are designed to support XA/JTA and
it seems like there is a lot of XA support in the various aries bundles
(aries.transaction. and aries.jpa.). However, I'm not sure of what
combination of these bundles+configuration will provide what I am after.
Since many of these bundles seem to be designed/documented around general
JTA usage - like performing a transaction that spans JMS and JDBC - I'm not
confident if that model works with my particular use case.

To further add to this, I would like for my JDBC connections to be pooled
for performance. ops4j-pax-jdbc-pool-dbcp2 seems like a logical choice, but
again I'm not sure if it will play nicely with the transaction management
features from aries.

I think I have a working model of this, but I'm rather new to XA/JTA
especially from OSGi so I want to make sure I'm getting this correct. So, I
guess my questions are:

1. Can ops4j-pax-jdbc-pool and it's dbcp2 sibling work with
aries-jpa/transaction to pool and auto-enlist XADataSources?

2. If not, will ops4j-pax-jdbc-pool-aries work? And if so, can its pooling
semantics be configured? E.g. minIdle, maxIdle, etc.

3. If neither of the above works, is there a recommended set of bundles to
accomplish pooled XADataSources with eclipselink+JTA?


Thanks,


Matthew Pitts
 
Developer
Security Solutions Design & Automation
 
Wells Fargo Bank | Tel 336.608.3332 | Cell 336.202.3913 | Kernersville, NC |
MAC D9693-010
matthew.w.pitts@wellsfargo.com