You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cayenne.apache.org by "Durchholz, Joachim" <Jo...@hennig-fahrzeugteile.de> on 2012/05/02 11:11:48 UTC
RE: Standalone app, Threads and Object contexts
> Unfortunately, I may have up to 200 simultaneous requests
> at the database.
Not a problem.
> I'm sure the database administrator won't like to see my
> app opening and closing connections to their database
> so I would prefer to open up a pool of connections and
> then share them among the incoming requests much like a
> web app.
Standalone apps tend to work by opening a single Connection
and using that for all requests.
The snag here is that connections will break sooner or later
(due to networking problems, server restarts, idle time, or
a multitude of other causes), so this needs a library that
recreates a broken connection on the fly.
Connection pooling libraries do that as a side effect. That
means that you use such a library, but not for the pooling
(no need to share connection across "requests", there is no
such thing in a J2SE application unless your design calls
for one due to other reasons) but for the side effect of
recreating connections as needed.
Since I'm still stuck with Hibernate, I can't advise very
well about how to best integrate a connection pooling
library and Cayenne.
I'd probably look up the Cayenne docs on connection pooling,
and just make sure that everything is configured explicitly
that the web container would do for you implicitly.
BTW I've been working on a J2SE application for the last
three years, and it's probably really a good idea to design
the application around "requests" or "transactions".
Transactions can fail due to transient errors, so you want
them restartable, which means you need to wrap them in
objects, and there you have your requests. (This pattern
also sucks greatly because it isn't easy to pass results
back to the caller. Essentially, you're doing the exactly
same web service architecture, except it's running inside a
single JVM.)
Re: Standalone app, Threads and Object contexts
Posted by Joe Baldwin <jf...@earthlink.net>.
Not sure if this helps, but I have used both the default (Cayenne) connection pool, and DBCP.
http://commons.apache.org/dbcp/
Cayenne's implementation seems to be adequate, but if you determine that you are going to need something a bit more sophisticated, then DBCP is very reliable IMHO. I have been using DBCP for over a year and have had no issues yet (however, I also upgraded from MySQL 5.0 because of connection bugs reported in parts of that version branch).
Joe
On May 2, 2012, at 10:18 AM, John Huss wrote:
> The number of ObjectContexts (or DataContexts) has no relation to the
> number of database connections. ObjectContexts are cheap to create and
> use, so go wild. Cayenne will do connection pooling out of the box by
> using the minimum and maximum connection limits that you define in Modeler.
>
> John
>
> On Wed, May 2, 2012 at 8:05 AM, Andrew Willerding
> <aw...@itsurcom.com>wrote:
>
>> On 05/02/2012 05:11 AM, Durchholz, Joachim wrote:
>>
>>> I'd probably look up the Cayenne docs on connection pooling, and just
>>> make sure that everything is configured explicitly that the web container
>>> would do for you implicitly. BTW I've been working on a J2SE application
>>> for the last three years, and it's probably really a good idea to design
>>> the application around "requests" or "transactions". Transactions can fail
>>> due to transient errors, so you want them restartable, which means you need
>>> to wrap them in objects, and there you have your requests. (This pattern
>>> also sucks greatly because it isn't easy to pass results back to the
>>> caller. Essentially, you're doing the exactly same web service
>>> architecture, except it's running inside a single JVM.)
>>>
>> I got the impression that there was some sort of connection pooling
>> built-in to Cayenne because in the Modeller there is a setting for minimum
>> and maximum connections. Now I'm getting confused and perhaps I'm using
>> incorrect terminology.
>>
>> In my mind there is the actual connection to the database as defined
>> during the creation of the DataContext. In the application code I am
>> creating an ObjectContext based on the DataContext with a call to
>> DataContext.**createChildContext(). It is this ObjectContext where I can
>> create/delete manipulate the contents of the database.
>>
>> But in a multi-threaded application do I need to create several
>> ObjectContexts or can one ObjectContext be shared among all the threads?
>> If the ObjectContext is sharable then do I need to synchronize access to
>> it? If it needs to by synchronized then the "200th" thread will be in a
>> long line-up which is why I would see the need to create several pooled
>> ObjectContexts. But then this takes be back to connection pooling as
>> defined in the Modeller. If everything goes back through a single
>> connection then it's pointless to create and manage ObjectContexts as the
>> bottleneck may be "deeper" in the Cayenne library.
>>
>> Help! I getting trapped in my own recursive thoughts.
>>
>> ;-)
>>
Re: Standalone app, Threads and Object contexts
Posted by Mike Kienenberger <mk...@gmail.com>.
Unless something has changed recently, DataContexts are not thread-safe.
As has been stated before, creating a DataContext is a cheap operation
(populating it from the database is expensive, copying objects from
another data context is cheap).
DataContexts are not tied to connnection pools -- a connection is
temporarily assigned whenever you commit or query the database, and
then released back to the pool.
On Wed, May 2, 2012 at 11:34 AM, Andrew Willerding
<aw...@itsurcom.com> wrote:
> On 05/02/2012 10:18 AM, John Huss wrote:
>>
>> The number of ObjectContexts (or DataContexts) has no relation to the
>> number of database connections. ObjectContexts are cheap to create and
>> use, so go wild. Cayenne will do connection pooling out of the box by
>> using the minimum and maximum connection limits that you define in
>> Modeler.
>>
>> John
>>
>>
> That was what I was hoping for. Continue to let the "magic" take care of
> the nitty-gritty details so I can focus on my app.
>
> But just to confirm my understanding. I can create one
> DataContext(ObjectContext) and perform as many "simultaneous i.e. from
> multiple threads" object creations, object updates, stored procedure
> executions without concern as long as these creations/updates/stored
> procedure executions are non-related and independent meaning one thread
> that's doing a creation/update/stored procedure execution is not dependent
> on a creation/update/execution going on in another thread. Something like
> this
>
> public class Main{
> public static synchronized DataContext getDataContext() {
> if (dc == null) {
> String fileName = "etc/cayenne.xml";
> org.apache.cayenne.conf.FileConfiguration conf = new
> org.apache.cayenne.conf.FileConfiguration(fileName);
>
> org.apache.cayenne.conf.Configuration.initializeSharedConfiguration(conf);
>
> dc = DataContext.createDataContext();
> }
> return dc;
> }
> }
>
> public class myThread(){
>
> public void run(){
> DataContext dc = Main.getDataContext();
>
> // etc on object creation/update/stored procedure executions
> dc.commitChanges();
> }
> }
>
>
>
> Andrew
Re: Standalone app, Threads and Object contexts
Posted by Michael Gentry <mg...@masslight.net>.
Hi Andrew,
I do not believe you want to share a context like that. A DataContext
is NOT thread safe between different threads. Also, even if you
synchronized access to it further (to be thread safe), a single commit
failure (such as a constraint exception) would doom all future commits
because the shared DataContext is now dirty with uncommitable changes
and nothing else can commit, either.
Use individual DataContexts for related changes. If you re-use a
DataContext, it should be applicable to a previous request (just like
a web session re-attaches your HTTP Session).
mrg
On Wed, May 2, 2012 at 11:34 AM, Andrew Willerding
<aw...@itsurcom.com> wrote:
> But just to confirm my understanding. I can create one
> DataContext(ObjectContext) and perform as many "simultaneous i.e. from
> multiple threads" object creations, object updates, stored procedure
> executions without concern as long as these creations/updates/stored
> procedure executions are non-related and independent meaning one thread
> that's doing a creation/update/stored procedure execution is not dependent
> on a creation/update/execution going on in another thread. Something like
> this
Re: Standalone app, Threads and Object contexts
Posted by Andrew Willerding <aw...@itsurcom.com>.
On 05/02/2012 10:18 AM, John Huss wrote:
> The number of ObjectContexts (or DataContexts) has no relation to the
> number of database connections. ObjectContexts are cheap to create and
> use, so go wild. Cayenne will do connection pooling out of the box by
> using the minimum and maximum connection limits that you define in Modeler.
>
> John
>
>
That was what I was hoping for. Continue to let the "magic" take care
of the nitty-gritty details so I can focus on my app.
But just to confirm my understanding. I can create one
DataContext(ObjectContext) and perform as many "simultaneous i.e. from
multiple threads" object creations, object updates, stored procedure
executions without concern as long as these creations/updates/stored
procedure executions are non-related and independent meaning one thread
that's doing a creation/update/stored procedure execution is not
dependent on a creation/update/execution going on in another thread.
Something like this
public class Main{
public static synchronized DataContext getDataContext() {
if (dc == null) {
String fileName = "etc/cayenne.xml";
org.apache.cayenne.conf.FileConfiguration conf = new
org.apache.cayenne.conf.FileConfiguration(fileName);
org.apache.cayenne.conf.Configuration.initializeSharedConfiguration(conf);
dc = DataContext.createDataContext();
}
return dc;
}
}
public class myThread(){
public void run(){
DataContext dc = Main.getDataContext();
// etc on object creation/update/stored procedure executions
dc.commitChanges();
}
}
Andrew
Re: Standalone app, Threads and Object contexts
Posted by John Huss <jo...@gmail.com>.
The number of ObjectContexts (or DataContexts) has no relation to the
number of database connections. ObjectContexts are cheap to create and
use, so go wild. Cayenne will do connection pooling out of the box by
using the minimum and maximum connection limits that you define in Modeler.
John
On Wed, May 2, 2012 at 8:05 AM, Andrew Willerding
<aw...@itsurcom.com>wrote:
> On 05/02/2012 05:11 AM, Durchholz, Joachim wrote:
>
>> I'd probably look up the Cayenne docs on connection pooling, and just
>> make sure that everything is configured explicitly that the web container
>> would do for you implicitly. BTW I've been working on a J2SE application
>> for the last three years, and it's probably really a good idea to design
>> the application around "requests" or "transactions". Transactions can fail
>> due to transient errors, so you want them restartable, which means you need
>> to wrap them in objects, and there you have your requests. (This pattern
>> also sucks greatly because it isn't easy to pass results back to the
>> caller. Essentially, you're doing the exactly same web service
>> architecture, except it's running inside a single JVM.)
>>
> I got the impression that there was some sort of connection pooling
> built-in to Cayenne because in the Modeller there is a setting for minimum
> and maximum connections. Now I'm getting confused and perhaps I'm using
> incorrect terminology.
>
> In my mind there is the actual connection to the database as defined
> during the creation of the DataContext. In the application code I am
> creating an ObjectContext based on the DataContext with a call to
> DataContext.**createChildContext(). It is this ObjectContext where I can
> create/delete manipulate the contents of the database.
>
> But in a multi-threaded application do I need to create several
> ObjectContexts or can one ObjectContext be shared among all the threads?
> If the ObjectContext is sharable then do I need to synchronize access to
> it? If it needs to by synchronized then the "200th" thread will be in a
> long line-up which is why I would see the need to create several pooled
> ObjectContexts. But then this takes be back to connection pooling as
> defined in the Modeller. If everything goes back through a single
> connection then it's pointless to create and manage ObjectContexts as the
> bottleneck may be "deeper" in the Cayenne library.
>
> Help! I getting trapped in my own recursive thoughts.
>
> ;-)
>
RE: Standalone app, Threads and Object contexts
Posted by "Durchholz, Joachim" <Jo...@hennig-fahrzeugteile.de>.
> One thing I found that I must create at least one
> ObjectContextChild before the DataContext becomes
> usable otherwise I'm not seeing the connection being
> made to the database in the log files.
Just in case it's not already clear: A DataContext does not need a connection to be usable. It will grab a connection as soon as it actually needs one, i.e. when the application starts querying away.
Re: Standalone app, Threads and Object contexts
Posted by Michael Gentry <mg...@masslight.net>.
Hi Andrew,
I've written applications that never create a child context. Sounds
like you have something else going on if you must create a child
context before anything is usable. (This would be database
independent.)
mrg
On Wed, May 2, 2012 at 11:43 AM, Andrew Willerding
<aw...@itsurcom.com> wrote:
> One thing I found that I must create at least one ObjectContextChild before
> the DataContext becomes usable otherwise I'm not seeing the connection being
> made to the database in the log files. I'm using Cayenne 3.0.2 and an
> Oracle database. I can provide a log trace if required.
Re: Standalone app, Threads and Object contexts
Posted by Andrew Willerding <aw...@itsurcom.com>.
On 05/02/2012 09:46 AM, Durchholz, Joachim wrote:
> No, a DataContext is a context within which retrieving the same rows
> via some query will always give you identical Java objects. That's a
> pretty powerful (and important) property, but not necessarily related
> to a connection. (Connections become important as soon as you need
> transaction-related guarantees.) You don't need to create an
> ObjectContext to work with a DataContext - any DataContext already is
> an ObjectContext. You don't roll back a database transaction here, you
> roll back to whatever was last loaded from the database. If you use
> child contexts, you can roll back to whatever the state was when that
> child context was started. I believe that this means that you may have
> data loaded in different transactions in your application, and that
> that is the price you pay for getting rid of
> LazyInitializationExceptions. (True Cayenne experts, please correct if
> that's wrong. I'm writing this both to help Andrew along and to get
> helped along ;-) )
One thing I found that I must create at least one ObjectContextChild
before the DataContext becomes usable otherwise I'm not seeing the
connection being made to the database in the log files. I'm using
Cayenne 3.0.2 and an Oracle database. I can provide a log trace if
required.
RE: Standalone app, Threads and Object contexts
Posted by "Durchholz, Joachim" <Jo...@hennig-fahrzeugteile.de>.
> I got the impression that there was some sort of connection
> pooling built-in to Cayenne because in the Modeller there
> is a setting for minimum and maximum connections. Now I'm
> getting confused and perhaps I'm using incorrect terminology.
As I said, I'm still stuck with Hibernate, so I'm quite fuzzy about exact details here.
However, I think it is entirely possible that Cayenne already has some built-in connection pooling. In a J2SE context, it would essentially just be a connection recreator.
> In my mind there is the actual connection to the database
> as defined during the creation of the DataContext.
No, a DataContext is a context within which retrieving the same rows via some query will always give you identical Java objects.
That's a pretty powerful (and important) property, but not necessarily related to a connection. (Connections become important as soon as you need transaction-related guarantees.)
You don't need to create an ObjectContext to work with a DataContext - any DataContext already is an ObjectContext.
You don't roll back a database transaction here, you roll back to whatever was last loaded from the database.
If you use child contexts, you can roll back to whatever the state was when that child context was started.
I believe that this means that you may have data loaded in different transactions in your application, and that that is the price you pay for getting rid of LazyInitializationExceptions. (True Cayenne experts, please correct if that's wrong. I'm writing this both to help Andrew along and to get helped along ;-) )
Re: Standalone app, Threads and Object contexts
Posted by Andrew Willerding <aw...@itsurcom.com>.
On 05/02/2012 05:11 AM, Durchholz, Joachim wrote:
> I'd probably look up the Cayenne docs on connection pooling, and just
> make sure that everything is configured explicitly that the web
> container would do for you implicitly. BTW I've been working on a J2SE
> application for the last three years, and it's probably really a good
> idea to design the application around "requests" or "transactions".
> Transactions can fail due to transient errors, so you want them
> restartable, which means you need to wrap them in objects, and there
> you have your requests. (This pattern also sucks greatly because it
> isn't easy to pass results back to the caller. Essentially, you're
> doing the exactly same web service architecture, except it's running
> inside a single JVM.)
I got the impression that there was some sort of connection pooling
built-in to Cayenne because in the Modeller there is a setting for
minimum and maximum connections. Now I'm getting confused and perhaps
I'm using incorrect terminology.
In my mind there is the actual connection to the database as defined
during the creation of the DataContext. In the application code I am
creating an ObjectContext based on the DataContext with a call to
DataContext.createChildContext(). It is this ObjectContext where I can
create/delete manipulate the contents of the database.
But in a multi-threaded application do I need to create several
ObjectContexts or can one ObjectContext be shared among all the
threads? If the ObjectContext is sharable then do I need to synchronize
access to it? If it needs to by synchronized then the "200th" thread
will be in a long line-up which is why I would see the need to create
several pooled ObjectContexts. But then this takes be back to
connection pooling as defined in the Modeller. If everything goes back
through a single connection then it's pointless to create and manage
ObjectContexts as the bottleneck may be "deeper" in the Cayenne library.
Help! I getting trapped in my own recursive thoughts.
;-)
RE: Standalone app, Threads and Object contexts
Posted by "Durchholz, Joachim" <Jo...@hennig-fahrzeugteile.de>.
> But maybe I'm misunderstanding what "standalone" means.
> Does it mean Swing? Or a daemon with no UI? Or an
> application which needs to be a double clickable
> executable but still serves web pages?
In the sense that I understand the word, the first two would
usually be designed as a standalone (J2SE) app and the latter
one as a container (J2EE) app.
> In all three cases, Tomcat/Jetty/etc could still be
> part of the solution.
Could, but often wouldn't.
Tomcat etc. are built for high concurrency. Which means ultra-short transaction and connections and many parallel connections. That's the use case for containers and connection pooling.
If high concurrency is not an issue, you could have just a handful of Swing application banging on the same database, or even just a sole daemon. In these scenarios, you don't need the container for its high-concurrency guarantees.
Of course that doesn't mean that a container would necessarily be useless in such scenarios. A container enforces a design that makes retrying transactions easier, a connection pool ensures that that broken connections will be recreated ASAP. Both services may or may not be useful to a "standalone" (i.e. non-web-based) application.
Re: Standalone app, Threads and Object contexts
Posted by Andrew Willerding <aw...@itsurcom.com>.
I'm a little confused by this email thread because it started off as
questions about derby and then morphed into connection pooling. There
are lots of connection pooling libraries out there such as c3po and
dbcp. But I'll step out of this conversation since perhaps I'm not
understanding what is being asked.
>
> Ari
>
>
My mistake. I accidentally posted a reply instead of creating a new post.
Re: Standalone app, Threads and Object contexts
Posted by Aristedes Maniatis <ar...@maniatis.org>.
On 2/05/12 8:38pm, Durchholz, Joachim wrote:
>> On 2/05/12 7:11pm, Durchholz, Joachim wrote:
>>
>>> I'd probably look up the Cayenne docs on connection pooling, and just
>>> make sure that everything is configured explicitly that the web
>>> container would do for you implicitly.
>>
>> There is actually nothing at all to do in Cayenne. If you are using
>> tomcat, put your datasource into conf/server.xml and you are done.
>
> The Subject says "standalone app".
> My understanding is that this implies not having Tomcat or any other J2EE container.
No idea why that would be implied. But maybe I'm misunderstanding what "standalone" means. Does it mean Swing? Or a daemon with no UI? Or an application which needs to be a double clickable executable but still serves web pages? In all three cases, Tomcat/Jetty/etc could still be part of the solution.
I'm a little confused by this email thread because it started off as questions about derby and then morphed into connection pooling. There are lots of connection pooling libraries out there such as c3po and dbcp. But I'll step out of this conversation since perhaps I'm not understanding what is being asked.
Ari
--
-------------------------->
Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
RE: Standalone app, Threads and Object contexts
Posted by "Durchholz, Joachim" <Jo...@hennig-fahrzeugteile.de>.
> On 2/05/12 7:11pm, Durchholz, Joachim wrote:
>
>> I'd probably look up the Cayenne docs on connection pooling, and just
>> make sure that everything is configured explicitly that the web
>> container would do for you implicitly.
>
> There is actually nothing at all to do in Cayenne. If you are using
> tomcat, put your datasource into conf/server.xml and you are done.
The Subject says "standalone app".
My understanding is that this implies not having Tomcat or any other J2EE container.
Re: Standalone app, Threads and Object contexts
Posted by Aristedes Maniatis <ar...@maniatis.org>.
On 2/05/12 7:11pm, Durchholz, Joachim wrote:
> I'd probably look up the Cayenne docs on connection pooling,
> and just make sure that everything is configured explicitly
> that the web container would do for you implicitly.
There is actually nothing at all to do in Cayenne. If you are using tomcat, put your datasource into conf/server.xml and you are done. Cayenne will simply use the connection pool you define.
http://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html
Same goes for most other containers.
Ari
--
-------------------------->
Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A