You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cayenne.apache.org by Andrew Willerding <aw...@itsurcom.com> on 2012/04/30 19:36:48 UTC
Standalone app, Threads and Object contexts
I have built a few Web apps using Cayenne quite successfully but now I
am building my first standalone app and have a few questions.
In a Web app, I have typically just setup my Tomcat configuration and in
my code I execute something like ObjectContext context =
BaseContext.getThreadObjectContext() and let the "magic" happen.
In my standalone application I don't think it will be quite that easy.
I will have multiple short lived threads requesting database information
simultaneously so I'm not quite sure how to manage the Cayenne
components. At first I thought I could simple do a
DataContext.createDataContext() and I'd be fine but "Creating a new
DataContext for each request is not a recommended practice." So I
understand the need for a single DataContext and so I think I found the
answer in createChildContext() but,
1) Do I need to manually create a child context in each thread? But
what should I do when the thread has completed its transactions and is
ready to terminate? I don't see a corresponding "destroyChildContext"
ObjectContext oc = getDataContext().createChildContext();
2) Or is it better to create a pool of ObjectContexts that I have to
manually manage access to within the threads - or does Cayenne have
something like this already built-in?
3) Is the maximum simultaneous createChildContext calls tied to the
ConnectionPool limits in the Modeler? To me the "createChildContext"
implies creation without limit so if they are somehow linked how will I
know that the underlying ConnectionPool has run out of resources?
4) Or am I way off base and making things more complicated than they
should be? ;-)
Sorry if I've missed the answers somewhere in the documentation but I
haven't been able to Google successfully for something I've fully
understood.
Thanks,
Andrew
Re: Standalone app, Threads and Object contexts
Posted by Joe Baldwin <jf...@earthlink.net>.
Not sure if this helps, but I have used both the default (Cayenne) connection pool, and DBCP.
http://commons.apache.org/dbcp/
Cayenne's implementation seems to be adequate, but if you determine that you are going to need something a bit more sophisticated, then DBCP is very reliable IMHO. I have been using DBCP for over a year and have had no issues yet (however, I also upgraded from MySQL 5.0 because of connection bugs reported in parts of that version branch).
Joe
On May 2, 2012, at 10:18 AM, John Huss wrote:
> The number of ObjectContexts (or DataContexts) has no relation to the
> number of database connections. ObjectContexts are cheap to create and
> use, so go wild. Cayenne will do connection pooling out of the box by
> using the minimum and maximum connection limits that you define in Modeler.
>
> John
>
> On Wed, May 2, 2012 at 8:05 AM, Andrew Willerding
> <aw...@itsurcom.com>wrote:
>
>> On 05/02/2012 05:11 AM, Durchholz, Joachim wrote:
>>
>>> I'd probably look up the Cayenne docs on connection pooling, and just
>>> make sure that everything is configured explicitly that the web container
>>> would do for you implicitly. BTW I've been working on a J2SE application
>>> for the last three years, and it's probably really a good idea to design
>>> the application around "requests" or "transactions". Transactions can fail
>>> due to transient errors, so you want them restartable, which means you need
>>> to wrap them in objects, and there you have your requests. (This pattern
>>> also sucks greatly because it isn't easy to pass results back to the
>>> caller. Essentially, you're doing the exactly same web service
>>> architecture, except it's running inside a single JVM.)
>>>
>> I got the impression that there was some sort of connection pooling
>> built-in to Cayenne because in the Modeller there is a setting for minimum
>> and maximum connections. Now I'm getting confused and perhaps I'm using
>> incorrect terminology.
>>
>> In my mind there is the actual connection to the database as defined
>> during the creation of the DataContext. In the application code I am
>> creating an ObjectContext based on the DataContext with a call to
>> DataContext.**createChildContext(). It is this ObjectContext where I can
>> create/delete manipulate the contents of the database.
>>
>> But in a multi-threaded application do I need to create several
>> ObjectContexts or can one ObjectContext be shared among all the threads?
>> If the ObjectContext is sharable then do I need to synchronize access to
>> it? If it needs to by synchronized then the "200th" thread will be in a
>> long line-up which is why I would see the need to create several pooled
>> ObjectContexts. But then this takes be back to connection pooling as
>> defined in the Modeller. If everything goes back through a single
>> connection then it's pointless to create and manage ObjectContexts as the
>> bottleneck may be "deeper" in the Cayenne library.
>>
>> Help! I getting trapped in my own recursive thoughts.
>>
>> ;-)
>>
Re: Standalone app, Threads and Object contexts
Posted by Mike Kienenberger <mk...@gmail.com>.
Unless something has changed recently, DataContexts are not thread-safe.
As has been stated before, creating a DataContext is a cheap operation
(populating it from the database is expensive, copying objects from
another data context is cheap).
DataContexts are not tied to connnection pools -- a connection is
temporarily assigned whenever you commit or query the database, and
then released back to the pool.
On Wed, May 2, 2012 at 11:34 AM, Andrew Willerding
<aw...@itsurcom.com> wrote:
> On 05/02/2012 10:18 AM, John Huss wrote:
>>
>> The number of ObjectContexts (or DataContexts) has no relation to the
>> number of database connections. ObjectContexts are cheap to create and
>> use, so go wild. Cayenne will do connection pooling out of the box by
>> using the minimum and maximum connection limits that you define in
>> Modeler.
>>
>> John
>>
>>
> That was what I was hoping for. Continue to let the "magic" take care of
> the nitty-gritty details so I can focus on my app.
>
> But just to confirm my understanding. I can create one
> DataContext(ObjectContext) and perform as many "simultaneous i.e. from
> multiple threads" object creations, object updates, stored procedure
> executions without concern as long as these creations/updates/stored
> procedure executions are non-related and independent meaning one thread
> that's doing a creation/update/stored procedure execution is not dependent
> on a creation/update/execution going on in another thread. Something like
> this
>
> public class Main{
> public static synchronized DataContext getDataContext() {
> if (dc == null) {
> String fileName = "etc/cayenne.xml";
> org.apache.cayenne.conf.FileConfiguration conf = new
> org.apache.cayenne.conf.FileConfiguration(fileName);
>
> org.apache.cayenne.conf.Configuration.initializeSharedConfiguration(conf);
>
> dc = DataContext.createDataContext();
> }
> return dc;
> }
> }
>
> public class myThread(){
>
> public void run(){
> DataContext dc = Main.getDataContext();
>
> // etc on object creation/update/stored procedure executions
> dc.commitChanges();
> }
> }
>
>
>
> Andrew
Re: Standalone app, Threads and Object contexts
Posted by Michael Gentry <mg...@masslight.net>.
Hi Andrew,
I do not believe you want to share a context like that. A DataContext
is NOT thread safe between different threads. Also, even if you
synchronized access to it further (to be thread safe), a single commit
failure (such as a constraint exception) would doom all future commits
because the shared DataContext is now dirty with uncommitable changes
and nothing else can commit, either.
Use individual DataContexts for related changes. If you re-use a
DataContext, it should be applicable to a previous request (just like
a web session re-attaches your HTTP Session).
mrg
On Wed, May 2, 2012 at 11:34 AM, Andrew Willerding
<aw...@itsurcom.com> wrote:
> But just to confirm my understanding. I can create one
> DataContext(ObjectContext) and perform as many "simultaneous i.e. from
> multiple threads" object creations, object updates, stored procedure
> executions without concern as long as these creations/updates/stored
> procedure executions are non-related and independent meaning one thread
> that's doing a creation/update/stored procedure execution is not dependent
> on a creation/update/execution going on in another thread. Something like
> this
Re: Standalone app, Threads and Object contexts
Posted by Andrew Willerding <aw...@itsurcom.com>.
On 05/02/2012 10:18 AM, John Huss wrote:
> The number of ObjectContexts (or DataContexts) has no relation to the
> number of database connections. ObjectContexts are cheap to create and
> use, so go wild. Cayenne will do connection pooling out of the box by
> using the minimum and maximum connection limits that you define in Modeler.
>
> John
>
>
That was what I was hoping for. Continue to let the "magic" take care
of the nitty-gritty details so I can focus on my app.
But just to confirm my understanding. I can create one
DataContext(ObjectContext) and perform as many "simultaneous i.e. from
multiple threads" object creations, object updates, stored procedure
executions without concern as long as these creations/updates/stored
procedure executions are non-related and independent meaning one thread
that's doing a creation/update/stored procedure execution is not
dependent on a creation/update/execution going on in another thread.
Something like this
public class Main{
public static synchronized DataContext getDataContext() {
if (dc == null) {
String fileName = "etc/cayenne.xml";
org.apache.cayenne.conf.FileConfiguration conf = new
org.apache.cayenne.conf.FileConfiguration(fileName);
org.apache.cayenne.conf.Configuration.initializeSharedConfiguration(conf);
dc = DataContext.createDataContext();
}
return dc;
}
}
public class myThread(){
public void run(){
DataContext dc = Main.getDataContext();
// etc on object creation/update/stored procedure executions
dc.commitChanges();
}
}
Andrew
Re: Standalone app, Threads and Object contexts
Posted by John Huss <jo...@gmail.com>.
The number of ObjectContexts (or DataContexts) has no relation to the
number of database connections. ObjectContexts are cheap to create and
use, so go wild. Cayenne will do connection pooling out of the box by
using the minimum and maximum connection limits that you define in Modeler.
John
On Wed, May 2, 2012 at 8:05 AM, Andrew Willerding
<aw...@itsurcom.com>wrote:
> On 05/02/2012 05:11 AM, Durchholz, Joachim wrote:
>
>> I'd probably look up the Cayenne docs on connection pooling, and just
>> make sure that everything is configured explicitly that the web container
>> would do for you implicitly. BTW I've been working on a J2SE application
>> for the last three years, and it's probably really a good idea to design
>> the application around "requests" or "transactions". Transactions can fail
>> due to transient errors, so you want them restartable, which means you need
>> to wrap them in objects, and there you have your requests. (This pattern
>> also sucks greatly because it isn't easy to pass results back to the
>> caller. Essentially, you're doing the exactly same web service
>> architecture, except it's running inside a single JVM.)
>>
> I got the impression that there was some sort of connection pooling
> built-in to Cayenne because in the Modeller there is a setting for minimum
> and maximum connections. Now I'm getting confused and perhaps I'm using
> incorrect terminology.
>
> In my mind there is the actual connection to the database as defined
> during the creation of the DataContext. In the application code I am
> creating an ObjectContext based on the DataContext with a call to
> DataContext.**createChildContext(). It is this ObjectContext where I can
> create/delete manipulate the contents of the database.
>
> But in a multi-threaded application do I need to create several
> ObjectContexts or can one ObjectContext be shared among all the threads?
> If the ObjectContext is sharable then do I need to synchronize access to
> it? If it needs to by synchronized then the "200th" thread will be in a
> long line-up which is why I would see the need to create several pooled
> ObjectContexts. But then this takes be back to connection pooling as
> defined in the Modeller. If everything goes back through a single
> connection then it's pointless to create and manage ObjectContexts as the
> bottleneck may be "deeper" in the Cayenne library.
>
> Help! I getting trapped in my own recursive thoughts.
>
> ;-)
>
RE: Standalone app, Threads and Object contexts
Posted by "Durchholz, Joachim" <Jo...@hennig-fahrzeugteile.de>.
> One thing I found that I must create at least one
> ObjectContextChild before the DataContext becomes
> usable otherwise I'm not seeing the connection being
> made to the database in the log files.
Just in case it's not already clear: A DataContext does not need a connection to be usable. It will grab a connection as soon as it actually needs one, i.e. when the application starts querying away.
Re: Standalone app, Threads and Object contexts
Posted by Michael Gentry <mg...@masslight.net>.
Hi Andrew,
I've written applications that never create a child context. Sounds
like you have something else going on if you must create a child
context before anything is usable. (This would be database
independent.)
mrg
On Wed, May 2, 2012 at 11:43 AM, Andrew Willerding
<aw...@itsurcom.com> wrote:
> One thing I found that I must create at least one ObjectContextChild before
> the DataContext becomes usable otherwise I'm not seeing the connection being
> made to the database in the log files. I'm using Cayenne 3.0.2 and an
> Oracle database. I can provide a log trace if required.
Re: Standalone app, Threads and Object contexts
Posted by Andrew Willerding <aw...@itsurcom.com>.
On 05/02/2012 09:46 AM, Durchholz, Joachim wrote:
> No, a DataContext is a context within which retrieving the same rows
> via some query will always give you identical Java objects. That's a
> pretty powerful (and important) property, but not necessarily related
> to a connection. (Connections become important as soon as you need
> transaction-related guarantees.) You don't need to create an
> ObjectContext to work with a DataContext - any DataContext already is
> an ObjectContext. You don't roll back a database transaction here, you
> roll back to whatever was last loaded from the database. If you use
> child contexts, you can roll back to whatever the state was when that
> child context was started. I believe that this means that you may have
> data loaded in different transactions in your application, and that
> that is the price you pay for getting rid of
> LazyInitializationExceptions. (True Cayenne experts, please correct if
> that's wrong. I'm writing this both to help Andrew along and to get
> helped along ;-) )
One thing I found that I must create at least one ObjectContextChild
before the DataContext becomes usable otherwise I'm not seeing the
connection being made to the database in the log files. I'm using
Cayenne 3.0.2 and an Oracle database. I can provide a log trace if
required.
RE: Standalone app, Threads and Object contexts
Posted by "Durchholz, Joachim" <Jo...@hennig-fahrzeugteile.de>.
> I got the impression that there was some sort of connection
> pooling built-in to Cayenne because in the Modeller there
> is a setting for minimum and maximum connections. Now I'm
> getting confused and perhaps I'm using incorrect terminology.
As I said, I'm still stuck with Hibernate, so I'm quite fuzzy about exact details here.
However, I think it is entirely possible that Cayenne already has some built-in connection pooling. In a J2SE context, it would essentially just be a connection recreator.
> In my mind there is the actual connection to the database
> as defined during the creation of the DataContext.
No, a DataContext is a context within which retrieving the same rows via some query will always give you identical Java objects.
That's a pretty powerful (and important) property, but not necessarily related to a connection. (Connections become important as soon as you need transaction-related guarantees.)
You don't need to create an ObjectContext to work with a DataContext - any DataContext already is an ObjectContext.
You don't roll back a database transaction here, you roll back to whatever was last loaded from the database.
If you use child contexts, you can roll back to whatever the state was when that child context was started.
I believe that this means that you may have data loaded in different transactions in your application, and that that is the price you pay for getting rid of LazyInitializationExceptions. (True Cayenne experts, please correct if that's wrong. I'm writing this both to help Andrew along and to get helped along ;-) )
Re: Standalone app, Threads and Object contexts
Posted by Andrew Willerding <aw...@itsurcom.com>.
On 05/02/2012 05:11 AM, Durchholz, Joachim wrote:
> I'd probably look up the Cayenne docs on connection pooling, and just
> make sure that everything is configured explicitly that the web
> container would do for you implicitly. BTW I've been working on a J2SE
> application for the last three years, and it's probably really a good
> idea to design the application around "requests" or "transactions".
> Transactions can fail due to transient errors, so you want them
> restartable, which means you need to wrap them in objects, and there
> you have your requests. (This pattern also sucks greatly because it
> isn't easy to pass results back to the caller. Essentially, you're
> doing the exactly same web service architecture, except it's running
> inside a single JVM.)
I got the impression that there was some sort of connection pooling
built-in to Cayenne because in the Modeller there is a setting for
minimum and maximum connections. Now I'm getting confused and perhaps
I'm using incorrect terminology.
In my mind there is the actual connection to the database as defined
during the creation of the DataContext. In the application code I am
creating an ObjectContext based on the DataContext with a call to
DataContext.createChildContext(). It is this ObjectContext where I can
create/delete manipulate the contents of the database.
But in a multi-threaded application do I need to create several
ObjectContexts or can one ObjectContext be shared among all the
threads? If the ObjectContext is sharable then do I need to synchronize
access to it? If it needs to by synchronized then the "200th" thread
will be in a long line-up which is why I would see the need to create
several pooled ObjectContexts. But then this takes be back to
connection pooling as defined in the Modeller. If everything goes back
through a single connection then it's pointless to create and manage
ObjectContexts as the bottleneck may be "deeper" in the Cayenne library.
Help! I getting trapped in my own recursive thoughts.
;-)
RE: Standalone app, Threads and Object contexts
Posted by "Durchholz, Joachim" <Jo...@hennig-fahrzeugteile.de>.
> But maybe I'm misunderstanding what "standalone" means.
> Does it mean Swing? Or a daemon with no UI? Or an
> application which needs to be a double clickable
> executable but still serves web pages?
In the sense that I understand the word, the first two would
usually be designed as a standalone (J2SE) app and the latter
one as a container (J2EE) app.
> In all three cases, Tomcat/Jetty/etc could still be
> part of the solution.
Could, but often wouldn't.
Tomcat etc. are built for high concurrency. Which means ultra-short transaction and connections and many parallel connections. That's the use case for containers and connection pooling.
If high concurrency is not an issue, you could have just a handful of Swing application banging on the same database, or even just a sole daemon. In these scenarios, you don't need the container for its high-concurrency guarantees.
Of course that doesn't mean that a container would necessarily be useless in such scenarios. A container enforces a design that makes retrying transactions easier, a connection pool ensures that that broken connections will be recreated ASAP. Both services may or may not be useful to a "standalone" (i.e. non-web-based) application.
Re: Standalone app, Threads and Object contexts
Posted by Andrew Willerding <aw...@itsurcom.com>.
I'm a little confused by this email thread because it started off as
questions about derby and then morphed into connection pooling. There
are lots of connection pooling libraries out there such as c3po and
dbcp. But I'll step out of this conversation since perhaps I'm not
understanding what is being asked.
>
> Ari
>
>
My mistake. I accidentally posted a reply instead of creating a new post.
Re: Standalone app, Threads and Object contexts
Posted by Aristedes Maniatis <ar...@maniatis.org>.
On 2/05/12 8:38pm, Durchholz, Joachim wrote:
>> On 2/05/12 7:11pm, Durchholz, Joachim wrote:
>>
>>> I'd probably look up the Cayenne docs on connection pooling, and just
>>> make sure that everything is configured explicitly that the web
>>> container would do for you implicitly.
>>
>> There is actually nothing at all to do in Cayenne. If you are using
>> tomcat, put your datasource into conf/server.xml and you are done.
>
> The Subject says "standalone app".
> My understanding is that this implies not having Tomcat or any other J2EE container.
No idea why that would be implied. But maybe I'm misunderstanding what "standalone" means. Does it mean Swing? Or a daemon with no UI? Or an application which needs to be a double clickable executable but still serves web pages? In all three cases, Tomcat/Jetty/etc could still be part of the solution.
I'm a little confused by this email thread because it started off as questions about derby and then morphed into connection pooling. There are lots of connection pooling libraries out there such as c3po and dbcp. But I'll step out of this conversation since perhaps I'm not understanding what is being asked.
Ari
--
-------------------------->
Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
RE: Standalone app, Threads and Object contexts
Posted by "Durchholz, Joachim" <Jo...@hennig-fahrzeugteile.de>.
> On 2/05/12 7:11pm, Durchholz, Joachim wrote:
>
>> I'd probably look up the Cayenne docs on connection pooling, and just
>> make sure that everything is configured explicitly that the web
>> container would do for you implicitly.
>
> There is actually nothing at all to do in Cayenne. If you are using
> tomcat, put your datasource into conf/server.xml and you are done.
The Subject says "standalone app".
My understanding is that this implies not having Tomcat or any other J2EE container.
Re: Standalone app, Threads and Object contexts
Posted by Aristedes Maniatis <ar...@maniatis.org>.
On 2/05/12 7:11pm, Durchholz, Joachim wrote:
> I'd probably look up the Cayenne docs on connection pooling,
> and just make sure that everything is configured explicitly
> that the web container would do for you implicitly.
There is actually nothing at all to do in Cayenne. If you are using tomcat, put your datasource into conf/server.xml and you are done. Cayenne will simply use the connection pool you define.
http://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html
Same goes for most other containers.
Ari
--
-------------------------->
Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
RE: Standalone app, Threads and Object contexts
Posted by "Durchholz, Joachim" <Jo...@hennig-fahrzeugteile.de>.
> Unfortunately, I may have up to 200 simultaneous requests
> at the database.
Not a problem.
> I'm sure the database administrator won't like to see my
> app opening and closing connections to their database
> so I would prefer to open up a pool of connections and
> then share them among the incoming requests much like a
> web app.
Standalone apps tend to work by opening a single Connection
and using that for all requests.
The snag here is that connections will break sooner or later
(due to networking problems, server restarts, idle time, or
a multitude of other causes), so this needs a library that
recreates a broken connection on the fly.
Connection pooling libraries do that as a side effect. That
means that you use such a library, but not for the pooling
(no need to share connection across "requests", there is no
such thing in a J2SE application unless your design calls
for one due to other reasons) but for the side effect of
recreating connections as needed.
Since I'm still stuck with Hibernate, I can't advise very
well about how to best integrate a connection pooling
library and Cayenne.
I'd probably look up the Cayenne docs on connection pooling,
and just make sure that everything is configured explicitly
that the web container would do for you implicitly.
BTW I've been working on a J2SE application for the last
three years, and it's probably really a good idea to design
the application around "requests" or "transactions".
Transactions can fail due to transient errors, so you want
them restartable, which means you need to wrap them in
objects, and there you have your requests. (This pattern
also sucks greatly because it isn't easy to pass results
back to the caller. Essentially, you're doing the exactly
same web service architecture, except it's running inside a
single JVM.)
Re: Standalone app, Threads and Object contexts
Posted by Andrew Willerding <aw...@itsurcom.com>.
Hi Mike,
Unfortunately, I may have up to 200 simultaneous requests at the
database. I'm sure the database administrator won't like to see my app
opening and closing connections to their database so I would prefer to
open up a pool of connections and then share them among the incoming
requests much like a web app. I have no problem building the pool
solution but I just need to understand the best way to utilize the
Cayenne components and not duplicate or over-build what may already exist.
Or am I misunderstanding how createDataContext() works in that it is
already managing the pool of connections will only establish a new
connection when the requests exceed the number of already established
until the ConnectionPool limit is reached. If so, then what happens
when the (ConnectionPool + 1) createDataContext() request is made?
Andrew
On 04/30/2012 02:06 PM, Mike Kienenberger wrote:
>> At first I thought I could simple do a DataContext.createDataContext() and
>> I'd be fine but "Creating a new DataContext for each request is not a
>> recommended practice."
> DataContext.createDataContext() is probably fine. The "request"
> discussed above is a web server http request. It's unlikely that
> you'll have hundreds of simultaneous threads in your standalone app.
>
> Start with createDataContext() and see if it works well enough. If
> not, then you can worry about finding another way to optimize it.
>
> On Mon, Apr 30, 2012 at 1:36 PM, Andrew Willerding
> <aw...@itsurcom.com> wrote:
>> I have built a few Web apps using Cayenne quite successfully but now I am
>> building my first standalone app and have a few questions.
>>
>> In a Web app, I have typically just setup my Tomcat configuration and in my
>> code I execute something like ObjectContext context =
>> BaseContext.getThreadObjectContext() and let the "magic" happen.
>>
>> In my standalone application I don't think it will be quite that easy. I
>> will have multiple short lived threads requesting database information
>> simultaneously so I'm not quite sure how to manage the Cayenne components.
>> At first I thought I could simple do a DataContext.createDataContext() and
>> I'd be fine but "Creating a new DataContext for each request is not a
>> recommended practice." So I understand the need for a single DataContext
>> and so I think I found the answer in createChildContext() but,
>>
>> 1) Do I need to manually create a child context in each thread? But what
>> should I do when the thread has completed its transactions and is ready to
>> terminate? I don't see a corresponding "destroyChildContext"
>>
>> ObjectContext oc = getDataContext().createChildContext();
>>
>> 2) Or is it better to create a pool of ObjectContexts that I have to
>> manually manage access to within the threads - or does Cayenne have
>> something like this already built-in?
>>
>> 3) Is the maximum simultaneous createChildContext calls tied to the
>> ConnectionPool limits in the Modeler? To me the "createChildContext"
>> implies creation without limit so if they are somehow linked how will I know
>> that the underlying ConnectionPool has run out of resources?
>>
>> 4) Or am I way off base and making things more complicated than they should
>> be? ;-)
>>
>> Sorry if I've missed the answers somewhere in the documentation but I
>> haven't been able to Google successfully for something I've fully
>> understood.
>>
>> Thanks,
>>
>> Andrew
>>
>>
>>
>>
>>
Re: Standalone app, Threads and Object contexts
Posted by Mike Kienenberger <mk...@gmail.com>.
> At first I thought I could simple do a DataContext.createDataContext() and
> I'd be fine but "Creating a new DataContext for each request is not a
> recommended practice."
DataContext.createDataContext() is probably fine. The "request"
discussed above is a web server http request. It's unlikely that
you'll have hundreds of simultaneous threads in your standalone app.
Start with createDataContext() and see if it works well enough. If
not, then you can worry about finding another way to optimize it.
On Mon, Apr 30, 2012 at 1:36 PM, Andrew Willerding
<aw...@itsurcom.com> wrote:
> I have built a few Web apps using Cayenne quite successfully but now I am
> building my first standalone app and have a few questions.
>
> In a Web app, I have typically just setup my Tomcat configuration and in my
> code I execute something like ObjectContext context =
> BaseContext.getThreadObjectContext() and let the "magic" happen.
>
> In my standalone application I don't think it will be quite that easy. I
> will have multiple short lived threads requesting database information
> simultaneously so I'm not quite sure how to manage the Cayenne components.
> At first I thought I could simple do a DataContext.createDataContext() and
> I'd be fine but "Creating a new DataContext for each request is not a
> recommended practice." So I understand the need for a single DataContext
> and so I think I found the answer in createChildContext() but,
>
> 1) Do I need to manually create a child context in each thread? But what
> should I do when the thread has completed its transactions and is ready to
> terminate? I don't see a corresponding "destroyChildContext"
>
> ObjectContext oc = getDataContext().createChildContext();
>
> 2) Or is it better to create a pool of ObjectContexts that I have to
> manually manage access to within the threads - or does Cayenne have
> something like this already built-in?
>
> 3) Is the maximum simultaneous createChildContext calls tied to the
> ConnectionPool limits in the Modeler? To me the "createChildContext"
> implies creation without limit so if they are somehow linked how will I know
> that the underlying ConnectionPool has run out of resources?
>
> 4) Or am I way off base and making things more complicated than they should
> be? ;-)
>
> Sorry if I've missed the answers somewhere in the documentation but I
> haven't been able to Google successfully for something I've fully
> understood.
>
> Thanks,
>
> Andrew
>
>
>
>
>
Re: Standalone app, Threads and Object contexts
Posted by Andrew Willerding <aw...@itsurcom.com>.
On 05/02/2012 05:40 PM, Michael Gentry wrote:
> On Wed, May 2, 2012 at 4:52 PM, Andrew Willerding
> <aw...@itsurcom.com> wrote:
>> Is is simply a matter of doing a DataContext.createDataContext() inside of a
>> thread?
> Yes.
>
>> Will the new DataContexts use the SharedConfiguration of the first
>> DataContext that was created?
> And yes. :-) (Although to be more precise, you initialize the shared
> configuration and then they all use it -- has nothing to do with the
> first DC that was created.)
>
> mrg
Thank you Michael. You have made my day!
Re: Standalone app, Threads and Object contexts
Posted by Michael Gentry <mg...@masslight.net>.
On Wed, May 2, 2012 at 4:52 PM, Andrew Willerding
<aw...@itsurcom.com> wrote:
> Is is simply a matter of doing a DataContext.createDataContext() inside of a
> thread?
Yes.
> Will the new DataContexts use the SharedConfiguration of the first
> DataContext that was created?
And yes. :-) (Although to be more precise, you initialize the shared
configuration and then they all use it -- has nothing to do with the
first DC that was created.)
mrg
Re: Standalone app, Threads and Object contexts
Posted by Andrew Willerding <aw...@itsurcom.com>.
On 05/02/2012 04:26 PM, Michael Gentry wrote:
> We should change that wording in the documentation. Thanks for
> pointing it out.
I'm glad I could contribute something!
> 200 threads simultaneously using a DataContext is going to be bad. You
> want 200 DataContexts. Also, you don't need to initialize the shared
> configuration every time you create a DataContext. Just once should be
> fine. When you create a DataContext, it doesn't open/close a
> connection to the database. Doing dataContext.performQuery() or
> dataContext.commitChanges() (and similar fetch/commit calls) will get
> a database connection from the connection pool, use it, then return it.
How do I create the 200 new DataContexts? (This is where I came to the
conclusion that a createChildContext() call was needed.) Is is simply
a matter of doing a DataContext.createDataContext() inside of a
thread? Will the new DataContexts use the SharedConfiguration of the
first DataContext that was created? (This is why I thought I needed to
specify the underlying cayenne.xml file for each DataContext)
> If you have a maximum of 20 connections and you have 20 simultaneous
> fetches/commits going on, the 21st will have to wait or fail. If you
> use the DBCPDataSourceFactory method (instead of
> DriverDataSourceFactory), you get more control in configuring such
> things. See http://cayenne.apache.org/doc/dbcpdatasourcefactory.html
> and cayenne.dbcp.maxWait, for example. Keep in mind a DataContext only
> grabs a database connection when it actually needs to talk to the
> database. If your request takes 1000ms to process, but only needs
> 100ms of database connection usage time, that's 900ms of time where
> the DataContext is not using a connection and another
> thread/DataContext can use the connection. Furthermore, of that 100ms
> of database connection usage time, let's imagine that you first do a
> fetch which takes 20ms and then you do an update which takes 80ms. The
> DataContext is going to checkout (and checkin) the database connection
> from (and to) the pool twice: once for the select, once for the
> update. You may get the same connection the second time or perhaps a
> different connection. In this manner connections can be shared/re-used
> among different DataContexts.
I will look further into the DBCPDataSourceFactory method to see what I
need to learn there!
Re: Standalone app, Threads and Object contexts
Posted by Michael Gentry <mg...@masslight.net>.
Hi again Andrew!
On Wed, May 2, 2012 at 2:54 PM, Andrew Willerding
<aw...@itsurcom.com> wrote:
> On 05/02/2012 10:35 AM, Michael Gentry wrote:
>>
>> Creating a new DataContext may or may not be the right approach for your
>> application, but to say it "is not a recommended practice" would be
>> incorrect. If your requests are naturally isolated from other requests,
>> there is no harm in creating a new DataContext/request.
>
> I got that quote from the Cayenne Guide
>
> http://cayenne.apache.org/doc30/obtaining-datacontext.html
We should change that wording in the documentation. Thanks for pointing it out.
> so that was got me thinking about the whole thread topic and what might be
> the recommended practice for a multi-threaded stand-alone app.
>
>
>> DataContexts (parent/child/whatever) are not tied at all to the connection
>> pool limits. Creating a DataContext is cheap, too. Create all that you need.
>
>
> I am beginning to understand this now but in my "extreme" case of 200
> simultaneous requests to a DataContext can I simply create 200 new
> DataContexts using code like
>
>
> String fileName = "etc/cayenne.xml";
> org.apache.cayenne.conf.FileConfiguration conf = new
> org.apache.cayenne.conf.FileConfiguration(fileName);
>
> org.apache.cayenne.conf.Configuration.initializeSharedConfiguration(conf);
> DataContext dc = DataContext.createDataContext();
>
>
> without causing multiple open/close connections to the underlying database?
200 threads simultaneously using a DataContext is going to be bad.
You want 200 DataContexts. Also, you don't need to initialize the
shared configuration every time you create a DataContext. Just once
should be fine.
When you create a DataContext, it doesn't open/close a connection to
the database. Doing dataContext.performQuery() or
dataContext.commitChanges() (and similar fetch/commit calls) will get
a database connection from the connection pool, use it, then return
it.
> And where does the underlying connection pool set for lets say 20 maximum
> connections come in to play? In the code example above, will a
> createDataContext() fail at the 21st request?
If you have a maximum of 20 connections and you have 20 simultaneous
fetches/commits going on, the 21st will have to wait or fail. If you
use the DBCPDataSourceFactory method (instead of
DriverDataSourceFactory), you get more control in configuring such
things. See http://cayenne.apache.org/doc/dbcpdatasourcefactory.html
and cayenne.dbcp.maxWait, for example.
Keep in mind a DataContext only grabs a database connection when it
actually needs to talk to the database. If your request takes 1000ms
to process, but only needs 100ms of database connection usage time,
that's 900ms of time where the DataContext is not using a connection
and another thread/DataContext can use the connection.
Furthermore, of that 100ms of database connection usage time, let's
imagine that you first do a fetch which takes 20ms and then you do an
update which takes 80ms. The DataContext is going to checkout (and
checkin) the database connection from (and to) the pool twice: once
for the select, once for the update. You may get the same connection
the second time or perhaps a different connection. In this manner
connections can be shared/re-used among different DataContexts.
>>> 4) Or am I way off base and making things more complicated than they
>>> should
>>> be? ;-)
>>
>>
>> Probably. :-)
>
> My thanks to the forum monitors for your patience and explanations!!
>
>>
>> PS. You keep mentioning child contexts. Do you *need* child contexts?
>
>
Re: Standalone app, Threads and Object contexts
Posted by Andrew Willerding <aw...@itsurcom.com>.
On 05/02/2012 10:35 AM, Michael Gentry wrote:
> Creating a new DataContext may or may not be the right approach for
> your application, but to say it "is not a recommended practice" would
> be incorrect. If your requests are naturally isolated from other
> requests, there is no harm in creating a new DataContext/request.
I got that quote from the Cayenne Guide
http://cayenne.apache.org/doc30/obtaining-datacontext.html
so that was got me thinking about the whole thread topic and what might
be the recommended practice for a multi-threaded stand-alone app.
> DataContexts (parent/child/whatever) are not tied at all to the
> connection pool limits. Creating a DataContext is cheap, too. Create
> all that you need.
I am beginning to understand this now but in my "extreme" case of 200
simultaneous requests to a DataContext can I simply create 200 new
DataContexts using code like
String fileName = "etc/cayenne.xml";
org.apache.cayenne.conf.FileConfiguration conf = new org.apache.cayenne.conf.FileConfiguration(fileName);
org.apache.cayenne.conf.Configuration.initializeSharedConfiguration(conf);
DataContext dc = DataContext.createDataContext();
without causing multiple open/close connections to the underlying database?
And where does the underlying connection pool set for lets say 20
maximum connections come in to play? In the code example above, will a
createDataContext() fail at the 21st request?
>> 4) Or am I way off base and making things more complicated than they should
>> be? ;-)
>
> Probably. :-)
My thanks to the forum monitors for your patience and explanations!!
>
> PS. You keep mentioning child contexts. Do you *need* child contexts?
Re: Standalone app, Threads and Object contexts
Posted by Michael Gentry <mg...@masslight.net>.
Hi Andrew, I'm coming to the party late, but a few quick comments inline.
mrg
On Mon, Apr 30, 2012 at 1:36 PM, Andrew Willerding
<aw...@itsurcom.com> wrote:
> I have built a few Web apps using Cayenne quite successfully but now I am
> building my first standalone app and have a few questions.
>
> In a Web app, I have typically just setup my Tomcat configuration and in my
> code I execute something like ObjectContext context =
> BaseContext.getThreadObjectContext() and let the "magic" happen.
>
> In my standalone application I don't think it will be quite that easy. I
> will have multiple short lived threads requesting database information
> simultaneously so I'm not quite sure how to manage the Cayenne components.
> At first I thought I could simple do a DataContext.createDataContext() and
> I'd be fine but "Creating a new DataContext for each request is not a
> recommended practice." So I understand the need for a single DataContext
> and so I think I found the answer in createChildContext() but,
Creating a new DataContext may or may not be the right approach for
your application, but to say it "is not a recommended practice" would
be incorrect. If your requests are naturally isolated from other
requests, there is no harm in creating a new DataContext/request.
> 1) Do I need to manually create a child context in each thread? But what
> should I do when the thread has completed its transactions and is ready to
> terminate? I don't see a corresponding "destroyChildContext"
>
> ObjectContext oc = getDataContext().createChildContext();
This is more of a design decision (sorry to be fuzzy). If it makes
sense to you to create a child context for your requirements, do it.
As for destroying a child context, all DataContexts are free to be
garbage collected if you don't hold onto them.
> 2) Or is it better to create a pool of ObjectContexts that I have to
> manually manage access to within the threads - or does Cayenne have
> something like this already built-in?
You typically don't care to create a pool of re-usable contexts.
There are some exceptions to this for read-mostly data, though.
> 3) Is the maximum simultaneous createChildContext calls tied to the
> ConnectionPool limits in the Modeler? To me the "createChildContext"
> implies creation without limit so if they are somehow linked how will I know
> that the underlying ConnectionPool has run out of resources?
DataContexts (parent/child/whatever) are not tied at all to the
connection pool limits. Creating a DataContext is cheap, too. Create
all that you need.
> 4) Or am I way off base and making things more complicated than they should
> be? ;-)
Probably. :-)
PS. You keep mentioning child contexts. Do you *need* child contexts?