You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cayenne.apache.org by jonathan_ms2006 <jo...@yahoo.com> on 2015/01/02 16:43:12 UTC
Closing Cayennes' DataContext
Hi Andrus,
Good day! Is there a way to close cayennes' DataContext.? In my web
application I'm using this code "DataContext context =
ServletUtil.getSessionContext(session);" as suggested on cayenne website.
However the problem is, it is creating new connection every time a page
loads which as a result, returns an error when there are multiple users
accessing the web application:
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Data
source rejected establishment of connection, message from server: "Too many
connections".
Any ideas how to overcome this error in cayenne.? Thank you in advance.
Best Regards,
Jonathan
--
View this message in context: http://cayenne.195.n3.nabble.com/Closing-Cayennes-DataContext-tp4026629.html
Sent from the Cayenne - User mailing list archive at Nabble.com.
Re: Closing Cayennes' DataContext
Posted by jonathan_ms2006 <jo...@yahoo.com>.
Hi Michael,
Good day! Here are the answers to your inquiries:
Q: Have you switched to using JNDI lookups yet?
A: Not yet, Currently I barely rely on cayennes' DataContext for
connection pooling/reference, etc.
Q: what kind of load do you have on your system?
A: Upon deployment, I'm expecting to have at least 500 to 1000 users per
day. That's just a conservative estimate. The reason I configured 10,000
connection pool in fear of multiple requests that will surely exceed the
connection pool limit which will prevent users from accessing the system.
I'm not comfortable in this setting too as this will eat up much resources.
Also regarding on your statement: "Whenever you perform an operation it
checks out a connection, uses it, then returns it to the pool." I'm
expecting the same thing in cayennes' DataContext, but during testing phase
this error occurred. Below is a test scenario on how I ended up with the
error mentioned on this thread. This might also give you an idea if there's
something wrong with the way I do things. Hope this helps you.
Test Scenario (pre-deployment):
* I executed the application in eclipse via run at server (tomcat), and
when the page is loaded, I tried
switching from one JSP page to another several times (65+ switches)
then the error is triggered.
Both JSP page has an include JSP file which has the DataContext
intitialization something like :
DataContext context = ServletUtil.getSessionContext(session);
Note:
I haven't deployed yet the system in tomcat and thus haven't performed
regression testing since there are still updates to be done.
Additional Web Application Info:
I'm using traditional/legacy JSP response and request page using JSP
useBean as my form handler and there are still some scriptlets on the page.
Sooner or later I will move to JSF.
Best Regards,
Jonathan
--
View this message in context: http://cayenne.195.n3.nabble.com/Closing-Cayennes-DataContext-tp4026629p4026638.html
Sent from the Cayenne - User mailing list archive at Nabble.com.
Re: Closing Cayennes' DataContext
Posted by jonathan_ms2006 <jo...@yahoo.com>.
Hi Ari,
You're absolutely right! I also read the link you shared and it's a good
one except that the MySQL Thread Pool Plugin is for commercial MySQL
databases only. haha.. :))
Best Regards,
Jonathan
--
View this message in context: http://cayenne.195.n3.nabble.com/Closing-Cayennes-DataContext-tp4026629p4026639.html
Sent from the Cayenne - User mailing list archive at Nabble.com.
Re: Closing Cayennes' DataContext
Posted by Aristedes Maniatis <ar...@maniatis.org>.
On 9/01/2015 4:03am, Michael Gentry wrote:
> Also, what kind of load do you have on your system? Our connection pools
> are rarely larger than 25-50 and work just fine. 10,000 connections smells
> to me of a larger problem.
I agree. I read an interesting article about the tradeoff between queuing and threading. That is, if you get lots of requests are you better creating lots and lots of threads to handle them all concurrently, or queuing the requests (eg. by having a connection pool of only about 30 connections) and handling them 30 at a time.
The author of the article I can't right now find was of the opinion that most people have way too many connections in their pool. The number of connections should be closer (within an order of magnitude) to the number of cores in the CPU. For example, mysql creates a separate unix thread for every connection and that scales poorly over about 60. I think this is improved in mysql 5.7, but I've not tried it. [1]
Ari
[1] http://dev.mysql.com/doc/refman/5.6/en/thread-pool-plugin.html
--
-------------------------->
Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
Re: Closing Cayennes' DataContext
Posted by Michael Gentry <mg...@masslight.net>.
Hi Jonathan,
Sorry for the delays.
I have a few questions for you...
Have you switched to using JNDI lookups yet? (Since you are clearly using
a web-based container.) I'd say JNDI is by far the preferred way to go for
almost all web applications.
Also, what kind of load do you have on your system? Our connection pools
are rarely larger than 25-50 and work just fine. 10,000 connections smells
to me of a larger problem.
Cayenne works great with database connection pools. Whenever you perform
an operation it checks out a connection, uses it, then returns it to the
pool.
Thanks,
mrg
On Sun, Jan 4, 2015 at 10:10 PM, jonathan_ms2006 <jo...@yahoo.com>
wrote:
> Hi Ari,
>
> I think you have a point there. Also I read some article about
> connection pooling using web application container (tomcat), assuming we
> have properly installed and configured Database Connection Pool (DBCP for
> example), shall not this interfere or be in conflict with the connection
> pool configured in cayennes' dataNodeDriver.xml.? will it be safe to remove
> cayennes' dataNodeDriver connection pool.? In your case, have you removed
> it.? Any ideas.? Thank you in advance... :))
>
> Best Regards,
>
> Jonathan
>
>
>
> --
> View this message in context:
> http://cayenne.195.n3.nabble.com/Closing-Cayennes-DataContext-tp4026629p4026635.html
> Sent from the Cayenne - User mailing list archive at Nabble.com.
>
Re: Closing Cayennes' DataContext
Posted by jonathan_ms2006 <jo...@yahoo.com>.
Hi Ari,
I think you have a point there. Also I read some article about
connection pooling using web application container (tomcat), assuming we
have properly installed and configured Database Connection Pool (DBCP for
example), shall not this interfere or be in conflict with the connection
pool configured in cayennes' dataNodeDriver.xml.? will it be safe to remove
cayennes' dataNodeDriver connection pool.? In your case, have you removed
it.? Any ideas.? Thank you in advance... :))
Best Regards,
Jonathan
--
View this message in context: http://cayenne.195.n3.nabble.com/Closing-Cayennes-DataContext-tp4026629p4026635.html
Sent from the Cayenne - User mailing list archive at Nabble.com.
Re: Closing Cayennes' DataContext
Posted by Aristedes Maniatis <ar...@maniatis.org>.
I use a connection pool from the application container (tomcat in my case). That might give you more choice and flexibility. I presume if you are using JSP, then you are using tomcat, jetty or one of their friends.
Ari
On 4/01/2015 3:14pm, jonathan_ms2006 wrote:
> Hi Aristedes,
>
> That's a good tip, from now on I will set MySQL max_connection
> attributes greater than cayennes' connection pool (dataNodeDriver.xml). But
> still I think this will not resolve the issue, there must be a way wherein
> cayennes' DataContext class should strategically close unused database
> connections.
>
> Lately I was thinking if we can replicate using cayennes' DataContext
> the traditional way of connecting Java to MySQL wherein programmatically
> we're setting the JDBC driver, MySQL login credentials, MySQL port number,
> etc. in a java class and then strategically we close unused or idle
> connection/s to avoid such error as mentioned on this thread.
>
> Also if there's a way where cayennes' DataContext (open connection) can
> be re-used instead opening a new one every time a JSP page loads.? Any ideas
> on this will be much appreciated. :))
>
> Best Regards,
>
> Jonathan
>
>
>
>
>
>
> --
> View this message in context: http://cayenne.195.n3.nabble.com/Closing-Cayennes-DataContext-tp4026629p4026633.html
> Sent from the Cayenne - User mailing list archive at Nabble.com.
>
--
-------------------------->
Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
Re: Closing Cayennes' DataContext
Posted by jonathan_ms2006 <jo...@yahoo.com>.
Hi Aristedes,
That's a good tip, from now on I will set MySQL max_connection
attributes greater than cayennes' connection pool (dataNodeDriver.xml). But
still I think this will not resolve the issue, there must be a way wherein
cayennes' DataContext class should strategically close unused database
connections.
Lately I was thinking if we can replicate using cayennes' DataContext
the traditional way of connecting Java to MySQL wherein programmatically
we're setting the JDBC driver, MySQL login credentials, MySQL port number,
etc. in a java class and then strategically we close unused or idle
connection/s to avoid such error as mentioned on this thread.
Also if there's a way where cayennes' DataContext (open connection) can
be re-used instead opening a new one every time a JSP page loads.? Any ideas
on this will be much appreciated. :))
Best Regards,
Jonathan
--
View this message in context: http://cayenne.195.n3.nabble.com/Closing-Cayennes-DataContext-tp4026629p4026633.html
Sent from the Cayenne - User mailing list archive at Nabble.com.
Re: Closing Cayennes' DataContext
Posted by Aristedes Maniatis <ar...@maniatis.org>.
10,000 is a pretty high number. The default is about 150. It may be that mysql isn't able to cope with that many settings.
Also, you should always have more connections available in your database than you configure in your connection pool, otherwise you are going to run into problems when you connect to your database for admin purposes, backups, or other things.
Ari
On 4/01/2015 1:37am, jonathan_ms2006 wrote:
> Hi Michael,
>
> In my cayenne NodeDriver.xml file, I have configured a connection pool
> that goes like this:
> <connectionPool min="1" max="10000"/>. Obviously I allowed 10000 maximum
> number or simultaneous connections.
>
> On MySQL server installation directory (my.ini), I have configured
> max_connections attribute to 10000 same with my connection pool defined in
> cayennes' dataNodeDriver.xml while the rest of my.ini file attributes are
> not modified upon installation.
>
> Are there other things I missed.? Many thanks.. :))
>
> Best Regards,
>
> Jonathan
>
>
>
> --
> View this message in context: http://cayenne.195.n3.nabble.com/Closing-Cayennes-DataContext-tp4026629p4026631.html
> Sent from the Cayenne - User mailing list archive at Nabble.com.
>
--
-------------------------->
Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
Re: Closing Cayennes' DataContext
Posted by jonathan_ms2006 <jo...@yahoo.com>.
Hi Michael,
In my cayenne NodeDriver.xml file, I have configured a connection pool
that goes like this:
<connectionPool min="1" max="10000"/>. Obviously I allowed 10000 maximum
number or simultaneous connections.
On MySQL server installation directory (my.ini), I have configured
max_connections attribute to 10000 same with my connection pool defined in
cayennes' dataNodeDriver.xml while the rest of my.ini file attributes are
not modified upon installation.
Are there other things I missed.? Many thanks.. :))
Best Regards,
Jonathan
--
View this message in context: http://cayenne.195.n3.nabble.com/Closing-Cayennes-DataContext-tp4026629p4026631.html
Sent from the Cayenne - User mailing list archive at Nabble.com.
Re: Closing Cayennes' DataContext
Posted by Michael Gentry <mg...@masslight.net>.
Hi Jonathan,
How many connections are you allocating and how many is your MySQL server
set up to allow?
mrg
On Fri, Jan 2, 2015 at 10:43 AM, jonathan_ms2006 <jo...@yahoo.com>
wrote:
> Hi Andrus,
>
> Good day! Is there a way to close cayennes' DataContext.? In my web
> application I'm using this code "DataContext context =
> ServletUtil.getSessionContext(session);" as suggested on cayenne website.
> However the problem is, it is creating new connection every time a page
> loads which as a result, returns an error when there are multiple users
> accessing the web application:
> com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Data
> source rejected establishment of connection, message from server: "Too
> many
> connections".
>
> Any ideas how to overcome this error in cayenne.? Thank you in advance.
>
> Best Regards,
>
> Jonathan
>
>
>
> --
> View this message in context:
> http://cayenne.195.n3.nabble.com/Closing-Cayennes-DataContext-tp4026629.html
> Sent from the Cayenne - User mailing list archive at Nabble.com.
>
Re: Closing Cayennes' DataContext
Posted by Andrus Adamchik <an...@objectstyle.org>.
> Is there a way to close cayennes' DataContext.?
While the rest of the thread correctly focused on the size of the connection pool, I wanted to clarify this point. There's no need to "close" a DataContext in Cayenne. The DataContext by itself does not hold any open connections.
Andrus
> On Jan 2, 2015, at 6:43 PM, jonathan_ms2006 <jo...@yahoo.com> wrote:
>
> Hi Andrus,
>
> Good day! Is there a way to close cayennes' DataContext.? In my web
> application I'm using this code "DataContext context =
> ServletUtil.getSessionContext(session);" as suggested on cayenne website.
> However the problem is, it is creating new connection every time a page
> loads which as a result, returns an error when there are multiple users
> accessing the web application:
> com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Data
> source rejected establishment of connection, message from server: "Too many
> connections".
>
> Any ideas how to overcome this error in cayenne.? Thank you in advance.
>
> Best Regards,
>
> Jonathan