You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by tiffany <ti...@future.co.jp> on 2011/05/31 11:16:27 UTC
DIH: Exception with "Too many connections"
Hi all,
I'm using DIH and getting the following error.
My Solr version is Solr3.1.
=====
...
Caused by:
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could
not create connection to database server. Attempted reconnect 3 times.
Giving up.
at sun.reflect.GeneratedConstructorAccessor98.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
at com.mysql.jdbc.Util.getInstance(Util.java:381)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:985)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:926)
at
com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2364)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:781)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46)
at sun.reflect.GeneratedConstructorAccessor94.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
at
com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:352)
at
com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:284)
at
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:161)
at
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:128)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcDataSource.java:363)
at
org.apache.solr.handler.dataimport.JdbcDataSource.access$200(JdbcDataSource.java:39)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:240)
... 11 more
Caused by:
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Data
source rejected establishment of connection, message from server: "Too many
connections"
at sun.reflect.GeneratedConstructorAccessor98.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
at com.mysql.jdbc.Util.getInstance(Util.java:381)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:985)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1104)
at
com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2292)
... 24 more
=====
My dataSource setting is something like this:
<dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver"
url="jdbc:mysql://database01/test?autoReconnect=true" user="xxx"
password="xxx" batchSize="-1" />
Any idea to solve this problem?
Thank you!
--
View this message in context: http://lucene.472066.n3.nabble.com/DIH-Exception-with-Too-many-connections-tp3005213p3005213.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: DIH: Exception with "Too many connections"
Posted by tiffany <ti...@future.co.jp>.
Stephan,
Your advice (check the process list) gave me an important clue for my
solution.
I changed my database connection to the slave instead of master, so that I
can use more threads.
Thank you very much!
***
François,
My setting is the default value:
max_connections = 151
max_user_connections = 0
I will think of changing the max_connections when I increase the number of
cores.
Thanks!
***
Fuad,
So far, I can handle DIH with my setting.
Thanks for letting me know!
Regards,
Tiffany
--
View this message in context: http://lucene.472066.n3.nabble.com/DIH-Exception-with-Too-many-connections-tp3005213p3009206.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: DIH: Exception with "Too many connections"
Posted by Fuad Efendi <fu...@efendi.ca>.
Hi,
There is existing bug in DataImportHandler described (and patched) at
https://issues.apache.org/jira/browse/SOLR-2233
It is not used in a thread safe manner, and it is not appropriately closed &
reopened (why?); and new connection is opened unpredictably. It may cause
"Too many connections" even for huge SQL-side max_connections.
If you are interested, I can continue work on SOLR-2233. CC: dev@lucene (is
anyone working on DIH improvements?)
Thanks,
Fuad Efendi
http://www.tokenizer.ca/
-----Original Message-----
From: François Schiettecatte [mailto:fschiettecatte@gmail.com]
Sent: May-31-11 7:44 AM
To: solr-user@lucene.apache.org
Subject: Re: DIH: Exception with "Too many connections"
Hi
You might also check the 'max_user_connections' settings too if you have
that set:
# Maximum number of connections, and per user
max_connections = 2048
max_user_connections = 2048
http://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html
Cheers
François
>> So, if the number of threads in the process list is larger than
>> max_connections, I would get the "too many connections" error. Am I
>> thinking the right way?
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
RE: DIH: Exception with "Too many connections"
Posted by Fuad Efendi <fu...@efendi.ca>.
Hi,
There is existing bug in DataImportHandler described (and patched) at
https://issues.apache.org/jira/browse/SOLR-2233
It is not used in a thread safe manner, and it is not appropriately closed &
reopened (why?); and new connection is opened unpredictably. It may cause
"Too many connections" even for huge SQL-side max_connections.
If you are interested, I can continue work on SOLR-2233. CC: dev@lucene (is
anyone working on DIH improvements?)
Thanks,
Fuad Efendi
http://www.tokenizer.ca/
-----Original Message-----
From: François Schiettecatte [mailto:fschiettecatte@gmail.com]
Sent: May-31-11 7:44 AM
To: solr-user@lucene.apache.org
Subject: Re: DIH: Exception with "Too many connections"
Hi
You might also check the 'max_user_connections' settings too if you have
that set:
# Maximum number of connections, and per user
max_connections = 2048
max_user_connections = 2048
http://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html
Cheers
François
>> So, if the number of threads in the process list is larger than
>> max_connections, I would get the "too many connections" error. Am I
>> thinking the right way?
>
Re: DIH: Exception with "Too many connections"
Posted by François Schiettecatte <fs...@gmail.com>.
Hi
You might also check the 'max_user_connections' settings too if you have that set:
# Maximum number of connections, and per user
max_connections = 2048
max_user_connections = 2048
http://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html
Cheers
François
On May 31, 2011, at 7:39 AM, Stefan Matheis wrote:
> Tiffany,
>
> On Tue, May 31, 2011 at 12:45 PM, tiffany <ti...@future.co.jp> wrote:
>> I executed the SHOW PROCESSLIST; command. (Is it what you mean? I've never
>> tried it before...)
>
> Exactly this, yes :)
>
> On Tue, May 31, 2011 at 12:45 PM, tiffany <ti...@future.co.jp> wrote:
>> So, if the number of threads in the process list is larger than
>> max_connections, I would get the "too many connections" error. Am I
>> thinking the right way?
>
> Yepp, right
>
> On Tue, May 31, 2011 at 12:45 PM, tiffany <ti...@future.co.jp> wrote:
>> If it is right, maybe I should think of the commit timing, changing the
>> number of max_connections, and/or some other ways...
>
> You may lift the allowed Number of Connections for the MySQL-Server?
> Or, of course - if possible - tweak your SOLR-Settings, correct
>
> Regards
> Stefan
Re: DIH: Exception with "Too many connections"
Posted by Stefan Matheis <ma...@googlemail.com>.
Tiffany,
On Tue, May 31, 2011 at 12:45 PM, tiffany <ti...@future.co.jp> wrote:
> I executed the SHOW PROCESSLIST; command. (Is it what you mean? I've never
> tried it before...)
Exactly this, yes :)
On Tue, May 31, 2011 at 12:45 PM, tiffany <ti...@future.co.jp> wrote:
> So, if the number of threads in the process list is larger than
> max_connections, I would get the "too many connections" error. Am I
> thinking the right way?
Yepp, right
On Tue, May 31, 2011 at 12:45 PM, tiffany <ti...@future.co.jp> wrote:
> If it is right, maybe I should think of the commit timing, changing the
> number of max_connections, and/or some other ways...
You may lift the allowed Number of Connections for the MySQL-Server?
Or, of course - if possible - tweak your SOLR-Settings, correct
Regards
Stefan
Re: DIH: Exception with "Too many connections"
Posted by tiffany <ti...@future.co.jp>.
Thanks Stefan!
I executed the SHOW PROCESSLIST; command. (Is it what you mean? I've never
tried it before...)
It seems that when I executed one delta-import command, several threads were
inserted into the table and removed after commit. Also it looks like the
number of threads are pretty much equal to the number of entity in my
db-data-config.xml.
So, if the number of threads in the process list is larger than
max_connections, I would get the "too many connections" error. Am I
thinking the right way?
If it is right, maybe I should think of the commit timing, changing the
number of max_connections, and/or some other ways...
If there are any other idea, please let me know =)
Thanks a lot!
--
View this message in context: http://lucene.472066.n3.nabble.com/DIH-Exception-with-Too-many-connections-tp3005213p3005401.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: DIH: Exception with "Too many connections"
Posted by Stefan Matheis <ma...@googlemail.com>.
Tiffany,
On Tue, May 31, 2011 at 11:16 AM, tiffany <ti...@future.co.jp> wrote:
> Any idea to solve this problem?
in Addition to Chandan: Check your mysql process list and have a look
what is displayed there
Regards
Stefan
Re: DIH: Exception with "Too many connections"
Posted by tiffany <ti...@future.co.jp>.
Thanks for your reply, Chandan.
Here is the additional information.
I'm also using the multi-core function, and I run the delta-import command
in parallel due to saving the running time. If I don't run in parallel, it
works fine. Each core accesses to the same database server but different
schema.
So, I don't know if I should change something in my database server side or
I can adjust something at the Solr side by adding some kind of property.
Tiffany
--
View this message in context: http://lucene.472066.n3.nabble.com/DIH-Exception-with-Too-many-connections-tp3005213p3005313.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: DIH: Exception with "Too many connections"
Posted by Chandan Tamrakar <ch...@nepasoft.com>.
looks like you are not being able to connect to database , pls see if you
get similar exception when you try to connect from other clients
On Tue, May 31, 2011 at 3:01 PM, tiffany <ti...@future.co.jp> wrote:
> Hi all,
>
> I'm using DIH and getting the following error.
> My Solr version is Solr3.1.
>
> =====
> ...
> Caused by:
> com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could
> not create connection to database server. Attempted reconnect 3 times.
> Giving up.
> at sun.reflect.GeneratedConstructorAccessor98.newInstance(Unknown
> Source)
> at
>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
> at com.mysql.jdbc.Util.getInstance(Util.java:381)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:985)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:926)
> at
> com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2364)
> at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:781)
> at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46)
> at sun.reflect.GeneratedConstructorAccessor94.newInstance(Unknown
> Source)
> at
>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
> at
> com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:352)
> at
> com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:284)
> at
>
> org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:161)
> at
>
> org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:128)
> at
>
> org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcDataSource.java:363)
> at
>
> org.apache.solr.handler.dataimport.JdbcDataSource.access$200(JdbcDataSource.java:39)
> at
>
> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:240)
> ... 11 more
> Caused by:
> com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Data
> source rejected establishment of connection, message from server: "Too
> many
> connections"
> at sun.reflect.GeneratedConstructorAccessor98.newInstance(Unknown
> Source)
> at
>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
> at com.mysql.jdbc.Util.getInstance(Util.java:381)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:985)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
> at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1104)
> at
> com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2292)
> ... 24 more
>
> =====
>
> My dataSource setting is something like this:
>
> <dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver"
> url="jdbc:mysql://database01/test?autoReconnect=true" user="xxx"
> password="xxx" batchSize="-1" />
>
> Any idea to solve this problem?
>
> Thank you!
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/DIH-Exception-with-Too-many-connections-tp3005213p3005213.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Chandan Tamrakar
*
*