You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Partha (JIRA)" <ji...@apache.org> on 2009/11/16 23:15:52 UTC

[jira] Created: (AMQ-2498) Broker goes down when database is restarted or goes down

Broker goes down when database is restarted or goes down
--------------------------------------------------------

                 Key: AMQ-2498
                 URL: https://issues.apache.org/activemq/browse/AMQ-2498
             Project: ActiveMQ
          Issue Type: Bug
          Components: Broker
    Affects Versions: 5.3.0
            Reporter: Partha
            Priority: Critical


When I database is idle for too long or I manually restart database, broker is shutting down. Does it not automatically try to reconnect? Am I missing anything?

I am setting validationQuery and testOnBorrow to true in broker-config.xml datasource bean.

Do I need to do anything else to tell broker for keep trying for database connection?

error
=========================

Failed to checkpoint a message store: java.util.concurrent.ExecutionException: java.io.IOException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
java.util.concurrent.ExecutionException: java.io.IOException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
 at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
 at java.util.concurrent.FutureTask.get(FutureTask.java:83)
 at org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:421)
 at org.apache.activemq.store.journal.JournalPersistenceAdapter$1.iterate(JournalPersistenceAdapter.java:124)
 at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:122)
 at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:43)
 at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
 at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:45)
 at org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:61)
 at org.apache.activemq.store.jdbc.TransactionContext.begin(TransactionContext.java:151)
 at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.beginTransaction(JDBCPersistenceAdapter.java:406)
 at org.apache.activemq.store.journal.JournalPersistenceAdapter.beginTransaction(JournalPersistenceAdapter.java:216)
 at org.apache.activemq.util.TransactionTemplate.run(TransactionTemplate.java:41)
 at org.apache.activemq.store.journal.JournalMessageStore.checkpoint(JournalMessageStore.java:258)
 at org.apache.activemq.store.journal.JournalTopicMessageStore.checkpoint(JournalTopicMessageStore.java:171)
 at org.apache.activemq.store.journal.JournalPersistenceAdapter$5.call(JournalPersistenceAdapter.java:408)
 at org.apache.activemq.store.journal.JournalPersistenceAdapter$5.call(JournalPersistenceAdapter.java:407)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 ... 3 more
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
 at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(Unknown Source)
 at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(Unknown Source)
 at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(Unknown Source)
 at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
 at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:294)
 at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:974)
 at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:96)
 at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:880)
 at org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:54)
 ... 13 more
2009-11-12 14:30:12,705 INFO  [STDOUT] 2009-11-12 14:30:12,705 [Q Cleanup Timer] INFO  BrokerService                  - ActiveMQ JMS Message Broker (ess.broker1, ID:VR1Server-8183-1258064801205-0:0) stopped

============

Thanks,
PG

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (AMQ-2498) Broker goes down when database is restarted or goes down

Posted by "Mark Gellings (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/activemq/browse/AMQ-2498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=55549#action_55549 ] 

Mark Gellings commented on AMQ-2498:
------------------------------------

I have handled this problem using a connection pooler.  Master goes down, slave attempts to connect to database and retries however many times configured...  New slave then needs to be started up manually (or automatically through some automation).


  <bean id="mssql-ds" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
    <property name="driverClass" value="com.microsoft.sqlserver.jdbc.SQLServerDriver"/>
    <property name="jdbcUrl" value="jdbc:sqlserver://CAGR.qg.com:1433;databaseName=AtvMMS"/>
    <property name="user" value="user"/>
    <property name="password" value="password"/>
    <property name="acquireRetryAttempts" value="100"/>
    <property name="testConnectionOnCheckout" value="true"/>
    <property name="testConnectionOnCheckin" value="true"/>
    <property name="automaticTestTable" value="ACTIVEMQ_CONN_TEST"/>
    <property name="acquireRetryDelay" value="5000"/>
    <property name="preferredTestQuery" value="SELECT 1"/>
  </bean>

> Broker goes down when database is restarted or goes down
> --------------------------------------------------------
>
>                 Key: AMQ-2498
>                 URL: https://issues.apache.org/activemq/browse/AMQ-2498
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.3.0
>            Reporter: Partha
>            Priority: Critical
>         Attachments: server.log
>
>
> When I database is idle for too long or I manually restart database, broker is shutting down. Does it not automatically try to reconnect? Am I missing anything?
> I am setting validationQuery and testOnBorrow to true in broker-config.xml datasource bean.
> Do I need to do anything else to tell broker for keep trying for database connection?
> error
> =========================
> Failed to checkpoint a message store: java.util.concurrent.ExecutionException: java.io.IOException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
> java.util.concurrent.ExecutionException: java.io.IOException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
>  at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>  at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:421)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter$1.iterate(JournalPersistenceAdapter.java:124)
>  at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:122)
>  at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:43)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>  at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
>  at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:45)
>  at org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:61)
>  at org.apache.activemq.store.jdbc.TransactionContext.begin(TransactionContext.java:151)
>  at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.beginTransaction(JDBCPersistenceAdapter.java:406)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter.beginTransaction(JournalPersistenceAdapter.java:216)
>  at org.apache.activemq.util.TransactionTemplate.run(TransactionTemplate.java:41)
>  at org.apache.activemq.store.journal.JournalMessageStore.checkpoint(JournalMessageStore.java:258)
>  at org.apache.activemq.store.journal.JournalTopicMessageStore.checkpoint(JournalTopicMessageStore.java:171)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter$5.call(JournalPersistenceAdapter.java:408)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter$5.call(JournalPersistenceAdapter.java:407)
>  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>  at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>  ... 3 more
> Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
>  at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(Unknown Source)
>  at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(Unknown Source)
>  at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(Unknown Source)
>  at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
>  at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:294)
>  at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:974)
>  at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:96)
>  at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:880)
>  at org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:54)
>  ... 13 more
> 2009-11-12 14:30:12,705 INFO  [STDOUT] 2009-11-12 14:30:12,705 [Q Cleanup Timer] INFO  BrokerService                  - ActiveMQ JMS Message Broker (ess.broker1, ID:VR1Server-8183-1258064801205-0:0) stopped
> ============
> Thanks,
> PG

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (AMQ-2498) Broker goes down when database is restarted or goes down

Posted by "Bruce Snyder (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/activemq/browse/AMQ-2498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bruce Snyder updated AMQ-2498:
------------------------------

    Fix Version/s: 5.5.0
                       (was: 5.4.1)

> Broker goes down when database is restarted or goes down
> --------------------------------------------------------
>
>                 Key: AMQ-2498
>                 URL: https://issues.apache.org/activemq/browse/AMQ-2498
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.3.0
>            Reporter: Partha
>            Priority: Critical
>             Fix For: 5.5.0
>
>         Attachments: server.log
>
>
> When I database is idle for too long or I manually restart database, broker is shutting down. Does it not automatically try to reconnect? Am I missing anything?
> I am setting validationQuery and testOnBorrow to true in broker-config.xml datasource bean.
> Do I need to do anything else to tell broker for keep trying for database connection?
> error
> =========================
> Failed to checkpoint a message store: java.util.concurrent.ExecutionException: java.io.IOException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
> java.util.concurrent.ExecutionException: java.io.IOException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
>  at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>  at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:421)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter$1.iterate(JournalPersistenceAdapter.java:124)
>  at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:122)
>  at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:43)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>  at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
>  at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:45)
>  at org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:61)
>  at org.apache.activemq.store.jdbc.TransactionContext.begin(TransactionContext.java:151)
>  at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.beginTransaction(JDBCPersistenceAdapter.java:406)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter.beginTransaction(JournalPersistenceAdapter.java:216)
>  at org.apache.activemq.util.TransactionTemplate.run(TransactionTemplate.java:41)
>  at org.apache.activemq.store.journal.JournalMessageStore.checkpoint(JournalMessageStore.java:258)
>  at org.apache.activemq.store.journal.JournalTopicMessageStore.checkpoint(JournalTopicMessageStore.java:171)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter$5.call(JournalPersistenceAdapter.java:408)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter$5.call(JournalPersistenceAdapter.java:407)
>  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>  at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>  ... 3 more
> Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
>  at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(Unknown Source)
>  at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(Unknown Source)
>  at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(Unknown Source)
>  at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
>  at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:294)
>  at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:974)
>  at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:96)
>  at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:880)
>  at org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:54)
>  ... 13 more
> 2009-11-12 14:30:12,705 INFO  [STDOUT] 2009-11-12 14:30:12,705 [Q Cleanup Timer] INFO  BrokerService                  - ActiveMQ JMS Message Broker (ess.broker1, ID:VR1Server-8183-1258064801205-0:0) stopped
> ============
> Thanks,
> PG

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (AMQ-2498) Broker goes down when database is restarted or goes down

Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/activemq/browse/AMQ-2498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56064#action_56064 ] 

Gary Tully commented on AMQ-2498:
---------------------------------

In the standalone case, simple broker, not master slave. disable the lock thread for the jdbc persistence adapter. set  useDatabaseLock=false. This is what is initiating the shutdown of the broker when the connection to the DB dies, at it assumes that another broker is ready to take its place. It is not needed in the standalone case.
Otherwise if look into a connection pool that will do retries for you.

> Broker goes down when database is restarted or goes down
> --------------------------------------------------------
>
>                 Key: AMQ-2498
>                 URL: https://issues.apache.org/activemq/browse/AMQ-2498
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.3.0
>            Reporter: Partha
>            Priority: Critical
>         Attachments: server.log
>
>
> When I database is idle for too long or I manually restart database, broker is shutting down. Does it not automatically try to reconnect? Am I missing anything?
> I am setting validationQuery and testOnBorrow to true in broker-config.xml datasource bean.
> Do I need to do anything else to tell broker for keep trying for database connection?
> error
> =========================
> Failed to checkpoint a message store: java.util.concurrent.ExecutionException: java.io.IOException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
> java.util.concurrent.ExecutionException: java.io.IOException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
>  at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>  at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:421)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter$1.iterate(JournalPersistenceAdapter.java:124)
>  at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:122)
>  at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:43)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>  at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
>  at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:45)
>  at org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:61)
>  at org.apache.activemq.store.jdbc.TransactionContext.begin(TransactionContext.java:151)
>  at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.beginTransaction(JDBCPersistenceAdapter.java:406)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter.beginTransaction(JournalPersistenceAdapter.java:216)
>  at org.apache.activemq.util.TransactionTemplate.run(TransactionTemplate.java:41)
>  at org.apache.activemq.store.journal.JournalMessageStore.checkpoint(JournalMessageStore.java:258)
>  at org.apache.activemq.store.journal.JournalTopicMessageStore.checkpoint(JournalTopicMessageStore.java:171)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter$5.call(JournalPersistenceAdapter.java:408)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter$5.call(JournalPersistenceAdapter.java:407)
>  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>  at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>  ... 3 more
> Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
>  at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(Unknown Source)
>  at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(Unknown Source)
>  at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(Unknown Source)
>  at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
>  at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:294)
>  at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:974)
>  at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:96)
>  at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:880)
>  at org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:54)
>  ... 13 more
> 2009-11-12 14:30:12,705 INFO  [STDOUT] 2009-11-12 14:30:12,705 [Q Cleanup Timer] INFO  BrokerService                  - ActiveMQ JMS Message Broker (ess.broker1, ID:VR1Server-8183-1258064801205-0:0) stopped
> ============
> Thanks,
> PG

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Issue Comment Edited: (AMQ-2498) Broker goes down when database is restarted or goes down

Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/activemq/browse/AMQ-2498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56064#action_56064 ] 

Gary Tully edited comment on AMQ-2498 at 11/24/09 6:36 AM:
-----------------------------------------------------------

In the standalone case, simple broker, not master slave. disable the lock thread for the jdbc persistence adapter. set  useDatabaseLock=false. This is what is initiating the shutdown of the broker when the connection to the DB dies, at it assumes that another broker is ready to take its place. It is not needed in the standalone case.
Otherwise,look into a connection pool that will do retries for you.

      was (Author: gtully):
    In the standalone case, simple broker, not master slave. disable the lock thread for the jdbc persistence adapter. set  useDatabaseLock=false. This is what is initiating the shutdown of the broker when the connection to the DB dies, at it assumes that another broker is ready to take its place. It is not needed in the standalone case.
Otherwise if look into a connection pool that will do retries for you.
  
> Broker goes down when database is restarted or goes down
> --------------------------------------------------------
>
>                 Key: AMQ-2498
>                 URL: https://issues.apache.org/activemq/browse/AMQ-2498
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.3.0
>            Reporter: Partha
>            Priority: Critical
>         Attachments: server.log
>
>
> When I database is idle for too long or I manually restart database, broker is shutting down. Does it not automatically try to reconnect? Am I missing anything?
> I am setting validationQuery and testOnBorrow to true in broker-config.xml datasource bean.
> Do I need to do anything else to tell broker for keep trying for database connection?
> error
> =========================
> Failed to checkpoint a message store: java.util.concurrent.ExecutionException: java.io.IOException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
> java.util.concurrent.ExecutionException: java.io.IOException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
>  at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>  at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:421)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter$1.iterate(JournalPersistenceAdapter.java:124)
>  at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:122)
>  at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:43)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>  at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
>  at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:45)
>  at org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:61)
>  at org.apache.activemq.store.jdbc.TransactionContext.begin(TransactionContext.java:151)
>  at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.beginTransaction(JDBCPersistenceAdapter.java:406)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter.beginTransaction(JournalPersistenceAdapter.java:216)
>  at org.apache.activemq.util.TransactionTemplate.run(TransactionTemplate.java:41)
>  at org.apache.activemq.store.journal.JournalMessageStore.checkpoint(JournalMessageStore.java:258)
>  at org.apache.activemq.store.journal.JournalTopicMessageStore.checkpoint(JournalTopicMessageStore.java:171)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter$5.call(JournalPersistenceAdapter.java:408)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter$5.call(JournalPersistenceAdapter.java:407)
>  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>  at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>  ... 3 more
> Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
>  at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(Unknown Source)
>  at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(Unknown Source)
>  at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(Unknown Source)
>  at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
>  at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:294)
>  at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:974)
>  at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:96)
>  at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:880)
>  at org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:54)
>  ... 13 more
> 2009-11-12 14:30:12,705 INFO  [STDOUT] 2009-11-12 14:30:12,705 [Q Cleanup Timer] INFO  BrokerService                  - ActiveMQ JMS Message Broker (ess.broker1, ID:VR1Server-8183-1258064801205-0:0) stopped
> ============
> Thanks,
> PG

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (AMQ-2498) Broker goes down when database is restarted or goes down

Posted by "Partha (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/activemq/browse/AMQ-2498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=55475#action_55475 ] 

Partha commented on AMQ-2498:
-----------------------------

==========================================================================
Re: Broker goes down when database is restarted

by Dejan Bosanac :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message
Hi,

what version are you using?

Cheers
-- 
Dejan Bosanac - http://twitter.com/dejanb

Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net


On Thu, Nov 12, 2009 at 11:47 PM, au.pg <pg...@...> wrote:

>
> Hi,
>
> When I database is idle for too long or I manually restart database, broker
> is shutting down. Does it not automatically try to reconnect? Am I missing
> anything?
>
> I am setting validationQuery and testOnBorrow to true in broker-config.xml
> datasource bean.
>
> Do I need to do anything else to tell broker for keep trying for database
> connection?
>
> error
> =========================
>
> Failed to checkpoint a message store:
> java.util.concurrent.ExecutionException: java.io.IOException: The TCP/IP
> connection to the host  has failed. java.net.ConnectException: Connection
> refused: connect
> java.util.concurrent.ExecutionException: java.io.IOException: The TCP/IP
> connection to the host  has failed. java.net.ConnectException: Connection
> refused: connect
>  at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>  at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>  at
>
> org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:421)
>  at
>
> org.apache.activemq.store.journal.JournalPersistenceAdapter$1.iterate(JournalPersistenceAdapter.java:124)
>  at
>
> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:122)
>  at
> org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:43)
>  at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>  at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>  at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: The TCP/IP connection to the host  has
> failed. java.net.ConnectException: Connection refused: connect
>  at
>
> org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:45)
>  at
>
> org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:61)
>  at
>
> org.apache.activemq.store.jdbc.TransactionContext.begin(TransactionContext.java:151)
>  at
>
> org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.beginTransaction(JDBCPersistenceAdapter.java:406)
>  at
>
> org.apache.activemq.store.journal.JournalPersistenceAdapter.beginTransaction(JournalPersistenceAdapter.java:216)
>  at
>
> org.apache.activemq.util.TransactionTemplate.run(TransactionTemplate.java:41)
>  at
>
> org.apache.activemq.store.journal.JournalMessageStore.checkpoint(JournalMessageStore.java:258)
>  at
>
> org.apache.activemq.store.journal.JournalTopicMessageStore.checkpoint(JournalTopicMessageStore.java:171)
>  at
>
> org.apache.activemq.store.journal.JournalPersistenceAdapter$5.call(JournalPersistenceAdapter.java:408)
>  at
>
> org.apache.activemq.store.journal.JournalPersistenceAdapter$5.call(JournalPersistenceAdapter.java:407)
>  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>  at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>  ... 3 more
> Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP
> connection to the host  has failed. java.net.ConnectException: Connection
> refused: connect
>  at
> com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(Unknown
> Source)
>  at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(Unknown
> Source)
>  at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(Unknown Source)
>  at
>
> org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
>  at
>
> org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:294)
>  at
>
> org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:974)
>  at
>
> org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:96)
>  at
>
> org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:880)
>  at
>
> org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:54)
>  ... 13 more
> 2009-11-12 14:30:12,705 INFO  [STDOUT] 2009-11-12 14:30:12,705 [Q Cleanup
> Timer] INFO  BrokerService                  - ActiveMQ JMS Message Broker
> (ess.broker1, ID:VR1Server-8183-1258064801205-0:0) stopped
>
> ============
>
> Thanks,
> PG
> --
> View this message in context:
> http://old.nabble.com/Broker-goes-down-when-database-is-restarted-tp26327953p26327953.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
Dejan Bosanac

Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net

	
Re: Broker goes down when database is restarted

by au.pg :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message
Hi Dejan,

I am using ActiveMQ 5.3.

Thanks,
PG

	
Re: Broker goes down when database is restarted

by Gary Tully :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message
Can you create a jira and attach the full log, a checkpoint failure should
not force a broker shutdown, that is, that stack trace should not. There
should be something else in the logs I think.

do the validationQuery and testOnBorrow on the pool really take effect?
Seems odd that connection refused is the exception rather than EOF.

But it looks like some sort of failover Jdbc pool is required here.


2009/11/13 au.pg <pg...@...>

>
> Hi Dejan,
>
> I am using ActiveMQ 5.3.
>
> Thanks,
> PG
> --
> View this message in context:
> http://old.nabble.com/Broker-goes-down-when-database-is-restarted-tp26327953p26338023.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>


-- 
http://blog.garytully.com

Open Source Integration
http://fusesource.com
==========================================================================

> Broker goes down when database is restarted or goes down
> --------------------------------------------------------
>
>                 Key: AMQ-2498
>                 URL: https://issues.apache.org/activemq/browse/AMQ-2498
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.3.0
>            Reporter: Partha
>            Priority: Critical
>         Attachments: server.log
>
>
> When I database is idle for too long or I manually restart database, broker is shutting down. Does it not automatically try to reconnect? Am I missing anything?
> I am setting validationQuery and testOnBorrow to true in broker-config.xml datasource bean.
> Do I need to do anything else to tell broker for keep trying for database connection?
> error
> =========================
> Failed to checkpoint a message store: java.util.concurrent.ExecutionException: java.io.IOException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
> java.util.concurrent.ExecutionException: java.io.IOException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
>  at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>  at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:421)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter$1.iterate(JournalPersistenceAdapter.java:124)
>  at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:122)
>  at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:43)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>  at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
>  at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:45)
>  at org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:61)
>  at org.apache.activemq.store.jdbc.TransactionContext.begin(TransactionContext.java:151)
>  at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.beginTransaction(JDBCPersistenceAdapter.java:406)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter.beginTransaction(JournalPersistenceAdapter.java:216)
>  at org.apache.activemq.util.TransactionTemplate.run(TransactionTemplate.java:41)
>  at org.apache.activemq.store.journal.JournalMessageStore.checkpoint(JournalMessageStore.java:258)
>  at org.apache.activemq.store.journal.JournalTopicMessageStore.checkpoint(JournalTopicMessageStore.java:171)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter$5.call(JournalPersistenceAdapter.java:408)
>  at org.apache.activemq.store.journal.JournalPersistenceAdapter$5.call(JournalPersistenceAdapter.java:407)
>  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>  at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>  ... 3 more
> Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host  has failed. java.net.ConnectException: Connection refused: connect
>  at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(Unknown Source)
>  at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(Unknown Source)
>  at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(Unknown Source)
>  at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
>  at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:294)
>  at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:974)
>  at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:96)
>  at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:880)
>  at org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:54)
>  ... 13 more
> 2009-11-12 14:30:12,705 INFO  [STDOUT] 2009-11-12 14:30:12,705 [Q Cleanup Timer] INFO  BrokerService                  - ActiveMQ JMS Message Broker (ess.broker1, ID:VR1Server-8183-1258064801205-0:0) stopped
> ============
> Thanks,
> PG

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.