[jira] [Commented] (AMQ-5435) Persistence Adapter Starting Thread is still alive after stopping a slave broker with lease database locker

["Shi Lei (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AMQ-5435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14228075#comment-14228075 ] 

Shi Lei commented on AMQ-5435:
------------------------------

This is the stop method
public void stop() throws Exception {
        if (stopped.compareAndSet(false, true)) {
            stopping.set(true);
            ServiceStopper stopper = new ServiceStopper();
            try {
                doStop(stopper);
            } catch (Exception e) {
                stopper.onException(this, e);
            } finally {
                postStop(stopper);
            }
            stopped.set(true);
            started.set(false);
            stopping.set(false);
            for(ServiceListener l:this.serviceListeners) {
                l.stopped(this);
            }
            stopper.throwFirstException();
        }
    }

The problem is that after being stopped, stopping is false,

So LeaseDatabaseLocker.doStart will still in loop after the locker is stopped.

> Persistence Adapter Starting Thread is still alive after stopping a slave broker with lease database locker
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-5435
>                 URL: https://issues.apache.org/jira/browse/AMQ-5435
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.10.0
>         Environment: Windows, JDK7
>            Reporter: Shi Lei
>         Attachments: patch2.txt
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> I am using jdbc master/slave with lease database lock.
> <amq:broker id="broker" startAsync="true">
> I found if I call broker.stop to stop a slave broker service (which means it tries to get a lease locker and has not got yet), its Persistence Adapter Starting Thread is still alive. If I create and start a new broker in the same java VM, there will be 2 Persistence Adapter Starting Threads inside the same java VM. At this time, if the master broker is down, the stopped broker will get the database lease locker, but somehow it cannot start broker. Now I have 2 broker service in the same VM. One has got the locker, but cannot start broker, the other one is still requesting the locker.
> The root cause is that after stopping broker, LeaseDatabaseLocker.isStopping() is false, LeaseDatabaseLocker.isStopped() is true,
> In LeaseDatabaseLocker.doStart
> {code}
>  while (!isStopping()) {
>             Connection connection = null;
>             PreparedStatement statement = null;
>             try {
>                 connection = getConnection();
>                 initTimeDiff(connection);
>                 statement = connection.prepareStatement(sql);
>                 setQueryTimeout(statement);
>                 now = System.currentTimeMillis() + diffFromCurrentTime;
>                 statement.setString(1, getLeaseHolderId());
>                 statement.setLong(2, now + lockAcquireSleepInterval);
>                 statement.setLong(3, now);
>                 int result = statement.executeUpdate();
>                 if (result == 1) {
>                     // we got the lease, verify we still have it
>                     if (keepAlive()) {
>                         break;
>                     }
>                 }
>                 reportLeasOwnerShipAndDuration(connection);
>             } catch (Exception e) {
>                 LOG.debug(getLeaseHolderId() + " lease acquire failure: "+ e, e);
>                 if (isStopping()) {
>                     throw new Exception(
>                             "Cannot start broker as being asked to shut down. "
>                                     + "Interrupted attempt to acquire lock: "
>                                     + e, e);
>                 }
>                 if (handleStartException) {
>                     lockable.getBrokerService().handleIOException(IOExceptionSupport.create(e));
>                 }
>             } finally {
>                 close(statement);
>                 close(connection);
>             }
>             LOG.info(getLeaseHolderId() + " failed to acquire lease.  Sleeping for " + lockAcquireSleepInterval + " milli(s) before trying again...");
>             TimeUnit.MILLISECONDS.sleep(lockAcquireSleepInterval);
>         }
>         if (isStopping()) {
>             throw new RuntimeException(getLeaseHolderId() + " failing lease acquire due to stop");
>         }
>         LOG.info(getLeaseHolderId() + ", becoming master with lease expiry " + new Date(now) + " on dataSource: " + dataSource);
>     }
> {code}
> I think we should replace isStopping() with isStopping() or isStopped().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)