[jira] [Created] (AMBARI-11713) Deleting a service does not remove alert_definition, leading to a deadlock when re-adding the service

["Benoit Perroud (JIRA)" <ji...@apache.org>]

Benoit Perroud created AMBARI-11713:
---------------------------------------

             Summary: Deleting a service does not remove alert_definition, leading to a deadlock when re-adding the service
                 Key: AMBARI-11713
                 URL: https://issues.apache.org/jira/browse/AMBARI-11713
             Project: Ambari
          Issue Type: Bug
          Components: ambari-server
    Affects Versions: 2.0.1
            Reporter: Benoit Perroud
            Priority: Minor


When removing a service (via the API, for instance {{curl -u $ambariuser -i -H 'X-Requested-By: ambari' -X DELETE http://daplab-wn-06:8080/api/v1/clusters/CLUSTER01/services/AMBARI_METRICS}}), the entries in {{alert_definition}} table are not removed.

When re-adding the service, the {{uni_alert_def_name}} constraint fails, leading to a deadlock (looks like the lock is not released properly).

Constraint exception:
{code}
Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "uni_alert_def_name"
  Detail: Key (cluster_id, definition_name)=(2, metrics_monitor_process_percent) already exists.
Error Code: 0
Call: INSERT INTO alert_definition (definition_id, cluster_id, component_name, definition_name, description, ENABLED, HASH, ignore_host, label, schedule_interval, scope, service_name, alert_source, source_type) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
        bind => [14 parameters bound]
        at org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:340)
        at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.processExceptionForCommError(DatabaseAccessor.java:1611)
        at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeDirectNoSelect(DatabaseAccessor.java:898)
        at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeNoSelect(DatabaseAccessor.java:962)
        at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:631)
        at org.eclipse.persistence.internal.databaseaccess.ParameterizedSQLBatchWritingMechanism.executeBatch(ParameterizedSQLBatchWritingMechanism.java:149)
        at org.eclipse.persistence.internal.databaseaccess.ParameterizedSQLBatchWritingMechanism.executeBatchedStatements(ParameterizedSQLBatchWritingMechanism.java:134)
        at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.writesCompleted(DatabaseAccessor.java:1836)
        at org.eclipse.persistence.internal.sessions.AbstractSession.writesCompleted(AbstractSession.java:4244)
        at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.writesCompleted(UnitOfWorkImpl.java:5594)
        at org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork.writeChanges(RepeatableWriteUnitOfWork.java:453)
        at org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush(EntityManagerImpl.java:863)
        at org.eclipse.persistence.internal.jpa.QueryImpl.performPreQueryFlush(QueryImpl.java:963)
        at org.eclipse.persistence.internal.jpa.QueryImpl.executeReadQuery(QueryImpl.java:207)
        at org.eclipse.persistence.internal.jpa.QueryImpl.getSingleResult(QueryImpl.java:517)
        at org.eclipse.persistence.internal.jpa.EJBQueryImpl.getSingleResult(EJBQueryImpl.java:400)
        at org.apache.ambari.server.orm.dao.DaoUtils.selectSingle(DaoUtils.java:71)
        at org.apache.ambari.server.orm.dao.AlertDispatchDAO.findDefaultServiceGroup(AlertDispatchDAO.java:346)
        at org.apache.ambari.server.orm.AmbariLocalSessionInterceptor.invoke(AmbariLocalSessionInterceptor.java:53)
        at org.apache.ambari.server.orm.dao.AlertDefinitionDAO.create(AlertDefinitionDAO.java:307)
        at org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.invoke(AmbariJpaLocalTxnInterceptor.java:68)
        at org.apache.ambari.server.events.listeners.alerts.AlertServiceStateListener.onAmbariEvent(AlertServiceStateListener.java:148)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at com.google.common.eventbus.EventHandler.handleEvent(EventHandler.java:74)
        at com.google.common.eventbus.EventBus.dispatch(EventBus.java:314)
        at com.google.common.eventbus.AsyncEventBus.access$001(AsyncEventBus.java:34)
        at com.google.common.eventbus.AsyncEventBus$1.run(AsyncEventBus.java:100)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "uni_alert_def_name"
  Detail: Key (cluster_id, definition_name)=(2, metrics_monitor_process_percent) already exists.
        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161)
        at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890)
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
{code}

 
Stacktrace
{code}
"qtp-client-24" prio=10 tid=0x00007fdc656e4800 nid=0xe79 waiting on condition [0x00007fdc266bb000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x000000079e3364f0> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197)
        at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:945)
        at org.apache.ambari.server.state.configgroup.ConfigGroupImpl.persistConfigMapping(ConfigGroupImpl.java:419)
        at org.apache.ambari.server.state.configgroup.ConfigGroupImpl.saveIfPersisted(ConfigGroupImpl.java:456)
        at org.apache.ambari.server.state.configgroup.ConfigGroupImpl.persist(ConfigGroupImpl.java:323)
        at org.apache.ambari.server.controller.internal.ConfigGroupResourceProvider.updateConfigGroups(ConfigGroupResourceProvider.java:600)
        - locked <0x00000007a0067b30> (a org.apache.ambari.server.controller.internal.ConfigGroupResourceProvider)
        at org.apache.ambari.server.controller.internal.ConfigGroupResourceProvider.access$300(ConfigGroupResourceProvider.java:58)
        at org.apache.ambari.server.controller.internal.ConfigGroupResourceProvider$4.invoke(ConfigGroupResourceProvider.java:269)
{code}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)