[jira] [Created] (TRAFODION-2310) DTM Lead Logic on very busy system resulted in trafodion crash

["Gonzalo E Correa (JIRA)" <ji...@apache.org>]

Gonzalo E Correa created TRAFODION-2310:
-------------------------------------------

             Summary:  DTM Lead Logic on very busy system resulted in trafodion crash
                 Key: TRAFODION-2310
                 URL: https://issues.apache.org/jira/browse/TRAFODION-2310
             Project: Apache Trafodion
          Issue Type: Bug
          Components: foundation
    Affects Versions: 2.1-incubating
            Reporter: Gonzalo E Correa
            Assignee: Gonzalo E Correa
             Fix For: 2.1-incubating


The root cause of this problem is that the monitor in node 0 was starved out of CPU cycles and the watchdog timer expired. Consequently, the node was brought down by the SQWatchdog process.
 
This caused a sequence events where the TM leader was still in node 0 as far as all the remote monitors were concerned, but the $TM0 process no longer existed. The TM processes on the other nodes got the death message before the node was marked down which caused them to send a TM Leader request to their local monitor which checks to make sure the process exists, if not it aborts (this really should re-drive the selection of a new TM leader). However, the node down processing is what currently select a new TM leader, but there should be logic that reassigns a new TM leader when the leader dies. This is a bug that needs fixing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)