[jira] [Created] (GOBBLIN-1621) Make HelixRetriggeringJobCallable emit job skip event when job is dropped due to another job is running

["Zihan Li (Jira)" <ji...@apache.org>]

Zihan Li created GOBBLIN-1621:
---------------------------------

             Summary: Make HelixRetriggeringJobCallable emit job skip event when job is dropped due to another job is running
                 Key: GOBBLIN-1621
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1621
             Project: Apache Gobblin
          Issue Type: Improvement
            Reporter: Zihan Li


Now, when we enable concurrency on gobblin service but disable that on gobblin cluster, gobblin cluster manager will drop the job silently if previous job is running. And from gobblin service, since no update heard back, it will think that job is still waiting for start and then once we exceed job start sla, it will cancel the job, which in turn will cancel the previous running job. And make the long running job never finish in this case. 

 

The solution is to emit job skip event when job get dropped, and on job status monitoring service, we listen to this event and mark the job status as cancelled in this case.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)