[jira] [Created] (AIRAVATA-2689) Distributed email clients to improve email monitoring

["Dimuthu Upeksha (JIRA)" <ji...@apache.org>]

Dimuthu Upeksha created AIRAVATA-2689:
-----------------------------------------

             Summary: Distributed email clients to improve email monitoring 
                 Key: AIRAVATA-2689
                 URL: https://issues.apache.org/jira/browse/AIRAVATA-2689
             Project: Airavata
          Issue Type: New Feature
            Reporter: Dimuthu Upeksha
            Assignee: Dimuthu Upeksha


Once Airavata submits a job to a compute resource, scheduler in compute resource sends emails about the status of the job. Content in the email is different to each application type so we have written a set of parsers [2] which can extract correct information form email messages. Airavata has an email monitoring system which reads those emails, parse them and perform necessary actions depending on the content of the emails. However this email monitoring system is tightly coupled into the task execution engine so we can't easily replicate it to have high availability.

Idea is to come up with a standalone email monitoring client that reads emails from a given email account, parse them and convert it into a standard message format. Once the message is parsed into the known message format, put it in to a queue ( rabbitmq, kafka) in order to be consumed by task execution engine. There are few non functional requirements
 # To improve the availability, we need to have more than one monitoring client to be running at a given time. However we need to make sure only exactly one client consumes a given email. So we need the coordination among email clients
 #  In future, this will be deployed as a micro service, so final packaging should be compatible with docker

Current email monitor implementation is this [1]. Set of parsers available depending on the application [2]

[1] [https://github.com/apache/airavata/blob/master/modules/gfac/gfac-impl/src/main/java/org/apache/airavata/gfac/monitor/email/EmailBasedMonitor.java]

[2] https://github.com/apache/airavata/tree/master/modules/gfac/gfac-impl/src/main/java/org/apache/airavata/gfac/monitor/email/parser



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)