[jira] [Commented] (AIRAVATA-2689) Distributed email clients to improve email monitoring

["Suresh Marru (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRAVATA-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16373464#comment-16373464 ] 

Suresh Marru commented on AIRAVATA-2689:
----------------------------------------

[~apoorvp2] has worked on this issues as part of GSOC last year and has architecture diagram and discussion at - [http://markmail.org/thread/cqvdomb2i5hh4jgw.|http://markmail.org/thread/cqvdomb2i5hh4jgw] I am still looking for the code related to this contribution. 

> Distributed email clients to improve email monitoring 
> ------------------------------------------------------
>
>                 Key: AIRAVATA-2689
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-2689
>             Project: Airavata
>          Issue Type: New Feature
>            Reporter: Dimuthu Upeksha
>            Assignee: Dimuthu Upeksha
>            Priority: Major
>              Labels: HackIllinois2018
>
> Once Airavata submits a job to a compute resource, scheduler in compute resource sends emails about the status of the job. Content in the email is different to each application type so we have written a set of parsers [2] which can extract correct information form email messages. Airavata has an email monitoring system which reads those emails, parse them and perform necessary actions depending on the content of the emails. However this email monitoring system is tightly coupled into the task execution engine so we can't easily replicate it to have high availability.
> Idea is to come up with a standalone email monitoring client that reads emails from a given email account, parse them and convert it into a standard message format. Once the message is parsed into the known message format, put it in to a queue ( rabbitmq, kafka) in order to be consumed by task execution engine. There are few non functional requirements
>  # To improve the availability, we need to have more than one monitoring client to be running at a given time. However we need to make sure only exactly one client consumes a given email. So we need the coordination among email clients
>  #  In future, this will be deployed as a micro service, so final packaging should be compatible with docker
> Current email monitor implementation is this [1]. Set of parsers available depending on the application [2]
> [1] [https://github.com/apache/airavata/blob/master/modules/gfac/gfac-impl/src/main/java/org/apache/airavata/gfac/monitor/email/EmailBasedMonitor.java]
> [2] https://github.com/apache/airavata/tree/master/modules/gfac/gfac-impl/src/main/java/org/apache/airavata/gfac/monitor/email/parser



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)