[jira] [Commented] (AIRAVATA-2065) Introduce more robust logging mechanism for Airavata

["Lahiru Gunathilake (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRAVATA-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15436415#comment-15436415 ] 

Lahiru Gunathilake commented on AIRAVATA-2065:
----------------------------------------------

It would be nice to have following logging features to airavata.

1. Central location of viewing logs of all the components (preferably web gui)
2. Filter the logs with properties like (component, deployed host).
      If we deploy multiple instances of gfac given the gfac node name get all the logs in that gfac node.
      Given the experiment Id get all the logs for that experiment.
      Get all the logs with given log level (get all the errors in gfac, get all the warnings in orchestrator)

I think for this kind of usage elastic search is going to very useful because it provide very powerful way of querying data with simple query language.

Dataflow would look like this,

Airavata dump all the logs to some messaging middlware (ex: kafka) -> logstash[1] listen on the topic -> push to elastic search

Use some tool like kibana [2] to view the logs or implement our own log viewer to view the logs.


[1]https://www.elastic.co/products/logstash
[2]https://www.elastic.co/products/kibana


> Introduce more robust logging mechanism for Airavata
> ----------------------------------------------------
>
>                 Key: AIRAVATA-2065
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-2065
>             Project: Airavata
>          Issue Type: Story
>            Reporter: Lahiru Gunathilake
>            Assignee: Lahiru Gunathilake
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)