[jira] Updated: (CHUKWA-94) SALSA state-machine extraction from Hadoop logs

["Jiaqi Tan (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CHUKWA-94?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jiaqi Tan updated CHUKWA-94:
----------------------------

    Attachment: tan.pdf

SALSA paper from the Workshop on Analysis of System Logs '08, San Diego, CA

> SALSA state-machine extraction from Hadoop logs
> -----------------------------------------------
>
>                 Key: CHUKWA-94
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-94
>             Project: Hadoop Chukwa
>          Issue Type: New Feature
>          Components: Data Processors
>            Reporter: Jiaqi Tan
>         Attachments: tan.pdf
>
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> This is a proposed feature addition to extract state-machine views from Hadoop's logs (TaskTracker, JobTracker, and DataNode currently supported, NameNode soon). These views are as described in http://www.usenix.org/event/wasl08/tech/full_papers/tan/tan_html/ and will enable analysis and diagnosis algorithms to be built on top of them.
> Building a full SALSA view involves two steps:
> 1. Incrementally parsing log entries on a per-node basis to extract states (line-by-line reading, assuming the entire log file from a given node is available to the same process)
> 2. "Stitching" and correlating states across all logs (across nodes and across types) to build a full state machine.
> My idea is to add SALSA as two jobs in the demux stage, with the first parsing job in demux, and either having: 
> (a) the parsing job write its output to the permanent store with the correlating job reading/writing from/to the permanent store, or 
> (b) the parsing job write its output back to the sinkfile and having the correlating job reading from the sink file and writing to the permanent store.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.