You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@eagle.apache.org by 绿飕飕 <qi...@gmail.com> on 2017/12/08 05:34:28 UTC

Need help about eagle's data flow.Thanks!

*1.* For Install 'Hdfs Audit Log Monitor', I have set following config:

    1.create two kafka topics: *hdfs_audit_log_sandbox*,
*hdfs_audit_log_enriched_sandbox*

    2.stream audit log into topic *hdfs_audit_log_sandbox*

    3.Kafka Consumer Topic for HDFS Auditlog : *hdfs_audit_log_sandbox*

    4. Kafka Topic for Auditlog Event Sink:
*hdfs_audit_log_enriched_sandbox*

    5. The Policy is from *HDFS_AUDIT_LOG_ENRICHED_STREAM_SANDBOX* insert
into *hdfs_audit_log_enriched_stream_out*


(a).  But the monitor not work, are there any error about set these config?

(b). I think some topic such as hdfs_audit_log_enriched_stream_out will be
create by eagle, is it right?


*2.* Is it right of the data flow?Or I miss some steps ?

data flow:   a-b-c-d-e-f

a. hdfa --> *hdfs_audit_log*

b. kafka topic -->  *hdfs_audit_log_sandbox*


 c.* HDFS_AUDIT_LOG_ENRICHED_STREAM_SANDBOX :*

        storm Spouts --> parserBolt

        storm Bolts --> sensitivityJoin

        storm Bolts --> ipZoneJoin

        storm Bolts -->kafkaSink

d. kafka topic --> *hdfs_audit_log_enriched_sandbox*​


e. the Policy handle the *hdfs_audit_log_enriched_sandbox* and send the
alert result to *hdfs_audit_log_enriched_stream_out*

f. the error message would put into the storage



*3. *could you recommend the version about the dependence of eagle v0.5.0?


Thanks,

    Qilv Wu