[jira] [Updated] (METRON-371) Errors seen in enrichment bolts for squid logs

["Casey Stella (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/METRON-371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Casey Stella updated METRON-371:
--------------------------------
    Fix Version/s: 0.2.2BETA

> Errors seen in enrichment bolts for squid logs
> ----------------------------------------------
>
>                 Key: METRON-371
>                 URL: https://issues.apache.org/jira/browse/METRON-371
>             Project: Metron
>          Issue Type: Improvement
>    Affects Versions: 0.3.0BETA
>         Environment: 12 node setup created on openstack running build as of Aug 8th. See git log snippet below:
> {code}
> [root@metron-test-13 metron-deployment]# git log
> commit b9282b438422d56fac23301dc854a39ae7d83a83
> Author: mmiklavc <mi...@gmail.com>
> Date:   Mon Aug 8 15:25:20 2016 -0400
>     METRON-356 Modify Storm topology.classpath via configuration (mmiklavc via cestella) closes apache/incubator-metron#204
> <snip>
> {code}
>            Reporter: Anand Subramanian
>            Assignee: Casey Stella
>            Priority: Minor
>             Fix For: 0.2.2BETA
>
>         Attachments: zkconfig.txt
>
>
> When I ran a test for the squid proxy sensor, I could see the following errors being thrown in the enrichment kafkaspout log file. 
> {code}
> 2016-08-11 09:07:26.629 o.a.m.e.b.EnrichmentSplitterBolt [ERROR] Unable to retrieve a sensor enrichment config of squid
> 2016-08-11 09:07:26.630 o.a.m.e.b.EnrichmentJoinBolt [ERROR] Unable to retrieve a sensor enrichment config of squid
> 2016-08-11 09:07:26.631 o.a.m.e.b.EnrichmentSplitterBolt [ERROR] Unable to retrieve sensor config: squid
> 2016-08-11 09:07:26.631 o.a.m.e.b.ThreatIntelJoinBolt [ERROR] Unable to retrieve sensor config: squid
> {code}
> *Testing Steps*
> 1) Ensure squid topology is up.
> 2) Inject the following message to the kafka-producer to ingest 
> {code}
> "1461576382.642    161 127.0.0.1 TCP_MISS/200 103701 GET http://www.abc.com/ - DIRECT/199.27.79.73 text/html"
> {code}
> 3) Wait for the enrichment and index to be generated. 
> 4) Review the enrichment kafkaspout log file and the error can be seen. 
> After discussing with [~dlyle], this error is apparently due to the missing enrichments for squid (see attached zkconfig.txt). If the squid enrichment were added manually, then the error messages are not seen. 
> Also that for some of the sensors (squid, in this case), it might be normal to not enrich some types of data.
> Now, this message showing up as ERROR is not representative of the above statement where we do not want to enrich some fields, on purpose. WARNNG or INFO might be a more appropriate way to log these messages. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)